[cschanhniem]

Class

Benchmark honesty / scientific credibility (Priority 1)

Why it matters

Until now, there was no documented comparison of the pipeline against trivial features. We didn't know whether the pipeline's 0.78 AUROC came from sophisticated scoring or from charge alone. This benchmark answers that question honestly.

Key finding

Charge density alone (AUROC 0.8166) outperforms the full pipeline ensemble (0.7792, Δ=−0.0374) on AMP-vs-Swiss-Prot-decoy discrimination.

Why this is expected

1. AMPs are cationic — net positive charge is the single strongest predictor of antimicrobial function
2. The pipeline optimizes for 4 objectives (activity 0.40, safety 0.25, synthesis 0.15, novelty 0.20)
3. The safety scorer penalizes high-charge peptides (hemolytic risk), reducing raw discrimination

Implication

The pipeline's value is in multi-objective selection (finding safe, novel, synthesizable AMPs), not in basic AMP/non-AMP discrimination. The current AMP-vs-decoy benchmark primarily tests charge, not the pipeline's actual strengths.

Changes

• scripts/baseline_trivial.py — new script computing trivial baselines (length, charge, charge density, length+charge)
• Makefile — make bench-easy-baseline target
• .github/workflows/ci.yml — informational CI step (non-gating)
• docs/50_LOOP_PLAN.md — Loop 12 ✅
• docs/METRICS_CURRENT.md — Easy Baseline Benchmark section + changelog entry
• docs/ROADMAP.md — v0.5.30 entry

Verification

• ✅ 1723 tests pass
• ✅ Lint clean on new file
• ✅ make bench-easy-baseline runs correctly

Risk / uncertainty

• The benchmark tests AMP-vs-Swiss-Prot-decoy discrimination, which is not the pipeline's design target
• A benchmark that tests safe-AMP detection or charge-matched discrimination would more honestly assess the pipeline's value
• Finding is consistent with known AMP literature (charge as dominant predictor)

Next loop

Loop 13 — Order-dependent features / strict triage