[cschanhniem]

Summary

Addresses the v1.1 ROADMAP item: "benchmark candidate triage against a reference panel that includes selective AMPs, hemolytic positives, inactive peptides, and random controls."

Prior benchmarks tested two separate 2-class problems (AMP vs decoy, hemolytic vs selective). No benchmark tested the combined triage task that the virtual assay layer must solve: rank selective AMPs above hemolytic AMPs above random decoys in a single panel.

Dataset

125 selective AMPs (HC50 >= 100 µg/mL) + 45 hemolytic AMPs (HC50 < 25 µg/mL) + 96 random background decoys = 266 total.

Key findings (documented honestly)

Scorer	sel > decoy	hem > decoy	sel > hem	Triages correctly?
ensemble	0.848	0.891	0.466	NO (anti-selective)
activity	0.885	0.934	0.430	NO
selectivity_proxy	0.782	0.795	0.610	YES
triage_score	0.863	0.902	0.462	NO
safe_weighted_ensemble	0.849	0.890	0.483	NO

1. Ensemble does NOT triage correctly. It ranks hemolytic AMPs above selective AMPs (AUROC 0.466 < 0.5) due to the activity scorer's anti-selective bias.
2. Only selectivity_proxy triages correctly but sacrifices decoy discrimination (0.782 vs ensemble 0.848).
3. Naive triage_score does NOT fix the bias because hemolysis_risk is too weak (detection AUROC 0.565, not significant).
4. triage_score shifts top-20 favorably (16 selective vs 14, 4 hemolytic vs 6) but pairwise AUROC remains < 0.5.

New components

• src/openamp_foundry/benchmark/triage.py — multi-class triage benchmark module
• CLI: openamp-foundry bench triage --hemolysis-csv ... --decoy-csv ...
• Makefile: make bench-triage
• 16 new tests (15 benchmark + 1 CLI integration)
• Docs: METRICS_CURRENT.md, ROADMAP.md, ARCHITECTURE.md updated

Verification

• 1487 tests pass, 7 skipped
• Pre-push hook passed (pytest -q)
• Activity AUROC unchanged: 0.7832 (pipeline.yaml), 0.7448 (phase3.yaml)

Implication for virtual assay layer

This establishes the baseline that any future virtual assay module must beat: triage correctly (all 3 AUROCs > 0.5) while maintaining decoy discrimination (sel_vs_decoy > 0.80). The selectivity_proxy achieves correct triage but at the cost of decoy discrimination. A successful virtual assay must achieve both.

Constraint: Uses existing datasets (hemolysis_reference.csv + random_background.csv)
Rejected: Adding MODERATE class to binary task | Would muddy the triage signal
Directive: Do not claim triage_score as a virtual assay — it is a baseline composite
Confidence: high
Scope-risk: narrow
Tested: 1487 tests pass, 7 skipped; benchmark run on full 266-peptide panel

Co-authored-by: OmX noreply@omx.ai