diff --git a/docs/METRICS_CURRENT.md b/docs/METRICS_CURRENT.md index 719d7085..6a983af0 100644 --- a/docs/METRICS_CURRENT.md +++ b/docs/METRICS_CURRENT.md @@ -413,6 +413,7 @@ A scorer that triages correctly should have all three AUROCs > 0.5: | synthesis | 0.590 | 0.634 | 0.469 | NO | | hemolysis_risk (inverted) | 0.485 | 0.492 | 0.488 | NO | | serum_stability | 0.217 | 0.160 | 0.569 | NO | +| **gate_triage** (activity × rich_sel) | **0.779** | **0.686** | **0.666** | **YES** | **Key findings:** @@ -431,6 +432,16 @@ A scorer that triages correctly should have all three AUROCs > 0.5: is too weak (detection AUROC 0.565, not significant on expanded benchmark). A naive virtual-assay composite does not outperform the ensemble. +6. **The gate_triage scorer (activity × rich_selectivity) is the first scorer + to triage correctly with strong selective_vs_hemolytic separation** (0.666). + Unlike the old triage_score, it uses rich_selectivity (detection AUROC 0.714, + significant) instead of hemolysis_risk (not significant). It also achieves + selective_vs_decoy 0.779 and hemolytic_vs_decoy 0.686, and ranks 16 selective + / 1 hemolytic / 3 decoys in its top-20 — the best distribution of any benchmarked + scorer. However, its AMP-vs-decoy discrimination is weaker than the ensemble + (0.779 vs 0.848) because the rich_selectivity gate penalizes AMP-like features. + It must NOT replace the ensemble activity gate; it is a complementary signal. + 4. **Top-20 distribution shift:** The triage_score moves 2 more selective AMPs into the top-20 (16 vs 14 for ensemble), removing 2 hemolytic AMPs (4 vs 6). The shift is in the right direction but modest — the hemolysis_risk penalty @@ -475,6 +486,7 @@ The MODERATE class (HC50 25-100, n=68) is excluded from the binary task. | expert_composite | 0.757 | **0.510** | 0.545 | 0.545 | YES | **NO** | | triage_score | 0.863 | **0.674** | 0.462 | 0.462 | NO | NO | | hemolysis_risk | 0.485 | 0.617 | 0.488 | 0.488 | NO | NO | +| gate_triage | 0.779 | **0.624** | 0.666 | 0.666 | YES | **NO** | **What this reveals:** @@ -496,6 +508,13 @@ The MODERATE class (HC50 25-100, n=68) is excluded from the binary task. 5. **The ensemble admits 7 scrambled decoys into top-20** (vs 0 with random decoys) — it cannot distinguish real AMPs from scrambled versions of themselves. +6. **gate_triage retains partial order-dependent signal** (sel_vs_dec 0.624, + hem_vs_dec 0.489). It fails strict triage because rich_selectivity penalizes + the AMP-like composition that hemolytic AMPs share with their scrambled + versions. But its selective_vs_decoy remains above 0.5, unlike selectivity_proxy + which collapses to exactly 0.500 — suggesting the activity gate contributes + order-dependent signal that the selectivity gate alone lacks. + **Implication:** The pipeline's triage signal is almost entirely composition-driven. The real bottleneck is selective-vs-hemolytic discrimination, which requires structural or contextual features beyond what current 1D physicochemical scorers @@ -595,6 +614,62 @@ protective), `longest_repeat_run` (0.5946), `length` (0.5900). Run: `make bench-selectivity` (rich_selectivity is included in the selectivity benchmark output) +## Two-Gate Triage Composite (v0.5.17 — added 2026-07-03) + +> The triage benchmark showed that no scorer could pass all three pairwise +> AUROC conditions (selective_vs_decoy, hemolytic_vs_decoy, selective_vs_hemolytic) +> with strong selective-vs-hemolytic separation. selectivity_proxy passed but +> had weak separation (0.610). expert_composite passed but admitted 5 decoys +> into top-20. The old triage_score used hemolysis_risk (not significant). +> +> This scorer combines two complementary signals as a multiplicative gate: +> activity (strong AMP-vs-decoy, AUROC 0.885-0.934) × rich_selectivity +> (strong selective-vs-hemolytic, AUROC 0.745, significant). +> +> Run: `make bench-triage` + +**Key result: gate_triage is the first scorer to pass all three standard triage conditions +with selective_vs_hemolytic > 0.65.** + +| Scorer | sel > decoy | hem > decoy | sel > hem | Top-20 (sel/hem/dec) | Correct? | +|--------|:-----------:|:-----------:|:---------:|:---------------------:|:--------:| +| ensemble | 0.848 | 0.891 | 0.466 | 14/6/0 | NO | +| selectivity_proxy | 0.782 | 0.795 | 0.610 | — | YES (weak) | +| expert_composite | 0.757 | 0.746 | 0.545 | 15/0/5 | YES (decoy leak) | +| triage_score (old) | 0.863 | 0.902 | 0.462 | 16/4/0 | NO | +| **gate_triage** | **0.779** | **0.686** | **0.666** | **16/1/3** | **YES** | + +**Design rationale:** + +The two gates solve complementary problems: +- activity gate: detects AMP-likeness (composition + amphipathicity) — + strong vs random decoys but anti-selective (rewards hemolytic AMPs) +- rich_selectivity gate: detects hemolysis risk from 8 evidence-identified + features — strong vs hemolytic AMPs but anti-AMP (penalizes AMP-like composition) + +Their product leverages both: a candidate must score high on BOTH AMP-likeness +AND selectivity. Hemolytic AMPs score high on activity but low on rich_selectivity. +Decoys score low on activity. Selective AMPs score moderately on both. + +**Honest limitations:** + +1. gate_triage does NOT pass strict triage (composition-matched decoys). + Its hemolytic_vs_decoy drops to 0.489 because rich_selectivity penalizes + the AMP-like composition that hemolytic AMPs share with their scrambled + versions. It retains partial order-dependent signal (sel_vs_dec 0.624), + but this is from the activity gate, not the selectivity gate. + +2. gate_triage is weaker than ensemble on pure AMP-vs-decoy detection + (0.779 vs 0.848). It must NOT replace the ensemble activity gate. + It is a complementary triage signal, not a replacement. + +3. A decoy leaks into the top-20 (3 decoys vs 0 for ensemble). The + selectivity gate removes some hemolytic AMPs but admits some decoys + that happen to have moderate activity and moderate selectivity. + +4. This is still a dry-lab triage signal. Wet-lab hemolysis assay + remains mandatory for all candidates. + ## Test Suite | Metric | Value | @@ -636,5 +711,6 @@ Run: `make bench-selectivity` (rich_selectivity is included in the selectivity b | 2026-07-02 | **Strict triage benchmark added:** composition-matched scrambled decoys replace random background. No scorer triages correctly — standard triage "success" of selectivity_proxy (0.782 sel_vs_dec) and expert_composite (0.757) was inflated by trivially distinguishable decoys. selectivity_proxy collapses to 0.500 (purely composition-driven), ensemble drops to 0.572. Real bottleneck (selective_vs_hemolytic) unchanged. | OpenAMP loop | | 2026-07-03 | **Rich selectivity scorer added:** composite of 8 evidence-identified features from the feature decomposition benchmark. Detection AUROC=0.7138 (CI 0.63-0.80) on n=179 — first pipeline score with statistically significant selective_vs_hemolytic discrimination. Old selectivity_proxy=0.5744 (CI 0.50-0.66). Honest limitation: does not triage AMP-vs-decoy (0.19); must be combined with activity gate. | OpenAMP loop | | 2026-07-03 | **Rich selectivity integrated into production pipeline:** rich_selectivity_score now computed in score_candidates() (pipeline.py), replaces hemolysis_safety as the expert composite hemolysis-risk component (weight 0.10), used in pilot_priority formula, displayed in pilot panel report, and included in evidence certificates. Expert AUROC drops 0.7119→0.7097 (−0.0022) — acceptable tradeoff: the expert now includes a significant hemolysis detector (CI excludes 0.5) instead of the old non-significant one. | OpenAMP loop | +| 2026-07-03 | **Two-gate triage composite added:** gate_triage = activity × rich_selectivity, added to triage benchmark. First scorer to pass all three standard triage conditions with strong selective_vs_hemolytic separation (0.666). Top-20: 16 selective / 1 hemolytic / 3 decoy — best distribution. Does NOT pass strict triage (hem_vs_dec 0.489) — honest limitation. Must not replace ensemble activity gate. | OpenAMP loop | | 2026-07-03 | **Feature decomposition benchmark added:** per-feature selective_vs_hemolytic AUROC for all 30 scalar physicochemical features. hydrophobic_fraction is the strongest single discriminative feature (0.6745, CI 0.58-0.77) but is NOT used by the selectivity proxy. 8/30 features have significant signal; 6 of those are unused. Provides actionable diagnostic for why composite scorers fail selective_vs_hemolytic discrimination. | OpenAMP loop | | 2026-06-29 | Initial — expanded benchmark (PR #110) | OpenAMP CI | diff --git a/docs/ROADMAP.md b/docs/ROADMAP.md index 51696cb7..2c4b37d5 100644 --- a/docs/ROADMAP.md +++ b/docs/ROADMAP.md @@ -366,6 +366,36 @@ The `rich_selectivity` component is anti-AMP by design (AUROC=0.1973 for AMP-vs-decoy) because it penalises high hydrophobicity and charge that define AMPs — this is the correct tradeoff. +## v0.5.18 — Two-Gate Triage Composite ✓ (2026-07-03) + +The triage benchmark (v0.5.12) showed no scorer could pass all three pairwise +AUROC conditions with strong selective-vs-hemolytic separation. The old +`triage_score` (activity × (1 - hemolysis_risk)) used a non-significant +hemolysis detector. This version adds `gate_triage` = activity × +rich_selectivity, combining the two strongest complementary signals. + +Changes: +- `benchmark/triage.py`: `gate_triage` added to `_score_all()` and the scorer + list in both `run_triage_benchmark()` and `run_strict_triage_benchmark()`. + Top-20 class distribution breakdowns added. +- `benchmark/metrics_snapshot.py`: `top_20_by_gate_triage` added to both triage + and strict_triage snapshot sections. +- `tests/test_triage_benchmark.py`: 8 new tests in `TestGateTriageFindings` + covering structure, standard triage success, selective-vs-hemolytic threshold, + improvement over old triage_score, top-20 distribution, hemolytic reduction vs + ensemble, best-scorer assertion, and strict triage honest-failure test. +- `outputs/metrics_snapshot.json`: regenerated with gate_triage results. + +Key result: **gate_triage is the first scorer to pass all three standard triage +conditions with selective_vs_hemolytic > 0.65** (sel_vs_dec=0.779, +hem_vs_dec=0.686, sel_vs_hem=0.666). Top-20: 16 selective / 1 hemolytic / 3 decoy. + +Honest limitation: gate_triage does NOT pass strict triage (composition-matched +decoys). Its hemolytic_vs_decoy drops to 0.489 because rich_selectivity +penalizes the AMP-like composition that hemolytic AMPs share with their +scrambled versions. It also retains 3 decoys in top-20 (vs 0 for ensemble). +It must NOT replace the ensemble activity gate — it is a complementary signal. + ## v1.0 — Validated dry-lab-to-wet-lab loop - independently reviewed assay batch (expert_review.yml GitHub issue template); diff --git a/outputs/metrics_snapshot.json b/outputs/metrics_snapshot.json index 093fa55c..1e67292c 100644 --- a/outputs/metrics_snapshot.json +++ b/outputs/metrics_snapshot.json @@ -638,7 +638,7 @@ "recall_at_43": 0.4211 }, "strict_triage": { - "best_scorer": "rich_selectivity", + "best_scorer": "gate_triage", "decoy_type": "composition_matched_scrambled", "n_decoy": 125, "n_hemolytic": 45, @@ -662,6 +662,12 @@ "selective_vs_hemolytic": 0.5745, "triages_correctly": false }, + "gate_triage": { + "hemolytic_vs_decoy": 0.4887, + "selective_vs_decoy": 0.6242, + "selective_vs_hemolytic": 0.6663, + "triages_correctly": false + }, "hemolysis_risk": { "hemolytic_vs_decoy": 0.6199, "selective_vs_decoy": 0.6173, @@ -721,6 +727,11 @@ "HEMOLYTIC": 0, "SELECTIVE": 11 }, + "top_20_by_gate_triage": { + "DECOY": 4, + "HEMOLYTIC": 1, + "SELECTIVE": 15 + }, "top_20_by_triage_score": { "DECOY": 2, "HEMOLYTIC": 4, @@ -728,7 +739,7 @@ } }, "triage": { - "best_scorer": "selectivity_proxy", + "best_scorer": "gate_triage", "per_scorer": { "activity": { "hemolytic_vs_decoy": 0.934, @@ -748,6 +759,12 @@ "selective_vs_hemolytic": 0.5745, "triages_correctly": true }, + "gate_triage": { + "hemolytic_vs_decoy": 0.6863, + "selective_vs_decoy": 0.779, + "selective_vs_hemolytic": 0.6663, + "triages_correctly": true + }, "hemolysis_risk": { "hemolytic_vs_decoy": 0.4924, "selective_vs_decoy": 0.485, @@ -807,6 +824,11 @@ "HEMOLYTIC": 0, "SELECTIVE": 15 }, + "top_20_by_gate_triage": { + "DECOY": 3, + "HEMOLYTIC": 1, + "SELECTIVE": 16 + }, "top_20_by_triage_score": { "DECOY": 0, "HEMOLYTIC": 4, diff --git a/src/openamp_foundry/benchmark/metrics_snapshot.py b/src/openamp_foundry/benchmark/metrics_snapshot.py index 26990b94..0b1418f6 100644 --- a/src/openamp_foundry/benchmark/metrics_snapshot.py +++ b/src/openamp_foundry/benchmark/metrics_snapshot.py @@ -154,6 +154,7 @@ def build_metrics_snapshot( "top_20_by_ensemble": triage["top_20_by_ensemble"], "top_20_by_triage_score": triage["top_20_by_triage_score"], "top_20_by_expert_composite": triage["top_20_by_expert_composite"], + "top_20_by_gate_triage": triage["top_20_by_gate_triage"], "per_scorer": { scorer: { "selective_vs_decoy": info["selective_vs_decoy"]["auroc"], @@ -173,6 +174,7 @@ def build_metrics_snapshot( "top_20_by_ensemble": strict_triage["top_20_by_ensemble"], "top_20_by_triage_score": strict_triage["top_20_by_triage_score"], "top_20_by_expert_composite": strict_triage["top_20_by_expert_composite"], + "top_20_by_gate_triage": strict_triage["top_20_by_gate_triage"], "per_scorer": { scorer: { "selective_vs_decoy": info["selective_vs_decoy"]["auroc"], diff --git a/src/openamp_foundry/benchmark/triage.py b/src/openamp_foundry/benchmark/triage.py index 58e88a55..894ec916 100644 --- a/src/openamp_foundry/benchmark/triage.py +++ b/src/openamp_foundry/benchmark/triage.py @@ -124,6 +124,13 @@ def _score_all(rows: list[dict[str, Any]], weights: dict[str, float]) -> list[di raw_adjusted = dict(raw) raw_adjusted["safety"] = safe_adjusted safe_weighted_ens = ensemble_score(raw_adjusted, weights) + # Two-gate triage: activity (AMP detection) * rich_selectivity (hemolysis separation). + # activity is strong at AMP-vs-decoy (AUROC ~0.88-0.93) but anti-selective + # (penalizes hydrophobicity/charge that hemolytic AMPs have). + # rich_selectivity is strong at selective-vs-hemolytic (AUROC ~0.74, significant) + # but anti-AMP (penalizes features that make AMPs look AMP-like vs decoys). + # Their product should leverage both: reward AMP-likeness AND selectivity. + gate_triage = round(act * rich_sel, 4) scored.append({ **row, "ensemble": round(ens, 4), @@ -138,6 +145,7 @@ def _score_all(rows: list[dict[str, Any]], weights: dict[str, float]) -> list[di "expert_composite": exp.composite, "triage_score": triage_score, "safe_weighted_ensemble": round(safe_weighted_ens, 4), + "gate_triage": gate_triage, }) return scored @@ -198,6 +206,7 @@ def run_triage_benchmark( "ensemble", "activity", "safety", "synthesis", "selectivity_proxy", "rich_selectivity", "hemolysis_risk", "serum_stability", "expert_composite", "triage_score", "safe_weighted_ensemble", + "gate_triage", ] per_scorer = {} @@ -280,6 +289,14 @@ def run_triage_benchmark( "DECOY": top_20_expert_classes.count("DECOY"), } + by_gate = sorted(all_scored, key=lambda x: x["gate_triage"], reverse=True) + top_20_gate_classes = [r["triage_class"] for r in by_gate[:20]] + top_20_gate_dist = { + "SELECTIVE": top_20_gate_classes.count("SELECTIVE"), + "HEMOLYTIC": top_20_gate_classes.count("HEMOLYTIC"), + "DECOY": top_20_gate_classes.count("DECOY"), + } + return { "benchmark": "multi_class_triage", "n_selective": len(selective_scored), @@ -291,6 +308,7 @@ def run_triage_benchmark( "top_20_by_ensemble": top_20_dist, "top_20_by_triage_score": top_20_triage_dist, "top_20_by_expert_composite": top_20_expert_dist, + "top_20_by_gate_triage": top_20_gate_dist, "known_blind_spots": [ "The ensemble ranks hemolytic AMPs above selective AMPs " "(anti-selective bias) because hemolytic AMPs have stronger " @@ -435,6 +453,7 @@ def run_strict_triage_benchmark( "ensemble", "activity", "safety", "synthesis", "selectivity_proxy", "rich_selectivity", "hemolysis_risk", "serum_stability", "expert_composite", "triage_score", "safe_weighted_ensemble", + "gate_triage", ] per_scorer = {} @@ -510,6 +529,14 @@ def run_strict_triage_benchmark( "DECOY": top_20_expert_classes.count("DECOY"), } + by_gate = sorted(all_scored, key=lambda x: x["gate_triage"], reverse=True) + top_20_gate_classes = [r["triage_class"] for r in by_gate[:20]] + top_20_gate_dist = { + "SELECTIVE": top_20_gate_classes.count("SELECTIVE"), + "HEMOLYTIC": top_20_gate_classes.count("HEMOLYTIC"), + "DECOY": top_20_gate_classes.count("DECOY"), + } + return { "benchmark": "strict_multi_class_triage", "decoy_type": "composition_matched_scrambled", @@ -524,6 +551,7 @@ def run_strict_triage_benchmark( "top_20_by_ensemble": top_20_dist, "top_20_by_triage_score": top_20_triage_dist, "top_20_by_expert_composite": top_20_expert_dist, + "top_20_by_gate_triage": top_20_gate_dist, "known_blind_spots": [ "Composition-matched decoys control for amino acid composition, " "so only order-dependent features (hydrophobic moment, amphipathic " diff --git a/tests/test_metrics_snapshot.py b/tests/test_metrics_snapshot.py index 29e3f7ba..cdbac9ae 100644 --- a/tests/test_metrics_snapshot.py +++ b/tests/test_metrics_snapshot.py @@ -17,7 +17,11 @@ def test_metrics_snapshot_matches_current_benchmark_truth(tmp_path): assert snapshot["cluster_split"]["full_auroc"] == 0.7832 assert snapshot["expert_ablation"]["expert_auroc"] == 0.7097 assert snapshot["selectivity"]["n_hemolytic"] == 54 - assert snapshot["triage"]["best_scorer"] == "selectivity_proxy" + assert snapshot["triage"]["best_scorer"] == "gate_triage" + assert "gate_triage" in snapshot["triage"]["per_scorer"] + assert snapshot["triage"]["per_scorer"]["gate_triage"]["triages_correctly"] is True + assert "top_20_by_gate_triage" in snapshot["triage"] + assert "top_20_by_gate_triage" in snapshot["strict_triage"] assert "expert_composite" in snapshot["triage"]["per_scorer"] assert snapshot["triage"]["per_scorer"]["expert_composite"]["triages_correctly"] is True assert snapshot["triage"]["top_20_by_expert_composite"]["DECOY"] > 0 diff --git a/tests/test_triage_benchmark.py b/tests/test_triage_benchmark.py index 18e217c0..14fcb333 100644 --- a/tests/test_triage_benchmark.py +++ b/tests/test_triage_benchmark.py @@ -44,7 +44,7 @@ def test_returns_required_keys(self, result): for key in ["benchmark", "n_selective", "n_hemolytic", "n_decoy", "n_total", "per_scorer", "best_scorer", "top_20_by_ensemble", "top_20_by_triage_score", - "top_20_by_expert_composite", + "top_20_by_expert_composite", "top_20_by_gate_triage", "known_blind_spots", "disclaimer"]: assert key in result, f"Missing key: {key}" @@ -62,6 +62,7 @@ def test_per_scorer_has_all_scorers(self, result): "ensemble", "activity", "safety", "synthesis", "selectivity_proxy", "hemolysis_risk", "serum_stability", "expert_composite", "triage_score", "safe_weighted_ensemble", + "gate_triage", } assert expected.issubset(set(result["per_scorer"].keys())) @@ -214,7 +215,7 @@ def test_returns_required_keys(self, strict_result): for key in ["benchmark", "decoy_type", "n_selective", "n_hemolytic", "n_decoy", "n_total", "per_scorer", "best_scorer", "top_20_by_ensemble", "top_20_by_triage_score", - "top_20_by_expert_composite", + "top_20_by_expert_composite", "top_20_by_gate_triage", "known_blind_spots", "disclaimer"]: assert key in strict_result, f"Missing key: {key}" @@ -322,3 +323,99 @@ def test_disclaimer_present(self, strict_result): def test_known_blind_spots_documented(self, strict_result): assert len(strict_result["known_blind_spots"]) >= 3 + +class TestGateTriageFindings: + """Test the two-gate triage composite: activity * rich_selectivity. + + This scorer combines two complementary signals: + - activity: strong at AMP-vs-decoy (AUROC ~0.88-0.93) but anti-selective + - rich_selectivity: strong at selective-vs-hemolytic (AUROC ~0.74, significant) + but anti-AMP (penalizes AMP-like features vs decoys) + + Their product should pass all three triage conditions in the standard + benchmark, which no previous scorer has achieved. + """ + + def test_gate_triage_is_present(self, result): + assert "gate_triage" in result["per_scorer"] + + def test_gate_triage_triages_correctly(self, result): + """The gate_triage scorer should pass all three pairwise AUROC > 0.5. + + This is the first scorer to achieve all-three-pass triage in the + standard benchmark. It leverages activity (AMP detection) and + rich_selectivity (hemolysis separation) as multiplicative gates. + """ + info = result["per_scorer"]["gate_triage"] + assert info["triages_correctly"] is True, ( + f"gate_triage should triage correctly. " + f"sel_vs_dec={info['selective_vs_decoy']['auroc']:.4f}, " + f"hem_vs_dec={info['hemolytic_vs_decoy']['auroc']:.4f}, " + f"sel_vs_hem={info['selective_vs_hemolytic']['auroc']:.4f}" + ) + + def test_gate_triage_selective_vs_hemolytic_above_threshold(self, result): + """gate_triage should achieve selective_vs_hemolytic > 0.60. + + The old triage_score (activity * (1 - hemolysis_risk)) fails this + condition because hemolysis_risk is not significant. gate_triage + uses rich_selectivity (AUROC=0.74, significant) instead. + """ + svh = result["per_scorer"]["gate_triage"]["selective_vs_hemolytic"]["auroc"] + assert svh > 0.60, ( + f"gate_triage selective_vs_hemolytic={svh:.4f}: expected > 0.60" + ) + + def test_gate_triage_beats_triage_score_on_selective_vs_hemolytic(self, result): + """gate_triage should beat the old triage_score on selective_vs_hemolytic. + + This validates that rich_selectivity (significant) is a better + hemolysis separator than hemolysis_risk (not significant). + """ + gate_svh = result["per_scorer"]["gate_triage"]["selective_vs_hemolytic"]["auroc"] + old_svh = result["per_scorer"]["triage_score"]["selective_vs_hemolytic"]["auroc"] + assert gate_svh > old_svh, ( + f"gate_triage svh={gate_svh:.4f} should beat triage_score svh={old_svh:.4f}" + ) + + def test_gate_triage_top_20_majority_selective(self, result): + """gate_triage top-20 should have majority selective AMPs.""" + dist = result["top_20_by_gate_triage"] + assert dist["SELECTIVE"] > 10, ( + f"gate_triage top-20 selective={dist['SELECTIVE']}: expected > 10" + ) + + def test_gate_triage_fewer_hemolytic_than_ensemble(self, result): + """gate_triage should rank fewer hemolytic AMPs in top-20 than ensemble. + + The ensemble has an anti-selective bias (ranks hemolytic AMPs highly). + gate_triage should correct this by penalizing hemolytic candidates + through the rich_selectivity gate. + """ + gate_hem = result["top_20_by_gate_triage"]["HEMOLYTIC"] + ens_hem = result["top_20_by_ensemble"]["HEMOLYTIC"] + assert gate_hem <= ens_hem, ( + f"gate_triage hemolytic in top-20={gate_hem} should be <= " + f"ensemble hemolytic in top-20={ens_hem}" + ) + + def test_gate_triage_is_best_scorer(self, result): + """gate_triage should be the best scorer by the composite metric.""" + assert result["best_scorer"] == "gate_triage", ( + f"Expected gate_triage as best scorer, got {result['best_scorer']}" + ) + + def test_gate_triage_does_not_pass_strict(self, strict_result): + """gate_triage should NOT pass strict triage (composition-matched decoys). + + In strict triage, hemolytic_vs_decoy drops below 0.5 because + rich_selectivity penalizes the AMP-like composition that hemolytic + AMPs share with their scrambled decoys. This is an honest limitation: + the gate_triage has no order-dependent signal for the hemolytic-vs-decoy + pair when composition is controlled. + """ + info = strict_result["per_scorer"]["gate_triage"] + assert info["triages_correctly"] is False, ( + "gate_triage should not pass strict triage. " + "If it does, it has order-dependent signal beyond composition." + )