Repository navigation
feat(bench): add two-gate triage composite (activity x rich_selectivity) - #132
Merged
Merged
Conversation
The triage benchmark showed no scorer could pass all three pairwise AUROC conditions with strong selective-vs-hemolytic separation. The old triage_score (activity × (1 - hemolysis_risk)) used a non-significant hemolysis detector (detection AUROC 0.565, CI includes 0.5). This PR adds gate_triage = activity × rich_selectivity, combining the two strongest complementary signals: - activity: strong AMP-vs-decoy (AUROC 0.885-0.934) but anti-selective - rich_selectivity: strong selective-vs-hemolytic (AUROC 0.745, significant) but anti-AMP (penalizes AMP-like features vs decoys) Key result: gate_triage is the first scorer to pass all three standard triage conditions with selective_vs_hemolytic > 0.65 (sel_vs_dec=0.779, hem_vs_dec=0.686, sel_vs_hem=0.666). Top-20: 16 selective / 1 hemolytic / 3 decoy — best distribution. Honest limitation: does NOT pass strict triage (composition-matched decoys). Hemolytic_vs_decoy drops to 0.489 because rich_selectivity penalizes the AMP-like composition that hemolytic AMPs share with their scrambled versions. Must NOT replace the ensemble activity gate — it is a complementary signal. Changes: - benchmark/triage.py: gate_triage added to _score_all() and scorer lists in both run_triage_benchmark() and run_strict_triage_benchmark() - benchmark/metrics_snapshot.py: top_20_by_gate_triage added to snapshot - tests/test_triage_benchmark.py: 8 new tests in TestGateTriageFindings - tests/test_metrics_snapshot.py: updated best_scorer assertion - outputs/metrics_snapshot.json: regenerated with gate_triage results - docs/METRICS_CURRENT.md: gate_triage row in triage tables, new v0.5.17 section - docs/ROADMAP.md: v0.5.18 entry Verified: 1579 passed, 7 skipped, ruff clean, git diff --check clean
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds gate_triage = activity x rich_selectivity to the triage benchmark. First scorer to pass all three standard triage conditions with selective_vs_hemolytic above 0.65.
Key result
Honest limitations
Does NOT pass strict triage. Weaker than ensemble on AMP-vs-decoy. Must not replace ensemble activity gate. Wet-lab assay remains mandatory.
Verification