feat(bench): per-feature selective_vs_hemolytic decomposition benchmark

Makefile

@@ -1,4 +1,4 @@
-.PHONY: help demo test lint ci clean bench-leakage bench-baseline bench-hidden-active bench-cluster-split bench-expert-ablation bench-selectivity generate phase3 pilot validate-scoring validate-scoring-phase3 validate-scoring-strict external-predict pilot-confident presynth-qc gold-standard diversity synthesis-order novelty-broad external-consensus questionnaire gate-check ip-report benchmark-card wave0-5-gate-check wave0-5-novelty-audit wave0-5-novelty-audit-v2 wave0-5-panel wave0-5-evidence wave0-5-fill-external wave0-5b-generate wave0-5b-filter
+.PHONY: help demo test lint ci clean bench-leakage bench-baseline bench-hidden-active bench-cluster-split bench-expert-ablation bench-selectivity bench-feature-decomp generate phase3 pilot validate-scoring validate-scoring-phase3 validate-scoring-strict external-predict pilot-confident presynth-qc gold-standard diversity synthesis-order novelty-broad external-consensus questionnaire gate-check ip-report benchmark-card wave0-5-gate-check wave0-5-novelty-audit wave0-5-novelty-audit-v2 wave0-5-panel wave0-5-evidence wave0-5-fill-external wave0-5b-generate wave0-5b-filter
 
 PYTHON := $(shell [ -f .venv/bin/python ] && echo .venv/bin/python || echo python3)
 PYTEST  := $(shell [ -f .venv/bin/pytest ] && echo .venv/bin/pytest || echo pytest)
@@ -31,6 +31,7 @@ help:
 	@echo "  make bench-cluster-split      Cluster-aware bootstrap CI (de-inflates near-duplicates)"
 	@echo "  make bench-expert-ablation    Expert composite vs ensemble ablation (honesty check)"
 	@echo "  make bench-selectivity        Within-AMP selectivity (hemolytic vs selective)"
+	@echo "  make bench-feature-decomp     Per-feature selective_vs_hemolytic decomposition"
 	@echo "  make bench-hidden-active      Hidden-positive recovery on mixed benchmark set"
 	@echo ""
 	@echo "  make wave0-5-gate-check     Run Wave 0.5 gates W0.5-1 through W0.5-7"
@@ -158,6 +159,10 @@ bench-selectivity:
 bench-triage:
 	PYTHONPATH=src $(PYTHON) -m openamp_foundry.cli bench triage \
 		--out outputs/triage_benchmark_report.json
+
+bench-feature-decomp:
+	PYTHONPATH=src $(PYTHON) -m openamp_foundry.cli bench feature-decomp \
+		--out outputs/feature_decomp_report.json
 validate-scoring-strict:
 	PYTHONPATH=src $(PYTHON) -m openamp_foundry.cli validate-scoring \
 		--amp-csv examples/validation/known_amps.csv \

docs/ARCHITECTURE.md

@@ -40,7 +40,7 @@ The purpose of the added layers is not simulation theater. It is to improve whic
 | `openamp_foundry.selection` | ranking (ensemble or expert composite) and diversity selection |
 | `openamp_foundry.evidence` | JSON certificate generation and validation |
 | `openamp_foundry.reports` | human-readable and machine-readable batch, calibration, and wet-lab review reports |
-| `openamp_foundry.benchmark` | leakage checks, cluster-split benchmark, expert ablation benchmark, within-AMP selectivity benchmark, triage benchmark, and evaluation scaffolding |
+| `openamp_foundry.benchmark` | leakage checks, cluster-split benchmark, expert ablation benchmark, within-AMP selectivity benchmark, triage benchmark, per-feature selectivity decomposition, and evaluation scaffolding |
 | `openamp_foundry.generators` | safe, bounded toy candidate generation |
 | `openamp_foundry.simulation` | membrane/selectivity/stability proxy modeling (scaffolded) |
 

docs/METRICS_CURRENT.md

@@ -5,7 +5,7 @@ Machine-readable snapshot: `outputs/metrics_snapshot.json` regenerated with `mak
 > **Purpose:** One authoritative table of current pipeline metrics. If any doc disagrees
 > with this file, this file wins. Updated whenever benchmark/benchmark config changes.
 >
-> **Last updated:** 2026-07-02 (strict triage benchmark added)
+> **Last updated:** 2026-07-03 (feature decomposition benchmark added)
 > **Pipeline version:** v0.5.x
 > **Branch:** main
 
@@ -502,6 +502,61 @@ can capture. Any future virtual assay layer must demonstrate order-dependent
 triage signal on this strict benchmark before claiming to improve candidate
 selection.
 
+## Feature Decomposition: Per-Feature Selective vs Hemolytic (v0.5.15 — added 2026-07-03)
+
+> The strict triage benchmark (v0.5.14) proved that NO composite scorer passes
+> selective_vs_hemolytic discrimination (AUROC 0.43-0.54). But it did not explain
+> *why*. This benchmark tests every scalar physicochemical feature individually
+> for selective_vs_hemolytic AUROC, with bootstrap confidence intervals.
+
+**Key finding: the selectivity proxy ignores the strongest discriminative features.**
+
+The selectivity proxy uses only `net_charge_ph74` and `gravy`. The top feature,
+`hydrophobic_fraction` (AUROC 0.6745, CI 0.58-0.77), is NOT used by the proxy.
+Six of eight significant features are not used by the current selectivity model.
+
+| Feature | Detection AUROC | CI 95% | Direction | Used by proxy? |
+|---------|-----------------|--------|-----------|----------------|
+| hydrophobic_fraction | **0.6745** | 0.58-0.77 | risk | **NO** |
+| helix_propensity | **0.6489** | 0.54-0.75 | risk | **NO** |
+| net_charge_proxy | **0.6394** | 0.54-0.73 | risk | **NO** |
+| net_charge_ph74 | **0.6332** | 0.54-0.73 | risk | YES |
+| selectivity_proxy | **0.6095** | 0.52-0.70 | protective | YES |
+| interior_trypsin_sites | **0.6089** | 0.51-0.70 | risk | **NO** |
+| longest_repeat_run | **0.5946** | 0.52-0.68 | risk | **NO** |
+| length | **0.5785** | 0.51-0.66 | risk | **NO** |
+
+**What this reveals:**
+
+1. **`hydrophobic_fraction` is the strongest single discriminative feature**
+   (AUROC 0.6745), yet the selectivity proxy does not use it. The proxy relies
+   on charge and overall hydrophobicity (GRAVY), but the *fraction* of
+   hydrophobic residues carries more signal.
+
+2. **All significant risk indicators point in the expected direction**
+   (higher = more hemolytic). The features the pipeline already tracks (charge,
+   hydrophobicity, helix propensity) have real signal for hemolysis, but the
+   composite scorers cancel it out.
+
+3. **The selectivity proxy itself has weak but significant signal** (0.6095)
+   as a protective indicator. It is doing the right thing but is underpowered
+   because it ignores the strongest axes.
+
+4. **22 of 30 features tested have NO significant signal** for selective vs
+   hemolytic discrimination. This confirms the strict triage finding: 1D
+   physicochemical descriptors alone cannot solve this task well.
+
+**Implication for next steps:**
+
+A richer selectivity scorer combining `hydrophobic_fraction`, `helix_propensity`,
+`net_charge`, and `interior_trypsin_sites` in a learned or hand-tuned model
+could plausibly improve selective_vs_hemolytic AUROC above the current 0.55
+ceiling. However, the best single feature (0.6745) is still modest, and
+the CI is wide. 3D structural modelling or sequence-pattern features may
+ultimately be needed for clinically meaningful discrimination.
+
+Run: `make bench-feature-decomp` or `python -m openamp_foundry.cli bench feature-decomp`
+
 ## Test Suite
 
 | Metric | Value |
@@ -540,4 +595,5 @@ selection.
 | 2026-07-01 | Within-AMP selectivity benchmark added: safety scorer FAILS hemolysis detection (AUROC=0.3844); synthesis is only significant risk detector (AUROC=0.8027); expert composite better than ensemble but not significant (0.5119 vs 0.3486) | OpenAMP loop |
 | 2026-07-01 | Expert composite ranking integration: `score_candidates()` now computes `expert_composite` and `hemolysis_risk`; `--ranking-mode expert` CLI flag; expert-ranked top-5 have lower mean hemolysis_risk than ensemble | OpenAMP loop |
 | 2026-07-02 | **Strict triage benchmark added:** composition-matched scrambled decoys replace random background. No scorer triages correctly — standard triage "success" of selectivity_proxy (0.782 sel_vs_dec) and expert_composite (0.757) was inflated by trivially distinguishable decoys. selectivity_proxy collapses to 0.500 (purely composition-driven), ensemble drops to 0.572. Real bottleneck (selective_vs_hemolytic) unchanged. | OpenAMP loop |
+| 2026-07-03 | **Feature decomposition benchmark added:** per-feature selective_vs_hemolytic AUROC for all 30 scalar physicochemical features. hydrophobic_fraction is the strongest single discriminative feature (0.6745, CI 0.58-0.77) but is NOT used by the selectivity proxy. 8/30 features have significant signal; 6 of those are unused. Provides actionable diagnostic for why composite scorers fail selective_vs_hemolytic discrimination. | OpenAMP loop |
 | 2026-06-29 | Initial — expanded benchmark (PR #110) | OpenAMP CI |

docs/ROADMAP.md

@@ -276,6 +276,33 @@ confirms the real bottleneck — selective-vs-hemolytic discrimination — is
 unchanged and requires structural or contextual features beyond current 1D
 physicochemical scorers.
 
+## v0.5.15 — Feature Decomposition Benchmark ✓ (2026-07-03)
+
+The strict triage benchmark (v0.5.14) proved no composite scorer passes
+selective_vs_hemolytic discrimination (AUROC 0.43-0.54). But it did not explain
+*why* — only that the aggregate fails. This benchmark decomposes the failure
+into per-feature contributions: which individual physicochemical features have
+statistically significant signal for distinguishing hemolytic from selective AMPs?
+
+Changes:
+- `benchmark/feature_decomp.py`: `run_feature_decomposition_benchmark()` — tests
+  all 30 scalar features from `compute_features()` individually for selective_vs_hemolytic
+  AUROC with bootstrap CIs, direction, significance, and selectivity-proxy usage flags
+- `benchmark/metrics_snapshot.py`: feature_decomposition section added to snapshot
+- `cli/commands/benchmark.py`: `bench feature-decomp` CLI command
+- `Makefile`: `bench-feature-decomp` target
+- `tests/test_feature_decomp.py`: 20 tests covering structure, findings, and snapshot integration
+- `outputs/metrics_snapshot.json`: regenerated with feature decomposition results
+- `docs/METRICS_CURRENT.md`: feature decomposition results table and interpretation
+
+Key honest finding: **The selectivity proxy ignores the strongest discriminative
+features.** `hydrophobic_fraction` (AUROC 0.6745, CI 0.58-0.77) is the single best
+feature for selective_vs_hemolytic discrimination, yet the proxy uses only charge
+and GRAVY. 8 of 30 features have significant signal; 6 of those 8 are NOT used
+by the current selectivity model. This converts the v0.5.14 aggregate failure
+into actionable diagnostic information: the next loop knows exactly which feature
+axes to combine into a richer selectivity scorer.
+
 ## v1.0 — Validated dry-lab-to-wet-lab loop
 
 - independently reviewed assay batch (expert_review.yml GitHub issue template);