cschanhniem · Jul 6, 2026 · Jul 6, 2026
diff --git a/Makefile b/Makefile
@@ -439,7 +439,11 @@ wave0-5-evidence:
 
 bench-simulation-ablation:
 	PYTHONPATH=src $(PYTHON) scripts/benchmark_simulation_ablation.py \
-		--out outputs/simulation_ablation.json
+		--mode amp-vs-decoy --out outputs/simulation_ablation.json
+
+bench-simulation-ablation-within-amp:
+	PYTHONPATH=src $(PYTHON) scripts/benchmark_simulation_ablation.py \
+		--mode within-amp --out outputs/simulation_ablation_within_amp.json
 
 bench-cross-dataset:
 	PYTHONPATH=src $(PYTHON) scripts/benchmark_cross_dataset.py \

diff --git a/docs/50_LOOP_PLAN.md b/docs/50_LOOP_PLAN.md
@@ -102,7 +102,7 @@ Build the multi-resolution virtual assay layer: structure proxies, membrane inte
 | 31 ✅ | No membrane interaction proxy. The simulation module had only a dummy placeholder | `simulation/membrane.py`: MembraneProxy with Wimley-White interfacial and octanol scales, bacterial/mammalian binding scores, selectivity ratio, and uncertainty. BomanBaseline clamped. v0.5.52 | 30 tests, 1873 total. Magainin more selective than melittin (verified by test). Membrane proxy initialised, benchmark deferred to Loop 34 |
 | 32 ✅ | No structure ensemble proxy. Current pipeline assumes all AMPs are helical | `simulation/structure.py`: StructureProxy with Chou-Fasman 3-state propensities, normalized ensemble weights, non_helical flag. v0.5.53 | 34 tests, 1905 total. Protegrin/tachyplesin non-helical (✅), magainin not non-helical (✅). Melittin ambiguous — Chou-Fasman limitation documented |
 | 33 ✅ | No per-module ablation: do simulation modules improve discrimination? (Original 'selectivity ratio' superseded by Loop 31 membrane proxy — replaced with more useful ablation benchmark) | `scripts/benchmark_simulation_ablation.py`: tests MembraneProxy + StructureProxy on 500-AMP AMP-vs-decoy benchmark. Reports per-module AUROC, combined composite AUROC, and verdict. v0.5.54 | 17 tests, 1922 total. Honest finding: composite degrades AMP-vs-decoy (delta=-0.1153), but bacterial_binding alone achieves 0.7512 AUROC — genuine non-charge signal from Wimley-White + hydrophobic moment |
-| 34 | No benchmark set for membrane proxy calibration | `examples/validation/membrane_reference.csv`: 30+ AMPs with known membrane activity (pore-forming, carpet, toroidal pore). Literature-sourced with citations | Dataset card exists with source, citation, and known limitations |
+| 34 ✅ | No within-AMP simulation ablation: do simulation modules improve hemolysis detection? (Original 'membrane reference CSV' deferred — higher leverage to test modules on their actual task first) | Extended `scripts/benchmark_simulation_ablation.py` with `--mode within-amp`. Tests MembraneProxy + StructureProxy on 45 hemolytic vs 125 selective AMPs. Reports per-score detection AUROC. v0.5.55 | 23 tests, 1928 total. Honest finding: best simulation helix_weight AUROC 0.6458; rich_selectivity still best at 0.7453. Simulation modules do NOT improve over existing scorers for hemolysis detection |
 | 35 | No per-module ablation: each simulation module must beat a cheap heuristic baseline before affecting selection | `benchmark/simulation_ablation.py`: compares each simulation module against trivial baseline (single-feature proxy). Reports delta AUROC for each | Any module with delta ≤ 0 is flagged as `NO_IMPROVEMENT` and disabled from production pipeline |
 | 36 | No uncertainty-aware composite that folds simulation outputs into the ranking | `scoring/simulation_composite.py`: weighted combination of simulation modules, weighted by their benchmarked AUROC. Uncertainty from `SimulationResult` propagates to final score | Uncertainty reported alongside every candidate score in evidence certificate |
 | 37 | No simulation module that meets the bar to affect selection. All may be experimental | Add `--simulation-mode` flag to the `rank` command: `off` (default, current pipeline), `info` (simulation scores in report only, no ranking impact), `weighted` (simulation affects ranking if benchmarked AUROC > 0.6) | Each mode produces distinct, documented output |
@@ -180,7 +180,7 @@ If no data arrives, virtual assay scaffolding continues independently.
 Phase 0: ✅ Complete (Loops 1–8)
 Phase 1: ✅ Complete (Loops 9–17)
 Phase 2: ✅ Complete (Loops 18–29)
-Phase 3: In progress — Loop 33 ✅ (Loops 30–39)
+Phase 3: In progress — Loop 34 ✅ (Loops 30–39)
 Phase 4: Not started (Loops 40–49)
 ```
 
@@ -241,7 +241,7 @@ Phase 4: Not started (Loops 40–49)
 | 28 ✅ | Policy version bump workflow for when real data arrives | `scripts/bump_recalibration_policy.py`: standalone script with `--dry-run`, decision-log guard, auto-increment + write. CI guard in `ci.yml` validates policy version changes against base branch. v0.5.49. 9 tests. | CI rejects policy PRs without valid decision log; `make bump-policy-version` and `make bump-policy-version-dry-run` available |
 | 29 ✅ | No public negative-result archive format. If Wave 1 yields all negatives, where do they go? | `docs/NEGATIVE_RESULT_ARCHIVE.md`: full template with entry schema, procedures, automation notes, and limitations. Covers pre-selection rejects, selected-untested, lab inactives, lab toxic, control failures. v0.5.50. | Template complete enough for a lab partner to fill; schema defines 18 fields with required/conditional markers |
 
-**Next loop:** Loop 34 — Phase 3 (membrane reference benchmark set).
+**Next loop:** Loop 35 — Phase 3 (membrane reference benchmark set or simulation-mode CLI flag).
 
 **Phase 2 exit criteria (all 5 met ✅):**
 - ✅ `make calibration-loop` runs from clean checkout, produces batch-2 manifest

diff --git a/docs/BENCHMARKING.md b/docs/BENCHMARKING.md
@@ -59,6 +59,7 @@ documented result in `docs/METRICS_CURRENT.md` and
 | `bench-expert-ablation` | Ablate expert composite components on n=191 | Document per-component AUROC; expert lower than ensemble on AMP-vs-decoy (by design) | `make bench-expert-ablation` |
 | `bench-expert-ablation-500` | Ablate expert composite on expanded n=1000 | Re-run on diverse set; 2 components reclassified from n=191 | `make bench-expert-ablation-500` |
 | `bench-simulation-ablation` | Ablate MembraneProxy + StructureProxy on 500-AMP benchmark | Composite degrades AMP-vs-decoy (expected — modules are for within-AMP tasks); bacterial_binding alone AUROC 0.7512 (non-charge signal) | `make bench-simulation-ablation` |
+| `bench-simulation-ablation-within-amp` | Ablate simulation modules on within-AMP hemolysis detection | Best simulation helix_weight AUROC 0.6458; rich_selectivity best at 0.7453. No improvement | `make bench-simulation-ablation-within-amp` |
 
 ### Regression / CI
 

diff --git a/docs/METRICS_CURRENT.md b/docs/METRICS_CURRENT.md
@@ -5,7 +5,8 @@ Machine-readable snapshot: `outputs/metrics_snapshot.json` regenerated with `mak
 > **Purpose:** One authoritative table of current pipeline metrics. If any doc disagrees
 > with this file, this file wins. Updated whenever benchmark/benchmark config changes.
 >
-> **Last updated:** 2026-07-06 (simulation ablation — v0.5.54)
+> **Last updated:** 2026-07-06 (within-AMP simulation ablation — v0.5.55)
+> **New in v0.5.55:** Within-AMP simulation ablation — `--mode within-amp` tests MembraneProxy + StructureProxy on hemolysis detection (45 hemolytic vs 125 selective). Honest finding: best simulation `helix_weight` AUROC 0.6458; `rich_selectivity` still best at 0.7453. 23 tests. 1928 total.
 > **New in v0.5.54:** Simulation ablation benchmark — tests MembraneProxy + StructureProxy on 500-AMP AMP-vs-decoy benchmark. Honest finding: composite degrades (delta=−0.1153) but `bacterial_binding` alone achieves AUROC 0.7512 — genuine non-charge signal. 17 new tests. 1922 total.
 > **New in v0.5.53:** Structure ensemble proxy — `StructureProxy` using Chou-Fasman 3-state helix/sheet/coil propensities. `non_helical` flag for helic-biased scorer warnings. `HelicityBaseline`. 34 new tests. 1905 total.
 > **New in v0.5.52:** Membrane interaction proxy — `MembraneProxy` using Wimley-White interfacial/octanol scales for coarse-grained bacterial and mammalian binding energy scores. `BomanBaseline` clamped. 30 new tests. 1873 total.
@@ -35,7 +36,7 @@ Machine-readable snapshot: `outputs/metrics_snapshot.json` regenerated with `mak
 > **New in v0.5.31:** Added dipeptide-order features for sequence-order awareness. `dipeptide_order_score` achieves AUROC 0.7861 on AMP-vs-scrambled discrimination — the strongest order-dependent feature in the pipeline. Only 7/31 features survive scrambling (amphipathicity/helix-wheel + dipeptide). All composition features are purely position-independent (exactly 0.5000 AUROC on scrambled test).
 > **New in v0.5.30:** Easy baseline benchmark added — charge density alone (AUROC 0.8166) outperforms the full pipeline ensemble (0.7792) on AMP-vs-Swiss-Prot-decoy discrimination. Honest finding documented: expected because pipeline optimizes for safety, not raw discrimination.
 > **New in v0.5.29:** Expanded benchmark to 500 AMPs + 500 composition-matched decoys (n=1000). AUROC 0.7792 (CI₉₅: 0.7505–0.8065) confirms signal generalizes. Cluster-aware CI: 0.746–0.8102. Representative AUROC: 0.778. Standard benchmark (n=191) retained for backward comparison.
-> **Pipeline version:** v0.5.54
+> **Pipeline version:** v0.5.55
 > **Branch:** main
 
 ---
@@ -1105,7 +1106,7 @@ Decoys score low on activity. Selective AMPs score moderately on both.
 
 | Metric | Value |
 |--------|-------|
-| Total tests | 1922 |
+| Total tests | 1928 |
 | Coverage (branch) | 99% (6 CLI guard lines only) |
 | Source modules at 100% | All pipeline, QC, scoring modules |
 

diff --git a/docs/ROADMAP.md b/docs/ROADMAP.md
@@ -1,5 +1,31 @@
 # Roadmap
 
+## v0.5.55 — Within-AMP Simulation Ablation (Loop 34) ✓ (2026-07-06)
+
+Extended the simulation ablation benchmark to test modules on their
+designed task: within-AMP hemolysis detection. Original Loop 34 plan
+(membrane reference CSV) deferred — higher leverage to test modules
+on actual task first.
+
+Changes:
+- ``scripts/benchmark_simulation_ablation.py`` — Extended with ``--mode
+  within-amp`` flag. Tests 45 hemolytic vs 125 selective AMPs from
+  ``examples/validation/hemolysis_reference.csv``. Reports per-score
+  hemolysis detection AUROC (inverted for safety-type scorers).
+- ``Makefile`` — Added ``bench-simulation-ablation-within-amp`` target.
+- ``docs/50_LOOP_PLAN.md`` — Loop 34 ✅ (re-interpreted). Next: Loop 35.
+- ``docs/METRICS_CURRENT.md`` — Pipeline version v0.5.55. Test count: 1928.
+- 1928 tests passing (1905 existing + 23 updated).
+
+**Honest finding:**
+- ``rich_selectivity`` remains the best hemolysis detector (AUROC 0.7453).
+- Best simulation score: ``helix_weight`` (AUROC 0.6458) — hemolytic AMPs tend
+  to be more helical. Biologically meaningful but not competitive.
+- Simulation composite delta=-0.0995 vs rich_selectivity — modules do NOT
+  improve over existing scorers for hemolysis detection.
+- Implication: membrane and structure proxies need structural or contextual
+  features beyond 1D Chou-Fasman propensities and Wimley-White averages.
+
 ## v0.5.54 — Simulation Ablation Benchmark (Loop 33) ✓ (2026-07-06)
 
 First honest test of whether Phase 3 simulation modules improve