Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 3 additions & 3 deletions docs/50_LOOP_PLAN.md
Original file line number Diff line number Diff line change
Expand Up @@ -100,7 +100,7 @@ Build the multi-resolution virtual assay layer: structure proxies, membrane inte
|------|-----------|-------------|-------------|
| 30 ✅ | No explicit virtual-assay scope document; stale BENCHMARKING.md; stale test count in METRICS_CURRENT.md; broken script ref in NOVELTY_AUDIT_GUIDE.md | `docs/VIRTUAL_ASSAY_SCOPE.md` — scope doc with modeled/NOT-modeled effects, uncertainty policy, ablation requirement, integration modes. Rewrote `docs/BENCHMARKING.md` with all 19 benchmarks. Fixed test count, script ref, scripts/__init__.py. | Scope doc reviewable; BENCHMARKING.md matches Makefile; 1843 tests |
| 31 ✅ | No membrane interaction proxy. The simulation module had only a dummy placeholder | `simulation/membrane.py`: MembraneProxy with Wimley-White interfacial and octanol scales, bacterial/mammalian binding scores, selectivity ratio, and uncertainty. BomanBaseline clamped. v0.5.52 | 30 tests, 1873 total. Magainin more selective than melittin (verified by test). Membrane proxy initialised, benchmark deferred to Loop 34 |
| 32 | No structure ensemble proxy. Current pipeline assumes all AMPs are helical | `simulation/structure.py`: secondary structure prediction (helix/coil/sheet propensity from sequence). Generates 3-state ensemble weights. Flags non-helical candidates | For known beta-sheet AMPs (protegrin, tachyplesin), reports low helix confidence |
| 32 ✅ | No structure ensemble proxy. Current pipeline assumes all AMPs are helical | `simulation/structure.py`: StructureProxy with Chou-Fasman 3-state propensities, normalized ensemble weights, non_helical flag. v0.5.53 | 34 tests, 1905 total. Protegrin/tachyplesin non-helical (✅), magainin not non-helical (✅). Melittin ambiguous — Chou-Fasman limitation documented |
| 33 | No selectivity ratio module that combines bacterial and mammalian membrane scores | `simulation/selectivity_ratio.py`: ratio of bacterial membrane binding to RBC membrane binding proxy. Reports as `selectivity_ratio` with uncertainty | Distinguishes known selective (magainin) from known hemolytic (melittin) at ratio > 2x |
| 34 | No benchmark set for membrane proxy calibration | `examples/validation/membrane_reference.csv`: 30+ AMPs with known membrane activity (pore-forming, carpet, toroidal pore). Literature-sourced with citations | Dataset card exists with source, citation, and known limitations |
| 35 | No per-module ablation: each simulation module must beat a cheap heuristic baseline before affecting selection | `benchmark/simulation_ablation.py`: compares each simulation module against trivial baseline (single-feature proxy). Reports delta AUROC for each | Any module with delta ≤ 0 is flagged as `NO_IMPROVEMENT` and disabled from production pipeline |
Expand Down Expand Up @@ -180,7 +180,7 @@ If no data arrives, virtual assay scaffolding continues independently.
Phase 0: ✅ Complete (Loops 1–8)
Phase 1: ✅ Complete (Loops 9–17)
Phase 2: ✅ Complete (Loops 18–29)
Phase 3: In progress — Loop 31 ✅ (Loops 30–39)
Phase 3: In progress — Loop 32 ✅ (Loops 30–39)
Phase 4: Not started (Loops 40–49)
```

Expand Down Expand Up @@ -241,7 +241,7 @@ Phase 4: Not started (Loops 40–49)
| 28 ✅ | Policy version bump workflow for when real data arrives | `scripts/bump_recalibration_policy.py`: standalone script with `--dry-run`, decision-log guard, auto-increment + write. CI guard in `ci.yml` validates policy version changes against base branch. v0.5.49. 9 tests. | CI rejects policy PRs without valid decision log; `make bump-policy-version` and `make bump-policy-version-dry-run` available |
| 29 ✅ | No public negative-result archive format. If Wave 1 yields all negatives, where do they go? | `docs/NEGATIVE_RESULT_ARCHIVE.md`: full template with entry schema, procedures, automation notes, and limitations. Covers pre-selection rejects, selected-untested, lab inactives, lab toxic, control failures. v0.5.50. | Template complete enough for a lab partner to fill; schema defines 18 fields with required/conditional markers |

**Next loop:** Loop 32 — Phase 3 (structure ensemble proxy).
**Next loop:** Loop 33 — Phase 3 (selectivity ratio module).

**Phase 2 exit criteria (all 5 met ✅):**
- ✅ `make calibration-loop` runs from clean checkout, produces batch-2 manifest
Expand Down
2 changes: 1 addition & 1 deletion docs/ARCHITECTURE.md
Original file line number Diff line number Diff line change
Expand Up @@ -42,7 +42,7 @@ The purpose of the added layers is not simulation theater. It is to improve whic
| `openamp_foundry.reports` | human-readable and machine-readable batch, calibration, and wet-lab review reports |
| `openamp_foundry.benchmark` | leakage checks, cluster-split benchmark, expert ablation benchmark, within-AMP selectivity benchmark, triage benchmark, per-feature selectivity decomposition, and evaluation scaffolding |
| `openamp_foundry.generators` | safe, bounded toy candidate generation |
| `openamp_foundry.simulation` | membrane proxy (Loop 31), dummy stub, interfaces — structure/selectivity/stability scaffolding |
| `openamp_foundry.simulation` | membrane proxy (Loop 31), structure proxy (Loop 32), dummy stub, interfaces |
| `openamp_foundry.calibration` | lab-result intake (v0.5.19), pre-registered recalibration policy + gate (v0.5.20), engine + report (v0.5.36/v0.5.44), policy version tracking (v0.5.40) |
| `openamp_foundry.active_learning` | batch-2 selector (v0.5.45) and recovery benchmark (v0.5.46) for choosing informative next experiments under uncertainty |
| `openamp_foundry.analysis` | diversity clustering, panel similarity, family structural warnings, audit helpers |
Expand Down
7 changes: 4 additions & 3 deletions docs/METRICS_CURRENT.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,8 @@ Machine-readable snapshot: `outputs/metrics_snapshot.json` regenerated with `mak
> **Purpose:** One authoritative table of current pipeline metrics. If any doc disagrees
> with this file, this file wins. Updated whenever benchmark/benchmark config changes.
>
> **Last updated:** 2026-07-06 (membrane proxy — v0.5.52)
> **Last updated:** 2026-07-06 (structure proxy — v0.5.53)
> **New in v0.5.53:** Structure ensemble proxy — `StructureProxy` using Chou-Fasman 3-state helix/sheet/coil propensities. `non_helical` flag for helic-biased scorer warnings. `HelicityBaseline`. 34 new tests. 1905 total.
> **New in v0.5.52:** Membrane interaction proxy — `MembraneProxy` using Wimley-White interfacial/octanol scales for coarse-grained bacterial and mammalian binding energy scores. `BomanBaseline` clamped. 30 new tests. 1873 total.
> **New in v0.5.51:** Virtual assay scope document — `docs/VIRTUAL_ASSAY_SCOPE.md` with uncertainty policy, ablation requirements, integration modes. Doc drift fixes: BENCHMARKING.md, METRICS_CURRENT.md test count, NOVELTY_AUDIT_GUIDE.md script ref. No code changes.
> **New in v0.5.50:** Negative-result archive template — `docs/NEGATIVE_RESULT_ARCHIVE.md` with full 18-field entry schema, automation notes, and limitations. Phase 2 exit criteria: all 5 met ✅. No code changes.
Expand Down Expand Up @@ -33,7 +34,7 @@ Machine-readable snapshot: `outputs/metrics_snapshot.json` regenerated with `mak
> **New in v0.5.31:** Added dipeptide-order features for sequence-order awareness. `dipeptide_order_score` achieves AUROC 0.7861 on AMP-vs-scrambled discrimination — the strongest order-dependent feature in the pipeline. Only 7/31 features survive scrambling (amphipathicity/helix-wheel + dipeptide). All composition features are purely position-independent (exactly 0.5000 AUROC on scrambled test).
> **New in v0.5.30:** Easy baseline benchmark added — charge density alone (AUROC 0.8166) outperforms the full pipeline ensemble (0.7792) on AMP-vs-Swiss-Prot-decoy discrimination. Honest finding documented: expected because pipeline optimizes for safety, not raw discrimination.
> **New in v0.5.29:** Expanded benchmark to 500 AMPs + 500 composition-matched decoys (n=1000). AUROC 0.7792 (CI₉₅: 0.7505–0.8065) confirms signal generalizes. Cluster-aware CI: 0.746–0.8102. Representative AUROC: 0.778. Standard benchmark (n=191) retained for backward comparison.
> **Pipeline version:** v0.5.52
> **Pipeline version:** v0.5.53
> **Branch:** main

---
Expand Down Expand Up @@ -1103,7 +1104,7 @@ Decoys score low on activity. Selective AMPs score moderately on both.

| Metric | Value |
|--------|-------|
| Total tests | 1873 |
| Total tests | 1905 |
| Coverage (branch) | 99% (6 CLI guard lines only) |
| Source modules at 100% | All pipeline, QC, scoring modules |

Expand Down
20 changes: 20 additions & 0 deletions docs/ROADMAP.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,25 @@
# Roadmap

## v0.5.53 — Structure Ensemble Proxy (Loop 32) ✓ (2026-07-06)

Second Phase 3 simulation module. Chou-Fasman 3-state secondary structure
prediction (helix/sheet/coil). Flags non-helical candidates where the
helic-centric activity scorer is unreliable.

Changes:
- ``src/openamp_foundry/simulation/structure.py`` — ``StructureProxy`` implementing
``VirtualAssayProxy`` with Chou-Fasman helix/sheet/coil propensity parameters.
Generates normalized 3-state ensemble weights, raw propensities, and
``non_helical`` flag (True if dominant state != helix or helix_weight < 0.33).
``HelicityBaseline`` for cheap-baseline comparison.
- ``tests/test_structure_proxy.py`` — 34 tests covering scale lookup,
normalization, dominant state, non-helical flag, known reference comparisons
(magainin helical, protegrin/tachyplesin beta-sheet, proline-rich non-helical,
melittin ambiguous — documented limitation), and empty sequence.
- ``docs/50_LOOP_PLAN.md`` — Loop 32 ✅. Next: Loop 33.
- ``docs/METRICS_CURRENT.md`` — Pipeline version v0.5.53. Test count: 1905.
- 1905 tests passing (1871 existing + 34 new).

## v0.5.52 — Membrane Interaction Proxy (Loop 31) ✓ (2026-07-06)

Phase 3 begins implementation. Membrane proxy uses Wimley-White interfacial
Expand Down
2 changes: 2 additions & 0 deletions src/openamp_foundry/simulation/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -6,10 +6,12 @@
VirtualAssayProxy,
)
from .membrane import MembraneProxy
from .structure import StructureProxy

__all__ = [
"SimulationResult",
"VirtualAssayProxy",
"EmulatorBaseline",
"MembraneProxy",
"StructureProxy",
]
156 changes: 156 additions & 0 deletions src/openamp_foundry/simulation/structure.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,156 @@
"""Structure ensemble proxy for virtual assay layer.

Computes 3-state secondary structure propensities (helix/sheet/coil)
using Chou-Fasman parameters. Generates normalized ensemble weights
and flags non-helical candidates where the helic-centric activity
scorer is unreliable.

Reference: Chou PY & Fasman GD (1974) Biochemistry 13:222-245.
"""

from .interfaces import EmulatorBaseline, SimulationResult, VirtualAssayProxy

# Chou-Fasman alpha-helix propensity parameters (Palpha).
# Source: Chou & Fasman (1974) Biochemistry 13:222-245, Table IV.
# Values > 1.0 are helix-forming; < 0.75 helix-breaking.
_HELIX: dict[str, float] = {
"A": 1.42, "R": 0.98, "N": 0.67, "D": 1.01, "C": 0.70,
"Q": 1.11, "E": 1.51, "G": 0.57, "H": 1.00, "I": 1.08,
"L": 1.21, "K": 1.16, "M": 1.45, "F": 1.13, "P": 0.57,
"S": 0.77, "T": 0.83, "W": 1.08, "Y": 0.69, "V": 1.06,
}

# Chou-Fasman beta-sheet propensity parameters (Pbeta).
_SHEET: dict[str, float] = {
"A": 0.83, "R": 0.93, "N": 0.89, "D": 0.54, "C": 1.19,
"Q": 1.10, "E": 0.37, "G": 0.75, "H": 0.87, "I": 1.60,
"L": 1.30, "K": 0.74, "M": 1.05, "F": 1.38, "P": 0.55,
"S": 0.75, "T": 1.19, "W": 1.37, "Y": 1.47, "V": 1.70,
}

# Chou-Fasman coil / turn propensity parameters (Pt / Pc).
_COIL: dict[str, float] = {
"A": 0.66, "R": 0.95, "N": 1.56, "D": 1.46, "C": 0.70,
"Q": 0.98, "E": 0.74, "G": 1.56, "H": 0.95, "I": 0.47,
"L": 0.59, "K": 1.01, "M": 0.60, "F": 0.60, "P": 1.52,
"S": 1.43, "T": 0.96, "W": 0.96, "Y": 1.14, "V": 0.50,
}


def _mean_propensity(sequence: str, scale: dict[str, float]) -> float:
if not sequence:
return 1.0
total = 0.0
count = 0
for aa in sequence.upper():
if aa in scale:
total += scale[aa]
count += 1
return total / count if count else 1.0


def _normalize_weights(h: float, e: float, c: float) -> tuple[float, float, float]:
"""Normalize three propensity scores to sum to 1.0.

Converts Chou-Fasman propensities (centered at 1.0) to fractional
ensemble weights by taking their relative magnitudes.
"""
total = h + e + c
if total <= 0.0:
return (0.0, 0.0, 0.0)
return (h / total, e / total, c / total)


def _dominant_state(helix: float, sheet: float, coil: float) -> str:
"""Return 'helix', 'sheet', or 'coil' as the predicted dominant state."""
if helix >= sheet and helix >= coil:
return "helix"
if sheet >= coil:
return "sheet"
return "coil"


def _is_non_helical(dominant: str, helix_weight: float) -> bool:
"""Return True if the candidate is likely non-helical.

Non-helical means the dominant state is NOT helix, or helix
weight is below 0.33 (indifferent / ambiguous).
"""
return dominant != "helix" or helix_weight < 0.33


def _uncertainty(sequence: str) -> float:
"""Uncertainty estimate for structure predictions.

Higher uncertainty when:
- Sequence is short (< 12 AA): insufficient residues for pattern
- Sequence is very long (> 40 AA): Chou-Fasman less reliable for multi-domain
"""
length = len(sequence) if sequence else 0
if length == 0:
return 1.0
if length < 12:
return round(1.0 - length / 12.0, 4)
if length > 40:
return round(min((length - 40) / 40.0, 1.0), 4)
return 0.15 # baseline uncertainty for well-behaved sequences


class HelicityBaseline(EmulatorBaseline):
"""Mean Chou-Fasman helix propensity as a cheap baseline."""

def evaluate(self, sequence: str) -> float:
return _mean_propensity(sequence, _HELIX)


class StructureProxy(VirtualAssayProxy):
"""Secondary structure ensemble proxy.

Predicts helix/sheet/coil propensities from sequence using
Chou-Fasman parameters. Flags non-helical candidates where
the helic-centric activity scorer is unreliable.
"""

def __init__(self, version: str = "0.1.0") -> None:
self._version = version

def simulate(self, sequence: str) -> SimulationResult:
h = _mean_propensity(sequence, _HELIX)
e = _mean_propensity(sequence, _SHEET)
c = _mean_propensity(sequence, _COIL)

w_h, w_e, w_c = _normalize_weights(h, e, c)
dominant = _dominant_state(w_h, w_e, w_c)
non_helical = _is_non_helical(dominant, w_h)
uncertainty = _uncertainty(sequence)

return SimulationResult(
module="structure_proxy",
version=self._version,
scope=[
"secondary_structure_prediction",
"non_helical_flag",
],
scores={
"helix_weight": round(w_h, 4),
"sheet_weight": round(w_e, 4),
"coil_weight": round(w_c, 4),
"helix_propensity_raw": round(h, 4),
"sheet_propensity_raw": round(e, 4),
"coil_propensity_raw": round(c, 4),
"non_helical": 1.0 if non_helical else 0.0,
},
uncertainty=uncertainty,
calibration_set=None,
validated_against=[],
notes=[
"3-state Chou-Fasman secondary structure prediction.",
f"Dominant predicted state: {dominant}.",
"Non-helical flag: True if dominant != helix or helix_weight < 0.33.",
"Not validated against any experimental structure data.",
"Uncertainty > 0.5 = experimental, must not affect selection.",
],
)

def get_baseline(self) -> EmulatorBaseline:
return HelicityBaseline()
Loading
Loading