Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -42,6 +42,7 @@ This repo gives you a safe starting point for:
- penalizing likely safety/synthesis risks;
- selecting diverse candidates;
- generating auditable JSON evidence certificates;
- bundling lab-review packs with chain-of-custody identity hashes;
- running a demo pipeline without downloading external biological datasets;
- expanding later with real predictors and qualified external validation.

Expand All @@ -61,6 +62,7 @@ candidate sequences
-> novelty score against references
-> ensemble rank
-> evidence certificate
-> chain-of-custody hash manifest
-> expert-review package, if human review approves
```

Expand Down
2 changes: 1 addition & 1 deletion docs/50_LOOP_PLAN.md
Original file line number Diff line number Diff line change
Expand Up @@ -130,7 +130,7 @@ Prepare for real wet-lab partners. Make the pack reviewable, the evidence tracea
| 42 | No expert-review GitHub template. Review process is ad-hoc | `.github/ISSUE_TEMPLATE/expert_review.yml`: structured form for expert reviewers covering novelty, safety, mechanism plausibility, experiment design | Template generates a review issue with all required fields |
| 43 ✅ | No decision-log format for human review. AGENTS.md requires human review for 6 decision types but no machine-readable log format exists | `schemas/decision_log.schema.json`: JSON Schema (Draft 2020-12) with 12 fields, 9 decision types, dissent-requires-notes conditional. v0.5.64 | 11 tests, 2000 total. Schema validates against real decision examples |
| 44 ✅ | No candidate batch pack that a lab partner can download as a single artifact | `scripts/build_lab_batch_pack.py`: generates a zip with candidate CSV, evidence certificates, protocol refs, controls manifest, and data return template. `make lab-batch-pack` target. v0.5.63 | 10 tests, 1989 total. Zip contains 36 evidence certs + 3 protocol docs + controls + data return template |
| 45 | No chain-of-custody hashing for lab samples. Lab receives peptides but can't verify identity | Add SHA-256 hash of each candidate sequence and synthesis order to the batch pack. Lab can verify received peptides match | Hash verification script works end-to-end on synthetic data |
| 45 ✅ | No chain-of-custody hashing for lab samples. Lab receives peptides but can't verify identity | `scripts/build_lab_batch_pack.py` now writes `chain_of_custody.json` + `MANIFEST.json` into the lab batch pack, including SHA-256 hashes for `panel.csv`, the ordered candidate list, each candidate sequence, and each evidence certificate. `--verify-pack` re-checks archive integrity end to end. v0.5.65 | Verification script passes on synthetic data and fails on tampered packs |
| 46 | No pre-registered analysis plan for Wave 1 results. What analysis runs when data comes back? | `docs/WAVE1_ANALYSIS_PLAN.md`: pre-registered statistical analysis (primary: MIC ≤ 32 hit rate; secondary: selectivity index; exploratory: per-family correlation). No post-hoc changes allowed | Plan is timestamped and locked before data return |
| 47 ✅ | No data-return validation tool. Lab sends CSV/JSON files; pipeline needs to validate them against schema | `scripts/validate_lab_data_return.py`: validates JSON files against `schemas/lab_result.schema.json`, reports per-file errors, detects control failures. v0.5.67 | 8 tests, 2008 total. Detects missing fields, invalid JSON, control failures |
| 48 | No public-data release package. If results are publishable, what goes in the supplement? | `docs/PUBLICATION_PACK.md`: checklist for public release — benchmark data, evidence certificates, candidate scores, negative results, analysis scripts, README for reproduction | Pack is complete enough for a reviewer to reproduce all findings |
Expand Down
12 changes: 12 additions & 0 deletions docs/LAB_PARTNER_ONBOARDING.md
Original file line number Diff line number Diff line change
Expand Up @@ -59,11 +59,23 @@ partner_packet/
unsupported_claims.md
pre_registered_pilot_template.md
result_intake_schema_reference.md
chain_of_custody.json
release_review.md
```

See [`EXTERNAL_REVIEW_PACKET.md`](EXTERNAL_REVIEW_PACKET.md) for the full packet standard.

Current lab batch packs generated by `scripts/build_lab_batch_pack.py` include
`chain_of_custody.json` and `MANIFEST.json`. A receiver can run:

```bash
python scripts/build_lab_batch_pack.py --verify-pack outputs/lab_batch_pack.zip
```

The check verifies archive identity only: candidate IDs, sequences, panel order,
and evidence certificate files. It does not verify synthesis, biological
activity, safety, or experimental provenance after receipt.

## What a lab partner does not receive from this repo

OpenAMP should not provide:
Expand Down
3 changes: 2 additions & 1 deletion docs/METRICS_CURRENT.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@ Machine-readable snapshot: `outputs/metrics_snapshot.json` regenerated with `mak
> with this file, this file wins. Updated whenever benchmark/benchmark config changes.
>
> **Last updated:** 2026-07-06 (data-return validation — v0.5.67)
> **New in v0.5.65:** `scripts/build_lab_batch_pack.py` now writes `chain_of_custody.json` and `MANIFEST.json` into lab batch packs. The custody file includes SHA-256 hashes for `panel.csv`, the ordered candidate list, each candidate sequence, and each evidence certificate. `--verify-pack` verifies archive integrity and detects tampering. These hashes prove identity/integrity only, not biological activity, safety, or synthesis success. 5 tests.
> **New in v0.5.64:** `schemas/decision_log.schema.json` — 12-field JSON Schema for human review decisions. Covers 9 decision types from AGENTS.md §8. Dissent conditional. 11 tests. 2000 total.
> **New in v0.5.63:** `scripts/build_lab_batch_pack.py` — generates single zip with candidate CSV, 36 evidence certs, protocol refs, controls manifest, data return template. `make lab-batch-pack`. 10 tests. 1989 total.
> **New in v0.5.62:** Pre-registered pass/fail criteria + simulation uncertainty in evidence. `configs/wave1_pass_fail.yaml` + `scripts/check_wave1_pass_fail.py`. `rank info` now propagates simulation uncertainty into evidence certs. 17 tests.
Expand Down Expand Up @@ -49,7 +50,7 @@ Machine-readable snapshot: `outputs/metrics_snapshot.json` regenerated with `mak
> **New in v0.5.31:** Added dipeptide-order features for sequence-order awareness. `dipeptide_order_score` achieves AUROC 0.7861 on AMP-vs-scrambled discrimination — the strongest order-dependent feature in the pipeline. Only 7/31 features survive scrambling (amphipathicity/helix-wheel + dipeptide). All composition features are purely position-independent (exactly 0.5000 AUROC on scrambled test).
> **New in v0.5.30:** Easy baseline benchmark added — charge density alone (AUROC 0.8166) outperforms the full pipeline ensemble (0.7792) on AMP-vs-Swiss-Prot-decoy discrimination. Honest finding documented: expected because pipeline optimizes for safety, not raw discrimination.
> **New in v0.5.29:** Expanded benchmark to 500 AMPs + 500 composition-matched decoys (n=1000). AUROC 0.7792 (CI₉₅: 0.7505–0.8065) confirms signal generalizes. Cluster-aware CI: 0.746–0.8102. Representative AUROC: 0.778. Standard benchmark (n=191) retained for backward comparison.
> **Pipeline version:** v0.5.62
> **Pipeline version:** v0.5.67
> **Branch:** main

---
Expand Down
22 changes: 22 additions & 0 deletions docs/ROADMAP.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,28 @@ Changes:
- ``docs/METRICS_CURRENT.md`` — Changelog updated. Test count: 2008.
- 2008 tests passing.

## v0.5.65 — Lab Batch Pack Chain-of-Custody Hashes (Loop 45) ✓ (2026-07-06)

Closed a real Phase 4 handoff gap: a lab batch pack could be zipped and sent,
but the receiver had no machine-checkable way to verify that candidate IDs,
sequences, ordering, and evidence files were still the intended ones.

Changes:
- `scripts/build_lab_batch_pack.py` — now writes `chain_of_custody.json` and
`MANIFEST.json` into each pack. The custody file records SHA-256 hashes for
`panel.csv`, the ordered candidate list, each candidate sequence, and each
bundled evidence certificate. Added `--verify-pack` to re-check those hashes
against an existing zip.
- `tests/test_lab_batch_pack.py` — covers custody-file generation, successful
verification, tamper detection, and CLI verification mode.
- `docs/50_LOOP_PLAN.md` — Loop 45 ✅. Next: Loop 46.
- `docs/METRICS_CURRENT.md` — version and changelog synced.

Honest boundary:
- These hashes verify identity and archive integrity only.
- They are not evidence of biological activity, safety, synthesis success, or
experimental provenance after a qualified lab receives material.

## v0.5.64 — Decision-Log Schema (Loop 43) ✓ (2026-07-06)

Machine-readable JSON Schema for human review decisions, covering the
Expand Down
141 changes: 138 additions & 3 deletions scripts/build_lab_batch_pack.py
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,7 @@
from __future__ import annotations

import csv
import hashlib
import json
import sys
import zipfile
Expand All @@ -15,6 +16,90 @@
from typing import Any


def _sha256_text(value: str) -> str:
return hashlib.sha256(value.encode("utf-8")).hexdigest()


def _sha256_bytes(value: bytes) -> str:
return hashlib.sha256(value).hexdigest()


def _read_panel_rows(panel_csv: Path) -> list[dict[str, str]]:
with panel_csv.open(newline="", encoding="utf-8") as f:
return list(csv.DictReader(f))


def _candidate_identity_rows(panel_rows: list[dict[str, str]]) -> list[dict[str, Any]]:
identities: list[dict[str, Any]] = []
for row in panel_rows:
candidate_id = row.get("candidate_id", "").strip()
sequence = row.get("sequence", "").strip().upper()
if not candidate_id or not sequence:
continue
identities.append({
"candidate_id": candidate_id,
"sequence_sha256": _sha256_text(sequence),
"sequence_length": len(sequence),
})
return identities


def _synthesis_order_digest(panel_rows: list[dict[str, str]]) -> str:
"""Hash the ordered candidate identity list, not vendor instructions."""
lines = []
for row in panel_rows:
candidate_id = row.get("candidate_id", "").strip()
sequence = row.get("sequence", "").strip().upper()
if candidate_id and sequence:
lines.append(f"{candidate_id},{sequence}")
return _sha256_text("\n".join(lines) + "\n")


def verify_batch_pack(zip_path: str | Path) -> dict[str, Any]:
"""Verify chain-of-custody hashes inside a lab batch pack zip."""
zip_p = Path(zip_path)
if not zip_p.exists():
return {"status": "error", "error": f"Pack not found: {zip_path}"}

with zipfile.ZipFile(zip_p) as zf:
names = set(zf.namelist())
required = {"panel.csv", "chain_of_custody.json", "MANIFEST.json"}
missing = sorted(required - names)
if missing:
return {"status": "failed", "missing_files": missing}

panel_bytes = zf.read("panel.csv")
custody = json.loads(zf.read("chain_of_custody.json").decode("utf-8"))
manifest = json.loads(zf.read("MANIFEST.json").decode("utf-8"))

panel_hash_ok = _sha256_bytes(panel_bytes) == custody.get("panel_csv_sha256")

panel_text = panel_bytes.decode("utf-8")
panel_rows = list(csv.DictReader(panel_text.splitlines()))
expected_identities = _candidate_identity_rows(panel_rows)
identities_ok = expected_identities == custody.get("candidate_identities", [])
order_ok = _synthesis_order_digest(panel_rows) == custody.get("synthesis_order_sha256")

evidence_ok = True
evidence_hashes = custody.get("evidence_certificate_sha256", {})
for archive_name, expected_hash in evidence_hashes.items():
if archive_name not in names or _sha256_bytes(zf.read(archive_name)) != expected_hash:
evidence_ok = False
break

ok = panel_hash_ok and identities_ok and order_ok and evidence_ok
return {
"status": "ok" if ok else "failed",
"pack": str(zip_p),
"panel_hash_ok": panel_hash_ok,
"candidate_identity_hashes_ok": identities_ok,
"synthesis_order_hash_ok": order_ok,
"evidence_hashes_ok": evidence_ok,
"candidate_count": len(expected_identities),
"manifest_candidate_count": manifest.get("candidate_count"),
}


def build_batch_pack(
panel_csv: str,
evidence_dir: str,
Expand All @@ -37,30 +122,44 @@ def build_batch_pack(
if not evidence_p.exists():
return {"error": f"Evidence directory not found: {evidence_dir}"}

panel_rows = _read_panel_rows(panel_p)
candidate_identities = _candidate_identity_rows(panel_rows)
panel_data = panel_p.read_bytes()
panel_csv_sha256 = _sha256_bytes(panel_data)
synthesis_order_sha256 = _synthesis_order_digest(panel_rows)

manifest: dict[str, Any] = {
"generated_at": datetime.now(timezone.utc).isoformat(),
"panel_csv": panel_csv,
"panel_csv_sha256": panel_csv_sha256,
"synthesis_order_sha256": synthesis_order_sha256,
"candidate_count": len(candidate_identities),
"evidence_count": 0,
"included_protocols": [],
"included_candidates": [],
"candidate_identities": candidate_identities,
"data_return_template": None,
"controls": controls or [],
}

with zipfile.ZipFile(out_p, "w", zipfile.ZIP_DEFLATED) as zf:
# 1. Panel CSV
panel_data = panel_p.read_bytes()
zf.writestr("panel.csv", panel_data)
manifest["panel_csv_size"] = len(panel_data)

# 2. Evidence certificates
ev_count = 0
evidence_hashes: dict[str, str] = {}
for f in sorted(evidence_p.iterdir()):
if f.suffix == ".json":
zf.write(str(f), f"evidence/{f.name}")
archive_name = f"evidence/{f.name}"
evidence_bytes = f.read_bytes()
zf.writestr(archive_name, evidence_bytes)
evidence_hashes[archive_name] = _sha256_bytes(evidence_bytes)
ev_count += 1
manifest["included_candidates"].append(f.stem)
manifest["evidence_count"] = ev_count
manifest["evidence_certificate_sha256"] = evidence_hashes

# 3. Protocol references (as a README)
protocols = protocol_docs or [
Expand Down Expand Up @@ -95,7 +194,6 @@ def build_batch_pack(

# 5. Data return template (empty JSON per schema)
if schema_p.exists():
schema = json.loads(schema_p.read_text())
template: dict[str, Any] = {
"result_id": "REPLACE_WITH_UUID",
"candidate_id": "REPLACE_WITH_CANDIDATE_ID",
Expand All @@ -114,6 +212,27 @@ def build_batch_pack(
zf.writestr("data_return_template.json", json.dumps(template, indent=2))
manifest["data_return_template"] = str(schema_p)

# 6. Chain-of-custody identity hashes
custody = {
"chain_of_custody_version": "1.0",
"scope": (
"Dry-lab identity check only. Hashes verify that candidate IDs, "
"sequences, panel ordering, and evidence certificate files did not drift."
),
"not_evidence_of": [
"biological activity",
"mammalian safety",
"synthesis success",
"clinical utility",
],
"panel_csv_sha256": panel_csv_sha256,
"synthesis_order_sha256": synthesis_order_sha256,
"candidate_identities": candidate_identities,
"evidence_certificate_sha256": evidence_hashes,
}
zf.writestr("chain_of_custody.json", json.dumps(custody, indent=2))
zf.writestr("MANIFEST.json", json.dumps(manifest, indent=2))

# 6. Batch README
readme = [
"# OpenAMP Foundry — Lab Batch Pack",
Expand All @@ -129,6 +248,16 @@ def build_batch_pack(
"- protocols/ — Assay protocol documents",
"- controls_manifest.json — Positive/negative control sequences",
"- data_return_template.json — Empty JSON template per lab_result.schema.json",
"- chain_of_custody.json — SHA-256 identity hashes for panel, sequences, and evidence files",
"- MANIFEST.json — Machine-readable archive manifest",
"",
"## Identity Verification",
"",
"Before using returned results, verify that panel and evidence files match:",
"",
" python scripts/build_lab_batch_pack.py --verify-pack <pack.zip>",
"",
"These hashes verify identity only. They are not evidence of activity, safety, synthesis success, or clinical utility.",
"",
"## Return Data",
"",
Expand All @@ -153,8 +282,14 @@ def main() -> int:
parser.add_argument("--evidence-dir", default="outputs/evidence_wave0_5")
parser.add_argument("--out", default="outputs/lab_batch_pack.zip")
parser.add_argument("--manifest-out", default=None, help="JSON manifest path")
parser.add_argument("--verify-pack", default=None, help="Verify an existing lab batch pack zip")
args = parser.parse_args()

if args.verify_pack:
result = verify_batch_pack(args.verify_pack)
print(json.dumps(result, indent=2))
return 0 if result.get("status") == "ok" else 3

manifest = build_batch_pack(
panel_csv=args.panel_csv,
evidence_dir=args.evidence_dir,
Expand Down
2 changes: 0 additions & 2 deletions tests/test_decision_log_schema.py
Original file line number Diff line number Diff line change
Expand Up @@ -51,7 +51,6 @@ def test_schema_decision_values():


def test_valid_decision_passes_validation():
schema = _load_schema()
example = {
"entry_id": "DR-2026-07-06-001",
"date": "2026-07-06",
Expand All @@ -67,7 +66,6 @@ def test_valid_decision_passes_validation():


def test_missing_required_field_fails():
schema = _load_schema()
example = {
"entry_id": "DR-001",
"date": "2026-07-06",
Expand Down
Loading
Loading