Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 7 additions & 1 deletion Makefile
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
.PHONY: help demo test lint ci clean bench-leakage bench-multi-negatives bench-baseline bench-hidden-active bench-cluster-split bench-expert-ablation bench-expert-ablation-500 bench-selectivity bench-feature-decomp bench-gate bench-easy-baseline bench-charge-matched bench-order-dependent bench-precision-at-k bench-per-family bench-simulation-ablation bench-simulation-ablation-within-amp simulation-gate regenerate-all generate phase3 pilot validate-scoring validate-scoring-phase3 validate-scoring-strict external-pilot pilot-confident presynth-qc gold-standard diversity synthesis-order novelty-broad external-consensus questionnaire gate-check ip-report benchmark-card wave0-5-gate-check wave0-5-novelty-audit wave0-5-novelty-audit-v2 wave0-5-panel wave0-5-evidence wave0-5-fill-external wave0-5b-generate wave0-5b-filter recalibration-engine recalibration-engine-dry-run validate-policy-version generate-synthetic-lab-results bump-policy-version generate-review-packet failed-candidate-report safe-publication-filter
.PHONY: help demo test lint ci clean bench-leakage bench-multi-negatives bench-baseline bench-hidden-active bench-cluster-split bench-expert-ablation bench-expert-ablation-500 bench-selectivity bench-feature-decomp bench-gate bench-easy-baseline bench-charge-matched bench-order-dependent bench-precision-at-k bench-per-family bench-simulation-ablation bench-simulation-ablation-within-amp simulation-gate regenerate-all generate phase3 pilot validate-scoring validate-scoring-phase3 validate-scoring-strict external-pilot pilot-confident presynth-qc gold-standard diversity synthesis-order novelty-broad external-consensus questionnaire gate-check ip-report benchmark-card wave0-5-gate-check wave0-5-novelty-audit wave0-5-novelty-audit-v2 wave0-5-panel wave0-5-evidence wave0-5-fill-external wave0-5b-generate wave0-5b-filter recalibration-engine recalibration-engine-dry-run validate-policy-version generate-synthetic-lab-results bump-policy-version generate-review-packet failed-candidate-report safe-publication-filter classify-negative-informativeness

PYTHON := $(shell [ -f .venv/bin/python ] && echo .venv/bin/python || echo python3)
PYTEST := $(shell [ -f .venv/bin/pytest ] && echo .venv/bin/pytest || echo pytest)
Expand Down Expand Up @@ -603,6 +603,12 @@ failed-candidate-report:
--out-md outputs/failed_candidate_report.md \
--validate-rejection-codes

classify-negative-informativeness:
PYTHONPATH=src $(PYTHON) scripts/classify_negative_result_informativeness.py \
--input examples/negative_result_entry_example.json \
--out-json outputs/negative_result_informativeness_result.json \
--out-md outputs/negative_result_informativeness_result.md

safe-publication-filter:
PYTHONPATH=src $(PYTHON) scripts/safe_publication_filter.py \
--input examples/safe_publication_filter_example_input.json \
Expand Down
1 change: 1 addition & 0 deletions docs/PROJECT_INDEX.md
Original file line number Diff line number Diff line change
Expand Up @@ -236,6 +236,7 @@ First useful contribution: make the repo easier to maintain without lowering saf
| [`docs/review/ASSAY_PREREGISTRATION.md`](review/ASSAY_PREREGISTRATION.md) | Candidate-selection pilot pre-registration template, not a protocol. |
| [`docs/review/COLLABORATION_PLAYBOOK.md`](review/COLLABORATION_PLAYBOOK.md) | External collaboration modes and boundaries. |
| [`docs/evidence/NEGATIVE_RESULT_ARCHIVE.md`](evidence/../evidence/NEGATIVE_RESULT_ARCHIVE.md) | Safe negative-result format. |
| [`docs/evidence/NEGATIVE_RESULT_INFORMATIVENESS_GUIDE.md`](evidence/../evidence/NEGATIVE_RESULT_INFORMATIVENESS_GUIDE.md) | 7-dimension framework for classifying negative-result entries as informative/neutral/non-informative. |

## Core safety docs

Expand Down
6 changes: 4 additions & 2 deletions docs/evidence/METRICS_CURRENT.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,9 @@ Machine-readable snapshot: `outputs/metrics_snapshot.json` regenerated with `mak
> **Purpose:** One authoritative table of current pipeline metrics. If any doc disagrees
> with this file, this file wins. Updated whenever benchmark/benchmark config changes.
>
> **Last updated:** 2026-07-09 (Phase F safe-publication filter — v0.5.76)
> **Last updated:** 2026-07-09 (Phase F negative-result informativeness — v0.5.77)
> **New in v0.5.77:** Negative-result informativeness guide (F6) — comprehensive documentation defining 7-dimension informativeness framework for negative-result entries, with 14 examples across all 6 reason categories, before/after transformation pairs, and quick-reference checklist. CLI classifier (`scripts/classify_negative_result_informativeness.py`) that scores entries on 7 dimensions and produces informative/neutral/non-informative classification. 37 tests. 2796 total.
>
> **New in v0.5.76:** Safe-publication filter (F5) — CLI that reads candidate panel data with safety metadata and checks each candidate against publication safety constraints before external release. FAIL-CLOSED: defaults to reject unless all checks pass. Checks: dry_lab_only (must be true), proof_ladder_level (≤4), toxicity_screened, hemolysis_screened, dual_use_reviewed (all must be true). Produces structured JSON filter result and human-readable Markdown summary. 33 tests. 2758 total.
>
> **New in v0.5.75:** Failed-candidate report generator (F4) — CLI that reads candidate rejection data + taxonomy, produces structured JSON report with summary aggregation (by category, severity, stage, evidence impact) and human-readable Markdown summary. 26 tests. 2724 total.
Expand Down Expand Up @@ -62,7 +64,7 @@ Machine-readable snapshot: `outputs/metrics_snapshot.json` regenerated with `mak
> **New in v0.5.31:** Added dipeptide-order features for sequence-order awareness. `dipeptide_order_score` achieves AUROC 0.7861 on AMP-vs-scrambled discrimination — the strongest order-dependent feature in the pipeline. Only 7/31 features survive scrambling (amphipathicity/helix-wheel + dipeptide). All composition features are purely position-independent (exactly 0.5000 AUROC on scrambled test).
> **New in v0.5.30:** Easy baseline benchmark added — charge density alone (AUROC 0.8166) outperforms the full pipeline ensemble (0.7792) on AMP-vs-Swiss-Prot-decoy discrimination. Honest finding documented: expected because pipeline optimizes for safety, not raw discrimination.
> **New in v0.5.29:** Expanded benchmark to 500 AMPs + 500 composition-matched decoys (n=1000). AUROC 0.7792 (CI₉₅: 0.7505–0.8065) confirms signal generalizes. Cluster-aware CI: 0.746–0.8102. Representative AUROC: 0.778. Standard benchmark (n=191) retained for backward comparison.
> **Pipeline version:** v0.5.76
> **Pipeline version:** v0.5.77
> **Branch:** main

---
Expand Down
Loading
Loading