Loop 74: Phase F F4 — failed-candidate report generator

Makefile

@@ -1,4 +1,4 @@
-.PHONY: help demo test lint ci clean bench-leakage bench-multi-negatives bench-baseline bench-hidden-active bench-cluster-split bench-expert-ablation bench-expert-ablation-500 bench-selectivity bench-feature-decomp bench-gate bench-easy-baseline bench-charge-matched bench-order-dependent bench-precision-at-k bench-per-family bench-simulation-ablation bench-simulation-ablation-within-amp simulation-gate regenerate-all generate phase3 pilot validate-scoring validate-scoring-phase3 validate-scoring-strict external-pilot pilot-confident presynth-qc gold-standard diversity synthesis-order novelty-broad external-consensus questionnaire gate-check ip-report benchmark-card wave0-5-gate-check wave0-5-novelty-audit wave0-5-novelty-audit-v2 wave0-5-panel wave0-5-evidence wave0-5-fill-external wave0-5b-generate wave0-5b-filter recalibration-engine recalibration-engine-dry-run validate-policy-version generate-synthetic-lab-results bump-policy-version generate-review-packet
+.PHONY: help demo test lint ci clean bench-leakage bench-multi-negatives bench-baseline bench-hidden-active bench-cluster-split bench-expert-ablation bench-expert-ablation-500 bench-selectivity bench-feature-decomp bench-gate bench-easy-baseline bench-charge-matched bench-order-dependent bench-precision-at-k bench-per-family bench-simulation-ablation bench-simulation-ablation-within-amp simulation-gate regenerate-all generate phase3 pilot validate-scoring validate-scoring-phase3 validate-scoring-strict external-pilot pilot-confident presynth-qc gold-standard diversity synthesis-order novelty-broad external-consensus questionnaire gate-check ip-report benchmark-card wave0-5-gate-check wave0-5-novelty-audit wave0-5-novelty-audit-v2 wave0-5-panel wave0-5-evidence wave0-5-fill-external wave0-5b-generate wave0-5b-filter recalibration-engine recalibration-engine-dry-run validate-policy-version generate-synthetic-lab-results bump-policy-version generate-review-packet failed-candidate-report
 
 PYTHON := $(shell [ -f .venv/bin/python ] && echo .venv/bin/python || echo python3)
 PYTEST  := $(shell [ -f .venv/bin/pytest ] && echo .venv/bin/pytest || echo pytest)
@@ -595,3 +595,10 @@ generate-review-packet:
 		--proof-ladder-level 2 \
 		--out outputs/review_packet_skeleton.json \
 		--validate
+
+failed-candidate-report:
+	PYTHONPATH=src $(PYTHON) scripts/generate_failed_candidate_report.py \
+		--input examples/failed_candidates_example.json \
+		--out-json outputs/failed_candidate_report.json \
+		--out-md outputs/failed_candidate_report.md \
+		--validate-rejection-codes

docs/evidence/FAILED_CANDIDATE_REPORT.md

@@ -0,0 +1,106 @@
+# Failed Candidate Report
+
+## Purpose
+
+Generate structured, reviewable reports from batches of rejected candidates.
+Each report uses the rejection taxonomy codes (`schemas/rejection_taxonomy.schema.json`)
+to provide consistent categorization, severity assessment, and evidence-impact
+tracking across all pipeline failures.
+
+## Source
+
+- **Generator:** `scripts/generate_failed_candidate_report.py`
+- **Input format:** JSON file with `candidates` array, each containing rejection codes from the taxonomy
+- **Output formats:** JSON (machine-readable) + Markdown (human-readable)
+
+## Usage
+
+Generate a report from an example input:
+
+```bash
+python scripts/generate_failed_candidate_report.py \
+    --input examples/failed_candidates_example.json \
+    --out-json outputs/failed_candidate_report.json \
+    --out-md outputs/failed_candidate_report.md \
+    --validate-rejection-codes
+```
+
+Or using the Makefile target:
+
+```bash
+make failed-candidate-report
+```
+
+## Input Format
+
+The input file is a JSON object with:
+
+| Field | Required | Description |
+|-------|----------|-------------|
+| `source_batch` | Yes | Batch identifier (e.g. `wave0.5`) |
+| `pipeline_version` | Yes | Pipeline version string |
+| `date` | Yes | Report date (ISO 8601) |
+| `candidates` | Yes | Array of candidate objects |
+
+Each candidate object:
+
+| Field | Required | Description |
+|-------|----------|-------------|
+| `candidate_id` | Yes | Unique identifier |
+| `sequence` | No | Amino-acid sequence (recommended) |
+| `rejections` | Yes | Array of rejection objects |
+
+Each rejection object:
+
+| Field | Required | Description |
+|-------|----------|-------------|
+| `code` | Yes | Rejection taxonomy code (e.g. `PIPE_ACTIVITY_LOW`) |
+| `detail` | No | Human-readable explanation with relevant scores/thresholds |
+
+## Output Formats
+
+### JSON report
+
+The JSON report includes:
+
+- **report_metadata**: generation timestamp, source batch, pipeline version, taxonomy source
+- **summary**: totals, unique codes, by-category aggregation, by-severity aggregation,
+  by-stage aggregation, rejection-code frequency table, unknown codes if validation was skipped
+- **candidates**: per-candidate breakdown with rejection details and taxonomy metadata
+- **caveats**: standard disclaimer array
+- **dry_lab_only**: always `true`
+
+### Markdown summary
+
+The Markdown summary includes:
+
+1. Report metadata table
+2. Summary metrics table
+3. By-category table
+4. By-severity table
+5. By-evidence-impact table
+6. Rejection-code frequency table
+7. Per-candidate breakdown with detailed tables
+8. Standard caveats
+
+## Validation
+
+The `--validate-rejection-codes` flag checks every rejection code in the input
+against the taxonomy. Unknown codes produce a non-zero exit code and error
+messages on stderr. This is recommended for automated pipelines.
+
+## Caveats
+
+- Rejection codes reflect pipeline or reviewer decisions, not biological inactivity unless confirmed by lab assay.
+- Soft rejections may be overcome by threshold or policy changes.
+- Hard rejections require new evidence to overturn.
+- This report is informational only and requires qualified review.
+- Computational outputs are hypotheses and review aids — they are not biological proof.
+
+## Cross-references
+
+- `schemas/rejection_taxonomy.schema.json` — Taxonomy defining all valid rejection codes
+- `examples/rejection_taxonomy_example.json` — Complete taxonomy with 21 entries
+- `docs/evidence/REJECTION_TAXONOMY.md` — Taxonomy reference documentation
+- `schemas/negative_result_entry.schema.json` — Negative-result archive entry schema
+- `docs/evidence/NEGATIVE_RESULT_ARCHIVE.md` — Archive format for persistent records

docs/evidence/METRICS_CURRENT.md

@@ -5,7 +5,9 @@ Machine-readable snapshot: `outputs/metrics_snapshot.json` regenerated with `mak
 > **Purpose:** One authoritative table of current pipeline metrics. If any doc disagrees
 > with this file, this file wins. Updated whenever benchmark/benchmark config changes.
 >
-> **Last updated:** 2026-07-09 (Phase F rejection taxonomy — v0.5.74)
+> **Last updated:** 2026-07-09 (Phase F report generator — v0.5.75)
+> **New in v0.5.75:** Failed-candidate report generator (F4) — CLI that reads candidate rejection data + taxonomy, produces structured JSON report with summary aggregation (by category, severity, stage, evidence impact) and human-readable Markdown summary. 26 tests. 2724 total.
+>
 > **New in v0.5.74:** Rejection reason taxonomy schema (F3) — machine-readable taxonomy with severity, evidence_impact, and applies_at_stage fields. Comprehensive REJECTION_TAXONOMY.md doc. 19 tests. 2698 total.
 >
 > **New in v0.5.73:** Safety-release decision schema (E4), pilot pre-registration schema (E5), review packet generator CLI (E6). 40 tests. 2679 total.
@@ -58,7 +60,7 @@ Machine-readable snapshot: `outputs/metrics_snapshot.json` regenerated with `mak
 > **New in v0.5.31:** Added dipeptide-order features for sequence-order awareness. `dipeptide_order_score` achieves AUROC 0.7861 on AMP-vs-scrambled discrimination — the strongest order-dependent feature in the pipeline. Only 7/31 features survive scrambling (amphipathicity/helix-wheel + dipeptide). All composition features are purely position-independent (exactly 0.5000 AUROC on scrambled test).
 > **New in v0.5.30:** Easy baseline benchmark added — charge density alone (AUROC 0.8166) outperforms the full pipeline ensemble (0.7792) on AMP-vs-Swiss-Prot-decoy discrimination. Honest finding documented: expected because pipeline optimizes for safety, not raw discrimination.
 > **New in v0.5.29:** Expanded benchmark to 500 AMPs + 500 composition-matched decoys (n=1000). AUROC 0.7792 (CI₉₅: 0.7505–0.8065) confirms signal generalizes. Cluster-aware CI: 0.746–0.8102. Representative AUROC: 0.778. Standard benchmark (n=191) retained for backward comparison.
-> **Pipeline version:** v0.5.74
+> **Pipeline version:** v0.5.75
 > **Branch:** main
 
 ---
@@ -1128,7 +1130,7 @@ Decoys score low on activity. Selective AMPs score moderately on both.
 
 | Metric | Value |
 |--------|-------|
-| Total tests | 2021 |
+| Total tests | 2724 |
 | Coverage (branch) | 99% (6 CLI guard lines only) |
 | Source modules at 100% | All pipeline, QC, scoring modules |
 

docs/research/ROADMAP.md

@@ -1,5 +1,41 @@
 # Roadmap
 
+## v0.5.75 — Loop 74: Phase F F4 — Failed-Candidate Report Generator ✓ (2026-07-09)
+
+Failed-candidate report generator: CLI that reads candidate rejection data validated
+against the rejection taxonomy and produces structured JSON + Markdown reports.
+Foundation for F5 (safe-publication filter).
+
+Changes:
+- `scripts/generate_failed_candidate_report.py` (F4) — Standalone CLI that loads a batch
+  of failed candidates with rejection codes, validates them against the taxonomy
+  (optional `--validate-rejection-codes`), and builds a structured report with:
+  per-candidate breakdown, summary statistics, by-category/severity/stage/evidence-impact
+  aggregation, and rejection-code frequency table.
+- `examples/failed_candidates_example.json` — Toy input file with 5 candidates spanning
+  6 rejection codes across pipeline, pre_selection, and diversity categories. Clearly
+  marked EXAMPLE.
+- `docs/evidence/FAILED_CANDIDATE_REPORT.md` — Reference doc with usage, input/output
+  format tables, validation instructions, caveats, and cross-references.
+- `Makefile` — Added `make failed-candidate-report` target.
+- `tests/evidence/test_failed_candidate_report.py` — 26 tests covering: taxonomy loading,
+  rejection code validation (valid + unknown), report build with summary aggregation,
+  category/severity/rejection-code frequency aggregation, metadata and caveat generation,
+  candidate entry enrichment with taxonomy metadata, markdown section content, input
+  loading (valid + missing file), CLI output (JSON + Markdown), validation success and
+  failure paths, missing input and empty candidates error paths, empty taxonomy handling,
+  and example input validation round-trip.
+
+Honest boundaries:
+- The report is a computational summary of pipeline rejection decisions; it is not
+  biological proof of inactivity unless confirmed by lab assay.
+- Soft rejections may be overcome by threshold or policy changes; the report does not
+  determine whether such changes are justified.
+- Code validation depends on taxonomy being loaded; without the taxonomy, unknown codes
+  are silently included in the report (flagged only in the `unknown_codes` field).
+- The report does not enforce that downstream tools or human reviewers actually use
+  the rejection taxonomy codes — adoption requires tool-chain updates.
+
 ## v0.5.74 — Loop 73: Phase F F3 — Rejection Reason Taxonomy Schema ✓ (2026-07-09)
 
 Negative-result infrastructure begins: machine-readable rejection reason taxonomy

examples/failed_candidates_example.json

@@ -0,0 +1,70 @@
+{
+  "source_batch": "wave0.5",
+  "pipeline_version": "v0.5.74",
+  "date": "2026-07-09",
+  "description": "EXAMPLE failed candidates from Wave 0.5 selection pipeline. NOT real rejected candidates — illustrative only.",
+  "candidates": [
+    {
+      "candidate_id": "WAVE0.5_EXAMPLE-001",
+      "sequence": "ACDEFGHIKLMNPQRSTVWY",
+      "rejections": [
+        {
+          "code": "PIPE_ACTIVITY_LOW",
+          "detail": "Activity score 0.42 below minimum threshold 0.50."
+        },
+        {
+          "code": "PIPE_SAFETY_LOW",
+          "detail": "Safety score 0.31 below minimum threshold 0.50 (driven by high hemolysis proxy 0.89)."
+        }
+      ]
+    },
+    {
+      "candidate_id": "WAVE0.5_EXAMPLE-002",
+      "sequence": "KLAKLAKKLAKLAK",
+      "rejections": [
+        {
+          "code": "PIPE_ENSEMBLE_LOW",
+          "detail": "Ensemble score 0.55 below synthesis gate threshold 0.65."
+        },
+        {
+          "code": "PIPE_NOVELTY_LOW",
+          "detail": "Novelty score 0.35 below minimum threshold 0.40 (near-duplicate of known AMP)."
+        }
+      ]
+    },
+    {
+      "candidate_id": "WAVE0.5_EXAMPLE-003",
+      "sequence": "RRWQWRMKKLG",
+      "rejections": [
+        {
+          "code": "DIV_TOO_SIMILAR",
+          "detail": "Sequence similarity 0.82 to already-selected candidate WAVE0.5-003 exceeds diversity threshold 0.70."
+        },
+        {
+          "code": "DIV_FAMILY_CAP",
+          "detail": "Family SEED-003 has reached its cap of 2 panel slots."
+        }
+      ]
+    },
+    {
+      "candidate_id": "WAVE0.5_EXAMPLE-004",
+      "sequence": "AAAABBBBCCCC",
+      "rejections": [
+        {
+          "code": "PRE_SEQ_INVALID",
+          "detail": "Sequence contains non-canonical amino-acid symbols 'B' at positions 5-8."
+        }
+      ]
+    },
+    {
+      "candidate_id": "WAVE0.5_EXAMPLE-005",
+      "sequence": "GIGKFLHSAKKWGKAFVGEIMNS",
+      "rejections": [
+        {
+          "code": "DIV_BUDGET",
+          "detail": "Panel budget limited to 24 candidates; lowest-ranked qualifying candidate excluded."
+        }
+      ]
+    }
+  ]
+}