Loop 78: Bulk rejection-event validator

Makefile

@@ -614,3 +614,9 @@ safe-publication-filter:
 		--input examples/safe_publication_filter_example_input.json \
 		--out-json outputs/safe_publication_filter_result.json \
 		--out-md outputs/safe_publication_filter_result.md
+
+validate-rejection-events:
+	PYTHONPATH=src $(PYTHON) scripts/validate_rejection_events.py \
+		--input examples/rejection_events_example.json \
+		--out-json outputs/rejection_events_validation.json \
+		--out-md outputs/rejection_events_validation.md

docs/evidence/METRICS_CURRENT.md

@@ -5,7 +5,9 @@ Machine-readable snapshot: `outputs/metrics_snapshot.json` regenerated with `mak
 > **Purpose:** One authoritative table of current pipeline metrics. If any doc disagrees
 > with this file, this file wins. Updated whenever benchmark/benchmark config changes.
 >
-> **Last updated:** 2026-07-09 (Phase F F7 — calibration link from negative-result entries to intake reports — v0.5.78)
+> **Last updated:** 2026-07-09 (Phase F F8 — bulk rejection-event validator — v0.5.79)
+> **New in v0.5.79:** Bulk rejection-event validator (F8) — CLI (`scripts/validate_rejection_events.py`) that reads a JSON list of rejection events, validates each `rejection_code` against the rejection taxonomy (`examples/rejection_taxonomy_example.json`), checks required fields (candidate_id, rejection_code, date, pipeline_version), and outputs a PASS/FAIL validation report with per-event errors, rejection-code distribution, and Markdown summary. Example file (`examples/rejection_events_example.json`) with 6 toy events. `make validate-rejection-events` target. 26 tests. 2849 total.
+>
 > **New in v0.5.78:** Calibration link from negative-result entries to intake reports (F7) — closes learning loop by tracing each negative-result entry back to its prediction-vs-actual data in the calibration intake report. Schema addition: `intake_report_id` optional field on `negative_result_entry.schema.json`. CLI (`scripts/link_negative_result_to_intake.py`) reads negative-result archive JSON + calibration intake report, links by candidate_id, reports matched/unmatched entries, orphan intake candidates, lab summary for matched entries, and validates intake_report_id references. Produces structured JSON + Markdown link report. 25 tests. 2822 total.
 >
 > **New in v0.5.77:** Negative-result informativeness guide (F6) — comprehensive documentation defining 7-dimension informativeness framework for negative-result entries, with 14 examples across all 6 reason categories, before/after transformation pairs, and quick-reference checklist. CLI classifier (`scripts/classify_negative_result_informativeness.py`) that scores entries on 7 dimensions and produces informative/neutral/non-informative classification. 37 tests. 2796 total.
@@ -66,7 +68,7 @@ Machine-readable snapshot: `outputs/metrics_snapshot.json` regenerated with `mak
 > **New in v0.5.31:** Added dipeptide-order features for sequence-order awareness. `dipeptide_order_score` achieves AUROC 0.7861 on AMP-vs-scrambled discrimination — the strongest order-dependent feature in the pipeline. Only 7/31 features survive scrambling (amphipathicity/helix-wheel + dipeptide). All composition features are purely position-independent (exactly 0.5000 AUROC on scrambled test).
 > **New in v0.5.30:** Easy baseline benchmark added — charge density alone (AUROC 0.8166) outperforms the full pipeline ensemble (0.7792) on AMP-vs-Swiss-Prot-decoy discrimination. Honest finding documented: expected because pipeline optimizes for safety, not raw discrimination.
 > **New in v0.5.29:** Expanded benchmark to 500 AMPs + 500 composition-matched decoys (n=1000). AUROC 0.7792 (CI₉₅: 0.7505–0.8065) confirms signal generalizes. Cluster-aware CI: 0.746–0.8102. Representative AUROC: 0.778. Standard benchmark (n=191) retained for backward comparison.
-> **Pipeline version:** v0.5.78
+> **Pipeline version:** v0.5.79
 > **Branch:** main
 
 ---

docs/research/ROADMAP.md

@@ -1,5 +1,43 @@
 # Roadmap
 
+## v0.5.79 — Loop 78: Phase F F8 — Bulk Rejection-Event Validator ✓ (2026-07-09)
+
+CLI that validates rejection events against the rejection taxonomy. Checks
+required fields, known rejection codes, and date format. Produces PASS/FAIL
+report with per-event errors.
+
+Changes:
+- `scripts/validate_rejection_events.py` (F8) — Standalone CLI that reads a JSON
+  list of rejection events, validates each rejection_code against the rejection
+  taxonomy, checks required fields (candidate_id, rejection_code, date,
+  pipeline_version), and outputs a structured validation report with per-event
+  errors. Supports `--out-json` and `--out-md` output. Exit 0 on all valid,
+  3 on any invalid, 2 on input errors.
+- `examples/rejection_events_example.json` (F8) — Toy example with 6 rejection
+  events spanning activity, safety, diversity, reviewer, and lab codes.
+  Clearly marked EXAMPLE.
+- `Makefile` — Added `validate-rejection-events` target.
+- `tests/evidence/test_validate_rejection_events.py` — 26 tests covering:
+  taxonomy code loading, valid events (single + multiple), missing required
+  fields (each of 4 individually), unknown rejection_code, invalid date format,
+  empty string fields, empty list, multi-error reporting, build_report structure,
+  rejection_code_summary counts, markdown generation (title, summary, errors,
+  caveat), and CLI exit codes (0, 3, 2) and output writing (JSON + Markdown).
+- `docs/evidence/METRICS_CURRENT.md` — v0.5.79 changelog. Pipeline version bumped.
+  Test count: 2849.
+- `tests/test_test_count_regression.py` — baseline updated to 2849.
+
+Honest boundaries:
+- Validates structural correctness of rejection events, not biological truth.
+- A PASS means the event has valid fields and a known rejection code; it does
+  not mean the rejection decision was correct.
+- Unknown rejection codes may indicate a stale taxonomy rather than an invalid
+  event — the report flags them as errors but the taxonomy may need updating.
+- The rejection_code distribution is a descriptive summary, not evidence of
+  bias or correctness in rejection decisions.
+- This validator does not enforce that downstream tools actually use the
+  rejection taxonomy — adoption requires tool-chain updates.
+
 ## v0.5.78 — Loop 77: Phase F F7 — Calibration Link from Negative-Result Entries to Intake Reports ✓ (2026-07-09)
 
 Closes the learning loop by tracing each negative-result entry back to its

examples/rejection_events_example.json

@@ -0,0 +1,44 @@
+[
+  {
+    "candidate_id": "WAVE0.5-001",
+    "rejection_code": "PIPE_ACTIVITY_LOW",
+    "date": "2026-07-09",
+    "pipeline_version": "v0.5.78",
+    "notes": "Activity score 0.42 below minimum threshold 0.50."
+  },
+  {
+    "candidate_id": "WAVE0.5-002",
+    "rejection_code": "PIPE_SAFETY_LOW",
+    "date": "2026-07-08",
+    "pipeline_version": "v0.5.77",
+    "notes": "Safety score 0.31 below threshold 0.50."
+  },
+  {
+    "candidate_id": "WAVE0.5-010",
+    "rejection_code": "DIV_TOO_SIMILAR",
+    "date": "2026-07-07",
+    "pipeline_version": "v0.5.76",
+    "notes": "Similarity 0.82 to already-selected WAVE0.5-003."
+  },
+  {
+    "candidate_id": "WAVE0.5-021",
+    "rejection_code": "REV_EXPERT_REJECT",
+    "date": "2026-07-06",
+    "pipeline_version": "v0.5.75",
+    "notes": "Expert reviewer recommended against proceeding."
+  },
+  {
+    "candidate_id": "WAVE0.5-030",
+    "rejection_code": "LAB_INACTIVE",
+    "date": "2026-07-05",
+    "pipeline_version": "v0.5.74",
+    "notes": "MIC > 128 ug/mL across all test strains."
+  },
+  {
+    "candidate_id": "WAVE0.5-045",
+    "rejection_code": "PIPE_ACTIVITY_LOW",
+    "date": "2026-07-04",
+    "pipeline_version": "v0.5.73",
+    "notes": "Activity score 0.38."
+  }
+]

scripts/validate_rejection_events.py

@@ -0,0 +1,205 @@
+"""Validate a batch of rejection events against the rejection taxonomy.
+
+Reads a JSON list of rejection events, validates each rejection_code against
+schemas/rejection_taxonomy.schema.json, checks required fields, and outputs a
+PASS/FAIL validation report with per-event errors.
+
+Usage:
+    python scripts/validate_rejection_events.py \\
+        --input rejection_events.json \\
+        --out-json outputs/validation_report.json
+"""
+
+from __future__ import annotations
+
+import json
+import sys
+from datetime import datetime
+from pathlib import Path
+
+from openamp_foundry.evidence.schemas import validate_json_schema
+from openamp_foundry.utils.io import read_json
+
+REQUIRED_FIELDS = {"candidate_id", "rejection_code", "date", "pipeline_version"}
+DATE_FORMAT = "%Y-%m-%d"
+_ROOT = Path(__file__).resolve().parent.parent
+TAXONOMY_DEFAULT_PATH = _ROOT / "examples" / "rejection_taxonomy_example.json"
+
+
+def load_taxonomy_codes(taxonomy_path: str | Path) -> set[str]:
+    taxonomy = read_json(taxonomy_path)
+    return {entry["code"] for entry in taxonomy}
+
+
+def validate_events(
+    events: list[dict],
+    taxonomy_codes: set[str],
+) -> dict:
+    total = len(events)
+    errors: list[dict] = []
+    valid_count = 0
+
+    for i, event in enumerate(events):
+        row_errors: list[str] = []
+
+        for field in REQUIRED_FIELDS:
+            val = event.get(field)
+            if not val or (isinstance(val, str) and not val.strip()):
+                row_errors.append(f"Missing required field: {field}")
+
+        rejection_code = event.get("rejection_code", "")
+        if rejection_code and rejection_code not in taxonomy_codes:
+            row_errors.append(
+                f"Unknown rejection_code: {rejection_code}"
+            )
+
+        date_val = event.get("date", "")
+        if date_val:
+            try:
+                datetime.strptime(str(date_val), DATE_FORMAT)
+            except ValueError:
+                row_errors.append(
+                    f"Invalid date format: {date_val} (expected YYYY-MM-DD)"
+                )
+
+        if row_errors:
+            errors.append({
+                "index": i,
+                "candidate_id": event.get("candidate_id", "?"),
+                "errors": row_errors,
+            })
+        else:
+            valid_count += 1
+
+    return {
+        "input": {"total_events": total},
+        "summary": {
+            "valid": valid_count,
+            "invalid": len(errors),
+            "all_valid": len(errors) == 0,
+        },
+        "errors": errors,
+    }
+
+
+def build_report(events: list[dict], taxonomy_codes: set[str]) -> dict:
+    result = validate_events(events, taxonomy_codes)
+
+    rejection_code_counts: dict[str, int] = {}
+    for event in events:
+        code = event.get("rejection_code", "unknown")
+        rejection_code_counts[code] = rejection_code_counts.get(code, 0) + 1
+
+    result["rejection_code_summary"] = dict(
+        sorted(rejection_code_counts.items())
+    )
+
+    result["_caveat"] = (
+        "This validation report is informational only. "
+        "A PASS result confirms structural validity of rejection events, "
+        "not biological correctness of rejection decisions. "
+        "Unknown rejection codes may indicate a stale taxonomy "
+        "rather than an invalid event."
+    )
+    return result
+
+
+def generate_markdown(result: dict) -> str:
+    lines: list[str] = []
+    lines.append("# Rejection Event Validation Report\n")
+    lines.append(f"**Total events:** {result['input']['total_events']}\n")
+    lines.append(
+        f"**Valid:** {result['summary']['valid']} | "
+        f"**Invalid:** {result['summary']['invalid']} | "
+        f"**All valid:** {result['summary']['all_valid']}\n"
+    )
+
+    if result.get("rejection_code_summary"):
+        lines.append("\n## Rejection Code Distribution\n")
+        lines.append("| Code | Count |")
+        lines.append("|------|-------|")
+        for code, count in result["rejection_code_summary"].items():
+            lines.append(f"| {code} | {count} |")
+
+    errors = result.get("errors", [])
+    if errors:
+        lines.append(f"\n## Errors ({len(errors)})\n")
+        for err in errors:
+            lines.append(
+                f"- **Event {err['index']}** "
+                f"(candidate: {err['candidate_id']}): "
+                f"{'; '.join(err['errors'])}"
+            )
+
+    lines.append(f"\n## Caveat\n\n{result.get('_caveat', '')}\n")
+    return "\n".join(lines)
+
+
+def main() -> int:
+    import argparse
+
+    parser = argparse.ArgumentParser(
+        description="Validate rejection events against the rejection taxonomy."
+    )
+    parser.add_argument("--input", required=True, help="Path to JSON events file")
+    parser.add_argument(
+        "--out-json", default=None, help="Output JSON report path"
+    )
+    parser.add_argument(
+        "--out-md", default=None, help="Output Markdown report path"
+    )
+    parser.add_argument(
+        "--taxonomy",
+        default=str(TAXONOMY_DEFAULT_PATH),
+        help="Path to rejection taxonomy JSON file (e.g. examples/rejection_taxonomy_example.json)",
+    )
+    args = parser.parse_args()
+
+    input_path = Path(args.input)
+    if not input_path.exists():
+        print(f"Error: Input not found: {args.input}", file=sys.stderr)
+        return 2
+
+    try:
+        events = read_json(input_path)
+    except Exception as e:
+        print(f"Error reading input: {e}", file=sys.stderr)
+        return 2
+
+    if not isinstance(events, list):
+        print("Error: Input must be a JSON array of rejection events", file=sys.stderr)
+        return 2
+
+    if not events:
+        print("Error: Input is an empty list (no events to validate)", file=sys.stderr)
+        return 2
+
+    taxonomy_path = Path(args.taxonomy)
+    if not taxonomy_path.exists():
+        print(f"Error: Taxonomy not found: {args.taxonomy}", file=sys.stderr)
+        return 2
+
+    try:
+        taxonomy_codes = load_taxonomy_codes(taxonomy_path)
+    except Exception as e:
+        print(f"Error loading taxonomy: {e}", file=sys.stderr)
+        return 2
+
+    result = build_report(events, taxonomy_codes)
+
+    print(json.dumps(result, indent=2))
+
+    if args.out_json:
+        Path(args.out_json).write_text(json.dumps(result, indent=2) + "\n")
+        print(f"\nJSON report written to: {args.out_json}")
+
+    if args.out_md:
+        md = generate_markdown(result)
+        Path(args.out_md).write_text(md + "\n")
+        print(f"Markdown report written to: {args.out_md}")
+
+    return 0 if result["summary"]["all_valid"] else 3
+
+
+if __name__ == "__main__":
+    sys.exit(main())