Repository navigation
Loop 74: Phase F F4 — failed-candidate report generator - #759
Merged
Merged
Conversation
CLI that reads candidate rejection data validated against the rejection taxonomy (F3) and produces structured JSON + Markdown reports. Changes: - scripts/generate_failed_candidate_report.py — standalone CLI with --input, --out-json, --out-md, --validate-rejection-codes, taxonomy loading, report building (summary aggregation, per-candidate breakdown, category/severity/stage/evidence-impact stats) - examples/failed_candidates_example.json — 5-candidate toy example - docs/evidence/FAILED_CANDIDATE_REPORT.md — usage reference - tests/evidence/test_failed_candidate_report.py — 25 tests (import, validation, report build, CLI, markdown, example round-trip) - Makefile — failed-candidate-report target - docs/evidence/METRICS_CURRENT.md — v0.5.75 changelog, test count 2724 - docs/research/ROADMAP.md — Loop 74 entry - tests/test_test_count_regression.py — baseline 2724 Honest boundaries: - Report is computational summary, not biological proof - Soft rejections may be overcome by policy changes - Taxonomy validation is optional (--validate-rejection-codes) - Report does not enforce downstream adoption of taxonomy codes
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Change
CLI script that reads batches of failed candidates (with rejection taxonomy codes), validates codes against the F3 taxonomy, and generates structured JSON + Markdown reports.
Why
F4 directly builds on F3 (rejection taxonomy). Without a report generator, the taxonomy codes exist but cannot produce reviewable batch summaries. This PR closes the gap between rejection taxonomy (machine-readable codes) and actionable reports (per-batch aggregation, categorization, frequency analysis).
Evidence
make failed-candidate-reportgenerates valid JSON + Markdown from the example inputpython3 -m pytest tests/evidence/test_failed_candidate_report.py -v— 25/25 passedSafety impact
None — this is a report generation tool. The output explicitly includes
dry_lab_only: trueand 5 caveat statements. No claims are strengthened.Proof level
N/A (infrastructure only). The report generator produces computational summaries that do not make biological claims.
Limitations
Human review needed
No