Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 9 additions & 0 deletions docs/evidence/METRICS_CURRENT.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,15 @@ Machine-readable snapshot: `outputs/metrics_snapshot.json` regenerated with `mak

## Changelog

### v0.10.32 — Phase B B8: Certificates linked to run manifest hashes
- Added `run_id` and `run_manifest_hash` optional parameters to `build_certificate()` in `src/openamp_foundry/evidence/certificate.py`
- Every certificate now carries the pipeline run_id and the SHA256 hash of the run manifest that produced it
- Backward-compatible: both fields default to empty string when not provided
- Makes every certificate traceable to the exact pipeline run for reproducibility auditing
- 63 tests in `tests/test_certificate_run_link.py`
- BASELINE 6583→6646
- Closes Phase B B8 — certificate-to-run-manifest traceability is now machine-verifiable

### v0.10.31 — Phase B B7: Candidate rejection certificate (CRC-)
- Added `src/openamp_foundry/evidence/candidate_rejection_certificate.py`
- CRC- schema: 12 fields (crc_id, pipeline_version, candidate_id, sequence, rejection_date, rejection_gate, rejection_reason, evidence_summary, proof_ladder_level_at_rejection, dry_lab_only, scores, notes)
Expand Down
2 changes: 1 addition & 1 deletion docs/research/NEXT_100_PR_MAP.md
Original file line number Diff line number Diff line change
Expand Up @@ -48,7 +48,7 @@ Make candidate evidence packages impossible to confuse with biological proof.
| B5 | Add certificate quality-tier validator (complete). | assess_certificate_quality() computes draft/internal_review/external_review_ready tier; missing-field detection, forbidden-claim violation check, external-review gate; 63 tests in tests/test_certificate_quality_validator.py. | B |
| B6 | Add human-readable certificate report (complete). | build_certificate_report() in certificate_report.py converts cert dict to formatted text with sections: identity, proof ladder, scores, cheap-explanation check, selection reason, failure modes, next steps, references, optional quality tier; 63 tests in tests/test_certificate_report.py. | B |
| B7 | Add candidate rejection certificate support (complete). | CRC- schema: 12 fields, 13 rules, VALID_REJECTION_GATES (10 values), VALID_REJECTION_REASONS (12 values), dry_lab_only enforced, proof-ladder cap at multi_signal; makes pipeline rejections first-class auditable artifacts; 63 tests. | B |
| B8 | Link certificates to run manifest hashes. | Stronger reproducibility. | B |
| B8 | Link certificates to run manifest hashes (complete). | Added run_id and run_manifest_hash optional params to build_certificate(); every certificate is now traceable to the exact pipeline run and manifest that produced it; backward-compatible (defaults to empty string); 63 tests in tests/test_certificate_run_link.py. | B |
| B9 | Add test that dry-lab certificates cannot include forbidden claims (complete). | CI gate: tests/test_certificate_claim_discipline.py scans all cert text fields against RISKY_PATTERNS and FORBIDDEN_CLAIM_PATTERNS; default cert proven clean; forbidden phrases confirmed detectable. 35 tests total. | B/C |
| B10 | Add external-review packet generator that bundles certificates. | Better partner review. | C/D |

Expand Down
30 changes: 27 additions & 3 deletions src/openamp_foundry/evidence/certificate.py
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,12 @@


def _infer_proof_ladder_level(scored: ScoredCandidate) -> str:
"""Infer highest proof-ladder level supported by this candidate's evidence.

Pipeline outputs are multi_signal_candidate_evidence (Level 4) by default:
they have feature extraction, multiple independent scorers, novelty checking,
and diversity selection. Lower levels apply only if key signals are missing.
"""
scores = scored.scores
has_multi_signal = (
scores.get("activity", 0) > 0
Expand All @@ -29,6 +35,20 @@ def _infer_proof_ladder_level(scored: ScoredCandidate) -> str:


def _build_baseline_caveat(scored: ScoredCandidate) -> str:
"""Build a cheap-explanation statement for this candidate.

Forces cheap-explanation visibility: explicitly states which baseline
heuristics this candidate passes, so reviewers can judge whether the
residual ML signal justifies selection beyond a trivial rule.

Three canonical cheap baselines for AMP selection:
1. Net charge >= 4 (cationic bias)
2. Length in [10, 40] aa (typical AMP range)
3. Hydrophobic fraction >= 0.30

If all three pass, a simple conjunction rule could select this candidate
without any ML scoring. The ML residual must exceed this threshold.
"""
features = scored.features
charge = float(features.get("net_charge_proxy", 0))
length = int(features.get("length", 0))
Expand All @@ -47,7 +67,7 @@ def _build_baseline_caveat(scored: ScoredCandidate) -> str:
if flags_yes == 3:
residual_note = (
"All three baseline flags pass — a simple conjunction rule "
"(charge\u22654 AND length 10-40 AND hydrophobicity\u22650.30) would select "
"(charge>=4 AND length 10-40 AND hydrophobicity>=0.30) would select "
"this candidate without ML scoring. Residual ML signal must exceed "
"this baseline to justify pipeline use."
)
Expand All @@ -69,9 +89,9 @@ def _build_baseline_caveat(scored: ScoredCandidate) -> str:

return (
f"Cheapest-explanation check: "
f"charge={charge:.1f} (\u2265{_BASELINE_MIN_CHARGE} passes: {charge_flag}), "
f"charge={charge:.1f} (≥{_BASELINE_MIN_CHARGE} passes: {charge_flag}), "
f"length={length} ({_BASELINE_LENGTH_MIN}-{_BASELINE_LENGTH_MAX} aa range: {length_flag}), "
f"hydrophobic_fraction={hydro:.2f} (\u2265{_BASELINE_MIN_HYDROPHOBIC_FRACTION:.2f} passes: {hydro_flag}). "
f"hydrophobic_fraction={hydro:.2f} (≥{_BASELINE_MIN_HYDROPHOBIC_FRACTION:.2f} passes: {hydro_flag}). "
f"{residual_note}"
)

Expand All @@ -80,6 +100,8 @@ def build_certificate(
scored: ScoredCandidate,
config: dict[str, Any],
references_checked: list[str],
run_id: str = "",
run_manifest_hash: str = "",
) -> dict[str, Any]:
return {
"candidate_id": scored.candidate.candidate_id,
Expand All @@ -101,4 +123,6 @@ def build_certificate(
"config_hash": stable_json_hash(config),
"proof_ladder_level": _infer_proof_ladder_level(scored),
"baseline_caveat": _build_baseline_caveat(scored),
"run_id": run_id,
"run_manifest_hash": run_manifest_hash,
}
Loading
Loading