Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
51 changes: 51 additions & 0 deletions docs/evidence/CANDIDATE_REJECTION_CERTIFICATE_GUIDE.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,51 @@
# Candidate Rejection Certificate Guide (CRC-)

## Purpose

A `CandidateRejectionCertificate` makes pipeline rejections first-class auditable artifacts.
When any gate rejects a candidate, a CRC- record documents: which gate, which reason, the evidence at time of rejection, and the proof-ladder level reached before rejection.

This closes the feedback loop: failures are as durable as passes.

## Required Fields

| Field | Type | Description |
|-------|------|-------------|
| `crc_id` | str | Unique ID starting with `CRC-` |
| `pipeline_version` | str | Pipeline version at rejection time |
| `candidate_id` | str | ID of the rejected candidate |
| `sequence` | str | Amino acid sequence |
| `rejection_date` | str | ISO 8601 date |
| `rejection_gate` | str | Gate that triggered rejection (controlled vocab) |
| `rejection_reason` | str | Reason code (controlled vocab) |
| `evidence_summary` | str | Human-readable evidence at rejection |
| `proof_ladder_level_at_rejection` | str | Highest level reached before rejection |
| `dry_lab_only` | bool | Must be True |

## Optional Fields

| Field | Default | Description |
|-------|---------|-------------|
| `scores` | {} | Score dict at rejection time |
| `notes` | "" | Reviewer notes (max 400 chars) |

## Validation Rules

1. `crc_id` must start with `CRC-`
2. `pipeline_version` non-empty
3. `candidate_id` non-empty
4. `sequence` non-empty, standard amino acids only
5. `rejection_date` ISO 8601 format
6. `rejection_gate` in `VALID_REJECTION_GATES`
7. `rejection_reason` in `VALID_REJECTION_REASONS`
8. `evidence_summary` non-empty
9. `proof_ladder_level_at_rejection` in `VALID_PROOF_LADDER_LEVELS`
10. `dry_lab_only` must be True
11. `proof_ladder_level_at_rejection` capped at `multi_signal_candidate_evidence` when `dry_lab_only=True`
12. `scores` must be a dict
13. `notes` ≤400 characters

## Claim Safety

CRC- records are dry-lab artifacts. `dry_lab_only=True` is enforced.
The proof ladder is capped at `multi_signal_candidate_evidence` (level 4).
9 changes: 9 additions & 0 deletions docs/evidence/METRICS_CURRENT.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,15 @@ Machine-readable snapshot: `outputs/metrics_snapshot.json` regenerated with `mak

## Changelog

### v0.10.31 — Phase B B7: Candidate rejection certificate (CRC-)
- Added `src/openamp_foundry/evidence/candidate_rejection_certificate.py`
- CRC- schema: 12 fields (crc_id, pipeline_version, candidate_id, sequence, rejection_date, rejection_gate, rejection_reason, evidence_summary, proof_ladder_level_at_rejection, dry_lab_only, scores, notes)
- 13 validation rules: CRC- prefix, sequence validation, ISO date, gate/reason controlled vocab (10 gates, 12 reasons), evidence_summary non-empty, dry_lab_only=True enforced, proof-ladder cap at multi_signal_candidate_evidence
- 2 warnings: short evidence_summary, empty notes
- 63 tests in `tests/evidence/test_candidate_rejection_certificate.py`
- BASELINE 6520→6583
- Closes Phase B B7 — pipeline rejections are now first-class auditable artifacts

### v0.10.30 — Phase B B6: Human-readable certificate report
- Added `src/openamp_foundry/evidence/certificate_report.py` — build_certificate_report() function
- Converts cert dict to formatted text with labelled sections: CANDIDATE, PROOF LADDER, SCORES (with not-biological-proof notice), CHEAP-EXPLANATION CHECK, SELECTION REASON, KNOWN FAILURE MODES, RECOMMENDED NEXT STEPS, REFERENCES CHECKED, optional QUALITY TIER
Expand Down
2 changes: 1 addition & 1 deletion docs/research/NEXT_100_PR_MAP.md
Original file line number Diff line number Diff line change
Expand Up @@ -47,7 +47,7 @@ Make candidate evidence packages impossible to confuse with biological proof.
| B4 | Add `release_status` field. | Supports staged release. | C/D |
| B5 | Add certificate quality-tier validator (complete). | assess_certificate_quality() computes draft/internal_review/external_review_ready tier; missing-field detection, forbidden-claim violation check, external-review gate; 63 tests in tests/test_certificate_quality_validator.py. | B |
| B6 | Add human-readable certificate report (complete). | build_certificate_report() in certificate_report.py converts cert dict to formatted text with sections: identity, proof ladder, scores, cheap-explanation check, selection reason, failure modes, next steps, references, optional quality tier; 63 tests in tests/test_certificate_report.py. | B |
| B7 | Add candidate rejection certificate support. | Failures become artifacts. | B |
| B7 | Add candidate rejection certificate support (complete). | CRC- schema: 12 fields, 13 rules, VALID_REJECTION_GATES (10 values), VALID_REJECTION_REASONS (12 values), dry_lab_only enforced, proof-ladder cap at multi_signal; makes pipeline rejections first-class auditable artifacts; 63 tests. | B |
| B8 | Link certificates to run manifest hashes. | Stronger reproducibility. | B |
| B9 | Add test that dry-lab certificates cannot include forbidden claims (complete). | CI gate: tests/test_certificate_claim_discipline.py scans all cert text fields against RISKY_PATTERNS and FORBIDDEN_CLAIM_PATTERNS; default cert proven clean; forbidden phrases confirmed detectable. 35 tests total. | B/C |
| B10 | Add external-review packet generator that bundles certificates. | Better partner review. | C/D |
Expand Down
194 changes: 194 additions & 0 deletions src/openamp_foundry/evidence/candidate_rejection_certificate.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,194 @@
"""CandidateRejectionCertificate — Phase B B7.

When a candidate fails any gate, this schema captures the rejection as a
first-class auditable artifact. Makes failures as durable as passes.

ID prefix: CRC-
"""

from __future__ import annotations

import re
from dataclasses import dataclass, field
from typing import List

CRC_PREFIX = "CRC-"

VALID_REJECTION_GATES = frozenset({
"wave0_5_safety_gate",
"novelty_gate",
"activity_threshold",
"diversity_filter",
"hemolysis_gate",
"toxicity_gate",
"synthesis_feasibility_gate",
"manual_review",
"claim_boundary_gate",
"calibration_gate",
})

VALID_REJECTION_REASONS = frozenset({
"safety_score_below_threshold",
"activity_score_below_threshold",
"insufficient_novelty",
"hemolysis_risk_too_high",
"toxicity_risk_too_high",
"synthesis_infeasible",
"duplicate_of_known",
"diversity_cluster_full",
"human_reviewer_rejected",
"claim_boundary_violation",
"calibration_refused",
"overclaiming_language_detected",
})

VALID_PROOF_LADDER_LEVELS = (
"valid_input",
"reproducible_dry_lab_features",
"baseline_triaged",
"leakage_aware_benchmark",
"multi_signal_candidate_evidence",
"expert_reviewed_assay_proposal",
"initial_qualified_assay_result",
"safety_adjusted_follow_up_signal",
"independent_replication",
"reusable_discovery_loop",
)

_ISO_DATE_RE = re.compile(r"^\d{4}-\d{2}-\d{2}(T[\d:.+Z-]+)?$")


@dataclass
class CandidateRejectionCertificate:
crc_id: str
pipeline_version: str
candidate_id: str
sequence: str
rejection_date: str
rejection_gate: str
rejection_reason: str
evidence_summary: str
proof_ladder_level_at_rejection: str
dry_lab_only: bool
scores: dict = field(default_factory=dict)
notes: str = ""

def validate(self) -> list[str]:
errors: list[str] = []
warnings: list[str] = []

# Rule 1: CRC- prefix
if not self.crc_id.startswith(CRC_PREFIX):
errors.append(f"crc_id must start with '{CRC_PREFIX}', got {self.crc_id!r}")

# Rule 2: pipeline_version non-empty
if not self.pipeline_version.strip():
errors.append("pipeline_version must not be empty")

# Rule 3: candidate_id non-empty
if not self.candidate_id.strip():
errors.append("candidate_id must not be empty")

# Rule 4: sequence non-empty and uppercase amino acids
if not self.sequence.strip():
errors.append("sequence must not be empty")
elif not re.match(r"^[ACDEFGHIKLMNPQRSTVWY]+$", self.sequence.upper()):
errors.append(
"sequence must contain only standard amino acid characters"
)

# Rule 5: rejection_date ISO format
if not _ISO_DATE_RE.match(self.rejection_date):
errors.append(
f"rejection_date must be ISO 8601 date, got {self.rejection_date!r}"
)

# Rule 6: rejection_gate in controlled vocab
if self.rejection_gate not in VALID_REJECTION_GATES:
errors.append(
f"rejection_gate must be one of {sorted(VALID_REJECTION_GATES)}, "
f"got {self.rejection_gate!r}"
)

# Rule 7: rejection_reason in controlled vocab
if self.rejection_reason not in VALID_REJECTION_REASONS:
errors.append(
f"rejection_reason must be one of {sorted(VALID_REJECTION_REASONS)}, "
f"got {self.rejection_reason!r}"
)

# Rule 8: evidence_summary non-empty
if not self.evidence_summary.strip():
errors.append("evidence_summary must not be empty")

# Rule 9: proof_ladder_level_at_rejection in controlled vocab
if self.proof_ladder_level_at_rejection not in VALID_PROOF_LADDER_LEVELS:
errors.append(
f"proof_ladder_level_at_rejection must be one of "
f"{VALID_PROOF_LADDER_LEVELS}, got {self.proof_ladder_level_at_rejection!r}"
)

# Rule 10: dry_lab_only must be True
if not self.dry_lab_only:
errors.append(
"dry_lab_only must be True — rejection certificates are dry-lab artifacts"
)

# Rule 11: proof_ladder_level_at_rejection <= multi_signal_candidate_evidence if dry_lab_only
dry_lab_cap_index = VALID_PROOF_LADDER_LEVELS.index("multi_signal_candidate_evidence")
if self.proof_ladder_level_at_rejection in VALID_PROOF_LADDER_LEVELS:
level_index = VALID_PROOF_LADDER_LEVELS.index(self.proof_ladder_level_at_rejection)
if self.dry_lab_only and level_index > dry_lab_cap_index:
errors.append(
f"dry_lab_only=True caps proof_ladder_level_at_rejection at "
f"'multi_signal_candidate_evidence' (level 4), "
f"got {self.proof_ladder_level_at_rejection!r} (level {level_index})"
)

# Rule 12: scores must be dict if provided
if not isinstance(self.scores, dict):
errors.append("scores must be a dict")

# Rule 13: notes length cap
if len(self.notes) > 400:
errors.append(f"notes must be ≤400 characters, got {len(self.notes)}")

# Warning: evidence_summary short
if len(self.evidence_summary.strip()) < 20:
warnings.append(
"evidence_summary is very short; provide enough context to "
"reconstruct the rejection decision"
)

# Warning: notes empty
if not self.notes.strip():
warnings.append(
"notes is empty; consider adding reviewer context or "
"follow-up instructions"
)

if errors:
return errors
return [f"WARNING: {w}" for w in warnings] if warnings else []


def validate_dict(data: dict) -> list[str]:
"""Validate a dict representation of CandidateRejectionCertificate."""
try:
record = CandidateRejectionCertificate(
crc_id=data.get("crc_id", ""),
pipeline_version=data.get("pipeline_version", ""),
candidate_id=data.get("candidate_id", ""),
sequence=data.get("sequence", ""),
rejection_date=data.get("rejection_date", ""),
rejection_gate=data.get("rejection_gate", ""),
rejection_reason=data.get("rejection_reason", ""),
evidence_summary=data.get("evidence_summary", ""),
proof_ladder_level_at_rejection=data.get("proof_ladder_level_at_rejection", ""),
dry_lab_only=data.get("dry_lab_only", False),
scores=data.get("scores", {}),
notes=data.get("notes", ""),
)
return record.validate()
except Exception as exc:
return [f"Construction error: {exc}"]
Loading
Loading