Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 8 additions & 0 deletions docs/evidence/METRICS_CURRENT.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,14 @@ Machine-readable snapshot: `outputs/metrics_snapshot.json` regenerated with `mak

## Changelog

### v0.10.28 — Phase B B3: baseline_caveat field in certificates
- Added `baseline_caveat` field to `build_certificate()` in `src/openamp_foundry/evidence/certificate.py`
- Auto-computes three cheap-baseline flags: charge≥4, length 10-40aa, hydrophobic_fraction≥0.30
- Warns when all three pass: "simple conjunction rule would select this candidate without ML scoring"
- 63 tests in `tests/test_certificate_baseline_caveat.py` covering presence, charge/length/hydro flags, all-YES/NO paths, claim discipline, integration
- BASELINE 6331→6394
- Closes Phase B B3 — cheap-explanation visibility is now mandatory in every certificate

### v0.10.27 — Phase B B2: CertificateClaimBoundary schema
- Added `CertificateClaimBoundary` (CCB-) schema
- 10 fields: ccb_id, pipeline_version, certificate_id, candidate_id, boundary_date, unsupported_claim_classes (8-value vocab), boundary_statement, dry_lab_only, all_listed_classes_unsupported, notes
Expand Down
2 changes: 1 addition & 1 deletion docs/research/NEXT_100_PR_MAP.md
Original file line number Diff line number Diff line change
Expand Up @@ -43,7 +43,7 @@ Make candidate evidence packages impossible to confuse with biological proof.
|---:|---|---|---|
| B1 | Add certificate field for `proof_ladder_level` (complete). | PLC- schema: 14 fields, 14 validation rules, dry_lab_only caps level at multi_signal_candidate_evidence, human_review_required enforced for levels ≥expert_reviewed_assay_proposal, unsupported_claims required as anti-overclaim guard; makes level claim machine-verifiable. | B |
| B2 | Add `unsupported_claims` field to certificates (complete). | CCB- schema: 10 fields, 13 validation rules, 8 claim class vocabulary, ≥3 classes required, dry_lab_only+all_listed_classes_unsupported enforced, no-duplicate check; negative complement of PLC- closes score-to-proof drift gap. | B |
| B3 | Add `baseline_caveat` field. | Forces cheap-explanation visibility. | B |
| B3 | Add `baseline_caveat` field (complete). | Added to build_certificate(): auto-computes charge/length/hydrophobicity flags and warns when all three cheap baselines pass; forces cheap-explanation visibility at certificate issue time. 63 tests in tests/test_certificate_baseline_caveat.py. | B |
| B4 | Add `release_status` field. | Supports staged release. | C/D |
| B5 | Add certificate quality-tier validator. | External-review readiness becomes measurable. | B |
| B6 | Add human-readable certificate report. | Domain experts can inspect faster. | B |
Expand Down
63 changes: 57 additions & 6 deletions src/openamp_foundry/evidence/certificate.py
Original file line number Diff line number Diff line change
Expand Up @@ -7,14 +7,16 @@
from openamp_foundry.types import ScoredCandidate
from openamp_foundry.utils.hashing import stable_json_hash

# Cheap-baseline thresholds for AMP selection heuristics.
# These define what a simple rule-based filter would pass —
# a candidate passing all three is explainable without ML.
_BASELINE_MIN_CHARGE = 4
_BASELINE_LENGTH_MIN = 10
_BASELINE_LENGTH_MAX = 40
_BASELINE_MIN_HYDROPHOBIC_FRACTION = 0.30

def _infer_proof_ladder_level(scored: ScoredCandidate) -> str:
"""Infer highest proof-ladder level supported by this candidate's evidence.

Pipeline outputs are multi_signal_candidate_evidence (Level 4) by default:
they have feature extraction, multiple independent scorers, novelty checking,
and diversity selection. Lower levels apply only if key signals are missing.
"""
def _infer_proof_ladder_level(scored: ScoredCandidate) -> str:
scores = scored.scores
has_multi_signal = (
scores.get("activity", 0) > 0
Expand All @@ -26,6 +28,54 @@ def _infer_proof_ladder_level(scored: ScoredCandidate) -> str:
return "multi_signal_candidate_evidence"


def _build_baseline_caveat(scored: ScoredCandidate) -> str:
features = scored.features
charge = float(features.get("net_charge_proxy", 0))
length = int(features.get("length", 0))
hydro = float(features.get("hydrophobic_fraction", 0.0))

charge_passes = charge >= _BASELINE_MIN_CHARGE
length_passes = _BASELINE_LENGTH_MIN <= length <= _BASELINE_LENGTH_MAX
hydro_passes = hydro >= _BASELINE_MIN_HYDROPHOBIC_FRACTION

flags_yes = sum([charge_passes, length_passes, hydro_passes])

charge_flag = "YES" if charge_passes else "NO"
length_flag = "YES" if length_passes else "NO"
hydro_flag = "YES" if hydro_passes else "NO"

if flags_yes == 3:
residual_note = (
"All three baseline flags pass — a simple conjunction rule "
"(charge\u22654 AND length 10-40 AND hydrophobicity\u22650.30) would select "
"this candidate without ML scoring. Residual ML signal must exceed "
"this baseline to justify pipeline use."
)
elif flags_yes == 2:
residual_note = (
"Two of three baseline flags pass — candidate partially explainable "
"by heuristic rules. Verify ML residual is non-trivial."
)
elif flags_yes == 1:
residual_note = (
"One of three baseline flags passes — candidate unlikely to be "
"selected by simple heuristics alone."
)
else:
residual_note = (
"No baseline flags pass — candidate cannot be explained by the "
"standard cheap heuristics; ML signal is dominant."
)

return (
f"Cheapest-explanation check: "
f"charge={charge:.1f} (\u2265{_BASELINE_MIN_CHARGE} passes: {charge_flag}), "
f"length={length} ({_BASELINE_LENGTH_MIN}-{_BASELINE_LENGTH_MAX} aa range: {length_flag}), "
f"hydrophobic_fraction={hydro:.2f} (\u2265{_BASELINE_MIN_HYDROPHOBIC_FRACTION:.2f} passes: {hydro_flag}). "
f"{residual_note}"
)


def build_certificate(
scored: ScoredCandidate,
config: dict[str, Any],
Expand All @@ -50,4 +100,5 @@ def build_certificate(
"pipeline_version": __version__,
"config_hash": stable_json_hash(config),
"proof_ladder_level": _infer_proof_ladder_level(scored),
"baseline_caveat": _build_baseline_caveat(scored),
}
Loading
Loading