feat: Phase P P2 recalibration refusal record schema -- auditable rejection of spurious recalibrations

docs/evidence/METRICS_CURRENT.md

@@ -5,7 +5,8 @@ Machine-readable snapshot: `outputs/metrics_snapshot.json` regenerated with `mak
 > **Purpose:** One authoritative table of current pipeline metrics. If any doc disagrees
 > with this file, this file wins. Updated whenever benchmark/benchmark config changes.
 >
-> **Last updated:** 2026-07-10 (Phase P P1 — batch selection proposal — v0.10.4)
+> **Last updated:** 2026-07-10 (Phase P P2 — recalibration refusal record — v0.10.5)
+> **New in v0.10.5 (Phase P P2):** RecalibrationRefusalEntry schema (10 fields: rrf_id, pipeline_version, trigger_id, recalibration_refused, refusal_reason, minimum_batches_required, batches_evaluated, refusal_notes, reviewer, dry_lab_only). recalibration_refused must be True — this schema only records valid refusals. 7 validation rules (RRF- prefix, CPS-/DRM- trigger prefix, recalibration_refused=True enforced, reason in 5 valid values, minimum_batches>=1, batches_evaluated>=0, notes<=400 chars). 2 warnings (insufficient_data but meets minimum; reviewer_override without documentation). Complements CalibrationImprovementRecord (O3). Added validate_recalibration_refusal() and validate_recalibration_refusal_dict(). Added 63 tests. CLI: openamp-foundry recalibration-refusal-check with --entry-json and --format text|json. Corrected inflated BASELINE from 4889 to 4784 (actual count 4721 + 63 = 4784).
 > **New in v0.10.4 (Phase P P1):** BatchSelectionProposalEntry schema (12 fields: bsp_id, pipeline_version, gate_id, gate_passed, candidate_ids, selection_strategy, exploitation_fraction, exploration_fraction, max_brier_score_allowed, proposal_notes, reviewer, dry_lab_only). gate_passed must be True — a batch can only be proposed when the calibration readiness gate passes. 9 validation rules (BSP- prefix, CRG- prefix, gate_passed=True enforced, candidate_ids >=1, strategy in 5 valid values, fraction bounds, fraction sum=1.0 to 0.001 tolerance, Brier bounds [0.0,1.0], notes <=400 chars). 3 warnings (pure exploitation >=0.99, single candidate, marginal Brier >=0.25). Added validate_batch_selection_proposal() and validate_batch_selection_proposal_dict(). Added 68 tests. CLI: openamp-foundry batch-selection-proposal-check with --entry-json and --format text|json. Closes the calibration-to-selection loop from Phase O.
 > **New in v0.10.3 (Phase O O5):** CalibrationReadinessEntry schema (11 fields, 8 validation rules, 2 warnings). Added validate_calibration_readiness() and validate_calibration_readiness_dict(). Added 68 tests. CLI: openamp-foundry calibration-readiness-check. Phase O (Calibration Quality Assurance) COMPLETE.
 > **New in v0.10.2 (Phase O O4):** CrossBatchAggregatorEntry schema (12 fields, 9 validation rules, 3 warnings). Added validate_cross_batch_aggregator() and validate_cross_batch_aggregator_dict(). Added 68 tests. CLI: openamp-foundry cross-batch-aggregator-check.
@@ -1230,6 +1231,7 @@ Decoys score low on activity. Selective AMPs score moderately on both.
 
 | Date | Change | Author |
 |------|--------|--------|
+| 2026-07-10 | **Recalibration refusal record schema (Phase P P2):** RecalibrationRefusalEntry (10 fields, recalibration_refused must be True). 7 validation rules, 2 warnings. 63 tests. CLI: openamp-foundry recalibration-refusal-check. Complements CalibrationImprovementRecord (O3) with the rejection path. Corrected inflated BASELINE 4889->4784. | OpenAMP Loop |
 | 2026-07-09 | **Release checklist and gate validator (J1, starts Phase J):** `docs/governance/RELEASE_CHECKLIST.md` with structured checklist. `src/openamp_foundry/governance/release_gate.py` with `RELEASE_TYPES` (5), `UNIVERSAL_GATES` (7), `EXTRA_GATES_BY_TYPE`, `ReleaseGateResult`, `validate_release_gate()`. CLI `release-gate-check`. `make release-gate-check`. 18 tests. **Phase J (Governance and release maturity) started. 3478 total.** | OpenAMP Loop 109 |
 | 2026-07-09 | **Adoption scorecard dashboard added (I10):** `src/openamp_foundry/adoption/scorecard.py` with `SCORECARD_DIMENSIONS` (5, weights sum 1.0), `ADOPTION_TIERS` (4), `DimensionScore`, `AdoptionScorecard`, `build_scorecard()`, `compute_adoption_tier()`. CLI `adoption-scorecard` with `--scores-json` and `--format`. `make adoption-scorecard`. 17 tests. **Phase I (Interoperability and Adoption) is now complete** — all 10 items I1–I10 implemented. **3446 total.** | OpenAMP Loop 108 |
 | 2026-07-09 | **Adapter author validator added (I6):** `src/openamp_foundry/adapters/adapter_validator.py` with `AdapterDeclaration` (14 fields), `AdapterValidationResult` (5 fields), `validate_adapter_declaration()` (10 checks enforcing ADAPTER_AUTHOR_GUIDE contract), `validate_adapter_dict()` (dict input with missing-fields guard). 4 valid-value sets (VALID_ADAPTER_MODES, VALID_OUTPUT_STATUSES, VALID_RANKING_EFFECTS, VALID_RELEASE_STATUSES). CLI (`openamp-foundry adapter-check`) with `--adapter-json`, `--format text|json`. `make adapter-author-check` target. 31 tests. **3387 total.** | OpenAMP Loop 104 |

docs/evidence/RECALIBRATION_REFUSAL_GUIDE.md

@@ -0,0 +1,68 @@
+# Recalibration Refusal Record Guide
+
+## Purpose
+
+A Recalibration Refusal Record (RRF) documents when a recalibration evaluation was
+conducted and the decision was to **not** recalibrate.
+
+Refusing to recalibrate on weak evidence is a quality outcome.  Without explicit
+refusal records, a pipeline might silently skip recalibration with no audit trail.
+With RRF records, every evaluated-but-rejected recalibration trigger is documented.
+
+## Relationship to other schemas
+
+| Schema | Records |
+|---|---|
+| `CalibrationPerformanceSummary` (CPS-) | Accuracy metrics per batch |
+| `PredictionDriftMonitor` (DRM-) | Detected drift between batches |
+| `CalibrationImprovementRecord` (CIR-) | Approved recalibrations |
+| `RecalibrationRefusalRecord` (RRF-) | Rejected recalibration evaluations |
+
+The trigger for an RRF must be a CPS- or DRM- entry.
+
+## Key constraint
+
+`recalibration_refused` must be `True`.  This schema only records valid refusals.
+If a recalibration was approved, use `CalibrationImprovementRecord` instead.
+
+## Schema fields
+
+| Field | Type | Description |
+|---|---|---|
+| `rrf_id` | str | Must start with `RRF-` |
+| `pipeline_version` | str | Pipeline version at evaluation time |
+| `trigger_id` | str | CPS- or DRM- entry that triggered evaluation |
+| `recalibration_refused` | bool | Must be `True` |
+| `refusal_reason` | str | One of 5 valid categories |
+| `minimum_batches_required` | int | Minimum batches needed to justify recalibration |
+| `batches_evaluated` | int | Batches actually available |
+| `refusal_notes` | str | Max 400 chars |
+| `reviewer` | str | Who made the refusal decision |
+| `dry_lab_only` | bool | Default True |
+
+## Refusal reasons
+
+| Reason | When to use |
+|---|---|
+| `insufficient_data` | Too few batches to justify recalibration |
+| `effect_within_noise` | Detected signal is within expected noise range |
+| `recent_recalibration` | Recalibration was done too recently |
+| `conflicting_signals` | Multiple signals point in different directions |
+| `reviewer_override` | Human reviewer overrode automated recommendation |
+
+## Honest boundaries
+
+- An RRF records a human or pipeline decision — it does not prove the decision was
+  correct. Future batches may reveal the refusal was wrong.
+- The RRF does not validate that `trigger_id` references a real CPS- or DRM- entry;
+  referential integrity requires a separate audit step.
+- `minimum_batches_required` is a policy value, not a statistical guarantee. The same
+  number of batches may be sufficient in some scenarios and insufficient in others.
+
+## Warnings
+
+The validator emits warnings (not errors) for:
+
+- `refusal_reason="insufficient_data"` but `batches_evaluated >= minimum_batches_required`
+  — consider a more precise reason.
+- `refusal_reason="reviewer_override"` — document the specific reason in `refusal_notes`.

docs/research/NEXT_100_PR_MAP.md

@@ -258,6 +258,7 @@ preventing the pipeline from selecting candidates under poor calibration.
 | PR | Task | Why it matters | Review class |
 |---:|---|---|---|
 | P1 | Add batch selection proposal schema (complete). | Enforces calibration gate must pass before proposing a next batch; documents exploitation/exploration strategy. | B/D |
+| P2 | Add recalibration refusal record schema (complete). | Documents when recalibration was correctly rejected; prevents spurious recalibrations and creates audit trail. | B/D |
 
 ## Prioritization rule
 

src/openamp_foundry/cli/commands/reports.py

@@ -3607,3 +3607,30 @@ def _run_batch_selection_proposal_check(args: argparse.Namespace) -> int:
             print(f"  WARN: {w}")
 
     return 0 if result.passed else 1
+
+
+def _run_recalibration_refusal_check(args) -> int:
+    import json
+    from openamp_foundry.evidence.recalibration_refusal_record import (
+        validate_recalibration_refusal_dict,
+    )
+    if args.entry_json:
+        try:
+            data = json.loads(args.entry_json)
+        except json.JSONDecodeError as exc:
+            print(f"Error: invalid JSON: {exc}", file=__import__("sys").stderr)
+            return 1
+    else:
+        data = json.load(__import__("sys").stdin)
+    result = validate_recalibration_refusal_dict(data)
+    if args.format == "json":
+        import dataclasses
+        print(json.dumps(dataclasses.asdict(result), indent=2))
+    else:
+        status = "PASS" if result.passed else "FAIL"
+        print(f"[{status}] Recalibration Refusal: {result.rrf_id}")
+        for err in result.errors:
+            print(f"  ERROR: {err}")
+        for warn in result.warnings:
+            print(f"  WARN:  {warn}")
+    return 0 if result.passed else 1

src/openamp_foundry/cli/main.py

@@ -24,6 +24,7 @@
     _run_cross_batch_aggregator_check,
     _run_calibration_readiness_check,
     _run_batch_selection_proposal_check,
+    _run_recalibration_refusal_check,
 )
 from openamp_foundry.cli.commands.gates import _run_gate_check, _run_release_gate_check
 from openamp_foundry.cli.commands.reports import _run_contribution_check
@@ -2365,6 +2366,14 @@ def build_parser() -> argparse.ArgumentParser:
     p_bsp.add_argument("--format", choices=["text", "json"], default="text")
     p_bsp.set_defaults(func=_run_batch_selection_proposal_check)
 
+    p_rrf = sub.add_parser(
+        "recalibration-refusal-check",
+        help="Validate recalibration refusal record entry",
+    )
+    p_rrf.add_argument("--entry-json", default=None)
+    p_rrf.add_argument("--format", choices=["text", "json"], default="text")
+    p_rrf.set_defaults(func=_run_recalibration_refusal_check)
+
     # ── Selection rationale check (Phase K K1) ───────────────────────
     src2 = sub.add_parser(
         "selection-rationale-check",
@@ -2743,6 +2752,9 @@ def main(argv: list[str] | None = None) -> int:
     if args.command == "batch-selection-proposal-check":
         return _run_batch_selection_proposal_check(args)
 
+    if args.command == "recalibration-refusal-check":
+        return _run_recalibration_refusal_check(args)
+
     parser.error("unknown command")
     return 2
 

src/openamp_foundry/evidence/recalibration_refusal_record.py

@@ -0,0 +1,180 @@
+"""Recalibration refusal record schema — Phase P P2.
+
+Documents when calibration recalibration was correctly rejected because the
+evidence was insufficient or the detected signal was within normal noise.
+
+A refusal is a quality outcome, not a failure.  Refusing to recalibrate on
+weak evidence preserves pipeline integrity.
+"""
+
+from __future__ import annotations
+
+from dataclasses import dataclass, field
+from typing import List
+
+RRF_PREFIX = "RRF-"
+VALID_TRIGGER_PREFIXES = frozenset({"CPS-", "DRM-"})
+VALID_REFUSAL_REASONS = frozenset(
+    {
+        "insufficient_data",
+        "effect_within_noise",
+        "recent_recalibration",
+        "conflicting_signals",
+        "reviewer_override",
+    }
+)
+REFUSAL_NOTES_MAX_LENGTH = 400
+MIN_BATCHES_FLOOR = 1
+POOR_BRIER_THRESHOLD = 0.25
+
+
+@dataclass
+class RecalibrationRefusalEntry:
+    """A record of a recalibration evaluation that was correctly rejected."""
+
+    rrf_id: str
+    pipeline_version: str
+    trigger_id: str
+    recalibration_refused: bool
+    refusal_reason: str
+    minimum_batches_required: int
+    batches_evaluated: int
+    refusal_notes: str
+    reviewer: str
+    dry_lab_only: bool = True
+
+
+@dataclass
+class RecalibrationRefusalResult:
+    rrf_id: str
+    pipeline_version: str
+    trigger_id: str
+    recalibration_refused: bool
+    refusal_reason: str
+    passed: bool
+    errors: List[str]
+    warnings: List[str]
+    dry_lab_only: bool = True
+
+
+def validate_recalibration_refusal(
+    entry: RecalibrationRefusalEntry,
+) -> RecalibrationRefusalResult:
+    errors: List[str] = []
+    warnings: List[str] = []
+
+    if not entry.rrf_id.startswith(RRF_PREFIX):
+        errors.append(
+            f"rrf_id must start with '{RRF_PREFIX}', got '{entry.rrf_id}'"
+        )
+
+    valid_trigger = any(
+        entry.trigger_id.startswith(p) for p in VALID_TRIGGER_PREFIXES
+    )
+    if not valid_trigger:
+        errors.append(
+            f"trigger_id must start with one of {sorted(VALID_TRIGGER_PREFIXES)}, "
+            f"got '{entry.trigger_id}'"
+        )
+
+    if not entry.recalibration_refused:
+        errors.append(
+            "recalibration_refused must be True — this schema documents refusals, "
+            "not approvals (use CalibrationImprovementRecord for approved recalibrations)"
+        )
+
+    if entry.refusal_reason not in VALID_REFUSAL_REASONS:
+        errors.append(
+            f"refusal_reason must be one of {sorted(VALID_REFUSAL_REASONS)}, "
+            f"got '{entry.refusal_reason}'"
+        )
+
+    if entry.minimum_batches_required < MIN_BATCHES_FLOOR:
+        errors.append(
+            f"minimum_batches_required must be >= {MIN_BATCHES_FLOOR}, "
+            f"got {entry.minimum_batches_required}"
+        )
+
+    if entry.batches_evaluated < 0:
+        errors.append(
+            f"batches_evaluated must be >= 0, got {entry.batches_evaluated}"
+        )
+
+    if len(entry.refusal_notes) > REFUSAL_NOTES_MAX_LENGTH:
+        errors.append(
+            f"refusal_notes exceeds {REFUSAL_NOTES_MAX_LENGTH} characters "
+            f"(got {len(entry.refusal_notes)})"
+        )
+
+    # Warnings
+    if (
+        entry.refusal_reason == "insufficient_data"
+        and entry.batches_evaluated >= entry.minimum_batches_required
+        and not errors
+    ):
+        warnings.append(
+            "refusal_reason is 'insufficient_data' but batches_evaluated "
+            f"({entry.batches_evaluated}) meets minimum_batches_required "
+            f"({entry.minimum_batches_required}) — consider 'effect_within_noise' "
+            "or 'conflicting_signals' as a more precise reason"
+        )
+
+    if entry.refusal_reason == "reviewer_override" and not errors:
+        warnings.append(
+            "refusal_reason is 'reviewer_override' — document the specific reason "
+            "in refusal_notes to maintain auditability"
+        )
+
+    return RecalibrationRefusalResult(
+        rrf_id=entry.rrf_id,
+        pipeline_version=entry.pipeline_version,
+        trigger_id=entry.trigger_id,
+        recalibration_refused=entry.recalibration_refused,
+        refusal_reason=entry.refusal_reason,
+        passed=len(errors) == 0,
+        errors=errors,
+        warnings=warnings,
+        dry_lab_only=entry.dry_lab_only,
+    )
+
+
+def validate_recalibration_refusal_dict(d: dict) -> RecalibrationRefusalResult:
+    missing = []
+    for k in (
+        "rrf_id",
+        "pipeline_version",
+        "trigger_id",
+        "recalibration_refused",
+        "refusal_reason",
+        "minimum_batches_required",
+        "batches_evaluated",
+        "refusal_notes",
+        "reviewer",
+    ):
+        if k not in d:
+            missing.append(k)
+    if missing:
+        return RecalibrationRefusalResult(
+            rrf_id=d.get("rrf_id", ""),
+            pipeline_version=d.get("pipeline_version", ""),
+            trigger_id=d.get("trigger_id", ""),
+            recalibration_refused=d.get("recalibration_refused", False),
+            refusal_reason=d.get("refusal_reason", ""),
+            passed=False,
+            errors=[f"missing required fields: {missing}"],
+            warnings=[],
+            dry_lab_only=d.get("dry_lab_only", True),
+        )
+    entry = RecalibrationRefusalEntry(
+        rrf_id=d["rrf_id"],
+        pipeline_version=d["pipeline_version"],
+        trigger_id=d["trigger_id"],
+        recalibration_refused=bool(d["recalibration_refused"]),
+        refusal_reason=d["refusal_reason"],
+        minimum_batches_required=int(d["minimum_batches_required"]),
+        batches_evaluated=int(d["batches_evaluated"]),
+        refusal_notes=d["refusal_notes"],
+        reviewer=d["reviewer"],
+        dry_lab_only=bool(d.get("dry_lab_only", True)),
+    )
+    return validate_recalibration_refusal(entry)