feat: Phase C C2 benchmark card schema (BMC-) -- prevents incomplete benchmark docs

docs/evidence/BENCHMARK_CARD_GUIDE.md

@@ -0,0 +1,57 @@
+# Benchmark Card Guide (BMC-)
+
+## Purpose
+
+A `BenchmarkCard` is the machine-readable documentation contract for every benchmark in the pipeline. Before a benchmark can have ranking authority over candidates, it must have a corresponding BMC- record that documents:
+
+1. What it measures (`measurement_target`)
+2. How data is split (`split_strategy`)
+3. What cheap baselines it must beat (`cheap_enemy_baselines`)
+4. What evaluation metrics are used (`evaluation_metrics`)
+5. What its known limitations are (`known_limitations`)
+
+This prevents benchmark sprawl and enforces anti-hype governance at the schema level.
+
+## Required Fields
+
+| Field | Type | Description |
+|-------|------|-------------|
+| `bmc_id` | str | Unique ID starting with `BMC-` |
+| `pipeline_version` | str | Pipeline version when benchmark was defined |
+| `benchmark_name` | str | Human-readable name |
+| `measurement_target` | str | What's being measured (controlled vocab) |
+| `split_strategy` | str | How data is split (controlled vocab) |
+| `cheap_enemy_baselines` | List[str] | ≥1 cheap baseline this benchmark must beat |
+| `evaluation_metrics` | List[str] | Evaluation metrics (controlled vocab) |
+| `known_limitations` | List[str] | ≥1 documented limitation |
+| `deprecated` | bool | Whether this benchmark is deprecated |
+| `created_date` | str | ISO 8601 date |
+| `last_updated_date` | str | ISO 8601 date |
+
+## Optional Fields
+
+| Field | Default | Description |
+|-------|---------|-------------|
+| `notes` | "" | Additional context (max 500 chars). Required when deprecated=True. |
+
+## Validation Rules
+
+1. `bmc_id` must start with `BMC-`
+2. `pipeline_version` non-empty
+3. `benchmark_name` non-empty
+4. `measurement_target` in `VALID_MEASUREMENT_TARGETS`
+5. `split_strategy` in `VALID_SPLIT_STRATEGIES`
+6. `cheap_enemy_baselines` ≥1 entry, all non-empty strings
+7. All entries in `cheap_enemy_baselines` non-empty
+8. `evaluation_metrics` ≥1 entry, all in `VALID_EVALUATION_METRICS`
+9. `known_limitations` ≥1 entry, all non-empty strings
+10. `created_date` ISO 8601 format
+11. `last_updated_date` ISO 8601 format
+12. If `deprecated=True`, `notes` must explain why
+13. `notes` ≤500 characters
+14. `evaluation_metrics` entries in controlled vocab
+
+## Anti-Hype Constraints
+
+- A benchmark with no `cheap_enemy_baselines` cannot prove the method beats trivial rules. **Warnings** are emitted when only one cheap enemy is declared; two or more are preferred.
+- Every benchmark has limitations. Documenting zero limitations is not allowed.

docs/evidence/METRICS_CURRENT.md

@@ -9,6 +9,15 @@ Machine-readable snapshot: `outputs/metrics_snapshot.json` regenerated with `mak
 
 ## Changelog
 
+### v0.10.33 — Phase C C2: Benchmark card schema (BMC-)
+- Added `src/openamp_foundry/evidence/benchmark_card.py` — BenchmarkCard schema
+- BMC- schema: 12 fields (bmc_id, pipeline_version, benchmark_name, measurement_target, split_strategy, cheap_enemy_baselines, evaluation_metrics, known_limitations, deprecated, created_date, last_updated_date, notes)
+- 14 validation rules: BMC- prefix, controlled vocabs for measurement_target (10 values)/split_strategy (10 values)/evaluation_metrics (12 values), cheap_enemy_baselines ≥1 required, known_limitations ≥1 required, deprecated+notes dependency, notes ≤500 chars
+- 2 warnings: <2 cheap enemies, <2 known limitations
+- 63 tests in `tests/evidence/test_benchmark_card.py`
+- BASELINE 6646→6709
+- Closes Phase C C2 — benchmark documentation is now machine-checkable; incomplete benchmark docs are blocked at schema level
+
 ### v0.10.32 — Phase B B8: Certificates linked to run manifest hashes
 - Added `run_id` and `run_manifest_hash` optional parameters to `build_certificate()` in `src/openamp_foundry/evidence/certificate.py`
 - Every certificate now carries the pipeline run_id and the SHA256 hash of the run manifest that produced it

docs/research/NEXT_100_PR_MAP.md

@@ -59,7 +59,7 @@ Make OpenAMP the hardest repo to fool in AMP dry-lab selection.
 | PR | Task | Why it matters | Review class |
 |---:|---|---|---|
 | C1 | Create machine-readable benchmark cards. | Governance becomes enforceable. | B |
-| C2 | Add benchmark-card schema. | Prevents incomplete benchmark docs. | B |
+| C2 | Add benchmark-card schema (complete). | BMC- schema: 12 fields, 14 validation rules, VALID_MEASUREMENT_TARGETS (10), VALID_SPLIT_STRATEGIES (10), VALID_EVALUATION_METRICS (12), cheap_enemy_baselines required (≥1, warns if <2), known_limitations required (≥1), deprecated+notes dependency; 63 tests. | B |
 | C3 | Add charge-distribution report for every benchmark. | Shortcut visibility. | B/C |
 | C4 | Add similarity-neighbor distribution report. | Novelty honesty. | B/C |
 | C5 | Add `bench-length-matched` control. | Tests length shortcut. | C |

src/openamp_foundry/evidence/benchmark_card.py

@@ -0,0 +1,209 @@
+"""BenchmarkCard schema — Phase C C2.
+
+Machine-readable documentation for each benchmark in the pipeline.
+Prevents incomplete benchmark docs by requiring: measurement target,
+split strategy, at least one cheap enemy baseline (must be beaten before
+an advanced scorer gets ranking authority), at least one known limitation.
+
+ID prefix: BMC-
+"""
+
+from __future__ import annotations
+
+import re
+from dataclasses import dataclass, field
+from typing import List
+
+BMC_PREFIX = "BMC-"
+
+VALID_SPLIT_STRATEGIES = frozenset({
+    "random_80_20",
+    "random_70_30",
+    "family_stratified",
+    "charge_stratified",
+    "length_stratified",
+    "temporal_split",
+    "scaffold_split",
+    "leakage_aware_split",
+    "cross_validation_5fold",
+    "cross_validation_10fold",
+})
+
+VALID_MEASUREMENT_TARGETS = frozenset({
+    "antimicrobial_activity_prediction",
+    "hemolysis_risk_prediction",
+    "toxicity_prediction",
+    "novelty_scoring",
+    "synthesis_feasibility",
+    "diversity_selection",
+    "ensemble_ranking",
+    "safety_scoring",
+    "selectivity_prediction",
+    "calibration_accuracy",
+})
+
+VALID_EVALUATION_METRICS = frozenset({
+    "precision_at_k",
+    "recall_at_k",
+    "auc_roc",
+    "auc_pr",
+    "mcc",
+    "f1",
+    "accuracy",
+    "spearman_rho",
+    "pearson_r",
+    "enrichment_factor",
+    "calibration_error",
+    "brier_score",
+})
+
+_MIN_CHEAP_ENEMIES = 1
+_MIN_KNOWN_LIMITATIONS = 1
+_NOTES_MAX_LENGTH = 500
+_ISO_DATE_RE = re.compile(r"^\d{4}-\d{2}-\d{2}(T[\d:.+Z-]+)?$")
+
+
+@dataclass
+class BenchmarkCard:
+    bmc_id: str
+    pipeline_version: str
+    benchmark_name: str
+    measurement_target: str
+    split_strategy: str
+    cheap_enemy_baselines: List[str]
+    evaluation_metrics: List[str]
+    known_limitations: List[str]
+    deprecated: bool
+    created_date: str
+    last_updated_date: str
+    notes: str = ""
+
+    def validate(self) -> list[str]:
+        errors: list[str] = []
+        warnings: list[str] = []
+
+        # Rule 1: BMC- prefix
+        if not self.bmc_id.startswith(BMC_PREFIX):
+            errors.append(f"bmc_id must start with '{BMC_PREFIX}', got {self.bmc_id!r}")
+
+        # Rule 2: pipeline_version non-empty
+        if not self.pipeline_version.strip():
+            errors.append("pipeline_version must not be empty")
+
+        # Rule 3: benchmark_name non-empty
+        if not self.benchmark_name.strip():
+            errors.append("benchmark_name must not be empty")
+
+        # Rule 4: measurement_target in controlled vocab
+        if self.measurement_target not in VALID_MEASUREMENT_TARGETS:
+            errors.append(
+                f"measurement_target must be one of {sorted(VALID_MEASUREMENT_TARGETS)}, "
+                f"got {self.measurement_target!r}"
+            )
+
+        # Rule 5: split_strategy in controlled vocab
+        if self.split_strategy not in VALID_SPLIT_STRATEGIES:
+            errors.append(
+                f"split_strategy must be one of {sorted(VALID_SPLIT_STRATEGIES)}, "
+                f"got {self.split_strategy!r}"
+            )
+
+        # Rule 6: cheap_enemy_baselines non-empty
+        if not self.cheap_enemy_baselines or len(self.cheap_enemy_baselines) < _MIN_CHEAP_ENEMIES:
+            errors.append(
+                f"cheap_enemy_baselines must have at least {_MIN_CHEAP_ENEMIES} entry — "
+                "a benchmark with no declared cheap enemies cannot prove the method beats "
+                "trivial baselines"
+            )
+
+        # Rule 7: all cheap enemy names non-empty strings
+        for i, enemy in enumerate(self.cheap_enemy_baselines):
+            if not isinstance(enemy, str) or not enemy.strip():
+                errors.append(f"cheap_enemy_baselines[{i}] must be a non-empty string")
+
+        # Rule 8: evaluation_metrics non-empty and in vocab
+        if not self.evaluation_metrics:
+            errors.append("evaluation_metrics must have at least one entry")
+        else:
+            for metric in self.evaluation_metrics:
+                if metric not in VALID_EVALUATION_METRICS:
+                    errors.append(
+                        f"evaluation_metrics entry {metric!r} must be one of "
+                        f"{sorted(VALID_EVALUATION_METRICS)}"
+                    )
+
+        # Rule 9: known_limitations non-empty
+        if not self.known_limitations or len(self.known_limitations) < _MIN_KNOWN_LIMITATIONS:
+            errors.append(
+                f"known_limitations must have at least {_MIN_KNOWN_LIMITATIONS} entry — "
+                "every benchmark has limitations; document them or do not ship the benchmark"
+            )
+
+        # Rule 10: all known_limitations non-empty strings
+        for i, lim in enumerate(self.known_limitations):
+            if not isinstance(lim, str) or not lim.strip():
+                errors.append(f"known_limitations[{i}] must be a non-empty string")
+
+        # Rule 11: created_date ISO format
+        if not _ISO_DATE_RE.match(self.created_date):
+            errors.append(
+                f"created_date must be ISO 8601 date, got {self.created_date!r}"
+            )
+
+        # Rule 12: last_updated_date ISO format
+        if not _ISO_DATE_RE.match(self.last_updated_date):
+            errors.append(
+                f"last_updated_date must be ISO 8601 date, got {self.last_updated_date!r}"
+            )
+
+        # Rule 13: if deprecated, notes must explain why
+        if self.deprecated and not self.notes.strip():
+            errors.append(
+                "notes must explain why the benchmark is deprecated when deprecated=True"
+            )
+
+        # Rule 14: notes length cap
+        if len(self.notes) > _NOTES_MAX_LENGTH:
+            errors.append(
+                f"notes must be ≤{_NOTES_MAX_LENGTH} characters, got {len(self.notes)}"
+            )
+
+        # Warning: only one cheap enemy (weak — two or more preferred)
+        if len(self.cheap_enemy_baselines) < 2:
+            warnings.append(
+                "cheap_enemy_baselines has only one entry; two or more cheap enemies "
+                "provide stronger anti-hype evidence"
+            )
+
+        # Warning: only one known limitation (probably incomplete)
+        if len(self.known_limitations) < 2:
+            warnings.append(
+                "known_limitations has only one entry; document at least two limitations "
+                "for a complete benchmark card"
+            )
+
+        if errors:
+            return errors
+        return [f"WARNING: {w}" for w in warnings] if warnings else []
+
+
+def validate_dict(data: dict) -> list[str]:
+    """Validate a dict representation of BenchmarkCard."""
+    try:
+        record = BenchmarkCard(
+            bmc_id=data.get("bmc_id", ""),
+            pipeline_version=data.get("pipeline_version", ""),
+            benchmark_name=data.get("benchmark_name", ""),
+            measurement_target=data.get("measurement_target", ""),
+            split_strategy=data.get("split_strategy", ""),
+            cheap_enemy_baselines=data.get("cheap_enemy_baselines", []),
+            evaluation_metrics=data.get("evaluation_metrics", []),
+            known_limitations=data.get("known_limitations", []),
+            deprecated=data.get("deprecated", False),
+            created_date=data.get("created_date", ""),
+            last_updated_date=data.get("last_updated_date", ""),
+            notes=data.get("notes", ""),
+        )
+        return record.validate()
+    except Exception as exc:
+        return [f"Construction error: {exc}"]