Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
68 changes: 68 additions & 0 deletions docs/evidence/EXTERNAL_SHARING_CLEARANCE_GUIDE.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,68 @@
# External Sharing Clearance Guide

## Purpose

An `ExternalSharingClearance` (ESC-) records the formal event of sharing a
`PilotEvidencePackage` with an external lab partner. It is the final release
gate in Phase Q — the moment that transforms an internal dry-lab artifact into
an externally shared package.

No PEP should leave the foundry without an ESC record.

## Fields

| Field | Type | Required | Description |
|-------|------|----------|-------------|
| `esc_id` | str | Yes | Unique ID, must start with `ESC-` |
| `pipeline_version` | str | Yes | Version of the pipeline that produced the PEP |
| `pep_id` | str | Yes | Which PEP is being shared (must start with `PEP-`) |
| `pre_id` | str | Yes | Linked pre-registration (must start with `PRE-`) |
| `external_lab_token` | str | Yes | Anonymized partner token — no PII |
| `sharing_date` | str | Yes | ISO YYYY-MM-DD date of sharing |
| `caveat_confirmed` | bool | Yes | Must be `True` — dry-lab caveat was communicated |
| `dry_lab_only_acknowledged` | bool | Yes | Must be `True` — partner acknowledged dry-lab status |
| `sharing_purpose` | str | Yes | `pilot_experiment`, `peer_review`, or `collaboration` |
| `sharing_notes` | str | Yes | Context notes (max 400 chars) |
| `reviewer` | str | Yes | Name of the human who authorized this sharing |
| `dry_lab_only` | bool | No | Default `True` — always True for this schema |

## Validation rules

1. `esc_id` must start with `ESC-`
2. `pep_id` must start with `PEP-`
3. `pre_id` must start with `PRE-`
4. `pipeline_version`, `external_lab_token`, `reviewer` must not be empty
5. `sharing_date` must be ISO format `YYYY-MM-DD`
6. `caveat_confirmed` must be `True`
7. `dry_lab_only_acknowledged` must be `True`
8. `sharing_purpose` must be one of `{pilot_experiment, peer_review, collaboration}`
9. `sharing_notes` must be at most 400 characters

## Warnings

- Empty `sharing_notes` — document the context
- `sharing_purpose = pilot_experiment` — extra reminder to confirm safety clearance with partner
- `sharing_purpose = collaboration` with no notes — document scope

## Sharing purposes

| Purpose | When to use |
|---------|-------------|
| `pilot_experiment` | Package is shared for actual wet-lab testing |
| `peer_review` | Package is shared for scientific peer review only |
| `collaboration` | Package is shared for joint research without immediate wet-lab commitment |

## Honest boundaries

- An ESC record does NOT guarantee the external partner will perform the experiment.
- An ESC record does NOT upgrade the proof level — all contents remain dry-lab-only.
- The `external_lab_token` must never contain PII (names, emails, institutions directly identifying individuals).
- An ESC with `caveat_confirmed=False` will always fail validation.

## CLI usage

```bash
openamp-foundry external-sharing-clearance-check \
--entry-json '{"esc_id":"ESC-001","pipeline_version":"v0.10.8","pep_id":"PEP-001","pre_id":"PRE-001","external_lab_token":"LAB-A","sharing_date":"2026-07-10","caveat_confirmed":true,"dry_lab_only_acknowledged":true,"sharing_purpose":"pilot_experiment","sharing_notes":"MIC validation run.","reviewer":"Dr. Smith"}' \
--format text
```
3 changes: 2 additions & 1 deletion docs/evidence/METRICS_CURRENT.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,8 @@ Machine-readable snapshot: `outputs/metrics_snapshot.json` regenerated with `mak
> **Purpose:** One authoritative table of current pipeline metrics. If any doc disagrees
> with this file, this file wins. Updated whenever benchmark/benchmark config changes.
>
> **Last updated:** 2026-07-10 (Phase Q Q2 — pre-registration entry — v0.10.8)
> **Last updated:** 2026-07-10 (Phase Q Q3 — external sharing clearance — v0.10.9)
> **New in v0.10.9 (Phase Q Q3):** ExternalSharingClearance — auditable release gate before external PEP sharing (12 fields, 9 validation rules, 3 warnings, 63 tests, BASELINE 5099→5162)
> **New in v0.10.8 (Phase Q Q2):** PreRegistrationEntry — pre-experiment plan lock (14 fields, 9 validation rules, 4 warnings, 63 tests, BASELINE 5036→5099)
> **New in v0.10.7 (Phase Q Q1):** PilotEvidencePackageEntry — external export bundle (14 fields, CCS+BSP+PSC+PRE+BCM refs, completeness+safety enforcement, 63 tests, BASELINE 4973→5036).
> **New in v0.10.5 (Phase P P2):** RecalibrationRefusalEntry schema (10 fields: rrf_id, pipeline_version, trigger_id, recalibration_refused, refusal_reason, minimum_batches_required, batches_evaluated, refusal_notes, reviewer, dry_lab_only). recalibration_refused must be True — this schema only records valid refusals. 7 validation rules (RRF- prefix, CPS-/DRM- trigger prefix, recalibration_refused=True enforced, reason in 5 valid values, minimum_batches>=1, batches_evaluated>=0, notes<=400 chars). 2 warnings (insufficient_data but meets minimum; reviewer_override without documentation). Complements CalibrationImprovementRecord (O3). Added validate_recalibration_refusal() and validate_recalibration_refusal_dict(). Added 63 tests. CLI: openamp-foundry recalibration-refusal-check with --entry-json and --format text|json. Corrected inflated BASELINE from 4889 to 4784 (actual count 4721 + 63 = 4784).
Expand Down
1 change: 1 addition & 0 deletions docs/research/NEXT_100_PR_MAP.md
Original file line number Diff line number Diff line change
Expand Up @@ -274,6 +274,7 @@ bundle.
|---:|---|---|---|
| Q1 | Add pilot evidence package schema (complete). | External export artifact: bundles CCS+BSP+PSC+PRE+BCM references, enforces completeness and safety clearance before external sharing. | B/D |
| Q2 | Add pre-registration entry schema (complete). | Locks hypothesis, endpoint, and candidate list before wet-lab begins; prevents HARKing; referenced by PEP as pre_registration_id. | B/D |
| Q3 | Add external sharing clearance schema (complete). | Auditable release gate: records who received a PEP, when, with what dry-lab caveat confirmation; no PEP leaves the foundry without an ESC record. | B/D |

## Prioritization rule

Expand Down
41 changes: 41 additions & 0 deletions src/openamp_foundry/cli/commands/reports.py
Original file line number Diff line number Diff line change
Expand Up @@ -3785,3 +3785,44 @@ def _run_pre_registration_entry_check(args) -> int:
for warn in result.warnings:
print(f" WARN: {warn}")
return 0 if result.passed else 1


def _run_external_sharing_clearance_check(args):
from openamp_foundry.evidence.external_sharing_clearance import (
validate_external_sharing_clearance_dict,
)
import json
import sys

data = json.loads(args.entry_json)
result = validate_external_sharing_clearance_dict(data)

if args.format == "json":
out = {
"esc_id": result.esc_id,
"pep_id": result.pep_id,
"pre_id": result.pre_id,
"sharing_purpose": result.sharing_purpose,
"passed": result.passed,
"errors": result.errors,
"warnings": result.warnings,
"dry_lab_only": result.dry_lab_only,
}
print(json.dumps(out, indent=2))
else:
status = "PASS" if result.passed else "FAIL"
print(f"External Sharing Clearance: {status}")
print(f" ESC ID: {result.esc_id}")
print(f" PEP ID: {result.pep_id}")
print(f" PRE ID: {result.pre_id}")
print(f" Purpose: {result.sharing_purpose}")
if result.errors:
print(" Errors:")
for e in result.errors:
print(f" - {e}")
if result.warnings:
print(" Warnings:")
for w in result.warnings:
print(f" - {w}")

sys.exit(0 if result.passed else 1)
15 changes: 14 additions & 1 deletion src/openamp_foundry/cli/main.py
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,8 @@
_run_simulation_scope_check, _run_simulation_evidence_packet, _run_artifact_version,
_run_candidate_manifest, _run_benchmark_card, _run_artifact_changelog, _run_integration_check,
_run_adapter_check, _run_license_check, _run_artifact_compat_check, _run_adoption_scorecard,
_run_reviewer_briefing_check, _run_audit_chain_check, _run_pre_registration_check,
_run_reviewer_briefing_check, _run_audit_chain_check, _run_pre_registration_check,
_run_external_sharing_clearance_check,
_run_hypothesis_outcome_check, _run_baseline_comparison_check, _run_negative_result_check,
_run_experiment_priority_check, _run_calibration_performance_check, _run_prediction_drift_check,
_run_calibration_improvement_check,
Expand Down Expand Up @@ -2278,6 +2279,15 @@ def build_parser() -> argparse.ArgumentParser:
)
pre_registration_parser.set_defaults(func=_run_pre_registration_check)

external_sharing_clearance_parser = sub.add_parser(
"external-sharing-clearance-check",
help="Validate an ExternalSharingClearance entry (ESC-).",
)
external_sharing_clearance_parser.add_argument("--entry-json", required=True)
external_sharing_clearance_parser.add_argument(
"--format", choices=["text", "json"], default="text"
)

# ── Hypothesis outcome check (Phase N N2) ────────────────────────
hypothesis_outcome_parser = sub.add_parser(
"hypothesis-outcome-check",
Expand Down Expand Up @@ -2772,6 +2782,9 @@ def main(argv: list[str] | None = None) -> int:
if args.command == "pre-registration-check":
return _run_pre_registration_check(args)

if args.command == "external-sharing-clearance-check":
return _run_external_sharing_clearance_check(args)

if args.command == "hypothesis-outcome-check":
return _run_hypothesis_outcome_check(args)

Expand Down
178 changes: 178 additions & 0 deletions src/openamp_foundry/evidence/external_sharing_clearance.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,178 @@
"""External sharing clearance schema — Phase Q Q3.

Records the formal clearance event when a PilotEvidencePackage (PEP) is
shared with an external lab partner. Makes external sharing auditable:
the dry-lab-only caveat must be confirmed, the external partner must be
identified by anonymous token, and the sharing event is timestamped.

This is the final release gate in Phase Q — no PEP should leave the
foundry without an ESC record.
"""

from __future__ import annotations

import re
from dataclasses import dataclass, field
from typing import List

ESC_PREFIX = "ESC-"
PEP_PREFIX = "PEP-"
PRE_PREFIX = "PRE-"

VALID_SHARING_PURPOSES = frozenset({
"pilot_experiment",
"peer_review",
"collaboration",
})

NOTES_MAX_LENGTH = 400


@dataclass
class ExternalSharingClearance:
"""Clearance record for sharing a PEP with an external lab partner."""

esc_id: str
pipeline_version: str
pep_id: str # must start with "PEP-"
pre_id: str # must start with "PRE-"
external_lab_token: str # anonymized lab identifier, no PII
sharing_date: str # ISO YYYY-MM-DD
caveat_confirmed: bool # must be True — dry-lab-only caveat was communicated
dry_lab_only_acknowledged: bool # must be True — partner acknowledged dry-lab status
sharing_purpose: str # {pilot_experiment, peer_review, collaboration}
sharing_notes: str # max 400 chars
reviewer: str
dry_lab_only: bool = True


@dataclass
class ExternalSharingClearanceResult:
esc_id: str
pep_id: str
pre_id: str
sharing_purpose: str
passed: bool
errors: List[str] = field(default_factory=list)
warnings: List[str] = field(default_factory=list)
dry_lab_only: bool = True


def validate_external_sharing_clearance(
entry: ExternalSharingClearance,
) -> ExternalSharingClearanceResult:
errors: List[str] = []
warnings: List[str] = []

# Rule 1: ESC- prefix
if not entry.esc_id.startswith(ESC_PREFIX):
errors.append(
f"esc_id must start with '{ESC_PREFIX}', got '{entry.esc_id}'"
)

# Rule 2: PEP- prefix
if not entry.pep_id.startswith(PEP_PREFIX):
errors.append(
f"pep_id must start with '{PEP_PREFIX}', got '{entry.pep_id}'"
)

# Rule 3: PRE- prefix
if not entry.pre_id.startswith(PRE_PREFIX):
errors.append(
f"pre_id must start with '{PRE_PREFIX}', got '{entry.pre_id}'"
)

# Rule 4: non-empty required strings
for fname, val in [
("pipeline_version", entry.pipeline_version),
("external_lab_token", entry.external_lab_token),
("reviewer", entry.reviewer),
]:
if not val or not val.strip():
errors.append(f"{fname} must not be empty")

# Rule 5: sharing_date must be ISO YYYY-MM-DD
if not re.match(r"^\d{4}-\d{2}-\d{2}$", entry.sharing_date):
errors.append(
f"sharing_date must be ISO format YYYY-MM-DD, got '{entry.sharing_date}'"
)

# Rule 6: caveat_confirmed must be True
if not entry.caveat_confirmed:
errors.append(
"caveat_confirmed must be True — dry-lab-only caveat must be communicated"
" before external sharing"
)

# Rule 7: dry_lab_only_acknowledged must be True
if not entry.dry_lab_only_acknowledged:
errors.append(
"dry_lab_only_acknowledged must be True — external partner must acknowledge"
" dry-lab-only status before receiving the package"
)

# Rule 8: valid sharing purpose
if entry.sharing_purpose not in VALID_SHARING_PURPOSES:
errors.append(
f"sharing_purpose must be one of {sorted(VALID_SHARING_PURPOSES)}, "
f"got '{entry.sharing_purpose}'"
)

# Rule 9: notes length
if len(entry.sharing_notes) > NOTES_MAX_LENGTH:
errors.append(
f"sharing_notes must be at most {NOTES_MAX_LENGTH} characters, "
f"got {len(entry.sharing_notes)}"
)

# Warning 1: no sharing_notes provided
if not entry.sharing_notes.strip():
warnings.append(
"sharing_notes is empty — consider documenting the context of this sharing event"
)

# Warning 2: pilot_experiment is highest stakes
if entry.sharing_purpose == "pilot_experiment":
warnings.append(
"sharing_purpose is 'pilot_experiment' — confirm wet-lab partner has reviewed"
" all safety clearance artifacts before proceeding"
)

# Warning 3: collaboration without notes is unusual
if entry.sharing_purpose == "collaboration" and not entry.sharing_notes.strip():
warnings.append(
"sharing_purpose is 'collaboration' with no sharing_notes — document the"
" scope of the collaboration"
)

passed = len(errors) == 0
return ExternalSharingClearanceResult(
esc_id=entry.esc_id,
pep_id=entry.pep_id,
pre_id=entry.pre_id,
sharing_purpose=entry.sharing_purpose,
passed=passed,
errors=errors,
warnings=warnings,
dry_lab_only=entry.dry_lab_only,
)


def validate_external_sharing_clearance_dict(
data: dict,
) -> ExternalSharingClearanceResult:
entry = ExternalSharingClearance(
esc_id=data.get("esc_id", ""),
pipeline_version=data.get("pipeline_version", ""),
pep_id=data.get("pep_id", ""),
pre_id=data.get("pre_id", ""),
external_lab_token=data.get("external_lab_token", ""),
sharing_date=data.get("sharing_date", ""),
caveat_confirmed=bool(data.get("caveat_confirmed", False)),
dry_lab_only_acknowledged=bool(data.get("dry_lab_only_acknowledged", False)),
sharing_purpose=data.get("sharing_purpose", ""),
sharing_notes=data.get("sharing_notes", ""),
reviewer=data.get("reviewer", ""),
dry_lab_only=bool(data.get("dry_lab_only", True)),
)
return validate_external_sharing_clearance(entry)
Loading
Loading