Skip to content

feat: Phase E E5 pilot pre-registration schema -- PRR- record freezes selection thresholds before wet-lab, prevents post-hoc adjustment, 63 tests - #896

Merged
cschanhniem merged 1 commit into
mainfrom
feat/e5-pilot-preregistration
Jul 10, 2026
Merged

cschanhniem merged 1 commit into
mainfrom
feat/e5-pilot-preregistration

Conversation

@cschanhniem

Copy link
Copy Markdown
Collaborator

Implements E5 from NEXT_100_PR_MAP: non-protocol pilot pre-registration schema.

What

  • pilot_preregistration.py: PilotPreregistration + ScoreThreshold + PreregistrationValidationResult
  • validate_pilot_preregistration(): 14 validation rules
  • format_pilot_preregistration(): human-readable output with VALID/INVALID/NOTICE
  • VALID_OUTCOME_METRICS: 8 controlled vocabulary terms
  • VALID_AMENDMENT_REASONS: 5 controlled terms
  • Key gates: dry_lab_only_declaration=True required, score thresholds declared before experiment, positive/negative controls required
  • 63 tests; BASELINE 7402→7465

Why

Post-hoc threshold adjustment is the most common form of silent overclaiming in computational-to-experimental pipelines. A locked PRR- record with frozen thresholds is the contract between dry-lab selection and wet-lab execution. Without pre-registration, "we selected candidates with score > 0.7" can become "score > 0.5 (after checking results)" with no audit trail.

Disconfirming pass

  • Threshold values are frozen at pre-registration time; amendment_reasons vocabulary limits what can be changed and why
  • Locked records with amendments produce a warning (surfaces amendment history, doesn't block)
  • No safety/release policy changed; no calibration policy touched

… selection thresholds before wet-lab, prevents post-hoc adjustment, 63 tests (#896)
@cschanhniem
cschanhniem merged commit 79d6266 into main Jul 10, 2026
1 check failed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant