Skip to content

feat: Phase O O2 prediction drift monitor schema -- earliest warning signal for pipeline score distribution shifts - #835

Merged
cschanhniem merged 1 commit into
mainfrom
feat/o2-prediction-drift-monitor
Jul 9, 2026
Merged

cschanhniem merged 1 commit into
mainfrom
feat/o2-prediction-drift-monitor

Conversation

@cschanhniem

Copy link
Copy Markdown
Collaborator

Summary

  • PredictionDriftEntry: 15 fields tracking mean/std score shifts between reference and evaluation batches
  • 8 validation rules: monitor_id prefix, batch id uniqueness, score bounds, std non-negative, shift magnitude consistency, population >= 1, notes length, notes required when drift_flag=True
  • 4 warning conditions: unreported large shift, small populations, variance explosion
  • 68 tests; BASELINE 4549→4617
  • CLI: openamp-foundry prediction-drift-check

Test plan

  • make agent-check passes
  • pytest tests/evidence/test_prediction_drift_monitor.py passes (68 tests)
  • BASELINE updated 4549→4617

…signal for pipeline score distribution shifts
@cschanhniem
cschanhniem merged commit 4f5ea0f into main Jul 9, 2026
1 check failed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant