Repository navigation
docs(sprint1): METRICS_CURRENT.md + CI stale-metric detector + breaking news cleanup - #106
Merged
Merged
Conversation
…reaking news cleanup Sprint 1 — Trust cleanup: 1. docs/METRICS_CURRENT.md: Single source of truth for all current pipeline metrics (AUROC 0.7832, n=191, phase3 AUROC 0.7448, etc.). Includes historical baselines, candidate panel summary, external predictor status, test suite stats, and limitations. 2. tests/test_docs_consistent.py: CI stale-metric detector — 5 tests that fail if: - Any doc uses stale AUROC in current context (not historical) - METRICS_CURRENT.md is missing or inconsistent - "breaking news" terminology remains (must be "high-impact scenario") - Overconfident 92-97% claims appear without honesty correction - WET_LAB_PROBABILITY.md lacks honest lower-bound estimate 3. Residual stale refs fixed across 5 docs (ROADMAP.md, DISCOVERY_PREDICTION.md, WET_LAB_PROBABILITY.md, WAVE2_PLAN.md, DISCOVERY_PREDICTION.md) - All "breaking news" → "high-impact scenario" (including headers and sidebars) - All stale 92-97% table entries with honest corrections - ROADMAP.md v0.4 entry qualified as "original demo set" Total: 1343 tests (5 new doc consistency tests). Co-authored-by: CommandCodeBot <noreply@commandcode.ai>
…g note Three HIGH issues fixed: 1. Hyphenated "breaking-news" in DISCOVERY_PREDICTION.md (2 occurrences) 2. Test regex updated to catch both space-separated and hyphenated forms 3. Tracking note self-reference fixed Co-authored-by: CommandCodeBot <noreply@commandcode.ai>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Sprint 1 — Trust cleanup
3 deliverables:
1. docs/METRICS_CURRENT.md
Single source of truth for current pipeline metrics:
2. CI stale-metric detector (tests/test_docs_consistent.py)
5 tests that auto-fail on stale docs:
3. Residual stale refs
All remaining 'breaking news' → 'high-impact scenario' across 5 docs.
All stale 92-97% table entries updated with honest corrections.
Tests: 1343 total (+5 doc consistency tests)