Repository navigation
docs: consolidate all benchmark and probability references after PR #110 - #105
Merged
Merged
Conversation
Fixes 6 items identified in external review: 1. METHODS.md: Benchmark section rewritten — expanded 95+96 (AUROC 0.7832) as primary reference; original demo set (0.8420) as historical baseline; PR timeline updated with PR #110 entry; limitation updated. 2. All stale AUROC 0.8420 "current" refs replaced: every current-state reference now says "expanded 95+96 benchmark, AUROC 0.7832". The 0.8420 value is only retained as "original demo set" historical context. 3. WET_LAB_PROBABILITY.md: Toned down "92-97%" to "upper bound; honest estimate 55-80%". Summary table updated with honest corrections. Publishable novel result lowered to 8-20%, high-impact to 2-8%. 4. "breaking news" → "high-impact scenario" across DISCOVERY_PREDICTION.md, WET_LAB_PROBABILITY.md, EXPERT_REVIEW_PACK.md. 5. NOVELTY_CHECKLIST.md: New doc — step-by-step external novelty check (APD3, DRAMP, dbAMP, UniProt BLASTp, patent search, PDB, PubMed). 6. REVIEWER_SUMMARY.md: New 2-page reviewer summary with pipeline overview, benchmark, panel table, assay protocol, limitations, verdict template. 7. Duplicated paragraph removed from WET_LAB_PROBABILITY.md disclaimer. EXPERT_REVIEW_PACK.md cross-references updated. Tests: 1338 passing. Co-authored-by: CommandCodeBot <noreply@commandcode.ai>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What this fixes
Six documentation issues identified in external review:
1. METHODS.md rewritten — benchmark section now references the expanded 95+96 (AUROC 0.7832) as primary. Original demo set (0.8420) retained as historical baseline. PR #110 entry added to timeline.
2. All stale AUROC 0.8420 'current' refs replaced — every current-state reference now says the expanded 95+96 benchmark value. The 0.8420 value only remains as 'original demo set' historical context. Affected: WET_LAB_PROBABILITY.md, DISCOVERY_PREDICTION.md, EXPERT_REVIEW_PACK.md, GOLD_STANDARD_CALIBRATION.md.
3. WET_LAB_PROBABILITY.md toned down — '92–97%' replaced with 'upper bound; honest estimate: 55–80%'. Summary table reflects honest corrections (publishable: 8–20%, high-impact: 2–8%). No misleading overconfidence.
4. 'breaking news' → 'high-impact scenario' across all docs.
5. NOVELTY_CHECKLIST.md — new doc with step-by-step external novelty verification: APD3, DRAMP, dbAMP, UniProt BLASTp, patent search (Google Patents, WIPO, USPTO), PDB structure search, PubMed literature check. Results table template included.
6. REVIEWER_SUMMARY.md — new 2-page reviewer summary: pipeline overview, benchmark table, 20-candidate/7-family panel, recommended assay protocol, safety precautions, known limitations, verdict template.
Tests: 1338 passing.