Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 11 additions & 0 deletions docs/evidence/DECISION_TREE_BENCHMARK_CHANGE.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
# Decision Tree: Benchmark Change Review

```mermaid
flowchart TD
A[Benchmark change proposed] --> B{Does it change AUROC?}
B -->|Yes, improvement| C[Verify no overfitting]
B -->|Yes, regression| D[Is regression > 0.02?]
D -->|Yes| E[Block — investigate]
D -->|No| F[Accept — document tradeoff]
B -->|No change| G[Verify benchmark still runs]
```
12 changes: 12 additions & 0 deletions docs/evidence/DECISION_TREE_EVIDENCE_DOWNGRADE.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
# Decision Tree: Evidence Downgrade

```mermaid
flowchart TD
A[Evidence quality concern] --> B{Is the evidence reproducible?}
B -->|No| C[Downgrade 1 level]
B -->|Yes| D{Are the caveats documented?}
D -->|No| E[Downgrade 1 level]
D -->|Yes| F{Is the data source reliable?}
F -->|No| G[Downgrade 1 level]
F -->|Yes| H[Maintain current level]
```
12 changes: 12 additions & 0 deletions docs/evidence/DECISION_TREE_FIRST_CONTRIBUTION.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
# Decision Tree: Choosing First Contribution

```mermaid
flowchart TD
A[Want to contribute?] --> B{Can you run Python?}
B -->|Yes| C{Can you write tests?}
B -->|No| D[Start with docs-only tasks]
C -->|Yes| E[Pick a test issue]
C -->|No| F{Can you write docs?}
F -->|Yes| G[Pick a docs issue]
F -->|No| H[Start with CONTRIBUTOR_CURRICULUM.md]
```
12 changes: 12 additions & 0 deletions docs/evidence/DECISION_TREE_OPENING_DOCS_PR.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
# Decision Tree: Opening a Documentation PR

```mermaid
flowchart TD
A[Change needed in docs] --> B{Is it a typo or formatting?}
B -->|Yes| C[Direct commit or minor PR]
B -->|No| D{Is it a policy change?}
D -->|Yes| E[Full PR + maintainer review]
D -->|No| F{Is it a new document?}
F -->|Yes| G[PR + link from PROJECT_INDEX.md]
F -->|No| H[Standard PR with review]
```
12 changes: 12 additions & 0 deletions docs/evidence/DECISION_TREE_PUBLIC_SHARING.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
# Decision Tree: Public Artifact Sharing

```mermaid
flowchart TD
A[Share artifact publicly?] --> B{Contains sensitive data?}
B -->|Yes| C[Redact or don't share]
B -->|No| D{Is it a release artifact?}
D -->|Yes| E[Follow PUBLICATION_PACK.md]
D -->|No| F{Is it a benchmark result?}
F -->|Yes| G[Include caveats]
F -->|No| H[Share with standard disclaimer]
```
16 changes: 16 additions & 0 deletions docs/evidence/GUIDE_RELEASE_READINESS.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
# Decision Guide: Release Readiness

Before releasing, verify:

- [ ] `make full-reproducibility-report` succeeds
- [ ] `python scripts/check_doc_links.py` — 0 broken links
- [ ] `python scripts/safety/check_claims.py` — 0 overclaiming
- [ ] `python3 -m pytest -q` — all tests pass
- [ ] METRICS_CURRENT.md is up to date
- [ ] PUBLICATION_PACK.md checklist is complete
- [ ] Decision log is current

## Decision
- All checks pass → Release candidate
- Minor issues → Fix before release
- Major issues → Block release
9 changes: 9 additions & 0 deletions docs/evidence/RUNBOOK_BROKEN_DOCS_LINKS.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
# Maintainer Runbook: Broken Docs Links

When doc links are broken:

1. Run `python scripts/check_doc_links.py` to find all broken links.
2. For each broken link, determine the correct target path.
3. Update the link in the source document.
4. If the target document was moved, add a redirect or update all references.
5. Re-run the check to confirm 0 broken links.
10 changes: 10 additions & 0 deletions docs/evidence/RUNBOOK_STALE_ARTIFACTS.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
# Maintainer Runbook: Stale Artifacts

When an artifact is stale:

1. Identify what's stale and why.
2. If it's a benchmark output, regenerate with `make bench-*`.
3. If it's a schema, update it to match current code.
4. If it's a config, update or document the discrepancy.
5. Update references in docs.
6. Add a test to prevent future staleness.
10 changes: 10 additions & 0 deletions docs/evidence/RUNBOOK_UNSAFE_CLAIMS.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
# Maintainer Runbook: Unsafe Claim Wording

When unsafe claim wording is found:

1. Run `python scripts/safety/check_claims.py` to see all flagged phrases.
2. Review each flagged phrase in context.
3. Rewrite overclaiming phrases using project-allowed terms.
4. If the claim is in a public-facing document, prioritize the fix.
5. Re-run the check to confirm the fix.
6. If the author disagrees, escalate to maintainers.
16 changes: 16 additions & 0 deletions docs/evidence/WALKTHROUGH_CLAIM_TO_EVIDENCE.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
# Guided Walkthrough: Claim-to-Evidence Matrices

How to map a claim to its supporting evidence.

## Steps
1. State the claim clearly — what exactly is being asserted?
2. Identify the proof level required (see PROOF_LADDER.md).
3. Find the evidence that supports the claim.
4. Check if the evidence level matches the claim strength.
5. If the claim is stronger than the evidence, weaken the claim or gather more evidence.

## Example
| Claim | Required Level | Available Evidence | Match? |
|-------|:--------------:|-------------------|:------:|
| "Candidate X is predicted antimicrobial" | 2 (baseline score) | Ensemble score 0.85 | ✅ |
| "Candidate X is a drug candidate" | 5 (lab assay) | No wet-lab data | ❌ |
20 changes: 20 additions & 0 deletions tests/test_decision_trees.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
"""Tests for batch of 10 docs."""
from pathlib import Path

FILES = [
"WALKTHROUGH_CLAIM_TO_EVIDENCE.md",
"DECISION_TREE_FIRST_CONTRIBUTION.md",
"DECISION_TREE_OPENING_DOCS_PR.md",
"DECISION_TREE_BENCHMARK_CHANGE.md",
"GUIDE_RELEASE_READINESS.md",
"DECISION_TREE_PUBLIC_SHARING.md",
"DECISION_TREE_EVIDENCE_DOWNGRADE.md",
"RUNBOOK_STALE_ARTIFACTS.md",
"RUNBOOK_BROKEN_DOCS_LINKS.md",
"RUNBOOK_UNSAFE_CLAIMS.md",
]


def test_all_exist():
for f in FILES:
assert Path("docs/evidence", f).exists(), f"Missing: {f}"
Loading