diff --git a/docs/evidence/GUIDE_TUTORIAL_CLEANUP.md b/docs/evidence/GUIDE_TUTORIAL_CLEANUP.md new file mode 100644 index 00000000..48e35edb --- /dev/null +++ b/docs/evidence/GUIDE_TUTORIAL_CLEANUP.md @@ -0,0 +1,21 @@ +# Tutorial Cleanup Guidance + +Each tutorial should include cleanup instructions. + +## Format +```markdown +## Cleanup + +To remove the files created in this tutorial: + +```bash +rm -rf outputs/demo_ranked.jsonl outputs/evidence/ outputs/demo_report.md +``` + +If you want to remove the entire project: + +```bash +cd .. +rm -rf OpenAMP-Foundry +``` +``` diff --git a/docs/evidence/STANDARD_TUTORIAL_EXPECTED_FILES.md b/docs/evidence/STANDARD_TUTORIAL_EXPECTED_FILES.md new file mode 100644 index 00000000..ce5c9e8e --- /dev/null +++ b/docs/evidence/STANDARD_TUTORIAL_EXPECTED_FILES.md @@ -0,0 +1,15 @@ +# Tutorial Expected Files Section Standard + +Each tutorial should include a section listing the files the reader should +have after completing the tutorial. + +## Format +```markdown +## Expected Files + +After completing this tutorial, you should have: + +- `outputs/demo_ranked.jsonl` — ranked candidate list +- `outputs/evidence/` — evidence certificates for selected candidates +- `outputs/demo_report.md` — human-readable report +``` diff --git a/docs/evidence/TEMPLATE_TUTORIAL_SAFETY_NOTE.md b/docs/evidence/TEMPLATE_TUTORIAL_SAFETY_NOTE.md new file mode 100644 index 00000000..9a291f77 --- /dev/null +++ b/docs/evidence/TEMPLATE_TUTORIAL_SAFETY_NOTE.md @@ -0,0 +1,11 @@ +# Tutorial Safety Boundary Note Template + +Use this template for tutorials that touch safety-adjacent topics. + +```markdown +> **Safety Note:** This tutorial demonstrates computational scoring only. +> No biological activity has been demonstrated. The scores shown are +> predictions, not proof of antimicrobial efficacy. Do not use these +> results for clinical, safety, or release decisions without qualified +> human review and wet-lab validation. +``` diff --git a/docs/evidence/TEMPLATE_TUTORIAL_TROUBLESHOOTING.md b/docs/evidence/TEMPLATE_TUTORIAL_TROUBLESHOOTING.md new file mode 100644 index 00000000..ae76ea0d --- /dev/null +++ b/docs/evidence/TEMPLATE_TUTORIAL_TROUBLESHOOTING.md @@ -0,0 +1,16 @@ +# Tutorial Troubleshooting Appendix Template + +```markdown +## Troubleshooting + +### `command not found: openamp-foundry` +Make sure you've installed the package: `pip install -e ".[dev]"` + +### `ModuleNotFoundError: No module named 'openamp_foundry'` +Make sure the virtual environment is activated and PYTHONPATH is set: +`source .venv/bin/activate && export PYTHONPATH=src` + +### Tests fail +Check your Python version: `python3 --version` (requires 3.11+) +If tests still fail, open an issue. +``` diff --git a/docs/evidence/TUTORIAL_INDEX_BY_LEVEL.md b/docs/evidence/TUTORIAL_INDEX_BY_LEVEL.md new file mode 100644 index 00000000..9089163c --- /dev/null +++ b/docs/evidence/TUTORIAL_INDEX_BY_LEVEL.md @@ -0,0 +1,8 @@ +# Tutorial Index by Reader Level + +| Level | Reader | Recommended Tutorials | +|-------|--------|---------------------| +| Beginner | First-time contributor | QUICKSTART.md, first contribution decision tree | +| Intermediate | Regular contributor | ARCHITECTURE.md, REVIEWER_CURRICULUM.md | +| Advanced | Maintainer | Calibration audit walkthrough, release dry-run walkthrough | +| Agent | AI agent | AGENTS.md, AGENT_TASKS.json, AGENT_CURRICULUM.md | diff --git a/docs/evidence/TUTORIAL_PREREQS_CHECKLIST.md b/docs/evidence/TUTORIAL_PREREQS_CHECKLIST.md new file mode 100644 index 00000000..881c3e5f --- /dev/null +++ b/docs/evidence/TUTORIAL_PREREQS_CHECKLIST.md @@ -0,0 +1,10 @@ +# Tutorial Prerequisites Checklist + +Before starting a tutorial, ensure: + +- [ ] Python 3.11+ is installed +- [ ] `git` is installed +- [ ] A virtual environment is created: `python3 -m venv .venv` +- [ ] Dependencies are installed: `pip install -e ".[dev]"` +- [ ] The tutorial's estimated time is accurate +- [ ] All commands in the tutorial work from a clean checkout diff --git a/docs/evidence/WALKTHROUGH_ARTIFACT_VALIDATION.md b/docs/evidence/WALKTHROUGH_ARTIFACT_VALIDATION.md new file mode 100644 index 00000000..d7e734e0 --- /dev/null +++ b/docs/evidence/WALKTHROUGH_ARTIFACT_VALIDATION.md @@ -0,0 +1,13 @@ +# Guided Walkthrough: Artifact Validation + +## Scenario +You've received lab result JSON files from a partner. Validate them. + +## Steps +1. Place the JSON files in a single directory. +2. Run: `python scripts/validate_lab_data_return.py --results-dir ` +3. Check the output: + - "Passed: N" — all files passed schema validation + - "Failed: M" — some files have errors (check the error messages) +4. If there are control failures, check the original assay data. +5. If there are schema validation errors, ask the partner to fix and resubmit. diff --git a/docs/evidence/WALKTHROUGH_BENCHMARK_REVIEW.md b/docs/evidence/WALKTHROUGH_BENCHMARK_REVIEW.md new file mode 100644 index 00000000..a04a1281 --- /dev/null +++ b/docs/evidence/WALKTHROUGH_BENCHMARK_REVIEW.md @@ -0,0 +1,13 @@ +# Guided Walkthrough: Benchmark Report Review + +## Scenario +You need to review a benchmark report to assess pipeline health. + +## Steps +1. Run: `make bench-500` +2. Check the AUROC value in `outputs/validate_scoring_500.json`. +3. Compare with the expected value in METRICS_CURRENT.md. +4. If AUROC dropped > 0.02, check for recent regressions. +5. Run: `make bench-gate` — this checks all regression gates. +6. Review the simulation ablation results in `outputs/simulation_ablation.json`. +7. Check that simulation verdict remains NO_IMPROVEMENT. diff --git a/docs/evidence/WALKTHROUGH_CALIBRATION_AUDIT.md b/docs/evidence/WALKTHROUGH_CALIBRATION_AUDIT.md new file mode 100644 index 00000000..e3b13a20 --- /dev/null +++ b/docs/evidence/WALKTHROUGH_CALIBRATION_AUDIT.md @@ -0,0 +1,13 @@ +# Guided Walkthrough: Calibration Audit Reports + +## Scenario +You need to audit the calibration pipeline after receiving lab results. + +## Steps +1. Place lab result JSONs in a directory. +2. Run: `openamp-foundry calibration-intake --predictions --results-dir --panel-name wave1` +3. Check the intake report for matched and orphan results. +4. Run: `openamp-foundry recalibration-gate --intake --policy configs/recalibration_policy.yaml` +5. Check the gate verdict: 0 = recalibration allowed, 3 = blocked. +6. If blocked, review which conditions failed. +7. Document the audit results in the decision log. diff --git a/docs/evidence/WALKTHROUGH_REVIEW_PACKETS.md b/docs/evidence/WALKTHROUGH_REVIEW_PACKETS.md new file mode 100644 index 00000000..154d56a0 --- /dev/null +++ b/docs/evidence/WALKTHROUGH_REVIEW_PACKETS.md @@ -0,0 +1,13 @@ +# Guided Walkthrough: Review Packets + +## Scenario +You need to prepare a review packet for an external expert. + +## Steps +1. Collect the evidence certificates for selected candidates. +2. Run: `make lab-batch-pack` to create a zip with all artifacts. +3. Include LIMITATIONS_OVERVIEW.md for honest caveats. +4. Include METRICS_CURRENT.md for benchmark context. +5. Include the specific claim or question being reviewed. +6. Run `python scripts/safety/check_claims.py` on all documents. +7. Send the packet with a clear description of what feedback is needed. diff --git a/tests/test_tutorials_batch.py b/tests/test_tutorials_batch.py new file mode 100644 index 00000000..e085b6e4 --- /dev/null +++ b/tests/test_tutorials_batch.py @@ -0,0 +1,20 @@ +"""Tests for tutorials and walkthroughs batch.""" +from pathlib import Path + +FILES = [ + "TUTORIAL_INDEX_BY_LEVEL.md", + "TUTORIAL_PREREQS_CHECKLIST.md", + "STANDARD_TUTORIAL_EXPECTED_FILES.md", + "GUIDE_TUTORIAL_CLEANUP.md", + "TEMPLATE_TUTORIAL_SAFETY_NOTE.md", + "TEMPLATE_TUTORIAL_TROUBLESHOOTING.md", + "WALKTHROUGH_ARTIFACT_VALIDATION.md", + "WALKTHROUGH_BENCHMARK_REVIEW.md", + "WALKTHROUGH_REVIEW_PACKETS.md", + "WALKTHROUGH_CALIBRATION_AUDIT.md", +] + + +def test_all_exist(): + for f in FILES: + assert Path("docs/evidence", f).exists(), f"Missing: {f}"