diff --git a/docs/evidence/ARCHITECTURE_OVERVIEW.md b/docs/evidence/ARCHITECTURE_OVERVIEW.md new file mode 100644 index 00000000..925daece --- /dev/null +++ b/docs/evidence/ARCHITECTURE_OVERVIEW.md @@ -0,0 +1,29 @@ +# Architecture Overview for Contributors + +## High-Level Structure +``` +src/openamp_foundry/ — pipeline code + cli/ — command-line interface + pipeline.py — main scoring pipeline + scoring/ — individual scorers + features/ — feature extraction + simulation/ — virtual assay modules + calibration/ — recalibration engine + evidence/ — certificate generation +tests/ — test suite +docs/ — documentation +scripts/ — utility scripts +configs/ — configuration files +schemas/ — artifact schemas +``` + +## Key Files +- `src/openamp_foundry/pipeline.py` — main scoring pipeline +- `src/openamp_foundry/cli/main.py` — CLI entry points +- `configs/pipeline.yaml` — scoring weights and filters +- `docs/evidence/METRICS_CURRENT.md` — current benchmark metrics + +## Data Flow +1. CLI parses arguments and calls `pipeline.run_ranking_pipeline()`. +2. Pipeline loads candidates, scores them, and writes outputs. +3. Outputs include ranked JSONL, evidence certificates, and reports. diff --git a/docs/evidence/DIAGRAM_ARTIFACT_FLOW.md b/docs/evidence/DIAGRAM_ARTIFACT_FLOW.md new file mode 100644 index 00000000..ac4b2cbf --- /dev/null +++ b/docs/evidence/DIAGRAM_ARTIFACT_FLOW.md @@ -0,0 +1,19 @@ +# End-to-End Artifact Flow Diagram + +```mermaid +flowchart LR + A[Candidate CSV] --> B[score_candidates] + B --> C[Ranked JSONL] + B --> D[Evidence certs] + C --> E[Batch report] + C --> F[Run manifest] + D --> G[Lab batch pack] + G --> H[Lab results] + H --> I[calibration-intake] + I --> J[recalibration-gate] + J -->|pass| K[Weight proposal] + J -->|fail| L[Blocked] +``` + +This diagram shows the flow from candidate input through scoring, reporting, +lab partnership, and calibration. diff --git a/docs/evidence/DIAGRAM_DATA_FLOW.md b/docs/evidence/DIAGRAM_DATA_FLOW.md new file mode 100644 index 00000000..cf7d7ec7 --- /dev/null +++ b/docs/evidence/DIAGRAM_DATA_FLOW.md @@ -0,0 +1,36 @@ +# Data Flow and Data Boundary Map + +```mermaid +flowchart LR + subgraph Input + A[Candidate CSV] + B[Reference CSV] + C[Config YAML] + end + subgraph Pipeline + D[score_candidates] + E[rank_candidates] + F[select_diverse] + end + subgraph Output + G[Ranked JSONL] + H[Evidence certs] + I[Run manifest] + J[Batch report] + end + A --> D + B --> D + C --> D + D --> E + E --> F + F --> G + D --> H + D --> I + E --> J +``` + +## Data Boundaries +- **Input boundary**: Candidate CSV, reference CSV, config YAML +- **Pipeline boundary**: `score_candidates` → `rank_candidates` → `select_diverse` +- **Output boundary**: JSONL, evidence certs, manifest, report +- **External boundary**: Lab results (returned by partners) diff --git a/docs/evidence/DIAGRAM_DEPENDENCY_GRAPH.md b/docs/evidence/DIAGRAM_DEPENDENCY_GRAPH.md new file mode 100644 index 00000000..f39c5ec7 --- /dev/null +++ b/docs/evidence/DIAGRAM_DEPENDENCY_GRAPH.md @@ -0,0 +1,22 @@ +# Dependency Graph Explanation + +How pipeline components depend on each other. + +```mermaid +flowchart TD + CLI --> Pipeline + Pipeline --> Features + Pipeline --> Scoring + Pipeline --> Evidence + Scoring --> Features + Scoring --> Ensemble + Simulation --> Scoring + Calibration --> Pipeline +``` + +## Description +- **CLI** depends on **Pipeline** (calls `run_ranking_pipeline`) +- **Pipeline** depends on **Features** (feature extraction), **Scoring** (individual scorers), and **Evidence** (certificate generation) +- **Scoring** depends on **Features** (physicochemical features) and **Ensemble** (weighted combination) +- **Simulation** feeds into **Scoring** (via `--simulation-mode info`) +- **Calibration** feeds back into **Pipeline** (weight updates after human review) diff --git a/docs/evidence/GLOSSARY_BEGINNER_PATH.md b/docs/evidence/GLOSSARY_BEGINNER_PATH.md new file mode 100644 index 00000000..fd7593b8 --- /dev/null +++ b/docs/evidence/GLOSSARY_BEGINNER_PATH.md @@ -0,0 +1,14 @@ +# Beginner Glossary Path + +Recommended order to learn project terminology. + +1. Candidate — a peptide sequence being evaluated +2. Score — a numerical prediction of a property +3. Ensemble — weighted combination of individual scores +4. Benchmark — a test of pipeline performance against known data +5. Gate — a safety check that blocks or allows an action +6. Calibration — adjusting scores based on real data +7. Evidence certificate — machine-readable record of a candidate's scores +8. Manifest — record of inputs and outputs for a pipeline run +9. Ablation — removing a component to measure its contribution +10. Concept card — short explanation of a single concept diff --git a/docs/evidence/GUIDE_ARTIFACT_LIFECYCLE.md b/docs/evidence/GUIDE_ARTIFACT_LIFECYCLE.md new file mode 100644 index 00000000..20366467 --- /dev/null +++ b/docs/evidence/GUIDE_ARTIFACT_LIFECYCLE.md @@ -0,0 +1,20 @@ +# Artifact Lifecycle Guide + +The lifecycle of a pipeline artifact from creation to archive. + +## Stages +1. **Created** — generated by a pipeline run or script +2. **Validated** — checked against its schema +3. **Used** — consumed by downstream processes (reports, calibration) +4. **Reviewed** — included in a review packet or release +5. **Archived** — stored for historical reference after its active use + +## Rules +- Every artifact should be validated at creation time. +- Artifacts should not be modified after creation (immutable). +- If an artifact needs to change, create a new version. +- Artifacts used in releases should be archived with the release. + +## Related +- `docs/evidence/ARTIFACT_MAP.md` — artifact catalog +- `docs/evidence/STANDARD_GENERATED_FILE_WARNING.md` — generated file warnings diff --git a/docs/evidence/GUIDE_MAKE_TARGETS.md b/docs/evidence/GUIDE_MAKE_TARGETS.md new file mode 100644 index 00000000..8583cd92 --- /dev/null +++ b/docs/evidence/GUIDE_MAKE_TARGETS.md @@ -0,0 +1,20 @@ +# Make Target Guide + +How to use Makefile targets in the project. + +## Common Targets +| Target | Purpose | +|--------|---------| +| `make demo` | Run the demo pipeline | +| `make test` | Run all tests | +| `make bench-500` | Run 500-AMP benchmark | +| `make bench-gate` | Check benchmark regression gates | +| `make regenerate-all` | Regenerate all outputs | +| `make full-reproducibility-report` | Generate release report | +| `make lab-batch-pack` | Build lab partner zip | + +## Adding a New Target +1. Add the target to the Makefile. +2. Add it to the `.PHONY` list if it doesn't create a file. +3. Add a description to the help section. +4. Document the target in COMMAND_SURFACE.md if it's user-facing. diff --git a/docs/evidence/GUIDE_QUICKSTART_DIAGNOSIS.md b/docs/evidence/GUIDE_QUICKSTART_DIAGNOSIS.md new file mode 100644 index 00000000..498ab872 --- /dev/null +++ b/docs/evidence/GUIDE_QUICKSTART_DIAGNOSIS.md @@ -0,0 +1,12 @@ +# Failed Quickstart Diagnosis Table + +Common quickstart failures and how to fix them. + +| Symptom | Likely Cause | Solution | +|---------|-------------|----------| +| `command not found: openamp-foundry` | Package not installed | `pip install -e ".[dev]"` | +| `ModuleNotFoundError: No module named 'openamp_foundry'` | PYTHONPATH not set | `export PYTHONPATH=src` | +| `FileNotFoundError: examples/...` | Wrong working directory | Run from repo root | +| Tests fail | Wrong Python version | `python3 --version` (requires 3.11) | +| `make demo` produces no output | Previous run cleaned up | Re-run `make demo` | +| Git error | Not in a git repo | `git clone` first | diff --git a/docs/evidence/GUIDE_QUICKSTART_OUTPUT_GALLERY.md b/docs/evidence/GUIDE_QUICKSTART_OUTPUT_GALLERY.md new file mode 100644 index 00000000..b39219f1 --- /dev/null +++ b/docs/evidence/GUIDE_QUICKSTART_OUTPUT_GALLERY.md @@ -0,0 +1,21 @@ +# Expected Output Gallery for Quickstart + +After running `make demo`, you should see these outputs. + +## Files Created +``` +outputs/demo_ranked.jsonl — ranked candidate list +outputs/demo_report.md — human-readable report +outputs/evidence/ — evidence certificates directory +outputs/evidence/AMPF-*.json — one certificate per candidate +outputs/run_manifest.json — run manifest +``` + +## Sample Output +The first line of `outputs/demo_ranked.jsonl` should contain a JSON object +with `candidate_id`, `sequence`, `scores`, and `features` fields. + +## If Output Differs +- Check that `make demo` completed without errors. +- Check that the input files exist at `examples/sequences/demo_candidates.csv`. +- Check that the reference file exists at `examples/known_reference/demo_known_amps.csv`. diff --git a/docs/evidence/MAP_COMMAND_TO_ARTIFACT.md b/docs/evidence/MAP_COMMAND_TO_ARTIFACT.md new file mode 100644 index 00000000..99516c47 --- /dev/null +++ b/docs/evidence/MAP_COMMAND_TO_ARTIFACT.md @@ -0,0 +1,15 @@ +# Command-to-Artifact Map + +Maps CLI commands to their output artifacts. + +| Command | Output Artifact | Format | Location | +|---------|----------------|:-----:|----------| +| `rank` | Ranked candidates | JSONL | `outputs/*_ranked.jsonl` | +| `rank` | Evidence certificates | JSON | `outputs/evidence/` | +| `rank` | Run manifest | JSON | `outputs/run_manifest.json` | +| `rank --report` | Batch report | MD + JSON | `outputs/*_report.md` | +| `validate` | Validation result | stdout | — | +| `pilot-panel` | Pilot panel CSV | CSV | `outputs/*_panel.csv` | +| `lab-result-report` | Lab result report | JSON | `outputs/*_report.json` | +| `calibration-intake` | Intake report | JSON | `outputs/*_intake.json` | +| `recalibration-gate` | Gate verdict | JSON | `outputs/*_verdict.json` | diff --git a/docs/evidence/POLICY_CLI_REFERENCE_GENERATION.md b/docs/evidence/POLICY_CLI_REFERENCE_GENERATION.md new file mode 100644 index 00000000..b6a2ee51 --- /dev/null +++ b/docs/evidence/POLICY_CLI_REFERENCE_GENERATION.md @@ -0,0 +1,14 @@ +# CLI Reference Generation Policy + +CLI reference docs should be generated from the code, not written manually. + +## Current State +CLI reference is maintained in docs/getting-started/COMMAND_SURFACE.md. + +## Policy +- Add new commands to COMMAND_SURFACE.md when they are added. +- Update COMMAND_SURFACE.md when command flags change. +- Consider auto-generating the reference from argparse in the future. + +## Related +- `docs/getting-started/COMMAND_SURFACE.md` — CLI reference diff --git a/tests/test_arch_quickstart_batch.py b/tests/test_arch_quickstart_batch.py new file mode 100644 index 00000000..efee261e --- /dev/null +++ b/tests/test_arch_quickstart_batch.py @@ -0,0 +1,21 @@ +"""Tests for architecture and quickstart batch.""" +from pathlib import Path + +FILES = [ + "POLICY_CLI_REFERENCE_GENERATION.md", + "GUIDE_MAKE_TARGETS.md", + "GUIDE_QUICKSTART_OUTPUT_GALLERY.md", + "GUIDE_QUICKSTART_DIAGNOSIS.md", + "GLOSSARY_BEGINNER_PATH.md", + "DIAGRAM_ARTIFACT_FLOW.md", + "ARCHITECTURE_OVERVIEW.md", + "GUIDE_ARTIFACT_LIFECYCLE.md", + "DIAGRAM_DATA_FLOW.md", + "MAP_COMMAND_TO_ARTIFACT.md", + "DIAGRAM_DEPENDENCY_GRAPH.md", +] + + +def test_all_exist(): + for f in FILES: + assert Path("docs/evidence", f).exists(), f"Missing: {f}"