diff --git a/docs/evidence/ARCHITECTURE_OVERVIEW_CONTRIBUTORS.md b/docs/evidence/ARCHITECTURE_OVERVIEW_CONTRIBUTORS.md new file mode 100644 index 00000000..7076a4f7 --- /dev/null +++ b/docs/evidence/ARCHITECTURE_OVERVIEW_CONTRIBUTORS.md @@ -0,0 +1,27 @@ +# Architecture Overview for Contributors + +A quick tour of the codebase structure. + +``` +src/openamp_foundry/ # Pipeline code + cli/ # CLI commands (rank, validate, bench, etc.) + pipeline.py # Main scoring pipeline + scoring/ # Individual scorers (activity, safety, etc.) + features/ # Physicochemical feature extraction + simulation/ # Virtual assay modules (membrane, structure) + calibration/ # Recalibration engine and gate + evidence/ # Evidence certificate generation + benchmark/ # Benchmark infrastructure +tests/ # Test suite + benchmarks/ # Benchmark regression tests + calibration/ # Calibration tests + lab/ # Lab handoff tests +configs/ # Configuration files (pipeline.yaml, etc.) +schemas/ # JSON schemas for artifacts +scripts/ # Utility scripts +docs/ # Documentation +``` + +## Entry Points +- `src/openamp_foundry/cli/main.py` — CLI entry points. +- `src/openamp_foundry/pipeline.py` — `run_ranking_pipeline()` is the main function. diff --git a/docs/evidence/CATALOG_AGENT_ISSUES.md b/docs/evidence/CATALOG_AGENT_ISSUES.md new file mode 100644 index 00000000..655d1074 --- /dev/null +++ b/docs/evidence/CATALOG_AGENT_ISSUES.md @@ -0,0 +1,21 @@ +# Agent-Safe Issue Examples Catalog + +Example issues that are safe for AI agents to work on. + +## Docs +- Fix broken links in a document +- Add missing concept card for an artifact type +- Update QUICKSTART.md for a CLI change +- Add a test for a documentation file +- Create a runbook for a contributor workflow + +## Tests +- Add missing-file error test for a CLI command +- Add metric polarity test +- Add schema validation test +- Add CLI exit code test + +## Scripts +- Add a new local check script +- Improve pattern list in check_claims.py +- Add a metric naming check diff --git a/docs/evidence/GUIDE_ALLOWED_CLAIMS.md b/docs/evidence/GUIDE_ALLOWED_CLAIMS.md new file mode 100644 index 00000000..eb4809f3 --- /dev/null +++ b/docs/evidence/GUIDE_ALLOWED_CLAIMS.md @@ -0,0 +1,18 @@ +# Allowed Public Claim Examples + +Examples of allowed and forbidden public claims. + +## Allowed +- "Computationally nominated 24 antimicrobial peptide candidates." +- "Pipeline achieves AUROC 0.78 on benchmark — limitations apply." +- "All candidates are dry-lab predictions requiring wet-lab validation." +- "Full reproducible evidence trail available for scientific review." + +## Forbidden +- "Discovered a new antibiotic." (Requires wet-lab validation) +- "Pipeline-validated antimicrobial." (Computation does not validate) +- "Safe, effective peptide therapy." (No clinical evidence) +- "AI drug discovery breakthrough." (Overclaiming) + +## Rule +When in doubt, use the weakest accurate claim. Add a caveat. diff --git a/docs/evidence/GUIDE_DECISION_VALIDATOR.md b/docs/evidence/GUIDE_DECISION_VALIDATOR.md new file mode 100644 index 00000000..27476bdf --- /dev/null +++ b/docs/evidence/GUIDE_DECISION_VALIDATOR.md @@ -0,0 +1,18 @@ +# Decision Record Template Validator + +Validates decision log entries against the schema. + +## Usage +```bash +python -m openamp_foundry.cli validate \ + --certificate decision_log_entry.json \ + --schema schemas/decision_log.schema.json +``` + +## Checks +The schema validates: +- Required fields are present +- Decision type is one of the allowed values +- Decision is approved, rejected, or deferred +- If dissent_flag is true, dissent_notes are required +- Date is in ISO 8601 format diff --git a/docs/evidence/GUIDE_ISSUE_TRIAGE.md b/docs/evidence/GUIDE_ISSUE_TRIAGE.md new file mode 100644 index 00000000..653ab4f1 --- /dev/null +++ b/docs/evidence/GUIDE_ISSUE_TRIAGE.md @@ -0,0 +1,16 @@ +# Issue Triage Matrix + +How to triage incoming issues. + +| Factor | High Priority | Medium Priority | Low Priority | +|--------|:-------------:|:---------------:|:------------:| +| Safety impact | Direct safety concern | Indirect safety concern | No safety impact | +| User impact | Blocks all users | Blocks some users | Minor inconvenience | +| Effort | Small fix (< 1 day) | Medium fix (1-3 days) | Large fix (> 3 days) | +| Alignment | Core mission | Supporting | Nice-to-have | + +## Process +1. Apply appropriate labels (bug, enhancement, docs, safety). +2. Assign priority (P0-P3). +3. If P0, notify maintainers immediately. +4. If good-first-issue, add the label. diff --git a/docs/evidence/GUIDE_PLATFORM_VERIFICATION.md b/docs/evidence/GUIDE_PLATFORM_VERIFICATION.md new file mode 100644 index 00000000..0eb26f1f --- /dev/null +++ b/docs/evidence/GUIDE_PLATFORM_VERIFICATION.md @@ -0,0 +1,15 @@ +# Platform Quickstart Verification Notes + +Quickstart verification results by platform. + +| Platform | Status | Notes | +|----------|:------:|-------| +| macOS (Intel) | ✅ | Fully supported | +| macOS (Apple Silicon) | ✅ | Fully supported (native) | +| Ubuntu 22.04 | ✅ | Fully supported | +| Ubuntu 24.04 | ✅ | Fully supported | +| Windows (WSL2) | ⚠️ | Works with WSL2, not tested natively | +| Windows (native) | ❌ | Not tested — use WSL2 | + +## If Something Doesn't Work +Check the troubleshooting guide: `docs/evidence/GUIDE_QUICKSTART_DIAGNOSIS.md` diff --git a/docs/evidence/GUIDE_REPRODUCIBILITY_CHECK.md b/docs/evidence/GUIDE_REPRODUCIBILITY_CHECK.md new file mode 100644 index 00000000..ebac84fb --- /dev/null +++ b/docs/evidence/GUIDE_REPRODUCIBILITY_CHECK.md @@ -0,0 +1,22 @@ +# Hidden-State Reproducibility Check + +How to verify that pipeline results are reproducible. + +## What to Check +1. Same inputs + same config + same version = same outputs. +2. Output hashes match between runs. +3. Random seeds produce deterministic results. + +## How to Verify +```bash +# Run twice and compare outputs +make demo +cp outputs/demo_ranked.jsonl /tmp/run1.jsonl +make demo +diff outputs/demo_ranked.jsonl /tmp/run1.jsonl # Should be identical +``` + +## What If Outputs Differ +- Check for non-deterministic components (timestamps, UUIDs). +- Check that random seeds are fixed. +- Check that the code version hasn't changed between runs. diff --git a/docs/evidence/GUIDE_REVIEW_DASHBOARD.md b/docs/evidence/GUIDE_REVIEW_DASHBOARD.md new file mode 100644 index 00000000..3b77566f --- /dev/null +++ b/docs/evidence/GUIDE_REVIEW_DASHBOARD.md @@ -0,0 +1,15 @@ +# Maintainer Review Dashboard + +Metrics for tracking review performance. + +## Dashboard Metrics +| Metric | Target | Collection | +|--------|:------:|------------| +| Open PRs | < 10 | `gh pr list` | +| PRs awaiting review | < 5 | `gh pr list --review-requested` | +| Median time to review | < 24h | Tracked in review latency ledger | +| Oldest unreviewed PR | < 72h | `gh pr list --sort created` | +| Issues without assignee | < 20 | `gh issue list --no-assignee` | + +## How to Use +Run weekly. If any metric exceeds target, investigate bottlenecks. diff --git a/docs/evidence/INDEX_ADR.md b/docs/evidence/INDEX_ADR.md new file mode 100644 index 00000000..0cc9d0e6 --- /dev/null +++ b/docs/evidence/INDEX_ADR.md @@ -0,0 +1,19 @@ +# Architecture Decision Record Index + +Index of architecture decision records. + +| ADR | Title | Date | Status | +|:---:|-------|:----:|:------:| +| 001 | Use heuristic scoring over ML | 2026-06 | Accepted | +| 002 | Dry-lab first, wet-lab gated | 2026-06 | Accepted | +| 003 | JSON schemas for all artifacts | 2026-06 | Accepted | +| 004 | 50-loop execution plan | 2026-06 | Accepted | +| 005 | Simulation modules experimental | 2026-07 | Accepted | + +## Format +ADRs are stored in `docs/evidence/` with the prefix `ADR-`. + +## Rules +- Major architecture decisions should be recorded as ADRs. +- ADRs are numbered sequentially. +- ADRs are never deleted — only superseded. diff --git a/docs/evidence/MATRIX_CONTRIBUTOR_ENV.md b/docs/evidence/MATRIX_CONTRIBUTOR_ENV.md new file mode 100644 index 00000000..86d06408 --- /dev/null +++ b/docs/evidence/MATRIX_CONTRIBUTOR_ENV.md @@ -0,0 +1,21 @@ +# Contributor Environment Matrix + +Supported environments for contributors. + +| OS | Python | Shell | Status | +|---|:-------:|:-----:|:------:| +| macOS 14+ | 3.11-3.14 | zsh | ✅ | +| macOS 13 | 3.11-3.14 | zsh | ✅ | +| Ubuntu 22.04+ | 3.11-3.14 | bash | ✅ | +| Ubuntu 20.04 | 3.11-3.13 | bash | ✅ | +| Windows (WSL2/Ubuntu) | 3.11-3.13 | bash | ⚠️ | + +## Requirements +- Python 3.11+ (3.14 recommended) +- pip 21+ +- git 2.25+ +- make (optional, for Makefile targets) + +## Notes +- Windows native is not supported — use WSL2. +- Other Linux distributions likely work but are not tested in CI. diff --git a/tests/test_batch_v2.py b/tests/test_batch_v2.py new file mode 100644 index 00000000..f357ab52 --- /dev/null +++ b/tests/test_batch_v2.py @@ -0,0 +1,20 @@ +"""Tests for latest batch.""" +from pathlib import Path + +FILES = [ + "CATALOG_AGENT_ISSUES.md", + "GUIDE_ALLOWED_CLAIMS.md", + "GUIDE_ISSUE_TRIAGE.md", + "GUIDE_REVIEW_DASHBOARD.md", + "INDEX_ADR.md", + "GUIDE_DECISION_VALIDATOR.md", + "GUIDE_PLATFORM_VERIFICATION.md", + "MATRIX_CONTRIBUTOR_ENV.md", + "GUIDE_REPRODUCIBILITY_CHECK.md", + "ARCHITECTURE_OVERVIEW_CONTRIBUTORS.md", +] + + +def test_all_exist(): + for f in FILES: + assert Path("docs/evidence", f).exists(), f"Missing: {f}"