diff --git a/.gitignore b/.gitignore index d7e19813..284b60ed 100644 --- a/.gitignore +++ b/.gitignore @@ -79,6 +79,7 @@ logs/ # Generated pipeline outputs outputs/* !outputs/.gitkeep +!outputs/README.md reports/* !reports/.gitkeep runs/* diff --git a/docs/getting-started/TROUBLESHOOTING.md b/docs/getting-started/TROUBLESHOOTING.md new file mode 100644 index 00000000..9e3c0149 --- /dev/null +++ b/docs/getting-started/TROUBLESHOOTING.md @@ -0,0 +1,148 @@ +# Troubleshooting Install and Test Failures (A10) + +This document covers the most common failures when setting up OpenAMP Foundry for the first time and how to fix them. + +## Install failures + +### `pip install -e .` fails with dependency conflict + +**Symptom:** `ERROR: pip's dependency resolver does not currently take into account all the packages...` + +**Fix:** +```bash +python -m venv .venv +source .venv/bin/activate +pip install --upgrade pip +pip install -e ".[dev]" +``` + +Using a fresh virtual environment avoids conflicts with system packages. + +### `make doctor` reports missing packages + +**Symptom:** `MISSING: [package]` in the doctor output. + +**Fix:** +```bash +pip install -e ".[dev]" +``` + +If a specific package still fails: +```bash +pip install [package-name] +``` + +### Wrong Python version + +**Symptom:** `make doctor` reports Python version mismatch or syntax errors on import. + +**Fix:** OpenAMP Foundry requires Python 3.9+. Check your version: +```bash +python --version +``` + +If you have multiple Python versions, use `python3.9` or `python3.11` explicitly: +```bash +python3.11 -m venv .venv +``` + +## Test failures + +### `pytest` fails with `ModuleNotFoundError` + +**Symptom:** +``` +ModuleNotFoundError: No module named 'openamp_foundry' +``` + +**Fix:** The package is not installed in editable mode. Run: +```bash +pip install -e . +``` + +Or set `PYTHONPATH` explicitly: +```bash +PYTHONPATH=src pytest tests/ +``` + +### Test count regression fails + +**Symptom:** +``` +AssertionError: Test count 8158 deviates from BASELINE 8410 by more than 420. +``` + +**Cause:** Your local branch is behind `origin/main` and is missing test files that were merged. Or a PR updated the BASELINE but you haven't pulled. + +**Fix:** +```bash +git fetch origin +git reset --hard origin/main +``` + +### `make agent-check` fails with claim violation + +**Symptom:** +``` +VIOLATION: Found forbidden claim language in [file] +``` + +**Cause:** A file contains language that overclaims computational results (e.g., "validated," "confirmed efficacy," "biological proof"). + +**Fix:** Replace the flagged language with approved computational-claim terms ("computationally nominated," "dry-lab candidate," "selected for review"). See `docs/CLAIM_BOUNDARY_RULES.md`. + +### `make doc-links-check` reports broken links + +**Symptom:** +``` +BROKEN: docs/X.md → docs/Y.md (file not found) +``` + +**Fix:** The linked file was either renamed or deleted. Either update the link or restore the missing file. Check git log to see if it was recently moved: +```bash +git log --all --full-history -- docs/Y.md +``` + +### Benchmark test fails with threshold mismatch + +**Symptom:** +``` +AssertionError: precision_at_k 0.48 < PRECISION_AT_K_THRESHOLD 0.50 +``` + +**Cause:** The benchmark result is below the passing threshold. Do NOT lower the threshold to make the test pass. Thresholds are governance decisions. + +**Fix:** Investigate why the benchmark dropped. Check if a recent change to the scoring pipeline lowered quality. File an issue if the drop is unexpected. + +## Environment issues + +### `opencode` not found + +**Symptom:** `command not found: opencode` + +**Cause:** The `opencode` CLI is an optional dependency used in the agent dispatch loop. + +**Fix:** Install via the opencode documentation or skip agent dispatch steps if running manually. + +### `gh` not found (GitHub CLI) + +**Symptom:** `command not found: gh` + +**Fix:** +```bash +brew install gh # macOS +gh auth login # authenticate +``` + +### Pre-push hook is slow + +**Symptom:** `git push` runs the full pytest suite and takes several minutes. + +**Fix:** The pre-push hook runs tests by design (it prevents bad pushes). For fast iteration, use `git push --no-verify` only for feature branches where CI will catch failures. Never use `--no-verify` on main. + +## Getting more help + +- Check `CLAUDE.md` for the full operating contract +- Check `AGENTS.md` for agent-specific guidance +- File an issue with the `troubleshooting` label +- For test infrastructure questions, see `tests/README.md` if it exists diff --git a/docs/operations/MAINTAINER_AGENT_TASK_GUIDE.md b/docs/operations/MAINTAINER_AGENT_TASK_GUIDE.md new file mode 100644 index 00000000..a9025c45 --- /dev/null +++ b/docs/operations/MAINTAINER_AGENT_TASK_GUIDE.md @@ -0,0 +1,146 @@ +# Maintainer Guide: Assigning Agent-Safe Tasks (D7) + +This document provides maintainers with ready-to-use prompts and guidelines for assigning tasks to AI agents within the OpenAMP Foundry repository. + +## When to use an agent + +Use an agent for tasks that are: +- Self-contained (one schema, one test file, one doc) +- Fully dry-lab (no real candidate data, no wet-lab consequences) +- Bounded by a known acceptance criterion +- Classifiable via `AGENT_TASKS.json` + +Do NOT use an agent for tasks that: +- Touch safety policy, release policy, or calibration thresholds +- Involve real (non-toy) candidate sequences +- Could weaken a safety gate or novelty filter +- Require human judgment about scientific claims + +## Pre-assignment checklist + +Before assigning any task to an agent, confirm: + +- [ ] The task is in `AGENT_TASKS.json` under `allowed_zones` (not `forbidden_zones`) +- [ ] The scope is bounded to ≤3 files (plus BASELINE update) +- [ ] The acceptance criterion is measurable (`pytest` passes, `make agent-check` passes) +- [ ] The safety classification is explicit (see issue template in `AGENT_SAFE_ISSUES.md`) +- [ ] The task does NOT introduce real sequence data + +## Prompt templates + +### Template 1: Add a new evidence schema (63 tests) + +``` +You are implementing [SCHEMA_NAME] for the OpenAMP Foundry repository. + +Write exactly 2 files: + +FILE 1: src/openamp_foundry/evidence/[module_name].py +- ID prefix: [PREFIX]- +- Fields: [list 12-15 fields] +- Validation rules: [list 8-12 rules] +- Controlled vocabularies as frozenset (VALID_*) +- Constants: [list thresholds/limits] +- validate_[name]() and validate_[name]_dict() functions +- dry_lab_only=True enforced + +FILE 2: tests/evidence/test_[module_name].py +- Exactly 63 tests in 5 classes: + TestValid[Name] (10 tests) + Test[Field1]Validation (10 tests) + Test[Field2]Validation (10 tests) + TestConsistencyValidation (10 tests) + TestDictValidation (10 tests) + TestMultipleErrors (5 tests) + TestConstants (8 tests) + +After writing the files, run: +cd /Volumes/SSD/openamp-foundry +git fetch origin +git reset --hard origin/main +git checkout -b feat/[prefix-lower]-[schema-name] +git add src/openamp_foundry/evidence/[module_name].py +git add tests/evidence/test_[module_name].py +# Update BASELINE in tests/test_test_count_regression.py by +63 +git add tests/test_test_count_regression.py +git commit -m "feat: Phase [X] [X#] [schema name] -- [PREFIX]- schema: [N] fields, [N] validation rules, [key feature]; [N] tests (#[N])" +git push --no-verify origin feat/[prefix-lower]-[schema-name] +gh pr create --title "[title]" --body "[body]" --base main +gh pr merge --squash --admin 2>/dev/null || gh pr merge --squash 2>/dev/null || true +``` + +### Template 2: Add a docs-only file + +``` +You are implementing [ITEM_ID] for the OpenAMP Foundry repository. This is a docs-only item with no code changes. + +Write exactly 1 file: + +FILE: docs/[path/to/FILE.md] +[full file content here] + +After writing the file, run: +cd /Volumes/SSD/openamp-foundry +git fetch origin +git reset --hard origin/main +git checkout -b docs/[item-id-lower]-[short-name] +git add docs/[path/to/FILE.md] +git commit -m "docs: [description] ([item_id]) (#[N])" +git push --no-verify origin docs/[item-id-lower]-[short-name] +gh pr create --title "[title]" --body "[body]" --base main +gh pr merge --squash --admin 2>/dev/null || gh pr merge --squash 2>/dev/null || true +``` + +### Template 3: Mark NEXT_100_PR_MAP items complete + +``` +You are updating docs/research/NEXT_100_PR_MAP.md to mark newly completed items. + +Read the file first. Then for each item below, find the row and append "(complete)" to the Task column and add the implementation summary. + +Items to mark complete: +[list items with summaries] + +After updating the file, run: +cd /Volumes/SSD/openamp-foundry +git fetch origin +git reset --hard origin/main +git checkout -b chore/update-next100-map-completions +git add docs/research/NEXT_100_PR_MAP.md +git commit -m "chore: mark [items] complete in NEXT_100_PR_MAP (#[N])" +git push --no-verify origin chore/update-next100-map-completions +gh pr create --title "[title]" --body "[body]" --base main +gh pr merge --squash --admin 2>/dev/null || gh pr merge --squash 2>/dev/null || true +``` + +## Labeling convention + +When creating a GitHub issue for an agent task, use these labels: + +| Label | Meaning | +|---|---| +| `agent-safe` | Pre-cleared for agent pickup without clarification | +| `docs-only` | No code changes expected | +| `schema-addition` | New schema + 63 tests + BASELINE update | +| `test-completion` | Existing schema needs more tests to reach 63 | +| `map-update` | NEXT_100_PR_MAP.md metadata update only | +| `review-required` | Human must review before agent proceeds | + +## Agent failure signals + +If an agent returns any of these, do NOT auto-merge and request human review: + +- "I changed [safety/release/calibration/threshold]..." +- "I updated proof_ladder_level..." +- "I used real sequence data..." +- "STOP CONDITION TRIGGERED" (agent correctly stopped) +- Any diff touching `forbidden_zones` in `AGENT_TASKS.json` + +## Expected agent behavior + +A well-behaved agent will: +1. Check `AGENT_TASKS.json` to classify the task +2. Write files exactly as specified (no extra features, no cleanups) +3. Update `tests/test_test_count_regression.py` BASELINE if test count changes +4. Run `make agent-check` before committing +5. Stop and leave a decision-log draft if a stop condition fires diff --git a/outputs/README.md b/outputs/README.md new file mode 100644 index 00000000..b515270c --- /dev/null +++ b/outputs/README.md @@ -0,0 +1,43 @@ +# Outputs Directory + +This directory contains generated artifacts produced by OpenAMP Foundry pipeline runs. All artifacts are **dry-lab only** — computational outputs that require independent wet-lab validation before any biological conclusions can be drawn. + +## What is in this directory + +| File pattern | Description | +|---|---| +| `bench_*.json` | Benchmark results (precision@k, calibration, per-family) | +| `batch_*.fasta` | FASTA exports of candidate sequences for partner review | +| `calibration_audit_example.json` | Example calibration audit output | +| `baseline_trivial.json` | Trivial baseline scores for cheap-enemy comparison | +| `*.json` | Pipeline run outputs — see schema documentation | + +## Dry-lab-only notice + +**None of these outputs constitute biological proof.** They represent computational nominations based on dry-lab scoring pipelines. The pipeline does not predict in-vivo efficacy, toxicity, or any wet-lab outcome. + +Terms that apply to outputs in this directory: +- "computationally nominated" +- "dry-lab candidate" +- "selected for review" +- "evidence package" + +Terms that do NOT apply: +- "validated" +- "confirmed" +- "proven effective" +- "biologically active" + +## What is git-ignored + +Most pipeline run outputs are listed in `.gitignore` and are NOT committed. This directory contains only curated example outputs that are small, safe, and useful for reviewers and new contributors to understand what good output looks like. + +To regenerate these examples: `make demo` + +## Interpreting outputs + +Each output file is generated by a schema in `src/openamp_foundry/`. The corresponding test file shows what fields are required and what validation rules apply. + +For evidence certificates (CERT- prefix) and related records, see `docs/evidence/` for interpretation guidance. + +For benchmark outputs, see `docs/BENCHMARK_CAVEATS.md` before drawing conclusions.