diff --git a/docs/evidence/DOCS_IMPROVEMENT_ROADMAP.md b/docs/evidence/DOCS_IMPROVEMENT_ROADMAP.md new file mode 100644 index 00000000..3b85597a --- /dev/null +++ b/docs/evidence/DOCS_IMPROVEMENT_ROADMAP.md @@ -0,0 +1,27 @@ +# Documentation Improvement Roadmap + +Planned improvements to the documentation system. + +## Current +- [x] Documentation governance charter +- [x] Documentation role definitions +- [x] Documentation review cadence +- [x] Documentation retirement policy +- [x] Documentation minor-edit policy +- [x] Quickstart guide +- [x] Contributor curriculum +- [x] Concept cards for all artifact types +- [x] Documentation maps for all areas +- [x] Audit checklists for all doc types + +## Next +- [ ] Auto-generate PROJECT_INDEX.md from docs structure +- [ ] Add documentation coverage CI check +- [ ] Create documentation health dashboard +- [ ] Integrate docs metrics into release report + +## Future +- [ ] Multilingual documentation +- [ ] Video tutorials +- [ ] Interactive documentation (Jupyter notebooks) +- [ ] Documentation contribution analytics diff --git a/docs/evidence/DOCS_MAINTENANCE_CALENDAR.md b/docs/evidence/DOCS_MAINTENANCE_CALENDAR.md new file mode 100644 index 00000000..b1bfb3eb --- /dev/null +++ b/docs/evidence/DOCS_MAINTENANCE_CALENDAR.md @@ -0,0 +1,23 @@ +# Documentation Maintenance Calendar + +Regular maintenance tasks for documentation. + +## Weekly +- [ ] Review and merge documentation PRs +- [ ] Check for broken links (run `check_doc_links.py`) + +## Monthly +- [ ] Review new issues tagged `docs` and assign priorities +- [ ] Update METRICS_CURRENT.md if benchmarks changed +- [ ] Check QUICKSTART.md commands still work + +## Quarterly +- [ ] Full documentation audit (run all audit checklists) +- [ ] Review stale documents and archive or update +- [ ] Update learning paths and curricula +- [ ] Run `check_claims.py` on all public-facing docs + +## Annually +- [ ] Review documentation governance charter +- [ ] Review safety documentation +- [ ] Archive documents that are no longer relevant diff --git a/docs/evidence/GUIDE_CLI_EXIT_CODES.md b/docs/evidence/GUIDE_CLI_EXIT_CODES.md new file mode 100644 index 00000000..9b60296b --- /dev/null +++ b/docs/evidence/GUIDE_CLI_EXIT_CODES.md @@ -0,0 +1,15 @@ +# CLI Exit Code Guide + +Exit codes used by OpenAMP Foundry commands. + +| Code | Meaning | Example | +|:----:|---------|---------| +| 0 | Success | Command completed successfully | +| 1 | Internal error | Unexpected exception | +| 2 | Input error | Missing file, invalid arguments | +| 3 | Validation failure | Gate check failed, data invalid | + +## Consistency Rules +- All CLI commands should follow this exit code convention. +- New scripts should use exit code 2 for input errors, 3 for validation failures. +- Exit code 1 should be reserved for unexpected errors (bugs). diff --git a/docs/evidence/GUIDE_DEPENDENCY_REVIEW.md b/docs/evidence/GUIDE_DEPENDENCY_REVIEW.md new file mode 100644 index 00000000..af8b11e2 --- /dev/null +++ b/docs/evidence/GUIDE_DEPENDENCY_REVIEW.md @@ -0,0 +1,17 @@ +# Dependency Review Guide + +How to review dependency changes in PRs. + +## Steps +1. Verify the new dependency is necessary for the change. +2. Check the license is compatible with Apache 2.0. +3. Check for known security vulnerabilities. +4. Verify the change works on all supported platforms. +5. Update pyproject.toml and lock file if applicable. +6. Verify tests pass with the new dependency. + +## Red Flags +- Dependency added but not used in the PR +- Dependency with a restrictive license +- Dependency with known vulnerabilities +- Major version bump without documented breaking changes diff --git a/docs/evidence/GUIDE_MINIMAL_OUTPUT.md b/docs/evidence/GUIDE_MINIMAL_OUTPUT.md new file mode 100644 index 00000000..162117f9 --- /dev/null +++ b/docs/evidence/GUIDE_MINIMAL_OUTPUT.md @@ -0,0 +1,20 @@ +# Minimal Output Mode Guide + +How to use minimal output mode for scripting and automation. + +## Usage +Pass `--quiet` or set `LOG_LEVEL=WARNING` to reduce output. + +## What's Included +- Errors and warnings only +- Final result summary (one line) + +## What's Suppressed +- Progress messages +- Debug information +- Non-critical warnings + +## Best Practices +- Use minimal mode in CI/CD pipelines. +- Use verbose mode (`--verbose`) when debugging. +- Default mode is appropriate for interactive use. diff --git a/docs/evidence/GUIDE_PACKAGE_METADATA.md b/docs/evidence/GUIDE_PACKAGE_METADATA.md new file mode 100644 index 00000000..90a993a6 --- /dev/null +++ b/docs/evidence/GUIDE_PACKAGE_METADATA.md @@ -0,0 +1,22 @@ +# Package Metadata Guide + +How to maintain package metadata in pyproject.toml. + +## Required Fields +- `name` — must match the import name (hyphens allowed) +- `version` — semantic versioning +- `description` — one-line summary +- `requires-python` — minimum Python version +- `license` — SPDX identifier + +## Optional but Recommended +- `readme` — path to README file +- `authors` — list of maintainers +- `classifiers` — PyPI classifiers +- `keywords` — search keywords +- `urls` — project URLs (repository, docs, issues) + +## Common Mistakes +- Version not bumped after changes +- Missing dependencies in `dependencies` or `[project.optional-dependencies]` +- Stale `classifiers` that don't match current Python version diff --git a/docs/evidence/GUIDE_REPORT_MANIFEST.md b/docs/evidence/GUIDE_REPORT_MANIFEST.md new file mode 100644 index 00000000..ad582f7f --- /dev/null +++ b/docs/evidence/GUIDE_REPORT_MANIFEST.md @@ -0,0 +1,21 @@ +# Report Manifest Reading Guide + +How to read a run manifest. + +## Required Fields +| Field | Description | +|-------|-------------| +| `run_id` | UUID for the run | +| `pipeline_version` | Version string | +| `config_hash` | SHA-256 of config | +| `generated_at` | ISO 8601 timestamp | +| `inputs` | List of input file paths | +| `input_hashes` | SHA-256 per input file | +| `outputs` | List of output paths | + +## How to Verify +1. Check that `pipeline_version` matches the expected version. +2. Verify `config_hash` matches the config used. +3. Check that all expected inputs are listed. +4. Verify that all expected outputs exist. +5. If an input hash doesn't match, the input file has changed since the run. diff --git a/docs/evidence/GUIDE_STDOUT_STDERR.md b/docs/evidence/GUIDE_STDOUT_STDERR.md new file mode 100644 index 00000000..aeea9c68 --- /dev/null +++ b/docs/evidence/GUIDE_STDOUT_STDERR.md @@ -0,0 +1,19 @@ +# Stdout and Stderr Behavior Guide + +How CLI commands should use stdout and stderr. + +## Rules +- **stdout**: Machine-readable output (JSON, CSV) or formatted results +- **stderr**: Human-readable messages (progress, warnings, errors) + +## Examples +| Command | stdout | stderr | +|---------|--------|--------| +| `validate` | JSON validation result | Error messages (if any) | +| `bench` | JSON benchmark results | Progress messages | +| `rank` | JSON summary | Progress messages | + +## Why +- Separating output from diagnostics allows piping stdout to files or other commands. +- stderr is visible to the user even when stdout is redirected. +- This is a standard Unix convention. diff --git a/docs/evidence/GUIDE_STRUCTURED_LOGGING.md b/docs/evidence/GUIDE_STRUCTURED_LOGGING.md new file mode 100644 index 00000000..3ffcd82f --- /dev/null +++ b/docs/evidence/GUIDE_STRUCTURED_LOGGING.md @@ -0,0 +1,23 @@ +# Structured Logging Reading Guide + +How to interpret structured log output. + +## Log Format +``` +timestamp LEVEL module:line message key=value key=value +``` + +## Levels +| Level | Meaning | Action | +|-------|---------|--------| +| DEBUG | Detailed diagnostic info | Only needed for debugging | +| INFO | Normal operation | Monitor progress | +| WARNING | Unexpected but not an error | Investigate if persistent | +| ERROR | Operation failed | Investigate and fix | +| CRITICAL | System cannot continue | Immediate attention | + +## Structured Fields +- `command` — which CLI command was running +- `candidate_id` — which candidate was being processed +- `duration_ms` — how long the operation took +- `exit_code` — exit code of the operation diff --git a/docs/evidence/GUIDE_WARNING_TAXONOMY.md b/docs/evidence/GUIDE_WARNING_TAXONOMY.md new file mode 100644 index 00000000..e7c94b06 --- /dev/null +++ b/docs/evidence/GUIDE_WARNING_TAXONOMY.md @@ -0,0 +1,16 @@ +# Warning Taxonomy Guide + +Types of warnings and how to handle them. + +| Warning Type | Meaning | Example | Action | +|-------------|---------|---------|--------| +| `stale_config` | Config option is deprecated | `deprecated key 'foo'` | Update config | +| `missing_field` | Optional field is absent | `field 'bar' not found` | Check source | +| `low_confidence` | Score uncertainty is high | `uncertainty > 0.5` | Treat as experimental | +| `regression` | Metric dropped | `AUROC dropped 0.02` | Investigate | +| `overclaim` | Risky wording detected | `found 'proven'` | Rewrite | + +## Rules +- Warnings should not change behavior. +- Warnings should include actionable information. +- Warnings should be logged to stderr, not stdout. diff --git a/tests/test_cli_guides_batch.py b/tests/test_cli_guides_batch.py new file mode 100644 index 00000000..07885374 --- /dev/null +++ b/tests/test_cli_guides_batch.py @@ -0,0 +1,20 @@ +"""Tests for CLI guides and maintenance docs.""" +from pathlib import Path + +FILES = [ + "GUIDE_DEPENDENCY_REVIEW.md", + "GUIDE_PACKAGE_METADATA.md", + "GUIDE_STRUCTURED_LOGGING.md", + "GUIDE_CLI_EXIT_CODES.md", + "GUIDE_STDOUT_STDERR.md", + "GUIDE_WARNING_TAXONOMY.md", + "GUIDE_MINIMAL_OUTPUT.md", + "GUIDE_REPORT_MANIFEST.md", + "DOCS_MAINTENANCE_CALENDAR.md", + "DOCS_IMPROVEMENT_ROADMAP.md", +] + + +def test_all_exist(): + for f in FILES: + assert Path("docs/evidence", f).exists(), f"Missing: {f}"