Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 12 additions & 0 deletions docs/evidence/CHECKLIST_MAINTAINER_HANDOFF.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
# Maintainer Handoff Checklist

When a maintainer is unavailable or leaving, use this checklist.

- [ ] Identify all PRs the maintainer is reviewing.
- [ ] Reassign PRs to other maintainers.
- [ ] Identify all issues the maintainer is assigned to.
- [ ] Reassign or unassign issues.
- [ ] Document any in-progress work the maintainer was doing.
- [ ] Update the folder ownership map if needed.
- [ ] Notify the team of the change.
- [ ] Update the decision log.
15 changes: 15 additions & 0 deletions docs/evidence/CHECKLIST_PR_RISK.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
# Pull Request Risk Checklist

Use this to assess PR risk before review.

## Questions
- [ ] Does the PR change scoring weights? (High risk)
- [ ] Does the PR change calibration pipeline? (High risk)
- [ ] Does the PR change safety policy? (High risk)
- [ ] Does the PR add new dependencies? (Medium risk)
- [ ] Does the PR change CLI output format? (Medium risk)
- [ ] Does the PR add new functionality? (Medium risk)
- [ ] Does the PR fix a bug? (Low risk)
- [ ] Does the PR improve documentation? (Low risk)

Based on answers, assign the appropriate review depth per the PR review rubric.
21 changes: 21 additions & 0 deletions docs/evidence/GUIDE_REVIEW_LATENCY.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
# Review Latency Tracking Ledger

Track review latency to identify bottlenecks.

## Metrics
| Metric | Target | How to Measure |
|--------|:------:|----------------|
| Time to first review | < 24h | Time from PR open to first review comment |
| Time to merge | < 72h | Time from PR open to merge |
| Review depth | All issues addressed | Count of review comments resolved |

## Ledger Format
```csv
pr_number,author,reviewer,opened_at,first_review_at,merged_at,comments
#NNN,name,name,datetime,datetime,datetime,N
```

## Rules
- Review latency is tracked monthly.
- If targets are consistently missed, investigate bottlenecks.
- Review latency data is used for process improvement, not blame.
15 changes: 15 additions & 0 deletions docs/evidence/GUIDE_TASK_SIZE.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
# Contributor Task Size Guide

How to estimate task size for issues.

| Size | Lines of Change | Review Time | Example |
|:----:|:---------------:|:-----------:|---------|
| Small | < 50 | < 30 min | Typo fix, single doc update |
| Medium | 50-200 | 1-2 hours | New doc, small feature, test addition |
| Large | 200-500 | 2-4 hours | New script, policy document |
| XLarge | > 500 | > 4 hours | Major feature, architecture change |

## Rules
- Large or XLarge tasks should be broken into smaller issues.
- If an issue is XLarge, create a design doc first.
- Task size should be estimated before starting work.
20 changes: 10 additions & 10 deletions docs/evidence/GUIDE_WARNING_TAXONOMY.md
Original file line number Diff line number Diff line change
@@ -1,16 +1,16 @@
# Warning Taxonomy Guide

Types of warnings and how to handle them.
Types of warnings produced by the pipeline.

| Warning Type | Meaning | Example | Action |
|-------------|---------|---------|--------|
| `stale_config` | Config option is deprecated | `deprecated key 'foo'` | Update config |
| `missing_field` | Optional field is absent | `field 'bar' not found` | Check source |
| `low_confidence` | Score uncertainty is high | `uncertainty > 0.5` | Treat as experimental |
| `regression` | Metric dropped | `AUROC dropped 0.02` | Investigate |
| `overclaim` | Risky wording detected | `found 'proven'` | Rewrite |
| Type | Meaning | Example |
|------|---------|---------|
| stale_config | Deprecated config key | "key 'foo' is deprecated" |
| missing_field | Optional field missing | "field 'bar' not found" |
| low_confidence | High uncertainty | "uncertainty > 0.5" |
| regression | Metric declined | "AUROC dropped 0.02" |
| overclaim | Risky wording | "found 'proven'" |

## Rules
- Warnings should not change behavior.
- Warnings are logged to stderr, not stdout.
- Warnings should include actionable information.
- Warnings should be logged to stderr, not stdout.
- Warnings must not change pipeline behavior.
16 changes: 16 additions & 0 deletions docs/evidence/POLICY_QUIET_MODE.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
# Quiet Mode Policy for CLI Commands

CLI commands should support a quiet mode for scripting.

## Behavior
- `--quiet` flag suppresses all non-error output.
- Errors and the final result summary are still printed.
- Progress bars and informational messages are suppressed.

## Implementation
- Parse `--quiet` in the CLI handler.
- Set log level to WARNING when quiet is enabled.
- Suppress progress bars when quiet is enabled.

## Related
- `docs/evidence/GUIDE_MINIMAL_OUTPUT.md`
23 changes: 23 additions & 0 deletions docs/evidence/POLICY_SAFE_FILENAMES.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
# Safe Filename Policy

Filenames in the repository must follow these rules.

## Allowed Characters
- Lowercase letters: `a-z`
- Digits: `0-9`
- Hyphens: `-`
- Underscores: `_`
- Dots: `.`

## Rules
- No spaces in filenames.
- No special characters except `-` and `_`.
- Use `.md` for documentation, `.py` for Python, `.json` for JSON.
- Use snake_case for Python files and data files.
- Use lowercase for all filenames.

## Examples
- ✅ `check_doc_links.py`
- ✅ `METRICS_CURRENT.md`
- ❌ `My File.py`
- ❌ `data-file(v2).csv`
20 changes: 20 additions & 0 deletions docs/evidence/POLICY_SECRET_SCAN.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
# Secret-Scan Policy for Fixtures and Docs

Fixtures and documentation must not contain secrets.

## What to Scan For
- API keys (`sk-...`, `api_key=...`)
- Passwords (`password=...`)
- Tokens (`token=...`, `auth=...`)
- Private keys (`-----BEGIN...`)
- Connection strings (`mongodb://...`, `postgres://...`)

## Rules
- All fixtures and examples are scanned before commit.
- If a secret is found, remove it and rotate if it was ever committed.
- Use environment variables for any required credentials.
- The `.env` file is git-ignored.

## Enforcement
- Add secret scanning to CI (e.g., `trufflehog` or `git-secrets`).
- PRs that add new fixtures should be reviewed for secrets.
15 changes: 15 additions & 0 deletions docs/evidence/REGISTRY_TUTORIAL_FIXTURES.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
# Tutorial Fixture Registry

All tutorial fixtures must be registered here.

| Fixture | Location | Used By |
|---------|----------|---------|
| Demo candidates | examples/sequences/demo_candidates.csv | QUICKSTART.md |
| Demo references | examples/known_reference/demo_known_amps.csv | QUICKSTART.md |
| Lab results (synthetic) | examples/lab_results/ | calibration walkthrough |
| Hemolysis reference | examples/validation/hemolysis_reference.csv | selectivity benchmark |

## Rules
- Add new fixtures to this registry when created.
- Mark fixtures as `synthetic` or `demo` in their metadata.
- Never use real candidate data as tutorial fixtures.
21 changes: 21 additions & 0 deletions docs/evidence/STANDARD_WALKTHROUGH_SNAPSHOTS.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
# Walkthrough Expected-Output Snapshots

Walkthroughs should include expected output snapshots so users can verify
they got the right result.

## Format
```markdown
## Expected Output

After running `openamp-foundry rank ...`, the first line of the output file
should contain a JSON object with these keys:
- `candidate_id` — string
- `sequence` — string
- `scores` — dict with at least `ensemble`, `activity`, `safety`
- `selected` — boolean
```

## Rules
- Show the structure, not exact values.
- If exact values are shown, mark them as examples.
- Update snapshots when output format changes.
20 changes: 20 additions & 0 deletions tests/test_batch_latest.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
"""Tests for batch of 10 docs."""
from pathlib import Path

FILES = [
"REGISTRY_TUTORIAL_FIXTURES.md",
"STANDARD_WALKTHROUGH_SNAPSHOTS.md",
"GUIDE_TASK_SIZE.md",
"CHECKLIST_PR_RISK.md",
"GUIDE_REVIEW_LATENCY.md",
"CHECKLIST_MAINTAINER_HANDOFF.md",
"POLICY_SECRET_SCAN.md",
"POLICY_SAFE_FILENAMES.md",
"GUIDE_WARNING_TAXONOMY.md",
"POLICY_QUIET_MODE.md",
]


def test_all_exist():
for f in FILES:
assert Path("docs/evidence", f).exists(), f"Missing: {f}"
Loading