Development Policy Enforcement (tests-as-spec, commit intent, diff guards)

[flyingrobots]

Summary

We’ve agreed that:

The tests ARE the spec.

and that all Newton-style merges and future work on main should be:

• Test-first (failing test → implementation → passing test).
• Main-only (no new feature branches).
• History-safe (no rebases, no force pushes, no amend).

This issue tracks mechanizing that policy so we don’t have to rely on memory or vibes.

---

Goals

1. Enforce commit intent via commit message prefixes (feat:, fix:, refactor:, chore:, docs:).
2. Detect behavior changes without tests:
   • If code (src/**) changes under a feat:/fix: commit, require tests to change, too.
3. Allow refactors/chore commits without forcing tests, while still running the full test suite.
4. (Future) Tie coverage to changes:
   • Ensure new/changed code is exercised by tests (coverage doesn’t silently drop where it matters).

---

Policy

1. Commit Intent

• Behavior commits:
  • Prefix: feat:, fix:.
  • Expectations:
    • Some test files change (src/**/*.{test,spec}.{ts,tsx} or src/**/__tests__/**), or
    • Coverage for the changed code improves or changes in a meaningful way.
• Non-behavior commits:
  • Prefix: refactor:, chore:, docs:.
  • Expectations:
    • Tests may or may not change, but:
      • No intentional behavior change is allowed.
      • The full test suite must still pass.

2. Diff-Based Guardrails

On every push to main (or PR targeting main):

1. Compute diff vs origin/main:

   git diff --name-only origin/main...HEAD
   

2. Classify changed files:

   • Code files: src/** minus:
     • src//tests/
     • src/**/*.{test,spec}.{ts,tsx}
   • Test files:
     • src//tests/
     • src/**/*.{test,spec}.{ts,tsx}

3. Enforcement:

   • If there are code changes and no test changes:
     • AND the commit message starts with feat: or fix:
       • → Fail the check: “Behavioral commit with src changes but no tests.”
     • AND the commit message starts with refactor:, chore:, or docs:
       • → Allowed; rely on passing tests as guardrail.

This doesn’t perfectly detect behavior changes, but it catches most “changed behavior, forgot tests” cases and
forces the author to state intent in the commit message.

3. Coverage (Future)

Once we have coverage reports from Vitest (e.g., npm test -- --coverage):

• For each changed code file, check:
  • Coverage is above some threshold (e.g., ≥ 80%), or
  • Coverage for that file did not drop from a previous baseline.

This step is more expensive and can be added later; for now, the diff + intent approach is the MVP.

———

Implementation Plan

Phase 1 – Diff + Commit Prefix Heuristic

• Add a script under scripts/ (e.g., scripts/check-dev-policy.js) that:
  • Takes a base ref (default origin/main).
  • Computes changed files.
  • Classifies src vs tests.
  • Reads the latest commit message.
  • Exits with non-zero code if:
    • Commit starts with feat: or fix:, and
    • There are src changes, and
    • There are no test changes.
• Add a simple GitHub Action (or other CI job) that:
  • Runs on push to main and on PRs into main.
  • Runs npm test.
  • Runs node scripts/check-dev-policy.js origin/main.
  • Fails the build if the policy script exits non-zero.

Phase 2 – Coverage Integration (Optional)

• Enable coverage in Vitest:
  • e.g., npm test -- --coverage or Vitest config.
• Extend check-dev-policy script to:
  • Read coverage JSON (e.g., from .coverage/coverage-final.json).
  • For each changed src file, report coverage and optionally fail if below threshold.
• Wire coverage check into CI (can be a separate step so it’s easy to toggle).

Phase 3 – Refinements

• Allow per-directory overrides (e.g., some auto-generated files may be exempt).
• Optionally enforce that each feat:/fix: commit references specific test files or uses a standard “test
  narrative” in the commit message (for human review).

———

Acceptance Criteria

• Any feat: or fix: commit that changes src/** but not tests causes the policy script to fail.
• refactor:, chore:, and docs: commits are allowed to change src/** without tests (as long as the test
  suite passes).
• The policy script can be run locally (node scripts/check-dev-policy.js) and in CI.
• Policy is documented in AGENTS.md and discoverable in the repo for new contributors.