Skip to content

Fix: two spec-discovery defects that understate coverage and invent drift - #19

Merged
0xLeif merged 1 commit into
mainfrom
fix/spec-discovery-and-drift
Sep 19, 2026
Merged

0xLeif merged 1 commit into
mainfrom
fix/spec-discovery-and-drift

Conversation

@0xLeif

@0xLeif 0xLeif commented Sep 19, 2026

Copy link
Copy Markdown
Contributor

Summary

Two defects in crates/atlas-cli/src/main.rs, both found while taking a real project from a reported 64% to 100%. Between them they understated that project's coverage by 1,252 lines and put a spec that already existed at the top of its action plan.

  • load_specs applied SKIP_DIRS below the specs/ root. That list holds build and vendor directory names, and a spec module is named after what it governs β€” so specs/out/, specs/build/, specs/target/ and specs/coverage/ were never parsed, and every file they governed was reported as an orphan. The walk now carries an in_specs flag and prunes only dot-directories below that point. The list is scoped, not disabled: a .spec.md a build copied into out/ is still skipped.
  • enrich_drift scraped verdicts out of the report's raw text. It found the first "<module>" anywhere in the document, took the next 240 bytes, and reported the first verdict word in them. The name matched inside any string, so the window often described a different spec; the window ran past the end of the entry, so the last spec in specs picked up the following top-level key and "stale": [] marked it stale β€” a needs-review flag on a spec that is in sync; and out[pos..pos + 240] panics outright when the cut lands inside a multi-byte character. It now parses the JSON and matches a module by its specs entry or by name in stale.

The WASM engine already collected .spec.md from anywhere without the skip list, so this narrows a pre-existing divergence between the two engines rather than creating one.

specs/engine/engine.spec.md records both rules: the load_specs row in the Public API table, an extension to invariant 4, and a new invariant 4a for reading drift as JSON.

Test Plan

  • fledge lanes run verify β€” fmt, lint, test, build, wasm-test, wasm-build, all green
  • 7 new regression tests (22 in atlas-cli, 26 in atlas-core, 0 failures)
    • load_specs_finds_a_module_named_after_a_build_directory β€” out, build, dist, target, coverage and a control module are all discovered
    • load_specs_still_skips_a_build_tree_outside_specs β€” a .spec.md under out/, node_modules/ or target/ is not adopted
    • the_last_spec_in_a_report_is_not_made_stale_by_the_next_key β€” the false positive that started this
    • a_module_named_in_the_stale_list_is_stale β€” bare name, {name} object, and {path} object
    • an_explicit_verdict_on_the_module_s_own_entry_is_reported
    • a_module_name_appearing_in_another_spec_s_entry_is_not_its_verdict
    • a_report_carrying_multibyte_text_does_not_panic β€” the old slice would have panicked
    • a_report_that_is_not_json_is_ignored_rather_than_guessed_at
  • Verified end to end against two real repositories: the affected project goes from 64% / 4 orphans / one false needs_review to 100% / 0 orphans / 0 needs_review; this repo stays healthy with no new review flags

πŸ€– Generated with Claude Code

…rift

`load_specs` applied `SKIP_DIRS` below the `specs/` root. That list holds build
and vendor directory *names*, and a spec module is named after what it governs,
so `specs/out/`, `specs/build/`, `specs/target/` and `specs/coverage/` were all
undiscoverable: the spec was never parsed and every file it governed was
reported as an orphan. Found on a project whose `specs/out/out.spec.md` was
invisible, understating its coverage by 1,252 lines and putting "write a spec
for src/out.rs" at the top of its action plan. The walk now carries an
`in_specs` flag and prunes only dot-directories below that point; a `.spec.md`
a build copied into `out/` is still skipped, so the list is scoped rather than
disabled.

`enrich_drift` scraped verdicts out of the report's raw text: find the first
`"<module>"` anywhere in the document, take the next 240 bytes, report the
first verdict word in them. Three things were wrong with that. The name matched
inside any string, so the window often described a different spec. The window
ran past the end of the entry, so the *last* spec in `specs` picked up the
following top-level key β€” `"stale": []` marked it stale, a needs-review flag on
a spec that is in sync. And `out[pos..pos + 240]` panics outright when the cut
lands inside a multi-byte character. It now parses the JSON and matches a
module by its `specs` entry or by name in `stale`.

Seven regression tests, and the engine spec's Public API table, invariant 4 and
a new invariant 4a record both rules.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@0xLeif
0xLeif merged commit b1ca524 into main Sep 19, 2026
6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant