Skip to content

E2E contract corpus, milestone files, scaffold-seeded plans (Claude Opus 5) - #24

Merged
A merged 41 commits into
release/1.0.1from
refactor/e2e-contract-corpus-pilot
Aug 12, 2026
Merged

E2E contract corpus, milestone files, scaffold-seeded plans (Claude Opus 5)#24
A merged 41 commits into
release/1.0.1from
refactor/e2e-contract-corpus-pilot

Conversation

@A

@A A commented Aug 11, 2026

Copy link
Copy Markdown
Owner

Overview

booping scaffold's pytest suite described the Python implementation — Node object graphs, exception fields, exact message text — so it broke on no-op refactors and would not survive a rewrite of the CLI in another language.

This PR replaces it with a contract corpus: one txtar file per case carrying its fixture tree, the invocation, and everything asserted about the result (stdout, stderr, exit code, files left behind), executed by a standalone runner that shells bin/booping as a black box. scaffold is the pilot; the harness and format are what later plans convert the remaining subcommands onto.

booping-python/e2e/

  • _txtar.py — vendored txtar parse/serialize (Go semantics, stdlib only).
  • README.md — the format contract, written as the document a future non-Python runner is built from: section vocabulary, sandbox and env bindings, multi-line cmd, {CWD}/{HOME}/{XDG} tokens and the [..] wildcard, exit codes, the --update loop.
  • run.py — fresh home//xdg//cwd/ sandbox per case, subprocess invocation, labeled unified diffs, {N} passed, {M} failed, exits 0 pass / 1 fail / 2 malformed. --update rebaselines and is a no-op on a green corpus.
  • 38 cases covering every CLI-observable scaffold behavior — destinations and receipt shape, seed rendering, filename keys, error paths, the real core config trees, the scaffold→scaffold→query chain, the vault log line — replacing both superseded test files (917 lines deleted).

Two coverage decisions are recorded in e2e/README.md: exit-2-on-OS-fault is dropped (no file-mode fixture syntax), and atomicity is asserted via an empty stdout receipt, the format having no absent-file section.

Wired as just e2e [pattern…], added to just ci and the CI python job; basedpyright now includes e2e.

Sprint metrics

Development only — the grooming session is excluded, since the plan is shared with #25.

IN (uncached) is only the input that escaped prompt caching — the real input volume is the cached column.

Role Model IN (uncached) OUT cached in (write + read) Active
Orchestrator — develop driver claude-opus-5 857 70.1k 484.5k + 13.40M 39 min
Developer agents — 4 milestones claude-opus-5 404 59.4k 737.1k + 8.04M 23 min
Verify agent claude-sonnet-5 54 2.9k 56.1k + 980.2k 3 min
Total 1.3k 132.5k 1.28M + 22.42M 65 min

23.83M tokens total. Story points: 18 across 4 milestones.

🤖 Generated with Claude Code

A and others added 30 commits August 10, 2026 12:29
Tree keys go through the same Jinja env and --set globals as seed bodies,
with the filesystem-safety check re-run on the rendered name so an unsafe
render fails before any write.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
core.groom_playbook.milestone_scaffold seeds one file per milestone with a
fixed identity frontmatter and the three required headings; the shared
core.plans.milestones key describes where they live and how index.md's
table projects them. Ids are quoted so zero-padded values do not read back
as octal ints and scramble sorting.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Each template's Milestones section is now a generated table and a new
Milestone files section giving the per-file body shape for that surface;
checklists check the plan directory instead of a single file.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
draft-plan scaffolds one file per milestone and writes its body in place,
then generates index.md's table with booping query; present and
cross-review read the same source. sp moves to develop's refresh script.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
A milestone state machine addressed per instance writes status via
playbook-transition only; each edge runs refresh-milestone-table, which
rewrites index.md's Milestones table and re-sums the plan's sp from the
milestone files. develop-loop moves into a one-step milestones subgraph,
which is what makes the {instance} artifact legal.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
develop-loop composes one literal briefing block naming the milestone
files as contract and index.md as context; provision groups from a query
over milestone frontmatter and verify reads the milestone files. The
worker contract states which path binds it.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Uncommitted output of the docs-refresh run, captured before the
milestone-file sprint rewrites the same surfaces.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
retro, code-review and the lesson-check partial read index.md plus the
milestone files; the vault, develop and groom pages, README and CLAUDE.md
describe the plan directory, the milestone file contract and the
generated table.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Each hermetic fixture plan gains a milestones/ directory with two files
and an index table generated by refresh-milestone-table; cache-warmup
keeps none so the missing-sp branch stays covered.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
A milestone becomes a directory whose file is named after it, so a
runner-written feedback.md can sit beside the contract and a bare
wikilink still resolves. The worker now runs its own Verify and commits;
the runner validates the diff, keeps the bookkeeping and records failed
attempts in feedback.md, routing fixes to a fresh fallback_agent.
playbook-state strips the matched segment cleanly so instances key the
same in filename and directory positions.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…via local pi worker wire pi-developer agent into develop via vault config
A and others added 11 commits August 11, 2026 13:24
Plan: scaffold tests become a txtar contract corpus under booping-python/e2e/
with a standalone runner; superseded unit tests deleted. Drops the retired
reshape lesson pair (0008/0009).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…tests vendor txtar parser and write the case-format spec

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…tests standalone corpus runner over txtar cases
…tests scaffold behaviors ported to txtar corpus cases

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…tests delete superseded scaffold tests and wire docs
@A A changed the title E2E contract corpus, milestone files, scaffold-seeded plans E2E contract corpus for booping scaffold Aug 11, 2026
@A A changed the title E2E contract corpus for booping scaffold E2E contract corpus, milestone files, scaffold-seeded plans (Claude Opus 5) Aug 11, 2026
@A
A merged commit 2b69ae8 into release/1.0.1 Aug 12, 2026
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant