Migrated from Method backlog
This issue was created from a legacy filesystem backlog card. GitHub Issues are now the live work tracker; repository docs remain Method evidence.
Source backlog: docs/method/backlog/cool-ideas/METHOD_generated-playback.md
Original lane: cool-ideas
Original legend: METHOD
Original backlog card
Generated playback: derive the Phase 3 doc from RED tests + cycle goals
Legend: METHOD
The pain
METHOD Phase 3 asks the agent to run through every playback
question and record the witness. The expected artifact is a
playback document the human reviews and signs off on. In practice:
- Hand-written playback docs go stale the moment a test name
changes or a witness command moves.
- Authoring playback alongside RED-then-GREEN is a separate
context switch and routinely gets postponed.
- A stale playback is worse than no playback — it ratifies a state
the code no longer matches.
The structural answer: the playback doc should be a generated
artifact, derived from the same inputs that already exist by the
time Phase 3 starts.
What the generator takes as input
- The cycle goal lines from
docs/design/<cycle>/design.md
(already required).
- The cycle's RED test names — every
#[test] (Rust) or
t.test('...', ...) (Node) in the test files marked as part of
the cycle.
- The cycle's acceptance criteria from the design or backlog card
(already required to be there).
- The witness command set from the cycle's playback questions
scaffold (a one-line YAML or markdown frontmatter mapping
question → command).
What it emits
docs/method/playback/<cycle>/playback.md
A Markdown document where each playback question is a section, and
each section auto-fills:
- The question (from the scaffold).
- The matching RED test names and their pass/fail at last run.
- The witness command(s) and their last-recorded output.
- A boxed sign-off slot ("Sponsor 1: _ / Sponsor 2: _").
Determinism contract:
- Section order is canonical, sorted by explicit
question_id
from the scaffold.
- Test names within a section are canonical, sorted
lexicographically.
- Witness commands within a section preserve scaffold declaration
order.
question_id is an explicit immutable scaffold field, never
derived from mutable prose text, and sign-offs are keyed only by
that id.
The doc is regenerable. If a test renames, the next regen picks up
the new name; if a witness command emits new output, the doc
reflects it. Sign-off slots persist across regens (they live in a
sibling <cycle>/playback-signoffs.md keyed by question id, so the
generated doc can rewrite freely without nuking signatures).
Why this is on-brand
- "Architecture with a receipt" applied to the development process
itself. Phase 3 is the receipt; generating it from the same
inputs that produced Phase 2 GREEN closes the loop.
- It eliminates the largest source of "stale doc" in the METHOD
workflow without removing the human sign-off — the sign-off
remains the load-bearing artifact; the doc just stops lying
about what it certifies.
- It makes the "RED test name = playback question shape" coupling
explicit. RED authors write test names knowing they will
surface in the playback doc, which improves naming discipline.
Out of scope here
- Auto-generating the cycle design or RED tests from each other.
That direction breeds tautology; humans need to author the
goals.
- Replacing the human sign-off step. Generated docs do not
ratify; sponsors still do.
Trigger / acceptance
Resolve this card when:
- An
xtask method playback <cycle> command exists and produces
docs/method/playback/<cycle>/playback.md from the inputs
above.
- Re-running the command after a test rename produces a doc with
the new name and preserves prior sign-offs.
- METHOD Phase 3 documentation references the generator as the
canonical path (hand-written playback docs become an exception,
not the default).
- Two consecutive runs with identical inputs produce
byte-identical docs/method/playback/<cycle>/playback.md, and
sign-off association remains unchanged.
- The verification step re-runs
xtask method playback <cycle> on
identical inputs, compares output bytes, and checks the
playback-signoffs.md question-id mapping is unchanged.
Companion
docs/method/backlog/cool-ideas/METHOD_leash-files.md — same
spirit: structural records that the machine maintains, with
prose lanes alongside for the human-readable narrative.
Migrated from Method backlog
This issue was created from a legacy filesystem backlog card. GitHub Issues are now the live work tracker; repository docs remain Method evidence.
Source backlog:
docs/method/backlog/cool-ideas/METHOD_generated-playback.mdOriginal lane:
cool-ideasOriginal legend:
METHODOriginal backlog card
Generated playback: derive the Phase 3 doc from RED tests + cycle goals
Legend:
METHODThe pain
METHOD Phase 3 asks the agent to run through every playback
question and record the witness. The expected artifact is a
playback document the human reviews and signs off on. In practice:
changes or a witness command moves.
context switch and routinely gets postponed.
the code no longer matches.
The structural answer: the playback doc should be a generated
artifact, derived from the same inputs that already exist by the
time Phase 3 starts.
What the generator takes as input
docs/design/<cycle>/design.md(already required).
#[test](Rust) ort.test('...', ...)(Node) in the test files marked as part ofthe cycle.
(already required to be there).
scaffold (a one-line YAML or markdown frontmatter mapping
question → command).
What it emits
A Markdown document where each playback question is a section, and
each section auto-fills:
Determinism contract:
question_idfrom the scaffold.
lexicographically.
order.
question_idis an explicit immutable scaffold field, neverderived from mutable prose text, and sign-offs are keyed only by
that id.
The doc is regenerable. If a test renames, the next regen picks up
the new name; if a witness command emits new output, the doc
reflects it. Sign-off slots persist across regens (they live in a
sibling
<cycle>/playback-signoffs.mdkeyed by question id, so thegenerated doc can rewrite freely without nuking signatures).
Why this is on-brand
itself. Phase 3 is the receipt; generating it from the same
inputs that produced Phase 2 GREEN closes the loop.
workflow without removing the human sign-off — the sign-off
remains the load-bearing artifact; the doc just stops lying
about what it certifies.
explicit. RED authors write test names knowing they will
surface in the playback doc, which improves naming discipline.
Out of scope here
That direction breeds tautology; humans need to author the
goals.
ratify; sponsors still do.
Trigger / acceptance
Resolve this card when:
xtask method playback <cycle>command exists and producesdocs/method/playback/<cycle>/playback.mdfrom the inputsabove.
the new name and preserves prior sign-offs.
canonical path (hand-written playback docs become an exception,
not the default).
byte-identical
docs/method/playback/<cycle>/playback.md, andsign-off association remains unchanged.
xtask method playback <cycle>onidentical inputs, compares output bytes, and checks the
playback-signoffs.mdquestion-id mapping is unchanged.Companion
docs/method/backlog/cool-ideas/METHOD_leash-files.md— samespirit: structural records that the machine maintains, with
prose lanes alongside for the human-readable narrative.