From eaef914fd74e38d397c8163eda9c66cadd65e68e Mon Sep 17 00:00:00 2001 From: Diego Andres Rabaioli Date: Mon, 20 Jul 2026 22:33:00 +0200 Subject: [PATCH 1/2] Sync handoff + state across machines via refs/cdd/ (#22) The handoff (.md) and state record (.state.json) were out-of-tree, per-user files stranded on the machine that created them. cdd-worktree-resume recreated only the code branch. Carry the two files across machines through a per-task git ref. - cdd-state (seed + set): bundle whichever of the two files exist into a git tree under stable in-tree names (handoff.md/state.json), wrap it in a parentless commit, and force-push to refs/cdd/ on origin. Plumbing only (hash-object/mktree/commit-tree) so the live worktree is never touched; fixed cdd/cdd@local identity so it never depends on user git config. Sits under the existing jq guard. Advisory/best-effort: any failure warns and returns 0, never failing the state write. - cdd-worktree-resume: before it finishes, fetch refs/cdd/ and materialize both files into ~/.cdd/handoffs//. Handoff is immutable so it is written only when locally absent; state follows most-advanced-stage-wins (compare .stage enum indices), falling back to write-only-if-absent without jq. Byte-exact extraction. No ref -> resumes exactly as before, with an honest message. - cdd-worktree-done: best-effort delete of refs/cdd/ on origin when it deletes the branch, so the namespace does not accumulate. No slash-command changes (the push funnels through cdd-state). New smoke scripts/ref-sync-assert.sh covers the round-trip, byte-for-byte materialize, both directions of the heuristic, and the no-ref path; wired into CI and CLAUDE.md. Process doc SS2.8/SS2.13, shell-helpers.md, roadmap, and README updated. Co-Authored-By: Claude Opus 4.8 (1M context) --- .github/workflows/template-smoke.yml | 3 + CLAUDE.md | 5 + README.md | 2 +- doc/architecture/shell-helpers.md | 15 ++ .../claude-driven-development.md | 4 +- doc/knowledge_base/roadmap.md | 2 +- scripts/ref-sync-assert.sh | 211 ++++++++++++++++++ tools/cdd-state.sh | 59 ++++- tools/cdd-worktree.sh | 99 +++++++- 9 files changed, 386 insertions(+), 14 deletions(-) create mode 100755 scripts/ref-sync-assert.sh diff --git a/.github/workflows/template-smoke.yml b/.github/workflows/template-smoke.yml index ba61ad1..417348b 100644 --- a/.github/workflows/template-smoke.yml +++ b/.github/workflows/template-smoke.yml @@ -34,6 +34,9 @@ jobs: - name: Worktree-resume smoke (local bare repo as origin) run: ./scripts/worktree-resume-assert.sh + - name: Task-ref sync smoke (refs/cdd/ round-trip, local bare origin) + run: ./scripts/ref-sync-assert.sh + - name: Configure git identity for the scaffold commit run: | git config --global user.email "ci@example.com" diff --git a/CLAUDE.md b/CLAUDE.md index 839bd8c..c5bf14b 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -58,6 +58,11 @@ bash -n demo/setup.sh demo/teardown.sh demo/lib.sh # local bare repo as `origin`; a stubbed `claude` guards that it is never launched. ./scripts/worktree-resume-assert.sh +# Task-ref sync: seed/set on "machine A" push refs/cdd/ to a local bare +# `origin`; resume on "machine B" materializes the handoff + state (most-advanced +# wins); the no-ref path still resumes cleanly. +./scripts/ref-sync-assert.sh + # End-to-end smoke: bootstrap into a tmpdir and run the assertion script. rm -rf /tmp/cdd-smoke && mkdir -p /tmp/cdd-smoke ./tools/bootstrap-cdd-project.sh --name "Demo Project" \ diff --git a/README.md b/README.md index 1778b4a..a6a6e60 100644 --- a/README.md +++ b/README.md @@ -96,7 +96,7 @@ curl -fsSL https://raw.githubusercontent.com/drabaioli/cdd/main/tools/cdd-worktr && bash ~/.cdd/tools/cdd-worktree.sh install ``` -Either form wires `~/.bashrc` and `~/.zshrc` (idempotent); open a new shell afterwards. It spins up and tears down the per-task git worktree that an implementation session runs in, and `cdd-worktree-resume []` recreates that worktree on a second machine — tracking the existing remote branch, no handoff needed — so a task started elsewhere can be picked up to run `/cdd-process-pr`, `/cdd-merge-base`, or `/cdd-pre-pr`. +Either form wires `~/.bashrc` and `~/.zshrc` (idempotent); open a new shell afterwards. It spins up and tears down the per-task git worktree that an implementation session runs in, and `cdd-worktree-resume []` recreates that worktree on a second machine — tracking the existing remote branch — so a task started elsewhere can be picked up to run `/cdd-process-pr`, `/cdd-merge-base`, or `/cdd-pre-pr`. The task's handoff and state record ride along too, synced through a per-task git ref (advisory — resume still works without them). ## Questions? diff --git a/doc/architecture/shell-helpers.md b/doc/architecture/shell-helpers.md index 11fb706..7467786 100644 --- a/doc/architecture/shell-helpers.md +++ b/doc/architecture/shell-helpers.md @@ -68,3 +68,18 @@ The implementation session has no command file, so its two `cdd-state set` calls ## Resume discovery (`cdd-worktree-resume`) The no-argument discovery mode fetches with `--prune`, so remote-tracking refs for branches deleted on the remote (as GitHub does when a PR merges) drop out before the listing. What remains — the default branch plus the feature branches still live on the remote, minus those already checked out locally — is exactly the resumable set, whether or not a branch has a PR yet. + +## Task-ref sync (`refs/cdd/`) + +The handoff (`.md`) and state record (`.state.json`) are out-of-tree, per-user files that would otherwise stay on the machine that created them. To carry them to a machine that resumes the task, `cdd-state` pushes them to a per-task ref and `cdd-worktree-resume` fetches and materializes them. The whole path is advisory: every step is best-effort and any failure (no `origin`, offline, missing `jq`, no ref on the remote) warns and continues — a resume with no ref behaves exactly as it did before the sync existed. + +**Ref layout.** One ref per task at `refs/cdd/` on `origin`. The two files are bundled into a git *tree* under stable in-tree names — `handoff.md` and `state.json`, decoupled from the branch-named on-disk files — and the tree is wrapped in a **parentless (orphan) commit** that the ref points at. This is the side-ref-of-commits pattern git uses for `refs/stash` and `refs/notes/*`; a commit (rather than a bare tree/blob ref) is chosen because commits are push/fetch's native case and avoid cross-version transport surprises, and the commit can carry a message. `git notes` was rejected because it anchors metadata to a commit and so must chase the moving branch tip; the branch-keyed ref does not. + +**Push (in `cdd-state`, on `seed` and `set`).** The state helper is the transition funnel, so folding the push in there means no slash command has to remember it and it cannot drift. It sits under `cdd-state`'s existing `jq` guard, so a machine without `jq` skips the whole thing. The ref is built with plumbing only — `git hash-object -w` each file that exists, `git mktree`, `git commit-tree` — so it never touches the index or working tree of the live worktree. The commit uses a fixed `cdd`/`cdd@local` author+committer identity, so it neither depends on nor fails from an unset user git identity. The push is a force-push (`git push --force origin :refs/cdd/`): the metadata is advisory, latest-wins, so whichever machine last advanced the task holds the truth. A fresh orphan commit per push is fine precisely because of the force-push — the commit SHA is never load-bearing (the round-trip smoke asserts on file contents, not hashes). `seed` is what first lands the immutable handoff `.md`; each `set` refreshes the `.json`. + +**Fetch + materialize (in `cdd-worktree-resume`, before it finishes).** After the worktree is created, it fetches `refs/cdd/` (leaving the commit at `FETCH_HEAD`) and, if present, extracts the two blobs into `~/.cdd/handoffs//`. Extraction streams `git show FETCH_HEAD:` straight to a temp file then `mv`s it into place, so bytes are preserved exactly (no command-substitution newline mangling) and the write is atomic. The two files reconcile differently: + +- **Handoff `.md`** is immutable after seed, so it is materialized only when absent locally — a present local handoff is never clobbered. +- **State `.json`** follows **most-advanced-stage wins**: the `.stage` of the ref's record and the local record are mapped to their index in the lifecycle enum (least → most advanced), and the ref overwrites local only when it is strictly further along; otherwise local is kept. Absent local → take the ref; absent ref blob → keep local; unparseable stage → treated as least advanced. When `jq` is unavailable the comparison can't run, so it falls back to write-only-if-absent (never clobber). The enum order is mirrored in `cdd-worktree.sh` (`cdd-worktree-stage-index`) with `cdd-state.sh`'s `cdd-state-stages` as the source of truth — a small, deliberate duplication because the two helpers are separate self-installing files. + +**Cleanup (in `cdd-worktree-done`).** When `done` deletes the branch, it also best-effort deletes `refs/cdd/` on `origin` (`git push origin --delete`), so the namespace does not accumulate. Like everything else here, a failed delete warns and never blocks the teardown. diff --git a/doc/knowledge_base/claude-driven-development.md b/doc/knowledge_base/claude-driven-development.md index e7e3dbd..0af6475 100644 --- a/doc/knowledge_base/claude-driven-development.md +++ b/doc/knowledge_base/claude-driven-development.md @@ -100,7 +100,7 @@ A single, project-independent bash helper provides four commands — the same sc - `cdd-worktree `, creates a worktree for `` and launches Claude Code in plan mode in it, with the suggested first prompt already submitted. Requires a handoff file to exist. - `cdd-worktree-done`, run from a feature worktree once the PR has landed or the branch is being abandoned: returns to the default branch, removes the worktree, resolves the branch, and deletes the handoff — and its sibling state record (§2.13) — iff the branch was deleted. - `cdd-worktree-list`, lists active handoffs with worktree/branch/PR status, highlighting stale entries. -- `cdd-worktree-resume []`, picks up a task started on another machine that has only a clone of the repo: recreates a worktree tracking an **existing remote branch**, ready for a resume-side command (`/cdd-process-pr`, `/cdd-merge-base`, `/cdd-pre-pr`); with no argument it lists the resumable remote branches. Its scope is worktree and branch recreation only: the handoff (§2.6) and state record (§2.13) stay local per machine — cross-machine *sync* of those two artifacts is separate future work (issue #22). +- `cdd-worktree-resume []`, picks up a task started on another machine that has only a clone of the repo: recreates a worktree tracking an **existing remote branch**, ready for a resume-side command (`/cdd-process-pr`, `/cdd-merge-base`, `/cdd-pre-pr`); with no argument it lists the resumable remote branches. If the originating machine synced them, it also fetches and materializes the handoff (§2.6) and state record (§2.13) from a per-task ref before it finishes — advisory and best-effort, so a task with no synced ref resumes exactly as before (the resume-side commands read PR/branch state from `git`/`gh`, not the handoff). The sync mechanics live in `doc/architecture/shell-helpers.md`. The helper installs itself once per machine (`tools/cdd-worktree.sh install`) to a stable home that does not depend on a live CDD checkout; after that the commands work in every CDD project — including ones bootstrapped later — with no per-project setup. It is a machine-global toolchain dependency, like `git` or `gh`: newest wins, install idempotent, always from latest `main` — never pinned per project, since pinning would reintroduce the very conflict a single shared helper avoids. Its contract with projects is deliberately tiny and grows only additively: the command names above plus the `~/.cdd/handoffs//.md` layout; when that shared state must evolve, the change ships as a one-shot migration inside `install`, re-homing every project at once. The install and dispatch mechanics live in `doc/architecture/shell-helpers.md` and the script's own comments. @@ -150,7 +150,7 @@ A practice moves from expected to enforced in the same change that lands its mec A small JSON sibling of the handoff (§2.6) — same directory, same `` basename, same branch-scoped ephemeral lifecycle — recording where a task sits in its lifecycle (`stage`), its PR number once one exists, and, append-only, the chain of Claude Code sessions that have worked it, so a session can be found and resumed (`claude --resume `) without grepping shell history. -The record is **advisory** and **local**: a consumer that finds it missing or stale falls back to inferring state from handoffs, branches, and `gh`; a writer that finds it missing does not fabricate one (only `/cdd-next-step` seeds it); and it is a cache of work on *this* machine, not a cross-machine transfer mechanism (recreating a worktree elsewhere is `cdd-worktree-resume`, §2.8; syncing the record itself is future work, issue #22). Writes go through the `cdd-state` helper (§2.8), which keeps them atomic and well-formed and no-ops rather than failing the workflow. The slash commands call it at their stage transitions; the implementation session's calls are driven by a standing instruction in the handoff (§3.3). The schema and the stage-to-writer mapping live in `doc/architecture/shell-helpers.md`. +The record is **advisory**: a consumer that finds it missing or stale falls back to inferring state from handoffs, branches, and `gh`, and a writer that finds it missing does not fabricate one (only `/cdd-next-step` seeds it). It syncs across machines: every write also pushes the handoff and record to a per-task ref, which `cdd-worktree-resume` (§2.8) materializes on the picking-up machine — best-effort, so it degrades to purely local when there is no remote to reach. Writes go through the `cdd-state` helper (§2.8), which keeps them atomic and well-formed and no-ops rather than failing the workflow. The slash commands call it at their stage transitions; the implementation session's calls are driven by a standing instruction in the handoff (§3.3). The schema and the stage-to-writer mapping live in `doc/architecture/shell-helpers.md`. ## 3. Lifecycle diff --git a/doc/knowledge_base/roadmap.md b/doc/knowledge_base/roadmap.md index 070d8d4..27bc815 100644 --- a/doc/knowledge_base/roadmap.md +++ b/doc/knowledge_base/roadmap.md @@ -179,6 +179,6 @@ Give each task a machine-readable record of where it sits in its lifecycle and w - [ ] Harden the one outcome transition a tool call owns: a `PostToolUse` hook on `gh pr create` that parses the PR number and writes `pr_open`/`pr=NN` mechanically (`cdd-state` as the hook target), removing the model-remembering dependency. (A `UserPromptSubmit` hook fires deterministically on every `/cdd-*` call, but only at invocation — it can stamp "stage started", not outcomes like `checks_passed` or the PR number, which stay model-driven via `cdd-state set`.) - [ ] Consume the record: teach the `cdd-dash` dashboard to read `stage`/`sessions` instead of inferring task state. (`cdd-worktree-list` already infers worktree/branch/PR status fine and does not need the record — fold in only if a concrete need appears.) - [x] Multi-machine resume — worktree + branch (issue #22): `cdd-worktree-resume []` recreates a worktree on an existing remote branch (no handoff required, discovery mode when no branch given), ready for a resume-side command. — `tools/cdd-worktree.sh`, process doc §2.8/§2.13, `scripts/worktree-resume-assert.sh` (new) + CI step, README, both `CLAUDE.md` workflow sections. -- [ ] Multi-machine resume — handoff + state sync (issue #22): the resume above deliberately does **not** transfer the handoff (`.md`) or state record (`.state.json`); they stay local per machine. To carry them across, the chosen direction is a per-task ref namespace `refs/cdd/` holding the handoff + state JSON as a blob/tree, pushed by the helper/commands at lifecycle transitions and fetched + materialized by `cdd-worktree-resume` before launch, degrading gracefully when no ref exists. git notes (`refs/notes/cdd`) is the standard alternative but anchors metadata to a commit, so it must chase the moving branch tip — the branch-keyed ref avoids that. Committing these files into the work tree is rejected: they are deliberately out-of-tree, per-user artifacts. +- [x] Multi-machine resume — handoff + state sync (issue #22): the handoff (`.md`) and state record (`.state.json`) now sync across machines via a per-task ref `refs/cdd/` — a tree of the two files wrapped in a parentless commit, force-pushed (latest-wins) automatically inside `cdd-state` on `seed`/`set`, and fetched + materialized by `cdd-worktree-resume` before it finishes. Materialize reconciles by most-advanced-stage-wins (the handoff being immutable is never clobbered); `cdd-worktree-done` best-effort deletes the remote ref. Advisory end-to-end — no `origin`/offline/missing-`jq`/no-ref all warn-and-continue, so a resume with no ref behaves exactly as before. No slash-command changes (the push funnels through `cdd-state`). git notes was rejected (anchors to a commit, chases the moving tip). — `tools/cdd-state.sh` (push on seed/set), `tools/cdd-worktree.sh` (fetch/materialize in resume, remote-ref cleanup in done), process doc §2.8/§2.13, `doc/architecture/shell-helpers.md`, `scripts/ref-sync-assert.sh` (new) + CI step, README, CLAUDE.md build/workflow sections. **Milestone:** a task's lifecycle stage and its working sessions are recorded as data and surfaced by CDD tooling, not reconstructed by inference. diff --git a/scripts/ref-sync-assert.sh b/scripts/ref-sync-assert.sh new file mode 100755 index 0000000..0433c37 --- /dev/null +++ b/scripts/ref-sync-assert.sh @@ -0,0 +1,211 @@ +#!/usr/bin/env bash +# End-to-end smoke for the per-task ref sync (issue #22) against a local bare repo. +# +# The real cross-machine flow is hard to exercise in CI, so this stands in a local +# `git init --bare` for `origin` and two clones with separate $HOME dirs as +# "machine A" and "machine B". It sources both helpers (tools/cdd-state.sh and +# tools/cdd-worktree.sh) and asserts: +# - `cdd-state seed`/`set` on machine A push refs/cdd/ to origin +# - `cdd-worktree-resume ` on a fresh machine B materializes the handoff +# (byte-for-byte) and the state record (at the advanced stage), no `claude` +# - most-advanced-stage wins: a more-advanced ref overwrites a stale local record, +# a more-advanced local record is kept, and a present local handoff is never +# clobbered (it is immutable after seed) +# - the no-ref path still resumes cleanly (materializes nothing, exits 0) +# +# Usage: scripts/ref-sync-assert.sh +# Takes no arguments; it provisions and tears down its own temp tree. Requires jq +# (the push lives under cdd-state's jq guard); without it the test skips (advisory). + +set -euo pipefail + +REPO_ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)" +HELPER_WT="$REPO_ROOT/tools/cdd-worktree.sh" +HELPER_STATE="$REPO_ROOT/tools/cdd-state.sh" + +fail() { echo "FAIL: $*" >&2; exit 1; } +pass() { echo "ok: $*"; } + +[[ -f "$HELPER_WT" ]] || fail "helper not found: $HELPER_WT" +[[ -f "$HELPER_STATE" ]] || fail "helper not found: $HELPER_STATE" + +if ! command -v jq >/dev/null 2>&1; then + echo "skip: jq not available; ref sync is advisory and skips without it" + exit 0 +fi + +WORK="$(mktemp -d)" +trap 'rm -rf "$WORK"' EXIT + +# Isolate from the caller's git identity / signing config; keep runs deterministic. +export GIT_CONFIG_SYSTEM=/dev/null +export GIT_CONFIG_GLOBAL="$WORK/gitconfig" +cat > "$GIT_CONFIG_GLOBAL" <<'EOF' +[user] + name = CDD Smoke + email = smoke@example.com +[init] + defaultBranch = main +[commit] + gpgsign = false +EOF + +DEFAULT_BRANCH="main" +FEATURE="feat_sync" +FEATURE_NO="feat_noref" # pushed to origin, but never gets a refs/cdd/* ref + +# Stub `claude` on PATH as a negative guard: resume must never invoke it. +mkdir -p "$WORK/bin" +export CLAUDE_STUB_LOG="$WORK/claude.log" +cat > "$WORK/bin/claude" <<'EOF' +#!/usr/bin/env bash +echo "claude $*" >> "$CLAUDE_STUB_LOG" +exit 0 +EOF +chmod +x "$WORK/bin/claude" + +# 1. Bare repo standing in for origin, seeded with a default + two feature branches. +git init --bare -q "$WORK/origin.git" +git clone -q "$WORK/origin.git" "$WORK/seed" 2>/dev/null # empty-repo warning is expected +( + cd "$WORK/seed" + echo "# seed" > README.md + git add README.md + git commit -q -m "seed" + git push -q -u origin "$DEFAULT_BRANCH" + git switch -q -c "$FEATURE" + echo "a" > a.txt; git add a.txt; git commit -q -m "feature" + git push -q -u origin "$FEATURE" + git switch -q -c "$FEATURE_NO" "$DEFAULT_BRANCH" + echo "n" > n.txt; git add n.txt; git commit -q -m "no-ref feature" + git push -q -u origin "$FEATURE_NO" +) + +# The handoff dir is $HOME/.cdd/handoffs//, where is the clone's own +# directory basename (derived from the git common dir). Each machine has its own +# HOME and clone, so their repo names differ — the ref is keyed only by branch. +handoff_dir() { printf '%s/.cdd/handoffs/%s' "$1" "$(basename "$2")"; } + +# Run cdd-state in a clone with a given HOME. $1=clone, $2=HOME, then the command. +run_state() { + local dir="$1" home="$2"; shift 2 + ( + cd "$dir" + # shellcheck disable=SC2030,SC2031 # per-subshell HOME/PATH isolation is intended + export HOME="$home" PATH="$WORK/bin:$PATH" + # shellcheck source=/dev/null + source "$HELPER_STATE" + cdd-state "$@" + ) +} + +# Run cdd-worktree-resume in a clone with a given HOME. $1=clone, $2=HOME, $3=branch. +run_resume() { + local dir="$1" home="$2" branch="$3" + ( + cd "$dir" + # shellcheck disable=SC2030,SC2031 # per-subshell HOME/PATH isolation is intended + export HOME="$home" PATH="$WORK/bin:$PATH" + # shellcheck source=/dev/null + source "$HELPER_WT" + cdd-worktree-resume "$branch" + ) +} + +# 2. Machine A: seed + advance, which push refs/cdd/$FEATURE to origin. +HOME_A="$WORK/homeA" +git clone -q "$WORK/origin.git" "$WORK/machineA" +DIR_A="$(handoff_dir "$HOME_A" "$WORK/machineA")" +mkdir -p "$DIR_A" +HANDOFF_A="$DIR_A/$FEATURE.md" +printf '# Task: %s\n\nScoped handoff body.\nNo trailing weirdness.\n' "$FEATURE" > "$HANDOFF_A" + +run_state "$WORK/machineA" "$HOME_A" seed "$FEATURE" >/dev/null 2>&1 \ + || fail "cdd-state seed failed on machine A" +git -C "$WORK/machineA" ls-remote origin "refs/cdd/$FEATURE" | grep -q "refs/cdd/$FEATURE" \ + || fail "seed did not push refs/cdd/$FEATURE to origin" +pass "seed pushed refs/cdd/$FEATURE to origin" + +git -C "$WORK/machineA" switch -q "$FEATURE" +run_state "$WORK/machineA" "$HOME_A" set implementation_done >/dev/null 2>&1 \ + || fail "cdd-state set failed on machine A" +# Sanity: local record advanced. +[[ "$(jq -r '.stage' "$DIR_A/$FEATURE.state.json")" == "implementation_done" ]] \ + || fail "machine A state did not advance to implementation_done" +pass "set advanced the state and refreshed the ref" + +# 3. Machine B: fresh clone materializes handoff (byte-for-byte) + advanced state. +HOME_B="$WORK/homeB" +git clone -q "$WORK/origin.git" "$WORK/machineB" +DIR_B="$(handoff_dir "$HOME_B" "$WORK/machineB")" +: > "$CLAUDE_STUB_LOG" +set +e +run_resume "$WORK/machineB" "$HOME_B" "$FEATURE" >/dev/null 2>&1 +rc=$? +set -e +[[ "$rc" -eq 0 ]] || fail "resume on machine B exited $rc" +[[ -f "$DIR_B/$FEATURE.md" ]] || fail "resume did not materialize the handoff" +cmp -s "$HANDOFF_A" "$DIR_B/$FEATURE.md" \ + || fail "materialized handoff differs from machine A's (not byte-for-byte)" +[[ "$(jq -r '.stage' "$DIR_B/$FEATURE.state.json")" == "implementation_done" ]] \ + || fail "materialized state is not at the advanced stage implementation_done" +[[ ! -s "$CLAUDE_STUB_LOG" ]] || fail "resume must not launch claude" +pass "resume materialized handoff byte-for-byte and the advanced state, no claude" + +# 4. Most-advanced wins (ref ahead): a stale local record is overwritten, and a +# pre-existing local handoff is preserved (immutable after seed). +HOME_C="$WORK/homeC" +git clone -q "$WORK/origin.git" "$WORK/machineC" +DIR_C="$(handoff_dir "$HOME_C" "$WORK/machineC")" +mkdir -p "$DIR_C" +printf 'LOCAL HANDOFF — must be preserved\n' > "$DIR_C/$FEATURE.md" +jq -n '{schema_version:1, branch:"'"$FEATURE"'", stage:"plan_approved", pr:null, sessions:[]}' \ + > "$DIR_C/$FEATURE.state.json" +set +e +run_resume "$WORK/machineC" "$HOME_C" "$FEATURE" >/dev/null 2>&1 +rc=$? +set -e +[[ "$rc" -eq 0 ]] || fail "resume on machine C exited $rc" +[[ "$(jq -r '.stage' "$DIR_C/$FEATURE.state.json")" == "implementation_done" ]] \ + || fail "stale local state (plan_approved) should have been overwritten by the ref" +grep -q "LOCAL HANDOFF" "$DIR_C/$FEATURE.md" \ + || fail "present local handoff must not be clobbered by the ref" +pass "ref-ahead overwrites stale local state, preserves local handoff" + +# 5. Most-advanced wins (local ahead): a more-advanced local record is kept. +HOME_D="$WORK/homeD" +git clone -q "$WORK/origin.git" "$WORK/machineD" +DIR_D="$(handoff_dir "$HOME_D" "$WORK/machineD")" +mkdir -p "$DIR_D" +jq -n '{schema_version:1, branch:"'"$FEATURE"'", stage:"addressed", pr:7, sessions:[]}' \ + > "$DIR_D/$FEATURE.state.json" +set +e +run_resume "$WORK/machineD" "$HOME_D" "$FEATURE" >/dev/null 2>&1 +rc=$? +set -e +[[ "$rc" -eq 0 ]] || fail "resume on machine D exited $rc" +[[ "$(jq -r '.stage' "$DIR_D/$FEATURE.state.json")" == "addressed" ]] \ + || fail "more-advanced local state (addressed) should have been kept" +pass "local-ahead state record is kept over a less-advanced ref" + +# 6. No-ref path: a branch with no refs/cdd/* still resumes cleanly, materializes +# nothing, and exits 0. +HOME_E="$WORK/homeE" +git clone -q "$WORK/origin.git" "$WORK/machineE" +DIR_E="$(handoff_dir "$HOME_E" "$WORK/machineE")" +: > "$CLAUDE_STUB_LOG" +set +e +out="$(run_resume "$WORK/machineE" "$HOME_E" "$FEATURE_NO" 2>&1)" +rc=$? +set -e +[[ "$rc" -eq 0 ]] || fail "no-ref resume exited $rc (expected 0)" +[[ ! -f "$DIR_E/$FEATURE_NO.state.json" ]] \ + || fail "no-ref resume must not materialize a state record" +[[ ! -f "$DIR_E/$FEATURE_NO.md" ]] \ + || fail "no-ref resume must not materialize a handoff" +grep -q "No synced task ref" <<<"$out" \ + || fail "no-ref resume should print the honest no-transfer message" +[[ ! -s "$CLAUDE_STUB_LOG" ]] || fail "no-ref resume must not launch claude" +pass "no-ref path resumes cleanly without materializing anything" + +echo "all ref-sync smoke checks passed" diff --git a/tools/cdd-state.sh b/tools/cdd-state.sh index f8feb4d..b7f8870 100755 --- a/tools/cdd-state.sh +++ b/tools/cdd-state.sh @@ -42,6 +42,10 @@ # `dir` on a session entry is the worktree root the session ran in (from # `git rev-parse --show-toplevel`): the natural `cd` target for `claude --resume`. # +# Both `seed` and `set` also sync the handoff + record to a per-task ref +# `refs/cdd/` on origin (best-effort, advisory), so a resume on another +# machine can materialize them; see cdd-worktree-resume and shell-helpers.md. +# # Stages (a single enum; the record carries no separate status): # scoped plan_approved implementation_done merged checks_passed pr_open addressed @@ -67,6 +71,45 @@ cdd-state-write() { printf '%s\n' "$content" >"$tmp" && mv -f "$tmp" "$dest" } +# Sync the handoff + state record to a per-task ref refs/cdd/ on origin, so a +# resume on another machine can materialize them (see cdd-worktree-resume). Bundles +# whichever of the two files exist into a git tree (stable in-tree names handoff.md / +# state.json, decoupled from the branch-named on-disk files), wraps it in a parentless +# commit, and force-pushes (advisory, latest-wins). Best-effort end-to-end: no origin, +# offline, a missing object, or a rejected push warns once and returns 0 — it must +# never fail the state write that called it. Uses plumbing only (hash-object/mktree/ +# commit-tree), so it never touches the index or working tree. The commit uses a fixed +# cdd/cdd@local identity so it never depends on (or fails from) an unset user git +# identity; the SHA is irrelevant under force-push. See doc/architecture/shell-helpers.md. +cdd-state-push-ref() { + local handoff_md="$1" state_json="$2" branch="$3" + local entries="" blob + if [[ -f "$handoff_md" ]]; then + blob="$(git hash-object -w "$handoff_md" 2>/dev/null)" \ + || { echo "cdd-state: could not hash handoff; skipping ref sync (advisory)." >&2; return 0; } + entries+="100644 blob ${blob}"$'\t'"handoff.md"$'\n' + fi + if [[ -f "$state_json" ]]; then + blob="$(git hash-object -w "$state_json" 2>/dev/null)" \ + || { echo "cdd-state: could not hash state; skipping ref sync (advisory)." >&2; return 0; } + entries+="100644 blob ${blob}"$'\t'"state.json"$'\n' + fi + [[ -z "$entries" ]] && return 0 + local tree commit + tree="$(printf '%s' "$entries" | git mktree 2>/dev/null)" \ + || { echo "cdd-state: git mktree failed; skipping ref sync (advisory)." >&2; return 0; } + commit="$(GIT_AUTHOR_NAME=cdd GIT_AUTHOR_EMAIL=cdd@local \ + GIT_COMMITTER_NAME=cdd GIT_COMMITTER_EMAIL=cdd@local \ + git commit-tree "$tree" -m "cdd: sync ${branch}" 2>/dev/null)" \ + || { echo "cdd-state: git commit-tree failed; skipping ref sync (advisory)." >&2; return 0; } + if git push --force origin "${commit}:refs/cdd/${branch}" 2>/dev/null; then + echo "Synced task ref: refs/cdd/${branch}" + else + echo "cdd-state: could not push refs/cdd/${branch} (no origin/offline?); state stays local (advisory)." >&2 + fi + return 0 +} + cdd-state() { if ! command -v jq >/dev/null 2>&1; then echo "cdd-state: jq not found; skipping state update (advisory)." >&2 @@ -101,8 +144,11 @@ cdd-state() { --arg branch "$branch" \ --argjson sessions "$sessions" \ '{schema_version: $v, branch: $branch, stage: "scoped", pr: null, sessions: $sessions}')" || return 1 - cdd-state-write "${dir}/${branch}.state.json" "$content" \ - && echo "Seeded state: ${dir}/${branch}.state.json" + if cdd-state-write "${dir}/${branch}.state.json" "$content"; then + echo "Seeded state: ${dir}/${branch}.state.json" + # Land the handoff .md (immutable after seed) and the fresh state on the ref. + cdd-state-push-ref "${dir}/${branch}.md" "${dir}/${branch}.state.json" "$branch" + fi ;; set) local stage="$1"; shift 2>/dev/null @@ -144,8 +190,13 @@ cdd-state() { --arg sid "$sid" \ --arg dir "$toplevel" \ "$filter" "$file")" || { echo "cdd-state: failed to update $file" >&2; return 1; } - cdd-state-write "$file" "$content" \ - && echo "State: $(basename "$file") -> $stage${pr:+ (pr #$pr)}" + if cdd-state-write "$file" "$content"; then + echo "State: $(basename "$file") -> $stage${pr:+ (pr #$pr)}" + # Refresh the state .json on the ref (bundling the handoff .md if present). + # $file is /.state.json → strip the suffix for branch/handoff. + local base="${file%.state.json}" + cdd-state-push-ref "${base}.md" "$file" "$(basename "$base")" + fi ;; install|"") cdd-state-install "$@" diff --git a/tools/cdd-worktree.sh b/tools/cdd-worktree.sh index 76c445d..989f64e 100755 --- a/tools/cdd-worktree.sh +++ b/tools/cdd-worktree.sh @@ -210,6 +210,11 @@ cdd-worktree-done() { if (( branch_deleted )); then [[ -f "$handoff" ]] && rm "$handoff" && echo "Removed handoff: $handoff" [[ -f "$state_file" ]] && rm "$state_file" && echo "Removed state: $state_file" + # Best-effort: drop the per-task sync ref on origin so refs/cdd/* doesn't + # accumulate. Advisory — a failed delete (no such ref, offline) never blocks. + if git push origin --delete "refs/cdd/$branch" 2>/dev/null; then + echo "Removed remote task ref: refs/cdd/$branch" + fi else [[ -f "$handoff" ]] && echo "Kept handoff: $handoff" [[ -f "$state_file" ]] && echo "Kept state: $state_file" @@ -293,12 +298,91 @@ cdd-worktree-list() { done } +# Ordered CDD lifecycle stages, least → most advanced. Source of truth is +# cdd-state.sh's `cdd-state-stages`; mirrored here by hand (the two helpers are +# separate self-installing files). Prints the index of $1, or -1 when unknown. +cdd-worktree-stage-index() { + local stage="$1" i=0 s + for s in scoped plan_approved implementation_done merged checks_passed pr_open addressed; do + [[ "$s" == "$stage" ]] && { printf '%s\n' "$i"; return 0; } + i=$(( i + 1 )) + done + printf '%s\n' -1 +} + +# Atomically materialize FETCH_HEAD: to , preserving exact bytes +# (streams git show to a temp file in the same dir, then mv -f — no command +# substitution, so no trailing-newline mangling). Returns non-zero if the path is +# absent from the tree or the write fails. +cdd-worktree-extract() { + local intree="$1" dest="$2" tmp + tmp="$(mktemp "${dest}.XXXXXX")" || return 1 + if git show "FETCH_HEAD:${intree}" >"$tmp" 2>/dev/null; then + mv -f "$tmp" "$dest" + else + rm -f "$tmp" + return 1 + fi +} + +# Fetch the per-task ref refs/cdd/ from origin and materialize the handoff + +# state record into ~/.cdd/handoffs//. Advisory and best-effort: returns 0 when +# a ref was found (having printed what it did), 1 when there is no ref (offline, no +# origin, or none pushed) so the caller keeps the honest no-transfer messaging. +# Heuristics: the handoff .md is immutable after seed, so it is written only when +# absent locally; the state .json follows most-advanced-stage-wins (compare .stage +# indices, keep the further-along side), falling back to write-only-if-absent when jq +# is unavailable. Never clobbers a more-advanced local record. See shell-helpers.md. +cdd-worktree-materialize-ref() { + local branch="$1" + git fetch origin "refs/cdd/$branch" 2>/dev/null || return 1 + git rev-parse --quiet --verify FETCH_HEAD >/dev/null 2>&1 || return 1 + + local repo_name dir + repo_name="$(basename "$(dirname "$(git rev-parse --path-format=absolute --git-common-dir)")")" || return 1 + dir="$HOME/.cdd/handoffs/${repo_name}" + mkdir -p "$dir" + local handoff_dest="${dir}/${branch}.md" + local state_dest="${dir}/${branch}.state.json" + + # Handoff: immutable after seed → materialize only when absent locally. + if [[ ! -f "$handoff_dest" ]] && git cat-file -e FETCH_HEAD:handoff.md 2>/dev/null; then + cdd-worktree-extract handoff.md "$handoff_dest" \ + && echo "Materialized handoff: $handoff_dest" + fi + + # State record: most-advanced-stage wins. + if git cat-file -e FETCH_HEAD:state.json 2>/dev/null; then + if [[ ! -f "$state_dest" ]]; then + cdd-worktree-extract state.json "$state_dest" \ + && echo "Materialized state record: $state_dest" + elif command -v jq >/dev/null 2>&1; then + local ref_stage local_stage ref_idx local_idx + ref_stage="$(git show FETCH_HEAD:state.json 2>/dev/null | jq -r '.stage // empty' 2>/dev/null)" + local_stage="$(jq -r '.stage // empty' "$state_dest" 2>/dev/null)" + ref_idx="$(cdd-worktree-stage-index "$ref_stage")" + local_idx="$(cdd-worktree-stage-index "$local_stage")" + if (( ref_idx > local_idx )); then + cdd-worktree-extract state.json "$state_dest" \ + && echo "Updated state record from ref (stage ${local_stage:-?} -> ${ref_stage:-?})." + else + echo "Kept local state record (stage ${local_stage:-?} >= synced ${ref_stage:-?})." + fi + else + echo "Kept local state record (jq unavailable to compare stages)." + fi + fi + return 0 +} + # Recreate a worktree on an EXISTING remote branch so a task started on another # machine can be picked up here. Unlike cdd-worktree, this requires no handoff and -# tracks the remote branch rather than creating a new one. The handoff and state -# record are local-only and are NOT synced across machines (see process doc §2.8); -# the resume-side commands (/cdd-process-pr, /cdd-merge-base, /cdd-pre-pr) read -# PR/branch state from git and gh, not the handoff, so their absence is fine. +# tracks the remote branch rather than creating a new one. If the originating machine +# synced a per-task ref (refs/cdd/, see cdd-state), the handoff and state +# record are fetched and materialized here before launch (most-advanced-stage wins, +# advisory — a missing ref just means nothing to transfer); the resume-side commands +# (/cdd-process-pr, /cdd-merge-base, /cdd-pre-pr) read PR/branch state from git and +# gh, not the handoff, so its absence is still fine. cdd-worktree-resume() { local branch="${1:-}" @@ -405,8 +489,11 @@ cdd-worktree-resume() { echo echo "Resumed worktree for '$branch' on origin/$branch (now in $worktree_path)." - echo "Handoff/state were NOT transferred (they're local to the originating machine)." - echo "Resume-side commands read PR/branch state from git and gh, so this is fine." + # Fetch + materialize the handoff/state from refs/cdd/ if it was synced. + if ! cdd-worktree-materialize-ref "$branch"; then + echo "No synced task ref (refs/cdd/$branch); handoff/state not transferred." + echo "Resume-side commands read PR/branch state from git and gh, so this is fine." + fi echo "Next: start Claude Code here and run /cdd-process-pr, /cdd-merge-base, or /cdd-pre-pr." } From 75458dea38db8cb73c3762b8f83f42be231b1ad4 Mon Sep 17 00:00:00 2001 From: Diego Andres Rabaioli Date: Tue, 21 Jul 2026 22:15:31 +0200 Subject: [PATCH 2/2] Add cdd-worktree-gc to reap finished tasks' handoff/state/ref MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The per-task ref sync leaves cdd-worktree-done as the only reaper of the handoff, state record, and refs/cdd/, so they leak when done never runs, its ref delete fails offline, or a multi-machine resume leaves materialized copies on every machine but the one where done ran. cdd-worktree-gc [--force] sweeps them, reaping ONLY tasks whose PR has merged — a merged PR is the sole trustworthy "done" signal, since a scoped-but-unstarted task's handoff and ref exist before its branch does and so can't be told apart by branch/ref presence. Enumerates local handoffs plus refs/cdd/* so any machine can reap leaked refs and its own orphaned locals; dry-run unless --force; needs gh for PR state. scripts/gc-assert.sh (stubbed gh, local bare origin) + CI step; process doc §2.8, shell-helpers.md, roadmap, README, CLAUDE.md reconciled. Co-Authored-By: Claude Opus 4.8 (1M context) --- .github/workflows/template-smoke.yml | 3 + CLAUDE.md | 7 +- README.md | 4 +- doc/architecture/shell-helpers.md | 8 + .../claude-driven-development.md | 1 + doc/knowledge_base/roadmap.md | 1 + scripts/gc-assert.sh | 142 ++++++++++++++++++ tools/cdd-worktree.sh | 107 ++++++++++++- 8 files changed, 265 insertions(+), 8 deletions(-) create mode 100755 scripts/gc-assert.sh diff --git a/.github/workflows/template-smoke.yml b/.github/workflows/template-smoke.yml index 417348b..1135e05 100644 --- a/.github/workflows/template-smoke.yml +++ b/.github/workflows/template-smoke.yml @@ -37,6 +37,9 @@ jobs: - name: Task-ref sync smoke (refs/cdd/ round-trip, local bare origin) run: ./scripts/ref-sync-assert.sh + - name: Worktree GC smoke (reap merged tasks, keep scoped ones; stubbed gh) + run: ./scripts/gc-assert.sh + - name: Configure git identity for the scaffold commit run: | git config --global user.email "ci@example.com" diff --git a/CLAUDE.md b/CLAUDE.md index c5bf14b..8bb8eb1 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -63,6 +63,10 @@ bash -n demo/setup.sh demo/teardown.sh demo/lib.sh # wins); the no-ref path still resumes cleanly. ./scripts/ref-sync-assert.sh +# Worktree GC: with a stubbed `gh`, cdd-worktree-gc reaps a merged task's local +# handoff/state + remote refs/cdd/, but keeps a scoped-but-unstarted one. +./scripts/gc-assert.sh + # End-to-end smoke: bootstrap into a tmpdir and run the assertion script. rm -rf /tmp/cdd-smoke && mkdir -p /tmp/cdd-smoke ./tools/bootstrap-cdd-project.sh --name "Demo Project" \ @@ -74,7 +78,7 @@ rm -rf /tmp/cdd-demo-smoke demo/setup.sh mdr_demo_99 --base /tmp/cdd-demo-smoke --local-only ``` -The `template-smoke` GitHub Actions workflow runs the same checks on every PR: shellcheck, the command-set drift check, the prompt-seam check, the worktree-helper install smoke, the end-to-end smoke, and the demo seed-overlay step. +The `template-smoke` GitHub Actions workflow runs the same checks on every PR: shellcheck, the command-set drift check, the prompt-seam check, the worktree-helper install smoke, the task-ref sync smoke, the worktree GC smoke, the end-to-end smoke, and the demo seed-overlay step. When `/cdd-pre-pr` runs in this repo, the "build / format / lint / test" gates collapse into the checks above plus a doc reconciliation pass. @@ -113,4 +117,5 @@ This project uses CDD on itself. Every CDD session is a fresh context doing exac - **When main has advanced under a feature branch** (merge session): run `/cdd-merge-base` in a fresh context on the feature branch. - **Before opening a PR** (pre-PR session): run `/cdd-pre-pr` in a fresh context to verify the process doc and template are consistent and the roadmap reflects what landed; it auto-commits its own reconciliation edits (local, no push) and ends with an opt-in step to open the PR (adding `Closes #NN` when the branch carries the `gh_issue_NN` token). - **When a PR review leaves comments** (PR-review session): run `/cdd-process-pr` in a fresh context on the feature branch. +- **To clean up finished tasks' artifacts** (maintenance, not a session): run `cdd-worktree-gc` from the main worktree. It reaps the handoff, state record, and synced `refs/cdd/` of any task whose PR has merged — the backstop for `cdd-worktree-done` never running or its ref cleanup failing across machines. Dry-run by default; `--force` to delete. - Keep the process doc, template, and roadmap consistent as part of every change. Process-first, then template. diff --git a/README.md b/README.md index a6a6e60..5b0599b 100644 --- a/README.md +++ b/README.md @@ -88,7 +88,7 @@ CDD ships seven slash commands, all prefixed `cdd-` so they autocomplete as a gr | `/cdd‑retrofit` | Install or upgrade CDD in an existing project. | | `/cdd‑quick‑create` | Produce a one-off self-contained deliverable (script + README), no project substrate. | -`cdd-worktree` (and its companions `cdd-worktree-done`, `cdd-worktree-list`, and `cdd-worktree-resume`) is a **shell helper**, not a slash command. It's a single project-independent script — a machine-global toolchain dependency, like `git` or `gh` — that you install once and that then works in every CDD project. From a CDD repo checkout: `tools/cdd-worktree.sh install`. On a fresh machine with only a downstream project (no CDD repo), one command fetches and installs it: +`cdd-worktree` (and its companions `cdd-worktree-done`, `cdd-worktree-list`, `cdd-worktree-resume`, and `cdd-worktree-gc`) is a **shell helper**, not a slash command. It's a single project-independent script — a machine-global toolchain dependency, like `git` or `gh` — that you install once and that then works in every CDD project. From a CDD repo checkout: `tools/cdd-worktree.sh install`. On a fresh machine with only a downstream project (no CDD repo), one command fetches and installs it: ```bash curl -fsSL https://raw.githubusercontent.com/drabaioli/cdd/main/tools/cdd-worktree.sh \ @@ -96,7 +96,7 @@ curl -fsSL https://raw.githubusercontent.com/drabaioli/cdd/main/tools/cdd-worktr && bash ~/.cdd/tools/cdd-worktree.sh install ``` -Either form wires `~/.bashrc` and `~/.zshrc` (idempotent); open a new shell afterwards. It spins up and tears down the per-task git worktree that an implementation session runs in, and `cdd-worktree-resume []` recreates that worktree on a second machine — tracking the existing remote branch — so a task started elsewhere can be picked up to run `/cdd-process-pr`, `/cdd-merge-base`, or `/cdd-pre-pr`. The task's handoff and state record ride along too, synced through a per-task git ref (advisory — resume still works without them). +Either form wires `~/.bashrc` and `~/.zshrc` (idempotent); open a new shell afterwards. It spins up and tears down the per-task git worktree that an implementation session runs in, and `cdd-worktree-resume []` recreates that worktree on a second machine — tracking the existing remote branch — so a task started elsewhere can be picked up to run `/cdd-process-pr`, `/cdd-merge-base`, or `/cdd-pre-pr`. The task's handoff and state record ride along too, synced through a per-task git ref (advisory — resume still works without them). `cdd-worktree-gc` periodically reaps the handoff, state record, and synced ref of tasks whose PR has merged (dry-run unless `--force`), so those artifacts don't accumulate across machines. ## Questions? diff --git a/doc/architecture/shell-helpers.md b/doc/architecture/shell-helpers.md index 7467786..4e496c7 100644 --- a/doc/architecture/shell-helpers.md +++ b/doc/architecture/shell-helpers.md @@ -83,3 +83,11 @@ The handoff (`.md`) and state record (`.state.json`) are out-of- - **State `.json`** follows **most-advanced-stage wins**: the `.stage` of the ref's record and the local record are mapped to their index in the lifecycle enum (least → most advanced), and the ref overwrites local only when it is strictly further along; otherwise local is kept. Absent local → take the ref; absent ref blob → keep local; unparseable stage → treated as least advanced. When `jq` is unavailable the comparison can't run, so it falls back to write-only-if-absent (never clobber). The enum order is mirrored in `cdd-worktree.sh` (`cdd-worktree-stage-index`) with `cdd-state.sh`'s `cdd-state-stages` as the source of truth — a small, deliberate duplication because the two helpers are separate self-installing files. **Cleanup (in `cdd-worktree-done`).** When `done` deletes the branch, it also best-effort deletes `refs/cdd/` on `origin` (`git push origin --delete`), so the namespace does not accumulate. Like everything else here, a failed delete warns and never blocks the teardown. + +## Garbage collection (`cdd-worktree-gc`) + +`cdd-worktree-done` is the primary reaper, but it only fires when a human runs it from the worktree, and its remote-ref delete is a single best-effort attempt. Three gaps leak artifacts: `done` is never run; its ref delete fails while offline and is never retried; or a task resumed on several machines leaves a materialized handoff/state on each, while `done` (run on one machine) cleans only that one. `cdd-worktree-gc` is the periodic sweep that closes them. + +**Reap predicate — merged PR only.** The tempting signal, "the branch no longer exists on `origin`", is wrong: a freshly scoped task has a handoff and a `refs/cdd/` ref *before* `cdd-worktree` ever creates the branch (`cdd-state seed` pushes the ref; the branch reaches `origin` only at first push), so it is byte-for-byte indistinguishable from a merged-and-branch-deleted task by ref or branch presence alone. The only trustworthy "done" signal is a **merged PR** — the same signal `cdd-worktree-done` trusts (`gh pr list --head `). So GC reaps a task iff its PR state is `MERGED`; anything else (open PR, or no PR yet) is treated as in-flight or scoped and left untouched. This makes `gh` a hard dependency for GC: without an authenticated `gh` it cannot tell a finished task from a fresh one, so it reaps nothing and says so. + +**What it enumerates and removes.** Candidates are the union of local handoff/state basenames in `~/.cdd/handoffs//` and the `refs/cdd/*` names from `git ls-remote origin` — so a machine that never held a task's files can still reap that task's leaked remote ref, and a machine holding orphaned local copies can reap them even after the ref is already gone. For each merged candidate it removes whichever of the three artifacts are present (local `.md`, local `.state.json`, remote ref). It is **dry-run by default** — printing `reap … would remove …` / `keep …` lines — and only deletes under `--force`. Being conservative and idempotent, it is safe to run (or schedule) routinely. diff --git a/doc/knowledge_base/claude-driven-development.md b/doc/knowledge_base/claude-driven-development.md index 0af6475..aff55c2 100644 --- a/doc/knowledge_base/claude-driven-development.md +++ b/doc/knowledge_base/claude-driven-development.md @@ -100,6 +100,7 @@ A single, project-independent bash helper provides four commands — the same sc - `cdd-worktree `, creates a worktree for `` and launches Claude Code in plan mode in it, with the suggested first prompt already submitted. Requires a handoff file to exist. - `cdd-worktree-done`, run from a feature worktree once the PR has landed or the branch is being abandoned: returns to the default branch, removes the worktree, resolves the branch, and deletes the handoff — and its sibling state record (§2.13) — iff the branch was deleted. - `cdd-worktree-list`, lists active handoffs with worktree/branch/PR status, highlighting stale entries. +- `cdd-worktree-gc [--force]`, reaps the artifacts of **finished** tasks — the local handoff, the state record (§2.13), and the synced per-task ref (§2.13) — for any task whose PR has merged. It is the backstop for `cdd-worktree-done` never running, its ref cleanup failing offline, or a task resumed on several machines leaving materialized copies behind on all but the one where `done` ran. Deliberately conservative: it reaps only merged tasks (a merged PR is the sole trustworthy "done" signal — a scoped-but-unstarted task's handoff and ref exist *before* its branch does, so ref/branch presence alone cannot tell the two apart), and it is dry-run unless `--force`. - `cdd-worktree-resume []`, picks up a task started on another machine that has only a clone of the repo: recreates a worktree tracking an **existing remote branch**, ready for a resume-side command (`/cdd-process-pr`, `/cdd-merge-base`, `/cdd-pre-pr`); with no argument it lists the resumable remote branches. If the originating machine synced them, it also fetches and materializes the handoff (§2.6) and state record (§2.13) from a per-task ref before it finishes — advisory and best-effort, so a task with no synced ref resumes exactly as before (the resume-side commands read PR/branch state from `git`/`gh`, not the handoff). The sync mechanics live in `doc/architecture/shell-helpers.md`. The helper installs itself once per machine (`tools/cdd-worktree.sh install`) to a stable home that does not depend on a live CDD checkout; after that the commands work in every CDD project — including ones bootstrapped later — with no per-project setup. It is a machine-global toolchain dependency, like `git` or `gh`: newest wins, install idempotent, always from latest `main` — never pinned per project, since pinning would reintroduce the very conflict a single shared helper avoids. Its contract with projects is deliberately tiny and grows only additively: the command names above plus the `~/.cdd/handoffs//.md` layout; when that shared state must evolve, the change ships as a one-shot migration inside `install`, re-homing every project at once. The install and dispatch mechanics live in `doc/architecture/shell-helpers.md` and the script's own comments. diff --git a/doc/knowledge_base/roadmap.md b/doc/knowledge_base/roadmap.md index 27bc815..56d43d7 100644 --- a/doc/knowledge_base/roadmap.md +++ b/doc/knowledge_base/roadmap.md @@ -180,5 +180,6 @@ Give each task a machine-readable record of where it sits in its lifecycle and w - [ ] Consume the record: teach the `cdd-dash` dashboard to read `stage`/`sessions` instead of inferring task state. (`cdd-worktree-list` already infers worktree/branch/PR status fine and does not need the record — fold in only if a concrete need appears.) - [x] Multi-machine resume — worktree + branch (issue #22): `cdd-worktree-resume []` recreates a worktree on an existing remote branch (no handoff required, discovery mode when no branch given), ready for a resume-side command. — `tools/cdd-worktree.sh`, process doc §2.8/§2.13, `scripts/worktree-resume-assert.sh` (new) + CI step, README, both `CLAUDE.md` workflow sections. - [x] Multi-machine resume — handoff + state sync (issue #22): the handoff (`.md`) and state record (`.state.json`) now sync across machines via a per-task ref `refs/cdd/` — a tree of the two files wrapped in a parentless commit, force-pushed (latest-wins) automatically inside `cdd-state` on `seed`/`set`, and fetched + materialized by `cdd-worktree-resume` before it finishes. Materialize reconciles by most-advanced-stage-wins (the handoff being immutable is never clobbered); `cdd-worktree-done` best-effort deletes the remote ref. Advisory end-to-end — no `origin`/offline/missing-`jq`/no-ref all warn-and-continue, so a resume with no ref behaves exactly as before. No slash-command changes (the push funnels through `cdd-state`). git notes was rejected (anchors to a commit, chases the moving tip). — `tools/cdd-state.sh` (push on seed/set), `tools/cdd-worktree.sh` (fetch/materialize in resume, remote-ref cleanup in done), process doc §2.8/§2.13, `doc/architecture/shell-helpers.md`, `scripts/ref-sync-assert.sh` (new) + CI step, README, CLAUDE.md build/workflow sections. +- [x] Garbage-collect finished tasks' artifacts (`cdd-worktree-gc`): the per-task ref sync makes `cdd-worktree-done` the only reaper of the handoff, state record, and `refs/cdd/`, so they leak when `done` never runs, its ref delete fails offline, or a multi-machine resume leaves materialized copies behind on every machine but the one where `done` ran. `cdd-worktree-gc [--force]` sweeps them, reaping **only** tasks whose PR has merged (a merged PR is the sole trustworthy "done" signal — a scoped-but-unstarted task's handoff and ref exist before its branch does, so branch/ref presence can't distinguish them). Enumerates local handoffs ∪ `refs/cdd/*` so any machine can reap leaked refs and its own orphaned locals; dry-run unless `--force`; needs `gh`. — `tools/cdd-worktree.sh` (new command + PATH shim), process doc §2.8, `doc/architecture/shell-helpers.md`, `scripts/gc-assert.sh` (new) + CI step, README, CLAUDE.md. **Milestone:** a task's lifecycle stage and its working sessions are recorded as data and surfaced by CDD tooling, not reconstructed by inference. diff --git a/scripts/gc-assert.sh b/scripts/gc-assert.sh new file mode 100755 index 0000000..f26f5bc --- /dev/null +++ b/scripts/gc-assert.sh @@ -0,0 +1,142 @@ +#!/usr/bin/env bash +# Smoke for cdd-worktree-gc: it reaps a MERGED task's artifacts (local handoff + +# state and the remote refs/cdd/) but never a scoped-but-unstarted one. +# +# Like ref-sync-assert.sh this stands in a local `git init --bare` for origin and +# a clone with its own $HOME. PR state is the reap predicate, so `gh` is stubbed on +# PATH: `gh pr list --head feat_merged ...` reports MERGED, everything else reports +# no PR. It asserts: +# - dry-run: the merged branch is listed as "would remove", the scoped one kept, +# and NOTHING is actually deleted +# - --force: the merged branch's local files + remote ref are gone, while the +# scoped branch's files + ref are untouched +# +# Usage: scripts/gc-assert.sh (provisions and tears down its own temp tree) + +set -euo pipefail + +REPO_ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)" +HELPER_WT="$REPO_ROOT/tools/cdd-worktree.sh" +HELPER_STATE="$REPO_ROOT/tools/cdd-state.sh" + +fail() { echo "FAIL: $*" >&2; exit 1; } +pass() { echo "ok: $*"; } + +[[ -f "$HELPER_WT" ]] || fail "helper not found: $HELPER_WT" +[[ -f "$HELPER_STATE" ]] || fail "helper not found: $HELPER_STATE" + +if ! command -v jq >/dev/null 2>&1; then + echo "skip: jq not available; the ref push under test lives behind cdd-state's jq guard" + exit 0 +fi + +WORK="$(mktemp -d)" +trap 'rm -rf "$WORK"' EXIT + +export GIT_CONFIG_SYSTEM=/dev/null +export GIT_CONFIG_GLOBAL="$WORK/gitconfig" +cat > "$GIT_CONFIG_GLOBAL" <<'EOF' +[user] + name = CDD Smoke + email = smoke@example.com +[init] + defaultBranch = main +[commit] + gpgsign = false +EOF + +MERGED="feat_merged" # has a MERGED PR (per the gh stub) -> reap +SCOPED="feat_scoped" # no PR yet -> keep + +# Stub gh: `auth status` succeeds; `pr list --head feat_merged` reports MERGED, any +# other head reports nothing (empty -> "no PR yet"). Mirrors the --jq shape gc uses. +mkdir -p "$WORK/bin" +cat > "$WORK/bin/gh" <<'EOF' +#!/usr/bin/env bash +case "$1" in + auth) exit 0 ;; + pr) + branch="" + while [[ $# -gt 0 ]]; do + [[ "$1" == "--head" ]] && { branch="$2"; break; } + shift + done + [[ "$branch" == "feat_merged" ]] && echo "MERGED" + exit 0 ;; +esac +exit 0 +EOF +chmod +x "$WORK/bin/gh" + +# Bare origin + a clone that pushes both feature branches (so ls-remote has heads). +git init --bare -q "$WORK/origin.git" +git clone -q "$WORK/origin.git" "$WORK/seed" 2>/dev/null +( + cd "$WORK/seed" + echo "# seed" > README.md; git add README.md; git commit -q -m "seed" + git push -q -u origin main + git switch -q -c "$MERGED"; echo m > m.txt; git add m.txt; git commit -q -m m; git push -q -u origin "$MERGED" + git switch -q -c "$SCOPED" main; echo s > s.txt; git add s.txt; git commit -q -m s; git push -q -u origin "$SCOPED" +) + +HOME_A="$WORK/home" +git clone -q "$WORK/origin.git" "$WORK/machine" +handoff_dir() { printf '%s/.cdd/handoffs/%s' "$HOME_A" "$(basename "$WORK/machine")"; } +DIR="$(handoff_dir)" +mkdir -p "$DIR" + +# Seed both tasks on the machine: writes local handoff + state and pushes refs/cdd/*. +run_state() { + ( + cd "$WORK/machine" + # shellcheck disable=SC2030,SC2031 # per-subshell HOME/PATH isolation is intended + export HOME="$HOME_A" PATH="$WORK/bin:$PATH" + # shellcheck source=/dev/null + source "$HELPER_STATE" + cdd-state "$@" + ) +} +for b in "$MERGED" "$SCOPED"; do + printf '# Task: %s\n\nbody\n' "$b" > "$DIR/$b.md" + run_state seed "$b" >/dev/null 2>&1 || fail "cdd-state seed failed for $b" + git -C "$WORK/machine" ls-remote origin "refs/cdd/$b" | grep -q "refs/cdd/$b" \ + || fail "seed did not push refs/cdd/$b" +done +pass "seeded two tasks (handoff + state + refs/cdd/*) on the machine" + +run_gc() { + ( + cd "$WORK/machine" + # shellcheck disable=SC2030,SC2031 # per-subshell HOME/PATH isolation is intended + export HOME="$HOME_A" PATH="$WORK/bin:$PATH" + # shellcheck source=/dev/null + source "$HELPER_WT" + cdd-worktree-gc "$@" + ) +} + +# 1. Dry-run: merged -> "would remove", scoped -> "keep", and nothing deleted. +out="$(run_gc 2>&1)" || fail "gc dry-run exited non-zero" +grep -q "reap $MERGED (MERGED): would remove" <<<"$out" \ + || fail "dry-run did not mark the merged task for reaping. Output:\n$out" +grep -q "keep $SCOPED" <<<"$out" \ + || fail "dry-run did not keep the scoped task. Output:\n$out" +[[ -f "$DIR/$MERGED.md" && -f "$DIR/$MERGED.state.json" ]] \ + || fail "dry-run must not delete local files" +git -C "$WORK/machine" ls-remote origin "refs/cdd/$MERGED" | grep -q "refs/cdd/$MERGED" \ + || fail "dry-run must not delete the remote ref" +pass "dry-run reports the merged task, keeps the scoped one, deletes nothing" + +# 2. --force: merged artifacts gone; scoped artifacts untouched. +out="$(run_gc --force 2>&1)" || fail "gc --force exited non-zero" +[[ ! -f "$DIR/$MERGED.md" && ! -f "$DIR/$MERGED.state.json" ]] \ + || fail "--force did not remove the merged task's local files" +git -C "$WORK/machine" ls-remote origin "refs/cdd/$MERGED" | grep -q "refs/cdd/$MERGED" \ + && fail "--force did not delete the merged task's remote ref" +[[ -f "$DIR/$SCOPED.md" && -f "$DIR/$SCOPED.state.json" ]] \ + || fail "--force must not touch the scoped task's local files" +git -C "$WORK/machine" ls-remote origin "refs/cdd/$SCOPED" | grep -q "refs/cdd/$SCOPED" \ + || fail "--force must not delete the scoped task's remote ref" +pass "--force reaps the merged task (local + ref), leaves the scoped task intact" + +echo "all gc smoke checks passed" diff --git a/tools/cdd-worktree.sh b/tools/cdd-worktree.sh index 989f64e..7f3d922 100755 --- a/tools/cdd-worktree.sh +++ b/tools/cdd-worktree.sh @@ -62,6 +62,14 @@ # /cdd-pre-pr. With no argument, lists remote # feature branches not already checked out and # prompts for one. Run from the main worktree. +# +# cdd-worktree-gc [--force] +# Reap the artifacts of FINISHED tasks: the local +# handoff + state record and the remote sync ref +# (refs/cdd/) for any task whose PR has +# merged. Conservative — reaps only merged tasks +# (never a scoped-but-unstarted or open-PR one) and +# is dry-run unless --force. Needs gh. # Resolve the repo's default branch from origin's HEAD, falling back to "main". # The remote is assumed to be named "origin" (see template/BOOTSTRAP.md). @@ -298,6 +306,92 @@ cdd-worktree-list() { done } +# Garbage-collect the artifacts of FINISHED tasks: the local handoff + state record +# and the remote sync ref refs/cdd/. This is the safety net for the cleanup +# in cdd-worktree-done never running, its remote-ref delete failing while offline, or +# a task resumed on several machines leaving materialized copies behind on every +# machine but the one where `done` ran. It reaps ONLY tasks whose PR has merged — the +# same signal cdd-worktree-done trusts — so it never touches a task that is merely +# scoped-but-unstarted (the handoff and ref exist before the branch does, §2.6/§2.13) +# or one with an open PR: those are indistinguishable from a finished task by ref or +# branch presence alone, and only the PR state tells them apart. Dry-run by default; +# --force actually deletes. Needs an authenticated gh to read PR state; without it a +# merged task can't be told from a fresh one, so it reaps nothing. See shell-helpers.md. +cdd-worktree-gc() { + local force=0 + case "${1:-}" in + --force|-f) force=1 ;; + "") ;; + *) echo "usage: cdd-worktree-gc [--force]" >&2; return 2 ;; + esac + + if ! command -v gh >/dev/null 2>&1 || ! gh auth status >/dev/null 2>&1; then + echo "cdd-worktree-gc needs an authenticated gh to tell a merged task from a" >&2 + echo "just-scoped one; without it nothing can be safely reaped. Skipping (advisory)." >&2 + return 0 + fi + + local repo_name + repo_name="$(basename "$(dirname "$(git rev-parse --path-format=absolute --git-common-dir)")")" || return 1 + local handoff_dir="$HOME/.cdd/handoffs/${repo_name}" + + # Candidate branches = local handoff/state basenames ∪ remote refs/cdd/* names. + # Track which refs exist on origin so the reap reports and acts accurately. + local -A seen=() has_ref=() + local f branch ref + shopt -s nullglob + for f in "$handoff_dir"/*.md; do seen["$(basename "$f" .md)"]=1; done + for f in "$handoff_dir"/*.state.json; do seen["$(basename "$f" .state.json)"]=1; done + shopt -u nullglob + while IFS= read -r ref; do + [[ -z "$ref" ]] && continue + branch="${ref#refs/cdd/}" + seen["$branch"]=1 + has_ref["$branch"]=1 + done < <(git ls-remote origin 'refs/cdd/*' 2>/dev/null | awk '{print $2}') + + if (( ${#seen[@]} == 0 )); then + echo "No task artifacts found (no local handoffs, no refs/cdd/* on origin)." + return 0 + fi + + local reaped=0 kept=0 pr_state handoff state items joined + for branch in "${!seen[@]}"; do + pr_state="$(gh pr list --head "$branch" --state all --json state \ + --jq '.[0].state // empty' 2>/dev/null)" + if [[ "$pr_state" != "MERGED" ]]; then + kept=$(( kept + 1 )) + echo "keep $branch (${pr_state:-no PR yet} — in-flight or scoped, not reaped)" + continue + fi + + # Merged → finished → reap the local handoff/state and the remote ref. + reaped=$(( reaped + 1 )) + handoff="${handoff_dir}/${branch}.md" + state="${handoff_dir}/${branch}.state.json" + items=() + [[ -f "$handoff" ]] && items+=("handoff") + [[ -f "$state" ]] && items+=("state") + [[ -n "${has_ref[$branch]:-}" ]] && items+=("refs/cdd/$branch") + joined="$(IFS=,; echo "${items[*]}")" + if (( force )); then + [[ -f "$handoff" ]] && rm -f "$handoff" + [[ -f "$state" ]] && rm -f "$state" + [[ -n "${has_ref[$branch]:-}" ]] && git push origin --delete "refs/cdd/$branch" 2>/dev/null + echo "reap $branch (MERGED): removed ${joined:-nothing present}" + else + echo "reap $branch (MERGED): would remove ${joined:-nothing present}" + fi + done + + echo + if (( force )); then + echo "GC complete: reaped $reaped finished task(s), kept $kept." + else + echo "GC dry-run: would reap $reaped finished task(s), keep $kept. Re-run with --force to delete." + fi +} + # Ordered CDD lifecycle stages, least → most advanced. Source of truth is # cdd-state.sh's `cdd-state-stages`; mirrored here by hand (the two helpers are # separate self-installing files). Prints the index of $1, or -1 when unknown. @@ -587,15 +681,18 @@ exit 1 SHIM chmod +x "$bin_dir/$cmd" done - cat > "$bin_dir/cdd-worktree-list" <<'SHIM' + # These two change no cwd, so they get a real source+dispatch shim. + for cmd in cdd-worktree-list cdd-worktree-gc; do + cat > "$bin_dir/$cmd" <&2 ;;