diff --git a/.claude/skills/crypter-change/SKILL.md b/.claude/skills/crypter-change/SKILL.md index cc22ddc1..80f4172b 100644 --- a/.claude/skills/crypter-change/SKILL.md +++ b/.claude/skills/crypter-change/SKILL.md @@ -10,8 +10,9 @@ Carry a requirement from a sentence to a draft pull request whose checks pass. You own the whole run. Building and reviewing happen in the container; you hold the plan, the findings and every CI attempt, which is why the judgement calls are yours. -Run from the root of the main checkout. The container's mounts resolve against it, so a run -started from a worktree writes its plan where the container cannot read it. +Run from the root of a checkout — a worktree does as well as a main one. `/plans` and `/runs` +resolve against it, and the container is named after it, so the one you reach is always the one +reading the plan you wrote. There is one gate: the user approves the plan. Everything after it runs to a green draft pull request, or to a written account of why CI would not take it. @@ -40,43 +41,44 @@ The container needs both mounts, the run directory has to be writable from insid image has to carry the current tooling. Confirm before starting: ```bash -docker exec crypter-pipeline test -d /plans/{run-id} && \ - docker exec crypter-pipeline test -w /runs/{run-id}/findings && \ - docker exec crypter-pipeline test -x /usr/local/bin/crypter-workspace +.devcontainer/pipeline.sh exec -- test -d /plans/{run-id} && \ + .devcontainer/pipeline.sh exec -- test -w /runs/{run-id}/findings && \ + .devcontainer/pipeline.sh exec -- test -x /usr/local/bin/crypter-workspace ``` -The mount checks also settle which checkout the container belongs to: one created against a -different one reaches neither directory. The last check is separate because an older image -passes the first two and then fails at workspace creation with nothing but a missing executable -to go on. All three are `test` because `docker exec` runs a binary and not a shell, so a builtin -like `command -v` exits 127 whether or not the thing it was looking for is there. +`pipeline.sh` resolves the container from the checkout it sits in, so which container you get is +settled by where you are rather than by anything you check. The mount checks prove `/plans` and +`/runs` are both there and that uid 1001 can write the run directory. The last check is separate +because an older image passes the first two and then fails at workspace creation with nothing but +a missing executable to go on. All three are `test` because `docker exec` runs a binary and not a +shell, so a builtin like `command -v` exits 127 whether or not the thing it was looking for is +there. -**Do not `docker start` an exited container to fix any of this.** Mounts and image are fixed -when a container is created, so starting one built from another checkout, or from an older -image, brings back the same wrong container. Bring it up from here instead: +**Do not `docker start` an exited container to fix any of this.** The image is fixed when a +container is created, so starting an old one brings back the old tooling. Bring it up from here +instead: ```bash -docker compose -f .devcontainer/docker-compose.yml up -d --build +.devcontainer/pipeline.sh up ``` -That rebuilds the image and recreates the container against this checkout's mounts. It replaces -any container of the same name, so **ask the user before running it** — theirs may belong to -another checkout and hold work you cannot see. +That rebuilds the image and recreates this checkout's container. It cannot touch another +checkout's. What it does destroy is every workspace under `/work` in *this* container, because +`/work` is the container's own filesystem and not a volume — so **ask the user before running it +if another run may be live here.** -Then make the workspace the container builds in. It is a clone of your repository, taken from -the read-only `/host-git` mount, and it lasts exactly as long as this run: +Then make the workspace the container builds in. It is a clone of the repository taken from +GitHub, and it lasts exactly as long as this run: ```bash -git fetch origin -docker exec crypter-pipeline crypter-workspace create {run-id} +.devcontainer/pipeline.sh exec -- crypter-workspace create {run-id} ``` -Fetch first — the workspace takes its `origin/stable` from yours, so a stale remote-tracking ref -puts the whole run on an old base. A change of your own targets `stable`, which is what `create` -uses when no `--base` is given. **If either fails, stop and say so.** +A change of your own targets `stable`, which is what `create` uses when no `--base` is given. +**If it fails, stop and say so.** -The workspace holds only committed history. Uncommitted work in your checkout is not visible to -the container and never reaches the branch. +The workspace takes `stable` as the repository holds it, so nothing about your checkout — what +it is on, how stale it is, what is uncommitted in it — reaches the branch. ## 1. Plan @@ -88,7 +90,7 @@ It settles the plan with the user itself. **Do not continue until they have appr ## 2. Build ```bash -docker exec -w /work/{run-id} crypter-pipeline \ +.devcontainer/pipeline.sh exec -w /work/{run-id} -- \ claude --permission-mode auto -p "/crypter-devcontainer-implement {run-id} {branch}" ``` @@ -97,7 +99,7 @@ Keep the title and description it reports; `crypter-step-open-pull-request` need ## 3. Examine ```bash -docker exec -w /work/{run-id} crypter-pipeline \ +.devcontainer/pipeline.sh exec -w /work/{run-id} -- \ claude --permission-mode auto -p "/crypter-devcontainer-examine {run-id} {branch} origin/stable /plans/{run-id}/plan.md" ``` @@ -123,7 +125,7 @@ them. Where anything was accepted: ```bash -docker exec -w /work/{run-id} crypter-pipeline \ +.devcontainer/pipeline.sh exec -w /work/{run-id} -- \ claude --permission-mode auto -p "/crypter-devcontainer-remediate {run-id} {branch} /runs/{run-id}/triage.md" ``` @@ -159,7 +161,7 @@ The branch is on the fork and the artifacts are on your disk, so the workspace h to hold: ```bash -docker exec crypter-pipeline crypter-workspace remove {run-id} +.devcontainer/pipeline.sh exec -- crypter-workspace remove {run-id} ``` Remove it on every exit path, including the ones where you stopped early. Nothing under `/runs` diff --git a/.claude/skills/crypter-review/SKILL.md b/.claude/skills/crypter-review/SKILL.md index 92d5cd10..d9036575 100644 --- a/.claude/skills/crypter-review/SKILL.md +++ b/.claude/skills/crypter-review/SKILL.md @@ -26,7 +26,8 @@ your own inside it is a claim made in someone else's name. You are given a pull request number: `/crypter-review {pr-number}`. -Run from the root of the main checkout. The container's mounts resolve against it. +Run from the root of a checkout. `/plans` and `/runs` resolve against it, and the container is +named after it, so the one you reach is always the one whose artifacts you are reading. Use `pr-{number}` as the run id. @@ -40,35 +41,35 @@ The container's `agent` is uid 1001 and your files are uid 1000, so the agents w directories this side creates and grants. Creating them here also keeps you able to delete what they wrote. -Then confirm the running container is the one this checkout describes, before anything depends -on it: +Then confirm the container is up and current, before anything depends on it: ```bash -docker exec crypter-pipeline test -w /runs/pr-{number}/findings && \ - docker exec crypter-pipeline test -w /runs/pr-{number}/verification && \ - docker exec crypter-pipeline test -x /usr/local/bin/crypter-workspace +.devcontainer/pipeline.sh exec -- test -w /runs/pr-{number}/findings && \ + .devcontainer/pipeline.sh exec -- test -w /runs/pr-{number}/verification && \ + .devcontainer/pipeline.sh exec -- test -x /usr/local/bin/crypter-workspace ``` -The first proves the `/runs` mount reaches the directory you just made, which a container -created against a different checkout will not. The second proves the image carries the current -tooling. A container that fails either is not this checkout's, and every later step fails -against it in a way that reads like something else — a missing executable, findings written -somewhere you never look. +`pipeline.sh` resolves the container from the checkout it sits in, so which container you get is +settled by where you are rather than by anything you check. What the probes are for is the rest: +the first two prove `/runs` is mounted and that uid 1001 can write the directories you just +made, and the third proves the image carries the current tooling. Without them a later step +fails in a way that reads like something else — a missing executable, findings written somewhere +you never look. -Both are `test` because `docker exec` runs a binary and not a shell, so a builtin like +All three are `test` because `docker exec` runs a binary and not a shell, so a builtin like `command -v` exits 127 whether or not the thing it was looking for is there. -**Do not `docker start` an exited container to fix this.** Mounts and image are fixed when a -container is created, so starting one built from another checkout, or from an older image, -brings back the same wrong container. Bring it up from here instead: +**Do not `docker start` an exited container to fix this.** The image is fixed when a container is +created, so starting an old one brings back the old tooling. Bring it up from here instead: ```bash -docker compose -f .devcontainer/docker-compose.yml up -d --build +.devcontainer/pipeline.sh up ``` -That rebuilds the image and recreates the container against this checkout's mounts. It replaces -any container of the same name, so **ask the user before running it** — theirs may belong to -another checkout and hold work you cannot see. +That rebuilds the image and recreates this checkout's container. It cannot touch another +checkout's. What it does destroy is every workspace under `/work` in *this* container, because +`/work` is the container's own filesystem and not a volume — so **ask the user before running it +if another run may be live here.** ## 1. Read the pull request @@ -87,28 +88,28 @@ against the branch the pull request actually names. ## 2. Fetch it into a workspace -The container has no network remote. It clones from your repository through a read-only mount, -so the pull request head goes into your repository first and travels across from there: +The container clones from the repository itself, so the pull request head comes straight from +GitHub and nothing has to be staged in your checkout first: ```bash -git fetch origin +refs/pull/{number}/head:refs/pr/{number} {base-branch} -docker exec crypter-pipeline crypter-workspace create pr-{number} \ - --base {base-branch} '+refs/pr/{number}:refs/heads/pr-{number}' +.devcontainer/pipeline.sh exec -- crypter-workspace create pr-{number} \ + --base {base-branch} '+refs/pull/{number}/head:refs/heads/pr-{number}' ``` The refspec is forced, so reviewing a pull request again after its author rebased or amended -picks up the new head instead of being rejected. The base branch is fetched alongside it because -the workspace clones your repository, and a base you have never fetched is not there to diff -against. +picks up the new head instead of being rejected. The base branch needs no fetching of its own — +the clone brings every branch the repository has. -**If either fails, stop and say so.** +Whatever your checkout is on, and however stale it is, does not reach the review. + +**If it fails, stop and say so.** The workspace lasts for this review and no longer. ## 3. Examine ```bash -docker exec -w /work/pr-{number} crypter-pipeline \ +.devcontainer/pipeline.sh exec -w /work/pr-{number} -- \ claude --permission-mode auto -p "/crypter-devcontainer-examine pr-{number} pr-{number} origin/{base-branch}" ``` @@ -140,7 +141,7 @@ finding the author has to answer. So every finding is ruled on against the code anywhere — by a verifier in the container, one per finding, not by you. ```bash -docker exec -w /work/pr-{number} crypter-pipeline \ +.devcontainer/pipeline.sh exec -w /work/pr-{number} -- \ claude --permission-mode auto -p "/crypter-devcontainer-verify pr-{number} pr-{number} /runs/pr-{number}/findings/" ``` @@ -195,7 +196,7 @@ retry it against a different line.** ## 6. Tear down and report ```bash -docker exec crypter-pipeline crypter-workspace remove pr-{number} +.devcontainer/pipeline.sh exec -- crypter-workspace remove pr-{number} ``` Remove it on every exit path, including the ones where you stopped early. The findings and diff --git a/.claude/skills/crypter-step-open-pull-request/SKILL.md b/.claude/skills/crypter-step-open-pull-request/SKILL.md index b130708a..529e0f8b 100644 --- a/.claude/skills/crypter-step-open-pull-request/SKILL.md +++ b/.claude/skills/crypter-step-open-pull-request/SKILL.md @@ -19,7 +19,7 @@ The branch lives in the run's workspace. `git` reaches it over `docker exec`: ```bash git -c protocol.ext.allow=user fetch \ - "ext::docker exec -i crypter-pipeline git upload-pack /work/{run-id}" {branch}:{branch} + "ext::docker exec -i $(.devcontainer/pipeline.sh name) git upload-pack /work/{run-id}" {branch}:{branch} ``` `protocol.ext.allow` is passed per command and stays out of your config. **If this fails, stop diff --git a/.claude/skills/crypter-triage-review/SKILL.md b/.claude/skills/crypter-triage-review/SKILL.md index e4af386c..d2a02cb4 100644 --- a/.claude/skills/crypter-triage-review/SKILL.md +++ b/.claude/skills/crypter-triage-review/SKILL.md @@ -11,7 +11,8 @@ recorded as verified, and the verified ones become commits. Findings come from anywhere — the reviewer lenses, a person, another tool. They are treated the same way, because where a finding came from says nothing about whether it is true. -Run from the root of the main checkout. The container's mounts resolve against it. +Run from the root of a checkout. `/plans` and `/runs` resolve against it, and the container is +named after it, so the one you reach is always the one whose artifacts you are reading. ## Setup @@ -27,34 +28,34 @@ chmod 777 .claude/runs/pr-{number} .claude/runs/pr-{number}/verification The container's `agent` is uid 1001 and your files are uid 1000, so the agents write into directories this side creates and grants. -Then confirm the running container is the one this checkout describes, before anything depends -on it: +Then confirm the container is up and current, before anything depends on it: ```bash -docker exec crypter-pipeline test -w /runs/pr-{number}/verification && \ - docker exec crypter-pipeline test -x /usr/local/bin/crypter-workspace +.devcontainer/pipeline.sh exec -- test -w /runs/pr-{number}/verification && \ + .devcontainer/pipeline.sh exec -- test -x /usr/local/bin/crypter-workspace ``` -The first proves the `/runs` mount reaches the directory you just made, which a container -created against a different checkout will not. The second proves the image carries the current -tooling. A container that fails either is not this checkout's, and every later step fails -against it in a way that reads like something else — a missing executable, verification written -somewhere you never look. +`pipeline.sh` resolves the container from the checkout it sits in, so which container you get is +settled by where you are rather than by anything you check. What the probes are for is the rest: +the first proves `/runs` is mounted and that uid 1001 can write the directory you just made, and +the second proves the image carries the current tooling. Without them a later step fails in a way +that reads like something else — a missing executable, verification written somewhere you never +look. Both are `test` because `docker exec` runs a binary and not a shell, so a builtin like `command -v` exits 127 whether or not the thing it was looking for is there. -**Do not `docker start` an exited container to fix this.** Mounts and image are fixed when a -container is created, so starting one built from another checkout, or from an older image, -brings back the same wrong container. Bring it up from here instead: +**Do not `docker start` an exited container to fix this.** The image is fixed when a container is +created, so starting an old one brings back the old tooling. Bring it up from here instead: ```bash -docker compose -f .devcontainer/docker-compose.yml up -d --build +.devcontainer/pipeline.sh up ``` -That rebuilds the image and recreates the container against this checkout's mounts. It replaces -any container of the same name, so **ask the user before running it** — theirs may belong to -another checkout and hold work you cannot see. +That rebuilds the image and recreates this checkout's container. It cannot touch another +checkout's. What it does destroy is every workspace under `/work` in *this* container, because +`/work` is the container's own filesystem and not a volume — so **ask the user before running it +if another run may be live here.** ## 1. Collect the findings @@ -77,24 +78,23 @@ about intent. A question is for the author to answer, not for a verifier. ## 2. Fetch the head into a workspace -The container has no network remote. It clones from your repository through a read-only mount, -so the head goes into your repository first and travels across from there: +The container clones from the repository itself, so the head comes straight from GitHub and +nothing has to be staged in your checkout first: ```bash -git fetch origin +refs/pull/{number}/head:refs/pr/{number} -docker exec crypter-pipeline crypter-workspace create pr-{number} \ - '+refs/pr/{number}:refs/heads/{head-branch}' +.devcontainer/pipeline.sh exec -- crypter-workspace create pr-{number} \ + '+refs/pull/{number}/head:refs/heads/{head-branch}' ``` The branch in the workspace takes the pull request's own branch name, so the commits go back to the branch they came from. -**If either fails, stop and say so.** +**If it fails, stop and say so.** ## 3. Verify ```bash -docker exec -w /work/pr-{number} crypter-pipeline \ +.devcontainer/pipeline.sh exec -w /work/pr-{number} -- \ claude --permission-mode auto -p "/crypter-devcontainer-verify pr-{number} {head-branch} /runs/pr-{number}/review.md" ``` @@ -126,7 +126,7 @@ thread. Where `triage.md` has anything, and the head branch is one you can push to: ```bash -docker exec -w /work/pr-{number} crypter-pipeline \ +.devcontainer/pipeline.sh exec -w /work/pr-{number} -- \ claude --permission-mode auto -p "/crypter-devcontainer-remediate pr-{number} {head-branch} /runs/pr-{number}/triage.md" ``` @@ -139,7 +139,7 @@ stands, and the author does the fixing. Say so in the report. ## 6. Tear down and report ```bash -docker exec crypter-pipeline crypter-workspace remove pr-{number} +.devcontainer/pipeline.sh exec -- crypter-workspace remove pr-{number} ``` Remove it on every exit path, including the ones where you stopped early. The verdicts under diff --git a/.devcontainer/.env.example b/.devcontainer/.env.example index c10ed1a3..554041de 100644 --- a/.devcontainer/.env.example +++ b/.devcontainer/.env.example @@ -1,2 +1,5 @@ CRYPTER_GIT_EMAIL="" CRYPTER_GIT_NAME="" + +# The repository workspaces are cloned from. Set it to work against a fork. +#CRYPTER_REPO_URL="https://github.com/Crypter-File-Transfer/Crypter.git" diff --git a/.devcontainer/Dockerfile b/.devcontainer/Dockerfile index 98fc8031..e9cf0728 100644 --- a/.devcontainer/Dockerfile +++ b/.devcontainer/Dockerfile @@ -48,11 +48,6 @@ RUN dotnet tool install dotnet-ef --version '10.0.*' --tool-path "${DOTNET_TOOLS COPY .devcontainer/workspace.sh /usr/local/bin/crypter-workspace RUN chmod +x /usr/local/bin/crypter-workspace -# /host-git is the host's repository, owned by the host user. Git refuses to read a repository -# owned by anyone else until it is named as safe, and the whole point of the mount is that it -# belongs to somebody else. -RUN git config --system --add safe.directory /host-git - # The caches and the agent's Claude Code state are named volumes. Docker creates a mount point # that the image does not already contain as root, so creating these here is what gives the # volumes the right ownership. /work holds the ephemeral workspaces and is not a volume. diff --git a/.devcontainer/docker-compose.yml b/.devcontainer/docker-compose.yml index f6c9407f..fa48c29b 100644 --- a/.devcontainer/docker-compose.yml +++ b/.devcontainer/docker-compose.yml @@ -1,23 +1,26 @@ -name: crypter-pipeline - services: pipeline: - container_name: crypter-pipeline + # Named per checkout by pipeline.sh, so two checkouts get two containers instead of fighting + # over one. Required rather than defaulted: a bare `docker compose` here would create a + # container whose name says nothing about which checkout's mounts it holds. + container_name: ${CRYPTER_PIPELINE_CONTAINER:?run .devcontainer/pipeline.sh up instead} # Built here rather than pulled. The image carries tooling and nothing else, so a change to # it is a local rebuild instead of a publish someone has to approve. image: crypter-devcontainer:local build: context: .. dockerfile: .devcontainer/Dockerfile + labels: + # What `pipeline.sh list` reads to say which checkout a container belongs to. + com.crypter.pipeline.checkout: ${CRYPTER_PIPELINE_CHECKOUT:?run .devcontainer/pipeline.sh up instead} environment: CRYPTER_GIT_NAME: ${CRYPTER_GIT_NAME} CRYPTER_GIT_EMAIL: ${CRYPTER_GIT_EMAIL} + # Workspaces are cloned from here rather than from the host, so a run sees the branch as + # the repository holds it. Override for a fork. + CRYPTER_REPO_URL: ${CRYPTER_REPO_URL:-https://github.com/Crypter-File-Transfer/Crypter.git} volumes: - claude:/home/agent/.claude - # The host repository, read-only. Workspaces are cloned from it and the container writes - # nothing back. The host working tree is deliberately not mounted, so uncommitted work is - # not visible in here. - - ../.git:/host-git:ro # Plans are authored on the host and read from /plans. - ../.claude/plans:/plans:ro # Findings, conformance and triage are artifacts on the host, written from /runs. @@ -31,10 +34,16 @@ services: working_dir: /work command: sleep infinity +# External so every instance on the machine shares them: Claude Code is authenticated once and +# the caches are warmed once. A Compose-owned volume carries the project it was created for, and +# a second project mounting it fails on the mismatch. `pipeline.sh up` creates them. volumes: claude: + external: true name: crypter-pipeline-claude nuget: + external: true name: crypter-pipeline-nuget pnpm: + external: true name: crypter-pipeline-pnpm diff --git a/.devcontainer/pipeline.sh b/.devcontainer/pipeline.sh new file mode 100755 index 00000000..0c287fe8 --- /dev/null +++ b/.devcontainer/pipeline.sh @@ -0,0 +1,103 @@ +#!/usr/bin/env bash +# Resolve which pipeline container belongs to this checkout, and drive its lifecycle. +# +# /plans and /runs are relative mounts, so a container is welded to the checkout it was created +# from. Naming the container after that checkout is what lets several exist at once: the name a +# checkout resolves to is its own, so an orchestrator can never reach another checkout's mounts. +# +# The name is derived rather than configured. There is nothing to set, and nothing that can drift +# out of step with where the checkout actually is. +# +# Parallel runs within one checkout need none of this. They are already separated by the per-run +# workspaces at /work/{run-id}. +set -euo pipefail + +script_dir="$(cd "$(dirname "$(realpath "${BASH_SOURCE[0]}")")" && pwd)" +checkout="$(dirname "${script_dir}")" +compose_file="${script_dir}/docker-compose.yml" + +# Shared by every instance on the machine, which is why Claude Code is authenticated once and the +# package caches are warmed once. Declared external in the Compose file, so nothing creates them +# but this script. +volumes=(crypter-pipeline-claude crypter-pipeline-nuget crypter-pipeline-pnpm) + +usage() { + cat >&2 <<'EOF' +usage: pipeline.sh name print this checkout's container name + pipeline.sh exec [flags] -- {cmd} run a command in this checkout's container + pipeline.sh up create or recreate it, rebuilding the image + pipeline.sh down stop and remove it + pipeline.sh list every pipeline container, and its checkout +EOF + exit 64 +} + +# The name is both a container name and a Compose project name. Compose is the stricter of the +# two: lowercase, and no dots. The slug keeps `docker ps` readable and the hash of the real path +# separates two checkouts that share a basename. +container_name() { + local slug hash + slug="$(basename "${checkout}" | tr '[:upper:]' '[:lower:]' | tr -c 'a-z0-9' '-')" + slug="${slug#-}" + slug="${slug%-}" + slug="${slug:0:20}" + slug="${slug:-checkout}" + + hash="$(printf '%s' "${checkout}" | sha256sum | cut -c1-8)" + + printf 'crypter-pipeline-%s-%s\n' "${slug}" "${hash}" +} + +compose() { + local name + name="$(container_name)" + CRYPTER_PIPELINE_CONTAINER="${name}" \ + CRYPTER_PIPELINE_CHECKOUT="${checkout}" \ + docker compose --project-name "${name}" --file "${compose_file}" "$@" +} + +case "${1:-}" in + name) + container_name + ;; + + exec) + shift + # Flags for docker exec come first, then `--`, then the command. The separator is what keeps + # a command's own flags from being read as docker's. + flags=() + while [[ $# -gt 0 && "${1}" != "--" ]]; do + flags+=("${1}") + shift + done + [[ "${1:-}" == "--" ]] || usage + shift + [[ $# -gt 0 ]] || usage + + docker exec "${flags[@]}" "$(container_name)" "$@" + ;; + + up) + # Compose will not create an external volume, and a missing one fails the `up` rather than + # being made on the fly. Creating is idempotent, so this is safe on every run. + for volume in "${volumes[@]}"; do + docker volume create "${volume}" >/dev/null + done + + compose up --detach --build + ;; + + down) + compose down + ;; + + list) + docker ps --all \ + --filter 'label=com.crypter.pipeline.checkout' \ + --format 'table {{.Names}}\t{{.Status}}\t{{.Label "com.crypter.pipeline.checkout"}}' + ;; + + *) + usage + ;; +esac diff --git a/.devcontainer/workspace.sh b/.devcontainer/workspace.sh index 69a4ad77..98590295 100755 --- a/.devcontainer/workspace.sh +++ b/.devcontainer/workspace.sh @@ -1,16 +1,17 @@ #!/usr/bin/env bash # Create and remove the per-run workspaces the agents build and review in. # -# A workspace is a clone of the host repository taken from the read-only /host-git mount. It -# belongs to one run and is removed with it, so no copy of the repository outlives the state it -# was made from. +# A workspace is a clone of the repository taken from CRYPTER_REPO_URL. It belongs to one run and +# is removed with it, so no copy of the repository outlives the state it was made from. # -# The mount is read-only, so the container reads committed history and writes nothing back. The -# host's working tree is not mounted at all, which is what keeps uncommitted work invisible here. +# Cloning from the remote rather than from the host means a run sees the branch as the repository +# holds it, not as some checkout happens to have fetched it. Nothing on the host is mounted here, +# so neither uncommitted work nor a stale checkout can reach a run. +# +# The clone is anonymous and read-only. Branches leave a workspace by the host fetching from it +# over `docker exec ... git upload-pack`, so no credential is needed in here. set -euo pipefail -host_git="/host-git" - usage() { echo "usage: crypter-workspace create {run-id} [--base {branch}] [refspec]" >&2 echo " crypter-workspace remove {run-id}" >&2 @@ -51,47 +52,28 @@ case "${subcommand}" in create) : "${CRYPTER_GIT_NAME:?Set CRYPTER_GIT_NAME in .devcontainer/.env to the author name on the commits}" : "${CRYPTER_GIT_EMAIL:?Set CRYPTER_GIT_EMAIL in .devcontainer/.env to the author email on the commits}" - - if [[ ! -d "${host_git}" ]]; then - echo "No host repository at ${host_git}. The container was started without its mount." >&2 - exit 1 - fi + : "${CRYPTER_REPO_URL:?Set CRYPTER_REPO_URL to the repository the workspaces are cloned from}" if [[ -e "${workspace}" ]]; then echo "A workspace already exists at ${workspace}. Remove it or use another run id." >&2 exit 1 fi - # Resolve the base before anything is created, so a base that is not there leaves nothing - # behind to remove first. - if ! git -C "${host_git}" rev-parse --verify --quiet "refs/remotes/origin/${base}" >/dev/null + git clone --quiet "${CRYPTER_REPO_URL}" "${workspace}" + + # The base has to exist before the run is measured against it, and the clone is the first + # place that can be checked. A workspace without its base is no use, so it goes. + if ! git -C "${workspace}" rev-parse --verify --quiet "refs/remotes/origin/${base}" >/dev/null then - echo "The host repository has no origin/${base}. Fetch it there and try again." >&2 + echo "${CRYPTER_REPO_URL} has no ${base} branch." >&2 + rm -rf "${workspace}" exit 1 fi - # --no-hardlinks because the mount is read-only and owned by another uid, which is exactly - # the case where git's hardlink optimisation is unavailable. Copying is predictable. - git clone --quiet --no-hardlinks "${host_git}" "${workspace}" - - # A clone maps the source's local branches into origin/*, so origin/stable here would mean - # whatever the host has checked out rather than what the org repository holds. Point the - # remote at the host's own remote-tracking refs instead, so origin/{branch} means the same - # thing in a workspace as it does on the host. Configuring the refspec rather than fetching - # it once keeps a later bare `git fetch` from putting the host's local branches back. - git -C "${workspace}" config remote.origin.fetch \ - '+refs/remotes/origin/*:refs/remotes/origin/*' - git -C "${workspace}" fetch --quiet --prune origin - if [[ -n "${refspec}" ]]; then git -C "${workspace}" fetch --quiet origin "${refspec}" fi - # The clone takes origin/HEAD from the host's checked-out branch, which is the one thing in - # the origin namespace that would still mean the host rather than the org. Point it at the - # base, so a bare `origin` resolves to what the run is measured against. - git -C "${workspace}" remote set-head origin "${base}" - git -C "${workspace}" checkout --quiet -B "${base}" "refs/remotes/origin/${base}" git -C "${workspace}" config user.name "${CRYPTER_GIT_NAME}" diff --git a/Documentation/Development/Agentic Development Pipeline.md b/Documentation/Development/Agentic Development Pipeline.md index 9b16f614..f7c00f6b 100644 --- a/Documentation/Development/Agentic Development Pipeline.md +++ b/Documentation/Development/Agentic Development Pipeline.md @@ -34,13 +34,14 @@ ones to invoke. `/crypter-step-plan` is the one worth borrowing when you want a plan and nothing else, and `/crypter-step-open-pull-request` is safe to run repeatedly, which is how the CI loop uses it. -**Run the orchestrators from the root of your main checkout.** The container's mounts are -relative to `.devcontainer/`, so `.claude/plans` and `.claude/runs` resolve against that one -directory. Started from a worktree, a run writes its plan somewhere the container cannot read. +**Run the orchestrators from the root of a checkout**, main or worktree. `.claude/plans` and +`.claude/runs` are mounts relative to `.devcontainer/`, so they resolve against whichever checkout +you launch from, and the container is named after that checkout — so the one you reach is always +the one holding the artifacts you are reading. -**The container holds no GitHub credential and no network remote.** Every authenticated GitHub -operation happens in your session with your own access, and `/crypter-change` pushes and -re-pushes without stopping to ask. +**The container holds no GitHub credential.** It clones anonymously over https and can only read +a public repository. Every authenticated GitHub operation happens in your session with your own +access, and `/crypter-change` pushes and re-pushes without stopping to ask. The branch is pushed to the org repository and the pull request opens against it, base `stable`, the same route a branch of your own takes. `/crypter-change` leaves you a draft pull request to @@ -51,28 +52,29 @@ and its network egress is open. Treat it as a trust boundary rather than a sandb ## Workspaces -The agents build and review in a **workspace**: a clone of your repository at `/work/{run-id}`, +The agents build and review in a **workspace**: a clone of the repository at `/work/{run-id}`, made when a run starts and deleted when it ends. Nothing that holds a copy of the code outlives the run that made it, so there is no second checkout drifting away from yours. -Workspaces are cloned from `/host-git`, a read-only mount of your repository's `.git`. Read-only -is what makes this safe to share: the container reads committed history and cannot move a ref, -add an object, or touch anything in your repository. Your working tree is not mounted at all, so -uncommitted work is invisible in there and cannot reach a branch. +Workspaces are cloned from `CRYPTER_REPO_URL` — the org repository on GitHub, not your checkout. +Nothing on the host is mounted for them to read. **Your checkout is therefore irrelevant to what +a run builds or reviews**: it sees the branch as the repository holds it, whatever yours is on, +however stale it is, and whatever is uncommitted in it. A pull request head is fetched straight +from `refs/pull/{number}/head`, so nothing has to be staged on your side first. + +Because each workspace is a full clone taken when it is created, and nothing re-fetches +afterwards, a run is pinned to the commit it started from. Merge to `stable` while a run is going +and it keeps building against what it cloned; the next run gets the new commit. Two runs at +different bases are no trouble. The orchestrator owns the lifecycle. It creates the workspace in its setup and removes it when the run ends; the container skills use it and never create or destroy one. ```bash -docker exec crypter-pipeline crypter-workspace create {run-id} [--base {branch}] [refspec] -docker exec crypter-pipeline crypter-workspace remove {run-id} +.devcontainer/pipeline.sh exec -- crypter-workspace create {run-id} [--base {branch}] [refspec] +.devcontainer/pipeline.sh exec -- crypter-workspace remove {run-id} ``` -The org repository is `origin` on both sides. A clone would otherwise map your local branches -into the workspace's `origin/*`, so the create step points the remote at your remote-tracking -refs instead, and `origin/stable` in a workspace means what it means in your checkout. Fetch -before creating a workspace, or the run starts on a stale base. - `--base` is the branch the run is built or reviewed against, `stable` when it is not given. A pull request states its own base, and a release states `main`, so the review skills pass what they read rather than assuming. @@ -85,12 +87,14 @@ Everything crossing the container boundary goes through one of these: | Host | Container | Direction | Holds | |---|---|---|---| -| `.git` | `/host-git` | Read-only | Your committed history, which workspaces are cloned from | | `.claude/plans` | `/plans` | Read-only | `{run-id}/plan.md` | | `.claude/runs` | `/runs` | Writable | `{run-id}/conformance.md`, `{run-id}/findings/{lens}.md`, `{run-id}/review.md`, `{run-id}/verification/{id}.md`, `{run-id}/triage.md`, `{run-id}/ci-{n}.md` | -`.claude/plans` and `.claude/runs` are gitignored and live on your disk. Only `/runs` is -writable; the other two the container can read and nothing more. +Both are gitignored and live on your disk. Only `/runs` is writable; `/plans` the container can +read and nothing more. + +Source is not among them. Nothing of your repository is mounted, so a run cannot read your +working tree, your local branches, or a ref you have not pushed — it clones from GitHub instead. The plan goes in and cannot be rewritten by the agents. Findings come back out as files you can open, grep and keep, rather than as text in a transcript, and each is written by the agent that @@ -102,32 +106,52 @@ bind mount keeps host ownership, so the orchestrators create every directory und themselves and give it mode 777. Directories made on the host stay deletable from the host; a directory the container creates is one you need `docker exec` to remove. -The branch itself travels differently. It never passes through a mount: +A branch the agents built travels back out the same way it would from any remote, over a git +transport that runs `docker exec` instead of opening a socket: ```bash git -c protocol.ext.allow=user fetch \ - "ext::docker exec -i crypter-pipeline git upload-pack /work/{run-id}" {branch}:{branch} + "ext::docker exec -i $(.devcontainer/pipeline.sh name) git upload-pack /work/{run-id}" {branch}:{branch} +``` + +`protocol.ext.allow` is passed per command, so it stays out of your git config. Pushing is then +yours, with your credentials — which is what keeps a write token out of a container running +unattended agents. + +## One container per checkout + +`/plans` and `/runs` are relative paths in the Compose file, so they resolve against the checkout +you launch from and a container is stuck with whatever they resolved to when it was created. Each +checkout therefore gets its own container, named after it: + +```bash +.devcontainer/pipeline.sh name # crypter-pipeline-crypter-4f3a9c21 ``` -`protocol.ext.allow` is passed per command, so it stays out of your git config. +The name is derived from the checkout's real path — a readable slug, and a hash to separate two +clones that share a basename. Nothing to configure, and nothing that can drift out of step with +where the checkout actually is. Because a checkout resolves only to its own name, a run can never +reach another checkout's mounts, and recreating one container leaves the others alone. + +```bash +.devcontainer/pipeline.sh list # every instance, and the checkout it belongs to +``` -Because the mounts are relative paths in the Compose file, they resolve against the checkout you -launch from, and a container is stuck with whatever they resolved to when it was created. Run -the pipeline from a second checkout and the container it finds by name is the first one's: -`/runs` writes land under a repository you are not looking at, and `/host-git` clones a history -that is not the one you are reviewing. +This is not what makes runs parallel. Several runs share one container quite happily — they are +separated by their workspaces at `/work/{run-id}` and their artifacts at `/runs/{run-id}`, and +concurrent `docker exec` calls do not queue. Per-checkout naming is about which host directories +a container is wired to, nothing more. -Starting an exited container does not repair this, and neither does it pick up a newer image — -`docker start` reuses what the container was created with. Recreate it from the checkout you -mean to work in: +Starting an exited container does not pick up a newer image — `docker start` reuses what the +container was created with. Recreate it instead: ```bash -docker compose -f .devcontainer/docker-compose.yml up -d --build +.devcontainer/pipeline.sh up ``` -That rebuilds the image and recreates the container against the mounts as they resolve here. -There is one `crypter-pipeline` on the machine, so this takes it over from whichever checkout -held it. +That rebuilds the image and recreates this checkout's container. `/work` is the container's own +filesystem rather than a volume, so this destroys any workspace a run in this checkout is still +using. ## Running a change @@ -163,36 +187,43 @@ ignored by git. Copy the template and fill it in before the first `up`. cp .devcontainer/.env.example .devcontainer/.env ``` -| Variable | Value | -|---|---| -| `CRYPTER_GIT_NAME` | Author name on the agents' commits. | -| `CRYPTER_GIT_EMAIL` | Author email on the agents' commits. | +| Variable | Value | | +|---|---|---| +| `CRYPTER_GIT_NAME` | Author name on the agents' commits. | Required | +| `CRYPTER_GIT_EMAIL` | Author email on the agents' commits. | Required | +| `CRYPTER_REPO_URL` | The repository workspaces are cloned from. | Defaults to the org repository; set it for a fork | + +Both required ones fail workspace creation with a message naming the variable when left empty. -Both are required. Leaving one empty fails workspace creation with a message naming the -variable. +The container's name needs no configuration. It is derived from where the checkout is. ## Launching the container -The container is a Compose service in `.devcontainer/docker-compose.yml`. That is a separate -Compose project from the application stack at the repository root, so `docker compose up` and +The container is a Compose service in `.devcontainer/docker-compose.yml`, driven through +`pipeline.sh` — which supplies the per-checkout name Compose needs. That is a separate Compose +project from the application stack at the repository root, so `docker compose up` and `docker compose down` there never touch it, and the two share no network. ```bash mkdir -p .claude/plans .claude/runs -docker compose -f .devcontainer/docker-compose.yml up -d --build -docker compose -f .devcontainer/docker-compose.yml exec pipeline bash +.devcontainer/pipeline.sh up +.devcontainer/pipeline.sh exec -- bash # add -it for an interactive shell ``` Create the two mount sources first. They are gitignored, so a fresh clone has neither, and Docker creates a missing bind-mount source as root — which the orchestrators then cannot write into. -The first `--build` takes a few minutes, mostly installing the `wasm-tools` workload. After -that Docker's layer cache makes it quick, and a change to `workspace.sh` rebuilds only the last -couple of layers. Use `--build` whenever `.devcontainer/` has changed; plain `up -d` otherwise. +Running `docker compose` against this file directly fails, on purpose: the container name is a +required variable, and a container created without it would say nothing about which checkout's +mounts it holds. -Swap `up -d` for `down` to stop it. The named volumes outlive the container, so the next `up` -keeps your Claude Code credentials and your package caches. +The first `up` takes a few minutes, mostly installing the `wasm-tools` workload. After that +Docker's layer cache makes it quick, and a change to `workspace.sh` rebuilds only the last couple +of layers. + +`pipeline.sh down` stops it. The named volumes outlive the container, so the next `up` keeps your +Claude Code credentials and your package caches. ## What is in the container @@ -210,6 +241,9 @@ There is **no Docker in the container**, so `Crypter.Test` cannot run there — Testcontainers to start PostgreSQL. The agents build but never test locally; the test suite runs in CI once the pull request exists, and failures come back to the implementer from there. +It does need **outbound network**, both for the Anthropic API and now for cloning workspaces. +The pipeline does not work offline. + Three named volumes survive rebuilds, and none of them holds source: | Volume | Holds | @@ -218,6 +252,11 @@ Three named volumes survive rebuilds, and none of them holds source: | `crypter-pipeline-nuget` | The NuGet package cache | | `crypter-pipeline-pnpm` | The pnpm store | +**Every instance on the machine shares all three**, which is why Claude Code is authenticated +once rather than once per checkout, and why a second checkout's first build is not a cold +restore. They are declared external so that several Compose projects can mount them; `pipeline.sh +up` creates them. + The two caches exist because workspaces are ephemeral. Without them every run would restore NuGet and pnpm from nothing, which is most of a build. @@ -228,10 +267,14 @@ collide with the next one. To start over from nothing, take the container down and remove the volumes: ```bash -docker compose -f .devcontainer/docker-compose.yml down +.devcontainer/pipeline.sh down docker volume rm crypter-pipeline-claude crypter-pipeline-nuget crypter-pipeline-pnpm ``` +The `down` is per-checkout, but removing the volumes is not — it takes the credentials and caches +away from **every** instance. Use `pipeline.sh list` to see what else is on the machine first, +including containers left behind by checkouts that no longer exist. + ## Authenticate Claude Code The image ships Claude Code but no credentials. Run `claude` once inside the container and @@ -243,8 +286,9 @@ they survive container rebuilds. You only do this again after removing that volu Run the agents with `--permission-mode auto`. They work unattended, so a prompt they cannot answer is a run that stalls. What bounds the blast radius is the container itself: a workspace -that is thrown away at the end of the run, a read-only view of your repository, and no GitHub -credential to push with. +that is thrown away at the end of the run, no access to your repository at all, and no GitHub +credential to push with — its only reach into the repository is an anonymous read of what is +already public. ## Changing the image @@ -252,12 +296,15 @@ Needed when the tooling changes — a new tool the agents need, a runtime versio changes never require it, because the image carries no source. ```bash -docker compose -f .devcontainer/docker-compose.yml up -d --build +.devcontainer/pipeline.sh up ``` That is the whole loop. The image is local to your machine — it is never published, and nobody else consumes it — so a change to `workspace.sh` or the Dockerfile takes effect on your next -`up` and affects nothing but your own container. +`up`. + +The image tag is shared, so the rebuild is machine-wide; other instances pick the new image up +when they are next recreated, not before. `pr-build-devcontainer` builds the image on a pull request that touches `.devcontainer/`. It pushes nothing; it is there to catch a Dockerfile that does not build.