From 11208069d97832913e4a3f3704b2cffb47eaa511 Mon Sep 17 00:00:00 2001 From: Blake Bauman Date: Wed, 11 Nov 2026 16:03:14 -0700 Subject: [PATCH] Close the bypasses in the bash guard, and check the .claude toolkit in CI Probing guard-bash.sh with 17 commands found it allowed podman and docker-compose teardown with volumes, volume rm/prune, rm of the data roots with a trailing slash or glob, cp/mv/perl onto an applied migration, and the argv-pattern kill inside sh -c or behind xargs. Each is now denied, with a selftest case, alongside allow cases for the everyday commands next to them. validate.py now fails when a brief names a repository path that does not exist or the README's counts drift; fiber-devops named three files that were gone. make toolkit runs the validator, the selftest and shellcheck, and the check job runs it. Reviewers inherit the session model instead of pinning an older one. The allowlist no longer covers git branch -D. AGENTS.md carries the nine invariants and points at CLAUDE.md. Co-Authored-By: Claude Fable 5.2 --- .claude/README.md | 10 ++--- .claude/agents/fiber-devops.md | 4 +- .claude/agents/fiber-reviewer.md | 2 +- .claude/agents/fiber-security-reviewer.md | 2 +- .claude/hooks/guard-bash.sh | 45 ++++++++++++++++++--- .claude/hooks/selftest.sh | 28 +++++++++++++ .claude/settings.json | 3 +- .claude/validate.py | 49 +++++++++++++++++++++++ .github/workflows/ci.yml | 9 +++++ AGENTS.md | 32 +++++++++++++-- CLAUDE.md | 4 +- Makefile | 13 +++++- docs/development.md | 1 + 13 files changed, 178 insertions(+), 24 deletions(-) diff --git a/.claude/README.md b/.claude/README.md index cb8f98f..cd7d23a 100644 --- a/.claude/README.md +++ b/.claude/README.md @@ -15,7 +15,7 @@ Verify after any change here: ```bash python3 .claude/validate.py # frontmatter, spec compliance, hook registration -bash .claude/hooks/selftest.sh # 24 deny/allow cases against the guard hooks +bash .claude/hooks/selftest.sh # 53 deny/allow cases against the guard hooks ``` ## Skills — the procedures @@ -48,8 +48,8 @@ Delegate work that needs its own context. Claude routes on the `description`, or | `fiber-devops` | inherit | Compose, Dockerfiles, CI workflow, ports, deploys | | `fiber-docs-syncer` | inherit | Fixes docs drift against the code | | `fiber-smoke-runner` | inherit | Runs the smokes, triages to a cause | -| `fiber-reviewer` | opus | Reviews a diff against the real invariants (read-only) | -| `fiber-security-reviewer` | opus | Auth, secrets, tokens, webhooks, artifacts (read-only) | +| `fiber-reviewer` | inherit | Reviews a diff against the real invariants (read-only) | +| `fiber-security-reviewer` | inherit | Auth, secrets, tokens, webhooks, artifacts (read-only) | | `fiber-dx-reviewer` | sonnet | Onboarding friction and the five files that drift (read-only) | The three reviewers have no `Edit`/`Write` — they report, you decide. @@ -62,7 +62,7 @@ Deterministic, so they do not depend on the model remembering. Wired in `setting | Hook | Event | Does | |---|---|---| -| `guard-bash.sh` | PreToolUse(Bash) | Denies the argv-pattern kill that also kills this session's parent shell; denies volume-destroying Compose teardown; denies writes to *committed* migrations; denies wholesale deletion of `data/workspaces` or `data/artifacts` | +| `guard-bash.sh` | PreToolUse(Bash) | Denies the argv-pattern kill that also kills this session's parent shell (also inside `sh -c` and behind `xargs`); denies volume-destroying teardown under Docker or Podman, by Compose or by `volume rm`/`prune`; denies writing, copying or moving onto *committed* migrations, and deleting them; denies wholesale deletion of `data/`, `data/workspaces` or `data/artifacts` | | `guard-edits.sh` | PreToolUse(Edit\|Write) | Denies edits to applied migrations, `routeTree.gen.ts`, `data/`, `dist/`, `target/`, `Cargo.lock` | | `fmt-rust.sh` | PostToolUse(Edit\|Write) | `rustfmt` on edited `.rs` using the repo's `rustfmt.toml`, keeping `cargo fmt --check` green | | `fmt-ui.sh` | PostToolUse(Edit\|Write) | Biome (not Prettier) on edited `apps/ui` files | @@ -79,4 +79,4 @@ Two design notes. `guard-bash.sh` strips heredoc bodies before matching, so *doc - **New agent**: `agents/.md` with `name` and `description`. Reviewers should omit `Edit`/`Write`. - **New hook**: add the script to `hooks/`, `chmod +x`, register it in `settings.json`, and add deny *and* allow cases to `selftest.sh` — a guard that blocks legitimate work is worse than no guard. -Run `python3 .claude/validate.py` before committing changes to this directory. +Run `python3 .claude/validate.py` before committing changes to this directory. It also fails when a brief names a repository path that no longer exists, or when the counts in this file drift; CI runs it and the hook selftest in the `check` job. diff --git a/.claude/agents/fiber-devops.md b/.claude/agents/fiber-devops.md index ad1543e..1c94808 100644 --- a/.claude/agents/fiber-devops.md +++ b/.claude/agents/fiber-devops.md @@ -10,7 +10,7 @@ You own Fiber's build, packaging, and deployment surface. ## Territory -`deploy/docker-compose.yml`, `deploy/Dockerfile.api`, `deploy/Dockerfile.ui`, `deploy/nginx.conf`, `.github/workflows/ci.yml`, `Makefile`, `scripts/dev-env.sh`, `apps/ui/Dockerfile.ui`, the `.dockerignore` files. +`deploy/docker-compose.yml`, `deploy/Dockerfile` (one file, `fiber-api` and `fiber-agent` targets), `deploy/agent-docker.Dockerfile`, `deploy/fiber-agent.service`, `apps/ui/Dockerfile.ui`, `apps/ui/nginx.conf`, `.github/workflows/ci.yml`, `.github/workflows/release.yml`, `.github/workflows/advisories.yml`, `.github/dependabot.yml`, `deny.toml`, `Makefile`, `scripts/dev-env.sh`, `scripts/install-agent.sh`, the `.dockerignore` files. ## Fixed facts @@ -27,7 +27,7 @@ Ports are deliberately non-default to avoid colliding with whatever else is on t - Postgres is **17-alpine**. A volume created by 16 will not start under 17; the documented recovery destroys data, so never run it yourself — tell the user. - Compose service names are `fiber-postgres`, `fiber-redis`, `fiber-s3`, `fiber-api`, `fiber-ui`. The `fiber-` prefix is mandatory (`.claude/rules/naming.md`). - The Compose healthcheck uses `GET /ready` (Postgres + Redis reachable). `/health` is liveness only. -- CI is two jobs: `check` (fmt, clippy `-D warnings` across all six crates, `cargo build -p fiber-api -p fiber-agent -p fiber-cli`) and `ui` (pnpm + Node, `pnpm install --frozen-lockfile`, `pnpm exec tsc --noEmit`). `RUSTFLAGS: -Dwarnings` is set in the workflow env. +- CI (`ci.yml`) is six jobs: `check` (`make check`, shellcheck, actionlint, `make test-rust`, `make build`, lockfile drift), `deny` (`cargo deny check`), `ui` (Biome, `tsc --noEmit`, vitest, `pnpm build`), `docker` (image builds), `smoke` (the Compose smoke) and `smoke-host`. `check`, `ui`, `docker`, `smoke-host` and `deny` are required checks on the `main` ruleset, pinned by job name — renaming a job means editing the ruleset. The gate is `make check`; there is no `RUSTFLAGS` in the workflow env. - `fiber-api` applies migrations on boot, so rolling an image forward rolls the schema forward. Migrations must stay compatible with the *previous* image for the duration of a rolling deploy. - Redis carries only the `fiber:events` fan-out; Postgres leases are the source of truth. Multiple API replicas are fine; each agent holds a WS to one replica. diff --git a/.claude/agents/fiber-reviewer.md b/.claude/agents/fiber-reviewer.md index e474199..c1967a7 100644 --- a/.claude/agents/fiber-reviewer.md +++ b/.claude/agents/fiber-reviewer.md @@ -2,7 +2,7 @@ name: fiber-reviewer description: Reviews changed code against Fiber's real invariants — at-least-once replay safety, role gates, snapshot immutability, migration hygiene, cross-crate wire-type drift. Use proactively after implementing a change and before committing. tools: Read, Grep, Glob, Bash -model: opus +model: inherit color: yellow --- diff --git a/.claude/agents/fiber-security-reviewer.md b/.claude/agents/fiber-security-reviewer.md index 8fc49c4..c347df2 100644 --- a/.claude/agents/fiber-security-reviewer.md +++ b/.claude/agents/fiber-security-reviewer.md @@ -2,7 +2,7 @@ name: fiber-security-reviewer description: Security review for Fiber — a CI system that executes untrusted-ish shell on agents, holds project secrets, mints tokens, and verifies GitHub webhooks. Use before merging anything touching auth, secrets, tokens, webhooks, artifacts, or agent execution. tools: Read, Grep, Glob, Bash -model: opus +model: inherit color: red --- diff --git a/.claude/hooks/guard-bash.sh b/.claude/hooks/guard-bash.sh index b14b945..29365b6 100755 --- a/.claude/hooks/guard-bash.sh +++ b/.claude/hooks/guard-bash.sh @@ -26,10 +26,19 @@ deny() { printf 'Blocked by fiber toolkit: %s\n' "$1" >&2; exit 2; } # Match only where the binary is invoked as a command word (start of line, or # after a pipe/separator), not merely mentioned inside a longer string. -cmdword() { printf '%s' "$scan" | grep -Eq "(^|[;&|]|&&|\|\||\\bthen |\\bdo )[[:space:]]*(sudo[[:space:]]+)?$1"; } +# +# "Command word" also covers the two wrappers that run their argument as a command: +# `sh -c ''` (any of sh/bash/zsh, with or without -l) and `xargs `. Without +# them the kill below went straight through as `bash -c "pkill …"` or `… | xargs pkill …`. +cmdword() { + printf '%s' "$scan" | grep -Eq "(^|[;&|(]|&&|\|\||\\bthen |\\bdo |\\b(ba|z)?sh[[:space:]]+-l?c[[:space:]]+[\"']|\\bxargs[[:space:]]+(-[^[:space:]]+[[:space:]]+)*)[[:space:]]*(sudo[[:space:]]+)?$1" +} -# 1. pkill -f fiber-agent matches parent shells whose argv mentions the binary path. -if cmdword '(pkill|killall)[^;&|]*fiber-agent'; then +# 1. Killing by argv pattern matches parent shells whose argv mentions the binary path. +# Two shapes: any pkill/killall naming fiber-agent, and `-f` with any pattern that +# mentions fiber at all (`pkill -f target/debug/fiber` matches the same shells). +if cmdword '(pkill|killall)[^;&|]*fiber-agent' \ + || cmdword '(pkill|killall)[^;&|]*[[:space:]]-[a-zA-Z]*f[a-zA-Z]*[[:space:]][^;&|]*fiber'; then deny "that kill pattern also matches shells whose argv contains the binary path, and can kill this session's parent. Kill by PID instead: pgrep -x fiber-agent @@ -39,19 +48,40 @@ See docs/development.md 'Agent tips' and .claude/rules/dx.md." fi # 2. Destroying the Postgres volume wipes every run, artifact row, and the admin user. -if printf '%s' "$scan" | grep -Eq 'docker[[:space:]]+compose[^;&|]*[[:space:]]down([[:space:]]|$)[^;&|]*(-v|--volumes)'; then +# Engine-agnostic: the stack runs under Docker or Podman, and both have a plugin form +# (`docker compose`) and a standalone one (`docker-compose`). `-v` may be bundled with +# other short flags or come before or after `down`'s other options. +compose='(docker|podman)([[:space:]]+|-)compose' +if printf '%s' "$scan" | grep -Eq "${compose}[^;&|]*[[:space:]]down([[:space:]][^;&|]*)?[[:space:]](-[a-zA-Z]*v[a-zA-Z]*|--volumes)([[:space:]]|\$|[;&|])"; then deny "'compose down -v' destroys the fiber-postgres volume (all runs, artifacts, users, secrets). Use 'make down' to stop containers and keep data. If you truly intend to wipe (e.g. the documented Postgres 16->17 upgrade), ask the user to run it themselves." fi +# 2b. The same data, removed by name instead of through Compose. +if cmdword '(docker|podman)[[:space:]]+volume[[:space:]]+(rm|remove)[^;&|]*fiber' \ + || cmdword '(docker|podman)[[:space:]]+volume[[:space:]]+prune' \ + || cmdword '(docker|podman)[[:space:]]+system[[:space:]]+prune[^;&|]*--volumes'; then + deny "that removes the fiber-postgres volume (all runs, artifacts, users, secrets), or every unused volume including it. +Use 'make down' to stop containers and keep data. If a wipe is really intended, ask the user to run it themselves." +fi + # 3. sqlx migrations are applied on fiber-api boot and checksummed; rewriting an # applied file diverges from _sqlx_migrations on every existing database. # Only already-committed migrations are protected — writing a NEW numbered # file is the correct way to change the schema and must stay unobstructed. targets="$(printf '%s' "$scan" \ | grep -Eo 'crates/fiber-core/migrations/[0-9]{3}_[A-Za-z0-9_.-]+\.sql' | sort -u)" -if [ -n "$targets" ] && printf '%s' "$scan" | grep -Eq '(>>?|sed -i|tee|truncate)[^;&|]*crates/fiber-core/migrations/'; then +# Writers: redirection, in-place editors, and anything that copies or moves a file +# *onto* a migration (the path is the last argument). Copying a migration somewhere +# else, reading one, and `git checkout -- ` back to HEAD stay allowed. +mig='crates/fiber-core/migrations/' +writes_migration() { + printf '%s' "$scan" | grep -Eq "(>>?|sed[[:space:]]+(-[a-zA-Z]+[[:space:]]+)*-i|perl[[:space:]]+-[a-zA-Z]*i|tee|truncate|dd[^;&|]*of=)[^;&|]*$mig" \ + || printf '%s' "$scan" | grep -Eq "(^|[;&|]|&&)[[:space:]]*(cp|mv|install|rsync|ln)[[:space:]][^;&|]*${mig}[0-9]{3}_[A-Za-z0-9_.-]+\.sql[[:space:]]*(\$|[;&|])" \ + || printf '%s' "$scan" | grep -Eq "(^|[;&|]|&&)[[:space:]]*(rm|git[[:space:]]+rm)[[:space:]][^;&|]*$mig" +} +if [ -n "$targets" ] && writes_migration; then while IFS= read -r f; do [ -z "$f" ] && continue if git -C "${CLAUDE_PROJECT_DIR:-.}" ls-files --error-unmatch "$f" >/dev/null 2>&1; then @@ -65,7 +95,10 @@ EOF fi # 4. Truncating the workspace/artifact roots out from under a running agent. -if printf '%s' "$scan" | grep -Eq 'rm[[:space:]]+(-[a-zA-Z]+[[:space:]]+)*[^;&|]*(data/workspaces|data/artifacts)[[:space:]]*($|[;&|])'; then +# The root itself in any spelling: `data/workspaces`, with a trailing slash, with +# `/*` or `/.`, or behind `./`. A named subdirectory is one run and stays allowed. +if printf '%s' "$scan" | grep -Eq 'rm[[:space:]]+(-[a-zA-Z]+[[:space:]]+)*[^;&|]*(data/workspaces|data/artifacts)(/|/\*|/\.)?[[:space:]]*($|[;&|])' \ + || printf '%s' "$scan" | grep -Eq 'rm[[:space:]]+(-[a-zA-Z]+[[:space:]]+)*[^;&|]*[[:space:]](\./)?data(/|/\*)?[[:space:]]*($|[;&|])'; then deny "refusing to delete data/workspaces or data/artifacts wholesale — a running fiber-agent holds leases on them. Delete a specific run's subdirectory, or stop agents first." fi diff --git a/.claude/hooks/selftest.sh b/.claude/hooks/selftest.sh index 2933aa9..6ae9af5 100755 --- a/.claude/hooks/selftest.sh +++ b/.claude/hooks/selftest.sh @@ -32,6 +32,34 @@ expect deny "compose down --volumes" "$HOOKS/guard-bash.sh" "$(payload expect deny "append to applied migration" "$HOOKS/guard-bash.sh" "$(payload 'echo "ALTER TABLE runs ADD COLUMN x int;" >> crates/fiber-core/migrations/001_initial.sql')" expect deny "sed -i on applied migration" "$HOOKS/guard-bash.sh" "$(payload 'sed -i "" s/a/b/ crates/fiber-core/migrations/003_retention.sql')" expect deny "rm -rf data/workspaces" "$HOOKS/guard-bash.sh" "$(payload 'rm -rf data/workspaces')" +# Bypasses found by probing the guard in November 2026; each of these was allowed. +expect deny "pkill inside bash -c" "$HOOKS/guard-bash.sh" "$(payload 'bash -c "pkill -f fiber-agent"')" +expect deny "pkill behind xargs" "$HOOKS/guard-bash.sh" "$(payload 'echo x | xargs pkill -f fiber-agent')" +expect deny "pkill -f on the binary's path" "$HOOKS/guard-bash.sh" "$(payload 'pkill -f target/debug/fiber')" +expect deny "podman compose down -v" "$HOOKS/guard-bash.sh" "$(payload 'podman compose -f deploy/docker-compose.yml down -v')" +expect deny "docker-compose down -v" "$HOOKS/guard-bash.sh" "$(payload 'docker-compose down -v')" +expect deny "podman-compose down --volumes" "$HOOKS/guard-bash.sh" "$(payload 'podman-compose down --volumes')" +expect deny "compose down, -v after a flag" "$HOOKS/guard-bash.sh" "$(payload 'docker compose down --remove-orphans -v')" +expect deny "docker volume rm by name" "$HOOKS/guard-bash.sh" "$(payload 'docker volume rm fiber_fiber-postgres-data')" +expect deny "podman volume prune" "$HOOKS/guard-bash.sh" "$(payload 'podman volume prune -f')" +expect deny "rm -rf data/workspaces/" "$HOOKS/guard-bash.sh" "$(payload 'rm -rf data/workspaces/')" +expect deny "rm -rf ./data/artifacts/*" "$HOOKS/guard-bash.sh" "$(payload 'rm -rf ./data/artifacts/*')" +expect deny "rm -rf data" "$HOOKS/guard-bash.sh" "$(payload 'rm -rf data')" +expect deny "cp onto applied migration" "$HOOKS/guard-bash.sh" "$(payload 'cp /tmp/x.sql crates/fiber-core/migrations/001_initial.sql')" +expect deny "mv onto applied migration" "$HOOKS/guard-bash.sh" "$(payload 'mv /tmp/x.sql crates/fiber-core/migrations/001_initial.sql')" +expect deny "perl -pi on applied migration" "$HOOKS/guard-bash.sh" "$(payload 'perl -pi -e s/a/b/ crates/fiber-core/migrations/001_initial.sql')" +expect deny "rm an applied migration" "$HOOKS/guard-bash.sh" "$(payload 'rm crates/fiber-core/migrations/001_initial.sql')" +expect allow "copy a migration elsewhere" "$HOOKS/guard-bash.sh" "$(payload 'cp crates/fiber-core/migrations/001_initial.sql /tmp/x.sql')" +expect allow "cat an applied migration" "$HOOKS/guard-bash.sh" "$(payload 'cat crates/fiber-core/migrations/001_initial.sql')" +expect allow "git checkout migration to HEAD" "$HOOKS/guard-bash.sh" "$(payload 'git checkout -- crates/fiber-core/migrations/001_initial.sql')" +expect allow "cp a new numbered migration" "$HOOKS/guard-bash.sh" "$(payload 'cp /tmp/x.sql crates/fiber-core/migrations/999_new.sql')" +expect allow "podman compose down (no -v)" "$HOOKS/guard-bash.sh" "$(payload 'podman compose -f deploy/docker-compose.yml down')" +expect allow "compose up -d (a -d, not a -v)" "$HOOKS/guard-bash.sh" "$(payload 'docker compose -f deploy/docker-compose.yml up -d')" +expect allow "docker volume ls" "$HOOKS/guard-bash.sh" "$(payload 'docker volume ls')" +expect allow "rm a file under data/artifacts" "$HOOKS/guard-bash.sh" "$(payload 'rm data/artifacts/ab/cd/out.txt')" +expect allow "rm -rf target/debug" "$HOOKS/guard-bash.sh" "$(payload 'rm -rf target/debug')" +expect allow "pkill something unrelated" "$HOOKS/guard-bash.sh" "$(payload 'pkill -f vite')" +expect allow "kill a fiber-agent by PID" "$HOOKS/guard-bash.sh" "$(payload 'kill 12345')" expect allow "pgrep -x fiber-agent" "$HOOKS/guard-bash.sh" "$(payload 'pgrep -x fiber-agent')" expect allow "make down" "$HOOKS/guard-bash.sh" "$(payload 'make down')" expect allow "compose down (no volumes)" "$HOOKS/guard-bash.sh" "$(payload 'docker compose -f deploy/docker-compose.yml down')" diff --git a/.claude/settings.json b/.claude/settings.json index 3234bee..13604c9 100644 --- a/.claude/settings.json +++ b/.claude/settings.json @@ -71,7 +71,8 @@ "Bash(git diff:*)", "Bash(git log:*)", "Bash(git show:*)", - "Bash(git branch:*)", + "Bash(git branch --list:*)", + "Bash(git branch --show-current)", "Bash(docker ps:*)", "Bash(docker compose -f deploy/docker-compose.yml ps:*)", "Bash(docker compose -f deploy/docker-compose.yml logs:*)", diff --git a/.claude/validate.py b/.claude/validate.py index 35d2aac..ebfa18f 100644 --- a/.claude/validate.py +++ b/.claude/validate.py @@ -201,12 +201,61 @@ def check_hooks() -> None: warnings.append(f"hooks/{script.name} exists but is not registered in settings.json") +# A brief that names a file which no longer exists sends an agent looking for it, and +# then reporting on a layout the repository does not have. `fiber-devops` named +# `deploy/Dockerfile.api` and `deploy/nginx.conf` for two months after they were gone. +REPO = ROOT.parent +PATH_ROOTS = ("deploy/", "scripts/", "crates/", "apps/", "docs/", ".github/", ".claude/", "examples/") +# Build output: named in briefs as something not to edit, absent from a clean checkout. +GENERATED = {"apps/ui/dist/"} +PATH_RE = re.compile(r"`([A-Za-z0-9_.$/-]+)`") + + +def check_paths(md: Path) -> None: + """Every backticked repository path in a brief must exist.""" + rel = md.relative_to(ROOT) + for token in sorted(set(PATH_RE.findall(md.read_text()))): + if not token.startswith(PATH_ROOTS): + continue + # Only whole paths: something with a file extension, or a directory written + # with its trailing slash. `crates/fiber-api` style prose mentions are skipped. + if not (token.endswith("/") or re.search(r"\.[A-Za-z0-9]+$", token)): + continue + if token in GENERATED: + continue + if not (REPO / token).exists(): + errors.append(f"{rel}: names `{token}`, which does not exist") + + +def check_readme_counts() -> None: + """README.md states how many skills, agents and hooks there are; keep it true.""" + readme = (ROOT / "README.md").read_text() + actual = { + "skills": len(list((ROOT / "skills").glob("*/SKILL.md"))), + "subagents": len(list((ROOT / "agents").glob("*.md"))), + "hooks": len([p for p in (ROOT / "hooks").glob("*.sh") if p.name != "selftest.sh"]), + } + for noun, n in actual.items(): + m = re.search(rf"(\d+)\s+{noun}\b", readme) + if m and int(m.group(1)) != n: + errors.append(f"README.md says {m.group(1)} {noun}; there are {n}") + m = re.search(r"(\d+) deny/allow cases", readme) + if m: + selftest = (ROOT / "hooks" / "selftest.sh").read_text() + cases = len(re.findall(r"^expect (?:deny|allow) ", selftest, flags=re.M)) + if int(m.group(1)) != cases: + errors.append(f"README.md says {m.group(1)} deny/allow cases; selftest.sh has {cases}") + + def main() -> int: for skill_md in sorted((ROOT / "skills").glob("*/SKILL.md")): check_skill(skill_md) for agent in sorted((ROOT / "agents").glob("*.md")): check_agent(agent) check_hooks() + for md in sorted([*(ROOT / "skills").rglob("*.md"), *(ROOT / "agents").glob("*.md")]): + check_paths(md) + check_readme_counts() for w in warnings: print(f" warn {w}") diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index 17a61cf..6fd6add 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -58,6 +58,14 @@ jobs: - name: shellcheck run: shellcheck scripts/*.sh + # The .claude/ toolkit: briefs must not name files that are gone, and each guard + # hook must still deny what it says it denies. PyYAML ships with the runner's + # system Python; the install is a no-op there and keeps this honest if it stops. + - name: toolkit (validate.py, hook selftest) + run: | + python3 -c 'import yaml' 2>/dev/null || sudo apt-get install -y --no-install-recommends python3-yaml + make toolkit + # actionlint also runs shellcheck over every `run:` block. Pinned by version and # checked against the published checksum rather than taken from a mutable action # tag or Docker Hub tag; `actionlint_*_checksums.txt` on the release carries it. @@ -87,6 +95,7 @@ jobs: - name: Cargo.lock is unchanged run: git diff --exit-code Cargo.lock + # ⚠ `deny` is pinned as a required status check by the `main` ruleset — see `check`. # Supply chain: RUSTSEC advisories, the licence allowlist, and "every crate came from # crates.io". Config and rationale live in deny.toml; run `cargo deny check` locally # after `cargo install cargo-deny --locked`. Not part of `make check`, which has to diff --git a/AGENTS.md b/AGENTS.md index 55c4ba1..bc99576 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -1,6 +1,30 @@ # Fiber agent notes -- Start here: `make help` and [docs/development.md](docs/development.md). -- Naming: `.claude/rules/naming.md` — prefix is `fiber` / `FIBER_*`. -- DX: `.claude/rules/dx.md` — use Make + `scripts/dev-env.sh`. -- Roadmap (later SCM/Vault/cloud agents): [docs/roadmap.md](docs/roadmap.md). +**[CLAUDE.md](CLAUDE.md) is the guide for any coding agent working here**, whatever its +vendor: commands, architecture, the test strategy, and the gotchas. Read it first. The +rules below are the ones that produce a defect which compiles and passes review by +inspection; the reasoning for each is in +[.claude/skills/fiber-conventions/SKILL.md](.claude/skills/fiber-conventions/SKILL.md). + +1. Product prefix is `fiber` / `FIBER_*` everywhere — never `df` or `durablefibers` + ([.claude/rules/naming.md](.claude/rules/naming.md)). +2. `make check` is the gate (fmt + clippy `-D warnings`), and `make test` the suite. Use + Make targets and `scripts/dev-env.sh`, not ad-hoc shell + ([.claude/rules/dx.md](.claude/rules/dx.md)). +3. CI steps and durable fibers are at-least-once. Anything on the execution path must + survive running twice. +4. A run executes its definition snapshot, never the live pipeline row. +5. `step_attempts` and `log_lines` are append-only. +6. Migrations are append-only: add the next numbered file in + `crates/fiber-core/migrations/`, never edit a committed one. +7. All SQL is runtime `sqlx` — no `query!` macros — so a column mismatch is a runtime + error. Run schema-touching changes against a real Postgres. +8. Every project-scoped HTTP handler goes through `crates/fiber-api/src/access.rs`. +9. Wire types in `fiber-proto` have four consumers, one of them hand-mirrored + TypeScript (`apps/ui/src/lib/api.ts`). Change them together. + +Hazards: never kill agents by argv pattern (it matches your own parent shell — kill by +PID); never remove the Compose volumes to "reset"; `data/`, `target/`, `apps/ui/dist/` +and `apps/ui/src/routeTree.gen.ts` are generated. + +Docs index: [docs/README.md](docs/README.md). Roadmap: [docs/roadmap.md](docs/roadmap.md). diff --git a/CLAUDE.md b/CLAUDE.md index ab733e9..cfdf049 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -42,7 +42,7 @@ cd apps/ui && pnpm check && pnpm exec tsc --noEmit && pnpm build # what CI run End-to-end coverage is the **smoke scripts**, not integration tests — they drive the live API and print `SMOKE_OK`: ```bash -make smoke # authz + agent pools + artifacts (needs infra + api running) +make smoke # authz + artifacts + concurrency + runs + agent pools (needs infra + api running) make smoke-s3 # needs infra-s3 + api-s3 + a built fiber-agent make smoke-compose # full `compose up --build` smoke ``` @@ -65,7 +65,7 @@ Two independent durable systems share one Postgres and one API process — keep - **`fiber-api`** — axum. `routes.rs` is the whole HTTP surface; `ws.rs` holds both `/ws/agent` (agent protocol) and `/ws/runs/{id}` (UI event stream); `access.rs` is the role-gate helper layer; `auth.rs` has the `AuthUser` (bearer session) and `AuthAgent` (agent token) extractors; also `artifacts.rs` (local FS or S3 presign backend), `retention.rs`, `github.rs`, `otel.rs`. - **`fiber-scheduler`** — lease queue, agent registry/connections, offers, lease renewal, expired-lease reclaim, schedule loop, Redis `fiber:events` fan-out. - **`fiber-agent`** — single-binary worker: connects out over WS, claims offers, prepares git workspace, restores artifacts, runs shell or Docker, streams `LogChunk`, uploads artifacts. -- **`fiber-durable`** — separate memoturn-style runtime: `engine.rs` (run/resume one fiber), `context.rs` (`FiberSuspended`), `store.rs`, `scheduler.rs` (poller + stale-heartbeat reclaim), `registry.rs`, `tasks.rs` (built-ins `ping`, `sleep_demo`, `interval_task`). +- **`fiber-durable`** — separate memoturn-style runtime: `engine.rs` (run/resume one fiber), `context.rs` (`FiberSuspended`), `store.rs`, `scheduler.rs` (poller + stale-heartbeat reclaim), `registry.rs`, `tasks.rs` (built-ins `ping`, `sleep_demo`, `interval_task`), `http_task.rs` (`http_request`). - **`fiber-proto`** — wire + YAML-facing types shared by API, agent, and CLI (`AgentMessage`, `ServerMessage`, `RunEvent`, `PipelineDefinition`, `StepDefinition`). - **`fiber-cli`** — the `fiber` binary: `validate`, `login`, run, members, secrets, agents, fibers, and spawning an agent. diff --git a/Makefile b/Makefile index 642f6ec..aefba3e 100644 --- a/Makefile +++ b/Makefile @@ -1,4 +1,4 @@ -.PHONY: help infra infra-s3 infra-minio down api api-s3 agent ui cli build images check fmt fmt-check clippy deny test test-rust test-ui \ +.PHONY: help infra infra-s3 infra-minio down api api-s3 agent ui cli build images check fmt fmt-check clippy deny toolkit test test-rust test-ui \ smoke smoke-authz smoke-pools smoke-artifacts smoke-concurrency smoke-runs smoke-s3 smoke-compose \ login validate ready @@ -41,6 +41,7 @@ help: @echo " build cargo build workspace + fiber-cli" @echo " check THE gate: fmt --check + clippy --workspace --all-targets --locked -D warnings" @echo " deny cargo deny check (advisories, licences, sources) — needs cargo-deny" + @echo " toolkit Check .claude/: validate.py, the hook selftest, shellcheck on the hooks" @echo " test cargo test --workspace --locked + ui vitest" @echo " test-rust cargo test --workspace --locked" @echo " test-ui apps/ui vitest" @@ -55,7 +56,7 @@ help: @echo " login fiber-cli login (writes ~/.fiber/token)" @echo " validate Validate examples/fiber.yml" @echo " ready curl /ready" - @echo " smoke authz + artifacts + concurrency + pools (needs an agent; pools kills them last)" + @echo " smoke authz + artifacts + concurrency + runs + pools (needs an agent; pools kills them last)" @echo " smoke-authz / smoke-artifacts / smoke-concurrency / smoke-runs / smoke-pools one scenario each" @echo " smoke-s3 S3 presign path (needs infra-s3 + api-s3)" @echo " smoke-compose Full Compose stack + a pipeline on the containerised agent" @@ -98,6 +99,14 @@ deny: @command -v cargo-deny >/dev/null || { echo "cargo install cargo-deny --locked"; exit 1; } cargo deny check +# The Claude Code toolkit under .claude/ is code too: briefs that name files which no +# longer exist, and guards that stopped guarding, were both found by hand in the +# November 2026 audit. Needs python3 with PyYAML, jq and shellcheck. CI runs this in `check`. +toolkit: + python3 .claude/validate.py + bash .claude/hooks/selftest.sh + shellcheck -S warning .claude/hooks/*.sh + test: test-rust test-ui test-rust: diff --git a/docs/development.md b/docs/development.md index 571e6a4..64bbf00 100644 --- a/docs/development.md +++ b/docs/development.md @@ -32,6 +32,7 @@ make login # session → ~/.fiber/token make validate # examples/fiber.yml make check # THE gate: fmt --check + clippy --workspace --all-targets --locked -D warnings make deny # cargo deny check (advisories, licences, crates.io-only sources) +make toolkit # .claude/: validate.py, the hook selftest, shellcheck on the hooks make images # build the fiber-api / fiber-agent container images make test # cargo test --workspace --locked + apps/ui vitest make smoke # authz + artifacts + concurrency + runs + pools (pools last: it kills every agent)