diff --git a/.claude/README.md b/.claude/README.md index cb8f98f..cd7d23a 100644 --- a/.claude/README.md +++ b/.claude/README.md @@ -15,7 +15,7 @@ Verify after any change here: ```bash python3 .claude/validate.py # frontmatter, spec compliance, hook registration -bash .claude/hooks/selftest.sh # 24 deny/allow cases against the guard hooks +bash .claude/hooks/selftest.sh # 53 deny/allow cases against the guard hooks ``` ## Skills — the procedures @@ -48,8 +48,8 @@ Delegate work that needs its own context. Claude routes on the `description`, or | `fiber-devops` | inherit | Compose, Dockerfiles, CI workflow, ports, deploys | | `fiber-docs-syncer` | inherit | Fixes docs drift against the code | | `fiber-smoke-runner` | inherit | Runs the smokes, triages to a cause | -| `fiber-reviewer` | opus | Reviews a diff against the real invariants (read-only) | -| `fiber-security-reviewer` | opus | Auth, secrets, tokens, webhooks, artifacts (read-only) | +| `fiber-reviewer` | inherit | Reviews a diff against the real invariants (read-only) | +| `fiber-security-reviewer` | inherit | Auth, secrets, tokens, webhooks, artifacts (read-only) | | `fiber-dx-reviewer` | sonnet | Onboarding friction and the five files that drift (read-only) | The three reviewers have no `Edit`/`Write` — they report, you decide. @@ -62,7 +62,7 @@ Deterministic, so they do not depend on the model remembering. Wired in `setting | Hook | Event | Does | |---|---|---| -| `guard-bash.sh` | PreToolUse(Bash) | Denies the argv-pattern kill that also kills this session's parent shell; denies volume-destroying Compose teardown; denies writes to *committed* migrations; denies wholesale deletion of `data/workspaces` or `data/artifacts` | +| `guard-bash.sh` | PreToolUse(Bash) | Denies the argv-pattern kill that also kills this session's parent shell (also inside `sh -c` and behind `xargs`); denies volume-destroying teardown under Docker or Podman, by Compose or by `volume rm`/`prune`; denies writing, copying or moving onto *committed* migrations, and deleting them; denies wholesale deletion of `data/`, `data/workspaces` or `data/artifacts` | | `guard-edits.sh` | PreToolUse(Edit\|Write) | Denies edits to applied migrations, `routeTree.gen.ts`, `data/`, `dist/`, `target/`, `Cargo.lock` | | `fmt-rust.sh` | PostToolUse(Edit\|Write) | `rustfmt` on edited `.rs` using the repo's `rustfmt.toml`, keeping `cargo fmt --check` green | | `fmt-ui.sh` | PostToolUse(Edit\|Write) | Biome (not Prettier) on edited `apps/ui` files | @@ -79,4 +79,4 @@ Two design notes. `guard-bash.sh` strips heredoc bodies before matching, so *doc - **New agent**: `agents/.md` with `name` and `description`. Reviewers should omit `Edit`/`Write`. - **New hook**: add the script to `hooks/`, `chmod +x`, register it in `settings.json`, and add deny *and* allow cases to `selftest.sh` — a guard that blocks legitimate work is worse than no guard. -Run `python3 .claude/validate.py` before committing changes to this directory. +Run `python3 .claude/validate.py` before committing changes to this directory. It also fails when a brief names a repository path that no longer exists, or when the counts in this file drift; CI runs it and the hook selftest in the `check` job. diff --git a/.claude/agents/fiber-devops.md b/.claude/agents/fiber-devops.md index ad1543e..1c94808 100644 --- a/.claude/agents/fiber-devops.md +++ b/.claude/agents/fiber-devops.md @@ -10,7 +10,7 @@ You own Fiber's build, packaging, and deployment surface. ## Territory -`deploy/docker-compose.yml`, `deploy/Dockerfile.api`, `deploy/Dockerfile.ui`, `deploy/nginx.conf`, `.github/workflows/ci.yml`, `Makefile`, `scripts/dev-env.sh`, `apps/ui/Dockerfile.ui`, the `.dockerignore` files. +`deploy/docker-compose.yml`, `deploy/Dockerfile` (one file, `fiber-api` and `fiber-agent` targets), `deploy/agent-docker.Dockerfile`, `deploy/fiber-agent.service`, `apps/ui/Dockerfile.ui`, `apps/ui/nginx.conf`, `.github/workflows/ci.yml`, `.github/workflows/release.yml`, `.github/workflows/advisories.yml`, `.github/dependabot.yml`, `deny.toml`, `Makefile`, `scripts/dev-env.sh`, `scripts/install-agent.sh`, the `.dockerignore` files. ## Fixed facts @@ -27,7 +27,7 @@ Ports are deliberately non-default to avoid colliding with whatever else is on t - Postgres is **17-alpine**. A volume created by 16 will not start under 17; the documented recovery destroys data, so never run it yourself — tell the user. - Compose service names are `fiber-postgres`, `fiber-redis`, `fiber-s3`, `fiber-api`, `fiber-ui`. The `fiber-` prefix is mandatory (`.claude/rules/naming.md`). - The Compose healthcheck uses `GET /ready` (Postgres + Redis reachable). `/health` is liveness only. -- CI is two jobs: `check` (fmt, clippy `-D warnings` across all six crates, `cargo build -p fiber-api -p fiber-agent -p fiber-cli`) and `ui` (pnpm + Node, `pnpm install --frozen-lockfile`, `pnpm exec tsc --noEmit`). `RUSTFLAGS: -Dwarnings` is set in the workflow env. +- CI (`ci.yml`) is six jobs: `check` (`make check`, shellcheck, actionlint, `make test-rust`, `make build`, lockfile drift), `deny` (`cargo deny check`), `ui` (Biome, `tsc --noEmit`, vitest, `pnpm build`), `docker` (image builds), `smoke` (the Compose smoke) and `smoke-host`. `check`, `ui`, `docker`, `smoke-host` and `deny` are required checks on the `main` ruleset, pinned by job name — renaming a job means editing the ruleset. The gate is `make check`; there is no `RUSTFLAGS` in the workflow env. - `fiber-api` applies migrations on boot, so rolling an image forward rolls the schema forward. Migrations must stay compatible with the *previous* image for the duration of a rolling deploy. - Redis carries only the `fiber:events` fan-out; Postgres leases are the source of truth. Multiple API replicas are fine; each agent holds a WS to one replica. diff --git a/.claude/agents/fiber-reviewer.md b/.claude/agents/fiber-reviewer.md index e474199..c1967a7 100644 --- a/.claude/agents/fiber-reviewer.md +++ b/.claude/agents/fiber-reviewer.md @@ -2,7 +2,7 @@ name: fiber-reviewer description: Reviews changed code against Fiber's real invariants — at-least-once replay safety, role gates, snapshot immutability, migration hygiene, cross-crate wire-type drift. Use proactively after implementing a change and before committing. tools: Read, Grep, Glob, Bash -model: opus +model: inherit color: yellow --- diff --git a/.claude/agents/fiber-security-reviewer.md b/.claude/agents/fiber-security-reviewer.md index 8fc49c4..c347df2 100644 --- a/.claude/agents/fiber-security-reviewer.md +++ b/.claude/agents/fiber-security-reviewer.md @@ -2,7 +2,7 @@ name: fiber-security-reviewer description: Security review for Fiber — a CI system that executes untrusted-ish shell on agents, holds project secrets, mints tokens, and verifies GitHub webhooks. Use before merging anything touching auth, secrets, tokens, webhooks, artifacts, or agent execution. tools: Read, Grep, Glob, Bash -model: opus +model: inherit color: red --- diff --git a/.claude/hooks/guard-bash.sh b/.claude/hooks/guard-bash.sh index b14b945..29365b6 100755 --- a/.claude/hooks/guard-bash.sh +++ b/.claude/hooks/guard-bash.sh @@ -26,10 +26,19 @@ deny() { printf 'Blocked by fiber toolkit: %s\n' "$1" >&2; exit 2; } # Match only where the binary is invoked as a command word (start of line, or # after a pipe/separator), not merely mentioned inside a longer string. -cmdword() { printf '%s' "$scan" | grep -Eq "(^|[;&|]|&&|\|\||\\bthen |\\bdo )[[:space:]]*(sudo[[:space:]]+)?$1"; } +# +# "Command word" also covers the two wrappers that run their argument as a command: +# `sh -c ''` (any of sh/bash/zsh, with or without -l) and `xargs `. Without +# them the kill below went straight through as `bash -c "pkill …"` or `… | xargs pkill …`. +cmdword() { + printf '%s' "$scan" | grep -Eq "(^|[;&|(]|&&|\|\||\\bthen |\\bdo |\\b(ba|z)?sh[[:space:]]+-l?c[[:space:]]+[\"']|\\bxargs[[:space:]]+(-[^[:space:]]+[[:space:]]+)*)[[:space:]]*(sudo[[:space:]]+)?$1" +} -# 1. pkill -f fiber-agent matches parent shells whose argv mentions the binary path. -if cmdword '(pkill|killall)[^;&|]*fiber-agent'; then +# 1. Killing by argv pattern matches parent shells whose argv mentions the binary path. +# Two shapes: any pkill/killall naming fiber-agent, and `-f` with any pattern that +# mentions fiber at all (`pkill -f target/debug/fiber` matches the same shells). +if cmdword '(pkill|killall)[^;&|]*fiber-agent' \ + || cmdword '(pkill|killall)[^;&|]*[[:space:]]-[a-zA-Z]*f[a-zA-Z]*[[:space:]][^;&|]*fiber'; then deny "that kill pattern also matches shells whose argv contains the binary path, and can kill this session's parent. Kill by PID instead: pgrep -x fiber-agent @@ -39,19 +48,40 @@ See docs/development.md 'Agent tips' and .claude/rules/dx.md." fi # 2. Destroying the Postgres volume wipes every run, artifact row, and the admin user. -if printf '%s' "$scan" | grep -Eq 'docker[[:space:]]+compose[^;&|]*[[:space:]]down([[:space:]]|$)[^;&|]*(-v|--volumes)'; then +# Engine-agnostic: the stack runs under Docker or Podman, and both have a plugin form +# (`docker compose`) and a standalone one (`docker-compose`). `-v` may be bundled with +# other short flags or come before or after `down`'s other options. +compose='(docker|podman)([[:space:]]+|-)compose' +if printf '%s' "$scan" | grep -Eq "${compose}[^;&|]*[[:space:]]down([[:space:]][^;&|]*)?[[:space:]](-[a-zA-Z]*v[a-zA-Z]*|--volumes)([[:space:]]|\$|[;&|])"; then deny "'compose down -v' destroys the fiber-postgres volume (all runs, artifacts, users, secrets). Use 'make down' to stop containers and keep data. If you truly intend to wipe (e.g. the documented Postgres 16->17 upgrade), ask the user to run it themselves." fi +# 2b. The same data, removed by name instead of through Compose. +if cmdword '(docker|podman)[[:space:]]+volume[[:space:]]+(rm|remove)[^;&|]*fiber' \ + || cmdword '(docker|podman)[[:space:]]+volume[[:space:]]+prune' \ + || cmdword '(docker|podman)[[:space:]]+system[[:space:]]+prune[^;&|]*--volumes'; then + deny "that removes the fiber-postgres volume (all runs, artifacts, users, secrets), or every unused volume including it. +Use 'make down' to stop containers and keep data. If a wipe is really intended, ask the user to run it themselves." +fi + # 3. sqlx migrations are applied on fiber-api boot and checksummed; rewriting an # applied file diverges from _sqlx_migrations on every existing database. # Only already-committed migrations are protected — writing a NEW numbered # file is the correct way to change the schema and must stay unobstructed. targets="$(printf '%s' "$scan" \ | grep -Eo 'crates/fiber-core/migrations/[0-9]{3}_[A-Za-z0-9_.-]+\.sql' | sort -u)" -if [ -n "$targets" ] && printf '%s' "$scan" | grep -Eq '(>>?|sed -i|tee|truncate)[^;&|]*crates/fiber-core/migrations/'; then +# Writers: redirection, in-place editors, and anything that copies or moves a file +# *onto* a migration (the path is the last argument). Copying a migration somewhere +# else, reading one, and `git checkout -- ` back to HEAD stay allowed. +mig='crates/fiber-core/migrations/' +writes_migration() { + printf '%s' "$scan" | grep -Eq "(>>?|sed[[:space:]]+(-[a-zA-Z]+[[:space:]]+)*-i|perl[[:space:]]+-[a-zA-Z]*i|tee|truncate|dd[^;&|]*of=)[^;&|]*$mig" \ + || printf '%s' "$scan" | grep -Eq "(^|[;&|]|&&)[[:space:]]*(cp|mv|install|rsync|ln)[[:space:]][^;&|]*${mig}[0-9]{3}_[A-Za-z0-9_.-]+\.sql[[:space:]]*(\$|[;&|])" \ + || printf '%s' "$scan" | grep -Eq "(^|[;&|]|&&)[[:space:]]*(rm|git[[:space:]]+rm)[[:space:]][^;&|]*$mig" +} +if [ -n "$targets" ] && writes_migration; then while IFS= read -r f; do [ -z "$f" ] && continue if git -C "${CLAUDE_PROJECT_DIR:-.}" ls-files --error-unmatch "$f" >/dev/null 2>&1; then @@ -65,7 +95,10 @@ EOF fi # 4. Truncating the workspace/artifact roots out from under a running agent. -if printf '%s' "$scan" | grep -Eq 'rm[[:space:]]+(-[a-zA-Z]+[[:space:]]+)*[^;&|]*(data/workspaces|data/artifacts)[[:space:]]*($|[;&|])'; then +# The root itself in any spelling: `data/workspaces`, with a trailing slash, with +# `/*` or `/.`, or behind `./`. A named subdirectory is one run and stays allowed. +if printf '%s' "$scan" | grep -Eq 'rm[[:space:]]+(-[a-zA-Z]+[[:space:]]+)*[^;&|]*(data/workspaces|data/artifacts)(/|/\*|/\.)?[[:space:]]*($|[;&|])' \ + || printf '%s' "$scan" | grep -Eq 'rm[[:space:]]+(-[a-zA-Z]+[[:space:]]+)*[^;&|]*[[:space:]](\./)?data(/|/\*)?[[:space:]]*($|[;&|])'; then deny "refusing to delete data/workspaces or data/artifacts wholesale — a running fiber-agent holds leases on them. Delete a specific run's subdirectory, or stop agents first." fi diff --git a/.claude/hooks/selftest.sh b/.claude/hooks/selftest.sh index 2933aa9..6ae9af5 100755 --- a/.claude/hooks/selftest.sh +++ b/.claude/hooks/selftest.sh @@ -32,6 +32,34 @@ expect deny "compose down --volumes" "$HOOKS/guard-bash.sh" "$(payload expect deny "append to applied migration" "$HOOKS/guard-bash.sh" "$(payload 'echo "ALTER TABLE runs ADD COLUMN x int;" >> crates/fiber-core/migrations/001_initial.sql')" expect deny "sed -i on applied migration" "$HOOKS/guard-bash.sh" "$(payload 'sed -i "" s/a/b/ crates/fiber-core/migrations/003_retention.sql')" expect deny "rm -rf data/workspaces" "$HOOKS/guard-bash.sh" "$(payload 'rm -rf data/workspaces')" +# Bypasses found by probing the guard in November 2026; each of these was allowed. +expect deny "pkill inside bash -c" "$HOOKS/guard-bash.sh" "$(payload 'bash -c "pkill -f fiber-agent"')" +expect deny "pkill behind xargs" "$HOOKS/guard-bash.sh" "$(payload 'echo x | xargs pkill -f fiber-agent')" +expect deny "pkill -f on the binary's path" "$HOOKS/guard-bash.sh" "$(payload 'pkill -f target/debug/fiber')" +expect deny "podman compose down -v" "$HOOKS/guard-bash.sh" "$(payload 'podman compose -f deploy/docker-compose.yml down -v')" +expect deny "docker-compose down -v" "$HOOKS/guard-bash.sh" "$(payload 'docker-compose down -v')" +expect deny "podman-compose down --volumes" "$HOOKS/guard-bash.sh" "$(payload 'podman-compose down --volumes')" +expect deny "compose down, -v after a flag" "$HOOKS/guard-bash.sh" "$(payload 'docker compose down --remove-orphans -v')" +expect deny "docker volume rm by name" "$HOOKS/guard-bash.sh" "$(payload 'docker volume rm fiber_fiber-postgres-data')" +expect deny "podman volume prune" "$HOOKS/guard-bash.sh" "$(payload 'podman volume prune -f')" +expect deny "rm -rf data/workspaces/" "$HOOKS/guard-bash.sh" "$(payload 'rm -rf data/workspaces/')" +expect deny "rm -rf ./data/artifacts/*" "$HOOKS/guard-bash.sh" "$(payload 'rm -rf ./data/artifacts/*')" +expect deny "rm -rf data" "$HOOKS/guard-bash.sh" "$(payload 'rm -rf data')" +expect deny "cp onto applied migration" "$HOOKS/guard-bash.sh" "$(payload 'cp /tmp/x.sql crates/fiber-core/migrations/001_initial.sql')" +expect deny "mv onto applied migration" "$HOOKS/guard-bash.sh" "$(payload 'mv /tmp/x.sql crates/fiber-core/migrations/001_initial.sql')" +expect deny "perl -pi on applied migration" "$HOOKS/guard-bash.sh" "$(payload 'perl -pi -e s/a/b/ crates/fiber-core/migrations/001_initial.sql')" +expect deny "rm an applied migration" "$HOOKS/guard-bash.sh" "$(payload 'rm crates/fiber-core/migrations/001_initial.sql')" +expect allow "copy a migration elsewhere" "$HOOKS/guard-bash.sh" "$(payload 'cp crates/fiber-core/migrations/001_initial.sql /tmp/x.sql')" +expect allow "cat an applied migration" "$HOOKS/guard-bash.sh" "$(payload 'cat crates/fiber-core/migrations/001_initial.sql')" +expect allow "git checkout migration to HEAD" "$HOOKS/guard-bash.sh" "$(payload 'git checkout -- crates/fiber-core/migrations/001_initial.sql')" +expect allow "cp a new numbered migration" "$HOOKS/guard-bash.sh" "$(payload 'cp /tmp/x.sql crates/fiber-core/migrations/999_new.sql')" +expect allow "podman compose down (no -v)" "$HOOKS/guard-bash.sh" "$(payload 'podman compose -f deploy/docker-compose.yml down')" +expect allow "compose up -d (a -d, not a -v)" "$HOOKS/guard-bash.sh" "$(payload 'docker compose -f deploy/docker-compose.yml up -d')" +expect allow "docker volume ls" "$HOOKS/guard-bash.sh" "$(payload 'docker volume ls')" +expect allow "rm a file under data/artifacts" "$HOOKS/guard-bash.sh" "$(payload 'rm data/artifacts/ab/cd/out.txt')" +expect allow "rm -rf target/debug" "$HOOKS/guard-bash.sh" "$(payload 'rm -rf target/debug')" +expect allow "pkill something unrelated" "$HOOKS/guard-bash.sh" "$(payload 'pkill -f vite')" +expect allow "kill a fiber-agent by PID" "$HOOKS/guard-bash.sh" "$(payload 'kill 12345')" expect allow "pgrep -x fiber-agent" "$HOOKS/guard-bash.sh" "$(payload 'pgrep -x fiber-agent')" expect allow "make down" "$HOOKS/guard-bash.sh" "$(payload 'make down')" expect allow "compose down (no volumes)" "$HOOKS/guard-bash.sh" "$(payload 'docker compose -f deploy/docker-compose.yml down')" diff --git a/.claude/settings.json b/.claude/settings.json index 3234bee..13604c9 100644 --- a/.claude/settings.json +++ b/.claude/settings.json @@ -71,7 +71,8 @@ "Bash(git diff:*)", "Bash(git log:*)", "Bash(git show:*)", - "Bash(git branch:*)", + "Bash(git branch --list:*)", + "Bash(git branch --show-current)", "Bash(docker ps:*)", "Bash(docker compose -f deploy/docker-compose.yml ps:*)", "Bash(docker compose -f deploy/docker-compose.yml logs:*)", diff --git a/.claude/validate.py b/.claude/validate.py index 35d2aac..ebfa18f 100644 --- a/.claude/validate.py +++ b/.claude/validate.py @@ -201,12 +201,61 @@ def check_hooks() -> None: warnings.append(f"hooks/{script.name} exists but is not registered in settings.json") +# A brief that names a file which no longer exists sends an agent looking for it, and +# then reporting on a layout the repository does not have. `fiber-devops` named +# `deploy/Dockerfile.api` and `deploy/nginx.conf` for two months after they were gone. +REPO = ROOT.parent +PATH_ROOTS = ("deploy/", "scripts/", "crates/", "apps/", "docs/", ".github/", ".claude/", "examples/") +# Build output: named in briefs as something not to edit, absent from a clean checkout. +GENERATED = {"apps/ui/dist/"} +PATH_RE = re.compile(r"`([A-Za-z0-9_.$/-]+)`") + + +def check_paths(md: Path) -> None: + """Every backticked repository path in a brief must exist.""" + rel = md.relative_to(ROOT) + for token in sorted(set(PATH_RE.findall(md.read_text()))): + if not token.startswith(PATH_ROOTS): + continue + # Only whole paths: something with a file extension, or a directory written + # with its trailing slash. `crates/fiber-api` style prose mentions are skipped. + if not (token.endswith("/") or re.search(r"\.[A-Za-z0-9]+$", token)): + continue + if token in GENERATED: + continue + if not (REPO / token).exists(): + errors.append(f"{rel}: names `{token}`, which does not exist") + + +def check_readme_counts() -> None: + """README.md states how many skills, agents and hooks there are; keep it true.""" + readme = (ROOT / "README.md").read_text() + actual = { + "skills": len(list((ROOT / "skills").glob("*/SKILL.md"))), + "subagents": len(list((ROOT / "agents").glob("*.md"))), + "hooks": len([p for p in (ROOT / "hooks").glob("*.sh") if p.name != "selftest.sh"]), + } + for noun, n in actual.items(): + m = re.search(rf"(\d+)\s+{noun}\b", readme) + if m and int(m.group(1)) != n: + errors.append(f"README.md says {m.group(1)} {noun}; there are {n}") + m = re.search(r"(\d+) deny/allow cases", readme) + if m: + selftest = (ROOT / "hooks" / "selftest.sh").read_text() + cases = len(re.findall(r"^expect (?:deny|allow) ", selftest, flags=re.M)) + if int(m.group(1)) != cases: + errors.append(f"README.md says {m.group(1)} deny/allow cases; selftest.sh has {cases}") + + def main() -> int: for skill_md in sorted((ROOT / "skills").glob("*/SKILL.md")): check_skill(skill_md) for agent in sorted((ROOT / "agents").glob("*.md")): check_agent(agent) check_hooks() + for md in sorted([*(ROOT / "skills").rglob("*.md"), *(ROOT / "agents").glob("*.md")]): + check_paths(md) + check_readme_counts() for w in warnings: print(f" warn {w}") diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index 17a61cf..6fd6add 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -58,6 +58,14 @@ jobs: - name: shellcheck run: shellcheck scripts/*.sh + # The .claude/ toolkit: briefs must not name files that are gone, and each guard + # hook must still deny what it says it denies. PyYAML ships with the runner's + # system Python; the install is a no-op there and keeps this honest if it stops. + - name: toolkit (validate.py, hook selftest) + run: | + python3 -c 'import yaml' 2>/dev/null || sudo apt-get install -y --no-install-recommends python3-yaml + make toolkit + # actionlint also runs shellcheck over every `run:` block. Pinned by version and # checked against the published checksum rather than taken from a mutable action # tag or Docker Hub tag; `actionlint_*_checksums.txt` on the release carries it. @@ -87,6 +95,7 @@ jobs: - name: Cargo.lock is unchanged run: git diff --exit-code Cargo.lock + # ⚠ `deny` is pinned as a required status check by the `main` ruleset — see `check`. # Supply chain: RUSTSEC advisories, the licence allowlist, and "every crate came from # crates.io". Config and rationale live in deny.toml; run `cargo deny check` locally # after `cargo install cargo-deny --locked`. Not part of `make check`, which has to diff --git a/AGENTS.md b/AGENTS.md index 55c4ba1..bc99576 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -1,6 +1,30 @@ # Fiber agent notes -- Start here: `make help` and [docs/development.md](docs/development.md). -- Naming: `.claude/rules/naming.md` — prefix is `fiber` / `FIBER_*`. -- DX: `.claude/rules/dx.md` — use Make + `scripts/dev-env.sh`. -- Roadmap (later SCM/Vault/cloud agents): [docs/roadmap.md](docs/roadmap.md). +**[CLAUDE.md](CLAUDE.md) is the guide for any coding agent working here**, whatever its +vendor: commands, architecture, the test strategy, and the gotchas. Read it first. The +rules below are the ones that produce a defect which compiles and passes review by +inspection; the reasoning for each is in +[.claude/skills/fiber-conventions/SKILL.md](.claude/skills/fiber-conventions/SKILL.md). + +1. Product prefix is `fiber` / `FIBER_*` everywhere — never `df` or `durablefibers` + ([.claude/rules/naming.md](.claude/rules/naming.md)). +2. `make check` is the gate (fmt + clippy `-D warnings`), and `make test` the suite. Use + Make targets and `scripts/dev-env.sh`, not ad-hoc shell + ([.claude/rules/dx.md](.claude/rules/dx.md)). +3. CI steps and durable fibers are at-least-once. Anything on the execution path must + survive running twice. +4. A run executes its definition snapshot, never the live pipeline row. +5. `step_attempts` and `log_lines` are append-only. +6. Migrations are append-only: add the next numbered file in + `crates/fiber-core/migrations/`, never edit a committed one. +7. All SQL is runtime `sqlx` — no `query!` macros — so a column mismatch is a runtime + error. Run schema-touching changes against a real Postgres. +8. Every project-scoped HTTP handler goes through `crates/fiber-api/src/access.rs`. +9. Wire types in `fiber-proto` have four consumers, one of them hand-mirrored + TypeScript (`apps/ui/src/lib/api.ts`). Change them together. + +Hazards: never kill agents by argv pattern (it matches your own parent shell — kill by +PID); never remove the Compose volumes to "reset"; `data/`, `target/`, `apps/ui/dist/` +and `apps/ui/src/routeTree.gen.ts` are generated. + +Docs index: [docs/README.md](docs/README.md). Roadmap: [docs/roadmap.md](docs/roadmap.md). diff --git a/CLAUDE.md b/CLAUDE.md index ab733e9..cfdf049 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -42,7 +42,7 @@ cd apps/ui && pnpm check && pnpm exec tsc --noEmit && pnpm build # what CI run End-to-end coverage is the **smoke scripts**, not integration tests — they drive the live API and print `SMOKE_OK`: ```bash -make smoke # authz + agent pools + artifacts (needs infra + api running) +make smoke # authz + artifacts + concurrency + runs + agent pools (needs infra + api running) make smoke-s3 # needs infra-s3 + api-s3 + a built fiber-agent make smoke-compose # full `compose up --build` smoke ``` @@ -65,7 +65,7 @@ Two independent durable systems share one Postgres and one API process — keep - **`fiber-api`** — axum. `routes.rs` is the whole HTTP surface; `ws.rs` holds both `/ws/agent` (agent protocol) and `/ws/runs/{id}` (UI event stream); `access.rs` is the role-gate helper layer; `auth.rs` has the `AuthUser` (bearer session) and `AuthAgent` (agent token) extractors; also `artifacts.rs` (local FS or S3 presign backend), `retention.rs`, `github.rs`, `otel.rs`. - **`fiber-scheduler`** — lease queue, agent registry/connections, offers, lease renewal, expired-lease reclaim, schedule loop, Redis `fiber:events` fan-out. - **`fiber-agent`** — single-binary worker: connects out over WS, claims offers, prepares git workspace, restores artifacts, runs shell or Docker, streams `LogChunk`, uploads artifacts. -- **`fiber-durable`** — separate memoturn-style runtime: `engine.rs` (run/resume one fiber), `context.rs` (`FiberSuspended`), `store.rs`, `scheduler.rs` (poller + stale-heartbeat reclaim), `registry.rs`, `tasks.rs` (built-ins `ping`, `sleep_demo`, `interval_task`). +- **`fiber-durable`** — separate memoturn-style runtime: `engine.rs` (run/resume one fiber), `context.rs` (`FiberSuspended`), `store.rs`, `scheduler.rs` (poller + stale-heartbeat reclaim), `registry.rs`, `tasks.rs` (built-ins `ping`, `sleep_demo`, `interval_task`), `http_task.rs` (`http_request`). - **`fiber-proto`** — wire + YAML-facing types shared by API, agent, and CLI (`AgentMessage`, `ServerMessage`, `RunEvent`, `PipelineDefinition`, `StepDefinition`). - **`fiber-cli`** — the `fiber` binary: `validate`, `login`, run, members, secrets, agents, fibers, and spawning an agent. diff --git a/Makefile b/Makefile index 642f6ec..aefba3e 100644 --- a/Makefile +++ b/Makefile @@ -1,4 +1,4 @@ -.PHONY: help infra infra-s3 infra-minio down api api-s3 agent ui cli build images check fmt fmt-check clippy deny test test-rust test-ui \ +.PHONY: help infra infra-s3 infra-minio down api api-s3 agent ui cli build images check fmt fmt-check clippy deny toolkit test test-rust test-ui \ smoke smoke-authz smoke-pools smoke-artifacts smoke-concurrency smoke-runs smoke-s3 smoke-compose \ login validate ready @@ -41,6 +41,7 @@ help: @echo " build cargo build workspace + fiber-cli" @echo " check THE gate: fmt --check + clippy --workspace --all-targets --locked -D warnings" @echo " deny cargo deny check (advisories, licences, sources) — needs cargo-deny" + @echo " toolkit Check .claude/: validate.py, the hook selftest, shellcheck on the hooks" @echo " test cargo test --workspace --locked + ui vitest" @echo " test-rust cargo test --workspace --locked" @echo " test-ui apps/ui vitest" @@ -55,7 +56,7 @@ help: @echo " login fiber-cli login (writes ~/.fiber/token)" @echo " validate Validate examples/fiber.yml" @echo " ready curl /ready" - @echo " smoke authz + artifacts + concurrency + pools (needs an agent; pools kills them last)" + @echo " smoke authz + artifacts + concurrency + runs + pools (needs an agent; pools kills them last)" @echo " smoke-authz / smoke-artifacts / smoke-concurrency / smoke-runs / smoke-pools one scenario each" @echo " smoke-s3 S3 presign path (needs infra-s3 + api-s3)" @echo " smoke-compose Full Compose stack + a pipeline on the containerised agent" @@ -98,6 +99,14 @@ deny: @command -v cargo-deny >/dev/null || { echo "cargo install cargo-deny --locked"; exit 1; } cargo deny check +# The Claude Code toolkit under .claude/ is code too: briefs that name files which no +# longer exist, and guards that stopped guarding, were both found by hand in the +# November 2026 audit. Needs python3 with PyYAML, jq and shellcheck. CI runs this in `check`. +toolkit: + python3 .claude/validate.py + bash .claude/hooks/selftest.sh + shellcheck -S warning .claude/hooks/*.sh + test: test-rust test-ui test-rust: diff --git a/docs/development.md b/docs/development.md index 571e6a4..64bbf00 100644 --- a/docs/development.md +++ b/docs/development.md @@ -32,6 +32,7 @@ make login # session → ~/.fiber/token make validate # examples/fiber.yml make check # THE gate: fmt --check + clippy --workspace --all-targets --locked -D warnings make deny # cargo deny check (advisories, licences, crates.io-only sources) +make toolkit # .claude/: validate.py, the hook selftest, shellcheck on the hooks make images # build the fiber-api / fiber-agent container images make test # cargo test --workspace --locked + apps/ui vitest make smoke # authz + artifacts + concurrency + runs + pools (pools last: it kills every agent)