refactor(sql-fs): centralize sandbox fencing policy

.changeset/shared-sandbox-fencing.md

@@ -0,0 +1,5 @@
+---
+"sql-fs-api": patch
+---
+
+Centralize Postgres sandbox fencing while preserving transaction context, lock order, and statement counts. Add regression coverage for stale writes and concurrent transactions, and fix verification commands that could fail under shell pipefail.

.claude/skills/verify/SKILL.md

@@ -45,7 +45,7 @@ $S up --no-build # reuse the current dist/
 
 `up` refuses while a stack is recorded for this checkout. Reuse it (`$S doctor`) or `$S down` first. After a failed `up`, run `$S down` too, because the containers may be running.
 
-Server environment overrides: `SERVER_ENV="SQL_FS_WORKERS_ENABLED=false SQL_FS_WORKER_OUTPUT_BYTES=1048576" $S restart`. Overrides persist across later restarts until you pass `SERVER_ENV= $S restart`. Values are split on spaces, so use compact JSON with no spaces. `server.log` and `doctor` show only the override names; the values stay in the owner-only `state.env`.
+Server environment overrides: `SERVER_ENV="SQL_FS_WORKERS_ENABLED=false SQL_FS_WORKER_OUTPUT_BYTES=1048576" $S restart`. Overrides persist across later restarts until you pass `SERVER_ENV="" $S restart`. Values are split on spaces, so use compact JSON with no spaces. `server.log` and `doctor` show only the override names; the values stay in the owner-only `state.env`.
 
 ## Doctor
 

.claude/skills/verify/features/exec.md

@@ -26,7 +26,7 @@ Preconditions:
 - **SSE stream.** Run `$S api POST /v1/sandboxes/$ID/exec --json '{"script":"printf stream-proof"}'`. The output contains `event: stdout` with `"data":"stream-proof"` and exactly one `event: exit` with `"exitCode":0`.
 - **Batch.** Run `$S api POST /v1/sandboxes/$ID/exec-sync-batch --json '{"scripts":[{"id":"a","script":"echo one"},{"id":"b","script":"echo two"}]}' | jq -c '[.results[]|[.id,.exitCode,.stdout]]'`. The output is `[["a",0,"one\n"],["b",0,"two\n"]]`.
 - **Timeout prefix.** Run `EXPECT=408 $S api POST /v1/sandboxes/$ID/exec-sync --json '{"script":"echo p > /home/user/p.txt; sleep 2; echo s > /home/user/s.txt","timeoutMs":100}'`. The `code` is `EXEC_TIMEOUT`. Then `$S api GET /v1/sandboxes/$ID/files/home/user/p.txt` prints `p` and `EXPECT=404 $S api GET /v1/sandboxes/$ID/files/home/user/s.txt` succeeds.
-- **Worker modes.** Repeat the sync run after `SERVER_ENV="SQL_FS_WORKERS_ENABLED=false" $S restart --no-build`, then restore with `SERVER_ENV= $S restart --no-build`. Both modes must give identical results.
+- **Worker modes.** Repeat the sync run after `SERVER_ENV="SQL_FS_WORKERS_ENABLED=false" $S restart --no-build`, then restore with `SERVER_ENV="" $S restart --no-build`. Both modes must give identical results.
 
 ## Gotchas
 

.claude/skills/verify/scripts/journeys.sh

@@ -60,10 +60,10 @@ journey_exec() {
 	eq "$(EXPECT=408 api POST "/v1/sandboxes/$id/exec-sync" --json '{"script":"echo p > /home/user/p.txt; sleep 2; echo s > /home/user/s.txt","timeoutMs":100}' | jq -r .code)" EXEC_TIMEOUT "timeout -> 408"
 	eq "$(api GET "/v1/sandboxes/$id/files/home/user/p.txt")" p "timeout kept prefix"
 	eq "$(absent "/v1/sandboxes/$id/files/home/user/s.txt")" absent "timeout dropped suffix"
-	trap 'SERVER_ENV= $S restart --no-build >/dev/null' EXIT
+	trap 'SERVER_ENV="" $S restart --no-build >/dev/null' EXIT
 	SERVER_ENV="SQL_FS_WORKERS_ENABLED=false" $S restart --no-build >/dev/null
 	eq "$(api POST "/v1/sandboxes/$id/exec-sync" --json '{"script":"cat /home/user/a.txt"}' | jq -c '{exitCode,stdout}')" '{"exitCode":0,"stdout":"hi\n"}' "workers disabled: same result"
-	SERVER_ENV= $S restart --no-build >/dev/null
+	SERVER_ENV="" $S restart --no-build >/dev/null
 	trap - EXIT
 	drop "$id"
 }

.claude/skills/verify/scripts/stack.sh

@@ -97,7 +97,8 @@ cmd_up() {
 	command -v docker >/dev/null && docker info >/dev/null 2>&1 || die "Docker daemon is not running"
 	[[ -d node_modules ]] || die "run pnpm setup:dev first"
 	deps_current || die "node_modules does not match pnpm-lock.yaml; run: pnpm install --frozen-lockfile"
-	local id="stack-$(date +%s)-$$"
+	local id
+	id="stack-$(date +%s)-$$"
 	DIR="$runs/$id"
 	PROJECT="sqlfs-$id"
 	if docker compose version >/dev/null 2>&1; then COMPOSE="docker compose"; else COMPOSE="docker-compose"; fi
@@ -149,8 +150,10 @@ cmd_doctor() {
 	[[ "$code" == 200 ]] && ok "minted token accepted (GET /v1/sandboxes 200)" || fail "GET /v1/sandboxes returned $code"
 	deps_current && ok "node_modules matches pnpm-lock.yaml" ||
 		fail "node_modules does not match pnpm-lock.yaml -> run: pnpm install --frozen-lockfile && $0 restart"
-	local newer
-	newer=$(find src \( -name '*.ts' -o -name '*.sql' \) -newer dist/api/server.js -not -path '*/tests/*' 2>/dev/null | head -3)
+	local sources newer
+	[[ -f dist/api/server.js ]] || { fail "dist/api/server.js is missing -> run: $0 restart"; return 1; }
+	sources=$(find src \( -name '*.ts' -o -name '*.sql' \) -newer dist/api/server.js -not -path '*/tests/*' 2>/dev/null)
+	newer=$(head -3 <<<"$sources")
 	if [[ -n "$newer" ]]; then
 		fail "dist is stale; source changed after build: $(echo "$newer" | tr '\n' ' ') -> run: $0 restart"
 	elif [[ dist/api/server.js -nt "$DIR/state.env" ]]; then
@@ -210,9 +213,10 @@ suite_names() {
 cmd_suite() {
 	load_state
 	(($# >= 1)) || die "usage: $0 suite NAME [args]   (NAME: integration or a suite entry point in scripts/verify/)"
-	local name=$1
+	local name=$1 names
 	shift
-	suite_names | grep -qx "$name" || die "$name is not a suite entry point (have: $(suite_names | tr '\n' ' '))"
+	names=$(suite_names)
+	grep -qx "$name" <<<"$names" || die "$name is not a suite entry point (have: $(tr '\n' ' ' <<<"$names"))"
 	local -a suite_env=("VERIFY_ENV_FILE=none" "VERIFY_EVIDENCE_DIR=$DIR/suites" "VERIFY_PYTHON=$repo/clients/python/.venv/bin/python")
 	mkdir -p "$DIR/suites"
 	if [[ "$name" == integration ]]; then

docs/DEVELOPER.md

@@ -146,7 +146,7 @@ The session runtime opens a script scope only if the filesystem implements `IScr
 
 ## Ownership checks
 
-`pnpm check:architecture` enforces the worker module's named adapter imports and the script-scope completion owner. The allowed imports live in `scripts/check-architecture-boundaries.mjs`, with fixture tests beside the checker. Add a new adapter only when it has a real external responsibility; keep dispatch history, reservation state, and outcome classification inside worker ownership. The shared lease-loss context lives in `src/execution/owner-cancellation.ts` so filesystem fencing does not import worker internals.
+`pnpm check:architecture` enforces the worker module's named adapter imports, the script-scope completion owner, and the sandbox fence policy owner in `src/sql-fs/dialects/postgres/sandbox.ts`. The allowed imports live in `scripts/check-architecture-boundaries.mjs`, with fixture tests beside the checker. Add a new adapter only when it has a real external responsibility; keep dispatch history, reservation state, and outcome classification inside worker ownership. The shared lease-loss context lives in `src/execution/owner-cancellation.ts` so filesystem fencing does not import worker internals.
 
 ## Worker owners run scripts off the main thread
 

scripts/check-architecture-boundaries.mjs

@@ -5,7 +5,7 @@ import { fileURLToPath } from "node:url";
 import * as ts from "typescript/unstable/sync";
 import * as ast from "typescript/unstable/ast";
 
-const { API, SymbolFlags } = ts;
+const { API, SignatureKind, SymbolFlags, TypeFlags } = ts;
 const { SyntaxKind } = ast;
 
 const workerImportAllowlist = new Map([
@@ -44,6 +44,40 @@ for (const importer of [
 
 const scopeOperations = new Set(["beginScope", "endScope", "abortScope"]);
 
+const fencePolicyConsumers = new Set([
+	"src/sql-fs/dialects/postgres/namespace.ts",
+	"src/sql-fs/dialects/postgres/ingest.ts",
+]);
+const fencePolicyStatements = [/\bapp\.sandbox_epoch\b/i, /\bversion\s*=\s*(?:\w+\.)?version\s*\+/i];
+
+function forbiddenSqlApiAccess(node, checker) {
+	let receiver;
+	let name;
+	if (ast.isPropertyAccessExpression(node)) {
+		receiver = node.expression;
+		name = node.name.text;
+	} else if (ast.isElementAccessExpression(node) && ast.isStringLiteral(node.argumentExpression)) {
+		receiver = node.expression;
+		name = node.argumentExpression.text;
+	} else if (ast.isBindingElement(node) && ast.isObjectBindingPattern(node.parent)) {
+		receiver = node.parent;
+		const property = node.propertyName ?? node.name;
+		if (ast.isIdentifier(property) || ast.isStringLiteral(property)) name = property.text;
+	}
+	if (name !== "unsafe" && name !== "file") return false;
+	const type = checker.getTypeAtLocation(receiver);
+	return (
+		type === undefined ||
+		(type.flags & (TypeFlags.Any | TypeFlags.Unknown)) !== 0 ||
+		checker.getSignaturesOfType(type, SignatureKind.Call).length > 0
+	);
+}
+
+function templateLiteralText(template) {
+	if (ast.isNoSubstitutionTemplateLiteral(template)) return template.text;
+	return [template.head.text, ...template.templateSpans.map((span) => span.literal.text)].join(" ${} ");
+}
+
 function relative(root, filename) {
 	return path.relative(root, filename).split(path.sep).join("/");
 }
@@ -414,6 +448,34 @@ export function checkArchitectureBoundaries(root) {
 					}
 				}
 
+				if (fencePolicyConsumers.has(sourcePath)) {
+					if (forbiddenSqlApiAccess(node, checker)) {
+						diagnostics.push(
+							diagnostic(
+								absoluteRoot,
+								sourceFile,
+								node,
+								"FENCE_POLICY_RAW_SQL",
+								"tx.unsafe and tx.file bypass fence-policy inspection; use tagged SQL and safe value builders",
+							),
+						);
+					}
+					if (
+						ast.isTaggedTemplateExpression(node) &&
+						fencePolicyStatements.some((pattern) => pattern.test(templateLiteralText(node.template)))
+					) {
+						diagnostics.push(
+							diagnostic(
+								absoluteRoot,
+								sourceFile,
+								node,
+								"FENCE_POLICY_OWNER",
+								"epoch acceptance, epoch publication, and sandboxes.version advancement belong to the fence policy in sandbox.ts; use fenceAndAdvance or compositeFenceAndAdvance",
+							),
+						);
+					}
+				}
+
 				if (ast.isCallExpression(node)) {
 					const expression = unwrapParentheses(node.expression);
 					if (

scripts/check-architecture-boundaries.test.mjs

@@ -188,3 +188,112 @@ it("permits literal non-worker imports and rejects indirect scope method calls",
 		["SCRIPT_SCOPE_COMPLETION_OWNER", "SCRIPT_SCOPE_COMPLETION_OWNER", "SCRIPT_SCOPE_COMPLETION_OWNER"],
 	);
 });
+
+const sqlTag = "declare const tx: (strings: TemplateStringsArray, ...values: unknown[]) => unknown;\n";
+
+it("rejects fence policy copied into namespace and ingest SQL", () => {
+	const root = project({
+		"src/sql-fs/dialects/postgres/namespace.ts": `${sqlTag}export function copy(id: string, epoch: string | null) {
+	tx\`WITH ctx AS (SELECT s.version FROM sandboxes s WHERE s.id = \${id}
+		AND s.version = COALESCE(\${epoch}::bigint, NULLIF(current_setting('app.sandbox_epoch', true), '')::bigint, s.version))
+		SELECT 1 FROM ctx\`;
+	tx\`UPDATE sandboxes s
+		SET version = s.version + 1 WHERE s.id = \${id}\`;
+}
+`,
+		"src/sql-fs/dialects/postgres/ingest.ts": `${sqlTag}export function publish(version: string) {
+	tx\`SELECT set_config('app.sandbox_epoch', \${version}, true)\`;
+}
+`,
+	});
+	const diagnostics = checkArchitectureBoundaries(root).filter(({ code }) => code === "FENCE_POLICY_OWNER");
+	assert.deepEqual(
+		diagnostics.map(({ file, line }) => [file, line]),
+		[
+			["src/sql-fs/dialects/postgres/ingest.ts", 3],
+			["src/sql-fs/dialects/postgres/namespace.ts", 3],
+			["src/sql-fs/dialects/postgres/namespace.ts", 6],
+		],
+	);
+});
+
+it("permits epoch consumers, version reads, and the fence policy owner", () => {
+	const root = project({
+		"src/sql-fs/dialects/postgres/sandbox.ts": `${sqlTag}export function fence(id: string) {
+	return tx\`ctx AS (SELECT s.version FROM sandboxes s WHERE s.version = NULLIF(current_setting('app.sandbox_epoch', true), '')::bigint),
+		advanced AS (UPDATE sandboxes s SET version = s.version + 1 FROM ctx WHERE s.id = \${id} RETURNING s.version)\`;
+}
+`,
+		"src/sql-fs/dialects/postgres/namespace.ts": `${sqlTag}// app.sandbox_epoch is published by the fence policy
+const prose = "UPDATE sandboxes SET version = version + 1";
+export function consume(id: string, fence: unknown) {
+	tx\`WITH \${fence}, inserted AS (INSERT INTO inodes (sandbox_id) SELECT \${id} FROM epoch RETURNING id)
+		SELECT (SELECT count(*) FROM epoch)::int AS fenced, (SELECT id FROM inserted) AS id
+		FROM sandboxes WHERE id = \${id}\`;
+	tx\`SELECT version FROM sandboxes WHERE id = \${id}\`;
+	tx\`UPDATE sandboxes SET name = \${prose} WHERE id = \${id}\`;
+	tx\`UPDATE sandboxes SET name = \${prose} WHERE version = \${id}\`;
+}
+`,
+	});
+	assert.deepEqual(checkArchitectureBoundaries(root), []);
+});
+
+const sqlCapabilities = `declare const tx: {
+	(strings: TemplateStringsArray, ...values: unknown[]): unknown;
+	(values: unknown[] | Record<string, unknown>): unknown;
+	unsafe(query: string): unknown;
+	file(path: string): unknown;
+};
+`;
+
+it("rejects direct, extracted, destructured, and aliased raw SQL capabilities in both fence consumers", () => {
+	const accesses = [
+		'tx.unsafe("UPDATE sandboxes SET version = version + 1")',
+		'tx["unsafe"]("SELECT 1")',
+		'const run = tx.unsafe; run("SELECT 1")',
+		'const run = tx["unsafe"]; run("SELECT 1")',
+		'const { unsafe: run } = tx; run("SELECT 1")',
+		'const { "unsafe": run } = tx; run("SELECT 1")',
+		'const db = tx; db.unsafe("SELECT 1")',
+		'tx.file("query.sql")',
+		'tx["file"]("query.sql")',
+		'const run = tx.file; run("query.sql")',
+		'const run = tx["file"]; run("query.sql")',
+		'const { file: run } = tx; run("query.sql")',
+		'const db = tx; db.file("query.sql")',
+	];
+	const body = accesses.map((access) => `{ ${access}; }`).join("\n");
+	const files = ["namespace", "ingest"].map((name) => `src/sql-fs/dialects/postgres/${name}.ts`);
+	const root = project(Object.fromEntries(files.map((file) => [file, sqlCapabilities + body])));
+	const diagnostics = checkArchitectureBoundaries(root);
+	assert.equal(diagnostics.length, files.length * accesses.length);
+	for (const file of files) {
+		assert.deepEqual(
+			diagnostics.filter((item) => item.file === file).map(({ code, line }) => ({ code, line })),
+			accesses.map((_, index) => ({ code: "FENCE_POLICY_RAW_SQL", line: index + 7 })),
+		);
+	}
+});
+
+it("permits safe SQL builders, unrelated file data, and raw SQL in the sandbox lifecycle owner", () => {
+	const root = project({
+		"src/sql-fs/dialects/postgres/namespace.ts": `${sqlCapabilities}
+const db = tx;
+const prose = "UPDATE sandboxes SET version = version + 1";
+db([1, 2]);
+db({ name: prose });
+db\`SELECT \${prose}\`;
+const entry = { file: "notes.txt", unsafe: false };
+void entry.file;
+void entry["file"];
+const { file, unsafe } = entry;
+`,
+		"src/sql-fs/dialects/postgres/sandbox.ts": `${sqlCapabilities}
+tx.unsafe("UPDATE sandboxes SET version = version + 1");
+const { file: run } = tx;
+run("lifecycle.sql");
+`,
+	});
+	assert.deepEqual(checkArchitectureBoundaries(root), []);
+});

src/sql-fs/dialects/postgres/README.md

@@ -13,6 +13,21 @@ Group contracts in [types.ts](types.ts) derive from SqlDialect. Postgres-specifi
 
 The root blob commit and batch commit remain independent single statements outside the caller's namespace transaction. Ingest invokes its blob commit after directory and overwrite checks, before inserting file inodes. Cache backfill remains asynchronous.
 
-Writer-lock statements precede fenced CTEs so their snapshots follow any lock wait. Fenced CTE mutations reference and gate their writes on the epoch CTE; lower-level mutations run directly in the caller transaction. Both mvComposite statements and its two version advances remain together. Sandbox bootstrap inserts remain part of sandbox lifecycle rather than ordinary mkdir operations.
+Writer-lock statements precede fenced CTEs so READ COMMITTED statement snapshots follow any lock wait. REPEATABLE READ retains an established transaction snapshot. Fenced CTE mutations reference and gate their writes on the epoch CTE; lower-level mutations run directly in the caller transaction. Both mvComposite statements and its two version advances remain together. Sandbox bootstrap inserts remain part of sandbox lifecycle rather than ordinary mkdir operations.
+
+## Fencing
+
+| To change | Edit | Verify with |
+| --- | --- | --- |
+| How a namespace or ingest write accepts an epoch, advances the version, and publishes the transaction-local epoch | The private `fenceCtes` policy in [sandbox.ts](sandbox.ts). Lifecycle methods in the same file, such as retirement, context locking, and runtime updates, advance or publish directly. | [fence-policy](../../tests/integration/fence-policy.integration.test.ts), [fencing](../../tests/integration/fencing.integration.test.ts), and [epoch coverage](../../tests/integration/epoch-fence-coverage.integration.test.ts) integration suites |
+| A namespace operation | Its complete statement in [namespace.ts](namespace.ts) | The same suites, plus [move overwrite](../../tests/integration/mvcomposite-overwrite.integration.test.ts) for moves |
+
+`fenceAndAdvance` and `compositeFenceAndAdvance` return lazy SQL fragments that callers place in their own `WITH` clause. Keep them synchronous. An async builder would await the fragment, and awaiting a postgres.js query executes it. `fenceAndAdvance` projects `s.version` for callers that already set context, which covers lower-level namespace mutations and ingest. `compositeFenceAndAdvance` also sets `app.sandbox_id` and takes the reentrant advisory lock inline. Composite callers still run `takeWriterLock` first.
+
+Run the integration suites on the disposable verify stack, whose `sqlfs_app` role is neither superuser nor BYPASSRLS. A rejected write that rolls back looks the same as one that never ran, so the fence-policy and epoch coverage suites include cases that catch `ESTALE` and commit before checking state.
+
+The fence-policy suite pins the advisory-lock wait verdicts under READ COMMITTED and an established REPEATABLE READ snapshot; its `it.each` table lists them.
+
+`pnpm check:architecture` rejects `app.sandbox_epoch` references and `sandboxes.version` increments in tagged SQL in namespace.ts and ingest.ts. It also rejects `unsafe` and `file` raw SQL access in those two consumers. The check does not parse arbitrary SQL strings or track computed names, reflection, or capabilities hidden behind noncallable types. Sandbox lifecycle operations remain owned by sandbox.ts.
 
 GC uses the transaction and isolation selected by its caller. It does not create a nested transaction. Namespace and sandbox operations use the caller transaction. Ingest keeps namespace writes there, but commits its blob batch independently through the root pool.