diff --git a/docs/SURFACE.md b/docs/SURFACE.md index e231ff694..460f1d27f 100644 --- a/docs/SURFACE.md +++ b/docs/SURFACE.md @@ -494,6 +494,31 @@ The herdr model: first-party helpers are just plugins that ship in the box; the `flows build` seals a flow into a content-addressed, immutable bundle: canonical spec JSON, compiled TS with pinned deps, helper/plugin lockfile, assets, preflight declaration, identity signature — `flow@sha256:…`, pushed to a bucket/registry. `flows deploy` points a trigger at a digest; `flows run flow@sha256:…` executes from the bucket on any cell, no checkout. Preflight runs at build time for everything build-provable and again at deploy time for environment facts (credentials, workers, MCP servers). The working tree is for authoring; **production only ever runs digests.** +The first deployment slice supports `file://` buckets and self-contained +**declarative deterministic** bundles: + +```text +flows deploy @sha256:<64-hex-digest> --to file:///absolute/bucket +flows run @sha256:<64-hex-digest> --bucket file:///absolute/bucket +``` + +Deploy reads `dist/flows/@sha256:/` and publishes the complete +verified bundle at `//sha256//`. Re-deploy prints +`deploy_noop`; a partial copy is never published at the final path. +Run resolves `--bucket` before the nearest `flows.json`'s +`{"deploy":{"bucket":"file:///absolute/bucket"}}`. Verified payloads are cached +under `$XDG_CACHE_HOME/flows/bundles//`, falling back to +`~/.cache/flows/bundles//`. Every cache hit is verified before use. +The canonical spec goes through the existing journal run path with command +preflight before any daemon connection. Local authoring files are unnecessary. +Refusals exit 2; transport failure after copying starts exits 1 (`deploy_partial`). + +**Remaining work for #333:** S3 transport, trigger digest binding and conflict +checks, authored TypeScript execution, assets and placement, agent/LLM environment +preflight, and separating build-provable checks from environment checks. +Unsupported bundle execution is refused with `bundle_unsupported`; this slice +does not claim to implement the full production-digest contract above. + The local build and verification commands are available now: ```text diff --git a/evidence/spec-Adep/README.md b/evidence/spec-Adep/README.md new file mode 100644 index 000000000..7600290de --- /dev/null +++ b/evidence/spec-Adep/README.md @@ -0,0 +1,23 @@ +# Adep minimal slice for #333 + +This branch implements file-bucket deploy and digest-run for self-contained +YAML/declarative deterministic flows. It does not close all of #333. + +The new tests cover signed payload layout, idempotency, missing/unwritable +buckets, corruption before transport or journal creation, cache reuse and +corruption, bucket configuration/override, exact canonical spec submission, +and explicit refusal of asset execution. A real-kernel test deletes the local +source and build output before running the deployed digest to success. + +`verification.txt` contains literal commands, output, and exit codes. The live +check uses an already-built local relayflowd via the recorded RELAYFLOWD_BIN; +this branch does not modify the kernel. The initial regression run exposed +local dependency drift, then Bun missing from PATH. The failed outputs and +the reruns after restoring dependencies and adding the installed Bun to PATH +are retained. + +Follow-up: S3 transport; trigger digest pinning/conflict validation; authored TS, +assets and placement; agent/LLM environment checks; separate environment-only +preflight (this slice retains the existing checks). Deploy-time execution support +is deliberately refused for those bundle kinds. Linux artifact and packed-consumer +CI are left to the lead's PR workflow; they were not run on this node. diff --git a/evidence/spec-Adep/verification.txt b/evidence/spec-Adep/verification.txt new file mode 100644 index 000000000..81eaa4817 --- /dev/null +++ b/evidence/spec-Adep/verification.txt @@ -0,0 +1,186 @@ +Working directory: packages/sdk +RELAYFLOWD_BIN=/Users/khaliqgant/.relayflows-toolchain/target/1475297677/debug/relayflowd + +$ npm run typecheck + +> @relayflows/sdk@2.0.8 typecheck +> tsc --noEmit && tsc -p tsconfig.type-tests.json + +exit=0 + +$ npm run build + +> @relayflows/sdk@2.0.8 build +> tsc && node scripts/make-cli-executable.mjs + +exit=0 + +$ npm run typecheck:tests + +> @relayflows/sdk@2.0.8 typecheck:tests +> tsc -p tsconfig.tests.json + +exit=0 + +$ npx vitest run tests/deploy.test.ts tests/run-from-digest.test.ts tests/run-digest-live.test.ts tests/cli.test.ts tests/bundle.test.ts + + RUN v2.1.9 /Users/khaliqgant/flows-spec-Adep-deploy/packages/sdk + + ✓ tests/run-digest-live.test.ts (1 test) 535ms + ✓ executes a deployed digest on the real kernel after deleting the authoring tree 535ms + ✓ tests/deploy.test.ts (8 tests) 1990ms + ✓ flows deploy file buckets > publishes the full signed layout byte-for-byte and redeploys as a noop 451ms + ✓ flows deploy file buckets > never labels a corrupt existing deployment as a noop 374ms + ✓ tests/run-from-digest.test.ts (6 tests) 2192ms + ✓ flows run digest input > resolves deploy.bucket from flows.json and honors explicit override 602ms + ✓ flows run digest input > refuses an unconfigured bucket 382ms + ✓ flows run digest input > refuses tampered spec.canonical.json before creating run data 375ms + ✓ flows run digest input > refuses tampered identity.json before creating run data 361ms + ❯ tests/bundle.test.ts (21 tests | 1 failed) 2849ms + ✓ immutable bundles > builds and verifies the canonical YAML fixture through the compiled CLI 612ms + ✓ immutable bundles > emits the ephemeral warning on CLI stderr and uses the default output directory 387ms + × immutable bundles > builds a standalone TS fixture twice with identical executable hashes 185ms + → expected 'REFUSED [bundle_invalid] package-lock…' to be '' // Object.is equality + ✓ tests/cli.test.ts (63 tests) 13407ms + ✓ flows check CLI > binds a checked relative wrapper to the flow directory for worker execution 314ms + ✓ flows check CLI > refuses a typo model before probing or contacting relayflowd 933ms + ✓ flows check CLI > maps every input refusal path to its declared kind without raw exceptions 2535ms + ✓ flows run/resume CLI over the journal protocol > parses run options, submits the kernel dialect, and exits 0 on success 684ms + ✓ flows run/resume CLI over the journal protocol > exits 1 and emits the declared completionReason for a failed run 674ms + ✓ flows run/resume CLI over the journal protocol > exits 3 and names the parked llm step 759ms + ✓ flows run/resume CLI over the journal protocol > reports a needs_human agent step as parked for human recovery 713ms + ✓ flows run/resume CLI over the journal protocol > classifies a typed hello refusal as a protocol error, not an unreachable daemon 1011ms + ✓ flows run/resume CLI over the journal protocol > follows a dispatched worker step instead of reporting a protocol error 672ms + ✓ flows run/resume CLI over the journal protocol > bounds a worker wait by its lease and reports what it is waiting for 659ms + ✓ flows run/resume CLI over the journal protocol > resumes a parked run from snapshot step types without reading journal sequence one 678ms + ✓ flows run/resume CLI over the journal protocol > maps only run_not_found resumes to exit 2 1948ms + +⎯⎯⎯⎯⎯⎯⎯ Failed Tests 1 ⎯⎯⎯⎯⎯⎯⎯ + + FAIL tests/bundle.test.ts > immutable bundles > builds a standalone TS fixture twice with identical executable hashes +AssertionError: expected 'REFUSED [bundle_invalid] package-lock…' to be '' // Object.is equality + +- Expected ++ Received + ++ REFUSED [bundle_invalid] package-lock.json: node_modules/@jridgewell/sourcemap-codec does not match its pinned version; run npm ci before building ++ + + ❯ tests/bundle.test.ts:210:27 + 208| it('builds a standalone TS fixture twice with identical executable h… + 209| const result = invoke(['--out', await temp(), 'packages/sdk/tests/… + 210| expect(result.stderr).toBe(''); expect(result.status).toBe(0); + | ^ + 211| const bundle = result.stdout.trim(); + 212| const second = invoke(['--out', await temp(), 'packages/sdk/tests/… + +⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯[1/1]⎯ + + Test Files 1 failed | 4 passed (5) + Tests 1 failed | 98 passed (99) + Start at 19:08:34 + Duration 14.15s (transform 390ms, setup 0ms, collect 1.81s, tests 20.97s, environment 0ms, prepare 172ms) + +exit=1 + +Dependency repair: npm ci in packages/sdk, then link only node_modules/@relayflows/surface to the built packages/surface (same version). No lockfile or dependency manifest changes. + +$ npm run typecheck + +> @relayflows/sdk@2.0.8 typecheck +> tsc --noEmit && tsc -p tsconfig.type-tests.json + +exit=0 + +$ npm run build + +> @relayflows/sdk@2.0.8 build +> tsc && node scripts/make-cli-executable.mjs + +exit=0 + +$ npm run typecheck:tests + +> @relayflows/sdk@2.0.8 typecheck:tests +> tsc -p tsconfig.tests.json + +exit=0 + +$ npx vitest run tests/deploy.test.ts tests/run-from-digest.test.ts tests/run-digest-live.test.ts tests/cli.test.ts tests/bundle.test.ts + + RUN v2.1.9 /Users/khaliqgant/flows-spec-Adep-deploy/packages/sdk + + ✓ tests/run-digest-live.test.ts (1 test) 556ms + ✓ executes a deployed digest on the real kernel after deleting the authoring tree 556ms + ✓ tests/deploy.test.ts (8 tests) 2154ms + ✓ flows deploy file buckets > publishes the full signed layout byte-for-byte and redeploys as a noop 487ms + ✓ flows deploy file buckets > never labels a corrupt existing deployment as a noop 395ms + ✓ tests/run-from-digest.test.ts (6 tests) 2337ms + ✓ flows run digest input > resolves deploy.bucket from flows.json and honors explicit override 666ms + ✓ flows run digest input > refuses an unconfigured bucket 385ms + ✓ flows run digest input > refuses tampered spec.canonical.json before creating run data 382ms + ✓ flows run digest input > refuses tampered identity.json before creating run data 369ms + ❯ tests/bundle.test.ts (21 tests | 1 failed) 3075ms + ✓ immutable bundles > builds and verifies the canonical YAML fixture through the compiled CLI 682ms + ✓ immutable bundles > emits the ephemeral warning on CLI stderr and uses the default output directory 408ms + × immutable bundles > builds a standalone TS fixture twice with identical executable hashes 218ms + → expected 'REFUSED [bundle_invalid] TypeScript b…' to be '' // Object.is equality + ✓ tests/cli.test.ts (63 tests) 12718ms + ✓ flows check CLI > binds a checked relative wrapper to the flow directory for worker execution 319ms + ✓ flows check CLI > uses the raw Claude adapter model flag instead of accepting auth status as model proof 315ms + ✓ flows check CLI > refuses a typo model before probing or contacting relayflowd 1015ms + ✓ flows check CLI > maps every input refusal path to its declared kind without raw exceptions 2032ms + ✓ flows run/resume CLI over the journal protocol > parses run options, submits the kernel dialect, and exits 0 on success 616ms + ✓ flows run/resume CLI over the journal protocol > exits 1 and emits the declared completionReason for a failed run 567ms + ✓ flows run/resume CLI over the journal protocol > exits 3 and names the parked llm step 697ms + ✓ flows run/resume CLI over the journal protocol > reports a needs_human agent step as parked for human recovery 1152ms + ✓ flows run/resume CLI over the journal protocol > classifies a typed hello refusal as a protocol error, not an unreachable daemon 610ms + ✓ flows run/resume CLI over the journal protocol > follows a dispatched worker step instead of reporting a protocol error 549ms + ✓ flows run/resume CLI over the journal protocol > bounds a worker wait by its lease and reports what it is waiting for 750ms + ✓ flows run/resume CLI over the journal protocol > resumes a parked run from snapshot step types without reading journal sequence one 612ms + ✓ flows run/resume CLI over the journal protocol > maps only run_not_found resumes to exit 2 2254ms + +⎯⎯⎯⎯⎯⎯⎯ Failed Tests 1 ⎯⎯⎯⎯⎯⎯⎯ + + FAIL tests/bundle.test.ts > immutable bundles > builds a standalone TS fixture twice with identical executable hashes +AssertionError: expected 'REFUSED [bundle_invalid] TypeScript b…' to be '' // Object.is equality + +- Expected ++ Received + ++ REFUSED [bundle_invalid] TypeScript build requires Bun: spawnSync bun ENOENT ++ + + ❯ tests/bundle.test.ts:210:27 + 208| it('builds a standalone TS fixture twice with identical executable h… + 209| const result = invoke(['--out', await temp(), 'packages/sdk/tests/… + 210| expect(result.stderr).toBe(''); expect(result.status).toBe(0); + | ^ + 211| const bundle = result.stdout.trim(); + 212| const second = invoke(['--out', await temp(), 'packages/sdk/tests/… + +⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯[1/1]⎯ + + Test Files 1 failed | 4 passed (5) + Tests 1 failed | 98 passed (99) + Start at 19:09:41 + Duration 13.42s (transform 350ms, setup 0ms, collect 1.74s, tests 20.84s, environment 0ms, prepare 201ms) + +exit=1 + +Bun was installed but absent from PATH. +$ PATH=/Users/khaliqgant/.bun/bin:$PATH npx vitest run tests/bundle.test.ts + + RUN v2.1.9 /Users/khaliqgant/flows-spec-Adep-deploy/packages/sdk + + ✓ tests/bundle.test.ts (21 tests) 3717ms + ✓ immutable bundles > builds and verifies the canonical YAML fixture through the compiled CLI 542ms + ✓ immutable bundles > emits the ephemeral warning on CLI stderr and uses the default output directory 346ms + ✓ immutable bundles > builds a standalone TS fixture twice with identical executable hashes 1234ms + + Test Files 1 passed (1) + Tests 21 passed (21) + Start at 19:10:12 + Duration 4.20s (transform 99ms, setup 0ms, collect 289ms, tests 3.72s, environment 0ms, prepare 38ms) + +exit=0 diff --git a/packages/sdk/src/bundle-transport.ts b/packages/sdk/src/bundle-transport.ts new file mode 100644 index 000000000..ed0151032 --- /dev/null +++ b/packages/sdk/src/bundle-transport.ts @@ -0,0 +1,79 @@ +import { constants } from 'node:fs'; +import { access, cp, lstat, mkdir, mkdtemp, rename, rm } from 'node:fs/promises'; +import { dirname, isAbsolute, join } from 'node:path'; +import { homedir } from 'node:os'; +import { fileURLToPath } from 'node:url'; +import { verifyBundle } from './bundle.js'; + +export type BundleFailureKind = 'bucket_unconfigured' | 'bucket_unreachable' + | 'bundle_missing_locally' | 'bundle_signature_invalid' | 'bundle_unsupported' | 'deploy_partial'; +export class BundleFailure extends Error { + constructor(readonly kind: BundleFailureKind, message: string) { super(message); } +} +export interface DigestReference { name: string; digest: string } +export function parseDigestReference(value: string): DigestReference | undefined { + // Match the safe single-component name grammar used by sealBundle. + const match = /^([A-Za-z0-9][A-Za-z0-9._-]*)@sha256:([0-9a-f]{64})$/.exec(value); + return match ? { name: match[1]!, digest: match[2]! } : undefined; +} +export function bucketDirectory(uri: string, ref: DigestReference): string { + try { + const url = new URL(uri); + if (url.protocol !== 'file:' || url.search || url.hash) throw new Error('expected file:// URI'); + const path = fileURLToPath(url); + if (!isAbsolute(path)) throw new Error('expected absolute bucket path'); + return join(path, ref.name, 'sha256', ref.digest); + } catch { + throw new BundleFailure('bucket_unreachable', 'This slice requires an absolute file:// bucket URI.'); + } +} +export async function exists(path: string): Promise { + try { await lstat(path); return true; } + catch (error) { if ((error as NodeJS.ErrnoException).code === 'ENOENT') return false; throw error; } +} +export async function verifyDigest(directory: string, digest: string): Promise { + try { await verifyBundle(directory, digest); } + catch (error) { throw new BundleFailure('bundle_signature_invalid', String(error)); } +} +/** Publish only a complete, verified directory. Concurrent identical writers converge. */ +export async function copyBundle(source: string, target: string, digest: string): Promise { + if (await exists(target)) { await verifyDigest(target, digest); return false; } + await mkdir(dirname(target), { recursive: true }); + const staging = await mkdtemp(join(dirname(target), '.bundle-')); + try { + await cp(source, staging, { recursive: true, dereference: false, verbatimSymlinks: true }); + await verifyDigest(staging, digest); + try { await rename(staging, target); } + catch (error) { + if (!['EEXIST', 'ENOTEMPTY'].includes((error as NodeJS.ErrnoException).code ?? '')) throw error; + await verifyDigest(target, digest); + return false; + } + return true; + } finally { await rm(staging, { recursive: true, force: true }); } +} +export async function writableBucket(target: string): Promise { + try { + await mkdir(dirname(target), { recursive: true }); + await access(dirname(target), constants.W_OK); + const probe = await mkdtemp(join(dirname(target), '.probe-')); + await rm(probe, { recursive: true }); + } catch (error) { throw new BundleFailure('bucket_unreachable', String(error)); } +} +export async function fetchBundle(ref: DigestReference, bucket: string): Promise { + const cacheRoot = process.env['XDG_CACHE_HOME']; + const cache = join(cacheRoot && isAbsolute(cacheRoot) ? cacheRoot : join(homedir(), '.cache'), + 'flows', 'bundles', ref.digest); + // Verify every hit; a corrupt cache never executes or silently falls back. + if (await exists(cache)) { await verifyDigest(cache, ref.digest); return cache; } + const source = bucketDirectory(bucket, ref); + try { + if (!await exists(source)) throw new BundleFailure('bucket_unreachable', `Bundle is absent at ${bucket}.`); + await verifyDigest(source, ref.digest); + await copyBundle(source, cache, ref.digest); + return cache; + } catch (error) { + if (error instanceof BundleFailure) throw error; + throw new BundleFailure('bucket_unreachable', String(error)); + } +} diff --git a/packages/sdk/src/bundle.ts b/packages/sdk/src/bundle.ts index 73b28ef4b..470785f90 100644 --- a/packages/sdk/src/bundle.ts +++ b/packages/sdk/src/bundle.ts @@ -112,8 +112,9 @@ async function regularFile(root: string, path: string): Promise { } } -/** Verify an untrusted directory without following symlinks or manifest traversal. */ -export async function verifyBundle(directory: string): Promise { +/** Verify an untrusted directory without following symlinks or manifest traversal. + * Bucket/cache readers supply the requested digest; local builds bind it to the directory name. */ +export async function verifyBundle(directory: string, expectedDigest?: string): Promise { const root = resolve(directory); if (!(await lstat(root)).isDirectory()) throw new Error('manifest.json: bundle must be a directory, not a symlink'); const raw = (await regularFile(root, 'manifest.json')).toString('utf8'); @@ -146,7 +147,9 @@ export async function verifyBundle(directory: string): Promise { const canonical = canonicalize(manifest); if (raw !== canonical) throw new Error('manifest.json: noncanonical bytes'); const digest = sha256(canonical); - if (!basename(root).endsWith(`@sha256:${digest}`)) throw new Error('manifest.json: directory digest mismatch'); + const matchesDigest = expectedDigest === undefined + ? basename(root).endsWith(`@sha256:${digest}`) : digest === expectedDigest; + if (!matchesDigest) throw new Error('manifest.json: directory digest mismatch'); await rejectExtras(root, '', new Set([...paths, 'manifest.json', 'identity.json'])); try { const identity = JSON.parse((await regularFile(root, 'identity.json')).toString('utf8')); diff --git a/packages/sdk/src/cli.ts b/packages/sdk/src/cli.ts index a20bd2947..c6437bda9 100644 --- a/packages/sdk/src/cli.ts +++ b/packages/sdk/src/cli.ts @@ -24,6 +24,8 @@ import { parseReplayArgs, replayJournal, type ReplayArgs } from './cli/replay.js import { checkTypeScriptFlow } from './cli/check-typescript.js'; import { runCloudCli } from './cli/cloud-run.js'; import { isAuthoredFlowPath } from './direct-input.js'; +import { parseDeployArgs, runDeploy, type DeployArgs } from './cli/deploy.js'; +import { parseDigestReference } from './bundle-transport.js'; import { parseBuildArgs, runBuild, type BuildArgs } from './cli/build.js'; import { runHnMonitor } from './cli/hn-monitor.js'; import { runTickRunner } from './cli/tick-runner.js'; @@ -44,10 +46,11 @@ type CliExitCode = 0 | 1 | 2 | 3; type ParsedArgs = | ReplayArgs | BuildArgs + | DeployArgs | { command: 'serve-webhook'; dataDir: string; port: number } | { command: 'cloud-run'; value: string; json: boolean; wait: boolean } | { command: 'check'; json: boolean; watch: boolean; value: string } - | { command: 'run'; reuseFromRunId: string | undefined; localAgent: boolean; dataDir: string; input: string | undefined; json: boolean; spawn: boolean; noObserverLink: boolean; value: string } + | { command: 'run'; bucket: string | undefined; reuseFromRunId: string | undefined; localAgent: boolean; dataDir: string; input: string | undefined; json: boolean; spawn: boolean; noObserverLink: boolean; value: string } | { command: 'resume'; dataDir: string; json: boolean; spawn: boolean; noObserverLink: boolean; value: string } | { command: 'observer'; dataDir: string } | { command: 'hn-monitor'; sub: 'start'; dataDir: string; specPath: string; pollIntervalMs: number | undefined } @@ -60,6 +63,8 @@ const USAGE = [ 'Usage:', 'flows build [--out ] ', 'flows build --verify ', + 'flows deploy @sha256: --to ', + 'flows run @sha256: [--bucket ] [--data-dir ] [--json]', 'flows check [--watch] [--json] ', 'flows serve-webhook --data-dir --port

', 'flows run [--json] [--no-spawn] [--no-observer-link] [--data-dir

] [--local-agent] [--reuse-from ] ', @@ -108,6 +113,7 @@ export async function runCli( if (parsed.command === 'cloud-run') return runCloudCli(parsed, io); if (parsed.command === 'replay') return replayJournal(parsed, io); if (parsed.command === 'build') return runBuild(parsed, io); + if (parsed.command === 'deploy') return runDeploy(parsed, io); if (parsed.command === 'check') { if (parsed.watch) return watchCheck(parsed.value, parsed.json, io); @@ -176,6 +182,7 @@ export async function runCli( if (!parsed.json) for (const line of renderProgress([event])) io.stderr(line); }; const lifecycle = { + ...(parsed.command === 'run' ? { bucket: parsed.bucket } : {}), ...(parsed.command === 'run' && parsed.reuseFromRunId !== undefined ? { reuseFromRunId: parsed.reuseFromRunId } : {}), localAgent: parsed.command === 'run' && parsed.localAgent, onProgress: showProgress, @@ -404,6 +411,7 @@ function parseArgs(args: readonly string[]): ParsedArgs | undefined { const command = args[0]; if (command === 'replay') return parseReplayArgs(args.slice(1)); if (command === 'build') return parseBuildArgs(args.slice(1)); + if (command === 'deploy') return parseDeployArgs(args.slice(1)); if (command === 'serve-webhook') return parseWebhookArgs(args.slice(1)); if (command === 'hn-monitor') return parseHnMonitorArgs(args.slice(1)); if (command === 'tick') return parseTickArgs(args.slice(1)); @@ -422,6 +430,7 @@ function parseArgs(args: readonly string[]): ParsedArgs | undefined { let input: string | undefined; let sawInput = false; let reuseFromRunId: string | undefined; + let bucket: string | undefined; const positionals: string[] = []; for (let index = 1; index < args.length; index += 1) { const argument = args[index]!; @@ -475,6 +484,12 @@ function parseArgs(args: readonly string[]): ParsedArgs | undefined { index += 1; continue; } + if (argument === '--bucket') { + const value = args[++index]; + if (command !== 'run' || bucket !== undefined || !value || value.startsWith('-')) return undefined; + bucket = value; + continue; + } if (argument === '--input') { const value = args[index + 1]; if (command !== 'run' || sawInput || value === undefined || value.startsWith('--')) return undefined; @@ -488,6 +503,7 @@ function parseArgs(args: readonly string[]): ParsedArgs | undefined { } if (positionals.length !== 1) return undefined; + if (bucket !== undefined && (cloud || !parseDigestReference(positionals[0]!))) return undefined; if (cloud) { // `--cloud` submits the spec to Cloud, so every flag that only describes a // local run -- an inline input, a data dir, a suppressed daemon, a local @@ -503,7 +519,7 @@ function parseArgs(args: readonly string[]): ParsedArgs | undefined { return command === 'check' ? { command, json, watch, value: positionals[0]! } : command === 'run' - ? { command, reuseFromRunId, localAgent, dataDir, input, json, spawn, noObserverLink, value: positionals[0]! } + ? { command, bucket, reuseFromRunId, localAgent, dataDir, input, json, spawn, noObserverLink, value: positionals[0]! } : { command, dataDir, json, spawn, noObserverLink, value: positionals[0]! }; } diff --git a/packages/sdk/src/cli/bundle-preflight.ts b/packages/sdk/src/cli/bundle-preflight.ts new file mode 100644 index 000000000..bc0e98a36 --- /dev/null +++ b/packages/sdk/src/cli/bundle-preflight.ts @@ -0,0 +1,28 @@ +import { readFile } from 'node:fs/promises'; +import { join } from 'node:path'; +import { BundleFailure } from '../bundle-transport.js'; +import { compileSpec, kernelToAuthoring } from '../compile.js'; +import { checkAuthoredFlow, type CheckExecution } from './check.js'; + +/** Refuse unsupported execution: a TS declaration is not its body, and + * daemon-relative assets must never resolve in an unrelated checkout. */ +export async function checkRunnableBundle(directory: string, name: string): Promise { + const metadata = JSON.parse(await readFile(join(directory, 'metadata.json'), 'utf8')); + const manifest = JSON.parse(await readFile(join(directory, 'manifest.json'), 'utf8')) as { path: string }[]; + const flow = compileSpec(kernelToAuthoring(JSON.parse(await readFile(join(directory, 'spec.canonical.json'), 'utf8')))); + if (flow.name !== undefined && flow.name !== name) throw new BundleFailure('bundle_signature_invalid', 'Bundle name does not match the requested flow.'); + flow.name ??= name; + if (metadata.kind !== 'declarative' || flow.steps.some(step => step.type !== 'deterministic' + || step.requirements !== undefined) || (flow.triggers?.length ?? 0) > 0 + || manifest.some(entry => entry.path.startsWith('assets/'))) { + throw new BundleFailure('bundle_unsupported', + 'This slice runs declarative deterministic bundles without assets, placement requirements, or triggers.'); + } + // Keep existing fail-closed environment checks before transport/journal. + // Splitting static and environment checks is follow-up. + // This deterministic-only slice declares no project models or executors. + // Never discover flows.json in the deployment/cache tree or its ancestors. + return checkAuthoredFlow(flow, join(directory, 'spec.canonical.json'), { + directory, models: [], executors: [], + }); +} diff --git a/packages/sdk/src/cli/check.ts b/packages/sdk/src/cli/check.ts index eb4cc8934..2c294d09b 100644 --- a/packages/sdk/src/cli/check.ts +++ b/packages/sdk/src/cli/check.ts @@ -28,6 +28,7 @@ import { } from '../preflight.js'; export interface ProjectConfig { + deploy?: { bucket: string }; mcp?: Record; cli?: string; executors: string[]; @@ -63,7 +64,7 @@ export interface CheckExecution { flow?: FlowSpec; } -class CheckFailure extends Error { +export class CheckFailure extends Error { constructor(readonly kind: CheckFailureKind, message: string) { super(message); } @@ -101,10 +102,10 @@ export function checkFlow(path: string): CheckExecution { } /** Preflight a validated authored flow through the same path as YAML/JSON. */ -export function checkAuthoredFlow(authoring: FlowSpec, path: string): CheckExecution { +export function checkAuthoredFlow(authoring: FlowSpec, path: string, projectConfig?: ProjectConfig): CheckExecution { const absolutePath = resolve(path); try { - const config = readProjectConfig(dirname(absolutePath)); + const config = projectConfig ?? readProjectConfig(dirname(absolutePath)); const probes = systemProbes(dirname(absolutePath), config); const result = preflight(authoring, { projectCli: config.cli, @@ -198,8 +199,13 @@ export function readProjectConfig(start: string): ProjectConfig { } catch { throw new CheckFailure('config_invalid', `Project config "${configPath}" is not valid JSON.`); } - if (!isObject(value) || Object.keys(value).some((key) => !['cli', 'executors', 'models', 'mcp'].includes(key))) { - throw new CheckFailure('config_invalid', `Project config "${configPath}" expects only cli, executors, models, and mcp.`); + if (!isObject(value) || Object.keys(value).some((key) => !['cli', 'executors', 'models', 'mcp', 'deploy'].includes(key))) { + throw new CheckFailure('config_invalid', `Project config "${configPath}" expects only cli, executors, models, mcp, and deploy.`); + } + if (value['deploy'] !== undefined && (!isObject(value['deploy']) + || Object.keys(value['deploy']).some(key => key !== 'bucket') + || !isNonEmptyString(value['deploy']['bucket']))) { + throw new CheckFailure('config_invalid', `Project config "${configPath}" has invalid deploy.bucket.`); } if (value['cli'] !== undefined && !isNonEmptyString(value['cli'])) { throw new CheckFailure('config_invalid', `Project config "${configPath}" has an invalid cli.`); @@ -233,6 +239,7 @@ export function readProjectConfig(start: string): ProjectConfig { } } return { + ...(value['deploy'] !== undefined ? { deploy: value['deploy'] as { bucket: string } } : {}), ...(mcp !== undefined ? { mcp } : {}), ...(value['cli'] !== undefined ? { cli: value['cli'] as string } : {}), executors: (value['executors'] as string[] | undefined) ?? [], diff --git a/packages/sdk/src/cli/deploy.ts b/packages/sdk/src/cli/deploy.ts new file mode 100644 index 000000000..250583de0 --- /dev/null +++ b/packages/sdk/src/cli/deploy.ts @@ -0,0 +1,52 @@ +import { join } from 'node:path'; +import type { CliIo } from '../cli.js'; +import { BundleFailure, bucketDirectory, copyBundle, exists, parseDigestReference, + verifyDigest, writableBucket } from '../bundle-transport.js'; +import { checkRunnableBundle } from './bundle-preflight.js'; + +export interface DeployArgs { command: 'deploy'; value: string; to: string } +export function parseDeployArgs(args: readonly string[]): DeployArgs | undefined { + let value: string | undefined; + let to: string | undefined; + for (let i = 0; i < args.length; i++) { + const arg = args[i]!; + if (arg === '--to') { + if (to !== undefined || !args[i + 1] || args[i + 1]!.startsWith('-')) return undefined; + to = args[++i]; + } else if (arg.startsWith('-') || value !== undefined) return undefined; + else value = arg; + } + return value && to && parseDigestReference(value) ? { command: 'deploy', value, to } : undefined; +} +export async function runDeploy(args: DeployArgs, io: CliIo): Promise<0 | 1 | 2> { + let started = false; + try { + const ref = parseDigestReference(args.value); + if (!ref) throw new BundleFailure('bundle_missing_locally', 'Expected @sha256:<64 lowercase hex characters>.'); + const source = join(process.cwd(), 'dist', 'flows', args.value); + if (!await exists(source)) throw new BundleFailure('bundle_missing_locally', `Build ${args.value} locally first.`); + await verifyDigest(source, ref.digest); + const checked = await checkRunnableBundle(source, ref.name); + if (!checked.report.ok) { + for (const diagnostic of checked.report.diagnostics) io.stderr(`${diagnostic.severity.toUpperCase()} [${diagnostic.kind}] ${diagnostic.message}`); + return 2; + } + const target = bucketDirectory(args.to, ref); + if (await exists(target)) { + await verifyDigest(target, ref.digest); + io.stderr(`deploy_noop: ${args.value}`); + io.stdout(`SKIPPED (already-present) ${args.value} ${args.to}`); + return 0; + } + await writableBucket(target); + started = true; + const copied = await copyBundle(source, target, ref.digest); + if (!copied) io.stderr(`deploy_noop: ${args.value}`); + io.stdout(`${copied ? 'DEPLOYED' : 'SKIPPED (already-present)'} ${args.value} ${args.to}`); + return 0; + } catch (error) { + const kind = started ? 'deploy_partial' : error instanceof BundleFailure ? error.kind : 'bucket_unreachable'; + io.stderr(`${started ? 'FAILED' : 'REFUSED'} [${kind}] ${error instanceof Error ? error.message : String(error)}`); + return started ? 1 : 2; + } +} diff --git a/packages/sdk/src/cli/run-digest.ts b/packages/sdk/src/cli/run-digest.ts new file mode 100644 index 000000000..e8361a08a --- /dev/null +++ b/packages/sdk/src/cli/run-digest.ts @@ -0,0 +1,24 @@ +import { BundleFailure, fetchBundle, parseDigestReference } from '../bundle-transport.js'; +import { CheckFailure, inputFailureReport, readProjectConfig, type CheckExecution } from './check.js'; +import { checkRunnableBundle } from './bundle-preflight.js'; +import type { RunExecution } from './run.js'; + +export async function prepareDigestRun(reference: string, bucket?: string): Promise { + try { + const ref = parseDigestReference(reference)!; + const selected = bucket ?? readProjectConfig(process.cwd()).deploy?.bucket; + if (!selected) throw new BundleFailure('bucket_unconfigured', 'Set --bucket or deploy.bucket in flows.json.'); + const directory = await fetchBundle(ref, selected); + const checked = await checkRunnableBundle(directory, ref.name); + checked.report.path = reference; + return checked; + } catch (error) { + if (error instanceof CheckFailure) return { report: inputFailureReport(error, reference) }; + const kind = error instanceof BundleFailure && ( + error.kind === 'bucket_unconfigured' || error.kind === 'bucket_unreachable' + || error.kind === 'bundle_signature_invalid' || error.kind === 'bundle_unsupported' + ) ? error.kind : 'bundle_unsupported'; + return { exitCode: 2, report: { ok: false, command: 'run', path: reference, resolutions: [], + diagnostics: [{ severity: 'refusal', kind, message: error instanceof Error ? error.message : String(error) }] } }; + } +} diff --git a/packages/sdk/src/cli/run.ts b/packages/sdk/src/cli/run.ts index 7e2c28e19..86a29d78d 100644 --- a/packages/sdk/src/cli/run.ts +++ b/packages/sdk/src/cli/run.ts @@ -1,3 +1,5 @@ +import { parseDigestReference } from '../bundle-transport.js'; +import { prepareDigestRun } from './run-digest.js'; import { reuseSummary } from './reuse.js'; import { resumeSlackEffect } from '../authored-slack-effect.js'; import { AuthoredFlowExecutionError } from '../authored-flow-error.js'; @@ -65,6 +67,7 @@ export interface RunProgress { } export interface RunLifecycleOptions { + bucket?: string; reuseFromRunId?: string; onProgress?: (event: ProgressEvent) => void; localAgent?: boolean; @@ -83,7 +86,9 @@ export async function runFlow( dataDir: string, options: RunLifecycleOptions = {}, ): Promise { - const checked = checkFlow(path); + const prepared = parseDigestReference(path) ? await prepareDigestRun(path, options.bucket) : undefined; + if (prepared && 'exitCode' in prepared) return prepared; + const checked = prepared ?? checkFlow(path); if (!checked.report.ok || checked.flow === undefined) { return { exitCode: 2, report: fromCheckReport('run', checked.report) }; } diff --git a/packages/sdk/src/failure-kinds.ts b/packages/sdk/src/failure-kinds.ts index df9cb9746..7fbf59584 100644 --- a/packages/sdk/src/failure-kinds.ts +++ b/packages/sdk/src/failure-kinds.ts @@ -70,6 +70,10 @@ export const PREFLIGHT_WARNING_KINDS = [ * step failed. All of them are still exit 2 — refused before a journal write. */ export const RUN_FAILURE_KINDS = [ + 'bucket_unconfigured', + 'bucket_unreachable', + 'bundle_signature_invalid', + 'bundle_unsupported', 'reuse_spec_mismatch', 'reuse_run_not_found', 'reuse_journal_read_failed', diff --git a/packages/sdk/src/spec.ts b/packages/sdk/src/spec.ts index 895ddeb21..949785199 100644 --- a/packages/sdk/src/spec.ts +++ b/packages/sdk/src/spec.ts @@ -21,6 +21,7 @@ export type McpServerConfig = | { url: string; headers?: Record }; export interface FlowsJson { + deploy?: { bucket: string }; cli?: string; executors?: string[]; models?: string[]; diff --git a/packages/sdk/tests/bundle-preflight.test.ts b/packages/sdk/tests/bundle-preflight.test.ts new file mode 100644 index 000000000..4086d031c --- /dev/null +++ b/packages/sdk/tests/bundle-preflight.test.ts @@ -0,0 +1,60 @@ +import { afterEach, describe, expect, it, vi } from 'vitest'; +import { rm, writeFile } from 'node:fs/promises'; +import { basename, dirname, join } from 'node:path'; +import { buildFlow } from '../src/cli/build.js'; +import { checkRunnableBundle } from '../src/cli/bundle-preflight.js'; +import { fetchBundle, parseDigestReference } from '../src/bundle-transport.js'; +import { fixture } from './deploy-fixture.js'; + +const roots: string[] = []; +async function setup() { const f = await fixture(); roots.push(f.root); return f; } +afterEach(async () => { + vi.unstubAllEnvs(); + for (const root of roots.splice(0)) await rm(root, { recursive: true, force: true }); +}); + +describe('bundle execution preflight', () => { + it('ignores surrounding cache configuration on a verified cache hit', async () => { + const f = await setup(); + expect(f.invoke(['deploy', f.reference, '--to', f.bucket]).status).toBe(0); + vi.stubEnv('XDG_CACHE_HOME', join(f.root, 'cache')); + const ref = parseDigestReference(f.reference)!; + const cache = await fetchBundle(ref, f.bucket); + const before = await checkRunnableBundle(cache, ref.name); + expect(before.report.ok).toBe(true); + await writeFile(join(dirname(cache), 'flows.json'), '{invalid cache config'); + expect(await fetchBundle(ref, f.bucket)).toBe(cache); + expect(await checkRunnableBundle(cache, ref.name)).toEqual(before); + }); + + it('uses the built alias for a nameless flow even in a digest-only cache directory', async () => { + const f = await setup(); + await writeFile(join(f.root, 'nameless.yaml'), 'version: 0.1.0\nsteps:\n - id: greet\n type: deterministic\n command: echo nameless\n'); + const bundle = await buildFlow(join(f.root, 'nameless.yaml'), join(f.root, 'dist/flows'), () => {}); + const reference = basename(bundle); + expect(reference).toMatch(/^flow@sha256:/); + expect(f.invoke(['deploy', reference, '--to', f.bucket]).status).toBe(0); + vi.stubEnv('XDG_CACHE_HOME', join(f.root, 'cache')); + const ref = parseDigestReference(reference)!; + const cache = await fetchBundle(ref, f.bucket); + expect(basename(cache)).toBe(ref.digest); + const checked = await checkRunnableBundle(cache, ref.name); + expect(checked.report.ok).toBe(true); + expect(checked.flow?.name).toBe('flow'); + }); + + it('still rejects a named spec requested under another alias', async () => { + const f = await setup(); + await expect(checkRunnableBundle(f.bundle, 'other')).rejects.toMatchObject({ kind: 'bundle_signature_invalid' }); + }); + + it('still checks command availability without project configuration', async () => { + const f = await setup(); + await writeFile(join(f.root, 'missing.yaml'), 'version: 0.1.0\nname: missing\nsteps:\n - id: run\n type: deterministic\n command: flows-definitely-missing-command-337\n'); + const bundle = await buildFlow(join(f.root, 'missing.yaml'), join(f.root, 'dist/flows'), () => {}); + const checked = await checkRunnableBundle(bundle, 'missing'); + expect(checked.report.diagnostics).toContainEqual(expect.objectContaining({ + severity: 'warning', kind: 'command_unresolved', + })); + }); +}); diff --git a/packages/sdk/tests/bundle-transport.test.ts b/packages/sdk/tests/bundle-transport.test.ts new file mode 100644 index 000000000..4a990ba88 --- /dev/null +++ b/packages/sdk/tests/bundle-transport.test.ts @@ -0,0 +1,33 @@ +import { afterEach, describe, expect, it } from 'vitest'; +import { rm, writeFile } from 'node:fs/promises'; +import { basename, join } from 'node:path'; +import { parseDigestReference } from '../src/bundle-transport.js'; +import { buildFlow } from '../src/cli/build.js'; +import { fixture } from './deploy-fixture.js'; + +const roots: string[] = []; +afterEach(async () => { for (const root of roots.splice(0)) await rm(root, { recursive: true, force: true }); }); + +describe('digest references', () => { + it.each(['hello', 'Hello', 'hello.world', 'hello_world', '123', 'A_b.c-1'])( + 'accepts and deploys the build output for %s', async name => { + const f = await fixture(); roots.push(f.root); + await writeFile(join(f.root, 'named.yaml'), `version: 0.1.0\nname: ${JSON.stringify(name)}\nsteps:\n - id: greet\n type: deterministic\n command: echo deployed\n`); + const bundle = await buildFlow(join(f.root, 'named.yaml'), join(f.root, 'dist/flows'), () => {}); + const reference = basename(bundle); + expect(parseDigestReference(reference)).toEqual({ name, digest: reference.split('@sha256:')[1] }); + const result = f.invoke(['deploy', reference, '--to', f.bucket]); + expect(result.status, result.stderr).toBe(0); + }, + ); + it.each(['', '.', '..', '../hello', '/hello', 'a/b', 'a\\b', '@hello', '-hello', 'hello world'])( + 'rejects unsafe name %j', name => { + expect(parseDigestReference(`${name}@sha256:${'a'.repeat(64)}`)).toBeUndefined(); + }, + ); + it.each(['a'.repeat(63), 'a'.repeat(65), 'A'.repeat(64), 'g'.repeat(64)])( + 'rejects malformed digest %s', digest => { + expect(parseDigestReference(`hello@sha256:${digest}`)).toBeUndefined(); + }, + ); +}); diff --git a/packages/sdk/tests/deploy-fixture.ts b/packages/sdk/tests/deploy-fixture.ts new file mode 100644 index 000000000..f97282fe0 --- /dev/null +++ b/packages/sdk/tests/deploy-fixture.ts @@ -0,0 +1,23 @@ +import { mkdtemp, writeFile } from 'node:fs/promises'; +import { tmpdir } from 'node:os'; +import { basename, join } from 'node:path'; +import { pathToFileURL, fileURLToPath } from 'node:url'; +import { spawnSync } from 'node:child_process'; +import { buildFlow } from '../src/cli/build.js'; + +export const cli = fileURLToPath(new URL('../dist/cli.js', import.meta.url)); +export async function fixture() { + const root = await mkdtemp(join(tmpdir(), 'flows-deploy-')); + await writeFile(join(root, 'flows.json'), '{}'); + await writeFile(join(root, 'hello.yaml'), 'version: 0.1.0\nname: hello\nsteps:\n - id: greet\n type: deterministic\n command: echo deployed\n'); + const bundle = await buildFlow(join(root, 'hello.yaml'), join(root, 'dist/flows'), () => {}); + const reference = basename(bundle); + const digest = reference.split('@sha256:')[1]!; + const bucket = pathToFileURL(join(root, 'bucket')).href; + const target = join(root, 'bucket/hello/sha256', digest); + const invoke = (args: string[]) => spawnSync(process.execPath, [cli, ...args], { + cwd: root, encoding: 'utf8', timeout: 20_000, + env: { ...process.env, XDG_CACHE_HOME: join(root, 'cache') }, + }); + return { root, bundle, reference, digest, bucket, target, invoke }; +} diff --git a/packages/sdk/tests/deploy.test.ts b/packages/sdk/tests/deploy.test.ts new file mode 100644 index 000000000..23ec14c74 --- /dev/null +++ b/packages/sdk/tests/deploy.test.ts @@ -0,0 +1,90 @@ +import { afterEach, describe, expect, it, vi } from 'vitest'; +import { chmod, readFile, readdir, rm, writeFile } from 'node:fs/promises'; +import { join } from 'node:path'; +import { pathToFileURL } from 'node:url'; +import { sha256, verifyBundle, type BundleEntry } from '../src/bundle.js'; +import { buildFlow } from '../src/cli/build.js'; +import { basename } from 'node:path'; +import { fixture } from './deploy-fixture.js'; +import * as transport from '../src/bundle-transport.js'; +import { runDeploy } from '../src/cli/deploy.js'; + +const roots: string[] = []; +async function setup() { const f = await fixture(); roots.push(f.root); return f; } +afterEach(async () => { + vi.restoreAllMocks(); + for (const root of roots.splice(0)) await rm(root, { recursive: true, force: true }); +}); +describe('flows deploy file buckets', () => { + it('publishes the full signed layout byte-for-byte and redeploys as a noop', async () => { + const f = await setup(); + const result = f.invoke(['deploy', f.reference, '--to', f.bucket]); + expect(result.status, result.stderr).toBe(0); + expect(await verifyBundle(f.target, f.digest)).toBe(f.digest); + const entries = JSON.parse(await readFile(join(f.target, 'manifest.json'), 'utf8')) as BundleEntry[]; + for (const entry of entries) expect(sha256(await readFile(join(f.target, entry.path)))).toBe(entry.sha256); + for (const file of await readdir(f.bundle)) expect(await readFile(join(f.target, file))).toEqual(await readFile(join(f.bundle, file))); + const second = f.invoke(['deploy', '--to', f.bucket, f.reference]); + expect(second.status).toBe(0); expect(second.stderr).toContain('deploy_noop'); + expect(second.stdout).toContain('SKIPPED (already-present)'); + expect(second.stdout).not.toContain('DEPLOYED'); + }); + it('reports already-present when another writer publishes during the copy', async () => { + const f = await setup(); + vi.spyOn(process, 'cwd').mockReturnValue(f.root); + const copy = transport.copyBundle; + vi.spyOn(transport, 'copyBundle').mockImplementationOnce(async (source, target, digest) => { + await copy(source, target, digest); + return copy(source, target, digest); + }); + const stdout: string[] = []; const stderr: string[] = []; + expect(await runDeploy({ command: 'deploy', value: f.reference, to: f.bucket }, { + stdout: line => stdout.push(line), stderr: line => stderr.push(line), + })).toBe(0); + expect(await verifyBundle(f.target, f.digest)).toBe(f.digest); + expect(stdout.join('\n')).toContain('SKIPPED (already-present)'); + expect(stdout.join('\n')).not.toContain('DEPLOYED'); + expect(stderr.join('\n')).toContain('deploy_noop'); + }); + it('refuses a missing local bundle before creating the bucket', async () => { + const f = await setup(); + await rm(f.bundle, { recursive: true }); + const result = f.invoke(['deploy', f.reference, '--to', f.bucket]); + expect(result.status).toBe(2); expect(result.stderr).toContain('bundle_missing_locally'); + await expect(readdir(f.target)).rejects.toThrow(); + }); + it('refuses an unreachable bucket before copying', async () => { + const f = await setup(); + const result = f.invoke(['deploy', f.reference, '--to', pathToFileURL(join(f.root, 'hello.yaml')).href]); + expect(result.status).toBe(2); expect(result.stderr).toContain('bucket_unreachable'); + }); + it.skipIf(process.getuid?.() === 0)('refuses an unwritable bucket', async () => { + const f = await setup(); + await chmod(f.root, 0o555); + try { + const result = f.invoke(['deploy', f.reference, '--to', f.bucket]); + expect(result.status).toBe(2); expect(result.stderr).toContain('bucket_unreachable'); + } finally { await chmod(f.root, 0o755); } + }); + it.each(['spec.canonical.json', 'identity.json'])('refuses local tampering of %s', async file => { + const f = await setup(); await writeFile(join(f.bundle, file), '{}'); + const result = f.invoke(['deploy', f.reference, '--to', f.bucket]); + expect(result.status).toBe(2); expect(result.stderr).toContain('bundle_signature_invalid'); + await expect(readdir(f.target)).rejects.toThrow(); + }); + it('refuses asset bundles instead of using daemon-relative files', async () => { + const f = await setup(); + await writeFile(join(f.root, 'script.sh'), '#!/bin/sh\necho bundled\n'); + await writeFile(join(f.root, 'asset.yaml'), 'version: 0.1.0\nname: asset\nsteps:\n - id: run\n type: deterministic\n command: ./script.sh\n'); + const built = await buildFlow(join(f.root, 'asset.yaml'), join(f.root, 'dist/flows'), () => {}); + const result = f.invoke(['deploy', basename(built), '--to', f.bucket]); + expect(result.status).toBe(2); expect(result.stderr).toContain('bundle_unsupported'); + }); + it('never labels a corrupt existing deployment as a noop', async () => { + const f = await setup(); expect(f.invoke(['deploy', f.reference, '--to', f.bucket]).status).toBe(0); + await writeFile(join(f.target, 'identity.json'), '{}'); + const result = f.invoke(['deploy', f.reference, '--to', f.bucket]); + expect(result.status).toBe(2); expect(result.stderr).toContain('bundle_signature_invalid'); + expect(result.stderr).not.toContain('deploy_noop'); + }); +}); diff --git a/packages/sdk/tests/run-digest-live.test.ts b/packages/sdk/tests/run-digest-live.test.ts new file mode 100644 index 000000000..66a060a30 --- /dev/null +++ b/packages/sdk/tests/run-digest-live.test.ts @@ -0,0 +1,26 @@ +import { existsSync } from 'node:fs'; +import { readFile, rm } from 'node:fs/promises'; +import { join } from 'node:path'; +import { expect, it } from 'vitest'; +import { fixture } from './deploy-fixture.js'; + +// Opt in with the same real-kernel override used by existing SDK live tests. +it.skipIf(!process.env['RELAYFLOWD_BIN'] || !existsSync(process.env['RELAYFLOWD_BIN']))( + 'executes a deployed digest on the real kernel after deleting the authoring tree', async () => { + const f = await fixture(); const data = join(f.root, 'data'); + try { + expect(f.invoke(['deploy', f.reference, '--to', f.bucket]).status).toBe(0); + await rm(join(f.root, 'dist'), { recursive: true }); await rm(join(f.root, 'hello.yaml')); + const result = f.invoke(['run', f.reference, '--bucket', f.bucket, '--data-dir', data, + '--no-observer-link', '--json']); + expect(result.status, result.stderr || result.stdout).toBe(0); + expect(JSON.parse(result.stdout)).toMatchObject({ completionReason: 'success', completedSteps: 1 }); + } finally { + try { + const connection = JSON.parse(await readFile(join(data, 'connection.json'), 'utf8')); + process.kill(connection.pid, 'SIGTERM'); + } catch { /* No spawned daemon if preflight refused. */ } + await rm(f.root, { recursive: true, force: true }); + } + }, 30_000, +); diff --git a/packages/sdk/tests/run-digest.test.ts b/packages/sdk/tests/run-digest.test.ts new file mode 100644 index 000000000..2c8fe9cf8 --- /dev/null +++ b/packages/sdk/tests/run-digest.test.ts @@ -0,0 +1,23 @@ +import { afterEach, describe, expect, it } from 'vitest'; +import { readdir, rm, writeFile } from 'node:fs/promises'; +import { join } from 'node:path'; +import { fixture } from './deploy-fixture.js'; + +const roots: string[] = []; +afterEach(async () => { for (const root of roots.splice(0)) await rm(root, { recursive: true, force: true }); }); + +describe('digest run configuration refusals', () => { + it.each(['{invalid json', '{"deploy":{}}', '{"deploy":{"bucket":123}}', '{"deploy":{"bucket":""}}'])( + 'reports config_invalid before fetching or starting a run for %s', async config => { + const f = await fixture(); roots.push(f.root); + await writeFile(join(f.root, 'flows.json'), config); + const data = join(f.root, 'data'); + const result = f.invoke(['run', f.reference, '--data-dir', data, '--no-spawn', '--no-observer-link']); + expect(result.status).toBe(2); + expect(result.stderr).toContain('[config_invalid]'); + expect(result.stderr).not.toContain('bundle_unsupported'); + await expect(readdir(join(f.root, 'cache'))).rejects.toThrow(); + await expect(readdir(data)).rejects.toThrow(); + }, + ); +}); diff --git a/packages/sdk/tests/run-from-digest.test.ts b/packages/sdk/tests/run-from-digest.test.ts new file mode 100644 index 000000000..0cef820c0 --- /dev/null +++ b/packages/sdk/tests/run-from-digest.test.ts @@ -0,0 +1,72 @@ +import { afterEach, describe, expect, it, vi } from 'vitest'; +import { once } from 'node:events'; +import { readFile, readdir, rm, writeFile } from 'node:fs/promises'; +import { join } from 'node:path'; +import type { Server } from 'node:net'; +import { runCli } from '../src/cli.js'; +import { fetchBundle, parseDigestReference } from '../src/bundle-transport.js'; +import { socketPathFor } from '../src/daemon-connection.js'; +import { fixture } from './deploy-fixture.js'; +import { sendOk, sendResult, startLoopback } from './journal-client-loopback.js'; +const roots: string[] = []; +const servers: Server[] = []; +async function setup() { + const f = await fixture(); roots.push(f.root); + expect(f.invoke(['deploy', f.reference, '--to', f.bucket]).status).toBe(0); + vi.stubEnv('XDG_CACHE_HOME', join(f.root, 'cache')); + return f; +} +afterEach(async () => { + vi.unstubAllEnvs(); vi.restoreAllMocks(); + for (const server of servers.splice(0)) await new Promise(done => server.close(() => done())); + for (const root of roots.splice(0)) await rm(root, { recursive: true, force: true }); +}); +describe('flows run digest input', () => { + it('submits the sealed canonical spec through the normal journal path without checkout', async () => { + const f = await setup(); + const canonical = JSON.parse(await readFile(join(f.bundle, 'spec.canonical.json'), 'utf8')); + await rm(join(f.root, 'dist'), { recursive: true }); await rm(join(f.root, 'hello.yaml')); + const dataDir = join(f.root, 'data'); let submitted: unknown; + const server = startLoopback(socketPathFor(dataDir), { hello: sendOk, + 'run.start': (ctx, params) => { submitted = params['spec']; sendResult(ctx, { + run_id: 'digest-run', status: 'completed', completion_reason: 'success', completed_steps: 1, + }); }, + }); + servers.push(server); if (!server.listening) await once(server, 'listening'); + const stdout: string[] = []; const stderr: string[] = []; + const code = await runCli(['run', f.reference, '--bucket', f.bucket, '--data-dir', dataDir, + '--no-spawn', '--no-observer-link', '--json'], { stdout: s => stdout.push(s), stderr: s => stderr.push(s) }); + expect(code, stderr.join('\n')).toBe(0); expect(submitted).toEqual(canonical); + expect(JSON.parse(stdout.join('\n')).completionReason).toBe('success'); + expect(await readdir(join(f.root, 'cache/flows/bundles', f.digest))).toContain('identity.json'); + }); + it('uses a verified cache hit even after the bucket is removed', async () => { + const f = await setup(); const ref = parseDigestReference(f.reference)!; + const first = await fetchBundle(ref, f.bucket); + await rm(join(f.root, 'bucket'), { recursive: true }); + expect(await fetchBundle(ref, f.bucket)).toBe(first); + await writeFile(join(first, 'identity.json'), '{}'); + await expect(fetchBundle(ref, f.bucket)).rejects.toMatchObject({ kind: 'bundle_signature_invalid' }); + }); + it('resolves deploy.bucket from flows.json and honors explicit override', async () => { + const f = await setup(); await writeFile(join(f.root, 'flows.json'), JSON.stringify({ deploy: { bucket: f.bucket } })); + const args = ['run', f.reference, '--no-spawn', '--no-observer-link', '--data-dir', join(f.root, 'data')]; + const configured = f.invoke(args); + expect(configured.status).toBe(2); expect(configured.stderr).toContain('daemon_unreachable'); + await rm(join(f.root, 'cache'), { recursive: true }); + const overridden = f.invoke([...args, '--bucket', 'file:///definitely-absent-flows-bucket']); + expect(overridden.status).toBe(2); expect(overridden.stderr).toContain('bucket_unreachable'); + }); + it('refuses an unconfigured bucket', async () => { + const f = await setup(); + const result = f.invoke(['run', f.reference, '--no-observer-link']); + expect(result.status).toBe(2); expect(result.stderr).toContain('bucket_unconfigured'); + }); + it.each(['spec.canonical.json', 'identity.json'])('refuses tampered %s before creating run data', async file => { + const f = await setup(); await writeFile(join(f.target, file), '{}'); + const data = join(f.root, 'data'); + const result = f.invoke(['run', f.reference, '--bucket', f.bucket, '--data-dir', data, '--no-observer-link']); + expect(result.status).toBe(2); expect(result.stderr).toContain('bundle_signature_invalid'); + await expect(readdir(data)).rejects.toThrow(); + }); +}); diff --git a/packages/sdk/tsconfig.tests.json b/packages/sdk/tsconfig.tests.json index 6996328e6..c5882e2ee 100644 --- a/packages/sdk/tsconfig.tests.json +++ b/packages/sdk/tsconfig.tests.json @@ -11,6 +11,9 @@ "include": [ "src/**/*.ts", "tests/bundle.test.ts", + "tests/deploy.test.ts", + "tests/run-from-digest.test.ts", + "tests/run-digest-live.test.ts", "tests/fixtures/build.flow.ts", "tests/typed-output.test.ts", "tests/mcp.test.ts",