feat(sdk,surface): plugin registry — flows add helper-<name> (#305) #336
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,139 @@ | ||
| # Plugin registry minimal slice evidence | ||
|
|
||
| Ships npm installation, effect-only manifests/runtime, authored preflight, and | ||
| optional module augmentation. Follow-ups are listed in SURFACE §3. | ||
| No kernel source changes. No new skipped tests. | ||
|
|
||
| Dependency setup: `npm ci --ignore-scripts` in packages/sdk, followed by copying | ||
| this worktree's built surface `dist` into its installed surface package. Bun was | ||
| added to PATH and the kernel was built from this worktree for the final SDK run. | ||
| An earlier SDK run used an older shared daemon and lacked Bun; final results below | ||
| supersede that environment run. The last receipt metadata edit was covered by the | ||
| focused rerun and both typechecks after the full suite. | ||
|
|
||
| ## Focused checks | ||
|
|
||
| From packages/sdk: | ||
| ``` | ||
| RELAYFLOWD_BIN=/Users/khaliqgant/.relayflows-toolchain/target/2930744996/debug/relayflowd ./node_modules/.bin/vitest run tests/preflight.test.ts tests/plugin-add.test.ts tests/plugin-loader.test.ts | ||
| ``` | ||
| Captured output: | ||
| ``` | ||
|
|
||
| RUN v2.1.9 /Users/khaliqgant/flows-spec-J-plugin/packages/sdk | ||
|
|
||
| ✓ tests/preflight.test.ts (27 tests) 35ms | ||
| ✓ tests/plugin-loader.test.ts (5 tests) 146ms | ||
| ✓ tests/plugin-add.test.ts (7 tests) 772ms | ||
| ✓ installs a real offline npm fixture and includes declarations 369ms | ||
| ✓ typechecks the augmented verb and rejects unknown namespaces 392ms | ||
|
|
||
| Test Files 3 passed (3) | ||
| Tests 39 passed (39) | ||
| Start at 19:12:07 | ||
| Duration 1.31s (transform 281ms, setup 0ms, collect 960ms, tests 953ms, environment 0ms, prepare 98ms) | ||
|
|
||
| ``` | ||
|
|
||
| ## Types | ||
|
|
||
| From packages/sdk: `npm run typecheck && npm run typecheck:tests` (exit 0). | ||
| Captured output: | ||
| ``` | ||
|
|
||
| > @relayflows/sdk@2.0.8 typecheck | ||
| > tsc --noEmit && tsc -p tsconfig.type-tests.json | ||
|
|
||
|
|
||
| > @relayflows/sdk@2.0.8 typecheck:tests | ||
| > tsc -p tsconfig.tests.json | ||
|
|
||
| ``` | ||
|
|
||
| ## Kernel | ||
|
|
||
| From kernel: | ||
| ``` | ||
| PATH=/Users/khaliqgant/.rustup/toolchains/stable-aarch64-apple-darwin/bin:$PATH sh ../ops/cargo.sh test --workspace | ||
| ``` | ||
| Exit 0. Captured result lines (full local log: /private/tmp/spec-J-kernel.log): | ||
| ``` | ||
| test result: ok. 40 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.56s | ||
| test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s | ||
| test result: ok. 3 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.03s | ||
| test result: ok. 40 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 37.52s | ||
| test result: ok. 6 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.07s | ||
| test result: ok. 3 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.02s | ||
| test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.04s | ||
| test result: ok. 2 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.08s | ||
| test result: ok. 3 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 3.55s | ||
| test result: ok. 3 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.07s | ||
| test result: ok. 5 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.03s | ||
| test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.03s | ||
| test result: ok. 4 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.07s | ||
| test result: ok. 3 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.58s | ||
| test result: ok. 3 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.03s | ||
| test result: ok. 2 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.02s | ||
| test result: ok. 4 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.19s | ||
| test result: ok. 3 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.03s | ||
| test result: ok. 3 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.08s | ||
| test result: ok. 60 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.59s | ||
| test result: ok. 4 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.01s | ||
| test result: ok. 9 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.03s | ||
| test result: ok. 28 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.12s | ||
| test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s | ||
| test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s | ||
| test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s | ||
| ``` | ||
|
|
||
| ## Full SDK suite | ||
|
|
||
| From packages/sdk: | ||
| ``` | ||
| PATH=/Users/khaliqgant/.bun/bin:$PATH RELAYFLOWD_BIN=/Users/khaliqgant/.relayflows-toolchain/target/2930744996/debug/relayflowd ./node_modules/.bin/vitest run | ||
| ``` | ||
| Exit 1: the real Claude analyzer readiness probe is unavailable. This is an | ||
| unpassed acceptance test, not a green full suite. Existing opt-in adapter tests | ||
| remain skipped; no skip flags were enabled. Captured final output excerpt | ||
| (full local log: /private/tmp/spec-J-sdk-final.log): | ||
| ``` | ||
| ✓ built flows CLI against live relayflowd > preflights before journaling and names an unreachable socket 1972ms | ||
| ✓ built flows CLI against live relayflowd > starts exactly one daemon when two runs race for one empty data dir 967ms | ||
| ✓ surface resume after a real daemon kill > resumes a three-step run with each successful completion exactly once 1845ms | ||
|
|
||
| ⎯⎯⎯⎯⎯⎯⎯ Failed Tests 1 ⎯⎯⎯⎯⎯⎯⎯ | ||
|
|
||
| FAIL tests/live-kernel.test.ts > built flows CLI against live relayflowd > hn-monitor analyze-story reaches done through the real Claude analyzer CLI | ||
| Error: LIVE_ANALYZER_UNAVAILABLE: "/Users/khaliqgant/flows-spec-J-plugin/testdata/preflight/analyze-story-claude-cli auth status" exited 1: analyze-story-claude-cli: "claude -p --model claude-haiku-4-5-20251001" exited 1: — failing because gate-2 acceptance requires the real analyzer to execute. Set RELAYFLOWS_ALLOW_ANALYZER_SKIP=1 only if this run is not gate evidence. | ||
| ❯ tests/live-kernel.test.ts:1223:15 | ||
| 1221| const notice = `LIVE_ANALYZER_UNAVAILABLE: ${readiness.detail}`; | ||
| 1222| if (process.env['RELAYFLOWS_ALLOW_ANALYZER_SKIP'] !== '1') { | ||
| 1223| throw new Error( | ||
| | ^ | ||
| 1224| `${notice} — failing because gate-2 acceptance requires the … | ||
| 1225| + 'Set RELAYFLOWS_ALLOW_ANALYZER_SKIP=1 only if this run is … | ||
|
|
||
| ⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯[1/1]⎯ | ||
|
|
||
| Test Files 1 failed | 69 passed | 1 skipped (71) | ||
| Tests 1 failed | 1220 passed | 3 skipped (1224) | ||
| Start at 19:11:02 | ||
| Duration 57.03s (transform 1.25s, setup 0ms, collect 8.68s, tests 214.59s, environment 6ms, prepare 1.97s) | ||
|
|
||
| ``` | ||
|
|
||
| ## CLI smoke | ||
|
|
||
| Scratch project used a fixture-only npm shim that forwards to real npm's offline | ||
| local-package install, rather than contacting the public npm registry. | ||
| ``` | ||
| Command: node packages/sdk/dist/cli.js add helper-datadog (scratch project, npm stand-in installs local offline fixture) | ||
| Added @flows/helper-datadog | ||
| exit=0 | ||
| { | ||
| "plugins": [ | ||
| "@flows/helper-datadog" | ||
| ] | ||
| } | ||
|
|
||
| ``` |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,152 @@ | ||
| import { createHash, randomUUID } from 'node:crypto'; | ||
| import { compileSpec, toKernelSpec } from './compile.js'; | ||
| import { invokePlugin, type LoadedPlugin } from './plugin-loader.js'; | ||
| import type { PluginVerb } from './plugin-manifest.js'; | ||
| import type { AuthoredBudget } from './authored-budget.js'; | ||
| import { SPEC_SCHEMA_VERSION } from './spec.js'; | ||
| import type { JournalClient } from './journal-client.js'; | ||
| import type { StepDispatchEvent } from './protocol.js'; | ||
| import { AuthoredFlowExecutionError } from './authored-flow-error.js'; | ||
| import type { AuthoredFlowJournalStep } from './authored-flow-executor.js'; | ||
| import { readCompletedStepOutput } from './authored-step-output.js'; | ||
| import { withWorkerLease } from './worker-lease.js'; | ||
|
|
||
| export class PluginStepError extends AuthoredFlowExecutionError { | ||
| constructor(readonly diagnostic: string, runId: string) { | ||
| super('step_failed', diagnostic, 'worker_error', runId); | ||
| } | ||
| } | ||
|
|
||
| /** Helpers lower to an existing agent effect, never a fourth kernel primitive. | ||
| * The Plugin receipt lives in step.completed.output: { type, input, output, ... }. | ||
| * A private stream routes the step to this short-lived SDK worker exclusively. | ||
| */ | ||
| export async function runPluginEffect( | ||
| journal: JournalClient, flowName: string, id: string, plugin: LoadedPlugin, verb: PluginVerb, | ||
| args: unknown, journalSteps: AuthoredFlowJournalStep[], budget: AuthoredBudget, | ||
| ): Promise<unknown> { | ||
| const server = verb.namespace; | ||
| const tool = verb.method; | ||
| const idempotencyKey = `plugin:${server}:${tool}:${createHash('sha256').update(JSON.stringify(args)).digest('hex')}`; | ||
| const surfacePath = `/plugins/${pathPart(server)}/${pathPart(tool)}`; | ||
| const stream = `plugin-worker-${randomUUID()}`; | ||
| const instruction = JSON.stringify({ type: 'effect', server, tool, input: args }); | ||
| const peer = journal.createPeer(); | ||
| let diagnostic: string | undefined; | ||
| let settled!: () => void; | ||
| let failed!: (error: unknown) => void; | ||
| const completed = new Promise<void>((resolve, reject) => { settled = resolve; failed = reject; }); | ||
| void completed.catch(() => undefined); | ||
| let timer: ReturnType<typeof setTimeout> | undefined; | ||
| let work: Promise<void> | undefined; | ||
| let dispatchExpired = false; | ||
| const dispatch = (event: StepDispatchEvent): void => { | ||
| if (dispatchExpired) return; | ||
| const dispatched = event.spec as { instruction?: string; surfaces?: { streams?: { stream: string }[] } }; | ||
| if (event.step_id !== id || event.step_type !== 'agent' || work !== undefined | ||
| || dispatched?.instruction !== instruction | ||
| || !dispatched.surfaces?.streams?.some(pin => pin.stream === stream)) { | ||
| failed(new Error('Plugin worker received an unexpected dispatch')); | ||
| return; | ||
| } | ||
| clearTimeout(timer); | ||
| timer = undefined; | ||
| work = execute(event); | ||
| void work.then(settled, failed); | ||
| }; | ||
| async function execute(event: StepDispatchEvent): Promise<void> { | ||
| const receipt = { type: 'effect' as const, plugin: plugin.manifest.name, version: plugin.manifest.version, | ||
| namespace: server, method: tool, input: args, idempotencyKey: event.idempotency_key }; | ||
| let output: unknown; | ||
| let confirmed = false; | ||
| try { | ||
| // Renew the worker lease while the Plugin tool is in flight. Without this, | ||
| // a tool that runs longer than the initial lease loses ownership and | ||
| // the write-back path fails — see worker-lease.ts for the renewal contract. | ||
| confirmed = await withWorkerLease(peer, event, async () => { | ||
| return peer.performEffect({ | ||
| runId: event.run_id, stepId: id, attempt: event.attempt, idempotencyKey: event.idempotency_key, | ||
| surfacePath, revisionBefore: 'pending', revisionAfter: idempotencyKey, | ||
| }, async () => { | ||
| try { output = await invokePlugin(plugin, verb, args, event.idempotency_key); } | ||
| catch { diagnostic = 'plugin_execution_failed'; throw new PluginExecutionFailure(); } | ||
| }); | ||
| }); | ||
| // A confirmed election without its receipt is an interrupted writeback, | ||
| // not a successful result we may invent or a call we may safely repeat. | ||
| if (!confirmed) throw new Error('plugin_result_unavailable'); | ||
| } catch (error) { | ||
| if (error instanceof PluginExecutionFailure) diagnostic = 'plugin_execution_failed'; | ||
| else if (error instanceof Error && error.message === 'plugin_result_unavailable') diagnostic = error.message; | ||
| else throw error; // Journal failures remain fail-closed, never provider errors. | ||
| } | ||
| await peer.stepComplete(event.run_id, id, event.attempt, event.idempotency_key, | ||
| diagnostic === undefined ? 'success' : 'worker_error', { | ||
| output: { ...receipt, output: output ?? null, ...(diagnostic ? { diagnostic } : {}) }, | ||
| started_pins: event.pins, end_pins: event.pins, | ||
| ...(diagnostic ? { trajectory_tail: { ...receipt, diagnostic } } : {}), | ||
| effects: confirmed ? [{ surface_path: surfacePath, idempotency_key: event.idempotency_key }] : [], | ||
| }); | ||
| } | ||
| let childRunId: string | undefined; | ||
| async function cancelChildRun(): Promise<void> { | ||
| const runId = childRunId; | ||
| childRunId = undefined; | ||
| if (runId !== undefined) { | ||
| try { await journal.runCancel(runId); } catch { /* fail-open on cleanup */ } | ||
| } | ||
| } | ||
| const deadlineError = new Error('Plugin worker dispatch deadline exceeded'); | ||
| let finishDeadline!: () => void; | ||
| const deadline = new Promise<void>((resolve, reject) => { | ||
| finishDeadline = resolve; | ||
| // One deadline covers connection, initialization, run creation and dispatch. | ||
| timer = setTimeout(() => { | ||
| dispatchExpired = true; | ||
| failed(deadlineError); | ||
| reject(deadlineError); | ||
| }, 30_000); | ||
| }); | ||
| try { | ||
| await Promise.race([peer.connect(), deadline]); | ||
| await Promise.race([peer.hello('flows-plugin'), deadline]); | ||
| peer.on('step.dispatch', dispatch); | ||
| await Promise.race([peer.workerAttach(stream, ['agent'], { workspace: [], streams: [{ stream, read_offset: 0 }] }, 1), deadline]); | ||
| const spec = toKernelSpec(compileSpec({ version: SPEC_SCHEMA_VERSION, name: `${flowName}/${id}`, | ||
| steps: [{ id, type: 'agent', instruction, | ||
| surfaces: { streams: [{ stream }], external: [surfacePath] }, maxIterations: 1 }], | ||
| })); | ||
| return await Promise.race([budget.execute(journal, spec, async outcome => { | ||
| childRunId = outcome.run_id; | ||
| // run.start may answer after the deadline and the outer cleanup. | ||
| if (dispatchExpired) await cancelChildRun(); | ||
| await completed; | ||
| if (diagnostic !== undefined) { | ||
| try { await readCompletedStepOutput(journal, outcome.run_id, id, journalSteps); } | ||
| catch (error) { | ||
| if (!(error instanceof AuthoredFlowExecutionError) || error.code !== 'step_failed') throw error; | ||
| throw new PluginStepError(diagnostic, outcome.run_id); | ||
| } | ||
| } | ||
| const receipt = await readCompletedStepOutput(journal, outcome.run_id, id, journalSteps) as { output: unknown }; | ||
| return receipt.output; | ||
| }), deadline]); | ||
|
There was a problem hiding this comment. Deadline race abandons plugin executionHigh Severity The 30s dispatch timer starts at the beginning of   Reviewed by Cursor Bugbot for commit 512e31e. Configure here. |
||
| } finally { | ||
| clearTimeout(timer); | ||
| finishDeadline(); | ||
| peer.off('step.dispatch', dispatch); | ||
| // If the dispatch deadline fired we started a child run that will never | ||
| // be worked -- cancel it so its journal doesn't stay indeterminate. | ||
| // Best-effort: the parent flow already surfaced the deadline via the | ||
| // rejected `completed` promise, so a cancel error here must not mask it. | ||
| if (dispatchExpired) await cancelChildRun(); | ||
| peer.close(); | ||
| await work?.catch(() => undefined); | ||
| } | ||
| } | ||
|
|
||
| function pathPart(value: string): string { | ||
| return encodeURIComponent(value).replaceAll('.', '%2E'); | ||
| } | ||
|
|
||
| class PluginExecutionFailure extends Error {} | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.