From 432ef5136fcd06c797952f2b0dc204e0a5cf41e7 Mon Sep 17 00:00:00 2001 From: Khaliq Date: Wed, 2 Sep 2026 22:24:00 +0200 Subject: [PATCH 1/4] fix(core): restore the observer link with a scoped observer token MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit A workflow run stopped printing any way to watch itself. #27 removed the observer link because it carried `rk_live_` — an administrative credential (send, spawn, administer) in a URL, where query strings land in browser history, referrer headers, and proxy logs. That removal was right, and the engine rejects a workspace key on the realtime endpoint anyway, so the link had also stopped working. But the replacement it named — "requires a separately provisioned, read-only observer token" — was never built. Nothing in relayflows provisioned one, so runs became unwatchable: `ensureRelaycastApiKey` creates an anonymous workspace per run with no auth, holds the key in memory, and never persists or prints it. Once the run exits, nobody can reach that workspace again. Mint the token the old message pointed at. After resolving the workspace key, mint a scoped `ot_live_` token and print the link built from it: Workspace created for this workflow. Observer: https://agentrelay.com/observer?key=ot_live_... Channel: wf-ship-feature-a1b2c3 An observer token is the credential built for this job — read-only, expiring in 24h, individually revocable, and scopable to channels. Filters differ by who owns the workspace. An auto-created one exists only for this run, so the link covers all of it, DMs included. A user-supplied one may carry unrelated traffic, so the link is scoped to this run's channel with DMs excluded. A bring-your-own-key run previously printed nothing at all. Minting is best-effort throughout: network error, non-2xx, malformed body, or a 10s timeout all yield no link rather than a failed run. `buildObserverUrl` refuses anything but an `ot_live_` token, and the dashboard base must be http(s), so a bad `RELAY_OBSERVER_URL` cannot turn the link into a token exfiltration vector. Also fixes a filter bug this uncovered: `cli.ts` and `listr-renderer.ts` whitelisted only `Observer:` / `agentrelay.com` / `Channel: wf-`, so the `Observation:` and `Workspace created` lines were silently swallowed while listr owned the terminal — the existing fallback guidance never reached anyone. Both filters now share `isObserverGuidanceLine()`, with tests asserting every guidance line survives filtering. Verified against the live engine: both the firehose and channel-scoped mints return `ot_live_` tokens with a 24h expiry and produce a working link. Co-Authored-By: Claude Opus 5 (1M context) Claude-Session: https://claude.ai/code/session_01Rg77imZpwSho5Ngdp7wtRD --- README.md | 98 +++++++++ docs/observer.mdx | 125 +++++++++++ .../src/__tests__/channel-messenger.test.ts | 51 ++++- .../core/src/__tests__/observer-token.test.ts | 157 ++++++++++++++ packages/core/src/channel-messenger.ts | 62 +++++- packages/core/src/cli.ts | 5 +- packages/core/src/index.ts | 1 + packages/core/src/listr-renderer.ts | 7 +- packages/core/src/observer-token.ts | 205 ++++++++++++++++++ packages/core/src/runner.ts | 69 +++++- 10 files changed, 761 insertions(+), 19 deletions(-) create mode 100644 docs/observer.mdx create mode 100644 packages/core/src/__tests__/observer-token.test.ts create mode 100644 packages/core/src/observer-token.ts diff --git a/README.md b/README.md index 13265ba..f1f270f 100644 --- a/README.md +++ b/README.md @@ -70,6 +70,104 @@ result = ( ) ``` +## Watching a Run + +When a run starts, the runner prints a link you can open to follow it live: + +```text +[workflow 00:02] Workspace created for this workflow. +[workflow 00:02] Observer: https://agentrelay.com/observer?key=ot_live_... +[workflow 00:02] Channel: wf-ship-feature-a1b2c3 +``` + +Open the `Observer:` URL and you see messages, agent activity, handoffs, and +deliveries in real time. The link is safe to paste to a teammate: it carries a +**scoped observer token** (`ot_live_`), which is read-only, expires in 24 hours, +and can be revoked individually. + +### The two ways a run gets a workspace + +Which link you get depends on where the run's Relaycast workspace comes from. + +**No `RELAY_API_KEY` set** — the runner creates a throwaway workspace for this +run alone and mints an observer link covering all of it, DMs included. The +workspace is anonymous and disappears from your reach when the run ends, so +**copy the link while the run is going**. Nothing persists the underlying key, +and there is no way to recover it afterward. + +**`RELAY_API_KEY` set** — the runner uses your workspace and mints a link scoped +to just this run's channel, with agent DMs excluded, so the link does not expose +unrelated traffic in a shared workspace. This is the better setup for anything +you may want to revisit: the run is in a workspace you own, so you can mint +fresh links whenever you like. + +```bash +# One-time: create a workspace you own and keep the key +curl -sX POST https://api.relaycast.dev/v1/workspaces \ + -H 'content-type: application/json' -d '{"name":"my-workflows"}' \ + | jq -r '.data.api_key // .api_key' + +export RELAY_API_KEY=rk_live_... # put this in your shell profile +relayflows run workflow.yaml +``` + +If you already use the Agent Relay CLI, `agent-relay workspace key --reveal-secrets` +prints the key of your active workspace. Note it is **masked** without +`--reveal-secrets`. + +### Minting more links yourself + +With a workspace you own, `agent-relay observer` mints links on demand: + +```bash +agent-relay observer # read-only link, 24h, DMs excluded +agent-relay observer --channels wf-ship-a1b2c3 # scope to one run +agent-relay observer --include-dms # include agent DMs +agent-relay observer --expires 7d # longer-lived link +agent-relay observer list # what is outstanding +agent-relay observer revoke # cut one off immediately +``` + +### Never share the workspace key + +A workspace key (`rk_live_`) is an **administrative** credential — it can send +messages, spawn and remove agents, and change workspace settings. Do not put one +in an observer URL, a chat message, or a terminal transcript; query strings end +up in browser history, referrer headers, and proxy logs. The realtime endpoint +rejects it outright, so a link built from one cannot stream anyway. + +| | Workspace key (`rk_live_`) | Observer token (`ot_live_`) | +| --- | --- | --- | +| Read messages and activity | yes | yes | +| Send, spawn agents, administer | yes | **no** | +| Expires | no | yes | +| Revocable individually | no | yes | +| Scopable to channels | no | yes | + +The runner only ever prints `ot_live_` links, and scrubs `rk_live_` values out of +channel output. + +### Configuration + +| Variable | Purpose | +| --- | --- | +| `RELAY_API_KEY` | Workspace key to run against. Unset means a throwaway workspace per run. | +| `RELAY_OBSERVER_URL` | Observer dashboard base. Defaults to `https://agentrelay.com/observer`. | +| `RELAY_OBSERVER_EXPIRES` | Link lifetime as `30m` / `24h` / `7d`. Defaults to `24h`; unparseable or over `90d` falls back to `24h`. | +| `RELAYCAST_BASE_URL` | Relaycast engine base. Defaults to `https://api.relaycast.dev`. | + +### If no link appears + +- **`Observation: unavailable`** — the token could not be minted (engine + unreachable, or it rejected the request). The run is unaffected; minting is + best-effort by design and never fails a run. +- **No observer lines at all** — the run needed no broker: every step was + `deterministic`, `worktree`, `integration`, or `waitFor`, or an external + executor handled agent spawning, or Relaycast was disabled with + `AGENT_RELAY_WORKFLOW_DISABLE_RELAYCAST=1`. +- **`Observation: run agent-relay observer`** — you set `RELAY_API_KEY` and + minting failed. Mint a link by hand with `agent-relay observer`. + ## Consumer-Facing Apps + AI SDK Communicate Flows A good production split is: diff --git a/docs/observer.mdx b/docs/observer.mdx new file mode 100644 index 0000000..51d04be --- /dev/null +++ b/docs/observer.mdx @@ -0,0 +1,125 @@ +--- +title: 'Watch a run' +description: 'Follow a running workflow live at agentrelay.com with a read-only observer link, and understand where that link comes from.' +--- + +Every workflow run that uses a broker prints a link you can open to follow it in +real time — messages, agent activity, handoffs, and deliveries as they happen. + +```text +[workflow 00:02] Workspace created for this workflow. +[workflow 00:02] Observer: https://agentrelay.com/observer?key=ot_live_... +[workflow 00:02] Channel: wf-ship-feature-a1b2c3 +``` + +Open the `Observer:` URL, or paste it to a teammate. It carries a scoped +**observer token** (`ot_live_`): read-only, expiring in 24 hours by default, and +individually revocable. + +## Where the link comes from + +The runner needs a Relaycast workspace to coordinate agents. Where that +workspace comes from decides what your link covers. + +### Without `RELAY_API_KEY` + +The runner creates a throwaway workspace for this run alone, then mints a link +covering all of it, agent DMs included. + +That workspace is anonymous — it is not attached to your account, and nothing +persists its key. **Copy the link while the run is going.** Once the run ends +there is no way to recover access. + +### With `RELAY_API_KEY` + +The runner uses your workspace and mints a link scoped to just this run's +channel, with agent DMs excluded, so the link does not expose unrelated traffic +in a workspace you share with others. + +This is the better setup for anything you may want to revisit — the run lives in +a workspace you own, so you can mint fresh links at any time. + +```bash +# One-time: create a workspace and keep its key +curl -sX POST https://api.relaycast.dev/v1/workspaces \ + -H 'content-type: application/json' -d '{"name":"my-workflows"}' \ + | jq -r '.data.api_key // .api_key' + +export RELAY_API_KEY=rk_live_... +relayflows run workflow.yaml +``` + +If you already use the Agent Relay CLI, `agent-relay workspace key --reveal-secrets` +prints your active workspace key. Without `--reveal-secrets` it is masked. + +## Minting links yourself + +With a workspace you own: + +```bash +agent-relay observer # read-only link, 24h, DMs excluded +agent-relay observer --channels wf-ship-a1b2c3 # scope to one run's channel +agent-relay observer --include-dms # include agent DMs +agent-relay observer --expires 7d # longer-lived link +agent-relay observer list # what is outstanding +agent-relay observer revoke # cut one off immediately +``` + +## Never share a workspace key + +A workspace key (`rk_live_`) is an **administrative** credential: it can send +messages, spawn and remove agents, and change workspace settings. Do not put one +in an observer URL, a chat message, or a terminal transcript — query strings end +up in browser history, referrer headers, and proxy logs. + +| | Workspace key (`rk_live_`) | Observer token (`ot_live_`) | +| --- | --- | --- | +| Read messages and activity | yes | yes | +| Send, spawn agents, administer | yes | **no** | +| Expires | no | yes | +| Revocable individually | no | yes | +| Scopable to channels | no | yes | + +The realtime endpoint enforces this: it rejects a workspace key and accepts only +an observer token carrying `stream:read`. The runner prints `ot_live_` links +only, and scrubs `rk_live_` values out of channel output. + +## Configuration + +| Variable | Purpose | +| --- | --- | +| `RELAY_API_KEY` | Workspace key to run against. Unset means a throwaway workspace per run. | +| `RELAY_OBSERVER_URL` | Observer dashboard base. Defaults to `https://agentrelay.com/observer`. | +| `RELAY_OBSERVER_EXPIRES` | Link lifetime as `30m` / `24h` / `7d`. Defaults to `24h`; unparseable or over `90d` falls back to `24h`. | +| `RELAYCAST_BASE_URL` | Relaycast engine base. Defaults to `https://api.relaycast.dev`. | + +## If no link appears + + + + The token could not be minted — the engine was unreachable or rejected the + request. The run itself is unaffected: minting is best-effort by design and + never fails a run. + + + The run needed no broker: every step was `deterministic`, `worktree`, + `integration`, or `waitFor`, or an external executor handled agent + spawning, or Relaycast was disabled with + `AGENT_RELAY_WORKFLOW_DISABLE_RELAYCAST=1`. + + + You set `RELAY_API_KEY` and minting failed. Mint a link by hand with that + command. + + + +## See also + + + + Execute local workflow files and resume failed runs. + + + The observer dashboard itself. + + diff --git a/packages/core/src/__tests__/channel-messenger.test.ts b/packages/core/src/__tests__/channel-messenger.test.ts index 8d9f3ab..6f99302 100644 --- a/packages/core/src/__tests__/channel-messenger.test.ts +++ b/packages/core/src/__tests__/channel-messenger.test.ts @@ -5,6 +5,7 @@ import { ChannelMessenger, formatError, formatObserverGuidance, + isObserverGuidanceLine, formatStepOutput, scrubSecrets, sendToChannel, @@ -69,15 +70,57 @@ describe('channel messenger helpers', () => { expect(scrubSecrets(text)).toBe(text); }); - it('omits credential-bearing observer links from auto-created workspace guidance', () => { - const guidance = formatObserverGuidance('workflow-room'); + it('prints the minted observer link for an auto-created workspace', () => { + const guidance = formatObserverGuidance('workflow-room', { + workspaceCreated: true, + observerUrl: 'https://agentrelay.com/observer?key=ot_live_abc123', + }); expect(guidance).toEqual([ 'Workspace created for this workflow.', - ' Observation: requires a separately provisioned, read-only observer token', + ' Observer: https://agentrelay.com/observer?key=ot_live_abc123', + ' Channel: workflow-room', + ]); + }); + + it('never puts a workspace key in the guidance when minting failed', () => { + const guidance = formatObserverGuidance('workflow-room', { workspaceCreated: true }); + + expect(guidance).toEqual([ + 'Workspace created for this workflow.', + ' Observation: unavailable — could not mint a read-only observer token ' + + '(set RELAY_API_KEY to run against a workspace you own)', + ' Channel: workflow-room', + ]); + expect(guidance.join('\n')).not.toMatch(/rk_live_|observer\?key=/); + }); + + it.each([ + [{ workspaceCreated: true, observerUrl: 'https://agentrelay.com/observer?key=ot_live_a' }], + [{ workspaceCreated: true, observerUrl: 'http://localhost:4000/observer?key=ot_live_a' }], + [{ workspaceCreated: true }], + [{}], + ])('survives terminal output filtering for %j', (options) => { + // Every guidance line must clear the filter — a link the user cannot see is + // the bug this whole path exists to fix. + for (const line of formatObserverGuidance('wf-demo-ab12', options)) { + expect(isObserverGuidanceLine(`[workflow 00:03] ${line}`)).toBe(true); + } + }); + + it('does not whitelist ordinary workflow chatter', () => { + expect(isObserverGuidanceLine('[workflow 00:03] Resolving Relaycast API key...')).toBe(false); + expect(isObserverGuidanceLine('[broker] worker started')).toBe(false); + }); + + it('points a bring-your-own-key run at the observer command when minting failed', () => { + const guidance = formatObserverGuidance('workflow-room'); + + expect(guidance).toEqual([ + ' Observation: run `agent-relay observer` to mint a read-only link', ' Channel: workflow-room', ]); - expect(guidance.join('\n')).not.toMatch(/observer\?key=|\[REDACTED\]/); + expect(guidance.join('\n')).not.toContain('Workspace created'); }); it('formatError normalizes unknown errors', () => { diff --git a/packages/core/src/__tests__/observer-token.test.ts b/packages/core/src/__tests__/observer-token.test.ts new file mode 100644 index 0000000..954ecb0 --- /dev/null +++ b/packages/core/src/__tests__/observer-token.test.ts @@ -0,0 +1,157 @@ +import { describe, expect, it } from 'vitest'; + +import { + DEFAULT_OBSERVER_TTL_MS, + DEFAULT_OBSERVER_URL, + buildObserverUrl, + mintObserverToken, + parseObserverDuration, + resolveObserverBaseUrl, + resolveObserverTtlMs, +} from '../observer-token.js'; + +const OK_BODY = { + data: { id: 'ot_1', token: 'ot_live_abc123', expires_at: '2026-09-03T00:00:00.000Z' }, +}; + +function okFetch(capture?: { url?: string; init?: RequestInit }): typeof fetch { + return (async (url: string, init: RequestInit) => { + if (capture) { + capture.url = url; + capture.init = init; + } + return { ok: true, json: async () => OK_BODY } as Response; + }) as unknown as typeof fetch; +} + +describe('resolveObserverBaseUrl', () => { + it('prefers an explicit URL, then RELAY_OBSERVER_URL, then the hosted default', () => { + const env = { RELAY_OBSERVER_URL: 'https://observer.relaycast.dev' } as NodeJS.ProcessEnv; + expect(resolveObserverBaseUrl('https://staging.example/observer', env)).toBe( + 'https://staging.example/observer' + ); + expect(resolveObserverBaseUrl(undefined, env)).toBe('https://observer.relaycast.dev'); + expect(resolveObserverBaseUrl(undefined, {} as NodeJS.ProcessEnv)).toBe(DEFAULT_OBSERVER_URL); + }); + + it.each(['javascript:alert(1)', 'data:text/html,x', 'not a url'])( + 'rejects %s rather than building a link that leaks the token', + (value) => { + expect(() => resolveObserverBaseUrl(value, {} as NodeJS.ProcessEnv)).toThrow(); + } + ); +}); + +describe('buildObserverUrl', () => { + it('appends the observer token as the key parameter', () => { + expect(buildObserverUrl(DEFAULT_OBSERVER_URL, 'ot_live_abc123')).toBe( + 'https://agentrelay.com/observer?key=ot_live_abc123' + ); + }); + + it.each(['rk_live_secret', 'at_live_secret', 'plain'])( + 'refuses to put %s in a shareable URL', + (token) => { + expect(() => buildObserverUrl(DEFAULT_OBSERVER_URL, token)).toThrow(/scoped observer token/); + } + ); +}); + +describe('observer token lifetime', () => { + it.each([ + ['30m', 1_800_000], + ['24h', 86_400_000], + ['7d', 604_800_000], + ])('parses %s', (value, expected) => { + expect(parseObserverDuration(value)).toBe(expected); + }); + + it.each(['24', '0h', '-1d', '', 'soon', '91d'])('rejects %s', (value) => { + expect(parseObserverDuration(value)).toBeNull(); + }); + + it('falls back to the default rather than throwing on a typo', () => { + expect(resolveObserverTtlMs({ RELAY_OBSERVER_EXPIRES: 'oops' } as NodeJS.ProcessEnv)).toBe( + DEFAULT_OBSERVER_TTL_MS + ); + expect(resolveObserverTtlMs({} as NodeJS.ProcessEnv)).toBe(DEFAULT_OBSERVER_TTL_MS); + expect(resolveObserverTtlMs({ RELAY_OBSERVER_EXPIRES: '7d' } as NodeJS.ProcessEnv)).toBe( + 604_800_000 + ); + }); +}); + +describe('mintObserverToken', () => { + it('requests every read scope with the workspace key and an explicit expiry', async () => { + const capture: { url?: string; init?: RequestInit } = {}; + const minted = await mintObserverToken({ + baseUrl: 'https://api.relaycast.dev', + workspaceKey: 'rk_live_secret', + name: 'relayflows-wf-demo', + filters: { include_dms: false, channel_names: ['wf-demo'] }, + now: () => 0, + ttlMs: 1000, + fetchImpl: okFetch(capture), + }); + + expect(minted).toEqual({ + token: 'ot_live_abc123', + id: 'ot_1', + expiresAt: '2026-09-03T00:00:00.000Z', + }); + expect(capture.url).toBe('https://api.relaycast.dev/v1/observer-tokens'); + + const headers = capture.init?.headers as Record; + expect(headers.authorization).toBe('Bearer rk_live_secret'); + + const body = JSON.parse(String(capture.init?.body)); + expect(body.scopes).toContain('stream:read'); + expect(body.filters).toEqual({ include_dms: false, channel_names: ['wf-demo'] }); + expect(body.expires_at).toBe('1970-01-01T00:00:01.000Z'); + }); + + it('records include_dms explicitly rather than as an absence', async () => { + const capture: { url?: string; init?: RequestInit } = {}; + await mintObserverToken({ + baseUrl: 'https://api.relaycast.dev', + workspaceKey: 'rk_live_secret', + name: 'relayflows-wf-demo', + filters: { include_dms: true }, + fetchImpl: okFetch(capture), + }); + + expect(JSON.parse(String(capture.init?.body)).filters).toEqual({ include_dms: true }); + }); + + it.each([ + ['a non-2xx response', { ok: false, json: async () => ({}) }], + ['a body with no token', { ok: true, json: async () => ({ data: { id: 'ot_1' } }) }], + [ + 'a workspace key returned in the token field', + { ok: true, json: async () => ({ data: { id: 'ot_1', token: 'rk_live_nope' } }) }, + ], + ['a body with no id', { ok: true, json: async () => ({ data: { token: 'ot_live_abc' } }) }], + ])('returns null on %s', async (_label, response) => { + const minted = await mintObserverToken({ + baseUrl: 'https://api.relaycast.dev', + workspaceKey: 'rk_live_secret', + name: 'relayflows-wf-demo', + fetchImpl: (async () => response as Response) as unknown as typeof fetch, + }); + + expect(minted).toBeNull(); + }); + + it('returns null instead of throwing when the engine is unreachable', async () => { + const minted = await mintObserverToken({ + baseUrl: 'https://api.relaycast.dev', + workspaceKey: 'rk_live_secret', + name: 'relayflows-wf-demo', + fetchImpl: (async () => { + throw new Error('ECONNREFUSED'); + }) as unknown as typeof fetch, + }); + + expect(minted).toBeNull(); + }); +}); diff --git a/packages/core/src/channel-messenger.ts b/packages/core/src/channel-messenger.ts index cfc0cae..b3855f1 100644 --- a/packages/core/src/channel-messenger.ts +++ b/packages/core/src/channel-messenger.ts @@ -109,12 +109,62 @@ export function scrubSecrets(text: string): string { return result; } -export function formatObserverGuidance(channel: string): string[] { - return [ - 'Workspace created for this workflow.', - ' Observation: requires a separately provisioned, read-only observer token', - ` Channel: ${channel}`, - ]; +export interface ObserverGuidanceOptions { + /** Set when the runner auto-created a throwaway workspace for this run. */ + workspaceCreated?: boolean; + /** Read-only observer link (`ot_live_`), when one could be minted. */ + observerUrl?: string; +} + +/** + * The lines the runner prints so a human can follow a run. + * + * The link, when present, always carries a scoped `ot_live_` observer token — + * never the workspace key. See `observer-token.ts` for why. + * + * When minting fails the guidance depends on who owns the workspace: a + * user-supplied one can be re-minted against by hand, but an auto-created one + * holds a key the user never sees, so pointing them at `agent-relay observer` + * would send them after a credential they cannot obtain. + */ +export function formatObserverGuidance( + channel: string, + options: ObserverGuidanceOptions = {} +): string[] { + const lines: string[] = []; + if (options.workspaceCreated) { + lines.push('Workspace created for this workflow.'); + } + if (options.observerUrl) { + lines.push(` Observer: ${options.observerUrl}`); + } else if (options.workspaceCreated) { + lines.push( + ' Observation: unavailable — could not mint a read-only observer token ' + + '(set RELAY_API_KEY to run against a workspace you own)' + ); + } else { + lines.push(' Observation: run `agent-relay observer` to mint a read-only link'); + } + lines.push(` Channel: ${channel}`); + return lines; +} + +/** + * Whether a line is part of the observer guidance block. + * + * The terminal output filters drop `[workflow HH:MM]` chatter while listr owns + * the screen. The observer link is the one thing a user needs off that stream — + * swallowing it would defeat the point of printing it — so both filters ask + * here rather than each keeping its own copy of the substrings to spare. + */ +export function isObserverGuidanceLine(line: string): boolean { + return ( + line.includes('Observer:') || + line.includes('Observation:') || + line.includes('Workspace created') || + line.includes('agentrelay.com') || + line.includes('Channel: wf-') + ); } function stripMalformedPtyFrameGarbage(line: string): string { diff --git a/packages/core/src/cli.ts b/packages/core/src/cli.ts index 044a534..7a4ac41 100644 --- a/packages/core/src/cli.ts +++ b/packages/core/src/cli.ts @@ -15,6 +15,7 @@ import chalk from 'chalk'; import type { WorkflowEvent } from './runner.js'; import { WorkflowRunner } from './runner.js'; import { JsonFileWorkflowDb } from './file-db.js'; +import { isObserverGuidanceLine } from './channel-messenger.js'; function printUsage(): void { console.log( @@ -80,12 +81,12 @@ interface StepHandle { } // Filter [broker] and [workflow HH:MM] noise while listr owns the terminal, -// but let the observer URL and channel name through. +// but let the observer guidance block (link, channel) through. function installOutputFilter(): () => void { const orig = console.log.bind(console); console.log = (...args: unknown[]) => { const str = String(args[0] ?? ''); - if (str.includes('Observer:') || str.includes('agentrelay.com') || str.includes('Channel: wf-')) { + if (isObserverGuidanceLine(str)) { orig(...args); return; } diff --git a/packages/core/src/index.ts b/packages/core/src/index.ts index 2269a0b..9d100d0 100644 --- a/packages/core/src/index.ts +++ b/packages/core/src/index.ts @@ -3,6 +3,7 @@ export * from './runner.js'; export * from './custom-steps.js'; export * from './cli-session-collector.js'; export * from './channel-messenger.js'; +export * from './observer-token.js'; export * from './process-spawner.js'; export { createProcessBackendExecutor, diff --git a/packages/core/src/listr-renderer.ts b/packages/core/src/listr-renderer.ts index 6a44d10..0b0175e 100644 --- a/packages/core/src/listr-renderer.ts +++ b/packages/core/src/listr-renderer.ts @@ -1,16 +1,17 @@ import chalk from 'chalk'; import type { ListrTask } from 'listr2'; import type { WorkflowEvent, WorkflowEventListener } from './runner.js'; +import { isObserverGuidanceLine } from './channel-messenger.js'; // Filter console.log while listr owns the terminal. // Blocks [broker] noise and [workflow HH:MM] timing lines, but lets the -// observer URL and channel name through so users can track the run. +// observer guidance block through so users can track the run. function installOutputFilter(): () => void { const orig = console.log.bind(console); console.log = (...args: unknown[]) => { const str = String(args[0] ?? ''); - // Always show the observer URL and channel so users can follow the run - if (str.includes('Observer:') || str.includes('agentrelay.com') || str.includes('Channel: wf-')) { + // Always show the observer guidance so users can follow the run + if (isObserverGuidanceLine(str)) { orig(...args); return; } diff --git a/packages/core/src/observer-token.ts b/packages/core/src/observer-token.ts new file mode 100644 index 0000000..44f2cfe --- /dev/null +++ b/packages/core/src/observer-token.ts @@ -0,0 +1,205 @@ +/** + * Read-only observer links for workflow runs. + * + * A workspace key (`rk_live_`) is an administrative credential: it can send + * messages, spawn and remove agents, and change workspace settings. It has no + * business in a URL — query strings land in browser history, referrer headers, + * and proxy logs — and the engine rejects it on the realtime endpoint outright, + * so a link built from one cannot stream in the first place. + * + * The supported credential for "let a human watch this run" is a scoped + * observer token (`ot_live_`): read-only, individually revocable, expiring, and + * narrowable to specific channels. This module mints one per run and builds the + * observer URL from it, so the runner can print a link that is safe to paste. + * + * Mirrors `agent-relay observer` (relay `packages/cli/src/cli/commands/observer.ts`) + * so both produce identical links. + */ + +/** Where the hosted observer dashboard lives. */ +export const DEFAULT_OBSERVER_URL = 'https://agentrelay.com/observer'; + +/** Default token lifetime. Long enough to outlive a run, short enough to expire. */ +export const DEFAULT_OBSERVER_TTL_MS = 24 * 60 * 60 * 1000; + +/** Cap on how long a run's observer link may be kept alive. */ +const MAX_OBSERVER_TTL_MS = 90 * 24 * 60 * 60 * 1000; + +/** Minting must never delay a run start; give up rather than hang on the engine. */ +const MINT_TIMEOUT_MS = 10_000; + +/** + * Every read scope, mirroring the engine's `OBSERVER_SCOPES`. `stream:read` is + * what makes the token usable on the realtime endpoint (`GET /v1/ws`) — without + * it the token can read REST but never streams, which is the whole point here. + */ +export const OBSERVER_SCOPES = [ + 'stream:read', + 'messages:read', + 'threads:read', + 'dms:read', + 'channels:read', + 'search:read', + 'agents:read', + 'nodes:read', + 'deliveries:read', + 'activity:read', + 'files:read', + 'reactions:read', +] as const; + +/** Visibility filters narrowing what the minted token can see (engine wire format). */ +export interface ObserverTokenFilters { + /** Restrict to these channel names. Omit for every channel in the workspace. */ + channel_names?: string[]; + /** Include agent DM traffic. Off unless explicitly requested. */ + include_dms?: boolean; +} + +export interface MintObserverTokenOptions { + /** Relaycast engine base URL. */ + baseUrl: string; + /** Workspace key (`rk_live_...`) — only a workspace key may mint observer tokens. */ + workspaceKey: string; + /** Token name. Must be unique within the workspace. */ + name: string; + description?: string; + filters?: ObserverTokenFilters; + /** Token lifetime. Defaults to {@link DEFAULT_OBSERVER_TTL_MS}. */ + ttlMs?: number; + /** Injected in tests. */ + now?: () => number; + fetchImpl?: typeof fetch; +} + +export interface MintedObserverToken { + /** Raw `ot_live_` material. Returned once, at creation, and never again. */ + token: string; + /** Observer-token id, for `agent-relay observer revoke `. */ + id: string; + /** ISO-8601 expiry. */ + expiresAt: string; +} + +/** + * Resolve the observer dashboard base URL: explicit value, then + * `RELAY_OBSERVER_URL` (for self-hosted or staging dashboards), then the hosted + * default. + * + * @throws if the resolved value is not an http(s) URL — the token is appended to + * this URL's query string, so the scheme decides where a live credential ends + * up, and `new URL` happily accepts `data:` and `javascript:`. + */ +export function resolveObserverBaseUrl(explicit?: string, env: NodeJS.ProcessEnv = process.env): string { + const value = explicit?.trim() || env.RELAY_OBSERVER_URL?.trim() || DEFAULT_OBSERVER_URL; + let parsed: URL; + try { + parsed = new URL(value); + } catch { + throw new Error(`Invalid observer URL: ${value}`); + } + if (parsed.protocol !== 'https:' && parsed.protocol !== 'http:') { + throw new Error(`Observer URL must be http or https: ${value}`); + } + return value; +} + +/** + * Build the observer URL for a minted token. + * + * @throws if handed anything but an `ot_live_` token — a workspace key in this + * position would put an administrative credential in a shareable URL, the exact + * failure this module exists to prevent. + */ +export function buildObserverUrl(baseUrl: string, token: string): string { + if (!token.startsWith('ot_live_')) { + throw new Error('Observer URLs require a scoped observer token (ot_live_...).'); + } + const url = new URL(baseUrl); + url.searchParams.set('key', token); + return url.toString(); +} + +/** + * Parse a `30m` / `24h` / `7d` duration into milliseconds. Bare digits are + * rejected rather than guessed at — `24` is far more likely to mean hours than + * milliseconds, and silently picking either would be wrong. + * + * @returns the duration in ms, or `null` if the value is unparseable or out of range + */ +export function parseObserverDuration(value: string): number | null { + const match = /^(\d+)([mhd])$/.exec(value.trim()); + if (!match) return null; + const amount = Number(match[1]); + if (amount <= 0) return null; + const unitMs = { m: 60_000, h: 3_600_000, d: 86_400_000 }[match[2] as 'm' | 'h' | 'd']; + const total = amount * unitMs; + // A token outliving the workspace is a liability rather than a convenience. + return total > MAX_OBSERVER_TTL_MS ? null : total; +} + +/** + * Resolve the observer token lifetime from `RELAY_OBSERVER_EXPIRES`, falling + * back to the default. An unparseable value falls back rather than throwing: + * a typo in an env var should not cost the user their observer link. + */ +export function resolveObserverTtlMs(env: NodeJS.ProcessEnv = process.env): number { + const configured = env.RELAY_OBSERVER_EXPIRES?.trim(); + if (!configured) return DEFAULT_OBSERVER_TTL_MS; + return parseObserverDuration(configured) ?? DEFAULT_OBSERVER_TTL_MS; +} + +/** + * Mint a scoped, read-only observer token. + * + * Fails soft: any rejection (network, non-2xx, malformed body, timeout) returns + * `null` so the caller can carry on without a link. A run must never fail + * because an observability convenience could not be created. + * + * @returns the minted token, or `null` if it could not be minted + */ +export async function mintObserverToken( + options: MintObserverTokenOptions +): Promise { + const now = options.now ?? Date.now; + const doFetch = options.fetchImpl ?? fetch; + const expiresAt = new Date(now() + (options.ttlMs ?? DEFAULT_OBSERVER_TTL_MS)).toISOString(); + + try { + const res = await doFetch(new URL('/v1/observer-tokens', options.baseUrl).toString(), { + method: 'POST', + headers: { + authorization: `Bearer ${options.workspaceKey}`, + 'content-type': 'application/json', + }, + body: JSON.stringify({ + name: options.name, + ...(options.description === undefined ? {} : { description: options.description }), + scopes: [...OBSERVER_SCOPES], + // `include_dms` defaults to false server-side, but send it explicitly so + // the token's stored filters record the decision rather than an absence. + filters: { + include_dms: options.filters?.include_dms === true, + ...(options.filters?.channel_names?.length + ? { channel_names: options.filters.channel_names } + : {}), + }, + expires_at: expiresAt, + }), + signal: AbortSignal.timeout(MINT_TIMEOUT_MS), + }); + + if (!res.ok) return null; + + const body = (await res.json()) as Record; + const data = (body?.data ?? body) as Record; + const token = data?.token; + const id = data?.id; + if (typeof token !== 'string' || !token.startsWith('ot_live_')) return null; + if (typeof id !== 'string' || id.length === 0) return null; + + return { token, id, expiresAt: data?.expires_at ?? expiresAt }; + } catch { + return null; + } +} diff --git a/packages/core/src/runner.ts b/packages/core/src/runner.ts index 80092d9..b7c6764 100644 --- a/packages/core/src/runner.ts +++ b/packages/core/src/runner.ts @@ -69,6 +69,12 @@ import { formatObserverGuidance, scrubForChannel as scrubWorkflowOutputForChannel, } from './channel-messenger.js'; +import { + buildObserverUrl, + mintObserverToken, + resolveObserverBaseUrl, + resolveObserverTtlMs, +} from './observer-token.js'; import { InMemoryWorkflowDb } from './memory-db.js'; import { buildCommand as buildProcessCommand, spawnProcess } from './process-spawner.js'; import { createProcessBackendExecutor } from './process-backend-executor.js'; @@ -2434,6 +2440,59 @@ export class WorkflowRunner { }); } + /** + * Mint a read-only observer link for this run, so a human can follow along + * without being handed the workspace key. + * + * The workspace key is an administrative credential and the engine rejects it + * on the realtime endpoint anyway, so the link carries a scoped `ot_live_` + * token instead. Best-effort throughout: a run must never fail because an + * observability convenience could not be created. + * + * @param channel - This run's coordination channel + * @returns the observer URL, or `undefined` if no link could be minted + */ + private async mintRunObserverUrl(channel: string): Promise { + if (!this.relayApiKey) return undefined; + + const env = { ...process.env, ...(this.relayOptions.env ?? {}) }; + + // Resolve the dashboard URL BEFORE minting. A bad RELAY_OBSERVER_URL would + // otherwise mint a live token and then throw before printing it, leaving an + // active credential nobody ever saw. + let observerBase: string; + try { + observerBase = resolveObserverBaseUrl(undefined, env); + } catch (error) { + console.warn( + `[WorkflowRunner] Skipping observer link: ${error instanceof Error ? error.message : String(error)}` + ); + return undefined; + } + + const minted = await mintObserverToken({ + baseUrl: this.getRelaycastBaseUrl(), + workspaceKey: this.relayApiKey, + name: `relayflows-${channel}-${randomBytes(4).toString('hex')}`, + description: `Read-only follow-along for the relayflows run in #${channel}`, + // An auto-created workspace exists only for this run, so the link shows + // all of it. A user-supplied workspace may carry unrelated traffic — + // scope the link to this run's channel and leave agent DMs out. + filters: this.relayApiKeyAutoCreated + ? { include_dms: true } + : { include_dms: false, channel_names: [channel] }, + ttlMs: resolveObserverTtlMs(env), + }); + + if (!minted) return undefined; + + try { + return buildObserverUrl(observerBase, minted.token); + } catch { + return undefined; + } + } + private async loadCredentialProxyModule(): Promise { try { const dynamicImport = new Function('specifier', 'return import(specifier)') as ( @@ -4260,10 +4319,12 @@ export class WorkflowRunner { this.log('Resolving Relaycast API key...'); await this.ensureRelaycastApiKey(channel); this.log('API key resolved'); - if (this.relayApiKeyAutoCreated) { - for (const line of formatObserverGuidance(channel)) { - this.log(line); - } + const observerUrl = await this.mintRunObserverUrl(channel); + for (const line of formatObserverGuidance(channel, { + workspaceCreated: this.relayApiKeyAutoCreated, + observerUrl, + })) { + this.log(line); } } From ccbd9a983ac33ce18f928625952c7f24c17ec9bd Mon Sep 17 00:00:00 2001 From: Khaliq Date: Wed, 2 Sep 2026 22:33:44 +0200 Subject: [PATCH 2/4] fix(core): keep an auto-created workspace to its own run, and stop dropping custom channel guidance MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Two findings from review on this branch. An auto-created workspace was not, in fact, per-run. `ensureRelaycastApiKey` says "each run gets full isolation", but cleanup reset `channel`, `relaycast` and the rest while leaving `relayApiKey` and `relayApiKeyAutoCreated` set. On a reused runner instance the next run early-returns from that function and silently joins the previous run's workspace — where the previous run's observer link, unscoped and DM-enabled because the workspace was believed to be single-run, can watch it. Reset the auto-created key on cleanup so the promise the function already makes holds. A key supplied through RELAY_API_KEY belongs to the caller and is re-read from the environment. `isObserverGuidanceLine` matched `Channel: wf-`, inherited from the filters it replaced. A workflow may set `swarm.channel` to a name without the generated prefix, and that guidance line was then discarded as `[workflow HH:MM]` chatter by both Listr filters — the same swallowing bug this branch set out to fix, one case narrower. Match any channel name; `Creating channel:` chatter still fails the predicate, and there is a test for that. Co-Authored-By: Claude Opus 5 (1M context) Claude-Session: https://claude.ai/code/session_01Rg77imZpwSho5Ngdp7wtRD --- packages/core/src/__tests__/channel-messenger.test.ts | 8 ++++++++ packages/core/src/channel-messenger.ts | 6 +++++- packages/core/src/runner.ts | 10 ++++++++++ 3 files changed, 23 insertions(+), 1 deletion(-) diff --git a/packages/core/src/__tests__/channel-messenger.test.ts b/packages/core/src/__tests__/channel-messenger.test.ts index 6f99302..b9ba098 100644 --- a/packages/core/src/__tests__/channel-messenger.test.ts +++ b/packages/core/src/__tests__/channel-messenger.test.ts @@ -108,9 +108,17 @@ describe('channel messenger helpers', () => { } }); + it('survives filtering for a channel name without the generated wf- prefix', () => { + // `swarm.channel` may be set to anything; its guidance must not be dropped. + for (const line of formatObserverGuidance('team-room', { workspaceCreated: true })) { + expect(isObserverGuidanceLine(`[workflow 00:03] ${line}`)).toBe(true); + } + }); + it('does not whitelist ordinary workflow chatter', () => { expect(isObserverGuidanceLine('[workflow 00:03] Resolving Relaycast API key...')).toBe(false); expect(isObserverGuidanceLine('[broker] worker started')).toBe(false); + expect(isObserverGuidanceLine('[workflow 00:03] Creating channel: wf-demo...')).toBe(false); }); it('points a bring-your-own-key run at the observer command when minting failed', () => { diff --git a/packages/core/src/channel-messenger.ts b/packages/core/src/channel-messenger.ts index b3855f1..da21764 100644 --- a/packages/core/src/channel-messenger.ts +++ b/packages/core/src/channel-messenger.ts @@ -156,6 +156,10 @@ export function formatObserverGuidance( * the screen. The observer link is the one thing a user needs off that stream — * swallowing it would defeat the point of printing it — so both filters ask * here rather than each keeping its own copy of the substrings to spare. + * + * Matches `Channel: ` on any name: a workflow may set `swarm.channel` to + * something that does not carry the generated `wf-` prefix, and its guidance + * line has to survive too. */ export function isObserverGuidanceLine(line: string): boolean { return ( @@ -163,7 +167,7 @@ export function isObserverGuidanceLine(line: string): boolean { line.includes('Observation:') || line.includes('Workspace created') || line.includes('agentrelay.com') || - line.includes('Channel: wf-') + line.includes('Channel: ') ); } diff --git a/packages/core/src/runner.ts b/packages/core/src/runner.ts index b7c6764..879e0df 100644 --- a/packages/core/src/runner.ts +++ b/packages/core/src/runner.ts @@ -4538,6 +4538,16 @@ export class WorkflowRunner { this.relaycast = undefined; this.relaycastAgent = undefined; this.channel = undefined; + // An auto-created workspace belongs to the run that created it — + // `ensureRelaycastApiKey` promises "each run gets full isolation". + // Leaving the key set breaks that promise on a reused runner instance: + // the next run would silently join the same workspace, where the + // previous run's observer link can still see it. A key supplied through + // RELAY_API_KEY is the caller's and is re-read from the environment. + if (this.relayApiKeyAutoCreated) { + this.relayApiKey = undefined; + this.relayApiKeyAutoCreated = false; + } this.trajectory = undefined; this.abortController = undefined; this.currentConfig = undefined; From 752242aaa319974c93163d5384a9b903b64a5aa8 Mon Sep 17 00:00:00 2001 From: Khaliq Date: Wed, 2 Sep 2026 22:40:19 +0200 Subject: [PATCH 3/4] fix(core): require https off-machine, and stop overstating what the link is MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Four review findings. The observer URL carries a bearer token in a query parameter, and `resolveObserverBaseUrl` accepted plain `http:` — putting that token on the wire in cleartext. Allow `https:` anywhere, and `http:` only for loopback, where a self-hosted or staging dashboard may have no TLS to offer and nothing crosses a network. This deliberately diverges from `agent-relay observer`, which accepts any http host; diverging toward the safer behaviour, with an error that says why, is the right side to be on. Add the two-run reuse regression the earlier fix lacked: teardown must clear an auto-created key so the next run on the same instance gets its own workspace, and must leave a caller-supplied one alone. Mutation-checked — removing the reset fails the first test with `expected 'rk_live_autocreated' to be undefined`, and the second still passes. The docs oversold the link twice. "Every workflow run that uses a broker prints a link" is not true when minting fails, which is a supported outcome rather than an error. And "safe to paste to a teammate" skipped past what the link is: a bearer credential that anyone holding the URL can read with until it expires or is revoked. Far smaller blast radius than a workspace key — it cannot send, spawn, or administer — but not public, and the docs now say so and point at `agent-relay observer revoke`. Co-Authored-By: Claude Opus 5 (1M context) Claude-Session: https://claude.ai/code/session_01Rg77imZpwSho5Ngdp7wtRD --- README.md | 16 ++++-- docs/observer.mdx | 14 +++-- .../core/src/__tests__/observer-token.test.ts | 16 ++++++ .../src/__tests__/workflow-runner.test.ts | 54 +++++++++++++++++++ packages/core/src/observer-token.ts | 31 ++++++++--- 5 files changed, 118 insertions(+), 13 deletions(-) diff --git a/README.md b/README.md index f1f270f..d247241 100644 --- a/README.md +++ b/README.md @@ -72,7 +72,8 @@ result = ( ## Watching a Run -When a run starts, the runner prints a link you can open to follow it live: +When a run starts, the runner mints and prints a link you can open to follow it +live: ```text [workflow 00:02] Workspace created for this workflow. @@ -81,9 +82,16 @@ When a run starts, the runner prints a link you can open to follow it live: ``` Open the `Observer:` URL and you see messages, agent activity, handoffs, and -deliveries in real time. The link is safe to paste to a teammate: it carries a -**scoped observer token** (`ot_live_`), which is read-only, expires in 24 hours, -and can be revoked individually. +deliveries in real time. The link carries a **scoped observer token** +(`ot_live_`): read-only, expiring in 24 hours, and individually revocable. + +Minting is best-effort, so a link is not guaranteed — see +[If no link appears](#if-no-link-appears). A failed mint never fails the run. + +Treat the link itself as a shared secret. The token is a bearer credential in a +query parameter, so anyone who gets the URL can read the stream it covers until +it expires or you revoke it. It is far safer than a workspace key — it cannot +send, spawn, or administer — but it is not public. ### The two ways a run gets a workspace diff --git a/docs/observer.mdx b/docs/observer.mdx index 51d04be..89d66b6 100644 --- a/docs/observer.mdx +++ b/docs/observer.mdx @@ -3,8 +3,10 @@ title: 'Watch a run' description: 'Follow a running workflow live at agentrelay.com with a read-only observer link, and understand where that link comes from.' --- -Every workflow run that uses a broker prints a link you can open to follow it in -real time — messages, agent activity, handoffs, and deliveries as they happen. +A workflow run that uses a broker mints a link you can open to follow it in real +time — messages, agent activity, handoffs, and deliveries as they happen. +Minting is best-effort: it can fail without failing the run, and a run that +needs no broker prints no link at all. See [If no link appears](#if-no-link-appears). ```text [workflow 00:02] Workspace created for this workflow. @@ -12,10 +14,16 @@ real time — messages, agent activity, handoffs, and deliveries as they happen. [workflow 00:02] Channel: wf-ship-feature-a1b2c3 ``` -Open the `Observer:` URL, or paste it to a teammate. It carries a scoped +Open the `Observer:` URL, or hand it to a teammate. It carries a scoped **observer token** (`ot_live_`): read-only, expiring in 24 hours by default, and individually revocable. +Treat the link as a shared secret. The token is a bearer credential in a query +parameter, so anyone holding the URL can read the stream it covers until it +expires or is revoked. That is a far smaller blast radius than a workspace key — +it cannot send, spawn, or administer — but it is not public. Revoke one with +`agent-relay observer revoke `. + ## Where the link comes from The runner needs a Relaycast workspace to coordinate agents. Where that diff --git a/packages/core/src/__tests__/observer-token.test.ts b/packages/core/src/__tests__/observer-token.test.ts index 954ecb0..dd587d6 100644 --- a/packages/core/src/__tests__/observer-token.test.ts +++ b/packages/core/src/__tests__/observer-token.test.ts @@ -40,6 +40,22 @@ describe('resolveObserverBaseUrl', () => { expect(() => resolveObserverBaseUrl(value, {} as NodeJS.ProcessEnv)).toThrow(); } ); + + it.each([ + 'http://observer.example.com/observer', + 'http://192.168.1.10:4000/observer', + ])('rejects cleartext %s — the link carries a bearer token', (value) => { + expect(() => resolveObserverBaseUrl(value, {} as NodeJS.ProcessEnv)).toThrow(/https/); + }); + + it.each([ + 'http://localhost:4000/observer', + 'http://127.0.0.1:4000/observer', + 'http://[::1]:4000/observer', + 'http://dash.localhost/observer', + ])('allows cleartext %s — nothing crosses a network', (value) => { + expect(resolveObserverBaseUrl(value, {} as NodeJS.ProcessEnv)).toBe(value); + }); }); describe('buildObserverUrl', () => { diff --git a/packages/core/src/__tests__/workflow-runner.test.ts b/packages/core/src/__tests__/workflow-runner.test.ts index 23d597e..5b612b4 100644 --- a/packages/core/src/__tests__/workflow-runner.test.ts +++ b/packages/core/src/__tests__/workflow-runner.test.ts @@ -498,6 +498,60 @@ agents: // ── Execution ────────────────────────────────────────────────────────── describe('execute', () => { + // `ensureRelaycastApiKey` promises "each run gets full isolation" by creating + // a fresh workspace per run. It early-returns when `relayApiKey` is already + // set, so teardown has to clear an auto-created one — otherwise a second run + // on the same instance silently joins the first run's workspace, where the + // first run's observer link can still watch it. + it('clears an auto-created Relaycast key on teardown so the next run gets its own workspace', async () => { + (runner as any).relayApiKey = 'rk_live_autocreated'; + (runner as any).relayApiKeyAutoCreated = true; + + const config = { + version: '1', + name: 'teardown-resets-autocreated-key', + swarm: { pattern: 'dag' }, + agents: [], + workflows: [ + { + name: 'default', + steps: [{ name: 'noop', type: 'deterministic', command: 'true' }], + }, + ], + trajectories: false, + } as unknown as RelayYamlConfig; + + await runner.execute(config, 'default'); + + expect((runner as any).relayApiKey).toBeUndefined(); + expect((runner as any).relayApiKeyAutoCreated).toBe(false); + }); + + it('keeps a caller-supplied Relaycast key across runs', async () => { + // A key from RELAY_API_KEY belongs to the caller; clearing it would just + // make the next run re-read the same value from the environment. + (runner as any).relayApiKey = 'rk_live_caller_supplied'; + (runner as any).relayApiKeyAutoCreated = false; + + const config = { + version: '1', + name: 'teardown-keeps-supplied-key', + swarm: { pattern: 'dag' }, + agents: [], + workflows: [ + { + name: 'default', + steps: [{ name: 'noop', type: 'deterministic', command: 'true' }], + }, + ], + trajectories: false, + } as unknown as RelayYamlConfig; + + await runner.execute(config, 'default'); + + expect((runner as any).relayApiKey).toBe('rk_live_caller_supplied'); + }); + it('spawns a persona with its declared runtime and verifies readiness plus registration', async () => { mockRelayInstance.spawnPty.mockImplementation(async (input: { name: string; task?: string }) => { mockRelayInstance.listAgents.mockResolvedValueOnce([{ name: input.name }]); diff --git a/packages/core/src/observer-token.ts b/packages/core/src/observer-token.ts index 44f2cfe..4042f18 100644 --- a/packages/core/src/observer-token.ts +++ b/packages/core/src/observer-token.ts @@ -81,14 +81,29 @@ export interface MintedObserverToken { expiresAt: string; } +/** + * Whether a hostname never leaves the machine, so cleartext carries no + * credential across a network. + */ +function isLoopbackHost(hostname: string): boolean { + const host = hostname.toLowerCase().replace(/^\[|\]$/g, ''); + return host === 'localhost' || host === '127.0.0.1' || host === '::1' || host.endsWith('.localhost'); +} + /** * Resolve the observer dashboard base URL: explicit value, then * `RELAY_OBSERVER_URL` (for self-hosted or staging dashboards), then the hosted * default. * - * @throws if the resolved value is not an http(s) URL — the token is appended to - * this URL's query string, so the scheme decides where a live credential ends - * up, and `new URL` happily accepts `data:` and `javascript:`. + * The token is appended to this URL's query string, so the scheme decides where + * a live credential ends up. `new URL` happily accepts `data:` and + * `javascript:`, and plain `http:` would put a bearer token on the wire in + * cleartext. Only `https:` is allowed off-machine; `http:` is permitted solely + * for loopback, where a self-hosted or staging dashboard has no TLS to offer + * and nothing crosses a network. + * + * @throws if the resolved value is not a URL, or would transmit the token in + * cleartext to a remote host */ export function resolveObserverBaseUrl(explicit?: string, env: NodeJS.ProcessEnv = process.env): string { const value = explicit?.trim() || env.RELAY_OBSERVER_URL?.trim() || DEFAULT_OBSERVER_URL; @@ -98,10 +113,14 @@ export function resolveObserverBaseUrl(explicit?: string, env: NodeJS.ProcessEnv } catch { throw new Error(`Invalid observer URL: ${value}`); } - if (parsed.protocol !== 'https:' && parsed.protocol !== 'http:') { - throw new Error(`Observer URL must be http or https: ${value}`); + if (parsed.protocol === 'https:') return value; + if (parsed.protocol === 'http:' && isLoopbackHost(parsed.hostname)) return value; + if (parsed.protocol === 'http:') { + throw new Error( + `Observer URL must use https (the link carries a bearer token): ${value}` + ); } - return value; + throw new Error(`Observer URL must be http or https: ${value}`); } /** From 1372cd3fed23780f6bb8e10acd12cb768a33ce3c Mon Sep 17 00:00:00 2001 From: Khaliq Date: Wed, 2 Sep 2026 22:47:11 +0200 Subject: [PATCH 4/4] fix(core): reset before fallible teardown, tighten the guidance matcher, gate the printed link MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Three review findings, plus a rewrite of the regressions behind them. The auto-created key reset sat near the end of the run's `finally` block, after `stopRelayfileEventSubscriptions()` and `shutdownRelay()`. Either can reject, and a throw inside `finally` skips everything below it — so a failed broker shutdown would leave the key set and hand the next run the previous run's workspace. Move the reset to the top of the block, ahead of anything fallible. Mutation-checked: with the old ordering, the new shutdown-failure test fails with `expected 'rk_live_autocreated' to be undefined`. `isObserverGuidanceLine` matched a bare `Channel: `, which let any log line containing that substring bypass the terminal noise filter the guidance block needed an exemption from. The guidance line always carries its own two-space indent, so match ` Channel: ` — still correct for arbitrary channel names like `team-room`, without waving through `[broker]` chatter. `formatObserverGuidance` printed any truthy `observerUrl`. `buildObserverUrl` already refuses to construct a link around anything but an `ot_live_` token, but the formatter is exported and prints to a terminal, so it should not rely on every caller having built its input correctly. Validate at the printing site too: a URL whose `key` is not an `ot_live_` token falls back to the no-link guidance. Two independent gates on the invariant #27 existed to protect. The teardown regressions were also fair to criticise: they seeded private state and ran once, so they asserted the mechanism (a field is cleared) and not the behaviour (a second run gets its own workspace). Rewritten to drive the real path — mock the workspace endpoint, resolve a key, run, resolve again — and assert two distinct workspaces were provisioned. The caller-supplied companion asserts the opposite: same key across runs, and the workspace endpoint never called. Mutation-checked: dropping the reset fails with `expected 'rk_live_workspace_1' to be 'rk_live_workspace_2'`. Co-Authored-By: Claude Opus 5 (1M context) Claude-Session: https://claude.ai/code/session_01Rg77imZpwSho5Ngdp7wtRD --- .../src/__tests__/channel-messenger.test.ts | 21 ++++ .../src/__tests__/workflow-runner.test.ts | 104 ++++++++++++------ packages/core/src/channel-messenger.ts | 28 ++++- packages/core/src/runner.ts | 24 ++-- 4 files changed, 130 insertions(+), 47 deletions(-) diff --git a/packages/core/src/__tests__/channel-messenger.test.ts b/packages/core/src/__tests__/channel-messenger.test.ts index b9ba098..1196d8b 100644 --- a/packages/core/src/__tests__/channel-messenger.test.ts +++ b/packages/core/src/__tests__/channel-messenger.test.ts @@ -115,10 +115,31 @@ describe('channel messenger helpers', () => { } }); + it.each([ + ['a workspace key', 'https://agentrelay.com/observer?key=rk_live_secret'], + ['an agent token', 'https://agentrelay.com/observer?key=at_live_secret'], + ['no key at all', 'https://agentrelay.com/observer'], + ['a malformed URL', 'not a url'], + ])('refuses to print an observer link carrying %s', (_label, url) => { + const guidance = formatObserverGuidance('wf-demo', { + workspaceCreated: true, + observerUrl: url, + }); + + expect(guidance.join('\n')).not.toContain('Observer:'); + expect(guidance.join('\n')).not.toContain('secret'); + expect(guidance).toContain( + ' Observation: unavailable — could not mint a read-only observer token ' + + '(set RELAY_API_KEY to run against a workspace you own)' + ); + }); + it('does not whitelist ordinary workflow chatter', () => { expect(isObserverGuidanceLine('[workflow 00:03] Resolving Relaycast API key...')).toBe(false); expect(isObserverGuidanceLine('[broker] worker started')).toBe(false); expect(isObserverGuidanceLine('[workflow 00:03] Creating channel: wf-demo...')).toBe(false); + // A bare `Channel: ` elsewhere in a log line must not buy an exemption. + expect(isObserverGuidanceLine('[broker] joined Channel: wf-demo')).toBe(false); }); it('points a bring-your-own-key run at the observer command when minting failed', () => { diff --git a/packages/core/src/__tests__/workflow-runner.test.ts b/packages/core/src/__tests__/workflow-runner.test.ts index 5b612b4..bdb2228 100644 --- a/packages/core/src/__tests__/workflow-runner.test.ts +++ b/packages/core/src/__tests__/workflow-runner.test.ts @@ -503,53 +503,93 @@ agents: // set, so teardown has to clear an auto-created one — otherwise a second run // on the same instance silently joins the first run's workspace, where the // first run's observer link can still watch it. - it('clears an auto-created Relaycast key on teardown so the next run gets its own workspace', async () => { - (runner as any).relayApiKey = 'rk_live_autocreated'; - (runner as any).relayApiKeyAutoCreated = true; - - const config = { + const deterministicConfig = (name: string) => + ({ version: '1', - name: 'teardown-resets-autocreated-key', + name, swarm: { pattern: 'dag' }, agents: [], workflows: [ - { - name: 'default', - steps: [{ name: 'noop', type: 'deterministic', command: 'true' }], - }, + { name: 'default', steps: [{ name: 'noop', type: 'deterministic', command: 'true' }] }, ], trajectories: false, - } as unknown as RelayYamlConfig; + }) as unknown as RelayYamlConfig; + + it('provisions a separate workspace for each auto-created run on one instance', async () => { + vi.stubEnv('RELAY_API_KEY', ''); + const created: string[] = []; + const fetchSpy = vi.spyOn(globalThis, 'fetch').mockImplementation((async (url: string) => { + if (String(url).includes('/v1/workspaces')) { + const key = `rk_live_workspace_${created.length + 1}`; + created.push(key); + return { ok: true, json: async () => ({ data: { api_key: key } }) } as Response; + } + return { ok: true, json: async () => ({}) } as Response; + }) as unknown as typeof fetch); - await runner.execute(config, 'default'); + try { + // First run resolves a key, then tears down. + await (runner as any).ensureRelaycastApiKey('wf-first'); + expect((runner as any).relayApiKey).toBe('rk_live_workspace_1'); + await runner.execute(deterministicConfig('first-run'), 'default'); + + // Second run must provision its own workspace rather than early-returning + // onto the first run's — which the first run's observer link can watch. + await (runner as any).ensureRelaycastApiKey('wf-second'); + expect((runner as any).relayApiKey).toBe('rk_live_workspace_2'); + + const workspaceCalls = fetchSpy.mock.calls.filter((call) => + String(call[0]).includes('/v1/workspaces') + ); + expect(workspaceCalls).toHaveLength(2); + expect(created).toEqual(['rk_live_workspace_1', 'rk_live_workspace_2']); + } finally { + fetchSpy.mockRestore(); + vi.unstubAllEnvs(); + } + }); + + it('clears the auto-created key even when broker shutdown fails', async () => { + // The reset sits ahead of fallible teardown for exactly this reason: a + // rejecting `shutdownRelay()` skips the rest of the finally block, and + // this invariant must not be lost to a failed broker shutdown. + (runner as any).relayApiKey = 'rk_live_autocreated'; + (runner as any).relayApiKeyAutoCreated = true; + vi.spyOn(runner as any, 'shutdownRelay').mockRejectedValue(new Error('broker shutdown failed')); + + await runner + .execute(deterministicConfig('teardown-resets-despite-shutdown-failure'), 'default') + .catch(() => undefined); expect((runner as any).relayApiKey).toBeUndefined(); expect((runner as any).relayApiKeyAutoCreated).toBe(false); }); - it('keeps a caller-supplied Relaycast key across runs', async () => { - // A key from RELAY_API_KEY belongs to the caller; clearing it would just - // make the next run re-read the same value from the environment. - (runner as any).relayApiKey = 'rk_live_caller_supplied'; - (runner as any).relayApiKeyAutoCreated = false; + it('reuses a caller-supplied key across runs without provisioning a workspace', async () => { + // A key from RELAY_API_KEY belongs to the caller. Clearing it would make + // the next run re-read the same value; it must never trigger provisioning. + vi.stubEnv('RELAY_API_KEY', 'rk_live_caller_supplied'); + const fetchSpy = vi.spyOn(globalThis, 'fetch').mockImplementation((async () => { + return { ok: true, json: async () => ({}) } as Response; + }) as unknown as typeof fetch); - const config = { - version: '1', - name: 'teardown-keeps-supplied-key', - swarm: { pattern: 'dag' }, - agents: [], - workflows: [ - { - name: 'default', - steps: [{ name: 'noop', type: 'deterministic', command: 'true' }], - }, - ], - trajectories: false, - } as unknown as RelayYamlConfig; + try { + await (runner as any).ensureRelaycastApiKey('wf-first'); + expect((runner as any).relayApiKey).toBe('rk_live_caller_supplied'); + expect((runner as any).relayApiKeyAutoCreated).toBe(false); - await runner.execute(config, 'default'); + await runner.execute(deterministicConfig('supplied-key-run'), 'default'); - expect((runner as any).relayApiKey).toBe('rk_live_caller_supplied'); + await (runner as any).ensureRelaycastApiKey('wf-second'); + expect((runner as any).relayApiKey).toBe('rk_live_caller_supplied'); + + expect( + fetchSpy.mock.calls.filter((call) => String(call[0]).includes('/v1/workspaces')) + ).toHaveLength(0); + } finally { + fetchSpy.mockRestore(); + vi.unstubAllEnvs(); + } }); it('spawns a persona with its declared runtime and verifies readiness plus registration', async () => { diff --git a/packages/core/src/channel-messenger.ts b/packages/core/src/channel-messenger.ts index da21764..a6f4387 100644 --- a/packages/core/src/channel-messenger.ts +++ b/packages/core/src/channel-messenger.ts @@ -109,6 +109,23 @@ export function scrubSecrets(text: string): string { return result; } +/** + * Whether a URL carries a scoped observer token and nothing else. + * + * `buildObserverUrl` already refuses to construct anything else, so this is a + * second, independent gate on the same invariant — the one #27 existed to + * protect. This function is exported and prints to a terminal; it should not + * depend on every caller having built its input correctly to avoid emitting a + * workspace key. + */ +function isScopedObserverUrl(url: string): boolean { + try { + return new URL(url).searchParams.get('key')?.startsWith('ot_live_') === true; + } catch { + return false; + } +} + export interface ObserverGuidanceOptions { /** Set when the runner auto-created a throwaway workspace for this run. */ workspaceCreated?: boolean; @@ -135,7 +152,7 @@ export function formatObserverGuidance( if (options.workspaceCreated) { lines.push('Workspace created for this workflow.'); } - if (options.observerUrl) { + if (options.observerUrl && isScopedObserverUrl(options.observerUrl)) { lines.push(` Observer: ${options.observerUrl}`); } else if (options.workspaceCreated) { lines.push( @@ -157,9 +174,10 @@ export function formatObserverGuidance( * swallowing it would defeat the point of printing it — so both filters ask * here rather than each keeping its own copy of the substrings to spare. * - * Matches `Channel: ` on any name: a workflow may set `swarm.channel` to - * something that does not carry the generated `wf-` prefix, and its guidance - * line has to survive too. + * Matches the guidance line's own two-space indent rather than a `wf-` prefix: + * a workflow may set `swarm.channel` to any name, and its line has to survive + * too — but a bare `Channel: ` would wave through unrelated chatter that the + * filters exist to drop. */ export function isObserverGuidanceLine(line: string): boolean { return ( @@ -167,7 +185,7 @@ export function isObserverGuidanceLine(line: string): boolean { line.includes('Observation:') || line.includes('Workspace created') || line.includes('agentrelay.com') || - line.includes('Channel: ') + line.includes(' Channel: ') ); } diff --git a/packages/core/src/runner.ts b/packages/core/src/runner.ts index 879e0df..230e3dc 100644 --- a/packages/core/src/runner.ts +++ b/packages/core/src/runner.ts @@ -4507,6 +4507,20 @@ export class WorkflowRunner { } } } finally { + // First, ahead of any fallible teardown. An auto-created workspace + // belongs to the run that created it — `ensureRelaycastApiKey` promises + // "each run gets full isolation" — and leaving the key set breaks that + // promise on a reused runner instance: the next run silently joins the + // same workspace, where the previous run's observer link can still watch + // it. Everything below this can throw (`shutdownRelay`, subscription + // teardown), which would skip the rest of the block; this invariant is + // not one to lose to a failed broker shutdown. A key supplied through + // RELAY_API_KEY is the caller's and is re-read from the environment. + if (this.relayApiKeyAutoCreated) { + this.relayApiKey = undefined; + this.relayApiKeyAutoCreated = false; + } + this.lastFailedStepOutput.clear(); this.lastCustomVerificationFailure.clear(); for (const stream of this.ptyLogStreams.values()) stream.end(); @@ -4538,16 +4552,6 @@ export class WorkflowRunner { this.relaycast = undefined; this.relaycastAgent = undefined; this.channel = undefined; - // An auto-created workspace belongs to the run that created it — - // `ensureRelaycastApiKey` promises "each run gets full isolation". - // Leaving the key set breaks that promise on a reused runner instance: - // the next run would silently join the same workspace, where the - // previous run's observer link can still see it. A key supplied through - // RELAY_API_KEY is the caller's and is re-read from the environment. - if (this.relayApiKeyAutoCreated) { - this.relayApiKey = undefined; - this.relayApiKeyAutoCreated = false; - } this.trajectory = undefined; this.abortController = undefined; this.currentConfig = undefined;