Skip to content

feat(broker): run-token mode for system-held runs (rt_live_ via RELAY_RUN_TOKEN_FILE) - #1980

Draft
kjgbot wants to merge 4 commits into
mainfrom
feat/broker-run-token-mode
Draft

kjgbot wants to merge 4 commits into
mainfrom
feat/broker-run-token-mode

Conversation

@kjgbot

@kjgbot kjgbot commented Oct 18, 2026 •

Copy link
Copy Markdown
Contributor

DEPLOY HOLD

Do not merge or release this until all of the following have landed and deployed:

Legacy rk_ behavior is unchanged, so this is inert until a launcher sets RELAY_RUN_TOKEN_FILE. It still needs the server side before anything uses it. Spec: section 4 and the "FINAL Relaycast surface" section of relay-briefs/run-token-contract.md.

Summary

harness-driver

  • New RuntimeSpawnOptions.runTokenFile option, with RELAY_RUN_TOKEN_FILE from the caller or parent env as a fallback.
  • When either is set, the broker env gets only RELAY_RUN_TOKEN_FILE (the path, never the value). RELAY_API_KEY, RELAY_WORKSPACE_KEY, AGENT_RELAY_WORKSPACE_KEY, and RELAY_WORKSPACES_JSON are stripped. The legacy path is unchanged.

Broker run-token mode (enabled when RELAY_RUN_TOKEN_FILE is set and no rk_ key is configured)

  • New relaycast/run_token.rs:
    • Reads and validates {token, expiresAt} on every request. Nothing is cached, and the token is never copied into env, argv, or files.
    • On a 401, re-reads the file and retries once.
    • Debug output and error messages are redacted.
  • Broker self-registration (auth.rs) uses POST /v1/agents with the run token. The session holds no workspace key (api_key is empty).
  • It never creates a workspace. The fresh-workspace path and create_workspace fail closed whenever a run token file is configured.
  • Worker registration (ws.rs): all three create paths go through post_create_agent, which uses the run token. A name collision returns AlreadyExists instead of calling GET /v1/agents/:name plus takeover.
  • Worker release uses DELETE /v1/agents/:name with the run token:
    • A 404 (agent from another run) counts as done.
    • A local token-hash check replaces the server-side guard that the release route provided.
  • GET /v1/agents/:name is no longer called with the run token. Channel-scope checks use the worker's own at_live_, and presence probes use the broker's own at_live_. Attachment downloads also use the broker's at_live_.
  • Node minting (node_control.rs) uses POST /v1/nodes with the run token, re-reading the file on 401.
  • Worker env and MCP config:
    • Run-token mode delegates no workspace key or run token.
    • RELAY_RUN_TOKEN_FILE was added to INHERITED_RELAY_CREDENTIAL_ENV_KEYS, so workers never inherit the path.
    • spawn_env_vars skips the key variables when the key is empty.
  • Redaction now covers rt_live_ in Rust (redact.rs, node-mint logs) and TS (cli and cloud redact helpers).

Agent token rotation (run-token mode; relaycast #550 tokens expire after 15 minutes)

  • The broker rotates every agent token it owns with POST /v1/agents/:name/rotate-token, authenticated by that agent's own at_live_. It never uses takeover or delete-and-recreate.
  • Rotation is scheduled from token_expires_at (registration or rotation response) at the earlier of 2/3 of the lifetime or 5 minutes before expiry; 10 minutes for a 15-minute token.
  • Transient failures retry while the previous token's 60-second grace window still applies. A 401/403 fails closed and is logged without the token value. A token with unknown expiry is rotated once after 60 seconds to learn it.
  • Broker self: rotation starts from the startup registration's expiry. The node connection authenticates with the node token, so no WS or node session needs reconnecting.
  • Workers: each worker gets RELAY_AGENT_TOKEN_FILE instead of a static RELAY_AGENT_TOKEN. The file holds {token, expiresAt} and lives in a broker-owned 0700 directory (<state>/agent-tokens).
    • It is written as a 0600 temp file and renamed into place on every rotation, and removed when the worker is released.
    • The MCP config for every CLI carries the file path. RELAY_AGENT_TOKEN_FILE is also scrubbed from inherited env.
    • A record of the token hashes the broker rotated away from lets owned-identity cleanup still recognize the identity after its token changed.
  • Worker MCP server / CLI (agent-relay-mcp.ts, lib/sdk-client.ts): both honor RELAY_AGENT_TOKEN_FILE with no RELAY_API_KEY.
    • @relaycast/sdk takes only a static apiKey and has no token provider. The adapter re-reads the file before each request (stat-cached on inode, size, and mtime) and builds the per-request agent client with the current token.
    • On rotation the realtime bridge is re-created with the new token and existing subscriptions are kept.

Test results

  • cargo test -p agent-relay-broker -p relay-pty: all suites pass. That is 1575 broker lib tests (7 ignored), 269 relay-pty tests, and the integration suites.
  • New Rust tests cover:
    • the file is re-read every request and a rotated token is used on the next request (auth, worker registration, node mint);
    • a 401 triggers one re-read and retry, and a second 401 is surfaced;
    • no workspace is created in run-token mode (startup, a rejected token, and the direct fresh-workspace path);
    • a name collision fails closed without a lookup or takeover;
    • release uses DELETE;
    • the channel-scope check uses the worker's token;
    • a real spawned worker in run-token mode sees only RELAY_AGENT_TOKEN, with zero rk_/rt_ values in its env;
    • legacy mode is unchanged: an rk_ key wins over the run token file, and the worker env and spawn_env_vars are unchanged.
  • Rotation tests (Rust, with a fake clock):
    • rotation is scheduled at 10 minutes for a 15-minute token and happens before expiry; the next rotation uses the new expiry;
    • the worker file is replaced atomically (new inode), mode 0600, in a 0700 directory, with no temp files left behind;
    • a rejected rotation fails closed with no takeover or re-registration;
    • after rotation, release still deletes the identity and removes the file;
    • legacy mode never starts rotation;
    • a real spawned worker sees only RELAY_AGENT_TOKEN_FILE, with no rk_/rt_/at_ values in its env;
    • MCP configs carry the file path, not a token.
  • cargo clippy -p agent-relay-broker -p relay-pty --tests: no new warnings. Two existing warnings remain in test code.
  • vitest: spawn-config.test.ts (5 new run-token cases), the cli/cloud redact tests, lib/agent-token-file.test.ts, and the MCP startup tests pass. The MCP tests check that the token file works without a workspace key and that a rotated token is used on the very next request.
  • The full packages/cli suite: 2107 passed. The 10 failures are in files that also fail on the base commit locally (bootstrap/entrypoint/product-surfaces/node-definition-loader/bun loader; they need built packages or bun). broker-process.test.ts has 2 failures that also fail on clean origin/main locally; they are environment-dependent and unrelated.

Remaining gaps

  • Workspace-scoped calls that are still unsupported in run-token mode and degrade with a logged 401:
    • PATCH /v1/agents/:name metadata publish;
    • DM history (all_dm_conversations);
    • impersonating an agent this broker didn't register;
    • local-only/degraded mode.
  • Restart reclaim: a broker restarted in the same run under the same name fails closed on the 409. Same-run names are expected to be unique.
  • AI SDK native relay tools (packages/harnesses) and @agent-relay/session read only RELAY_AGENT_TOKEN. In run-token mode the native relay tools turn themselves off instead of using a stale token. Supporting the token file there is a follow-up.
  • Fleet-supplied worker tokens and node agent.register replies don't pass token_expires_at through yet, so those tokens are rotated once after 60 seconds to learn their expiry.
  • WebSocket revocation: closing node and WebSocket connections after DELETE /v1/runs/:runId/authority is the server's job (relaycast fix: pass --model flag to spawned CLI processes #559).

🤖 Generated with Claude Code


View with [code]smith Autofix with [code]smith
Need help on this PR? Tag @codesmith-bot with what you need.


Agent Relay sessions

  • claude session 79b75307 · opened via gh pr create · last active 2026-10-18

…_RUN_TOKEN_FILE)

When RELAY_RUN_TOKEN_FILE is set and no rk_ workspace key is configured, the
broker runs without any static workspace key:

- Registers itself and spawned workers via POST /v1/agents, mints its node via
  POST /v1/nodes, and releases workers via DELETE /v1/agents/:name, all with
  the run token read from the file on every request; a 401 re-reads the file
  and retries once.
- Never creates a workspace (fresh-workspace path and create_workspace fail
  closed whenever a run token file is configured).
- Avoids GET /v1/agents/:name with the run token: channel-scope and presence
  lookups use agent tokens; name collisions fail closed instead of takeover.
- Workers get only their own at_live_ token: no workspace key, run token, or
  RELAY_RUN_TOKEN_FILE in worker env or MCP config.

harness-driver: RuntimeSpawnOptions.runTokenFile (or RELAY_RUN_TOKEN_FILE)
passes only the file path to the broker and strips RELAY_API_KEY,
RELAY_WORKSPACE_KEY, AGENT_RELAY_WORKSPACE_KEY and RELAY_WORKSPACES_JSON.
Legacy workspace-key behavior is unchanged.

Redaction helpers (Rust and TS) now cover rt_live_.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
@coderabbitai

coderabbitai Bot commented Oct 18, 2026

Copy link
Copy Markdown
Contributor

Important

Draft PR not reviewed

Draft PRs are not automatically reviewed by default.

  • Trigger a manual review

To automatically review draft PRs, update your CodeRabbit configuration:

reviews:
  auto_review:
    drafts: true
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

github-actions Bot and others added 3 commits October 18, 2026 04:23
…ken mode

Relaycast #550 agent tokens expire after 15 minutes and run-token mode has no
workspace key to take an identity over, so the broker now rotates every agent
token it owns via POST /v1/agents/:name/rotate-token (authenticated by the
agent's own at_live_), never via takeover or delete-and-recreate:

- Rotation is scheduled from token_expires_at (registration or rotation
  response) at the earlier of 2/3 of the lifetime or 5 minutes before expiry.
  Transient failures retry within the previous-token grace window; a 401/403
  fails closed and is logged without the token value. Unknown expiry is
  probe-rotated once after 60s.
- Broker self: the startup registration's token_expires_at seeds rotation.
  The node connection uses the node token, so nothing needs reconnecting.
- Workers: get RELAY_AGENT_TOKEN_FILE (no static RELAY_AGENT_TOKEN), a
  per-worker file in a broker-owned 0700 directory, replaced atomically
  (0600 temp file + rename) on every rotation and removed on release.
  MCP configs for every CLI carry the file path. Rotation lineage keeps owned
  cleanup working after the token changes.
- Agent Relay MCP server and CLI honor RELAY_AGENT_TOKEN_FILE: stat-cached
  re-read before every request, no RELAY_API_KEY needed, realtime bridge
  re-established on rotation with subscriptions kept.

Legacy (rk_) mode is unchanged.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
}
let raw: string;
try {
raw = fs.readFileSync(this.path, 'utf8');

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants