Skip to content

herdr-socket-api.md

Latest commit

 

History

History
563 lines (477 loc) · 26.6 KB

herdr-socket-api.md

File metadata and controls

563 lines (477 loc) · 26.6 KB
Raw

Herdr Socket API — Ground Truth Reference

Status: Verified against a live Herdr server. This document supersedes the reverse-engineered guesses in project-idea.md for anything protocol-related.

  • Captured: 2026-07-28
  • Herdr version: 0.7.5 (stable channel)
  • Wire protocol version: 17
  • Schema version: 1
  • Method used to capture: herdr api schema --json + live socket probes + herdr api snapshot
  • Full raw schema saved alongside: run herdr api schema --json > herdr-schema.json to regenerate (~248 KB, 89 request methods).

This is the source of truth the VS Code extension client is built against. When Herdr bumps its protocol version, re-run the capture commands and diff.

1. Transport & framing

  • Server socket (control/API): ~/.config/herdr/herdr.sock (resolved absolute: /Users/<user>/.config/herdr/herdr.sock).
  • Client socket: ~/.config/herdr/herdr-client.sock (used by the TUI client; the API client uses the server socket above).
  • Remote sessions get their own socket: $TMPDIR/herdr-remote-<pid>-<host>-<session>.sock.
  • Framing: newline-delimited JSON (NDJSON). One JSON object per line, each terminated with \n. Send a request as JSON.stringify(req) + "\n"; every response/event is a single JSON object followed by \n.
  • Transport type: Unix domain stream socket (AF_UNIX, SOCK_STREAM).
  • Connection model — VERIFIED, critical:
    • Regular RPC methods are strictly one-request-per-connection. The server reads one request, writes exactly one response line, then closes the socket. A second request on the same connection is never answered (pipelining two pings yields only the first response, then close → EPIPE on further writes). Open a fresh connection for every RPC call. Do not pool or reuse RPC connections.
    • events.subscribe is the exception: it opens a long-lived streaming connection. The server replies {result:{type:"subscription_started"}} then streams {event,data} lines indefinitely until the client closes. Keep this connection open; use a separate connection for RPC calls.
    • Because RPC is one-shot, correlation by id matters only for the single in-flight response on that connection. On the events connection, distinguish the initial {id,result} ack from subsequent {event,data} pushes.

Verified probe

Sending {"id":"probe1","method":"ping","params":{}}\n returned:

{"id":"probe1","result":{"type":"pong","version":"0.7.5","protocol":17,"capabilities":{"live_handoff":true,"detached_server_daemon":true}}}

(followed by a trailing \n).

Discovering the socket path

  • herdr status prints server.socket: — the authoritative path.
  • Fallback: $HERDR_SOCK env if set, else ~/.config/herdr/herdr.sock.
  • Panes launched by Herdr also export: HERDR_ENV, HERDR_WORKSPACE_ID, HERDR_TAB_ID, HERDR_PANE_ID (identity channel for hooks/integrations).

2. Message envelopes

Request (client → server)

{ "id": "string",        // REQUIRED. Correlation id, echoed in the response.
  "method": "string",    // REQUIRED. One of the 89 methods (section 4).
  "params": { ... } }    // Method-specific. Use {} for EmptyParams methods.

Top-level required: ["id"]. method + params per the discriminated union.

Success response (server → client)

{ "id": "string", "result": <ResponseResult> }

Error response (server → client)

{ "id": "string", "error": <ErrorBody> }

A response is success iff it has a result key; error iff it has an error key.

Event (server → client, after subscribe)

{ "event": <EventKind>, "data": <EventData> }

Note: events use event + data; they do not carry an id. Distinguish events from responses by presence of event vs id.

3. Core enums

// Agent status as reported by the server (rollup + per pane).
type AgentStatus = "idle" | "working" | "blocked" | "done" | "unknown";

// State a client/integration may REPORT for a pane (note: no "done").
type PaneAgentState = "idle" | "working" | "blocked" | "unknown";

// pane.read / agent.read source region.
type ReadSource = "visible" | "recent" | "recent_unwrapped" | "detection";

// pane.read / agent.read output format.
type ReadFormat = "text" | "ansi";

done is a server-derived attention state (unseen completion), consistent with the project-idea.md model — clients receive it but cannot report it.

4. Request methods (89 total)

Grouped. → Params names reference section 5. Methods marked ★ are the ones Phase-1 MVP uses.

Server / lifecycle

  • ping → PingParams ★
  • server.stop, server.live_handoff, server.reload_config
  • server.agent_manifests, server.reload_agent_manifests
  • notification.show
  • client.window_title.set, client.window_title.clear

Session

  • session.snapshot → EmptyParams ★ (full topology + panes + agents in one call)

Workspace

  • workspace.create, workspace.list ★, workspace.get, workspace.focus ★, workspace.rename, workspace.move, workspace.report_metadata, workspace.close

Worktree

  • worktree.list, worktree.create, worktree.open, worktree.remove

Tab

  • tab.create, tab.list ★, tab.get, tab.focus, tab.rename, tab.move, tab.close

Agent

  • agent.list ★, agent.get ★, agent.read ★, agent.explain, agent.send_keys ★, agent.rename, agent.view.set, agent.view.clear, agent.focus ★, agent.start ★, agent.prompt ★, agent.wait ★

Pane

  • pane.split, pane.swap, pane.move, pane.zoom, pane.layout, pane.process_info, pane.neighbor, pane.edges, pane.focus_direction, pane.resize, pane.list ★, pane.current, pane.get, pane.focus, pane.rename, pane.send_text ★, pane.send_keys, pane.send_input, pane.read ★, pane.graphics.{set,clear,info}, pane.report_agent, pane.report_agent_session, pane.report_metadata, pane.clear_agent_authority, pane.release_agent, pane.close

Layout

  • layout.export, layout.apply, layout.set_split_ratio

Events

  • events.subscribe → EventsSubscribeParams ★
  • events.wait → EventsWaitParams
  • pane.wait_for_output

Integration / Plugin

  • integration.install, integration.uninstall
  • plugin.link, plugin.list, plugin.unlink, plugin.enable, plugin.disable
  • plugin.action.list, plugin.action.invoke, plugin.log.list
  • plugin.pane.{open,focus,close}
  • popup.close

5. Param shapes (Phase-1 methods)

? = nullable/optional. Types are the JSON-schema-resolved shapes.

PingParams          = {}                                  // EmptyParams
EmptyParams         = {}

// Targeting: `target` accepts an agent id / pane id / selector string.
AgentTarget         = { target: string }                  // agent.get, agent.focus

AgentReadParams     = { target: string, source: ReadSource,   // REQUIRED: target, source
                        format?: ReadFormat, lines?: number|null,
                        strip_ansi?: boolean }

AgentSendKeysParams = { target: string, keys: string[] }      // REQUIRED both

AgentStartParams    = { name: string, kind: string, pane_id: string, // REQUIRED
                        args?: string[], timeout_ms?: number|null }
// `kind` = agent kind id (e.g. "claude", "codex"); `name` = display name;
// `pane_id` = existing shell pane to start the agent in (pane != agent).

AgentPromptParams   = { target: string, text: string,        // REQUIRED target, text
                        wait?: AgentPromptWaitOptions|null }

AgentWaitParams     = { target: string,                      // REQUIRED target
                        until?: AgentStatus[], timeout_ms?: number|null }

PaneListParams      = { workspace_id?: string|null }
TabListParams       = { workspace_id?: string|null }

PaneSendTextParams  = { pane_id: string, text: string }      // REQUIRED both
PaneReadParams      = { pane_id: string, source: ReadSource, // REQUIRED pane_id, source
                        format?: ReadFormat, lines?: number|null,
                        strip_ansi?: boolean }

EventsSubscribeParams = { subscriptions: Subscription[] }    // REQUIRED

// Integrations report agent state into a pane (strongest signal):
PaneReportAgentParams = { pane_id: string, source: string,   // REQUIRED pane_id, source, agent, state
                          agent: string, state: PaneAgentState,
                          agent_session_id?: string|null,
                          agent_session_path?: string|null,
                          message?: string|null, seq?: number|null }

6. Subscriptions & events

events.subscribe takes subscriptions: [{ "type": "<name>" }, ...].

Subscription types (26)

workspace.created  workspace.updated  workspace.metadata_updated
workspace.renamed  workspace.moved    workspace.closed  workspace.focused
worktree.created   worktree.opened    worktree.removed
tab.created        tab.closed         tab.focused       tab.renamed  tab.moved
pane.created       pane.closed        pane.updated      pane.focused pane.moved
pane.exited        pane.agent_detected  pane.output_matched
pane.agent_status_changed             pane.scroll_changed
layout.updated

EventData payloads (25 variants, discriminated by data.type)

Key ones for the MVP:

// Agent status transition — THE core state signal.
{ "type": "pane_agent_status_changed",
  "pane_id": "string", "workspace_id": "string",
  "agent": "string|null", "display_agent": "string|null",
  "agent_status": AgentStatus,          // idle|working|blocked|done|unknown
  "title": "string|null", "state_labels": { ... } }

// Agent identity acquired/released on a pane.
{ "type": "pane_agent_detected",
  "pane_id": "string", "workspace_id": "string",
  "agent": "string|null", "final_status": "AgentStatus|null",
  "released": boolean }

// Pane content/topology changed — carries full PaneInfo.
{ "type": "pane_updated",   "pane": PaneInfo }
{ "type": "pane_created",   "pane": PaneInfo }
{ "type": "pane_closed",    "pane_id": "string", "workspace_id": "string" }
{ "type": "pane_output_changed", "pane_id": "string", "revision": integer, "workspace_id": "string" }
{ "type": "pane_focused",   "pane_id": "string", "workspace_id": "string" }
{ "type": "pane_exited",    "pane_id": "string", "workspace_id": "string" }

// Workspace / tab lifecycle carry full info structs.
{ "type": "workspace_created|workspace_updated|workspace_metadata_updated", "workspace": WorkspaceInfo }
{ "type": "workspace_focused", "workspace_id": "string" }
{ "type": "tab_created", "tab": TabInfo }
{ "type": "tab_focused", "tab_id": "string", "workspace_id": "string" }
{ "type": "layout_updated", "layout": PaneLayoutSnapshot }

Note the naming: subscription types use dots (pane.agent_status_changed) while the emitted data.type uses underscores (pane_agent_status_changed).

⚠️ Subscription gotchas (verified live — cost hours, read this)

  1. All-or-nothing validation. If any entry in subscriptions is unknown or malformed, the server rejects the entire events.subscribe, replies with an invalid_request error, and closes the connection. One bad name kills the whole stream. Validate names against the exact list above.

  2. Three subscriptions are PER-PANE, not global. These require a pane_id field in the subscription object and error with missing field \pane_id`` if omitted:

    • pane.agent_status_changed
    • pane.output_matched
    • pane.scroll_changed To watch one pane's status live: { "type": "pane.agent_status_changed", "pane_id": "w4:p1" }. You cannot get these globally. Everything else in the list is global.

  3. No global output event. Sending text to a pane produces no global event — pane.updated does not fire on terminal output. There is no global "pane content changed" signal. Consequences:

    • Terminal mirroring must poll pane.read on a timer (or use a per-pane pane.output_matched subscription with matchers).
    • Continuous agent status (working↔idle↔blocked) is only pushed via the per-pane pane.agent_status_changed. For a global view, either open a per-pane subscription per detected agent, or poll session.snapshot periodically. pane.agent_detected (global) fires on acquire/release and carries final_status, but not intermediate transitions.

  4. Recommended global set (all validated OK): workspace.{created,updated, metadata_updated,renamed,moved,closed,focused}, worktree.{created,opened, removed}, tab.{created,closed,focused,renamed,moved}, pane.{created,closed, updated,focused,moved,exited,agent_detected}, layout.updated. pane.updated carries full PaneInfo (so structural + agent field changes land here), but remember it does not fire on raw output.

7. Info structs (event + snapshot payloads)

WorkspaceInfo = {
  workspace_id: string; label: string; number: number;
  active_tab_id: string; agent_status: AgentStatus;
  focused: boolean; tab_count: number; pane_count: number;
  tokens: object; worktree: WorkspaceWorktreeInfo | null;
}

TabInfo = {
  tab_id: string; workspace_id: string; label: string; number: number;
  agent_status: AgentStatus; focused: boolean; pane_count: number;
}

PaneInfo = {
  pane_id: string; tab_id: string; workspace_id: string;
  terminal_id: string; label: string | null;
  agent: string | null; display_agent: string | null;
  agent_status: AgentStatus; agent_session: AgentSessionInfo | null;
  cwd: string | null; foreground_cwd: string | null;
  focused: boolean; revision: number;
  title: string | null; terminal_title: string | null;
  terminal_title_stripped: string | null;
  scroll: PaneScrollInfo | null; state_labels: object; tokens: object;
}

AgentSessionInfo = { agent: string; kind: AgentSessionRefKind; source: string; value: string }

8. Session snapshot shape (session.snapshot / herdr api snapshot)

Single call returns the whole live tree — ideal for initial hydration before subscribing to deltas. Verified live shape:

{ "id": "...", "result": { "type": "session_snapshot", "snapshot": {
  "protocol": 17, "version": "0.7.5",
  "focused_workspace_id": "w4", "focused_tab_id": "w4:t1", "focused_pane_id": "w4:p1",
  "workspaces": [ { "workspace_id":"w4","label":"~","number":1,"active_tab_id":"w4:t1",
                    "agent_status":"unknown","focused":true,"tab_count":1,"pane_count":1 } ],
  "tabs":  [ { "tab_id":"w4:t1","workspace_id":"w4","label":"1","number":1,
               "agent_status":"unknown","focused":true,"pane_count":1 } ],
  "panes": [ { "pane_id":"w4:p1","tab_id":"w4:t1","workspace_id":"w4",
               "terminal_id":"term_65785a8110cd54","agent_status":"unknown",
               "cwd":"/Users/me","foreground_cwd":"/Users/me","focused":true,
               "revision":0, "scroll":{ "viewport_rows":29,
                 "offset_from_bottom":0,"max_offset_from_bottom":0 } } ],
  "agents": [],
  "layouts": [ { "workspace_id":"w4","tab_id":"w4:t1","focused_pane_id":"w4:p1",
                 "zoomed":false, "area":{"x":26,"y":1,"width":54,"height":23},
                 "panes":[ {"pane_id":"w4:p1","focused":true,
                            "rect":{"x":26,"y":1,"width":54,"height":23}} ],
                 "splits":[] } ]
} } }

ID conventions (observed)

  • Workspace id: w<N> (e.g. w4).
  • Tab id: <workspaceId>:t<N> (e.g. w4:t1).
  • Pane id: <workspaceId>:p<N> (e.g. w4:p1).
  • Terminal id: term_<hex>.

9. Minimal client recipe (NDJSON over unix socket)

Because RPC is one-request-per-connection, call() opens a fresh socket per request and resolves when the single response line arrives. subscribe() keeps a separate long-lived socket for the event stream.

import * as net from "node:net";
import * as os from "node:os";
import * as path from "node:path";

const SOCK = process.env.HERDR_SOCK
  ?? path.join(os.homedir(), ".config", "herdr", "herdr.sock");

// --- one-shot RPC: new connection per call, server closes after the response ---
let seq = 0;
function call(method: string, params: object = {}): Promise<any> {
  return new Promise((resolve, reject) => {
    const id = `c${++seq}`;
    const sock = net.connect(SOCK);
    let buf = "";
    sock.on("error", reject);
    sock.on("data", (chunk) => {
      buf += chunk.toString("utf8");
      const nl = buf.indexOf("\n");
      if (nl < 0) return;                       // one line is all we get
      const msg = JSON.parse(buf.slice(0, nl));
      sock.end();
      "error" in msg ? reject(msg.error) : resolve(msg.result);
    });
    sock.on("connect", () =>
      sock.write(JSON.stringify({ id, method, params }) + "\n"));
  });
}

// --- long-lived event stream: keep this socket open ---
function subscribe(types: string[], onEvent: (e: any) => void): net.Socket {
  const sock = net.connect(SOCK);
  let buf = "";
  sock.on("data", (chunk) => {
    buf += chunk.toString("utf8");
    let nl;
    while ((nl = buf.indexOf("\n")) >= 0) {
      const line = buf.slice(0, nl); buf = buf.slice(nl + 1);
      if (!line) continue;
      const msg = JSON.parse(line);
      if ("event" in msg) onEvent(msg);         // {event, data}
      // else: initial {id, result:{type:"subscription_started"}} ack — ignore
    }
  });
  sock.on("connect", () => sock.write(JSON.stringify({
    id: "sub", method: "events.subscribe",
    params: { subscriptions: types.map((t) => ({ type: t })) },
  }) + "\n"));
  return sock; // caller keeps ref; reconnect on 'close'
}

// --- usage: hydrate via snapshot, then stream deltas ---
const snap = await call("session.snapshot");       // full tree
const evSock = subscribe([
  // GLOBAL events only — pane.agent_status_changed is per-pane (see §6 gotchas)
  "pane.agent_detected", "pane.updated", "pane.created", "pane.closed",
  "workspace.created", "workspace.updated", "workspace.closed",
  "tab.created", "tab.closed", "workspace.focused",
], (evt) => { /* apply delta to the model, refresh trees */ });
// Continuous agent status + terminal output are NOT pushed globally:
// poll session.snapshot for status; poll pane.read for a terminal mirror.

10. Gotchas / notes for implementers

  • Response discrimination: check ("result" in msg) for success, ("error" in msg) for failure, ("event" in msg) for a pushed event. Do not rely on id alone — events omit it.
  • Subscription vs event naming: dots on subscribe, underscores in data.type.
  • done is receive-only — PaneAgentState (what you can report) has no done.
  • Pane ≠ agent. agent.start needs an existing pane_id. Create the pane (shell) first, then start the agent into it.
  • agent.start requires an available shell — the pane must be sitting at an interactive shell prompt (foreground process is the shell, e.g. zsh). A pane already running a command or another agent errors with pane <id> is not an available shell. Reliable flow: tab.create (its root_pane is a fresh shell), then agent.start into root_pane.pane_id. tab.create returns { tab, root_pane }; pane.split returns a new pane too.
  • Target field names are inconsistent — check per method. agent.* methods take { target } (accepts agent/pane id). But pane.focus, pane.get, and pane.zoom take { pane_id }; pane.read/pane.send_text take { pane_id }; workspace.focus/workspace.get take { target }, but workspace.close takes { workspace_id }; tab.focus takes a tab target. Passing target to a workspace_id/pane_id method errors missing field \...``.
  • workspace.create takes { cwd, label, env, focus } and returns { workspace, tab, root_pane } — so "add a folder as a workspace" is a folder picker → workspace.create({ cwd }), and you can agent.start into root_pane.pane_id for a workspace-plus-agent flow.
  • Prefer snapshot-then-subscribe. session.snapshot hydrates the full tree atomically; events keep it live. Avoids N round-trips of list calls.
  • Reconnect strategy: RPC connections close themselves after each response — that is normal, not an error. For the event stream, watch for an unexpected close, then reconnect, re-session.snapshot to re-hydrate, and re-events.subscribe (the server does not replay events missed while disconnected).
  • events.wait uses a different param shape than events.subscribe: it requires a match_event field (one-shot wait for a single matching event), not subscriptions. Omitting it errors: missing field \match_event``.
  • CLI mirrors the API. Every herdr <group> <sub> command is a thin wrapper over these socket methods (herdr workspace list ≈ workspace.list). Useful for manual verification: herdr api snapshot, herdr agent list, etc.
  • No absolute PTY resize via the socket API. pane.resize only takes { direction, amount } (nudges split ratios — verified no-op on a solo pane's grid); pane.zoom also does not change the grid. There is no method to set a pane's PTY cols/rows. The pane's PTY size is dictated by Herdr's attached rendering clients, not by API consumers.
  • Two ways to show a pane's terminal — know the trade-off:
    1. API mirror (pane.read poll → repaint): non-intrusive, no client attached, but locked to Herdr's current grid (can't grow past it; the cap is Herdr's own pane width). Good for a read-only peek. Read the real grid and render 1:1 (see recipe below).
    2. Real attach (herdr agent attach <pane_id|name> in a real terminal): runs as a rendering client, so its PTY sizes to and reflows with the host terminal (e.g. a VS Code integrated terminal). This is how you get a full-size, resizable, interactive single-pane terminal — the equivalent of "open herdr fullscreen" for one pane. Requires a real TTY (panics without one: failed to initialize terminal: Device not configured). --takeover seizes input authority from other clients. This shares the pane PTY with Herdr's other clients (resizing here resizes it for them too — inherent to Herdr's multi-client model). Downside: it is a full TUI that enables mouse tracking (\x1b[?1000h/1002h/1003h/1006h), so the host terminal's native text selection is captured by the app.
    3. Raw terminal streams — a separate CLI surface from the JSON-RPC API (herdr terminal ..., not *. socket methods):
      • herdr terminal session observe <terminal_id|pane_id> [--cols N] [--rows N] streams the pane's terminal as newline-delimited JSON frames {"bytes":"<base64>"} (base64 → raw terminal bytes). Works headless (pipeable, no TTY). Verified: the stream does NOT enable mouse tracking or alt-screen — so a consumer preserves native selection. It is read-only and does NOT resize the PTY: --cols/--rows is the observer's requested view, but the shell still reports the real size (stty size stayed 23 54 under observe --cols 132). Great for a live, selection-safe mirror; does not fix width.
      • herdr terminal session control <target> [--takeover] [--cols N] [--rows N] is the interactive counterpart and can drive the PTY size: under a held control --takeover --cols 132 --rows 40, the shell eventually reported 40 132. So a background control-sizer (held open at the desired size, output discarded) is the viable way to force a wider PTY while displaying via observe/mirror. Detaching reverts toward the other attached client's size (e.g. the Herdr TUI at 54).
      • terminal_id is on PaneInfo.terminal_id (e.g. term_<hex>). Interactive attach/control detach with ctrl+b q.
    • ★ THE WINNING RECIPE (single terminal: full width + reflow + selection + interactivity). Corrects the caveats above — control is the answer when consumed programmatically, not rendered raw:
      • Spawn herdr terminal session control <terminal_id> --takeover --cols W --rows H headless with piped stdio (it stays alive and streams — no TTY, no node-pty needed). Decode its {"bytes":base64} frames and write them into a host-drawn terminal (VS Code Pseudoterminal). Do not run it as a raw terminal process — its stdout is the JSON protocol and would render as literal {"bytes":...} text.
      • Width/reflow: control --cols/--rows genuinely resizes the pane PTY (verified via the API: PaneInfo.scroll.viewport_rows went 23→45 under --rows 45). Pass the host terminal's dimensions and respawn on resize (debounced) to reflow.
      • Selection: the decoded control/observe stream contains no mouse-tracking escapes even for mouse-using agents (Herdr handles mouse in the protocol, not the content), so the host terminal never enters mouse mode → native selection works. (The earlier "control has no mouse" reading of the raw stream was a false positive: the escapes were base64-hidden. agent attach/terminal attach, which render raw, DO pass mouse tracking through → they break selection.)
      • Input/interactivity: deliver keystrokes over the socket with pane.send_text — verified to reach the shell/agent even while a control --takeover client holds authority. (control's own stdin expects a typed JSON command {"type":...}, not raw bytes — using the socket is simpler and works.)
    • Pane grid dimensions: PaneInfo.scroll.viewport_rows gives rows; the layout rect.width/rect.height give cols/rows; or measure the visible read (rows = line count, cols = max line width after stripping ANSI). All three agreed at 54×23 in testing.
    • Terminal-mirror recipe (VS Code specifics, but the escapes are generic): enter the alternate screen (\x1b[?1049h) and disable auto-wrap (\x1b[?7l) on open; repaint with \x1b[H + frame + \x1b[J; lock the renderer to the measured grid (VS Code: onDidOverrideDimensions). Alt-screen kills scrollback so scrolling can't reveal stale repainted frames; the locked grid + no-wrap makes a mismatched panel letterbox/clip cleanly instead of garbling. Restore with \x1b[?7h\x1b[?1049l on close.
  • Regenerate on protocol bump: herdr api schema --json + herdr api snapshot.

## 11. Capture commands (reproduce this doc)

```bash
herdr status                          # socket path + protocol version
herdr api schema --json > schema.json # full 248KB request/event/response schema
herdr api snapshot                    # live session tree
# framing probe: send {"id":"x","method":"ping","params":{}}\n to the socket