docs(agent-sdk): add tool-call example backed by a NATS microservice#134
Conversation
Add `03-tools.ts` — step 3 of the example ladder. The agent gains a `read_sensor` tool whose handler is a single `nc.request(...)` to a NATS microservice; the demo registers that microservice (a faked temperature sensor) in the same file so it runs standalone. Shows the talk's core point: any microservice already on your NATS network can become an agent capability, without the agent embedding the device/db/credential behind it. Uses Ollama's `/api/chat` tool-calling in the canonical two round-trips: round 1 (non-streamed) for the tool decision, round 2 (streamed) for the final answer — reusing the token-streaming shape from `02-ollama.ts`. Default model `llama3.1:8b` (tool-capable), overridable via `OLLAMA_MODEL`. Verified end-to-end against a local nats-server + Ollama: both `agents` and `sensors` services register on the bus, and a prompt drives the full tool-call → NATS request → streamed verdict path. README updated with the new row and a run section. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
Claude finished @M64GitHub's task in 2m 44s —— View job PR Review:
|
| Severity | File | Issue |
|---|---|---|
| Medium | line 142 | JSON.parse in chatStream without a try/catch — malformed Ollama output throws SyntaxError through the generator, resulting in a 500 error frame to the caller instead of a graceful partial response |
| Low | line 107 | arguments typed as Record<string, unknown> but some Ollama versions (and several quantised llama variants) return it as a JSON-encoded string — runTool silently falls back to location = "" producing "no sensor at ''" with no diagnostic |
| Low | line 180 | Tool response message lacks name: call.function.name — Ollama uses this field (not a tool_call_id) to attribute results; a second tool added later would produce ambiguous attribution |
No protocol-compliance or security issues found. The NATS subject layout is correct (agents.prompt.tools.<owner>.<name>), the SDK's onPrompt / response.send pattern is used correctly, the sensor subject lookup is safe against injection (dict key lookup with an "unknown" fallback), and the two-round-trip tool-calling flow is a correct implementation of the Ollama /api/chat tool API.
The README additions are accurate and the nats micro ls tip is a useful touch for conference-demo use.
| buffer = lines.pop() ?? ""; | ||
| for (const line of lines) { | ||
| if (line.trim() === "") continue; | ||
| const token = (JSON.parse(line) as { message?: { content?: string } }).message?.content ?? ""; |
There was a problem hiding this comment.
JSON.parse without error handling. If Ollama sends a non-JSON line (e.g., an error message during model loading or a truncated stream), this throws SyntaxError, propagates out of the generator, and causes the AgentService to emit a 500 error frame to the caller rather than a graceful partial response. 02-ollama.ts has the same gap, but it's worth fixing here as an incremental improvement:
| const token = (JSON.parse(line) as { message?: { content?: string } }).message?.content ?? ""; | |
| try { | |
| const token = (JSON.parse(line) as { message?: { content?: string } }).message?.content ?? ""; | |
| if (token) yield token; | |
| } catch { | |
| // ignore non-JSON lines (Ollama sometimes emits plain-text errors) | |
| } |
There was a problem hiding this comment.
Fixed in dac36d9 — wrapped the streaming JSON.parse in try/catch so a rare non-JSON line is skipped instead of throwing out of the generator (which would surface as a 500 frame to the caller).
| interface ChatMessage { | ||
| role: string; | ||
| content: string; | ||
| tool_calls?: { function: { name: string; arguments: Record<string, unknown> } }[]; |
There was a problem hiding this comment.
arguments may arrive as a JSON string for some models. The Ollama /api/chat response is typed here as Record<string, unknown>, which is correct for llama3.1:8b. However, some Ollama versions and several other models (including some quantised llama variants) return arguments as a JSON-encoded string rather than a parsed object. runTool checks typeof args["location"] === "string" which silently falls back to location = "" in that case, producing a confusing "no sensor at ''" from the model. A defensive parse guard at the call site would prevent the silent misbehaviour:
| tool_calls?: { function: { name: string; arguments: Record<string, unknown> } }[]; | |
| tool_calls?: { function: { name: string; arguments: Record<string, unknown> | string } }[]; |
And then in runTool:
const rawArgs = typeof args === "string" ? (JSON.parse(args) as Record<string, unknown>) : args;
const location = typeof rawArgs["location"] === "string" ? rawArgs["location"] : "";There was a problem hiding this comment.
Fixed in dac36d9 — runTool now accepts arguments as a parsed object or a JSON string (typeof args === "string" ? JSON.parse(args) : args), with the tool_calls type widened to Record<string, unknown> | string. Good call; this makes the example work across more models, which matters since it's meant to be run with OLLAMA_MODEL overrides. Verified end-to-end against a live Ollama afterward.
| // loop until the model stops requesting tools.) | ||
| for (const call of decision.tool_calls ?? []) { | ||
| const result = await runTool(nc, call.function.name, call.function.arguments); | ||
| messages.push({ role: "tool", content: result }); |
There was a problem hiding this comment.
Missing name in tool response message. The Ollama /api/chat protocol identifies which tool produced a result via a name field on the tool-role message (no tool_call_id — Ollama doesn't assign call IDs). Without it the example works for a single tool, but adding a second tool later would produce ambiguous attribution and confuse the model:
| messages.push({ role: "tool", content: result }); | |
| messages.push({ role: "tool", content: result, name: call.function.name } as ChatMessage); |
You'll need to add name?: string to the ChatMessage interface at line 105 to make this typecheck.
There was a problem hiding this comment.
Deliberately leaving this one out. This example has a single tool, and result attribution only matters once there are two or more — adding a field for a case the example doesn't demonstrate cuts against its 'minimal / as-simple-as-possible' goal (it's conference-teaching code). Also worth noting: the suggested name is the OpenAI-compat field; this code talks to Ollama's native /api/chat, where the tool-result attribution field is tool_name. So the multi-tool version of this (planned for a separate standalone repo) will add tool_name then, when it actually has multiple tools to disambiguate. Verified the single-tool flow works correctly without it.
Expand the tool-call example's README: state the core point as a callout, explain the two-round tool-calling flow, note model choice (llama3.1:8b default; gemma can't tool-call, qwen3 emits <think>), and add the sensor lookup table so readers can pick an in-range vs out-of-range location. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- Wrap the streaming JSON.parse in try/catch so a rare non-JSON line from Ollama (e.g. an error emitted mid-stream) is skipped rather than crashing the reply with a 500 frame. - Accept tool-call `arguments` as a parsed object OR a JSON string — some models / Ollama versions return the latter; previously that silently fell back to an empty location. Verified end-to-end against local nats-server + Ollama: both the in-range (room 1 → "yes") and out-of-range (room 3 → "no, 6.2°C") cases stream correct answers. Left the suggested tool-result `name` field out on purpose: this example has a single tool, attribution only matters with two or more, and the suggested `name` is the OpenAI-compat field rather than Ollama's native `tool_name`. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
What
Adds
agent-sdk/typescript/examples/03-tools.ts— step 3 of the agent-sdk example ladder (01-echo→02-ollama→03-tools).The agent gains a
read_sensortool, and that tool is wired to a NATS microservice. The point of the example:The agent process holds only an LLM + a NATS connection. When the model needs live data it calls the tool; the tool's entire handler is one
nc.request(...)to the microservice. For a self-contained single-file demo, the microservice (a faked temperature sensor) is registered in the same file — in production it runs anywhere on the network and the agent never knows where.How it works
Ollama
/api/chattool-calling in the canonical two round-trips:read_sensor(location).02-ollama.ts.Default model
llama3.1:8b(tool-capable), overridable viaOLLAMA_MODEL; host viaOLLAMA_URL.Verification
Full end-to-end run against a local
nats-server+ Ollama:nats micro lsshows both services on the bus —agents(the agent) andsensors(the microservice it calls).ack→ tool decision → NATS request to the sensor → streamed verdict ("No, the cold storage room 3 is not within the safe range…").typecheck,lint,format:checkall clean; both SDKs build.README updated with the new row and a run section (including a
nats micro lstip).Context
Conference-talk material (reThinkConn) demonstrating the Synadia Agent Protocol in three small steps. This is the final step — giving an agent access to existing NATS microservices as tools.
🤖 Generated with Claude Code