Skip to content

docs(agent-sdk): add tool-call example backed by a NATS microservice#134

Merged
M64GitHub merged 3 commits into
mainfrom
examples/tools-agent
May 30, 2026
Merged

docs(agent-sdk): add tool-call example backed by a NATS microservice#134
M64GitHub merged 3 commits into
mainfrom
examples/tools-agent

Conversation

@M64GitHub
Copy link
Copy Markdown
Contributor

What

Adds agent-sdk/typescript/examples/03-tools.tsstep 3 of the agent-sdk example ladder (01-echo02-ollama03-tools).

The agent gains a read_sensor tool, and that tool is wired to a NATS microservice. The point of the example:

any microservice already on your NATS network can become an agent capability — without the agent embedding the device, database, or credential behind it.

The agent process holds only an LLM + a NATS connection. When the model needs live data it calls the tool; the tool's entire handler is one nc.request(...) to the microservice. For a self-contained single-file demo, the microservice (a faked temperature sensor) is registered in the same file — in production it runs anywhere on the network and the agent never knows where.

// the tool handler — this one line is the whole lesson:
const reply = await nc.request(SENSOR_SUBJECT, location, { timeout: 5000 });

How it works

Ollama /api/chat tool-calling in the canonical two round-trips:

  1. Round 1 (non-streamed) — send the prompt + tool schema; the model asks to call read_sensor(location).
  2. Run the tool (the NATS request), feed the result back.
  3. Round 2 (streamed) — the model streams its final answer, reusing the exact token-streaming shape from 02-ollama.ts.

Default model llama3.1:8b (tool-capable), overridable via OLLAMA_MODEL; host via OLLAMA_URL.

Verification

Full end-to-end run against a local nats-server + Ollama:

  • nats micro ls shows both services on the bus — agents (the agent) and sensors (the microservice it calls).
  • A real prompt drove the complete path: ack → tool decision → NATS request to the sensor → streamed verdict ("No, the cold storage room 3 is not within the safe range…").
  • typecheck, lint, format:check all clean; both SDKs build.

README updated with the new row and a run section (including a nats micro ls tip).

Context

Conference-talk material (reThinkConn) demonstrating the Synadia Agent Protocol in three small steps. This is the final step — giving an agent access to existing NATS microservices as tools.

🤖 Generated with Claude Code

Add `03-tools.ts` — step 3 of the example ladder. The agent gains a
`read_sensor` tool whose handler is a single `nc.request(...)` to a NATS
microservice; the demo registers that microservice (a faked temperature
sensor) in the same file so it runs standalone. Shows the talk's core point:
any microservice already on your NATS network can become an agent capability,
without the agent embedding the device/db/credential behind it.

Uses Ollama's `/api/chat` tool-calling in the canonical two round-trips:
round 1 (non-streamed) for the tool decision, round 2 (streamed) for the
final answer — reusing the token-streaming shape from `02-ollama.ts`. Default
model `llama3.1:8b` (tool-capable), overridable via `OLLAMA_MODEL`.

Verified end-to-end against a local nats-server + Ollama: both `agents` and
`sensors` services register on the bus, and a prompt drives the full
tool-call → NATS request → streamed verdict path.

README updated with the new row and a run section.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@synadia-claude-reviewer
Copy link
Copy Markdown

synadia-claude-reviewer Bot commented May 30, 2026

Claude finished @M64GitHub's task in 2m 44s —— View job


PR Review: docs(agent-sdk): add tool-call example backed by a NATS microservice

  • Gather context
  • Read the diff and new files
  • Review for correctness, security, and protocol compliance
  • Post inline findings

The example is well-structured and clearly demonstrates the intended concept. Three issues worth addressing before merge, all in 03-tools.ts:

Severity File Issue
Medium line 142 JSON.parse in chatStream without a try/catch — malformed Ollama output throws SyntaxError through the generator, resulting in a 500 error frame to the caller instead of a graceful partial response
Low line 107 arguments typed as Record<string, unknown> but some Ollama versions (and several quantised llama variants) return it as a JSON-encoded string — runTool silently falls back to location = "" producing "no sensor at ''" with no diagnostic
Low line 180 Tool response message lacks name: call.function.name — Ollama uses this field (not a tool_call_id) to attribute results; a second tool added later would produce ambiguous attribution

No protocol-compliance or security issues found. The NATS subject layout is correct (agents.prompt.tools.<owner>.<name>), the SDK's onPrompt / response.send pattern is used correctly, the sensor subject lookup is safe against injection (dict key lookup with an "unknown" fallback), and the two-round-trip tool-calling flow is a correct implementation of the Ollama /api/chat tool API.

The README additions are accurate and the nats micro ls tip is a useful touch for conference-demo use.

buffer = lines.pop() ?? "";
for (const line of lines) {
if (line.trim() === "") continue;
const token = (JSON.parse(line) as { message?: { content?: string } }).message?.content ?? "";
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

JSON.parse without error handling. If Ollama sends a non-JSON line (e.g., an error message during model loading or a truncated stream), this throws SyntaxError, propagates out of the generator, and causes the AgentService to emit a 500 error frame to the caller rather than a graceful partial response. 02-ollama.ts has the same gap, but it's worth fixing here as an incremental improvement:

Suggested change
const token = (JSON.parse(line) as { message?: { content?: string } }).message?.content ?? "";
try {
const token = (JSON.parse(line) as { message?: { content?: string } }).message?.content ?? "";
if (token) yield token;
} catch {
// ignore non-JSON lines (Ollama sometimes emits plain-text errors)
}

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in dac36d9 — wrapped the streaming JSON.parse in try/catch so a rare non-JSON line is skipped instead of throwing out of the generator (which would surface as a 500 frame to the caller).

interface ChatMessage {
role: string;
content: string;
tool_calls?: { function: { name: string; arguments: Record<string, unknown> } }[];
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

arguments may arrive as a JSON string for some models. The Ollama /api/chat response is typed here as Record<string, unknown>, which is correct for llama3.1:8b. However, some Ollama versions and several other models (including some quantised llama variants) return arguments as a JSON-encoded string rather than a parsed object. runTool checks typeof args["location"] === "string" which silently falls back to location = "" in that case, producing a confusing "no sensor at ''" from the model. A defensive parse guard at the call site would prevent the silent misbehaviour:

Suggested change
tool_calls?: { function: { name: string; arguments: Record<string, unknown> } }[];
tool_calls?: { function: { name: string; arguments: Record<string, unknown> | string } }[];

And then in runTool:

const rawArgs = typeof args === "string" ? (JSON.parse(args) as Record<string, unknown>) : args;
const location = typeof rawArgs["location"] === "string" ? rawArgs["location"] : "";

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in dac36d9runTool now accepts arguments as a parsed object or a JSON string (typeof args === "string" ? JSON.parse(args) : args), with the tool_calls type widened to Record<string, unknown> | string. Good call; this makes the example work across more models, which matters since it's meant to be run with OLLAMA_MODEL overrides. Verified end-to-end against a live Ollama afterward.

// loop until the model stops requesting tools.)
for (const call of decision.tool_calls ?? []) {
const result = await runTool(nc, call.function.name, call.function.arguments);
messages.push({ role: "tool", content: result });
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Missing name in tool response message. The Ollama /api/chat protocol identifies which tool produced a result via a name field on the tool-role message (no tool_call_id — Ollama doesn't assign call IDs). Without it the example works for a single tool, but adding a second tool later would produce ambiguous attribution and confuse the model:

Suggested change
messages.push({ role: "tool", content: result });
messages.push({ role: "tool", content: result, name: call.function.name } as ChatMessage);

You'll need to add name?: string to the ChatMessage interface at line 105 to make this typecheck.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Deliberately leaving this one out. This example has a single tool, and result attribution only matters once there are two or more — adding a field for a case the example doesn't demonstrate cuts against its 'minimal / as-simple-as-possible' goal (it's conference-teaching code). Also worth noting: the suggested name is the OpenAI-compat field; this code talks to Ollama's native /api/chat, where the tool-result attribution field is tool_name. So the multi-tool version of this (planned for a separate standalone repo) will add tool_name then, when it actually has multiple tools to disambiguate. Verified the single-tool flow works correctly without it.

M64GitHub and others added 2 commits May 30, 2026 03:32
Expand the tool-call example's README: state the core point as a callout,
explain the two-round tool-calling flow, note model choice (llama3.1:8b
default; gemma can't tool-call, qwen3 emits <think>), and add the sensor
lookup table so readers can pick an in-range vs out-of-range location.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- Wrap the streaming JSON.parse in try/catch so a rare non-JSON line from
  Ollama (e.g. an error emitted mid-stream) is skipped rather than crashing
  the reply with a 500 frame.
- Accept tool-call `arguments` as a parsed object OR a JSON string — some
  models / Ollama versions return the latter; previously that silently fell
  back to an empty location.

Verified end-to-end against local nats-server + Ollama: both the in-range
(room 1 → "yes") and out-of-range (room 3 → "no, 6.2°C") cases stream correct
answers.

Left the suggested tool-result `name` field out on purpose: this example has
a single tool, attribution only matters with two or more, and the suggested
`name` is the OpenAI-compat field rather than Ollama's native `tool_name`.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@M64GitHub M64GitHub merged commit 8278333 into main May 30, 2026
20 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant