Skip to content

feat: optional tool-use loop helper (generateResponseWithTools) #28

@stackbilt-admin

Description

@stackbilt-admin

Summary

Every consumer of llm-providers that does tool-use manually implements the loop: `generateResponse → parse toolCalls → execute tools → append results → generateResponse` until no more tool calls. llm-providers already ships `validateToolCalls()` for sanitization but stops short of owning the loop. An optional helper would eliminate duplicated logic across consumers.

Motivation

AEGIS daemon has at least 2 copies of this loop (`executeGroqWithTools`, `executeCerebrasWithTools`). Future consumers (foodfiles, img-forge, stackbilt-web) will reinvent variations with subtle bugs — missing iteration caps, inconsistent cost accumulation, no abort-signal handling, divergent error propagation. Bolted-in tool-loop logic is the same failure class as bolted-in LLM routing: divergent copies of logic that should be canonical.

The whole point of llm-providers is to be the single source of truth for LLM orchestration. Tool-use loops are orchestration. They belong here.

Proposed API (optional — does not replace `generateResponse`)

```ts
interface ToolExecutor {
execute(name: string, arguments: unknown): Promise;
}

interface ToolLoopOptions {
maxIterations?: number; // default 10 — hard cap to prevent runaway
maxCostUSD?: number; // per-loop budget cap; abort if exceeded
onIteration?: (iter: number, state: ToolLoopState) => void | Promise;
abortSignal?: AbortSignal;
}

interface ToolLoopState {
iteration: number;
cumulativeCost: number;
messageCount: number;
lastToolCalls: ToolCall[];
}

class LLMProviderFactory {
async generateResponseWithTools(
request: LLMRequest,
toolExecutor: ToolExecutor,
opts?: ToolLoopOptions
): Promise; // final response after loop terminates
}
```

Loop semantics:

  1. Call `generateResponse(request)`
  2. If `toolCalls` empty → return as final response
  3. Else → run each tool call through `toolExecutor.execute()`, append tool results to messages, increment iteration
  4. Check: iteration cap, cost cap, abort signal — abort with a distinct error class if any hit
  5. Goto 1 with updated messages

Benefits

  • Eliminates duplicated loop logic across consumers (daemon has 2+ copies; any future consumer will have its own)
  • Consistent max-iteration enforcement — prevents runaway tool loops (a real failure mode when LLMs get stuck calling the same tool repeatedly)
  • Consistent per-loop cost caps — tool loops can rack up spend invisibly; capping is safety-critical
  • Better observability hooks — per-iteration telemetry via `onIteration`
  • Works with existing fallback chain — each LLM call inside the loop goes through the same provider selection, so tool loops inherit fallback automatically

Non-goals

  • NOT a replacement for raw `generateResponse()` — consumers can still drive the loop manually when they need custom per-iteration logic (memory injection, dynamic tool list, etc.)
  • NOT opinionated about tool implementation — the `ToolExecutor` interface is deliberately minimal. Consumers wire their own MCP clients, function registries, or hand-coded dispatchers.
  • NOT a tool schema validator — llm-providers already has `validateToolCalls()` for that.

Priority

MEDIUM — QoL improvement, not a migration blocker. Reduces the "LLM logic" surface area in consumers, which is the architectural point of the package.

Related

  • Will be consumed by AEGIS daemon Phase D migration as a drop-in replacement for `executeGroqWithTools` / `executeCerebrasWithTools`
  • Companion architectural rule: AEGIS `feedback_no_bolted_llm_logic.md` — no bolted-in LLM orchestration anywhere in the ecosystem

🤖 Filed by AEGIS during Phase D scoping session

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions