perf: 113-tool JSON-schema block (~17k tokens) sent on every turn regardless of intent

[webdevtodayjason]

Summary

Every agent turn currently ships the full 113-tool registry in the system prompt as JSON-schema, regardless of whether the operator's prompt is "what are the logs at?" (zero tools needed) or "delete file X" (one filesystem tool needed). At ~150 tokens of spec per tool, that's ~17,000 tokens of tool definitions on every single turn before the operator's question is even processed.

For openai-codex (gpt-5.3-codex, gpt-5.4-mini), that adds noticeable upload time, server-side parse time, and per-turn dollar cost. For local models on Ollama, it slows context build by hundreds of milliseconds and pushes context window usage.

Evidence

• [tony-stark] telemetry: tools=113 on every turn
• subctl, doing a comparable OAuth + 1-tool turn, completes in ~3s — likely because it registers a much smaller set of tools per session.
• For Argent: model time was 7.7s on a turn where only memory_recall + memory_timeline were used. The other 111 tool specs were paid for in tokens and time but never used.

Proposed fix (in order of complexity)

1. Toolkit gating (cheapest): per-agent operator config picks which toolkits to register. Operator who never uses Composio's GitHub tools can disable that toolkit and shave its specs out. Some scaffolding for this already exists per preferComposio work — extend it to all toolkits.
2. Intent classification (medium): pre-classify the operator's prompt with a small local model into intent buckets (filesystem / memory / web / scheduling / chat-only / etc.) and only inject the relevant toolkit's specs. Bucket "chat-only" gets zero tools.
3. Recency-weighted retrieval (advanced): rank tools by recent usage within this session + similarity to the prompt, and inject only the top-K.

Option 1 + a sane default-disabled set for most toolkits would likely cut the typical turn from 113 tools to 10-20 tools (-85% prompt block). Option 2 + 1 together would get most turns to <5 tools.

Acceptance criteria

• Operator can configure which toolkits to load per agent.
• Default tool count per turn for a chat-only operator (no filesystem / no memory) drops to <10.
• [tony-stark] tools=N telemetry reflects the per-turn injected count, not the registry size.
• No regression: turns that need a deselected toolkit get a graceful "this tool isn't available, enable X to use it" agent response, not a silent failure.

Related

• perf: chat turn 4x slower than equivalent subctl turn (13.5s vs 3s, same OAuth provider + tool use) #405 — broader perf gap to subctl.
• perf: ArgentSessionManager + 113-tool registry rebuilt every turn — cache across turns #406 — session/registry rebuild per turn (separate but compounds with this).

🤖 Filed from a live perf investigation.

[webdevtodayjason]

Fresh ground truth from the 2026-06-11 local-model investigation (#442):

• Native tools ARE passed for all openai-completions providers (verified in pi source) — the prompt mass is Argent's own system prompt + LM Studio's template rendering of 119 tool schemas: ~43k tokens on a default chat turn (LM Studio rejected at n_ctx 4096: n_keep 43019).
• Even a 2-tool worker turn carries enough preamble that 12B–35B local models lose tool-call format discipline (measured: gemma-4-12b-QAT + qwen3.6-35b-a3b both narrate/pseudo-call under the worker prompt, but call tools correctly on plain turns — PR fix(worker): textual pseudo-tool-call detection + format correction (#442) #444 added detection/correction, insufficient alone).
• Concrete acceptance for this issue when picked up: a worker-profile turn ships a prompt small enough that gemma-4-12b-QAT completes the 6-ticket conductor demo (scripts/demo/conductor-demo.mjs) — that's the mom-and-pop zero-API-bill deployment target.

[webdevtodayjason]

Design principle from Jason (2026-06-11) — this reframes lever 1 from optimization to correctness:

Workers are blank slates. A worker's prompt contains exactly four things: (1) company alignment, (2) the role contract (template: role prompt / SOP / success definition / boundaries), (3) explicitly granted tools, (4) explicitly granted knowledge (per-worker RAG via knowledge ACL). Everything else is default-deny — including Argent's identity stack (SOUL/IDENTITY/moods), channel instructions, skills preamble, and the operator's memory context.

Current reality: all agent runs share one prompt pipeline that layers the full Argent runtime scaffold regardless of role — there is no enforced worker-isolation profile. Beyond the ~40k token cost, this is a governance leak: operator-personal memory context rides into worker runs that may face client customers.

What already holds: per-template toolsAllow enforcement (verified: 2-tool worker turns), simulate-mode external-tool blocking, knowledge ACL primitives.

Revised acceptance for this issue: (a) worker-profile runs contain NO operator memory/identity/mood/skills preamble (assertable by prompt inspection), (b) gemma-4-12b-QAT completes the 6-ticket conductor demo locally, (c) chat-path selective tool injection remains the second lever for the main agent.

Also: the conductor demo script must stop defaulting --assign to 'main' — assigning jobs to Argent itself guarantees inheritance and muddied last night's measurements.

[webdevtodayjason]

Archaeology (2026-06-11): worker isolation was partially built and then suppressed. Legacy commit 32f9956aff (2026-03-21, 'stabilize chat surface and monitor controls') unmounted the operator's original WorkerWizard (import + state + onNewWorkerClick entry + pane); the files survived orphaned, were extracted to the Business repo 2026-05-05 still unmounted, returned in #435, and were re-mounted in #440. Creation-side blank-slate setup (identity/role/tools/intent) therefore already exists and is the operator's original design — see ArgentOS-Business docs/internal/archive/WORKER_WIZARD_PLAN.md. This issue's scope is the never-wired RUNTIME half: enforcing the blank-slate prompt at execution time.