perf: ArgentSessionManager + 113-tool registry rebuilt every turn — cache across turns

[webdevtodayjason]

Summary

Every agent turn rebuilds the entire session pipeline from scratch:

[argent-runtime] Using ArgentSessionManager for session
[argent-runtime] Using ArgentSettingsManager
[argent-runtime] Using createArgentAgentSession (tools=113)

Combined with the 113-tool registry construction, this accounts for ~5.5 seconds of pre-I/O wall-clock per turn (gateway tony-stark phase async_io_started=5509ms). Within a single ongoing conversation, none of this should change between turns — the same session, same settings, same tools.

Repro

Run two consecutive turns in the Argent TUI. Tail ~/.argent/logs/gateway.log for the [tony-stark] phase lines. Observe that async_io_started, tools_created, pre_session, session_created all hover around 5.5s on both turns even though the conversation, settings, and tool list have not changed.

Proposed fix

Cache the session-pipeline state keyed by (sessionId, agentId, settingsRevision, toolRegistryHash):

• First turn of a session: build everything as today (~5.5s).
• Subsequent turns: reuse the cached session manager, settings manager, agent session, and tool registry. Expected pre-I/O time: <100ms.
• Cache invalidation: bump on settings change, tool registry change (e.g. operator enables a new toolkit), or explicit session reset.

Acceptance criteria

• tony-stark telemetry shows async_io_started < 200ms on the 2nd+ turn of a session under no-change conditions.
• First turn is unchanged or improved (cache miss is fine).
• Cache invalidates correctly when settings, tools, or sessionId change. Add unit tests for each invalidation path.
• No regression: changing the model profile mid-conversation does the right thing.

Related

• perf: chat turn 4x slower than equivalent subctl turn (13.5s vs 3s, same OAuth provider + tool use) #405 — the broader 4x slowness this is one root cause of.
• perf: 113-tool JSON-schema block (~17k tokens) sent on every turn regardless of intent #407 — full 113-tool spec block sent on every turn (separate but related).

🤖 Filed from a live perf investigation.

[webdevtodayjason]

Measurement before building the cache (2026-06-04) — the premise needs revisiting

Pulled real [tony-stark] phase data from the gateway log: 2,448 actual operator turns (tools=107–119, openai-codex, idle/resumed outliers with async_io_started>10s excluded). Phases are cumulative from t0 (phaseTimes[name]=Date.now()-t0).

The 113-tool registry build is not the bottleneck.

Median full pre-LLM pipeline (session_created)	388ms
Turns with pipeline ≥ 2s	552 / 2448 (23%)
total is dominated by	prompt_done = the LLM call (13s–136s) — not cacheable

Decomposition of the slow tail (pipeline ≥ 2s), by phase:

Phase	Share of slow-pipeline time	Cacheable?
async_io_started (pre-tool: skills / room-reader / memory inject)	~30%	No (not the session pipeline)
tool build (async_io_started→tools_created)	~3% (~306ms)	yes — but tiny
sessionMgr + settings + split (tools_created→pre_session)	~41%	session-manager I/O dominates — mutable, must stay fresh
createArgentAgentSession (pre_session→session_created)	~26%	partly — but see timeout below

Two things the cache (as filed) would not fix:

1. A ~30s timeout, not a rebuild. 51 of the 552 slow turns (9%) show createArgentAgentSession clustered at ~30,000ms ±200ms — a timeout signature, not construction cost. That's a separate bug (something blocking in session creation), and a cache would mask it, not fix it. Worth its own issue.
2. The biggest cacheable chunk is session-manager I/O (ArgentSessionManager.open() = _loadFromFile(), scales with session-file size — cf. the 98MB sessions.json + lock-timeout history from the May perf sprint, perf(sessions): dedup skillsSnapshot to sidecar + auto-prune stale workflows (closes #410) #412). That is exactly the mutable, transcript-carrying piece that must be re-read fresh each turn — caching it is the correctness risk we set out to avoid.

Conclusion: the tool-registry rebuild this issue targets is ~3% of the cost. A session-pipeline cache scoped safely (excluding the session manager) would skip mostly-cheap work and miss the real bottleneck. The real levers are (a) the ~30s createArgentAgentSession timeout and (b) session-file I/O size/locking — both different fixes than the cache.

A reusable, fully-tested SessionPipelineCache was built as slice 1 (branch perf/session-pipeline-cache-406) before this measurement; holding it unwired pending a decision, since the data says wiring it would not deliver the speedup. Recommend repurposing #406 toward the session-creation timeout + session-file I/O, or closing in favor of those.

Honest note: this reverses the "wire the cache" plan — the measurement (which the issue rightly implies but I verified) shows the cache isn't the right tool here.

[webdevtodayjason]

Status update (2026-06-10): #405's root cause was fixed and merged (#436 — personal-skill block: writes fire-and-forget, reads cached per-agent + parallelized; measured 2→0 critical-path backend reads on warm turns). The 2026-06-04 measurement of 2,448 operator turns attributed ~3% of slow-turn cost to the session/tool-registry rebuild this issue targets, so the SessionPipelineCache from the slice-1 branch remains deliberately unwired. Leaving open as a low-priority candidate pending fresh measurements on post-#436 turns.