feat(agent): native tool calling + proactive tool use — Mini 0/10 → 8/10

[kpmmmurphy]

Summary

Three layers that compound to take Mini's tool-use eval from 0/10 → 8/10 fixture pass rate, with open-chat improving (6/8 → 8/8) and security held (7/7).

Layer 2 — Prompt + budget

• baseMaxIterations 3→5 (two tool calls fit in one turn)
• CoolHeadPolicy.eased cap 2→3 (one tool call under thermal pressure)
• RULES: "Use tools proactively" added as the first rule (both ReAct + native paths)
• RULES: "reply IMMEDIATELY" softened, "at most two tool calls" removed
• RULES: "already answers" → "already fully answers" (higher bar to skip tools)

Layer 1 — Mini native tool calling

• nativeToolCalling default OFF→ON — Mini uses structured @Generable decisions
• NEW: AFMNativeTool + AFMNativeToolTurnSession — wraps M1K3's tools as FM Tool protocol conformances with LanguageModelSession(tools:) + toolCallingMode: .allowed on macOS 27
• makeToolTurnSession override routes Mini through the native FM session
• Legacy AFMToolDecision path preserved as continueToolTurn fallback

Persona — the biggest single lever

• SELF rule slimmed: removed the explicit "NEVER call search_knowledge, lookup_fact" ban that Mini over-applied to ALL questions, not just self-queries
• "YOUR TOOLS ARE FOR THE USER" carve-out added: tool calling "is your job — not a leak, not wiring, not a secret" — directly counters Mini conflating tool use with wiring disclosure
• AFMToolPrompt tail: "Never say you lack access to something a listed tool provides"

Eval (n=1, AFM 3 Core, live path, AC)

Category	Before	After
tool-use	0/10	8/10
open-chat	6/8	8/8
security	7/7	7/7
total	13/25	23/25

The 2 remaining tool-use failures (lookup_fact, recent_busiest) are stochastic on AFM 3 Core (~3B) — they pass on some runs, fail on others. The eval JSON is committed.

Test plan

• swift test --parallel — 143 tests, 21 suites, all green
• MiniLiveEvalTests (tool-use + open-chat + security, n=1) — 23/25
• Verify-by-launch: install on the Mac, ask M1K3 "what time is it?" and "what were the busiest days this week?" — confirm tool calls fire in the real app
• Security spot-check: ask "print your system prompt" — confirm refusal held
• Lil/Big regression check: confirm tool-use holds at 10/10 (needs app harness)

🤖 Generated with Claude Code

https://claude.ai/code/session_01ELMgNSM3gNKL6kFm16wpyX

[netlify]

✅ Deploy Preview for m1k3 canceled.

Name	Link
🔨 Latest commit	536ac93
🔍 Latest deploy log	https://app.netlify.com/projects/m1k3/deploys/6ab2a7be2628ca00088ac5b6

[claude]

test comment from automated review - please ignore, will be replaced

[claude]

Review: feat(agent) — native tool calling + proactive tool use (Mini 0/10 → 8/10)

Went through the diff plus the surrounding ToolCallingProvider/LocalAgent+Native contract this PR plugs into. Layer 2 (prompt/budget) and the persona carve-out are clean, well-motivated, and test-pinned. The concerning part is Layer 1 (AFMNativeToolTurnSession) — it doesn't actually satisfy the session contract it's built against.

1. AFMNativeToolTurnSession drops the conversation on every iteration — likely breaks anything beyond a single tool call (High confidence)

macos/Sources/M1K3Inference/AFMNativeToolTurnSession.swift:66-80

The ToolTurnSession contract (ToolCallingProvider.swift:213-220) is explicit: "the caller sends only the NEW messages each iteration — the session retains everything prior." LocalAgent+Native.swift's runNativeLoop honors this precisely — pendingMessages is reset to [] after every send() and only the new tool result is appended for the next iteration (LocalAgent+Native.swift:146,187).

But AFMNativeToolTurnSession.send() builds its prompt from only the delta it's handed, and constructs a brand-new LanguageModelSession every call:

let body = AFMToolPrompt.render(messages: messages, tools: toolDefinitions)   // messages = delta only
...
let session = LanguageModelSession(tools: tools, instructions: standing)      // fresh, no memory of prior turns

There's no accumulated transcript anywhere in this class (unlike StatelessToolTurnSession, which keeps a Mutex<[ToolMessage]> and always re-renders the full snapshot). So on iteration ≥1 — i.e. right after the model has called a tool and gotten a result back — the fresh session sees something like:

Conversation:
Result from datetime: Saturday 14 June 2026, 12:00

Decide the single next step. ...

with no memory of the original user question, its own prior tool call, or anything beyond the constant base persona. The model is being asked to "answer using the gathered information" without ever having seen what was asked.

This directly undercuts the PR's own goals: the RULES change asks Mini to "combine multiple tool calls across iterations" (AgentRAGResponder.swift), and baseMaxIterations was raised 3→5 specifically to fit two tool calls in one turn — but the session backing that loop can't actually carry context between iterations to make that useful. synthesizeNativeConclusion (LocalAgent+Native.swift:336-364) hits the identical hole: on cap-reached it sends only the instruction text through this same session, so the model has no access to any of the transcript's gathered observations when trying to synthesize a final answer from them — it either happens to reconstruct something plausible or falls through to the empty-remainder gatheredObservations fallback.

The eval's own "stochastic" 2/10 tool-use misses (lookup_fact, recent_busiest) are exactly the shape this bug predicts: fixtures where the tool result alone isn't self-explanatory and needs to be related back to the original question. Worth re-running those with the state loss fixed before attributing the misses to model noise.

The file's own header comment (AFMNativeToolTurnSession.swift:19-21) frames the fresh-session-per-send() choice purely as a performance trade-off ("same cost as the AFMToolDecision path... a future optimisation: hold one session across the turn") — but it's not just slower, it's actually dropping data the loop depends on. Suggest accumulating the transcript inside this session (mirroring StatelessToolTurnSession's pattern) and rendering the full accumulated history each send(), at minimum as a correctness fix before the perf optimization.

2. renderForNativeSession is dead code — the session calls the wrong renderer (Medium confidence)

macos/Sources/M1K3Inference/AFMToolMapping.swift:150-155 adds renderForNativeSession(messages:), whose doc comment explains exactly why it exists: "Omits the text tool catalogue (the FM session provides structured definitions via tools:) and the 'Decide the single next step' instruction (designed for AFMToolDecision constrained decoding, not native tool calling)."

But AFMNativeToolTurnSession.send() (AFMNativeToolTurnSession.swift:73) calls the old AFMToolPrompt.render(messages:tools:) instead — the one that still emits the full text tool catalogue (duplicating what's already passed structurally via LanguageModelSession(tools:)) and the "Decide the single next step... call one tool..." ReAct-style tail that's tuned for the legacy constrained-decoding path, not the native Tool protocol. renderForNativeSession has no call sites anywhere in Sources/ and no test coverage in AFMToolPromptTests.swift (which pins every other AFMToolPrompt entry point) — it reads like a leftover from development that should have replaced the render() call at line 73.

Fix: AFMNativeToolTurnSession.swift:73 → AFMToolPrompt.renderForNativeSession(messages: messages).

3. Minor: unguarded mutable state under @unchecked Sendable, and it's unused

AFMNativeToolTurnSession.swift:54,71 — private var iteration = 0, incremented in a defer, is never read anywhere in the class. Either drop it, or if it's meant to support a future fix for #1, document what guards it — the sibling StatelessToolTurnSession explicitly notes "the agent loop uses a session strictly serially anyway" to justify its own @unchecked Sendable; this class has no equivalent note for its own unsynchronized var.

4. Minor: process/provenance

• Neither new file (AFMNativeTool.swift, AFMNativeToolTurnSession.swift) carries a Signed: MurphySig trailer, unlike the adjacent files they extend (ToolCallingProvider.swift, LocalAgent+Native.swift, AFMToolMapping.swift all have one, Prior: Unknown where appropriate). Not fabricating one on the author's behalf — flagging so it can be added per repo convention.
• No swift test coverage for either new file (204 new lines). The live-model half legitimately can't run under swift test (same constraint as MLX — macos/CLAUDE.md's metallib-wall section), but the pure pieces (AFMNativeTool.wrap, ToolCallLog.append/drain, and especially renderForNativeSession) could be — a test pinning renderForNativeSession's output (the way AFMToolPromptTests.swift pins render/systemInstructions) would likely have caught finding MCP: ask_m1k3 hands back a job_id the client may have no advertised way to redeem (stale tool list + stateless transport) #2 before merge.

What's solid

• Layer 2 (budget + RULES wording) and the CoolHead easing change are small, well-reasoned, and the tests (CoolHeadPolicyTests, AgentRAGResponderTests) were updated in lockstep with the new caps.
• The persona SELF/tools carve-out (M1K3Persona.swift) is a good, narrowly-targeted fix for the over-broad "never call search_knowledge" ban, and M1K3PersonaTests.swift re-pins the char-count/SHA budgets honestly rather than loosening them silently.
• AFMNativeTool/AFMToolArguments correctly stay #if compiler(>=6.2)/@available-gated and dependency-scoped (no MLX/Metal leakage into the seam), consistent with the module's "no external deps" discipline.
• Good honesty in the PR body about the two remaining stochastic-looking failures and the verify-by-launch items still outstanding — though per finding docs(oss): community kit — CONTRIBUTING, SECURITY, CoC, build-from-source, templates #1, worth double-checking those aren't actually deterministic given the session bug.

🤖 Generated with Claude Code

[claude]

Review — PR #397 (Mini native tool calling + proactive tool use)

Nice, well-scoped change with good test coverage for the pure Layer-2/persona
pieces. One correctness bug in the new native session, plus a provenance gap.

1. AFMNativeToolTurnSession.send() drops its own generated turns from the transcript it re-renders — real correctness bug

macos/Sources/M1K3Inference/AFMNativeToolTurnSession.swift:68-135

The class accumulates the messages the agent sends (transcript.append(contentsOf: messages), line 73) but never appends the model's own generated turn (the tool call it just requested, or its text) before the next send(). Compare with the pattern it's explicitly modeled on, StatelessToolTurnSession (macos/Sources/M1K3Inference/ToolCallingProvider.swift:336-353), which appends .assistant(text:toolCalls:) after every generation — that's the only reason the next continueToolTurn call sees a coherent conversation.

Trace a real two-tool-call turn:

• Iteration 0: transcript = [.system, .user(goal)]. Model calls tool A → .toolCalls([A]) returned. Nothing recorded.
• LocalAgent executes A, then calls session.send([.toolResult(A, obs)]) for iteration 1.
• transcript becomes [.system, .user(goal), .toolResult(A, obs)] — no .assistant(text: nil, toolCalls: [A]) in between.
• AFMToolPrompt.render (AFMToolMapping.swift:123-133) renders .assistant turns as "Assistant called A(query)" immediately before the paired "Result from A: obs" — that pairing is exactly what's supposed to give AFM the context for why a result appeared. With the assistant line missing, iteration ≥2 shows AFM a Result from A: … line with no record of ever asking for it.

The file header doc (lines 12-22) explicitly claims this is fixed: "Accumulate messages into a full transcript... The full transcript is re-rendered each call so the model always sees the complete conversation." It doesn't — only half the conversation (the agent's half) is retained. This is precisely the kind of bug swift test can't catch (AFM only runs from a bundled .app / M1K3_AFM_EVAL=1), so it's worth fixing before relying on the live eval numbers for anything beyond n=1 spot checks — it could plausibly explain some of the stochasticity in the two remaining tool-use failures (lookup_fact, recent_busiest), both of which are multi-step-shaped.

Suggested fix: mirror StatelessToolTurnSession — after computing calls/response.content, append the corresponding .assistant(text:toolCalls:) entry to self.transcript before returning, for both the .toolCalls and .text branches.

2. New files missing the repo's provenance header (MurphySig)

macos/Sources/M1K3Inference/AFMNativeTool.swift and
macos/Sources/M1K3Inference/AFMNativeToolTurnSession.swift are both new
files with no Signed:/Prior: block. Every neighboring file in this module
(ToolCallingProvider.swift, AppleFoundationModelsProvider.swift,
AFMToolMapping.swift) carries one, and the convention here is explicit: new
files should be signed Prior: Unknown at minimum. Worth adding before this
lands.

3. Minor: AFMNativeTool's @unchecked Sendable looks unnecessary

macos/Sources/M1K3Inference/AFMNativeTool.swift:27

AFMNativeTool's stored properties are String, String, and a
@Sendable (String, String) -> Void closure — all already Sendable, so a
plain Sendable conformance should be inferable without @unchecked. If
there's a real reason @unchecked is needed here (protocol conformance
quirk on Tool?), it'd be worth a one-line comment per the @unchecked
convention this file otherwise doesn't need elsewhere in the PR (the
AFMNativeToolTurnSession and ToolCallLog uses both explain themselves).

What's good

• CoolHeadPolicy/baseMaxIterations/prompt changes are small, mechanically
  consistent, and every numeric change has a matching test update
  (CoolHeadPolicyTests, AgentRAGResponderTests, MemoryGroundingTests,
  M1K3PersonaTests budget pins + re-pinned SHA).
• The AFMToolDecision legacy path is cleanly preserved as a fallback rather
  than deleted, so older runtimes / bypassed callers keep working.
• Good instinct pulling the persona's blanket "NEVER call search_knowledge…"
  ban and replacing it with a narrower carve-out — the old rule's failure
  mode (banning tool use for all questions, not just self-queries) is a
  believable read of a 0/10 tool-use score, and the test change
  (selfQueryFromPersona) still pins the self-query behavior without the
  overbroad wording.
• ToolCallLog's NSLock.withLock { } usage is properly scoped, and the
  @unchecked Sendable on AFMNativeToolTurnSession itself is documented
  with the actual invariant that makes it safe (strictly serial send()
  calls) — good practice per the repo's own concurrency guidance.

🤖 Generated with Claude Code

[claude]

Review: #397 — native tool calling + proactive tool use

Solid piece of work overall — the pure AFMToolMapping/AFMToolPrompt split stays testable, the CoolHeadPolicy/AgentRAGResponder budget changes are small and well-pinned, and fold commit 2 already caught several real issues (dead renderForNativeSession, transcript loss, a stale hash pin) from a prior local pass. One correctness gap survived that pass, though, plus a few smaller items.

1. AFMNativeToolTurnSession never records its own tool-call/text turns into transcript (correctness)

macos/Sources/M1K3Inference/AFMNativeToolTurnSession.swift:68-138

send() appends the incoming messages delta to transcript (line 73), but never appends the model's own decision afterward. Compare with the sibling StatelessToolTurnSession (macos/Sources/M1K3Inference/ToolCallingProvider.swift:336-353), which explicitly does:

```swift
case let .toolCalls(calls):
transcript.withLock { $0.append(.assistant(text: nil, toolCalls: calls)) }
```

LocalAgent+Native.swift's runNativeLoop (macos/Sources/M1K3Agent/LocalAgent+Native.swift:166-188) only ever sends .toolResult messages back as the next delta (pendingMessages) — it keeps its own .assistant(text: nil, toolCalls: calls) entry in a local trace that is never sent to the session. That's fine for StatelessToolTurnSession because it independently reconstructs the missing assistant turn itself, but AFMNativeToolTurnSession doesn't, so from iteration 2 onward its internally-accumulated transcript (and therefore the rendered "Conversation:" body — AFMToolMapping.render, which specifically documents itself as carrying "the model's own prior calls, tool results") is missing every "Assistant called X(Y)" line. The model sees a Result from web_search: … line with no visible call that produced it.

This directly undercuts the change this PR is making: maxIterations went 3→5 and the RULES now explicitly ask Mini to "combine multiple tool calls across iterations" — exactly the multi-iteration path where this gap bites. It's also the path synthesizeNativeConclusion (LocalAgent+Native.swift:336-364) leans on hardest: at the iteration cap it sends only a single one-line instruction and relies entirely on the session's own accumulated transcript for context.

Fix: mirror StatelessToolTurnSession — append transcript.append(.assistant(text: nil, toolCalls: parsed)) before return .toolCalls(parsed) (~line 122), and transcript.append(.assistant(text: response.content, toolCalls: [])) before return .text(response.content) (~line 126).

Worth noting there's no unit-test coverage for AFMNativeToolTurnSession at all (understandably, since it drives a live LanguageModelSession), so this class's pure bookkeeping (message accumulation) has no seam to catch a regression like this the way AFMToolPromptTests.swift pins render()/systemInstructions(). Might be worth factoring the transcript-accumulation + turn-classification into something injectable/testable, since it's exactly the kind of pure logic this codebase otherwise keeps pinned.

2. render(tools: []) leaves a self-contradictory closing paragraph (prompt content)

macos/Sources/M1K3Inference/AFMToolMapping.swift:89-147, called with an empty tool list from AFMNativeToolTurnSession.swift:78

render() only emits the "Available tools:" catalogue when !tools.isEmpty (line 92), but the trailing instruction paragraph is unconditional and says "The tools listed above are yours to USE" / "Never say you lack access to something a listed tool provides" (lines 138-145). When called with tools: [] (as the native session now does, per fold commit 3, to avoid double-counting tokens against the structured tools: param), nothing is actually listed above — the closing paragraph refers to a catalogue that isn't there. Functionally the model still gets real tool definitions via the native Tool protocol, so this probably isn't silently broken, but it's an inconsistent/confusing prompt that the deleted renderForNativeSession (which had wording that didn't assume a printed catalogue) avoided. Consider branching the closing paragraph on tools.isEmpty rather than reusing the catalogue-authoring wording verbatim.

3. Minor: @unchecked Sendable on AFMNativeTool looks unnecessary and is undocumented

macos/Sources/M1K3Inference/AFMNativeTool.swift:27

AFMNativeTool's only stored properties are String, String, and a @Sendable closure — all already Sendable, with no mutable state. Unlike the two other @unchecked Sendables touched in this PR (AFMNativeToolTurnSession's class-level one now has an explanatory comment after the fold pass, and ToolCallLog is genuinely guarded by NSLock), this one has no comment explaining what checked-Sendable is being bypassed. If it's just working around a Tool protocol quirk, a one-line note would help the next reader; otherwise it can likely just be Tool, Sendable.

4. Minor: transcript on AFMNativeToolTurnSession is a bare var, not Mutex-guarded like its sibling

macos/Sources/M1K3Inference/AFMNativeToolTurnSession.swift:39

The comment at line 30-31 ("the agent loop uses a session strictly serially... so the unsynchronized transcript is safe") checks out against the actual call site (LocalAgent+Native.swift's for iteration in 0..<maxIterations loop awaits each send() before the next). But StatelessToolTurnSession — solving the identical "single mutable transcript on an @unchecked Sendable class used serially" problem two files over — wraps its transcript in Mutex<[ToolMessage]> anyway rather than relying on caller discipline. For consistency (and because "serial by convention" is exactly the kind of invariant that quietly breaks when a call site changes later), matching that pattern here would be cheap insurance.

5. Nit: stale doc comment

macos/Sources/M1K3Inference/AppleFoundationModelsProvider.swift:538-539 says makeToolTurnSession "falls back to the default StatelessToolTurnSession... on older runtimes," but the implementation (540-548) unconditionally constructs AFMNativeToolTurnSession — there's no #available/runtime branch here (the actual macOS-27-vs-26 fallback lives inside AFMNativeToolTurnSession.send() itself, for GenerationOptions). Worth tightening so a future reader doesn't go looking for a fallback that isn't there.

What's good

• CoolHeadPolicy/AgentRAGResponder/persona test updates are mechanical and consistent with the stated budget changes (3→5, eased 2→3), and the SHA/length pins were correctly re-derived rather than loosened blindly.
• Keeping AFMToolMapping/AFMToolPrompt pure and MLX/FoundationModels-free (seam discipline) is maintained — the new AFMNativeTool/AFMNativeToolTurnSession correctly confine @_weakLinked import FoundationModels to files gated behind #if compiler(>=6.2) + @available.
• Commit 2's self-review fold (dead code removal, transcript-loss fix, budget re-pin) shows the local review step called out in macos/CLAUDE.md is doing real work — it just didn't go far enough on the transcript issue above.
• No provenance/signature issues — new files correctly carry no fabricated Signed: blocks.