feat(323): P0 gates measured + P1 — a model on this computer runs an agent

[munchist]

Plan 323: P0 (feasibility gates, measured) and P1 (one model on this computer runs an agent). The plan doc is updated to say what was built and what is still open.

P0 — measured, not researched

yarn models:local-gates <gate> runs each gate against the installed Ollama and prints JSON you can reproduce. Results for an Apple M3 with 24 GB on Ollama 0.40.5 are in docs/plans/323-p0/apple-m3-24gb.json; plan §9 summarises them. What they changed:

• The runners fail in opposite ways. The llama.cpp runner silently cuts an over-long prompt from the front, dropping the system message, and the answer comes back wrong. The MLX runner does not enforce num_ctx at all.
• prompt_eval_count is exact. The count includes any cached prefix; caching only changes the timing. That makes a one-token pre-count nearly free.
• Characters ÷ 4 undercounts. JSON is undercounted by about 40% and Chinese by about 2.5×.
• Replayed thinking reaches no model. The chat templates drop it from history.
• Locality is visible. GET /api/status reports the cloud-disable setting, and cloud models show remote_host in /api/tags.
• Pulls behave well. Two pulls of one tag join, a cancelled pull installs nothing, and the next pull resumes.
• Embeddings. Qwen3-Embedding 4B can produce 1536 dimensions, so the existing column works; EmbeddingGemma refuses. Both truncate silently unless truncate: false is sent.
• On MLX, model memory must be read from the process. /api/ps does not reflect the context size there.
• Model call inventory. Compaction was hard-coded to a cloud model, and memory embedding bypasses the space's credentials entirely.

P1 — what it adds

• An ollama provider on Ollama's native /api/chat and /api/embed. It is a credential-registry connection with no required secret, offered only by the local edition: the providers list, credential writes and /catalog/models all filter by edition.
• Context. Each request gets the connection's window, narrowed to the catalog's; the same window drives compaction (client.contextWindowFor). A prompt that may not fit is counted exactly by a one-token evaluation, the answer gets what the window leaves, and a prompt that doesn't fit is refused before it is sent. A response the runtime truncated fails instead of answering.
• Structured output. JSON is carried by a tool call when the runner refuses format (MLX returns 501). The adapter learns this per model from the runtime's answer.
• Native tool calling follows capabilities.functionCalling instead of a five-provider list. ⚠️ This also moves the three OpenRouter models, all declaring functionCalling: true, onto native tool calls. Their route already accepts tool definitions in the streaming path (models:verify-reasoning exercises it).
• Runtime admission.
  • A Redis lease keyed by the runtime, so every name for this machine's loopback counts as one runtime, sized by the connection's max_parallel.
  • A new optional StepHandler.admit: the step waits before it is started, without its executor slot and outside its timeout. A cancel ends the wait, and a draining executor hands the step back.
  • Callers outside a step take the same lease through createByokAiClientFactory(db, admission): titles decline when the runtime is busy, post-run judging waits.
  • The renewal timer is registered as ai.local_runtime_lease_renewal.
• Timeouts. Every AI operation on a local model gets a 15-minute idle window (AI_LOCAL_MODEL_IDLE_TIMEOUT_MS) with the agent-turn ceiling.
• Clerk and compaction. The clerk's auto mode takes a free space default, and compaction takes the clerk's model, so a space whose only provider is this computer can compact and title.
• Readiness. It no longer reports a local provider as ready with no credential, and the connect dialog can connect a provider that needs no key.
• yarn models:verify-local measures a build (digest, runner, quantisation, runtime version) and exits non-zero when its catalog entry drifts.
• First catalog entry: ollama-gemma4-26b (alias gemma-local).

Verified

• Unit tests: adapter, sizing, lease, admit hook, credential config, clerk, check probe and the edition filters.
• Full suites: yarn test with no database (Postgres suites skipped), tsc -b, format:check and repo-wide lint (0 errors).
• Against the live runtime (Ollama 0.40.5, gemma4:26b): plain text, two parallel native tool calls, streaming with thinking, JSON through the tool fallback after the 501, and a refusal before sending of a prompt the llama.cpp runner would have cut.

Not yet run

P1's exit: a Helmsman conversation through the web UI on the dev stack started from this branch, three concurrent sessions, and a long-prompt prefill. To try it:

1. Start the stack from this branch.
2. Add the Ollama connection under Settings → Credentials.
3. Enable Gemma 4 26B (this computer) for the tenant.
4. Assign it to Helmsman.

🤖 Generated with Claude Code