feat(models): Ollama structured outputs + challenger question-vs-task fix - #132
Merged
Conversation
… fix
Two reliability fixes for the local model, from the 2026-07-10 capability
review (spec_v3.md §4.2 model layer, §6.9 challenger):
- Structured outputs: OllamaProvider.complete() accepts output_schema and
sends it as Ollama's `format` (constrained decoding, Ollama >= 0.5) so
schema-invalid local output is impossible. The router threads each task
type's schema for Ollama targets only (post-escalation cloud fallbacks
never see the kwarg), gated by new config `ollama.structured_outputs`
(default off in code; enabled in donna_models.yaml).
- challenger_parse.md: questions about the user's own tasks/schedule
("what's on my plate this week?") are `question`, never `task` — the one
real routing miss in the review's scenario suite.
Also checks in the review's reusable scenario suites
(fixtures/scenarios_dispatch, fixtures/scenarios_chat_intent) and the
standalone eval driver (scripts/eval_driver_standalone.py, runs from the
host against localhost Ollama, passes the schema for ollama targets).
Verification (live, qwen2.5:32b-instruct-q4_K_M): dispatch suite 12/14 →
14/14 — the question-routing miss and the schema-validation error both
fixed; mean latency down (no parse retries). Chat-intent suite unchanged
at 11/12. Unit suite: 2243 passed; ruff clean.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01K7Ppai9Xbhv5iEuGBwi3ty
Resolve conflict in docs/superpowers/specs/followups.md by keeping both the calendar-OAuth (PR #130, now on main) and local-LLM-reliability follow-up entries. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0163xNY4AKcd3pHF8CYVGPH5
Resolve conflict in docs/superpowers/specs/followups.md by keeping both the output-standard (PR #131, now on main) and local-LLM-reliability follow-up entries. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0163xNY4AKcd3pHF8CYVGPH5
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Changes (spec_v3.md §4.2, §6.9)
OllamaProvider.complete()takesoutput_schemaand sends it as Ollama'sformat(constrained decoding, Ollama ≥ 0.5): schema-invalid local output becomes impossible, and the parse-retry latency tax disappears. Router threads each task type's schema to Ollama targets only (cloud fallbacks unaffected), gated by newollama.structured_outputsconfig (default off in code, on in YAML).question, nevertask.fixtures/scenarios_dispatch,fixtures/scenarios_chat_intent) and the standalone host-side eval driver.Verification (live on qwen2.5:32b-instruct-q4_K_M)
Follow-ups (logged)
Spec §4 / model-layer doc to absorb the structured-outputs contract; next tuning: num_ctx 16k + KV q8_0, qwen3:30b-a3b trial gated on these suites.
🤖 Generated with Claude Code
https://claude.ai/code/session_01K7Ppai9Xbhv5iEuGBwi3ty