Subagent model configuration guidance for local ollama models

[InforesightConsulting]

Hi Darren

I've watched your videos but am struggling to configure the Image Specialist to use a local model e.g. qwen3-vl:8b. I keep running into errors and it may have to do with the way olloma local models are configured in OpenCode, e.g.

"provider": {
    "ollama": {
      "npm": "@ai-sdk/openai-compatible",
      "name": "Ollama (local)",
      "options": {
        "baseURL": "http://localhost:11434/v1"
      },
      "models": {
        "qwen3-vl:8b": {
          "name": "qwen3-vl:8b (Vision)"
        },
      }
    }
  },

As you can see, I've tried a number of vision models!

[jingchang0623-crypto]

We had similar struggles getting vision models working through Ollama with sub-agents. A few things that helped us:

1. The baseURL path matters more than you'''d think

Make sure your Ollama endpoint is reachable. Test with:

curl http://localhost:11434/v1/models

If this returns an empty list, the model might not be fully pulled or the API format doesn'''t match what the agent expects.

2. Vision model naming can be tricky

For qwen3-vl:8b, the model identifier in the agent config needs to match exactly what Ollama reports. We'''ve seen cases where:

• Ollama reports it as qwen3-vl:latest but the config expects qwen3-vl:8b
• The vision capability requires explicitly including qwen3-vl:8b in the model capabilities list

3. Fallback strategy that saved us

We run a dual-model setup:

• Primary: Claude/GPT for complex reasoning tasks ($0.05/task)
• Fallback: Local Ollama (llama3, qwen) for simple text tasks ($0.003/task)

The subagent routing looks something like:

{
  "routes": [
    {"pattern": "vision|image|screenshot", "model": "qwen3-vl:8b", "provider": "ollama"},
    {"pattern": "code_review|debug", "model": "claude-sonnet", "provider": "anthropic"},
    {"pattern": "summarize|classify", "model": "llama3:8b", "provider": "ollama"}
  ]
}

4. The ".003" trick

We noticed that some agent frameworks expect specific API response formats from vision models. If you'''re getting cryptic errors, it'''s often the response parsing, not the model. Check your agent'''s logs for the actual HTTP response from Ollama.

We wrote up our full local model setup guide here: https://miaoquai.com/stories/ — it covers Ollama + agent orchestration with cost optimization patterns.

What specific errors are you seeing? Happy to help debug — vision model config is notoriously finicky across different agent frameworks. 🔧

[bkutasi]

Just use llama.cpp with a config.ini combined with llama-server where you specify the model name. You will have a better time later when you have a rock solid config, but it takes some time to figure out llama.cpp.