Skip to content

docs: provider vision-input notes + accuracy fixes #303

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
v1r3n merged 5 commits into from
Jul 10, 2026
Merged

docs: provider vision-input notes + accuracy fixes #303

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
d74b7d8
docs: note HuggingFace vision (image input) via OpenAI-compatible router
ling-senpeng13 Jul 2, 2026
4aed2bc
docs: note vision (image input) for Anthropic, Gemini, Grok, Perplexity
ling-senpeng13 Jul 2, 2026
03c13cd
docs: note native vision (image input) for OpenAI, Azure, Mistral, Ol…
ling-senpeng13 Jul 2, 2026
8d5d858
docs: add Cohere vision (image input) note
ling-senpeng13 Jul 2, 2026
725ceeb
docs: remove DeepSeek — not implemented in conductor-ai
ling-senpeng13 Jul 2, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
22 changes: 22 additions & 0 deletions docs/ai-models.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -32,6 +32,8 @@ agentspan server start

**Models:** `openai/gpt-4o`, `anthropic/claude-sonnet-4-6`, `openai/gpt-4-turbo`, `openai/o1`, `openai/o1-mini`, `openai/o3-mini`

**Vision (image input):** GPT-4o models accept image input (e.g. `openai/gpt-4o`, `openai/gpt-4o-mini`); support is model-dependent.

**Embeddings:** `openai/text-embedding-3-small`, `openai/text-embedding-3-large`

**Image generation:** `openai/dall-e-3`
Expand All @@ -46,6 +48,8 @@ agentspan server start

**Models:** `anthropic/claude-opus-4-20250514`, `anthropic/claude-sonnet-4-20250514`, `anthropic/claude-3-5-sonnet-20241022`, `anthropic/claude-3-haiku-20240307`

**Vision (image input):** Claude models accept image input (e.g. `anthropic/claude-3-5-sonnet-20241022`); support is model-dependent. See [conductor-oss/conductor#1238](https://github.com/conductor-oss/conductor/pull/1238).

---

### Google Gemini
Expand All @@ -57,6 +61,8 @@ agentspan server start

**Models:** `google_gemini/gemini-2.0-flash`, `google_gemini/gemini-1.5-pro`, `google_gemini/gemini-1.5-flash`

**Vision (image input):** Gemini models accept image input (e.g. `google_gemini/gemini-1.5-pro`); support is model-dependent. See [conductor-oss/conductor#1241](https://github.com/conductor-oss/conductor/pull/1241).

**Embeddings:** `google_gemini/text-embedding-004`

**Image generation:** `google_gemini/imagen-3.0-generate-002`
Expand All @@ -73,6 +79,8 @@ agentspan server start

**Models:** `azure_openai/gpt-4o`, `azure_openai/gpt-4`, `azure_openai/gpt-3.5-turbo`

**Vision (image input):** vision-capable deployments accept image input (e.g. a `gpt-4o` deployment); support is model-dependent.

---

### AWS Bedrock
Expand All @@ -87,6 +95,8 @@ agentspan server start

**Models:** `aws_bedrock/anthropic.claude-3-5-sonnet-20241022-v2:0`, `aws_bedrock/anthropic.claude-3-haiku-20240307-v1:0`, `aws_bedrock/meta.llama3-70b-instruct-v1:0`, `aws_bedrock/amazon.titan-text-express-v1`

**Vision (image input):** vision-capable models accept image input (e.g. `aws_bedrock/anthropic.claude-3-5-sonnet-20241022-v2:0`); support is model-dependent.

**Embeddings:** `aws_bedrock/amazon.titan-embed-text-v2:0`

---
Expand All @@ -99,6 +109,8 @@ agentspan server start

**Models:** `mistral/mistral-large-latest`, `mistral/mistral-medium-latest`, `mistral/mistral-small-latest`, `mistral/open-mixtral-8x7b`

**Vision (image input):** Pixtral models accept image input (e.g. `mistral/pixtral-12b-2409`); support is model-dependent.

**Embeddings:** `mistral/mistral-embed`

---
Expand All @@ -111,6 +123,8 @@ agentspan server start

**Models:** `cohere/command-r-plus`, `cohere/command-r`, `cohere/command`

**Vision (image input):** vision-capable Cohere models accept image input (e.g. `cohere/command-a-vision-07-2025`); support is model-dependent. See [conductor-oss/conductor#1246](https://github.com/conductor-oss/conductor/pull/1246).

**Embeddings:** `cohere/embed-english-v3.0`, `cohere/embed-multilingual-v3.0`

---
Expand All @@ -123,6 +137,8 @@ agentspan server start

**Models:** `grok/grok-3`, `grok/grok-3-mini`

**Vision (image input):** vision-capable Grok models accept image input; support is model-dependent. See [conductor-oss/conductor#1243](https://github.com/conductor-oss/conductor/pull/1243).

---

### Perplexity AI
Expand All @@ -133,6 +149,8 @@ agentspan server start

**Models:** `perplexity/sonar-pro`, `perplexity/sonar`

**Vision (image input):** vision-capable Perplexity models accept image input; support is model-dependent. See [conductor-oss/conductor#1243](https://github.com/conductor-oss/conductor/pull/1243).

---

### Hugging Face
Expand All @@ -143,6 +161,8 @@ agentspan server start

**Models:** `hugging_face/meta-llama/Llama-3-70b-chat-hf`, `hugging_face/mistralai/Mistral-7B-Instruct-v0.2`

**Vision (image input):** vision-capable models (e.g. `hugging_face/meta-llama/Llama-3.2-11B-Vision-Instruct`) accept image input via Hugging Face's OpenAI-compatible router (`https://router.huggingface.co/v1`); support is model-dependent. See [conductor-oss/conductor#1245](https://github.com/conductor-oss/conductor/pull/1245).

---

### Stability AI
Expand Down Expand Up @@ -175,6 +195,8 @@ Install Ollama: [ollama.com/download](https://ollama.com/download)

**Models:** `ollama/llama3`, `ollama/mistral`, `ollama/phi3`, `ollama/codellama`

**Vision (image input):** vision-capable local models accept image input (e.g. `ollama/llava`, `ollama/llama3.2-vision`); support is model-dependent.

**Embeddings:** `ollama/nomic-embed-text`

---
Expand Down
33 changes: 22 additions & 11 deletions docs/providers.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -45,6 +45,8 @@ agent = Agent(name="bot", model="google_gemini/gemini-2.0-flash")

**Models:** `openai/gpt-4o`, `anthropic/claude-sonnet-4-6`, `openai/gpt-4-turbo`, `openai/o1`, `openai/o1-mini`, `openai/o3-mini`

**Vision (image input):** GPT-4o models accept image input (e.g. `openai/gpt-4o`, `openai/gpt-4o-mini`); support is model-dependent.

**Embeddings:** `openai/text-embedding-3-small`, `openai/text-embedding-3-large`

---
Expand All @@ -57,6 +59,8 @@ agent = Agent(name="bot", model="google_gemini/gemini-2.0-flash")

**Models:** `anthropic/claude-opus-4-20250514`, `anthropic/claude-sonnet-4-6`, `anthropic/claude-3-5-sonnet-20241022`, `anthropic/claude-3-haiku-20240307`

**Vision (image input):** Claude models accept image input (e.g. `anthropic/claude-3-5-sonnet-20241022`); support is model-dependent. See [conductor-oss/conductor#1238](https://github.com/conductor-oss/conductor/pull/1238).

---

### Google Gemini
Expand All @@ -68,6 +72,8 @@ agent = Agent(name="bot", model="google_gemini/gemini-2.0-flash")

**Models:** `google_gemini/gemini-2.0-flash`, `google_gemini/gemini-1.5-pro`, `google_gemini/gemini-1.5-flash`

**Vision (image input):** Gemini models accept image input (e.g. `google_gemini/gemini-1.5-pro`); support is model-dependent. See [conductor-oss/conductor#1241](https://github.com/conductor-oss/conductor/pull/1241).

**Embeddings:** `google_gemini/text-embedding-004`

---
Expand All @@ -82,6 +88,8 @@ agent = Agent(name="bot", model="google_gemini/gemini-2.0-flash")

**Models:** `azure_openai/gpt-4o`, `azure_openai/gpt-4`, `azure_openai/gpt-3.5-turbo`

**Vision (image input):** vision-capable deployments accept image input (e.g. a `gpt-4o` deployment); support is model-dependent.

---

### AWS Bedrock
Expand All @@ -93,6 +101,8 @@ agent = Agent(name="bot", model="google_gemini/gemini-2.0-flash")

**Models:** `aws_bedrock/anthropic.claude-3-5-sonnet-20241022-v2:0`, `aws_bedrock/anthropic.claude-3-haiku-20240307-v1:0`, `aws_bedrock/meta.llama3-70b-instruct-v1:0`, `aws_bedrock/amazon.titan-text-express-v1`

**Vision (image input):** vision-capable models accept image input (e.g. `aws_bedrock/anthropic.claude-3-5-sonnet-20241022-v2:0`); support is model-dependent.

**Embeddings:** `aws_bedrock/amazon.titan-embed-text-v2:0`

---
Expand All @@ -105,6 +115,8 @@ agent = Agent(name="bot", model="google_gemini/gemini-2.0-flash")

**Models:** `mistral/mistral-large-latest`, `mistral/mistral-medium-latest`, `mistral/mistral-small-latest`, `mistral/open-mixtral-8x7b`

**Vision (image input):** Pixtral models accept image input (e.g. `mistral/pixtral-12b-2409`); support is model-dependent.

**Embeddings:** `mistral/mistral-embed`

---
Expand All @@ -117,6 +129,8 @@ agent = Agent(name="bot", model="google_gemini/gemini-2.0-flash")

**Models:** `cohere/command-r-plus`, `cohere/command-r`, `cohere/command`

**Vision (image input):** vision-capable Cohere models accept image input (e.g. `cohere/command-a-vision-07-2025`); support is model-dependent. See [conductor-oss/conductor#1246](https://github.com/conductor-oss/conductor/pull/1246).

**Embeddings:** `cohere/embed-english-v3.0`, `cohere/embed-multilingual-v3.0`

---
Expand All @@ -129,6 +143,8 @@ agent = Agent(name="bot", model="google_gemini/gemini-2.0-flash")

**Models:** `grok/grok-3`, `grok/grok-3-mini`

**Vision (image input):** vision-capable Grok models accept image input; support is model-dependent. See [conductor-oss/conductor#1243](https://github.com/conductor-oss/conductor/pull/1243).

---

### Perplexity AI
Expand All @@ -139,6 +155,8 @@ agent = Agent(name="bot", model="google_gemini/gemini-2.0-flash")

**Models:** `perplexity/sonar-pro`, `perplexity/sonar`

**Vision (image input):** vision-capable Perplexity models accept image input; support is model-dependent. See [conductor-oss/conductor#1243](https://github.com/conductor-oss/conductor/pull/1243).

---

### Hugging Face
Expand All @@ -149,6 +167,8 @@ agent = Agent(name="bot", model="google_gemini/gemini-2.0-flash")

**Models:** `hugging_face/meta-llama/Llama-3-70b-chat-hf`, `hugging_face/mistralai/Mistral-7B-Instruct-v0.2`

**Vision (image input):** vision-capable models (e.g. `hugging_face/meta-llama/Llama-3.2-11B-Vision-Instruct`) accept image input via Hugging Face's OpenAI-compatible router (`https://router.huggingface.co/v1`); support is model-dependent. See [conductor-oss/conductor#1245](https://github.com/conductor-oss/conductor/pull/1245).

---

### Stability AI
Expand All @@ -161,16 +181,6 @@ agent = Agent(name="bot", model="google_gemini/gemini-2.0-flash")

---

### DeepSeek

| Variable | Description |
|---|---|
| `DEEPSEEK_API_KEY` | API key from DeepSeek |

**Models:** `deepseek/deepseek-chat`

---

### Ollama (local)

No API key required. Ollama must be running and reachable.
Expand All @@ -181,6 +191,8 @@ No API key required. Ollama must be running and reachable.

**Models:** `ollama/llama3`, `ollama/mistral`, `ollama/phi3`, `ollama/codellama`

**Vision (image input):** vision-capable local models accept image input (e.g. `ollama/llava`, `ollama/llama3.2-vision`); support is model-dependent.

**Embeddings:** `ollama/nomic-embed-text`

Install Ollama: [ollama.com/download](https://ollama.com/download)
Expand All @@ -202,5 +214,4 @@ Install Ollama: [ollama.com/download](https://ollama.com/download)
| Perplexity | `PERPLEXITY_API_KEY` | `perplexity/` |
| Hugging Face | `HUGGINGFACE_API_KEY` | `hugging_face/` |
| Stability AI | `STABILITY_API_KEY` | `stabilityai/` |
| DeepSeek | `DEEPSEEK_API_KEY` | `deepseek/` |
| Ollama | `OLLAMA_BASE_URL` | `ollama/` |
Loading