Voice needs a hosted model — there is no local realtime speech model worth putting a call on yet — but the bots are plain text models and have no such excuse. Right now they run on OpenAI, Anthropic, Google, xAI or the Vercel AI Gateway (textModelProviderSchema in features/ai/model.schema.ts). An Ollama or vLLM box on the same network should be just as valid.
What it takes
A provider entry that takes a base url instead of a hosted name, built the same way the others are in features/ai/model.ts, plus the settings field for the url.
Done when
A bot with a local endpoint selected runs a job end to end — including tool calls, which is where a weaker local model usually falls over. Say in the PR which model you tested.
This is the most requested thing for a project like this, so it is worth doing properly rather than quickly.
Voice needs a hosted model — there is no local realtime speech model worth putting a call on yet — but the bots are plain text models and have no such excuse. Right now they run on OpenAI, Anthropic, Google, xAI or the Vercel AI Gateway (
textModelProviderSchemainfeatures/ai/model.schema.ts). An Ollama or vLLM box on the same network should be just as valid.What it takes
A provider entry that takes a base url instead of a hosted name, built the same way the others are in
features/ai/model.ts, plus the settings field for the url.Done when
A bot with a local endpoint selected runs a job end to end — including tool calls, which is where a weaker local model usually falls over. Say in the PR which model you tested.
This is the most requested thing for a project like this, so it is worth doing properly rather than quickly.