.ask: support an OpenAI-compatible API endpoint as an optional backend

[zlatent]

Problem

The .ask pipeline currently runs three local tiers (rules parser → Qwen 0.5B → Qwen 1.5B). This works great for privacy-sensitive environments, but the homepage itself acknowledges the tradeoff: "For top-tier accuracy on complex joins, a cloud model behind your own API key still beats us."

There's no way to opt in to a stronger model when privacy constraints allow it — even a locally-hosted one behind an OpenAI-compatible server (vLLM, Ollama, llama-server, LM Studio, etc.).

Proposal

Add an optional environment variable (e.g. SLOTHDB_ASK_API_URL) that points .ask at any OpenAI-compatible /v1/chat/completions endpoint. When set, the router would send the existing system prompt + schema context to the external model instead of the local Qwen tiers. The rules parser (Tier 1) would still run first — only model-dependent queries would go to the API.

A second variable like SLOTHDB_ASK_API_KEY could carry the bearer token, and SLOTHDB_ASK_API_MODEL could let users pick the model name to send in the request body.

Why OpenAI-compatible?

It's the de facto standard. A single implementation covers OpenAI, Azure OpenAI, Anthropic (via proxy), Ollama, vLLM, llama-server, LM Studio, Together, Groq, and others. Users who want to stay fully local can point it at http://localhost:11434/v1 (Ollama) and still never hit the internet.

Suggested behavior

• SLOTHDB_ASK_API_URL unset → current behavior, no change
• SLOTHDB_ASK_API_URL set → skip Qwen tiers, send to the API after rules parser miss
• SLOTHDB_ASK_CONFIRM=1 still works — SQL is printed and gated before execution regardless of source
• Existing SLOTHDB_ASK_MODEL=ON build flag remains independent — users could even have both compiled in and toggle via the env var

This would keep the zero-dependency default intact while letting power users swap in a stronger model when they choose to.

[SouravRoy-ETL]

Hi Sure. It will be integrated soon enough. Thanks for the Nudge!