From b3fed0744e370ba8bec0cbb48b9d55080137594c Mon Sep 17 00:00:00 2001 From: Stan Date: Thu, 3 Sep 2026 13:45:13 +0500 Subject: [PATCH 1/2] docs: cover aimlapi.com models on the LiteLLM LLM and vectors backends txtai delegates every hosted API to LiteLLM, so aimlapi.com already works without any code here. Two things about it are not discoverable from the existing docs and cost a user a failed run each: - The LLM pipeline reads AIML_API_KEY, not the AIMLAPI_API_KEY that the provider's own docs use. The general "set the API key via environment variable" comment does not say which variable, and there is no way to guess it. - LiteLLM registers `aiml` as an openai-compatible provider for chat completions only. Its embedding() dispatch does not consult that list, so an `aiml/` path in a vectors config fails with "Unmapped LLM provider for this endpoint". The working route is the openai-compatible one with api_base set, which is the same pattern already documented for a local OpenAI-compatible endpoint. The vectors note also warns against putting api_key in `vectors`. That dict is part of the index configuration and is serialised verbatim into config.json on save, which writes a live key to disk next to the index. Verified live against https://api.aimlapi.com/v1: LLM("aiml/openai/gpt-5-5") returns a completion, and the documented embeddings config returns 3072 dimensions with correct nearest neighbours over a five-row index. --- docs/embeddings/configuration/vectors.md | 13 +++++++++++++ docs/pipeline/llm/llm.md | 3 +++ 2 files changed, 16 insertions(+) diff --git a/docs/embeddings/configuration/vectors.md b/docs/embeddings/configuration/vectors.md index 0c214bc7c..a77908efb 100644 --- a/docs/embeddings/configuration/vectors.md +++ b/docs/embeddings/configuration/vectors.md @@ -43,6 +43,19 @@ Builds embeddings using a [llama.cpp](https://github.com/abetlen/llama-cpp-pytho Builds embeddings using a LiteLLM model. See the [LiteLLM documentation](https://litellm.vercel.app/docs/providers) for the options available with LiteLLM models. +Options for the underlying API call, such as the base URL of an OpenAI-compatible endpoint, are set with [vectors](#vectors). + +```yaml +path: openai/text-embedding-3-large +method: litellm +vectors: + api_base: https://api.aimlapi.com/v1 +``` + +The example above runs [aimlapi.com](https://aimlapi.com) embeddings and reads the key from `OPENAI_API_KEY`. Note that the `aiml/` model prefix used by the [LLM](../../../pipeline/llm/llm) pipeline is a chat-completions route only — LiteLLM raises `Unmapped LLM provider for this endpoint` when it's used with embeddings, so the OpenAI-compatible route above is required. + +Setting `api_key` in `vectors` also works but isn't recommended. The `vectors` configuration is stored in the index configuration, so [saving](../../methods/#txtai.embeddings.Embeddings.save) an index would write the key to disk in plain text. + ### model2vec Builds embeddings using a [Model2Vec](https://github.com/MinishLab/model2vec) model. Model2Vec is a knowledge-distilled version of a transformers model with static vectors. diff --git a/docs/pipeline/llm/llm.md b/docs/pipeline/llm/llm.md index 72b6044e1..482d618c7 100644 --- a/docs/pipeline/llm/llm.md +++ b/docs/pipeline/llm/llm.md @@ -70,6 +70,8 @@ The LLM pipeline automatically detects the underlying LLM framework. This can al See the [LiteLLM documentation](https://litellm.vercel.app/docs/providers) for the options available with LiteLLM models. +Each LLM API reads its key from its own environment variable. For example, [aimlapi.com](https://aimlapi.com) models are prefixed with `aiml/` and read `AIML_API_KEY`. + See the [OpenCode documentation](https://opencode.ai/docs/server/) for more on how to integrate the LLM pipeline with a running OpenCode instance. ```python @@ -107,6 +109,7 @@ llm = LLM("openai/gpt-oss", api_base="http://localhost:4000") llm = LLM("gpt-5.2") llm = LLM("claude-opus-4-5-20251101") llm = LLM("gemini/gemini-3-pro-preview") +llm = LLM("aiml/openai/gpt-5-5") # Local OpenCode server started via `opencode serve` llm = LLM("opencode") From 6ab9bc11c425923e146ad9ca25442431a96c7db2 Mon Sep 17 00:00:00 2001 From: Stan Date: Thu, 3 Sep 2026 13:46:22 +0500 Subject: [PATCH 2/2] =?UTF-8?q?chore(aimlapi):=20fork-only=20placement=20?= =?UTF-8?q?=E2=80=94=20do=20not=20send=20upstream?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Moves the aimlapi.com example to the top of the hand-ordered "LLM APIs" block in docs/pipeline/llm/llm.md. This is presentation, not information, and it is the only ordering change available in this repository: txtai has no provider registry, dropdown or badge mechanism, so there is nothing else to place or feature. Drop this commit before the change goes upstream. --- docs/pipeline/llm/llm.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/pipeline/llm/llm.md b/docs/pipeline/llm/llm.md index 482d618c7..d0661891c 100644 --- a/docs/pipeline/llm/llm.md +++ b/docs/pipeline/llm/llm.md @@ -106,10 +106,10 @@ llm = LLM("ollama/gpt-oss", api_base="http://localhost:11434") llm = LLM("openai/gpt-oss", api_base="http://localhost:4000") # LLM APIs - must also set API key via environment variable +llm = LLM("aiml/openai/gpt-5-5") llm = LLM("gpt-5.2") llm = LLM("claude-opus-4-5-20251101") llm = LLM("gemini/gemini-3-pro-preview") -llm = LLM("aiml/openai/gpt-5-5") # Local OpenCode server started via `opencode serve` llm = LLM("opencode")