docs: aimlapi.com via LiteLLM β LLM pipeline and embeddings vectors backend - #1
Open
Lookoff-AIMLAPI wants to merge 2 commits into
Open
docs: aimlapi.com via LiteLLM β LLM pipeline and embeddings vectors backend#1Lookoff-AIMLAPI wants to merge 2 commits into
Lookoff-AIMLAPI wants to merge 2 commits into
Conversation
txtai delegates every hosted API to LiteLLM, so aimlapi.com already works without any code here. Two things about it are not discoverable from the existing docs and cost a user a failed run each: - The LLM pipeline reads AIML_API_KEY, not the AIMLAPI_API_KEY that the provider's own docs use. The general "set the API key via environment variable" comment does not say which variable, and there is no way to guess it. - LiteLLM registers `aiml` as an openai-compatible provider for chat completions only. Its embedding() dispatch does not consult that list, so an `aiml/` path in a vectors config fails with "Unmapped LLM provider for this endpoint". The working route is the openai-compatible one with api_base set, which is the same pattern already documented for a local OpenAI-compatible endpoint. The vectors note also warns against putting api_key in `vectors`. That dict is part of the index configuration and is serialised verbatim into config.json on save, which writes a live key to disk next to the index. Verified live against https://api.aimlapi.com/v1: LLM("aiml/openai/gpt-5-5") returns a completion, and the documented embeddings config returns 3072 dimensions with correct nearest neighbours over a five-row index.
Moves the aimlapi.com example to the top of the hand-ordered "LLM APIs" block in docs/pipeline/llm/llm.md. This is presentation, not information, and it is the only ordering change available in this repository: txtai has no provider registry, dropdown or badge mechanism, so there is nothing else to place or feature. Drop this commit before the change goes upstream.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What this is
txtai has no provider registry.
GenerationFactory(src/python/txtai/pipeline/llm/factory.py)dispatches to five backends β
litellm,litert,llama.cpp,opencode,transformersβ andevery hosted API goes through LiteLLM. The vectors side is the same shape:
VectorsFactoryroutes hosted embeddings APIs to
src/python/txtai/vectors/dense/litellm.py.So aimlapi.com needs no code here; it already works. What it needed was documentation, because
two things about it are not discoverable and each costs a user a failed run.
Docs only. No Python changed.
The two things
1. The LLM pipeline reads
AIML_API_KEY. The existing docs say "must also set API key viaenvironment variable" without naming any variable. LiteLLM's
aimlprovider readsAIML_API_KEY(
litellm/llms/aiml/chat/transformation.py), which differs from theAIMLAPI_API_KEYused in theprovider's own documentation. There is no way to guess it.
2.
aiml/does not work for embeddings, and the docs now say so. LiteLLM listsaimlinopenai_compatible_providers(litellm/constants.py:843), but onlycompletion()andspeech()consult that list.
embedding()hardcodes its openai-compatible branch asopenai | together_ai | nvidia_nim | litellm_proxy(litellm/main.py:6220-6226), soaiml/onthe embeddings route raises:
The route that does work is the OpenAI-compatible one with
api_baseset β the same patterndocs/pipeline/llm/llm.mdalready documents for a local OpenAI-compatible endpoint. That is whatthe new
### litellmexample indocs/embeddings/configuration/vectors.mdshows.The same section warns against putting
api_keyinsidevectors. That dict is part of the indexconfiguration and is serialised verbatim into
config.jsononsave()β verified: a live keyappeared in plain text in the saved index directory.
api_basealone is safe to persist.Verified live
Base URL
https://api.aimlapi.com/v1, real key via env, through txtai's own classes β not curl,not a mock.
The
LLMexample, exactly as documented (onlyAIML_API_KEYset, noOPENAI_API_KEY):Also called on
aiml/anthropic/claude-sonnet-4.5,aiml/google/gemini-2.5-flashandaiml/deepseek/deepseek-chatβ all returned completions.The embeddings example, the YAML block from the docs loaded verbatim into
Embeddings(config):Real 3072-dimension vectors, five out of five nearest neighbours correct. Sibling embedding ids
called the same way:
text-embedding-3-small1536,qwen-text-embedding-v41024,voyage-code-21536.aiml/openai/text-embedding-3-smallwas also called and failed as described above β the docsstatement is tested in both directions.
Build and tests
1d45e88)mkdocs buildunittest -k testvectorsunittest -k testpipeline.testllmMINIMAL=1 pip install . && python -c "import txtai"Identical before and after, which is what a markdown-only diff should produce. The errors in both
columns are missing optional extras in the local environment (
fastapi,staticvectors,sentence-transformers,PIL,soundfile,timm,smolagents), not failures β the same testserror on a pristine checkout.
testvectors.testdense.testlitellmandtestpipeline.testllm.testlitellm, the two modules covering the documented code paths, pass inboth columns.
Not verified: the full
pip install .[all,dev]+make data coveragematrix that CI runs. Itneeds Java/Tika,
llama-cpp-pythonbuilt from source and the audio stack, none of which this diffcan reach β it touches two markdown files and no Python.
Rendered links in the built site were checked:
../../../pipeline/llm/llmand../../methods/#txtai.embeddings.Embeddings.saveboth resolve against generated pages and anchors.Two side observations, not fixed here
methodenum indocs/embeddings/configuration/vectors.md:15liststransformers|sentence-transformers|llama.cpp|litellm|model2vec|external|wordsbutVectorsFactoryalso acceptslitertandonnx.docs/embeddings/configuration/cloud.md:5anddocs/embeddings/indexing.md:41,61link to#txtai.embeddings.base.Embeddings.<method>anchors. mkdocstrings emits#txtai.embeddings.Embeddings.<method>(no.base.), so those anchors do not resolve. The newlink in this PR uses the working form.
Left alone to keep the diff minimal.
Commits
Two, deliberately separable:
docs: cover aimlapi.com models on the LiteLLM LLM and vectors backendsβ the content above.chore(aimlapi): fork-only placement β do not send upstreamβ moves the aimlapi.com line to thetop of the hand-ordered "LLM APIs" example block. Presentation only. Drop it before this goes
anywhere upstream.