feat(llm): add aimlapi.com as an OpenAI-compatible provider - #1
Open
Lookoff-AIMLAPI wants to merge 3 commits into
Open
feat(llm): add aimlapi.com as an OpenAI-compatible provider#1Lookoff-AIMLAPI wants to merge 3 commits into
Lookoff-AIMLAPI wants to merge 3 commits into
Conversation
added 2 commits
September 3, 2026 04:30
Hindsight operators who already route models through a gateway currently have OpenRouter, Requesty and Atlas Cloud to choose from. aimlapi.com is the same shape of thing — one key, one OpenAI-compatible endpoint, several hundred chat models — so it is wired the same way the last three gateways were rather than introducing any new plumbing: a named string in the factory list, the valid-provider list and the API-key-required list, plus a base-URL default. The gateway serves embeddings from the same host and key, so the embeddings factory gets the matching branch. Its key falls back through HINDSIGHT_API_AIMLAPI_API_KEY to HINDSIGHT_API_LLM_API_KEY, mirroring the chain OpenRouter and Requesty already use, so a single-key setup needs one variable. Only the /v1 root is declared: the gateway has no /v1/completions route, and declaring one would turn a 404 into a confusing "model is broken" report. Requests to the gateway carry attribution headers. There is precedent for them — create_llm_provider already threads default_headers into the OpenAI-compatible client for operators fronting a proxy — so this reuses that seam instead of adding one. They are merged *under* any operator-supplied headers so a configured HINDSIGHT_API_LLM_DEFAULT_HEADERS still wins, built into a fresh dict per client so the module constant is never mutated, and gated on both the provider name and the resolved host, so they cannot ride a request to another provider or to a private proxy that merely speaks the same wire format. A malformed partner id is dropped by the gateway without failing the request, i.e. invisibly, so its shape is pinned by a test rather than left to review. Default model openai/gpt-5-mini and embedding model openai/text-embedding-3-small were both checked against the live catalog (ids and aliases) rather than copied from another gateway's list.
Moves aimlapi.com to the front of the hand-ordered, user-facing provider lists: the docs provider grid/table source, the two provider enum rows in the configuration reference, the Provider Examples block, the .env.example supported-provider line and example blocks, and the embed control center's wizard dropdown catalog. The generated docs skill is refreshed from those sources. This is partnership placement, not a technical requirement. It is isolated in its own commit so it can be dropped with a single revert before the provider itself is offered upstream — reordering someone else's list to put ourselves first is not something a maintainer should have to argue about in review. No badge or "featured" flag was invented: llmProviders.json has no such field. The only marker used is the parenthetical the Provider Examples block already applies to Groq. Machine-readable orderings are untouched: the valid-provider and factory lists in llm_wrapper.py / openai_compatible_llm.py are validation sets whose order no user ever sees, and PROVIDER_DEFAULT_MODELS is a lookup table, so all of them keep aimlapi in the position the feature commit put it.
Lookoff-AIMLAPI
force-pushed
the
feat/aimlapi-provider
branch
from
September 2, 2026 23:31
b8eb2c5 to
90132f6
Compare
The placeholder part_hindsight was a readable stand-in chosen before the partner was registered. Registration mints the id server-side, so the real value is part_fO6J5vQsER4jNM0YFhQkld2H. A wrong or unknown partner id is accepted with a 200 and silently not attributed, so this would not have surfaced at runtime.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What this adds
aimlapias a named OpenAI-compatible LLM provider, plus the matching embeddingsbranch. Shaped deliberately after the last three gateway additions
(Requesty vectorize-io#2399, Atlas Cloud vectorize-io#2362, OpenRouter) — a named string in the factory
list, the
valid_providerslist and the API-key-required list, a base-URLdefault, one row in
llmProviders.json, and the configuration-reference rows.No new plumbing.
engine/llm_wrapper.pyaimlapiin the factory tuple +valid_providers; base URLhttps://api.aimlapi.com/v1engine/providers/openai_compatible_llm.pyengine/embeddings.pyaimlapiembeddings branch; optionaldefault_headersonOpenAIEmbeddings(defaults toNone, so no existing caller's request shape moves)config.pyENV_*/DEFAULT_*constants, dataclass fields,from_env()wiring, attribution constants + merge helperhindsight-embed/.../control_center/providers.py.env.example(+ embed copy), generated docs skillEnv vars follow the repo's own naming, not ours:
HINDSIGHT_API_AIMLAPI_API_KEY,HINDSIGHT_API_EMBEDDINGS_AIMLAPI_API_KEY,HINDSIGHT_API_EMBEDDINGS_AIMLAPI_MODEL, with the sameEMBEDDINGS_* -> shared key -> LLM_API_KEYfallback chain OpenRouter and Requestyalready use, so a single-key setup needs one variable.
Only the
/v1root is declared. The gateway has no/v1/completionsroute —it 404s — so nothing declares one; the OpenAI SDK appends
/chat/completions.The display label is
aimlapi.comeverywhere a human sees it; the machineidentifier is
aimlapi.Attribution headers
Four headers ride requests to this gateway:
X-AIMLAPI-Partner-ID,X-AIMLAPI-Source,HTTP-RefererandX-Title. The last two point atHindsight (
github.com/vectorize-io/hindsight/Hindsight), per theOpenRouter convention — they identify the calling application, not the gateway.
There is precedent for this:
create_llm_provider()already threadsdefault_headersinto the OpenAI-compatible client for operators fronting aproxy, so this reuses that seam rather than adding one. Three properties are
covered by tests:
HINDSIGHT_API_LLM_DEFAULT_HEADERSwins on a key clash.
the caller's dict are both left untouched.
provider == "aimlapi"andhostname == api.aimlapi.com. PointingHINDSIGHT_API_LLM_BASE_URLat aprivate proxy that merely speaks the same wire format sends no attribution, and
no other provider ever sees these headers.
A malformed partner id is dropped by the gateway without failing the request —
silently — so its shape (
^part_[A-Za-z0-9]{1,64}$) is asserted in a test ratherthan left to review.
Model ids
Both defaults were checked against the live catalog
(
GET https://api.aimlapi.com/v1/models, 936 entries, 353 of themtype == "openai/chat-completions"), against ids and aliases, not copied fromanother gateway's list:
openai/gpt-5-mini— present as an id on the chat surface. Chosen to line upwith Hindsight's own
gpt-5-minidefault rather than mirroring Requesty'sgpt-4o-mini.openai/text-embedding-3-small— present as an id on theopenai/embeddingssurface, and verified live to return 1536 dimensions.
Verification
Build / install:
uv sync --directory hindsight-api/anduv syncinhindsight-api-slim/both exit 0; the package imports cleanly.Tests — full
hindsight-api-slim/tests/suite,-n 8 --dist=loadfile, samemachine, same env, before and after:
main)Zero new failures. Comparing the failure sets rather than the counts: not
one name appears in the after-run that was not already in the baseline, and three
test_per_operation_llm_config.py::TestMockLLMProviderfailures cleared (flakyunder parallel load). The 1837 errors and the bulk of the failures are
environmental — there is no Docker on this machine, so every
testcontainers/Postgres-backed test errors identically in both runs. The 12 new
tests in
tests/test_aimlapi_provider.pyaccount for most of the +15.ruff checkandruff format --checkare clean for bothhindsight-api-slimandhindsight-embed.ty check hindsight_embedis clean;ty check hindsight_apireports 22
unresolved-importdiagnostics, all for optionallocal-mlextras(torch / transformers / sentence-transformers / mlx / flashrank / pg0) that are
not installed in this environment — none touch the changed files.
hindsight-embed/tests/test_env_template.pypasses:.env.examplewas re-copiedto the embed package.
./scripts/generate-docs-skill.shwas run and its outputcommitted, so
verify-generated-filesshould be clean.One real inference call, through the added code path (
LLMProvider.from_env()->
create_llm_provider()->OpenAICompatibleLLM) — not a mock, not raw curl:Tool calling and the embeddings surface were both exercised live.
Worth knowing about the default model:
openai/gpt-5-miniis a reasoningmodel. At
max_completion_tokens=64the tool-call probe returned no tool callsand no content — reasoning tokens consumed the whole budget, and the response
carries no error to say so. At 512 it works every time. That is standard
reasoning-model behaviour rather than anything specific to this gateway, but it
is worth knowing before setting a tight
HINDSIGHT_API_REFLECT_MAX_COMPLETION_TOKENS.Not done, deliberately
CLAUDE.mdforbids "Unreleased" entries; the releasescript writes them.
_CONFIGURABLE_FIELDSmembership for the new embeddings fields, matchinghow the OpenRouter and Requesty embeddings fields are already treated (static).
nothing claims it. Requesty is in the same position.
/v1/responsesrouting.openai-responsesremains a separate,explicitly chosen provider;
aimlapispeaks chat/completions only.versioned_docs/edits. Those are frozen release snapshots.The last commit is fork-only
chore(aimlapi): fork-only placement — do not send upstreammoves aimlapi.com tothe front of the hand-ordered, user-facing provider lists (docs grid/table source,
both docs enum rows, the Provider Examples block,
.env.example, and the embedwizard dropdown). It is a pure reordering — 34 insertions, 34 deletions, no
behaviour — isolated so it can be dropped with one revert before any upstream
conversation.
No badge or "featured" flag was invented:
llmProviders.jsonhas no such field.The only marker used is the parenthetical the Provider Examples block already
applies to Groq. Machine-readable orderings are untouched: the validation lists in
llm_wrapper.py/openai_compatible_llm.pyand thePROVIDER_DEFAULT_MODELSlookup table keep aimlapi wherever the feature commit put it.