Skip to content

feat(llm): add aimlapi.com as an OpenAI-compatible provider - #1

Open
Lookoff-AIMLAPI wants to merge 3 commits into
mainfrom
feat/aimlapi-provider
Open

feat(llm): add aimlapi.com as an OpenAI-compatible provider#1
Lookoff-AIMLAPI wants to merge 3 commits into
mainfrom
feat/aimlapi-provider

Conversation

@Lookoff-AIMLAPI

@Lookoff-AIMLAPI Lookoff-AIMLAPI commented Sep 2, 2026

Copy link
Copy Markdown
Member

What this adds

aimlapi as a named OpenAI-compatible LLM provider, plus the matching embeddings
branch. Shaped deliberately after the last three gateway additions
(Requesty vectorize-io#2399, Atlas Cloud vectorize-io#2362, OpenRouter) — a named string in the factory
list, the valid_providers list and the API-key-required list, a base-URL
default, one row in llmProviders.json, and the configuration-reference rows.
No new plumbing.

Surface Change
engine/llm_wrapper.py aimlapi in the factory tuple + valid_providers; base URL https://api.aimlapi.com/v1
engine/providers/openai_compatible_llm.py same three sites; attribution headers on the client
engine/embeddings.py aimlapi embeddings branch; optional default_headers on OpenAIEmbeddings (defaults to None, so no existing caller's request shape moves)
config.py ENV_* / DEFAULT_* constants, dataclass fields, from_env() wiring, attribution constants + merge helper
hindsight-embed/.../control_center/providers.py one entry in the config-wizard dropdown catalog (the sibling Atlas Cloud has and Requesty/OpenRouter do not)
docs, .env.example (+ embed copy), generated docs skill provider rows, env vars, an example block

Env vars follow the repo's own naming, not ours:
HINDSIGHT_API_AIMLAPI_API_KEY, HINDSIGHT_API_EMBEDDINGS_AIMLAPI_API_KEY,
HINDSIGHT_API_EMBEDDINGS_AIMLAPI_MODEL, with the same
EMBEDDINGS_* -> shared key -> LLM_API_KEY fallback chain OpenRouter and Requesty
already use, so a single-key setup needs one variable.

Only the /v1 root is declared. The gateway has no /v1/completions route
it 404s — so nothing declares one; the OpenAI SDK appends /chat/completions.

The display label is aimlapi.com everywhere a human sees it; the machine
identifier is aimlapi.

Attribution headers

Four headers ride requests to this gateway: X-AIMLAPI-Partner-ID,
X-AIMLAPI-Source, HTTP-Referer and X-Title. The last two point at
Hindsight (github.com/vectorize-io/hindsight / Hindsight), per the
OpenRouter convention — they identify the calling application, not the gateway.

There is precedent for this: create_llm_provider() already threads
default_headers into the OpenAI-compatible client for operators fronting a
proxy, so this reuses that seam rather than adding one. Three properties are
covered by tests:

  • Merged, never assigned. An operator's HINDSIGHT_API_LLM_DEFAULT_HEADERS
    wins on a key clash.
  • Never mutated. A fresh dict is built per client; the module constant and
    the caller's dict are both left untouched.
  • Scoped to provider and host. provider == "aimlapi" and
    hostname == api.aimlapi.com. Pointing HINDSIGHT_API_LLM_BASE_URL at a
    private proxy that merely speaks the same wire format sends no attribution, and
    no other provider ever sees these headers.

A malformed partner id is dropped by the gateway without failing the request
silently — so its shape (^part_[A-Za-z0-9]{1,64}$) is asserted in a test rather
than left to review.

Model ids

Both defaults were checked against the live catalog
(GET https://api.aimlapi.com/v1/models, 936 entries, 353 of them
type == "openai/chat-completions"), against ids and aliases, not copied from
another gateway's list:

  • openai/gpt-5-mini — present as an id on the chat surface. Chosen to line up
    with Hindsight's own gpt-5-mini default rather than mirroring Requesty's
    gpt-4o-mini.
  • openai/text-embedding-3-small — present as an id on the openai/embeddings
    surface, and verified live to return 1536 dimensions.

Verification

Build / install: uv sync --directory hindsight-api/ and uv sync in
hindsight-api-slim/ both exit 0; the package imports cleanly.

Tests — full hindsight-api-slim/tests/ suite, -n 8 --dist=loadfile, same
machine, same env, before and after:

baseline (pristine main) with this branch
passed 5808 5823 (+15)
failed 56 53
errors 1837 1837
skipped 252 252
xfailed 6 6

Zero new failures. Comparing the failure sets rather than the counts: not
one name appears in the after-run that was not already in the baseline, and three
test_per_operation_llm_config.py::TestMockLLMProvider failures cleared (flaky
under parallel load). The 1837 errors and the bulk of the failures are
environmental — there is no Docker on this machine, so every
testcontainers/Postgres-backed test errors identically in both runs. The 12 new
tests in tests/test_aimlapi_provider.py account for most of the +15.

ruff check and ruff format --check are clean for both hindsight-api-slim and
hindsight-embed. ty check hindsight_embed is clean; ty check hindsight_api
reports 22 unresolved-import diagnostics, all for optional local-ml extras
(torch / transformers / sentence-transformers / mlx / flashrank / pg0) that are
not installed in this environment — none touch the changed files.

hindsight-embed/tests/test_env_template.py passes: .env.example was re-copied
to the embed package. ./scripts/generate-docs-skill.sh was run and its output
committed, so verify-generated-files should be clean.

One real inference call, through the added code path (LLMProvider.from_env()
-> create_llm_provider() -> OpenAICompatibleLLM) — not a mock, not raw curl:

provider=aimlapi model=openai/gpt-5-mini base_url=https://api.aimlapi.com/v1
header HTTP-Referer: 'https://github.com/vectorize-io/hindsight'
header X-Title: 'Hindsight'
header X-AIMLAPI-Partner-ID: 'part_hindsight'
header X-AIMLAPI-Source: 'agent/hindsight'
CHAT RESULT: 'hindsight-aimlapi-ok'
TOOL CALLS: [LLMToolCall(id='call_WovH1J1xQdFFtZx6DFUKXU21', name='record_city', arguments={'city': 'Berlin'}, thought_signature=None)]
EMBEDDINGS: model=openai/text-embedding-3-small dim=1536 n=1

Tool calling and the embeddings surface were both exercised live.

Worth knowing about the default model: openai/gpt-5-mini is a reasoning
model. At max_completion_tokens=64 the tool-call probe returned no tool calls
and no content — reasoning tokens consumed the whole budget, and the response
carries no error to say so. At 512 it works every time. That is standard
reasoning-model behaviour rather than anything specific to this gateway, but it
is worth knowing before setting a tight
HINDSIGHT_API_REFLECT_MAX_COMPLETION_TOKENS.

Not done, deliberately

  • No changelog entryCLAUDE.md forbids "Unreleased" entries; the release
    script writes them.
  • No _CONFIGURABLE_FIELDS membership for the new embeddings fields, matching
    how the OpenRouter and Requesty embeddings fields are already treated (static).
  • No reranker branch. The gateway's rerank surface was not verified, so
    nothing claims it. Requesty is in the same position.
  • No /v1/responses routing. openai-responses remains a separate,
    explicitly chosen provider; aimlapi speaks chat/completions only.
  • No versioned_docs/ edits. Those are frozen release snapshots.

The last commit is fork-only

chore(aimlapi): fork-only placement — do not send upstream moves aimlapi.com to
the front of the hand-ordered, user-facing provider lists (docs grid/table source,
both docs enum rows, the Provider Examples block, .env.example, and the embed
wizard dropdown). It is a pure reordering — 34 insertions, 34 deletions, no
behaviour — isolated so it can be dropped with one revert before any upstream
conversation.

No badge or "featured" flag was invented: llmProviders.json has no such field.
The only marker used is the parenthetical the Provider Examples block already
applies to Groq. Machine-readable orderings are untouched: the validation lists in
llm_wrapper.py / openai_compatible_llm.py and the PROVIDER_DEFAULT_MODELS
lookup table keep aimlapi wherever the feature commit put it.

aimlapi added 2 commits September 3, 2026 04:30
Hindsight operators who already route models through a gateway currently have
OpenRouter, Requesty and Atlas Cloud to choose from. aimlapi.com is the same
shape of thing — one key, one OpenAI-compatible endpoint, several hundred chat
models — so it is wired the same way the last three gateways were rather than
introducing any new plumbing: a named string in the factory list, the
valid-provider list and the API-key-required list, plus a base-URL default.

The gateway serves embeddings from the same host and key, so the embeddings
factory gets the matching branch. Its key falls back through
HINDSIGHT_API_AIMLAPI_API_KEY to HINDSIGHT_API_LLM_API_KEY, mirroring the chain
OpenRouter and Requesty already use, so a single-key setup needs one variable.

Only the /v1 root is declared: the gateway has no /v1/completions route, and
declaring one would turn a 404 into a confusing "model is broken" report.

Requests to the gateway carry attribution headers. There is precedent for them
— create_llm_provider already threads default_headers into the OpenAI-compatible
client for operators fronting a proxy — so this reuses that seam instead of
adding one. They are merged *under* any operator-supplied headers so a configured
HINDSIGHT_API_LLM_DEFAULT_HEADERS still wins, built into a fresh dict per client
so the module constant is never mutated, and gated on both the provider name and
the resolved host, so they cannot ride a request to another provider or to a
private proxy that merely speaks the same wire format. A malformed partner id is
dropped by the gateway without failing the request, i.e. invisibly, so its shape
is pinned by a test rather than left to review.

Default model openai/gpt-5-mini and embedding model openai/text-embedding-3-small
were both checked against the live catalog (ids and aliases) rather than copied
from another gateway's list.
Moves aimlapi.com to the front of the hand-ordered, user-facing provider lists:
the docs provider grid/table source, the two provider enum rows in the
configuration reference, the Provider Examples block, the .env.example
supported-provider line and example blocks, and the embed control center's
wizard dropdown catalog. The generated docs skill is refreshed from those
sources.

This is partnership placement, not a technical requirement. It is isolated in
its own commit so it can be dropped with a single revert before the provider
itself is offered upstream — reordering someone else's list to put ourselves
first is not something a maintainer should have to argue about in review.

No badge or "featured" flag was invented: llmProviders.json has no such field.
The only marker used is the parenthetical the Provider Examples block already
applies to Groq. Machine-readable orderings are untouched: the valid-provider
and factory lists in llm_wrapper.py / openai_compatible_llm.py are validation
sets whose order no user ever sees, and PROVIDER_DEFAULT_MODELS is a lookup
table, so all of them keep aimlapi in the position the feature commit put it.
The placeholder part_hindsight was a readable stand-in chosen before the
partner was registered. Registration mints the id server-side, so the
real value is part_fO6J5vQsER4jNM0YFhQkld2H. A wrong or unknown partner id is accepted with a
200 and silently not attributed, so this would not have surfaced at runtime.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant