Skip to content

feat(llms,embeddings): add aimlapi.com as a named LLM and embedding provider - #1

Open
Lookoff-AIMLAPI wants to merge 3 commits into
mainfrom
feat/aimlapi-provider
Open

feat(llms,embeddings): add aimlapi.com as a named LLM and embedding provider#1
Lookoff-AIMLAPI wants to merge 3 commits into
mainfrom
feat/aimlapi-provider

Conversation

@Lookoff-AIMLAPI

Copy link
Copy Markdown
Member

Linked Issue

Not applicable — this PR targets the fork's own main. Upstream requires
Closes #<issue> on an issue carrying the accepted label; see Upstream gates
below before this is offered to mem0ai/mem0.

Description

Adds aimlapi as a named provider in both the LLM and the embedding
categories, in the Python SDK and the TypeScript SDK.

mem0 is a memory library, so a gateway is only useful here if it serves the
embedding path as well as the chat path. Two existing routes fall short:

  • provider: "litellm" with an aiml/... model covers chat only. LiteLLM ships
    llms/aiml/chat/ and image_generation/ but no embedding/ route, so an
    aiml/<embedding model> call fails with Unmapped LLM provider for this endpoint. Verified against upstream LiteLLM's tree.
  • provider: "openai" with openai_base_url works but leaves the provider
    unnamed in configs, docs and the factory, and silently shares the OpenAI env
    vars.

So this is a named provider on both sides, sharing one small connection module
(mem0/utils/aimlapi.py, mem0-ts/src/oss/src/utils/aimlapi.ts).

What was added

File Role
mem0/llms/aimlapi.py AimlapiLLM(LLMBase) over the OpenAI SDK
mem0/embeddings/aimlapi.py AimlapiEmbedding(EmbeddingBase)
mem0/configs/llms/aimlapi.py AimlapiConfig with aimlapi_base_url
mem0/utils/aimlapi.py base-URL resolution, attribution headers, drop_none
mem0/llms/configs.py, mem0/embeddings/configs.py allowlist entries
mem0/utils/factory.py LlmFactory + EmbedderFactory entries
mem0/configs/embeddings/base.py aimlapi_base_url parameter
mem0-ts/.../llms/aimlapi.ts, .../embeddings/aimlapi.ts TS twins
mem0-ts/.../utils/factory.ts case "aimlapi" in both factories
mem0-ts/.../types/index.ts, llms/openai.ts, embeddings/openai.ts optional defaultHeaders pass-through
tests/llms/test_aimlapi.py, tests/embeddings/test_aimlapi_embeddings.py 21 pytest cases
mem0-ts/.../tests/aimlapi.test.ts, factory.unit.test.ts 15 jest cases
docs/**, LLM.md, docs/llms.txt provider pages, cards, nav, param tables

Three details that are not cosmetic

1. Unset options are omitted from the payload, never sent as null.
The API answers 400 to an explicit null on tools, temperature, top_p,
seed, tool_choice, response_format, stream, stream_options,
parallel_tool_calls, max_tokens and max_completion_tokens on the strictest
models, rather than reading it as "unset". Which models are strict varies —
google/gemini-2.5-flash accepts null everywhere, openai/gpt-5-mini rejects
tools: null (confirmed live: 400, error.details[].path = "tools"). The
OpenAI SDK serialises a None-valued keyword straight through, so a provider
copied from a working OpenAI-compatible one succeeds on turn 1 and fails on the
turn where an agent loop clears its tools — with a fully green test suite.
drop_none() sits on the request path and
test_unset_parameters_are_omitted_not_nulled asserts no None reaches the wire.

2. The embedder sends strings, not token ids.
POST /v1/embeddings rejects the pre-tokenised integer-array form of input
with 400 and error.details[].path = "input". That is exactly what LangChain's
OpenAIEmbeddings sends by default, so anyone routing this gateway through
mem0's langchain embedder will hit it. mem0's own OpenAI-SDK embedder sends
strings, this one mirrors it, a test asserts the input elements are strings, and
the docs page carries the warning for the LangChain route.

3. Attribution headers are scoped to the origin and built per client.
X-AIMLAPI-Partner-ID / X-AIMLAPI-Source / HTTP-Referer / X-Title are
attached through the OpenAI SDK's default_headers, but only when the resolved
host is api.aimlapi.com — a user who repoints the provider at their own gateway
(aimlapi_base_url, AIMLAPI_API_BASE) gets None. HTTP-Referer/X-Title
name Mem0, matching how mem0/llms/openai.py already labels OpenRouter
traffic. Caller headers win on a key clash, and the module constant is never
mutated. A malformed partner id is dropped silently by the gateway and earns
nothing, so its shape is asserted in a test
(^part_[A-Za-z0-9]{1,64}$) in both languages rather than discovered in
production.

defaultHeaders had to be threaded through the TypeScript OpenAILLM and
OpenAIEmbedder, which had no way to set request headers. It is spread only
when present, so every other provider's client construction is byte-identical.

Defaults

  • LLM: openai/gpt-5-mini
  • Embedder: openai/text-embedding-3-small, 1536 dims — matches mem0's existing
    default embedding_model_dims, so the default vector-store collection works
    unchanged.
  • Key: AIMLAPI_API_KEY; base URL override AIMLAPI_API_BASE /
    aimlapi_base_url / TS baseURL.

Both defaults were checked against GET /v1/models (present as ids) and
called live — catalog membership alone is not proof: the catalog omits ids that
serve traffic and has been observed listing at least one that 404s.

Type of Change

  • New feature (non-breaking change that adds functionality)

AI Assistance

To be completed by the human author before any upstream submission — upstream's
template asks about the code, and upstream's CONTRIBUTING says the author must be
able to explain the diff without an AI tool.

Breaking Changes

None. Every change outside the new files is additive: three allowlist/registry
entries per language, one new optional parameter on BaseEmbedderConfig, one
optional field on the two TS config interfaces.

Test Coverage

  • I added/updated unit tests
  • I tested manually (live calls, below)

Baseline vs after

Python (pytest tests/ --continue-on-collection-errors, local env missing some
optional extras — azure-identity, sentence-transformers, vertexai,
chromadb, faiss — which accounts for every error and failure in both columns):

baseline (clean tree) after
passed 1101 1122
failed 20 20
skipped 42 42
collection errors 29 29

The failure and error sets are identical line-for-line; the +21 is the new tests.

TypeScript (pnpm run test):

baseline after
suites passed 94 95
tests passed 1528 1543
skipped 42 42
failed 0 0

Builds: uv buildmem0ai-2.0.20-py3-none-any.whl, exit 0.
pnpm run build → exit 0. ruff check mem0/ tests/ → clean.
ruff format --check on the six new/changed Python files → clean.
isort --check-only flags 18 pre-existing files, none of them mine.
python scripts/check-llms-txt-coverage.py → in sync.

Live calls

Python, through Memory.from_config with aimlapi for both the LLM and the
embedder (qdrant on disk):

1. Embedding path — real /v1/embeddings call through AimlapiEmbedding
   embed()      -> 1536 dims, first 3 = [-0.01147, -0.02693, 0.01121]
   embed_batch()-> 3 vectors, dims [1536, 1536, 1536]
   3-large      -> 3072 dims

2. LLM path — real /v1/chat/completions call through AimlapiLLM
   generate_response() -> 'hello from aimlapi'
   tool calling:
   generate_response(tools=...) -> {'content': None, 'tool_calls':
     [{'name': 'add_memory', 'arguments': {'data': 'The user is a vegetarian.'}}]}

3. Full Memory() round trip — aimlapi for LLM and embeddings
   add()    -> 3 ADD events extracted from
               "I'm Stan. I'm a vegetarian and I'm allergic to peanuts."
   search("What does the user eat?") ->
     ('User stated they are a vegetarian ...', 0.3866)
     ('User stated they are allergic to peanuts ...', 0.2729)
     ('User introduced themselves as Stan ...', 0.2033)

TypeScript, through LLMFactory.create("aimlapi") / EmbedderFactory.create("aimlapi")
on the built dist/oss bundle:

LLM  -> "hello from aimlapi ts"
EMB  -> 1536 dims, first 3 = [ 0.001667022705078125, 0.00470733642578125, -0.048095703125 ]
BATCH-> 3 vectors of 1536 dims

Notes for a future upstream PR

  • The second commit (chore(aimlapi): fork-only placement) must be dropped.
    It only moves the aimlapi.com entry to the head of the hand-ordered,
    user-facing docs lists (two card groups, two docs.json nav groups, LLM.md,
    two llms.txt sections). Reordering someone else's provider list is not ours
    to do upstream. mem0 has no "recommended"/featured badge concept in those
    lists, so none was invented. The Python allowlists and factory maps are left in
    append position in the first commit — their order is not user-visible.

  • Whether the attribution headers ship upstream in v1 or as a follow-up is a
    commercial call, not a technical one. They are two dict keys plus the
    origin guard; removing them means deleting _ATTRIBUTION_HEADERS /
    ATTRIBUTION_HEADERS and passing default_headers=None.

  • Upstream gates, verbatim from CONTRIBUTING.md:

    "Always open an issue before opening a pull request. [...] Every pull
    request must link to an issue using Closes #<issue-number>, and that issue
    must carry the accepted label."

    "We cannot accept or merge any pull request until you have signed our
    Contributor License Agreement (CLA).
    "

    "You must be able to explain what your changes do and how they interact
    with the rest of the codebase without the help of an AI tool.
    This is the
    one rule we will not bend on."

    .github/workflows/pr-gate.yml enforces the first one by closing the PR
    within a minute and reopening it when the label lands.

Things worth knowing that are not fixed here

  • max_tokens does not reliably bound reasoning tokens on this gateway, and
    completion_tokens excludes reasoning tokens on some models
    (google/gemini-2.5-flash returned prompt 12 / completion 3 / total 86 /
    reasoning 71 in probing). mem0 does not meter spend, so nothing breaks, but the
    docs page says not to treat max_tokens as a cost ceiling.
  • The model field echoed back does not always match the id requested
    (openai/gpt-5-5gpt-5.5-2026-04-23). mem0 does not pin or record the
    serving model, so this is informational.
  • No graph-store or reranker support was added — out of scope for a provider PR.

Checklist

  • My code follows the project's style guidelines
  • I have performed a self-review of my code
  • I have added tests that prove my feature works
  • New and existing tests pass locally
  • I have updated documentation

mem0 is a memory library, so a gateway is only usable here if it can serve
the embedding path as well as the LLM path. Reaching aimlapi.com through the
existing `litellm` provider covers chat but not embeddings — LiteLLM ships no
embedding route for its `aiml` provider and answers "Unmapped LLM provider for
this endpoint" — and reaching it through `openai` + `openai_base_url` leaves
the provider unnamed in configs and docs. Hence a named provider in both
categories, sharing one connection module.

The two providers are thin wrappers over the OpenAI SDK because the endpoint is
OpenAI-compatible, which is the same shape xai/deepseek/sarvam already use.

Three details are not cosmetic:

- Unset options are omitted from the chat payload rather than passed as None.
  The API answers 400 to an explicit `null` on `tools`, `temperature`, `top_p`,
  `seed`, `response_format` and others instead of reading it as "unset". The
  OpenAI SDK serialises a None-valued keyword straight through, so a provider
  that forwards None succeeds on a first turn and fails on the turn where an
  agent loop clears its tools.

- The embedder sends `input` as strings. The endpoint rejects the pre-tokenised
  integer-array form with a 400 naming `input`, which is what LangChain's
  OpenAIEmbeddings sends by default; going through mem0's own OpenAI-SDK
  embedder avoids it, and the docs page says so for anyone routing this gateway
  through the `langchain` embedder instead.

- Attribution headers are attached per client and only when the resolved host is
  api.aimlapi.com, so a user who repoints the provider at their own gateway does
  not forward them to a third party. HTTP-Referer/X-Title name Mem0, matching how
  the OpenAI provider already labels OpenRouter traffic.

`defaultHeaders` is threaded through the TypeScript OpenAI LLM and embedder,
which had no way to set request headers; it is spread only when present, so
every other provider's client construction is unchanged.
Moves the aimlapi.com entry to the head of every hand-ordered, user-facing
provider list: the two docs card groups, the two docs.json nav groups, the
LLM.md provider lists and the two llms.txt sections. No behaviour changes and
no code changes — docs ordering only.

This is deliberately isolated so it can be dropped before the change is offered
upstream, where self-promotion in someone else's provider list is not ours to
make. mem0 has no "recommended"/featured badge concept in these lists, so none
was invented.

The Python allowlists and the factory maps are left in append position: their
order is not user-visible and reordering them would be noise in the diff.
The placeholder part_mem0 was a readable stand-in chosen before the
partner was registered. Registration mints the id server-side, so the
real value is part_JNAROikm3sdRqpewzcZxLgrK. A wrong or unknown partner id is accepted with a
200 and silently not attributed, so this would not have surfaced at runtime.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant