Skip to content

docs(aimlapi): document aimlapi.com as an LLM/embedding provider - #1

Open
Lookoff-AIMLAPI wants to merge 2 commits into
mainfrom
docs/aimlapi-provider
Open

docs(aimlapi): document aimlapi.com as an LLM/embedding provider#1
Lookoff-AIMLAPI wants to merge 2 commits into
mainfrom
docs/aimlapi-provider

Conversation

@Lookoff-AIMLAPI

@Lookoff-AIMLAPI Lookoff-AIMLAPI commented Sep 3, 2026

Copy link
Copy Markdown
Member

Description

cognee already works on aimlapi.com today — nothing in the codebase had to change
for that. Every completion goes through litellm, and litellm has shipped a
first-class aiml provider (AIMLChatConfig, base https://api.aimlapi.com/v1)
since long before the litellm>=1.83.7 pin here. What was missing was any hint
that this is possible: grep -i aimlapi over the tree returns nothing, so a user
has no way to discover the route.

So this is a documentation PR, and it lands on the same surfaces the OpenRouter
route already occupies, plus the Integrations Hub card and a runnable example
that the catalog guide asks every new integration to point at.

I started out expecting to add a provider and stopped when the live calls showed
there was nothing to add. Two things did turn up that are worth writing down,
because both are silent failures a user cannot reason their way out of:

1. LLM_PROVIDER must be "custom". Setting only LLM_MODEL="aiml/..."
raises ProviderNotDeducibleError — cognee's provider enum is deliberately
small and aiml is not in it. This is the same situation OpenRouter is in, and
the error message does point at custom, but it fires at import time from a
pydantic validator, which is a confusing place to meet it.

2. Embeddings must NOT carry the aiml/ prefix. litellm registers no
embedding route for aiml, only chat and image generation. So
aiml/openai/text-embedding-3-large dies with litellm.BadRequestError: Unmapped LLM provider for this endpoint. The working shape is the unprefixed model id
plus EMBEDDING_ENDPOINT aimed at aimlapi.com, which routes through litellm's
OpenAI-compatible handler. That asymmetry between LLM_MODEL and
EMBEDDING_MODEL is the single least guessable thing here, so it is called out
in all four places.

A separate thing I hit on the way: EMBEDDING_MAX_TOKENS is a no-op. The
settings field is embedding_max_completion_tokens, nothing anywhere reads
embedding_max_tokens, and EmbeddingConfig has extra="allow" so the wrong
spelling is absorbed without complaint. Eight existing jobs in test_llms.yml
use the wrong spelling. I did not touch them — out of scope for this PR — but
the example here uses the spelling that actually works, which .env.template
already uses in its advanced section.

Acceptance Criteria

Every model id named in these docs was called for real, through cognee's own
GenericAPIAdapter, not asserted from the catalogue.
acreate_structured_output
returned a parsed pydantic model for each:

model result
aiml/openai/gpt-4o-mini Answer(capital='Paris', country='France')
aiml/openai/gpt-5-mini Answer(capital='Paris', country='France')
aiml/deepseek/deepseek-v4-flash Answer(capital='Paris', country='France')
aiml/google/gemini-2.5-flash Answer(capital='Paris', country='France')
aiml/anthropic/claude-sonnet-4.6 Answer(capital='Paris', country='France')

Tool calling works on all four families through the same route
(get_weather{"city":"Paris"}). One id that the aimlapi.com catalogue publishes
as a chat model, meta-llama/llama-3.3-70b-versatile, 404s on a real call — it
is deliberately not named anywhere here.

The new example runs end to end, LLM and embeddings both on aimlapi.com,
no OpenAI key involved:

$ LLM_API_KEY=*** python examples/guides/aimlapi_example.py
==============================================================================
REMEMBER USING AIMLAPI.COM
==============================================================================
Using LLM: aiml/openai/gpt-4o-mini
Using Embeddings: openai/text-embedding-3-large
Knowledge graph built successfully.

==============================================================================
RECALL
==============================================================================
Query: What does Cognee help developers do?
Recall Results:
Cognee helps developers turn documents into AI memory, build semantic graphs,
index entities, and store vectors for structured retrieval.

That the embedding call really reaches aimlapi.com and not api.openai.com is
confirmed by swapping in a bogus key: the 401 body comes back from aimlapi.com's
own billing page, so EMBEDDING_ENDPOINT is genuinely honoured.

Catalog validation:

$ uv run python -m catalog.loader
loaded 11 catalog entries
  integration: 5
  package: 2
  use-case: 4
$ uv run python -m catalog.inventory_sync
(coverage gaps only; informational, not a failure)

Tests. Baseline on a pristine main: 11 failed, 4670 passed, 69 skipped
for cognee/tests/unit. After this branch, cognee/tests/unit + catalog/tests:
11 failed, 4680 passed, 69 skipped — the same 11 failures, name for name (diffed),
plus the 10 catalog tests the after-run added. The 11 are pre-existing and unrelated
(retrieval / code-graph). Three files fail to collect on both trees for missing
optional deps (eval_framework ×2, neo4j_deadlock_test) and were excluded from
both runs. cognee/tests/unit/infrastructure/llm passes clean.

ruff check and ruff format --check pass on the new example.

Type of Change

  • Other: documentation + a runnable example + an Integrations Hub entry

Notes for review

The last commit, chore(aimlapi): fork-only placement — do not send upstream, is
placement only — it moves the entry to the top of the three hand-ordered provider
lists. It is kept separate so it can be dropped cleanly; provider ordering is a
maintainer decision, not a contributor one. The catalog's own ordering machinery
(sorted by kind then id in catalog/loader.py) and the test_llms.yml job
mapping are untouched, because neither order is editorial.

Not included, and why: a test-aimlapi job for .github/workflows/test_llms.yml,
mirroring test-openrouter one-for-one, was written and then left out — the
credential this branch was pushed with has no workflow scope, so GitHub refuses
any push that touches a workflow file. It is a ~28-line job and should be added
in a follow-up by someone who can push workflows; it also needs a repo secret,
AIMLAPI_API_KEY, provisioned by a maintainer, the same way OPENROUTER_API_KEY
is for the job it would sit beside.

One more note added after the first push: litellm's own env var for this provider
is AIML_API_KEY, and cognee never reads it — it passes api_key to litellm
explicitly, so a user who sets only AIML_API_KEY (following litellm's docs) gets
LLMAPIKeyNotSetError. Verified, and called out in .env.template and CLAUDE.md.

cognee dispatches every completion through litellm, and litellm has shipped a
first-class `aiml` provider (AIMLChatConfig, base https://api.aimlapi.com/v1)
since well before the version this repo pins. So aimlapi.com already works with
zero adapter code -- it was simply undiscoverable, because nothing in the repo
said so. This adds the same three surfaces the OpenRouter route already has
(.env.template block, CLAUDE.md snippet, integrations skill bullet), a runnable
example, and the Integrations Hub card.

Two facts here are not guessable and are the reason a user gives up:

- `LLM_PROVIDER` must be "custom". An "aiml/" model prefix on its own raises
  ProviderNotDeducibleError, because cognee's provider enum is deliberately
  small -- the same reason OpenRouter needs "custom".
- Embeddings must NOT carry the "aiml/" prefix. litellm registers no embedding
  route for `aiml`, so "aiml/openai/text-embedding-3-large" fails with
  "Unmapped LLM provider for this endpoint". The unprefixed id plus
  EMBEDDING_ENDPOINT pointed at aimlapi.com works, and that is what the example
  uses.

The example uses EMBEDDING_MAX_COMPLETION_TOKENS rather than the
EMBEDDING_MAX_TOKENS spelling most of test_llms.yml uses: EmbeddingConfig
declares `embedding_max_completion_tokens`, and nothing reads
`embedding_max_tokens`, so the shorter spelling is silently absorbed by
`extra="allow"` and has no effect.

Verified against the live API: the example runs end to end (remember + recall)
with both the LLM and the embeddings served by aimlapi.com, and every model id
named here was called through cognee's own GenericAPIAdapter, not asserted from
the catalogue.
Moves the aimlapi.com entry to the top of the three hand-ordered provider lists
(.env.template TIER 4, CLAUDE.md's LLM Provider Configuration, and the
cognee-integrations skill bullet list) and adds it first to CLAUDE.md's
"Supported providers" sentence. Placement only — no content change.

Kept separate so it can be dropped before an upstream PR, where provider order
is the maintainers' call.

Two lists are deliberately untouched because their order is machine-decided,
not editorial: catalog entries are sorted by kind then id in catalog/loader.py,
and the test_llms.yml jobs are an unordered GitHub Actions mapping that runs in
parallel.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant