Skip to content

feat(providers): add aimlapi.com as a gateway - #1

Open
Lookoff-AIMLAPI wants to merge 4 commits into
mainfrom
feat/aimlapi-provider
Open

feat(providers): add aimlapi.com as a gateway#1
Lookoff-AIMLAPI wants to merge 4 commits into
mainfrom
feat/aimlapi-provider

Conversation

@Lookoff-AIMLAPI

Copy link
Copy Markdown
Member

Description

Adds aimlapi.com (AI/ML API) as an OpenAI-compatible gateway, following the
shape of PR HKUDS#159 (Requesty / MiniMax / Forge) exactly: a ProviderSpec, the
matching ProvidersConfig field, an attribution-header block next to the
OpenRouter and Requesty ones, and a test file mirroring
tests/test_requesty_provider.py + tests/test_forge_provider.py.

No new backend is needed — backend="openai_compat", is_gateway=True.

Related Issues

None.

Changes Made

74fea880feat(providers): add aimlapi.com as a gateway

  • core/providers/registry.py — one ProviderSpec
    (name="aimlapi", display_name="aimlapi.com", env_key="AIMLAPI_API_KEY",
    default_api_base="https://api.aimlapi.com/v1", is_gateway=True,
    supports_prompt_caching=True).
  • core/config.py — the matching ProvidersConfig field. ProvidersConfig is
    read via getattr(providers, spec.name), so a spec without a field is
    silently skipped; both halves are required.
  • core/providers/openai_compat.py_DEFAULT_AIMLAPI_HEADERS and
    _uses_aimlapi_attribution, wired at the same call site as the OpenRouter
    and Requesty blocks.
  • tests/test_aimlapi_provider.py — registry, config and attribution.

3828e51dfix(catalog): scope aimlapi.com discovery to the chat surface

  • core/providers/catalog_service.py — two discovery fixes, both measured
    against the live directory:
    • The gateway serves one directory for every endpoint family it hosts.
      Unfiltered, GET /v1/models returned 936 rows for 785 distinct ids
      image, video, speech-to-text and embedding endpoints included, with ids
      repeating across families — so the /model picker offered 173 rows that a
      chat request rejects, several of them twice. Scoped to the chat surface it
      is 353 unique, usable models.
    • The gateway nests contextLength / outputMax / name under info
      instead of publishing OpenRouter's flat keys, so every model fell through
      to the built-in 128k/8k fallback. A 1M-context model was budgeted as 128k;
      a 32k model was also budgeted as 128k, which is the direction that
      overflows mid-turn. The nested read is placed after the flat names, so a
      provider publishing both is unaffected (covered by a test).

aae6a658chore(aimlapi): fork-only placement — do not send upstream

Placement only: moves the spec to position 0 of PROVIDERS and its config
field to the front. Every provider list DeepCode renders iterates PROVIDERS
in declaration order. No badge was invented — the repo has no
featured/recommended concept to reuse — and nothing else was reordered. It is
isolated as the last commit, with its own pinning test, so it can be dropped
whole
before the provider is offered upstream.

Attribution headers

_uses_aimlapi_attribution differs deliberately from the two helpers above it.
_uses_openrouter_attribution / _uses_requesty_attribution accept any base
URL containing the provider's name, which would also fire for a third-party
proxy that merely fronts the gateway. This one requires the resolved base
URL's host to be ours, so DeepCode's identity does not travel to someone
else's endpoint. Lookalike hosts (api.aimlapi.com.example.io,
notaimlapi.com) are covered by tests.

Headers are merged into a per-instance dict, never assigned over: a caller's
own extraHeaders win on a key clash, the module-level constant is never
mutated, and both are asserted. The partner id fails silently when
malformed — the gateway answers 200 and records the traffic as untagged — so
its shape is pinned by a regex test rather than left to be noticed in
production.

Checklist

  • Changes tested locally
  • Code reviewed
  • Documentation updated (if necessary) — no provider list in docs/; the
    registry display_name is the only label surface. The README news entry
    is written by maintainers per release, as it was for feat(providers): add Requesty, MiniMax and Forge HKUDS/DeepCode#159.
  • Unit tests added (tests/test_aimlapi_provider.py, 20 tests)

Verification

Buildpython -m build exit 0; twine check dist/* PASSED on both the
wheel and the sdist.

Testspython -m pytest -q, CPython 3.13.15, macOS:

result
baseline (main, pristine worktree) 5 failed / 1556 passed / 8 skipped
this branch 5 failed / 1575 passed / 8 skipped

The failure set is byte-identical (diff of the FAILED lines is empty) and
all five are environment, not code: they shell out to the system python3
(3.14, which has no pytest installed) or to npm.

Separately, this tree has pre-existing flaky concurrency tests — lease,
claim-ownership and scheduler cases that fail non-deterministically under load
and pass in isolation. A second pristine-main run produced 6 failures
(test_execution_coordination_integration), and branch runs produced 6 and 9.
Every extra failure passed when re-run on its own. This is not caused by the
change, but it is worth knowing before reading a red CI run.

Live inference — one real call through
core.config.make_llm_provider(...), i.e. the same path the CLI, TUI and App
Server use. Not a mock, not raw curl. Key passed via env; never written to the
tree.

provider class  : OpenAICompatProvider
base_url        : https://api.aimlapi.com/v1/
attribution     : {"HTTP-Referer": "https://github.com/HKUDS/DeepCode",
                   "X-Title": "DeepCode",
                   "X-AIMLAPI-Source": "agent/deepcode",
                   "X-AIMLAPI-Partner-ID": "part_deepcode"}

--- 1. chat ---
content         : 'DEEPCODE-AIMLAPI-OK'
finish_reason   : stop | usage: {'prompt_tokens': 21, 'completion_tokens': 9,
                                 'total_tokens': 30} | error: None

--- 2. tool calling ---
finish_reason   : tool_calls
tool_call       : get_weather {'city': 'Paris'}
usage           : {'prompt_tokens': 52, 'completion_tokens': 14, 'total_tokens': 66}

--- 3. streaming ---
content         : 'One, two, three.' | finish: stop

--- 4. model discovery ---
discovered      : 353 models
   aion-labs/aion-2.0               ctx=131072 out=32768
   aion-labs/aion-3.0               ctx=131072 out=32768
   aion-labs/aion-3.0-mini          ctx=131072 out=32768
   alibaba/glm-5.2                  ctx=1000000 out=131072

Prompt cachingsupports_prompt_caching=True is not copied from
OpenRouter, it was measured. Two identical 2,410-token requests through
OpenAICompatProvider:

call 0  usage {"prompt_tokens": 2410, "completion_tokens": 4, "total_tokens": 2414}
call 1  usage {"prompt_tokens": 2410, "completion_tokens": 4, "total_tokens": 2414,
               "cached_tokens": 2402}

Model ids — every id named in this diff or in its tests was checked against
the live catalog by id and aliases, not by id alone:
openai/gpt-4o-mini, anthropic/claude-sonnet-4.5 and alibaba/glm-5.2 are
all present as ids. No model list is hard-coded — the catalog is fetched live —
so there is no static array to go stale.

Lintruff check --ignore=E402 and ruff format --check clean at the
pinned .pre-commit-config.yaml version (v0.15.21).

Additional Notes

Not verified, and why:

  • Desktop / TUI rendering. The provider label reaches those surfaces
    through PROVIDERS (settings_service._view,
    llm_configuration_service), which is exercised by the Python tests, but the
    Electron app itself was not launched.
  • Key validation. GET /v1/models on this gateway returns 200 for a
    bogus key
    — only POST /v1/chat/completions returns 401. Nothing in this
    diff validates a key by listing models, and nothing should: DeepCode's
    test_connection already reports ok on a successful catalog fetch, so a
    typo'd key will read as a good connection until the first turn. That is
    pre-existing behaviour for every provider here and is out of scope for this
    PR.

AI/ML API is an OpenAI-compatible router over ~350 chat models, so it needs
no new backend: the ProviderSpec plus the matching ProvidersConfig field is
the whole wiring, exactly as Requesty and Forge landed in HKUDS#159. Both halves
are required — ProvidersConfig is read via getattr(providers, spec.name), so
a spec without a field is silently skipped everywhere.

The attribution block follows _DEFAULT_OPENROUTER_HEADERS /
_DEFAULT_REQUESTY_HEADERS, with one deliberate difference: the two existing
helpers accept any base URL containing the provider's name, which would also
fire for a third-party proxy that fronts the gateway. _uses_aimlapi_attribution
requires the resolved base URL's host to be ours, so DeepCode's identity does
not travel to someone else's endpoint. The partner id fails silently when
malformed — the gateway answers 200 and records the traffic as untagged — so
its shape is asserted in a test rather than left to be noticed in production.

Verified against the live gateway: chat, tool calling and streaming all
return through core.config.make_llm_provider, and cache_control markers come
back with cached_tokens in usage, which is why supports_prompt_caching is set.
Model ids were checked against the published catalog by id AND alias.
The gateway publishes one directory for every endpoint family it hosts. An
unfiltered GET /v1/models returns 936 rows for 785 distinct ids — image,
video, speech-to-text and embedding endpoints included, with ids repeating
across families — so the model picker offered 173 rows that a chat request
rejects, and offered several of them twice. Asking for the chat surface
narrows that to 353 unique, usable models.

Discovery also invented context windows for all of them: the gateway nests
its numbers under `info` rather than publishing OpenRouter's flat keys, so
every model fell through to the built-in 128k/8k fallback. A 1M-context
model was budgeted as 128k and a 32k one as 128k, which is the direction
that overflows mid-turn. The nested read is placed after the flat names, so
a provider that publishes both is unaffected.

Numbers verified against the live directory through ModelCatalogService.
Moves the aimlapi.com spec to position 0 of PROVIDERS and its ProvidersConfig
field to the front. Every provider list DeepCode renders iterates PROVIDERS in
declaration order, so this is placement, not behaviour: no badge is invented,
because the repo has no featured/recommended concept to reuse, and nothing
else is reordered.

Kept as its own commit so it can be dropped whole before the provider is
offered upstream. The test that pins position 0 lives in this commit too, so
dropping it leaves no dangling assertion.
The placeholder part_deepcode was a readable stand-in chosen before the
partner was registered. Registration mints the id server-side, so the
real value is part_CxcOejScJ3hI0O2cExWchiBy. A wrong or unknown partner id is accepted with a
200 and silently not attributed, so this would not have surfaced at runtime.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant