Skip to content

feat(llm): add aimlapi.com as an LLM provider - #1

Open
Lookoff-AIMLAPI wants to merge 4 commits into
mainfrom
feat/aimlapi-provider
Open

feat(llm): add aimlapi.com as an LLM provider#1
Lookoff-AIMLAPI wants to merge 4 commits into
mainfrom
feat/aimlapi-provider

Conversation

@Lookoff-AIMLAPI

Copy link
Copy Markdown
Member

What this does

Adds aimlapi.com as a selectable LLM provider (machine id aimlapi), following
the shape of AIHubMix — the most recently added provider — as closely as the
differences allow.

A user pastes one key and reaches 353 chat models from OpenAI, Anthropic, Google,
DeepSeek, Alibaba, xAI, Meta and Mistral through the existing provider combo, with
no per-vendor account. Chat, streaming, the Fetch-models button, tool calling and
Alpha Arena all work.

Files

File Change
services/llm/ProviderCatalog.{h,cpp} provider row (id, display name, base URL, brand colour, starter models) + new attribution_headers()
services/llm/LlmService.{h,cpp} streaming allow-list; merge attribution headers in get_headers()
services/llm/LlmModelsApi.cpp models-URL fallback, type-filtered catalogue parse, attribution on the Fetch request
services/llm/ModelCatalog.cpp wildcard output cap
services/llm/LlmRequestBuilders.cpp comment only — records why this provider is not in the max_completion_tokens list
services/alpha_arena/ArenaLlmClient.cpp attribution on the arena request path; reasoning-token accounting fix
tests/tst_provider_catalog.cpp, tests/CMakeLists.txt new unit suite (15 cases, 1 extra executable, 2 extra TUs)

Why some of it is not a copy of the AIHubMix row

The catalogue is one list for every modality. GET /v1/models returns 936 rows
for 353 chat models — the rest are image, video, TTS, speech-to-text, embeddings
and batch targets that answer /chat/completions with a 404. parse_models_response
filters on type == "openai/chat-completions". 71 ids appear on more than one row
under different types, so that filter also de-duplicates: the 353 chat rows carry
353 distinct ids. type is in the default response, so the Fetch button does not
pay for ?include=all (that flag adds pricing/capabilities/modalities and takes the
payload from ~0.5 MB to ~1.6 MB for fields the model combo never reads).

Call attribution. OpenRouter's HTTP-Referer/X-Title pair was hardcoded inline
in LlmService::get_headers. This adds ProviderCatalog::attribution_headers() so
the same idea is table-driven, and wires it into all three places the app originates
a request: chat, the Fetch button, and Alpha Arena — which builds its own
QNetworkRequest and never goes through LlmService. The helper:

  • returns a fresh map per call — no shared mutable constant to clobber;
  • is merged into the caller's headers and never assigned over them;
  • returns nothing at all unless base_url still resolves to api.aimlapi.com,
    so a row left on this provider but repointed at a proxy or another vendor cannot
    carry the identifiers off-site.

HTTP-Referer and X-Title name this project (https://fincept.in /
Fincept Terminal), matching the OpenRouter convention, not the gateway.

Model ids

Twelve starter ids. Every one of them was called live against
/v1/chat/completions on 2026-09-03 and returned HTTP 200 with a non-empty
completion — not merely looked up in /v1/models. That distinction matters here:
the catalogue omits ids that serve traffic (flux-pro/v1.1 is in neither id nor
aliases and returns 200) and publishes at least one that does not work
(meta-llama/llama-3.3-70b-versatile is listed as a chat model with an alias and
404s on both spellings). Catalogue membership is a necessary pre-filter, never proof.

Where the vendor publishes both a dotted and a dashed spelling the dotted one is
used — anthropic/claude-sonnet-4.6, not -4-6. x-ai/grok-4-6 and
openai/gpt-5-5 have no dotted twin in the catalogue and are correct as written.
Several ids echo back a different string than they were sent
(openai/gpt-5-5gpt-5.5-2026-04-23, x-ai/grok-4-6grok-4.6), which is
normal aliasing, but it is why the ids here were chosen by probing rather than by
copying another integration's array.

The full list is still one Fetch click away; this is only what the combo shows
before the user fetches.

Two traps this provider does not fall into, recorded in comments

  • max_tokens vs max_completion_tokens. AIHubMix is a pass-through and needs
    max_completion_tokens for OpenAI-family models. This gateway translates the
    parameter itself — openai/gpt-5-5 with max_tokens returns 200 — so it is
    deliberately absent from that list in apply_openai_token_limit.
  • Null-valued optional fields. This gateway rejects null on temperature,
    top_p, seed, tools, tool_choice, response_format, stream,
    max_tokens and others with a 400, and which fields it rejects varies per model —
    tools: null is the worst of them, because a host that clears tools between turns
    by nulling the field succeeds on turn 1 and 400s on turn 2 of every agent loop.
    build_openai_request already builds bodies by omitting unset keys rather than
    setting them to null, so the whole class of failure does not arise. Nothing was
    changed for this; it is called out so it stays that way.

max_tokens is not a spend ceiling here. On some reasoning models the billed
reasoning tokens run well past the value with finish_reason: "stop" — no signal at
all. ModelCatalog.cpp says so next to the entry, so nobody later presents that
control to a user as a cost cap.

The arena token-accounting commit

ArenaStore::token_totals and the decision log both bill a round as
prompt_tokens + completion_tokens. That assumes completion_tokens covers
everything generated, and on reasoning models it does not — and the split is
per-route, not per-vendor. Measured through one gateway with one request shape:

model prompt completion reasoning total
google/gemini-2.5-pro 11 1 57 69
x-ai/grok-4-6 650 1 94 745
openai/gpt-5-5 21 16 6 (inside completion) 37
deepseek/deepseek-v4-flash 95 24 22 (inside completion) 119

So the arena was reporting a reasoning agent's usage as a fraction of the work
actually billed for it, silently. Taking the shortfall from total_tokens is
provider-agnostic, needs no per-model table, and is a no-op wherever the two figures
already agree. It is a separate commit and can be dropped or split out if you would
rather see it on its own.

There is no cost or pricing display anywhere in the LLM or arena code — only
token counts — so nothing here reads the catalogue's pricing block. Worth knowing
if one is ever added: 227 of the 353 chat rows repeat their output-token price in
pricing.units[] and label every unit measure: "output", including the input
one, so a naive sum over that array roughly doubles the number.

How it was verified

Build. Configured with -DFINCEPT_BUILD_TESTS=ON, Ninja, Release, macOS arm64,
AppleClang 21, Qt 6.11.2 via -DFINCEPT_QT_PIN_MODE=ANY (Qt 6.8.3 was not available
on the build host — this is the documented escape hatch, not a change to the pin).
Full FinceptTerminal target: exit 0, 0 errors, before and after. Exit codes read
from the bare command, not through a pipe.

Tests. Baseline on a pristine checkout of this branch's merge base, then again
with the change:

executables test cases passed failed
baseline (73e1929) 3 52 52 0
with this change 4 67 67 0

ctest exit 0 both times. The 15 new cases are all in the new suite; no existing
case changed behaviour.

--selftest-llm-tools (the in-app provider/tool-format self-test) reports PASS
before and after. Its section 7 enumerates known_providers() and asserts every
provider resolves an OpenAI-shaped endpoint with the base_url field cleared, so this
provider picked up that coverage automatically.

Live inference through the added code path — not curl, not a mock. A harness
linking the real ProviderCatalog.cpp from this tree, using chat_endpoint() and
attribution_headers() verbatim plus the header-merge loop from get_headers() and
the body shape from build_openai_request(), over Qt's own QNetworkAccessManager:

display_name  : aimlapi.com
endpoint      : https://api.aimlapi.com/v1/chat/completions
header        : Authorization: Bearer <redacted>
header        : HTTP-Referer: https://fincept.in
header        : X-AIMLAPI-Partner-ID: part_finceptterminal
header        : X-AIMLAPI-Source: agent/finceptterminal
header        : X-Title: Fincept Terminal
request_body  : {"max_tokens":128,"messages":[...],"model":"anthropic/claude-sonnet-4.6"}
http_status   : 200
model_echo    : anthropic/claude-sonnet-4.6
finish_reason : stop
content       : A bond coupon is the periodic interest payment made to a bondholder,
                expressed as a percentage of the bond's face value.
usage         : {"completion_tokens":30,"prompt_tokens":30,"total_tokens":60}

Tool calling through the same path, openai/gpt-5-5:

http_status   : 200
finish_reason : tool_calls
tool_calls    : [{"function":{"arguments":"{\"symbol\":\"AAPL\"}","name":"get_quote"},
                 "id":"call_kDMeF8aZ5KZ3MLMiQTiC3UvR","type":"function"}]

Streaming was confirmed separately against /v1/chat/completions with
"stream": true — standard data: {...} SSE chunks with choices[].delta.content,
on both an OpenAI-family and an Anthropic-family model, which is what
provider_supports_streaming promises.

Not verified: the Settings and Alpha Arena screens were not driven by hand — this
was built and exercised on a headless host. The code those screens call is covered by
the unit suite and by the live call above, but nobody has clicked the combo.

Placement

The last commit, chore(aimlapi): fork-only placement — do not send upstream, is
separated so it can be dropped with a single revert.

It moves this provider to the head of ProviderCatalog::known_providers() — the only
provider list in the repo that is hand-ordered rather than sorted. The two lists a
user actually sees are left alone on purpose:
LlmConfigSection::providers_sorted()
and the Alpha Arena wizard both sort by display name at render time, and biasing a
general comparator toward one vendor is not a change this project should carry. The
repo has no featured/badge mechanism to hook into either — the nearest thing is the
literal (recommended) suffix inside Fincept's own display name, and this provider's
label has to stay the bare domain — so none was invented.

Scope

.github/CONTRIBUTING.md requires every PR to link an issue carrying
good-first-issue, help-wanted or scope:approved. No such issue is linked
here
, because this PR targets our own fork rather than upstream. Anyone taking this
upstream must open that issue and get it labelled first; the PR Scope Gate workflow
closes unlabelled PRs after 7 days.

No formatter was run over any file. The only reformatted lines are the
known_providers() initialiser, which clang-format re-wraps because the list grew by
one element. No docs were touched — the same policy file forbids unsolicited
docs/* changes, and README.md's provider list already omits several shipped
providers, so adding only this one there would be out of place.

Fincept already speaks to a dozen OpenAI-compatible backends, but every
aggregator it ships (AIHubMix, AstraFlow) publishes a different slice of the
model market. aimlapi.com routes 353 chat models — OpenAI, Anthropic, Google,
DeepSeek, Qwen, xAI, Mistral — through one /v1/chat/completions endpoint, so a
user with one key reaches all of them from the existing provider combo without
a per-vendor account.

Follows the AIHubMix shape exactly, which is the last provider added: one row
per catalogue function in ProviderCatalog, a models-URL fallback for when the
prefilled base_url has been cleared, a wildcard output cap, and an entry in the
streaming allow-list. The provider id stays `aimlapi`; the label users see is
the vendor's own name for itself, the bare domain.

Two things are not copies of the AIHubMix row and are worth the reviewer's eye:

* The catalogue is one list for every modality — 936 rows for 353 chat models —
  so parse_models_response filters on `type`. Without it the model combo fills
  with image, video, TTS and speech-to-text ids that answer /chat/completions
  with a 404. That field is in the default response, so the Fetch button does
  not pay for ?include=all.

* Call attribution. OpenRouter's HTTP-Referer/X-Title pair was hardcoded inline
  in LlmService::get_headers; this adds ProviderCatalog::attribution_headers()
  so the same idea is table-driven, and wires it into all three places the app
  originates a request (chat, the Fetch button, and Alpha Arena, which builds
  its own QNetworkRequest and never goes through LlmService). It is merged into
  the caller's header map, never assigned over it, returns a fresh map per call,
  and returns nothing at all unless base_url still resolves to the provider's
  own host — a row left on this provider but repointed at a proxy must not carry
  the identifiers off-site.

The starter model list is deliberately short and every id in it was verified
with a live POST to /v1/chat/completions, not just looked up in /v1/models: that
catalogue omits ids that serve traffic and publishes at least one that 404s, so
membership in it is not evidence either way.

New unit suite tst_provider_catalog covers the seam. The partner id assertion in
it exists because a malformed id is dropped silently by the gateway — the request
still succeeds and nothing anywhere reports the loss, so a regex in a test is the
only place a typo can ever be caught.
ArenaStore::token_totals and the decision log both bill a round as
prompt_tokens + completion_tokens. That assumes completion_tokens covers
everything the model generated, and on reasoning models it does not — the split
is per-route, not per-vendor.

Measured against one gateway with one request shape: a one-word reply from
gemini-2.5-pro came back as completion_tokens=1, reasoning_tokens=57,
total_tokens=69, and grok-4.6 as completion_tokens=1, reasoning_tokens=94,
total_tokens=745 — while gpt-5.5 and deepseek-v4 fold reasoning into
completion_tokens and reconcile exactly. So the arena was reporting a reasoning
agent's usage as a small fraction of the work actually billed for it, with no
error and nothing on screen to suggest the number was wrong.

Taking the shortfall from total_tokens is provider-agnostic and a no-op wherever
the two already agree, so it needs no per-model table to keep up to date.

Separable from the provider change it ships with, if the reviewer would rather
see it on its own; it is here because that route is where the discrepancy showed
up.
Moves aimlapi.com to the head of ProviderCatalog::known_providers(), the one
provider list in the repo that is ordered by hand rather than sorted.

Deliberately narrow. The two lists a user actually sees — the Settings provider
combo via LlmConfigSection::providers_sorted(), and the Alpha Arena wizard's
provider pane — both sort by display name at render time, and that sorting is
left untouched: reordering a provider there would mean special-casing one vendor
inside a general comparator, which is not a change this project should carry.
The repo has no featured/badge mechanism to hook into either (the only thing
resembling one is the literal "(recommended)" suffix inside Fincept's own
display name, and this provider's label has to stay the bare domain), so none is
invented here.

Separated into its own commit so it can be dropped with a single revert before
this work is offered anywhere outside our fork.
The placeholder part_finceptterminal was a readable stand-in chosen before the
partner was registered. Registration mints the id server-side, so the
real value is part_7BsLIBzelgOXobyArFlFtmup. A wrong or unknown partner id is accepted with a
200 and silently not attributed, so this would not have surfaced at runtime.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant