Skip to content

feat(providers): add aimlapi.com as an OpenAI-compatible gateway - #1

Open
Lookoff-AIMLAPI wants to merge 3 commits into
mainfrom
feat/aimlapi-provider
Open

feat(providers): add aimlapi.com as an OpenAI-compatible gateway#1
Lookoff-AIMLAPI wants to merge 3 commits into
mainfrom
feat/aimlapi-provider

Conversation

@Lookoff-AIMLAPI

Copy link
Copy Markdown
Member

Summary

Adds aimlapi.com as an OpenAI-compatible LLM gateway, following the wiring
shape already used for Requesty (issue RightNow-AI#995) and Novita (RightNow-AI#928): a base-URL
constant, a provider_defaults() arm, a ProviderInfo row, catalog model rows,
a metering fallback and an .env.example line. No new driver file — OpenFang
routes every OpenAI-compatible vendor through the shared tables, and this
follows that rather than adding a second shape to maintain.

Base URL https://api.aimlapi.com/v1, key env AIMLAPI_API_KEY, chat at
{base}/chat/completions. There is no {base}/completions endpoint on this
gateway (it 404s); the existing chat_url() never derives one, and the base-URL
constant carries a comment saying so.

Changes

File What
crates/openfang-types/src/model_catalog.rs AIMLAPI_BASE_URL
crates/openfang-runtime/src/drivers/mod.rs provider_defaults() arm; origin-scoped attribution headers; tests
crates/openfang-runtime/src/drivers/openai.rs with_extra_headers appends instead of assigning; tests
crates/openfang-runtime/src/model_catalog.rs ProviderInfo row + 5 model rows; tests
crates/openfang-kernel/src/metering.rs gateway cost arms + a catalog/fallback drift test
crates/openfang-cli/src/tui/screens/init_wizard.rs openfang init entry
crates/openfang-api/static/js/pages/settings.js aggregator categorisation
.env.example commented key line

Attribution headers

Partner headers (X-AIMLAPI-Partner-ID, X-AIMLAPI-Source, plus HTTP-Referer
and X-Title naming OpenFang, not the gateway) are keyed on the request
origin, not on the configured provider name. Any provider in OpenFang can be
pointed at any base_url, so a name-keyed check would let partner headers ride
a request to a different vendor. Tests cover OpenRouter, OpenAI, localhost, a
proxy path containing aimlapi, and the look-alike host
api.aimlapi.com.example.net — all get no headers.

OpenAIDriver::with_extra_headers previously assigned the header vector.
The only existing caller is Copilot IDE auth, which calls it once, so the
behaviour was correct but not composable: adding a second caller would have
silently dropped the first one's headers. It now appends, with a test.

The partner id is asserted against ^part_[A-Za-z0-9]{1,64}$ in a unit test.
This is not decoration: the gateway accepts a malformed id and then ignores it,
so a typo produces no error anywhere — it just silently stops earning
attribution. A test is the only thing that catches it.

Cost rates

Catalog rates are the gateway's rates, not the upstream vendors' direct
rates, taken from pricing.units[] in GET /v1/models?include=all. Without an
explicit arm, aimlapi/anthropic/claude-sonnet-4.6 would fall through to the
generic contains("sonnet") pattern and be billed at $3/$15 instead of
$4.13/$20.63 — a ~35% understatement. The metering arms are scoped to the
aimlapi/ prefix so the bare upstream spelling is untouched, and
test_aimlapi_fallback_rates_match_catalog fails if the two tables ever drift.

Caveat worth knowing: max_tokens does not reliably bound reasoning
tokens on this gateway. Some reasoning models return well over the cap and still
report finish_reason: "stop", so a max-tokens setting is not a cost bound.

Model selection

Five models. Each was checked against GET /v1/models?include=all as an id or
an alias, and then actually called — the catalog alone is not sufficient
proof in either direction (it omits ids that serve traffic, and publishes at
least one id that 404s).

Catalog id Wire id after strip_provider_prefix in/out $/M
aimlapi/anthropic/claude-sonnet-4.6 anthropic/claude-sonnet-4.6 4.13 / 20.63
aimlapi/openai/gpt-5-5 openai/gpt-5-5 6.50 / 39.00
aimlapi/google/gemini-2.5-flash google/gemini-2.5-flash 0.39 / 3.25
aimlapi/alibaba/qwen-max alibaba/qwen-max 2.08 / 8.32
aimlapi/meta-llama/Llama-3.3-70B-Instruct-Turbo meta-llama/Llama-3.3-70B-Instruct-Turbo 1.144 / 1.144

aimlapi/openai/gpt-5-5 echoes back gpt-5.5-2026-04-23 in the response
model field rather than the id that was requested. That is the gateway
resolving a floating id to a dated snapshot; the other four echo the id
unchanged. Nothing in OpenFang pins on the echoed value today.

max_output_tokens for Llama 3.3 70B is set to 32,768 rather than the 127,000
the catalog reports, which is all but the whole 128k context window and not a
usable output bound.

Testing

  • cargo clippy --workspace --all-targets -- -D warnings — see note below
  • cargo test --workspace passes
  • Live integration tested

Tests: baseline on a pristine checkout of this branch point was
2699 passed / 0 failed across 40 test binaries. After this PR:
2711 passed / 0 failed (+12 new tests). cargo build --workspace --lib
exits 0.

Clippy: this PR adds zero clippy findings — verified by running
cargo clippy --workspace --all-targets -- -D warnings with only the
pre-existing lint categories allowed, which exits 0. The unmodified
-D warnings command does not pass on this tree with Rust 1.98, for seven
findings in files this PR does not touch:

crates/openfang-memory/src/semantic.rs:503
crates/openfang-channels/src/feishu.rs:1052
crates/openfang-runtime/src/drivers/bedrock.rs:497
crates/openfang-runtime/src/embedding.rs:290
crates/openfang-api/src/routes.rs:580
crates/openfang-api/src/routes.rs:9530
crates/openfang-api/src/openai_compat.rs:318
crates/openfang-api/src/channel_bridge.rs:515

The lint categories involved are chunks_exact_to_as_chunks, question_mark,
drain_collect, useless_format, for_kv_map,
needless_borrows_for_generic_args and useless_borrows_in_formatting.

These are newer lints that landed in the toolchain after this code was written;
they are pre-existing and out of scope here.

Live calls: driven through create_driver("aimlapi")OpenAIDriver::complete(),
i.e. the code path this PR adds, not curl and not a mock. All five models
returned 200 with stop_reason: EndTurn:

COMPLETE aimlapi/google/gemini-2.5-flash                    -> text="OK" stop=EndTurn in=5  out=1
COMPLETE aimlapi/anthropic/claude-sonnet-4.6                -> text="OK" stop=EndTurn in=12 out=4
COMPLETE aimlapi/openai/gpt-5-5                             -> text="OK" stop=EndTurn in=11 out=16
COMPLETE aimlapi/alibaba/qwen-max                           -> text="OK" stop=EndTurn in=13 out=1
COMPLETE aimlapi/meta-llama/Llama-3.3-70B-Instruct-Turbo    -> text="OK" stop=EndTurn in=40 out=2

Tool calling exercised through the same path:

TOOLS aimlapi/google/gemini-2.5-flash     -> stop=ToolUse tool_calls=[("get_weather", {"city":"Warsaw"})]
TOOLS aimlapi/anthropic/claude-sonnet-4.6 -> stop=ToolUse tool_calls=[("get_weather", {"city":"Warsaw"})]

The harness that drove these was temporary and is not part of the diff.

Null-vs-omitted request fields

This gateway rejects an explicit null with a 400 on temperature, top_p,
seed, tools, tool_choice, response_format, stream, stream_options,
parallel_tool_calls, max_tokens and max_completion_tokens for at least
some models — and which models are strict varies per model, so one green live
call proves nothing about the rest. tools: null is the dangerous one: a host
that clears tools between turns by nulling the field succeeds on turn one and
400s on turn two of every agent loop.

OaiRequest already had skip_serializing_if on every optional field, so
OpenFang is not affected. test_unset_request_fields_are_omitted_not_null
locks that in, because the failure mode is a green test suite with 400s on every
real call.

Security

  • No new unsafe code
  • No secrets or API keys in diff
  • User input validated at boundaries

The origin check parses base_url with reqwest::Url and compares
host_str() case-insensitively, so a path segment or a suffixed domain cannot
impersonate the origin.

Fork-only commit

The final commit, chore(aimlapi): fork-only placement — do not send upstream,
is presentation only — it moves aimlapi.com to the head of the two hand-ordered
provider lists and names it in the settings page's aggregator group label. It is
isolated so it can be dropped before any upstream submission. The web settings
page sorts within a category (configured first, then alphabetical) and the API
returns the table unsorted; that ordering machinery was left alone.

docs/providers.md was not updated. It is a hand-numbered list that stops at 20
providers and already omits Requesty, Novita, MiniMax, Qwen, Zhipu, Venice,
Chutes and NVIDIA — neither of the two most recent provider PRs touched it, and
adding a 21st entry to a list missing 22 others would be inconsistent rather
than helpful.

OpenFang already routes every OpenAI-compatible vendor through the shared
driver tables, so a gateway needs a base-URL constant, a provider-defaults
arm, a ProviderInfo row and a handful of catalog rows rather than a new
driver file. This follows the Requesty (issue RightNow-AI#995) and Novita wiring
exactly so there is one shape to maintain, not two.

The five catalog rows are gateway-priced rather than upstream-priced: the
same model costs a different amount through a router than direct, and
letting `aimlapi/anthropic/claude-sonnet-4.6` fall through to the generic
`contains("sonnet")` arm would silently bill it at Anthropic's own rate.
The metering fallback carries matching arms scoped to the `aimlapi/`
prefix, with a test that fails if those rates ever drift from the catalog.

Attribution headers are keyed on the request *origin*, not on the
configured provider name. A user can point any provider at any base_url,
and partner headers must never ride a request to a different vendor or to
a third-party proxy that merely fronts the same API. `with_extra_headers`
now appends instead of assigning, so attribution cannot silently discard
headers an earlier caller configured (Copilot's IDE auth is the existing
caller).

The partner id shape is asserted in a unit test because a malformed id is
accepted by the gateway and then ignored — it fails silently, earning
nothing, with no runtime error to notice.
Moves aimlapi.com to the head of the two hand-ordered provider lists (the
builtin ProviderInfo table and the `openfang init` wizard) and names it in
the aggregator group label on the settings page.

This is presentation only and carries no functional change, which is why
it is isolated here: it reflects our own preference, not the project's,
and should be dropped before the provider wiring is offered upstream.

The web settings page orders providers within each category by
configured-first then alphabetically, and the API returns the table
unsorted; that ordering machinery is left untouched.
The placeholder part_openfang was a readable stand-in chosen before the
partner was registered. Registration mints the id server-side, so the
real value is part_Z6HbToJ3l2ht1dFSzHC98vk6. A wrong or unknown partner id is accepted with a
200 and silently not attributed, so this would not have surfaced at runtime.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant