fix(aiml): attribute aimlapi.com traffic, read its documented key env var, and repair its cost map - #1
Open
Lookoff-AIMLAPI wants to merge 5 commits into
Conversation
…nv var Chat completions for the aiml provider go out through the OpenAI SDK path, which never calls the provider config's validate_environment, so the provider had no way to put anything on the wire beyond the api base and the key. Every request therefore reached the provider untagged, and the provider could not tell LiteLLM traffic apart from anything else. Merging the headers in the chat dispatch is the only spot on that path where both the provider and the resolved api base are known. The headers are keyed on the request host, not on the provider name, so they cannot ride along to a different backend when someone points AIML_API_BASE at their own gateway, and caller supplied headers still win on a key clash. The provider documents AIMLAPI_API_KEY everywhere else, while LiteLLM has only ever read AIML_API_KEY. Renaming would break existing configs, so AIML_API_KEY stays primary and AIMLAPI_API_KEY joins the AIMLAPI_KEY fallback the image generation config already had.
… pricing aiml/dall-e-2 and aiml/dall-e-3 are gone. POSTing either to /v1/images/generations comes back 404 with "Model 'openai/dall-e-3' is no longer available (retired 2026-06-09)", so shipping cost entries for them only tells a user the ids are supported when they are not. flux-pro/v1.1-ultra was priced at $0.063 an image, but a real generation reports usd_spent 0.078, so every ultra call was under counted by a quarter. The provider's catalog now lists that pair under blackforestlabs prefixed ids, which route to the same models at the same prices, so both spellings are carried: the old ones still work and removing them would silently drop existing users to $0 spend. Checked by POSTing each id to /v1/images/generations with no prompt, which answers 400 for a model that exists and 404 for one that does not. Worth knowing for anyone repeating this: flux-pro/v1.1 and flux-pro/v1.1-ultra appear in neither the id nor the aliases list of GET /v1/models, yet both still serve traffic, so the catalog alone is not enough to call an id dead.
The provider's product, docs and billing all say aimlapi.com, so "AI/ML API" in the model add form, the credential picker, the public endpoint support response and the README table is a name a user has to translate before recognising it. The machine identifier stays aiml, so nobody's config changes.
Puts aimlapi.com first in the hand ordered lists a user actually reads: the Admin UI provider dropdowns, which render in enum declaration order, the provider table in provider_endpoints_support.json, and the README provider table. The provider create form already listed it first, so that file is untouched here. Placement only, no functional change. Drop this commit before sending anything from this branch upstream.
The placeholder part_litellm was a readable stand-in chosen before the partner was registered. Registration mints the id server-side, so the real value is part_O0eykPA6gQNIFEYaUBojBbU4. A wrong or unknown partner id is accepted with a 200 and silently not attributed, so this would not have surfaced at runtime.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
TLDR
Problem this solves:
How it solves it:
User Flow
Before: a developer routing an aimlapi.com model through the proxy cannot be recognised by the provider, cannot use the key env var the provider's own docs give them, and is billed short on one image model
model: aiml/openai/gpt-5-5in the config, and start the proxyaiml/flux-pro/v1.1-ultra, get an image back, and http://localhost:4000/ui/?page=logs shows $0.063 while the provider charged $0.078aiml/dall-e-3from the models LiteLLM lists for the provider and get a 404 saying the model was retired on 2026-06-09After: the same developer's key works under the name they were given, their requests are recognisable, and the ultra spend matches the invoice
model: aiml/openai/gpt-5-5in the config, and start the proxyaiml/flux-pro/v1.1-ultra, get an image back, and the logs page shows $0.078, matching the chargeaiml/dall-e-3is no longer among the models LiteLLM lists for the provider, so they never pick a retired idIf they point AIML_API_BASE at their own gateway instead of api.aimlapi.com, the four headers are not sent, so the attribution never rides to a host that did not earn it.
Relevant issues
Linear ticket
Pre-Submission checklist
uv run pytest tests/test_litellm/<your_test_file>.py -v. Leave the suites (make test-unit-*,make test-unit) to CI: it finishes in ~15 minutes where a laptop takes an hour or more@greptileaito re-request a review after pushing changes)Screenshots / Proof of Fix
Shared setup for every case below, run from the repo root on each side:
The proxy is started with the local cost map so the checked in prices are the ones under test, otherwise LiteLLM downloads the published map and the pricing case measures nothing:
Before (d2fe8af)
attribution headers on the outgoing request
curl -s -X POST http://localhost:4000/v1/chat/completions -H "Authorization: Bearer sk-1234" -H "Content-Type: application/json" -d '{"model":"aiml-gpt-5-5","messages":[{"role":"user","content":"What is the capital of France? One word."}],"max_tokens":400}'answers 200 with
"content":"Paris"grep -A 4 "POST Request Sent from LiteLLM" litellm.log | tail -1nothing identifies the caller
the provider's own api key env var
Restart the proxy with
AIMLAPI_API_KEYset andAIML_API_KEYunset, everything else the sameRepeat the same chat curl
image spend for flux-pro/v1.1-ultra
curl -s -X POST http://localhost:4000/v1/images/generations -H "Authorization: Bearer sk-1234" -H "Content-Type: application/json" -d '{"model":"aiml-flux-pro-ultra","prompt":"a red square"}'answers 200 with
"url":"https://s3.aimlapi.com/files/file-01a0648a-9984-73a9-abc2-f656bdcdc6d7"grep response_cost litellm.log | tail -1the provider's own response body on the same request reports
"usage": {"credits_used": 156000, "usd_spent": 0.078}After (69b1644)
attribution headers on the outgoing request
curl -s -X POST http://localhost:4000/v1/chat/completions -H "Authorization: Bearer sk-1234" -H "Content-Type: application/json" -d '{"model":"aiml-gpt-5-5","messages":[{"role":"user","content":"What is the capital of France? One word."}],"max_tokens":400}'answers 200 with
"content":"Paris", 20 total tokensgrep -A 4 "POST Request Sent from LiteLLM" litellm.log | tail -1the provider's own api key env var
Restart the proxy with
AIMLAPI_API_KEYset andAIML_API_KEYunset, everything else the sameRepeat the same chat curl
answers 200 with
"content":"Paris", modelaiml-gpt-5-5, 31 total tokensimage spend for flux-pro/v1.1-ultra
curl -s -X POST http://localhost:4000/v1/images/generations -H "Authorization: Bearer sk-1234" -H "Content-Type: application/json" -d '{"model":"aiml-flux-pro-ultra","prompt":"a red square"}'answers 200 with
"url":"https://s3.aimlapi.com/files/file-01a06496-5104-70a8-9dd7-d1660405f92c"grep response_cost litellm.log | tail -1The retired ids were checked directly rather than through the proxy, since the point is that the provider no longer serves them. POSTing a model with no prompt answers 400 when the model exists and 404 when it does not:
The 404 body names the retirement:
Model 'openai/dall-e-3' is no longer available (retired 2026-06-09).For the display name and the ordering, open http://localhost:4000/ui/?page=llm-playground and http://localhost:4000/ui/?page=new_model, click the provider dropdown, and the first entry reads aimlapi.com. The same name shows on http://localhost:4000/ui/?page=credentials when adding a credential.
Test numbers, same machine and same environment on both sides,
uv run pytest tests/test_litellm/llms tests/test_litellm/proxy/public_endpoints --tb=no -q -n 4:The failing and erroring node ids are identical on both sides, so the 18 extra passes are the new tests and nothing regressed.
make lintexits 0.npm run buildin ui/litellm-dashboard compiles, andnpx vitest run src/components/provider_info_helpers.test.tsxpasses 65 tests.Type
🐛 Bug Fix
Caveats (if any)
Medium
completion_costlogs "This model isn't mapped yet" and the proxy records zero. The provider returnsmeta.usage.usd_spenton every chat response, so the numbers are there to use. Out of scope here, worth its own changevalidate_environmenton the chat config is never called on the OpenAI SDK path, so putting them there would look right and send nothing. If that path ever starts honouring the provider config, this movesLow
chore(aimlapi): fork-only placement, do not send upstreamreorders provider lists and carries no behavior change. It is separated so it can be droppedprovider_endpoints_support.jsonhas duplicate provider keysa2aandcharity_engineeach appear twice, so anything that parses the file into a dict silently loses one of each. Untouched here, flagging it because a naive round trip throughjson.loaddrops entriesFinal Attestation