Skip to content

fix(aiml): attribute aimlapi.com traffic, read its documented key env var, and repair its cost map - #1

Open
Lookoff-AIMLAPI wants to merge 5 commits into
litellm_internal_stagingfrom
litellm_aimlapi_attribution_headers
Open

fix(aiml): attribute aimlapi.com traffic, read its documented key env var, and repair its cost map#1
Lookoff-AIMLAPI wants to merge 5 commits into
litellm_internal_stagingfrom
litellm_aimlapi_attribution_headers

Conversation

@Lookoff-AIMLAPI

Copy link
Copy Markdown
Member

TLDR

Problem this solves:

  • aimlapi.com requests leave LiteLLM carrying no attribution headers
  • The key env var the provider documents is never read
  • Two aiml image ids in the cost map are retired
  • flux-pro/v1.1-ultra is priced a quarter under what it costs

How it solves it:

  • Merge four headers into aiml chat and image requests
  • Key them on request host, so no other backend sees them
  • Read AIMLAPI_API_KEY as a fallback, AIML_API_KEY stays primary
  • Drop the dead ids, fix the price, add the current spellings

User Flow

Before: a developer routing an aimlapi.com model through the proxy cannot be recognised by the provider, cannot use the key env var the provider's own docs give them, and is billed short on one image model

  1. They export AIMLAPI_API_KEY, the spelling on the provider's docs and dashboard, put model: aiml/openai/gpt-5-5 in the config, and start the proxy
  2. They POST http://localhost:4000/v1/chat/completions and get back a 500 saying "The api_key client option must be set either by passing api_key to the client or by setting the AIML_API_KEY environment variable"
  3. They rename the variable to AIML_API_KEY, restart, and the same POST returns 200 with "Paris"
  4. On the provider's dashboard that request is indistinguishable from a raw curl: nothing on it says which tool sent it
  5. They POST http://localhost:4000/v1/images/generations for aiml/flux-pro/v1.1-ultra, get an image back, and http://localhost:4000/ui/?page=logs shows $0.063 while the provider charged $0.078
  6. They pick aiml/dall-e-3 from the models LiteLLM lists for the provider and get a 404 saying the model was retired on 2026-06-09

After: the same developer's key works under the name they were given, their requests are recognisable, and the ultra spend matches the invoice

  1. They export AIMLAPI_API_KEY, put model: aiml/openai/gpt-5-5 in the config, and start the proxy
  2. They POST http://localhost:4000/v1/chat/completions and get 200 with "Paris" on the first try
  3. That request reaches the provider carrying HTTP-Referer, X-Title, X-AIMLAPI-Partner-ID and X-AIMLAPI-Source, so it is attributable to LiteLLM
  4. They POST http://localhost:4000/v1/images/generations for aiml/flux-pro/v1.1-ultra, get an image back, and the logs page shows $0.078, matching the charge
  5. aiml/dall-e-3 is no longer among the models LiteLLM lists for the provider, so they never pick a retired id

If they point AIML_API_BASE at their own gateway instead of api.aimlapi.com, the four headers are not sent, so the attribution never rides to a host that did not earn it.

Relevant issues

Linear ticket

Pre-Submission checklist

  • I have added meaningful tests
  • The handful of test files covering my change pass locally, e.g. uv run pytest tests/test_litellm/<your_test_file>.py -v. Leave the suites (make test-unit-*, make test-unit) to CI: it finishes in ~15 minutes where a laptop takes an hour or more
  • My PR passes all required CI/CD checks (e.g., lint, schema.d.ts sync check, etc.)
  • My PR's scope is as isolated as possible; it only solves 1 specific problem
  • I have received a Greptile Confidence Score of at least 4/5 before requesting a maintainer review (Greptile reviews automatically once the PR is opened; only comment @greptileai to re-request a review after pushing changes)

Screenshots / Proof of Fix

Shared setup for every case below, run from the repo root on each side:

cat > /tmp/aiml.yaml <<'EOF'
model_list:
  - model_name: aiml-gpt-5-5
    litellm_params:
      model: aiml/openai/gpt-5-5
  - model_name: aiml-flux-pro-ultra
    litellm_params:
      model: aiml/flux-pro/v1.1-ultra
general_settings:
  master_key: sk-1234
EOF

The proxy is started with the local cost map so the checked in prices are the ones under test, otherwise LiteLLM downloads the published map and the pricing case measures nothing:

LITELLM_LOCAL_MODEL_COST_MAP=True AIML_API_KEY=<real key> \
  python litellm/proxy/proxy_cli.py --config /tmp/aiml.yaml --detailed_debug 2>&1 | tee litellm.log

Before (d2fe8af)

attribution headers on the outgoing request

  1. curl -s -X POST http://localhost:4000/v1/chat/completions -H "Authorization: Bearer sk-1234" -H "Content-Type: application/json" -d '{"model":"aiml-gpt-5-5","messages":[{"role":"user","content":"What is the capital of France? One word."}],"max_tokens":400}'

    answers 200 with "content":"Paris"

  2. grep -A 4 "POST Request Sent from LiteLLM" litellm.log | tail -1

    -d '{'model': 'openai/gpt-5-5', 'messages': [{'role': 'user', 'content': 'What is the capital of France? One word.'}], 'max_tokens': 400, 'extra_body': {}}'
    

    nothing identifies the caller

the provider's own api key env var

  1. Restart the proxy with AIMLAPI_API_KEY set and AIML_API_KEY unset, everything else the same

  2. Repeat the same chat curl

    {"error":{"message":"litellm.AuthenticationError: AuthenticationError: AimlException - The api_key client option must be set either by passing api_key to the client or by setting the AIML_API_KEY environment variable. Received Model Group=aiml-gpt-5-5\nAvailable Model Group Fallbacks=None","type":null,"param":null,"code":"500"}}
    

image spend for flux-pro/v1.1-ultra

  1. curl -s -X POST http://localhost:4000/v1/images/generations -H "Authorization: Bearer sk-1234" -H "Content-Type: application/json" -d '{"model":"aiml-flux-pro-ultra","prompt":"a red square"}'

    answers 200 with "url":"https://s3.aimlapi.com/files/file-01a0648a-9984-73a9-abc2-f656bdcdc6d7"

  2. grep response_cost litellm.log | tail -1

    05:53:20 - LiteLLM:DEBUG: litellm_logging.py:1716 - response_cost: 0.063
    

    the provider's own response body on the same request reports "usage": {"credits_used": 156000, "usd_spent": 0.078}

After (69b1644)

attribution headers on the outgoing request

  1. curl -s -X POST http://localhost:4000/v1/chat/completions -H "Authorization: Bearer sk-1234" -H "Content-Type: application/json" -d '{"model":"aiml-gpt-5-5","messages":[{"role":"user","content":"What is the capital of France? One word."}],"max_tokens":400}'

    answers 200 with "content":"Paris", 20 total tokens

  2. grep -A 4 "POST Request Sent from LiteLLM" litellm.log | tail -1

    -d '{'model': 'openai/gpt-5-5', 'messages': [{'role': 'user', 'content': 'What is the capital of France? One word.'}], 'max_tokens': 400, 'extra_body': {}, 'extra_headers': {'HTTP-Referer': 'https://github.com/BerriAI/litellm', 'X-Title': 'LiteLLM', 'X-AIMLAPI-Partner-ID': 'part_litellm', 'X-AIMLAPI-Source': 'agent/litellm'}}'
    

the provider's own api key env var

  1. Restart the proxy with AIMLAPI_API_KEY set and AIML_API_KEY unset, everything else the same

  2. Repeat the same chat curl

    answers 200 with "content":"Paris", model aiml-gpt-5-5, 31 total tokens

image spend for flux-pro/v1.1-ultra

  1. curl -s -X POST http://localhost:4000/v1/images/generations -H "Authorization: Bearer sk-1234" -H "Content-Type: application/json" -d '{"model":"aiml-flux-pro-ultra","prompt":"a red square"}'

    answers 200 with "url":"https://s3.aimlapi.com/files/file-01a06496-5104-70a8-9dd7-d1660405f92c"

  2. grep response_cost litellm.log | tail -1

    06:06:07 - LiteLLM:DEBUG: litellm_logging.py:1716 - response_cost: 0.078
    

The retired ids were checked directly rather than through the proxy, since the point is that the provider no longer serves them. POSTing a model with no prompt answers 400 when the model exists and 404 when it does not:

for m in dall-e-2 dall-e-3 flux-pro flux-pro/v1.1 flux-pro/v1.1-ultra flux-realism flux/dev \
         flux/kontext-max/text-to-image flux/kontext-pro/text-to-image flux/schnell \
         google/imagen-4.0-ultra-generate-001 google/nano-banana-pro openai/gpt-image-2 \
         blackforestlabs/flux-pro-1.1 blackforestlabs/flux-pro-1.1-ultra; do
  printf "%-42s %s\n" "$m" "$(curl -s -o /dev/null -w '%{http_code}' -X POST https://api.aimlapi.com/v1/images/generations \
    -H "Authorization: Bearer $AIML_API_KEY" -H 'Content-Type: application/json' -d "{\"model\":\"$m\"}")"
done
dall-e-2                                   404
dall-e-3                                   404
flux-pro                                   400
flux-pro/v1.1                              400
flux-pro/v1.1-ultra                        400
flux-realism                               400
flux/dev                                   400
flux/kontext-max/text-to-image             400
flux/kontext-pro/text-to-image             400
flux/schnell                               400
google/imagen-4.0-ultra-generate-001       400
google/nano-banana-pro                     400
openai/gpt-image-2                         400
blackforestlabs/flux-pro-1.1               400
blackforestlabs/flux-pro-1.1-ultra         400

The 404 body names the retirement: Model 'openai/dall-e-3' is no longer available (retired 2026-06-09).

For the display name and the ordering, open http://localhost:4000/ui/?page=llm-playground and http://localhost:4000/ui/?page=new_model, click the provider dropdown, and the first entry reads aimlapi.com. The same name shows on http://localhost:4000/ui/?page=credentials when adding a credential.

Test numbers, same machine and same environment on both sides, uv run pytest tests/test_litellm/llms tests/test_litellm/proxy/public_endpoints --tb=no -q -n 4:

  • d2fe8af: 11 failed, 11085 passed, 82 skipped, 2 errors
  • 69b1644: 11 failed, 11103 passed, 82 skipped, 2 errors

The failing and erroring node ids are identical on both sides, so the 18 extra passes are the new tests and nothing regressed. make lint exits 0. npm run build in ui/litellm-dashboard compiles, and npx vitest run src/components/provider_info_helpers.test.tsx passes 65 tests.

Type

🐛 Bug Fix

Caveats (if any)

Medium

  • Chat requests to this provider still record $0 spend
    • no aiml chat model has a cost entry, only image models, so completion_cost logs "This model isn't mapped yet" and the proxy records zero. The provider returns meta.usage.usd_spent on every chat response, so the numbers are there to use. Out of scope here, worth its own change
  • The headers ride on the chat dispatch, not on the provider config
    • validate_environment on the chat config is never called on the OpenAI SDK path, so putting them there would look right and send nothing. If that path ever starts honouring the provider config, this moves

Low

  • The PR solves more than one problem
    • the attribution fix, the key env var and the cost map corrections are separate concerns kept in separate commits. Splitting them into separate PRs is easy if you would rather
  • The last commit is placement only
    • chore(aimlapi): fork-only placement, do not send upstream reorders provider lists and carries no behavior change. It is separated so it can be dropped
  • provider_endpoints_support.json has duplicate provider keys
    • a2a and charity_engine each appear twice, so anything that parses the file into a dict silently loses one of each. Untouched here, flagging it because a naive round trip through json.load drops entries

Final Attestation

  • The tests check the right things, including the edge cases, and regressions in the respective real-world customer use-cases are not possible after this PR

…nv var

Chat completions for the aiml provider go out through the OpenAI SDK path,
which never calls the provider config's validate_environment, so the provider
had no way to put anything on the wire beyond the api base and the key. Every
request therefore reached the provider untagged, and the provider could not
tell LiteLLM traffic apart from anything else. Merging the headers in the chat
dispatch is the only spot on that path where both the provider and the resolved
api base are known.

The headers are keyed on the request host, not on the provider name, so they
cannot ride along to a different backend when someone points AIML_API_BASE at
their own gateway, and caller supplied headers still win on a key clash.

The provider documents AIMLAPI_API_KEY everywhere else, while LiteLLM has only
ever read AIML_API_KEY. Renaming would break existing configs, so AIML_API_KEY
stays primary and AIMLAPI_API_KEY joins the AIMLAPI_KEY fallback the image
generation config already had.
… pricing

aiml/dall-e-2 and aiml/dall-e-3 are gone. POSTing either to
/v1/images/generations comes back 404 with "Model 'openai/dall-e-3' is no
longer available (retired 2026-06-09)", so shipping cost entries for them only
tells a user the ids are supported when they are not.

flux-pro/v1.1-ultra was priced at $0.063 an image, but a real generation
reports usd_spent 0.078, so every ultra call was under counted by a quarter.
The provider's catalog now lists that pair under blackforestlabs prefixed ids,
which route to the same models at the same prices, so both spellings are
carried: the old ones still work and removing them would silently drop existing
users to $0 spend.

Checked by POSTing each id to /v1/images/generations with no prompt, which
answers 400 for a model that exists and 404 for one that does not. Worth
knowing for anyone repeating this: flux-pro/v1.1 and flux-pro/v1.1-ultra appear
in neither the id nor the aliases list of GET /v1/models, yet both still serve
traffic, so the catalog alone is not enough to call an id dead.
The provider's product, docs and billing all say aimlapi.com, so "AI/ML API" in
the model add form, the credential picker, the public endpoint support response
and the README table is a name a user has to translate before recognising it.
The machine identifier stays aiml, so nobody's config changes.
Puts aimlapi.com first in the hand ordered lists a user actually reads: the
Admin UI provider dropdowns, which render in enum declaration order, the
provider table in provider_endpoints_support.json, and the README provider
table. The provider create form already listed it first, so that file is
untouched here.

Placement only, no functional change. Drop this commit before sending anything
from this branch upstream.
The placeholder part_litellm was a readable stand-in chosen before the
partner was registered. Registration mints the id server-side, so the
real value is part_O0eykPA6gQNIFEYaUBojBbU4. A wrong or unknown partner id is accepted with a
200 and silently not attributed, so this would not have surfaced at runtime.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant