Skip to content

docs: document aimlapi.com models via LiteLLM - #1

Open
Lookoff-AIMLAPI wants to merge 2 commits into
mainfrom
docs/aimlapi-litellm-sample
Open

docs: document aimlapi.com models via LiteLLM#1
Lookoff-AIMLAPI wants to merge 2 commits into
mainfrom
docs/aimlapi-litellm-sample

Conversation

@Lookoff-AIMLAPI

Copy link
Copy Markdown
Member

What changed

Adds contributing/samples/models/hello_world_aimlapi/ — a runnable sample and
README documenting how to reach aimlapi.com models from ADK, following the
shape the NVIDIA NIM sample established for a LiteLLM-backed provider
(README.md, __init__.py, agent.py, main.py), plus two unit tests.

A second, separate commit spells aiml/.* out at the head of the two
hand-ordered LiteLLM pattern lists.

Why

Nothing was broken, and this PR fixes nothing. That was the finding, and it
was checked rather than assumed.

LLMRegistry.resolve (src/google/adk/models/registry.py:37,189) hands any
unmatched provider/model name to litellm.provider_list, and LiteLLM has
carried aiml as a first-class provider — base https://api.aimlapi.com/v1,
key AIML_API_KEY, optional AIML_API_BASE — since well before the
litellm>=1.84 floor this project pins. So Agent(model="aiml/openai/gpt-4o-mini")
already worked, with zero code changes.

What did not exist was any way for a user to find that out. grep -rIin "aiml"
over the tree returned 0 matches before this PR. The natural reading of the
registry is that the sixteen spelled-out providers are the supported set, so the
aggregator gets ruled out before it is ever tried. The gap is documentation, so
the change is documentation.

The one thing that is easy to get wrong

An aggregator id carries its own slash. aiml/openai/gpt-4o-mini has two, and
only the first is the LiteLLM provider. Resolution has to pass the whole string
through or the request routes to a model nobody asked for. Confirmed on the wire
(see below) and now covered by
test_new_llm_keeps_a_multi_segment_provider_model_name.

How it was verified

Build: uv buildgoogle_adk-2.8.0-py3-none-any.whl, exit 0.

Tests (pytest tests/unittests, Python 3.12, -p no:randomly):

passed failed skipped xfailed xpassed
baseline (65b382d, pristine) 13714 2 85 26 2
after 13717 2 85 26 2

The 2 failures are identical before and after — test_import_loading.py::test_entry_point_loads_only_allowlisted_packages[agent]
and [runner] — and are present on a clean tree in this environment.

The +3 collected items were diffed rather than guessed:
test_match_litellm_family[aiml/openai/gpt-4o-mini],
test_new_llm_keeps_a_multi_segment_provider_model_name, and
test_samples.py::test_sample_loads[models/hello_world_aimlapi], which the
repo's own sample harness picks up automatically. All three pass.

Live inference through ADK's model plumbing — not curl, not raw litellm,
not a mock. Running the shipped main.py (AIML_API_KEY supplied via env):

Testing aimlapi.com integration with ADK
Model: aiml/openai/gpt-4o-mini
------------------------------------
** User says: {'parts': [{'text': 'Hi, introduce yourself.'}], 'role': 'user'}
LiteLLM completion() model= openai/gpt-4o-mini; provider = aiml
** hello_world_aimlapi_agent: Hello! I'm an agent powered by aimlapi.com, known as
   "hello_world_aimlapi_agent." I'm here to roll dice of different sizes and check if
   numbers are prime.
** User says: {'parts': [{'text': 'Roll a die with 20 sides and check if the result is prime.'}], 'role': 'user'}
LiteLLM completion() model= openai/gpt-4o-mini; provider = aiml   (x3)
** hello_world_aimlapi_agent: I rolled a 20-sided die and got a result of 4. The number 4
   is not a prime number.

Four completion calls across two turns, exercising a tool call and its
follow-up. Exit 0.

Separately, driven through Runner with the plain model string (registry
path, not a hand-built LiteLlm):

LLMRegistry.resolve('aiml/openai/gpt-4o-mini') -> LiteLlm
** user: Name the capital of Portugal.
** aimlapi_no_tools: Lisbon.

** user: Roll a die with 20 sides.
** aimlapi_tools TOOL CALL: roll_die({'sides': 20})
** TOOL RESULT: roll_die -> {'result': 11}
** aimlapi_tools: You rolled a 20-sided die and got a result of 11.
** user: What number did you just roll? Answer with the number only.
** aimlapi_tools: 11

Wire check. A no-tools agent was run with httpx.AsyncClient.send
instrumented, to confirm what completion_args actually becomes. ADK builds
{"tools": None, "response_format": None, ...} unconditionally
(lite_llm.py:3153), and this API rejects tools: null with a 400 on its
stricter models — so it mattered whether those nulls survive. They do not;
LiteLLM drops them:

URL:  https://api.aimlapi.com/v1/chat/completions
BODY: {"messages": [...], "model": "openai/gpt-4o-mini"}
'tools' present: False | 'response_format' present: False

Base URL correct, and the model id's own slash arrives intact.

Every model id named in the README was called once, through
LiteLlm.generate_content_async, and all four answered:
aiml/openai/gpt-4o-mini, aiml/openai/gpt-5-5,
aiml/anthropic/claude-sonnet-4.5, aiml/google/gemini-2.5-flash.

Not verified / out of scope

  • Embeddings. LiteLLM's published provider page lists embeddings for
    aiml, but the shipped package has only llms/aiml/chat/ and
    llms/aiml/image_generation/ — no embedding route — and
    litellm.embedding(model="aiml/...") raises
    LiteLLMUnknownProvider: Unmapped LLM provider for this endpoint. This does
    not affect ADK, which never calls litellm.embedding/aembedding: its only
    LiteLLM entrypoints are completion, acompletion, and acreate_file, and
    its embedding surfaces (FilesRetrieval, the Spanner tools) never take a
    provider/model string. Worth an upstream LiteLLM issue, not an ADK one.
  • File attachments. aiml is not in _FILE_ID_REQUIRED_PROVIDERS
    (lite_llm.py:247), so PDFs go inline as a data URI rather than through
    acreate_file. That is the same treatment every non-OpenAI/Azure provider
    gets and was not probed here.
  • Streaming and live mode were not exercised.
  • Only Python 3.12 was run; tox's full 3.10–3.14 matrix was not.

No attribution headers

Deliberate. Traffic from ADK reaches this provider through LiteLLM, which
already carries its own channel attribution, and adding per-request headers here
would double-count the same traffic and introduce machinery this repo has no
precedent for.

Commit layout

Two commits, so the second can be dropped cleanly:

  • docs: add aimlapi.com model sample — the sample, README and tests.
  • chore(aimlapi): fork-only placement — do not send upstream — puts aiml/.*
    first in the hand-ordered lists in models/__init__.py and
    LiteLlm.supported_models(). Placement only; behaviour is identical either
    way
    , since the fallback already resolves these names. It contradicts the
    maintainers' documented rule that the list "only spells out the common ones
    and defers the rest to LiteLLM itself", which is exactly why it is isolated.

A user reaching for an aggregator has no way to learn from this repo that
one works here. Nothing in ADK names aimlapi.com, so the natural reading of
the registry is that only the sixteen spelled-out providers are usable, and
the aggregator gets ruled out before it is ever tried.

Nothing is broken: `LLMRegistry.resolve` already defers any `provider/model`
name to `litellm.provider_list`, and LiteLLM has carried the `aiml` provider
(base `https://api.aimlapi.com/v1`, key `AIML_API_KEY`) since well before the
`litellm>=1.84` floor this project pins. The gap is documentation, so this
change is documentation, following the shape the NVIDIA NIM sample set for a
LiteLLM-backed provider: README, agent, runner.

The two tests guard the one thing that is easy to get wrong. An aggregator id
carries its own slash, so `aiml/openai/gpt-4o-mini` has two of them, and only
the first is the provider. Resolution has to hand the whole string to LiteLLM
or the request routes to a model nobody asked for.

Verified against the live API rather than a mock: `main.py` completes both
turns, including a tool call and its follow-up, on `aiml/openai/gpt-4o-mini`.
Spells `aiml/.*` out at the head of both hand-ordered LiteLLM pattern lists
instead of leaving it to the provider-list fallback.

This is placement, not function. `LLMRegistry.resolve` already returns
`LiteLlm` for these names, so the resolution result is identical either way
and no behaviour changes; the entry only moves the decision from the fallback
into the explicit list. It is separated into its own commit so it can be
dropped before anything goes upstream, where the maintainers' stated rule is
that the list spells out the common providers and defers the rest.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant