docs: document aimlapi.com as a RULER judge provider

[Lookoff-AIMLAPI]

Summary

ART has no provider registry, no provider enum and no per-provider subclass directory. Every judge call in RULER is delegated wholesale to LiteLLM (src/art/rewards/ruler.py:236, await acompletion(model=judge_model, ...)), and _judge_provider (ruler.py:55-62) just splits the model string on the first /, so any LiteLLM provider prefix is accepted as-is.

The pinned LiteLLM (pyproject.toml: litellm>=1.71.1,<=1.82.0; uv.lock resolves 1.82.0) already ships a first-class aiml provider. aimlapi.com therefore already works in ART today with zero code changes — this was verified with a real inference call, not by reading. So this PR ships documentation plus one route-guarding test, not a provider implementation. Forcing a provider abstraction into a repo that has none would be the wrong change.

What changed

docs(ruler) commit — upstream-safe:

• docs/integrations/aimlapi.mdx (new) — setup, judge-model usage, model-id shape, base-URL override, and a working recipe for judge cost tracking.
• docs/fundamentals/ruler.mdx — one judge-model example and a <Note> pointing at the guide, in the existing "Judge Model" customization section.
• docs/docs.json — the new page registered in the Integrations navigation group, per AGENT.md:20-22 and CONTRIBUTING.md:156-166.
• tests/unit/test_aimlapi_litellm_route.py (new, 5 tests) — asserts aiml is in litellm.provider_list and litellm.openai_compatible_providers, that aiml/<vendor>/<model> resolves to https://api.aimlapi.com/v1 with the prefix stripped exactly once, that the key is read from AIML_API_KEY, and that _judge_provider attributes it to aiml. Offline, keyless, no network.

chore(aimlapi) commit — fork-only, drop before any upstream proposal:

• Moves the aimlapi.com entry to the front of the Integrations nav group and to the front of the judge-model examples. Preferential placement in someone else's docs, isolated so it can be dropped cleanly.

Why the two things documented are documented

Both are silent failures rather than errors, which is why they belong in prose:

1. The env var is AIML_API_KEY, not AIMLAPI_API_KEY. LiteLLM chose the shorter spelling. A wrong name surfaces only as a 401 on the first judge call.

2. Judge cost metrics are silently empty. art/api_costs.py:311-323 (_estimate_provider_cost) estimates cost from token counts only when the provider prefix is literally openai or anthropic; every other prefix returns None and raises a ValueError that ruler.py:84-86 deliberately swallows. Judging works; costs/<context>/judge/ruler never appears.

   Note this is not aimlapi-specific — groq/, together/, fireworks/, ollama/ judges are all affected identically. It also means register_model_pricing() alone does not help, which is counter-intuitive: pricing resolves, then _estimate_provider_cost discards it. register_cost_extractor("aiml", ...) is the route that works, and that is what the docs show. Both behaviours were confirmed live (see below). No change was made to api_costs.py — widening it is a behaviour change to cost metrics for every non-OpenAI provider and should be a maintainer decision, not a side effect of a docs PR.

Verification

Build/quality gate, per CONTRIBUTING.md:18-26 — uv run prek run --all-files:

hook	baseline (pristine main)	with this branch
ruff	Passed	Passed
ruff format	Passed	Passed
ty type checking	Failed, 4 diagnostics	Failed, same 4 diagnostics
uv.lock sync check	Passed	Passed

The 4 ty diagnostics are pre-existing and platform-related — os.sched_getaffinity, os.pidfd_open, os.pidfd_send_signal are Linux-only and this run was on macOS. Identical before and after; nothing here touches them.

Unit tests — uv run pytest --nbval --current-env --tb=short --continue-on-collection-errors tests/unit:

	baseline	after
passed	664	669 (+5, the new tests)
failed	25	25
skipped	15	15
collection errors	27	27

The failing set is byte-identical before and after (diffed, not eyeballed). All 27 collection errors are ModuleNotFoundError for torch (22), transformers (2) and tinker (2) — those live in the CUDA-oriented backend/megatron/tinker extras that uv sync --group dev does not install on macOS. CI runs in the CUDA image and does not have this gap.

Live inference call

One real call through ART's own documented public entry point — art.rewards.ruler_score_group, on real art.Trajectory objects, no mocks, no raw curl:

judge_model: aiml/openai/gpt-5-5
judged is None: False
  traj[0] reward=0.92 ruler_score=0.92 independent=0.0
    explanation: RULER explanation: Provides a clear, relevant computer-themed joke with a punchline. It fully satisfies the request, though it is a common joke.
  traj[1] reward=0.03 ruler_score=0.03 independent=0.0
    explanation: RULER explanation: States a basic fact about computers and does not attempt a joke or humor, so it fails the main goal.

RULER's structured-output path (response_format=Response, a Pydantic model) round-trips correctly, both trajectories were scored, ranking is correct, and rewards were written back onto the group.

Additional live checks:

• The low-level ruler() entry point, same result.
• With an active MetricsBuilder and no extractor: judging succeeds, pending_by_scope is {} — the cost metric is confirmed silently absent, and nothing crashes.
• With register_model_pricing(...): still {}. Confirms the counter-intuitive behaviour described above.
• With register_cost_extractor("aiml", ...): {'val': {'costs/val/judge/ruler': 0.00194375}} from prompt=451 completion=138. The documented recipe is the one that was actually run.
• Every model id named in the docs was resolved against the live catalog by the id-or-alias rule (not ids alone): openai/gpt-5-5, anthropic/claude-sonnet-4.6, google/gemini-2.5-flash — all present, all type == "openai/chat-completions". gpt-5-5 and gemini-2.5-flash were additionally driven live through the aiml/ route.

Tool calling was not exercised: RULER's judge call uses structured output and never sends tools, so there is no tool-calling surface on this path to test.

Scope notes

• No attribution headers. ART reaches aimlapi.com only through LiteLLM; it never constructs a request to the API itself. There is no header mechanism on this path to merge into, and attributing here would double-count traffic that the LiteLLM integration already accounts for. Adding a header-injection layer to a repo that has none is exactly the kind of new machinery that does not belong in a first PR.
• benchmarks/run.py and src/art/mcp/generate_scenarios.py take a user-supplied base_url and an OpenAI client, so they can be pointed at aimlapi.com, but they are generic and default to OpenRouter. Nothing aimlapi-specific was added there.
• No embeddings. ART makes no embedding API calls anywhere; the only embedding matches in the tree are neural-network weight embeddings in the Megatron/model-support code. Nothing on the embeddings endpoint can affect ART.