Dynamic model listing

[willbicks]

Summary

This PR replaces the static list of models supported by Fireworks with a call to their API to list the currently available models.

Additionally, it adds handling for the supports_image_input and supports_tools fields returned in the list of models, so that tools and vision can be used with Fireworks hosted inference.

Closes #2

Demos

Model listing:

$ # Fireworks hosted models as of 2026-04-04
$ llm models list | grep Fireworks
Fireworks: fireworks/models/deepseek-v3p2 (aliases: deepseek-v3p2)
Fireworks: fireworks/models/glm-4p7 (aliases: glm-4p7)
Fireworks: cogito/models/cogito-671b-v2-p1 (aliases: cogito-671b-v2-p1)
Fireworks: fireworks/models/gpt-oss-120b (aliases: gpt-oss-120b)
Fireworks: fireworks/models/glm-5 (aliases: glm-5)
Fireworks: fireworks/models/flux-kontext-max (aliases: flux-kontext-max)
Fireworks: fireworks/models/deepseek-v3p1 (aliases: deepseek-v3p1)
Fireworks: fireworks/models/kimi-k2p5 (aliases: kimi-k2p5)
Fireworks: fireworks/models/mixtral-8x22b-instruct (aliases: mixtral-8x22b-instruct)
Fireworks: fireworks/models/flux-kontext-pro (aliases: flux-kontext-pro)

Tool calling support:

$ llm -m kimi-k2p5 -T llm_time -T llm_version "What tools do you have? Can you try using one of them?"
I have two tools available:

1. **`llm_time`** - Returns the current time in both local time and UTC
2. **`llm_version`** - Returns the installed version of llm

Let me try using the `llm_time` tool to show you the current time:The `llm_time` tool returned the current time:

**Current Time:**
- **UTC:** April 4, 2026 at 21:09:43
- **Local (CDT):** April 4, 2026 at 16:09:43 (UTC-5:00, Daylight Saving Time is active)

This shows that the current timezone is Central Daylight Time (CDT), which is 5 hours behind UTC.

Vision support:

$ llm -m kimi-k2p5 describe -a 'https://static.simonwillison.net/static/2025/two-pelicans.jpg'
The image captures two brown pelicans in mid-flight against a pristine, cloudless blue sky.

The pelican in the lower left is actively flapping its wings, with both wings raised upward in a V-shape, revealing the dark flight feathers and lighter brown upper wing surfaces. Its long, orange-tipped bill points forward, and its throat pouch appears relaxed.

The second pelican, positioned in the upper right, glides with wings fully extended horizontally, showcasing the impressive wingspan and the contrast between the dark brown-black flight feathers and the lighter body plumage. This bird maintains a steady, soaring posture.

Both birds display characteristic features of brown pelicans: large, pouched bills, stocky bodies with pale undersides, and long, broad wings with distinct "finger-like" primary feathers at the wingtips. The bright daylight illumination highlights the texture of their feathers and creates a sharp silhouette against the vivid azure backdrop, emphasizing the grace and power of these large waterbirds in their aerial element.

[willbicks]

It turns out, there is unfortunately not a clear API from Fireworks to get a list of models that are available and return valid chat results.

I did a survey of their two endpoints (the OpenAI compatible one and the Fireworks native one) and the ways you can filter them to generate some sets of models as follows:

$ curl \
  -H "Authorization: Bearer $LLM_FIREWORKS_KEY" \
  https://api.fireworks.ai/inference/v1/models | \
  jq -r '.data[] | .id' \
  > inf_models.txt
$ curl \
  -H "Authorization: Bearer $LLM_FIREWORKS_KEY" \
  https://api.fireworks.ai/inference/v1/models | \
  jq -r '.data[] | select(.supports_chat) | .id' \
  > inf_models_chat.txt
$ curl \
  -H "Authorization: Bearer $LLM_FIREWORKS_KEY" \
  'https://api.fireworks.ai/v1/accounts/fireworks/models?filter=supports_serverless=true' | \
  jq -r '.models[] | .name' \
  > fw_serverless.txt

These formed my three sets of possible candidates. I then tried hitting https://api.fireworks.ai/inference/v1/chat/completions for each and seeing whether it returned any results. Finally, I checked whether that result was an actual response (and not, for example in the case of qwen3-embedding-8b, gibberish and foreign characters).

The results are as follows (as of today 2026-04-04):

Model Name / ID	in inf models	in inf models chat	in fw serverless	returns something	returns reasonable
accounts/cogito/models/cogito-671b-v2-p1	✅	✅	❌	✅	✅
accounts/fireworks/models/deepseek-v3p1	✅	✅	✅	✅	✅
accounts/fireworks/models/deepseek-v3p2	✅	✅	✅	✅	✅
accounts/fireworks/models/flux-1-dev-fp8	✅	❌	✅	❌	❌
accounts/fireworks/models/flux-1-schnell-fp8	✅	❌	✅	❌	❌
accounts/fireworks/models/flux-kontext-max	✅	✅	✅	❌	❌
accounts/fireworks/models/flux-kontext-pro	✅	✅	✅	❌	❌
accounts/fireworks/models/glm-4p7	✅	✅	✅	✅	✅
accounts/fireworks/models/glm-5	✅	✅	✅	✅	✅
accounts/fireworks/models/gpt-oss-120b	✅	✅	✅	✅	✅
accounts/fireworks/models/gpt-oss-20b	❌	❌	✅	✅	✅
accounts/fireworks/models/kimi-k2-instruct-0905	❌	❌	✅	✅	✅
accounts/fireworks/models/kimi-k2p5	✅	✅	✅	✅	✅
accounts/fireworks/models/kimi-k2-thinking	❌	❌	✅	✅	✅
accounts/fireworks/models/llama-v3p3-70b-instruct	❌	❌	✅	✅	✅
accounts/fireworks/models/minimax-m2p1	❌	❌	✅	✅	✅
accounts/fireworks/models/minimax-m2p5	❌	❌	✅	✅	✅
accounts/fireworks/models/mixtral-8x22b-instruct	✅	✅	✅	✅	✅
accounts/fireworks/models/qwen3-8b	❌	❌	✅	✅	✅
accounts/fireworks/models/qwen3-embedding-8b	❌	❌	✅	✅	❌
accounts/fireworks/models/qwen3p6-plus	❌	❌	✅	✅	✅
accounts/fireworks/models/qwen3-reranker-8b	❌	❌	✅	✅	✅
accounts/fireworks/models/qwen3-vl-30b-a3b-instruct	❌	❌	✅	✅	✅
accounts/fireworks/models/qwen3-vl-30b-a3b-thinking	❌	❌	✅	✅	✅

What this reveals is that there is seemingly no clearly documented way to get all the models that are valid for chatting out of the box. For each of the possible ways you could query for a list of models, none of them return all of the valid options, and all of them return at least one invalid option. This is cursed.

I reached out in Firework's discord for clarification on what is up here. Hopefully they can fix the /inference/v1/models endpoint to return all the valid options and not return invalid options.