Repository navigation
Fix azure deployed gpt-image image generation by not sending response_format - #195
Conversation
meffmadd
left a comment
There was a problem hiding this comment.
I would ideally like to avoid hardcoding specific models in the endpoints... could we maybe (ab)use the model_info config option for this?
The image_generation view injects response_format=b64_json into every
request body, but gpt-image models (Azure and OpenAI) reject that
parameter - they only return b64_json. Requests to a gpt-image
deployment therefore failed with
400 "Unknown parameter: 'response_format'".
Instead of hardcoding model names in the endpoint, whether
response_format is sent is now driven by the router config: a
deployment whose model_info sets the new `supports_response_format:
false` flag no longer receives the parameter. The flag defaults to
true, so existing configs (dall-e etc.) are unaffected.
Example router config entry for an Azure AI Foundry gpt-image
deployment (sanitized - api_base and api_key are placeholders):
- model_name: cloud-gpt-image-2
litellm_params:
model: azure/gpt-image-2-example
api_base: https://example.cognitiveservices.azure.com/openai/v1
api_key: <api_key>
api_version: 2025-04-01-preview
model_info:
id: cloud-gpt-image-2
mode: image_generation
supports_response_format: false
aliases:
- gpt-image-2
d815fc7 to
3bccc8c
Compare
|
Agreed, i added a dedicated supports_response_format option to each deployment's model_info (following LiteLLM's supports_* naming), read by a small helper in gateway/config.py alongside the existing aliases / request_limit_multiplier handling.
- model_name: cloud-gpt-image-2
litellm_params:
model: azure/gpt-image-2-example
...
model_info:
id: cloud-gpt-image-2
mode: image_generation
supports_response_format: falseVerified with unit tests for the helper plus integration tests that assert the actual upstream request body (no response_format for flagged models, b64_json for unflagged ones), and end-to-end against a live Azure gpt-image deployment (400 without the flag → 200 with it). Happy to adjust the option name if you'd prefer something else. |
The image_generation view injects response_format=b64_json into every request body, but gpt-image models (Azure and OpenAI) reject that parameter - they only return b64_json. Requests to a gpt-image deployment therefore failed with
400 "Unknown parameter: 'response_format'".
Drop response_format for gpt-image* relay models before the request is sent. dall-e deployments keep the parameter and are unaffected.
Example router config entry for an Azure AI Foundry gpt-image deployment (sanitized - api_base and api_key are placeholders):
litellm_params:
model: azure/gpt-image-2-example
api_base: https://example.cognitiveservices.azure.com/openai/v1
api_key: <api_key>
api_version: 2025-04-01-preview
model_info: id: cloud-gpt-image-2
mode: image_generation aliases: