Skip to content

fix(media): async submit (no more 524) + forward gpt-image size; remove dead KIE provider#15

Merged
duanbing merged 1 commit into
mainfrom
fix/media-async-submit-and-remove-kie
Jun 28, 2026
Merged

fix(media): async submit (no more 524) + forward gpt-image size; remove dead KIE provider#15
duanbing merged 1 commit into
mainfrom
fix/media-async-submit-and-remove-kie

Conversation

@duanbing

Copy link
Copy Markdown

Why

Two production issues on media generation (surfaced by openai/gpt-image-2 text-to-image), plus a dead-code removal.

1. 524 on media generation

NovitaProvider::infer_media_proxy blocked the /inference HTTP response for the entire render, polling the upstream async task to completion before returning. Renders over ~100s tripped Cloudflare's origin timeout (524) even though the generation succeeded server-side and the user was charged.

Fix: return the task_id as soon as the task is submitted, and run poll_async_result + the result callback in a detached spawn_ignoring_shutdown task. This matches the callback-driven contract the backend already implements (submit → pending → callback). A new post_media_callback_failure reports async failures so a gen that errors after submit settles/refunds instead of hanging in pending.

2. gpt-image size dropped

The GptImageTextToImage / GptImageEdit request-shape whitelists omitted size, so the chosen dimension never reached Novita (and tier pricing fell back to the flat rate on the RouterBase side). Added size to both.

3. Remove unused KIE provider

No active model routes through KIE (type = "kie" is used by zero models; every media variant has a .proxy), so the None-proxy KIE media fallback was dead. Removes providers/kie.rs and the ProviderConfig::KIE / ProviderType::KIE / KIEKind / KIEProviderTypeConfig plumbing. Media variants without a proxy now return an explicit config error instead of silently falling back.

Also

  • Added the missing media_generation: Default::default() to test InferenceParams initializers (the field was added earlier but ~28 test sites weren't updated), so the test target compiles again.

Verification

  • cargo check --package tensorzero-core
  • cargo check --package tensorzero-core --tests
  • cargo fmt (unrelated workspace-wide reformatting reverted to keep the diff focused)

🤖 Generated with Claude Code

…e dead KIE provider

Novita media proxy (`infer_media_proxy`) blocked the `/inference` response
for the entire render while polling the upstream task to completion. For
renders over ~100s this tripped Cloudflare's origin timeout (524) even
though the generation succeeded server-side. Now the proxy returns the
`task_id` as soon as the task is submitted and runs the poll + result
callback in a detached task; a new failure callback reports async errors so
the gen settles/refunds instead of hanging in `pending`.

Also:
- gpt-image-2 t2i/edit: add `size` to the request-shape whitelist so the
  chosen dimension actually reaches Novita (was silently dropped).
- Remove the unused KIE provider. No active model routes through it and
  every media variant uses a proxy, so the `None`-proxy KIE media fallback
  was dead. Deletes `providers/kie.rs` and the `ProviderConfig::KIE` /
  `ProviderType::KIE` / `KIEKind` / `KIEProviderTypeConfig` plumbing; media
  variants without a proxy now return an explicit config error.
- Add the missing `media_generation: Default::default()` to test
  `InferenceParams` initializers so the test target compiles.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@github-actions

Copy link
Copy Markdown


Thank you for your submission, we really appreciate it. Like many open-source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution. You can sign the CLA by just posting a Pull Request Comment same as the below format.


I have read the Contributor License Agreement (CLA) and hereby sign the CLA.


You can retrigger this bot by commenting recheck in this Pull Request. Posted by the CLA Assistant Lite bot.

@duanbing duanbing merged commit 06be2a5 into main Jun 28, 2026
6 of 7 checks passed
@duanbing duanbing deleted the fix/media-async-submit-and-remove-kie branch June 28, 2026 10:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant