Skip to content

feat: put image providers on one contract and drop CLI provider branches - #355

Merged
kiyeonjeon21 merged 1 commit into
mainfrom
feat/image-provider-contract
Oct 5, 2026
Merged

kiyeonjeon21 merged 1 commit into
mainfrom
feat/image-provider-contract

Conversation

@kiyeonjeon21

Copy link
Copy Markdown
Contributor

Image counterpart of the video provider work (#349-#353).

Why

Image generation and editing were duplicated per provider in five CLI places: generate image, the image executor, edit image, scene visuals, and two build helpers. Each had its own fetch, error parsing, model mapping, and URL-vs-base64 handling, and they had drifted apart (see the fixes below).

What changes

  • ImageGenerator contract (ai-providers/src/image):
    • createImage(request) returns image bytes; a request with input images is an edit.
    • maxEditImages declares the edit limit: OpenAI 16, Gemini 3 (Pro 14), Grok 1.
    • createImageGenerator(provider, key) opens a provider by name.
  • OpenAI, Gemini, and Grok implement it over providerRequest with ProviderError kinds.
    • Unknown models are refused before any request.
    • Gemini sends its key as a header instead of ?key= in the URL, and a safety block reads as moderation.
    • Grok asks for base64, so nothing downloads expiring URLs.
    • The old generateImage and editImage wrap the contract.
  • CLI on the contract: generate image, edit image, scene visuals, and build backdrops and keyframes all go through it, via executeImageGenerate and executeImageEdit. edit_image gains a provider parameter.

Fixes

  • OpenAI options ignored: generation ignored the model and the aspect ratio, and edits ignored the model (the spinner always said "GPT Image 1.5").
  • Wrong default model for edits: edit image -m defaulted to the Gemini alias flash for every provider.
  • Build model ignored: Grok backdrops ignored the build's image model.
  • Only one of several images saved: --count N saved only the first image. All are saved now (out.png, out-2.png, ...).
  • Wrong file format: scene visuals wrote Gemini JPEG bytes into .png files.
  • Dead option: generate image -p runway ran a Python script that no longer exists, so it always failed. The option is removed.
  • Wrong error code: a missing key surfaced as API_ERROR (exit 5, no hints) instead of API_KEY_MISSING (exit 4, setup hints).
  • Hardcoded model IDs: edit image used a hardcoded Gemini model-ID map instead of the catalog.
  • Tool options out of sync:
    • The MCP build tool allowed only OpenAI images, though builds support Gemini and Grok.
    • DALL-E's style option, which GPT Image rejects, is gone from the pipeline.

Verification

  • Tests:
    • New image contract suite over the fake network for all three providers: bytes and default model, edit limits refused before sending, unknown model, bad key. Plus provider specifics: OpenAI size from the ratio, Gemini header key and moderation, Gemini Pro limit, Grok base64 and 2.0 quality.
    • Build tests now mock createImageGenerator.
  • Checks: pnpm build, lint, typecheck, tests (providers 228, CLI 1342 including integration, MCP 73), pre-push gate.
  • Live:
    • generate image -p openai -r 16:9: 1536x1024, where it used to be 1024x1024.
    • -p gemini -m lite -r 16:9: 1376x768.
    • -p grok --count 2: two files saved, all PNG.
    • edit image -p openai -m flare and -p grok: both applied the edit with the scene kept.
    • A two-image Grok edit was refused before any request.
    • About $0.6.

https://claude.ai/code/session_011S1BLTaFctZNQzGERpkCuA

Image generation and editing were duplicated per provider in five CLI
places (generate image, the image executor, edit image, scene visuals, and
two build helpers), each with its own fetch, error parsing, model mapping,
and URL-vs-base64 handling.

- `ImageGenerator` (ai-providers/src/image): `createImage(request)` returns
  image bytes; a request with input images is an edit. `maxEditImages`
  declares each provider's edit limit (OpenAI 16, Gemini 3 / Pro 14,
  Grok 1). `createImageGenerator(provider, key)` opens one by name.
- OpenAI, Gemini, and Grok implement it over `providerRequest` with
  `ProviderError` kinds; unknown models are refused before any request,
  and the old `generateImage` / `editImage` methods wrap it.
- Gemini sends its key as a header instead of `?key=` in the URL, and a
  safety block reads as moderation. Grok asks for base64, so nothing
  downloads expiring URLs.
- The CLI's generate image, edit image, scene visuals, and build
  backdrops/keyframes all go through the contract (`executeImageGenerate`,
  `executeImageEdit`). `resolveCatalogModel` generalizes the video model
  resolver.

Fixes:
- OpenAI generation ignored the model and the aspect ratio; OpenAI edits
  ignored the model (and the spinner always said GPT Image 1.5).
- `edit image -m` defaulted to the Gemini alias `flash` for every provider.
- Grok backdrops ignored the build's image model.
- `--count N` saved only the first image; all are saved now (out-2.png...).
- Scene visuals wrote Gemini JPEG bytes into .png files.
- `generate image -p runway` ran a Python script that no longer exists.
- A missing image key surfaced as API_ERROR instead of API_KEY_MISSING.
- `edit image` used a hardcoded Gemini model-ID map instead of the catalog.
- The MCP build tool allowed only OpenAI images though builds support
  Gemini and Grok; DALL-E's `style` option is gone from the pipeline.

Claude-Session: https://claude.ai/code/session_011S1BLTaFctZNQzGERpkCuA
@vercel

vercel Bot commented Oct 5, 2026 •

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated
vibeframe Ready Ready Preview Oct 5, 2026 7:10am UTC

Request Review

@kiyeonjeon21
kiyeonjeon21 merged commit 33b68cc into main Oct 5, 2026
6 checks passed
@kiyeonjeon21
kiyeonjeon21 deleted the feat/image-provider-contract branch October 5, 2026 07:14

This branch was successfully deployed

1 active deployment
Preview — e497236e Deployed Oct 5, 2026 by vercel[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant