Skip to content

fix: adapt image generation/editing to new Grok cardAttachment API#392

Open
JinchengGao-Infty wants to merge 1 commit intochenyme:mainfrom
JinchengGao-Infty:fix-image-cardattachment
Open

fix: adapt image generation/editing to new Grok cardAttachment API#392
JinchengGao-Infty wants to merge 1 commit intochenyme:mainfrom
JinchengGao-Infty:fix-image-cardattachment

Conversation

@JinchengGao-Infty
Copy link
Contributor

Problem

Grok silently updated their web API, breaking both /v1/images/generations and /v1/images/edits:

  1. Image generation via app-chat with explicit modelName/modelMode no longer triggers image generation — Grok returns text instead
  2. Image results moved from modelResponse.generatedImageUrls to cardAttachment.jsonData.image_chunk format
  3. Image editing uses a different card type (render_edited_image) than generation (render_generated_image)

Root Cause

Captured browser network requests via Playwright and compared with grok2api payloads:

Field Browser (works) grok2api (broken)
modelName absent "grok-3"
modelMode absent "MODEL_MODE_FAST"
modeId "auto" absent
toolOverrides {"gmailSearch":false,...} {}
Image data location cardAttachment.image_chunk looked for modelResponse.generatedImageUrls

Fix

app_chat.py: build_payload() supports model=None — omits modelName/modelMode, allowing modeId-based routing via request_overrides.

image.py: _stream_app_chat() and _collect_app_chat() now use model=None, mode=None with modeId: "auto".

image_edit.py:

  • ImageStreamProcessor and ImageCollectProcessor parse new cardAttachment format with image_chunk.imageUrl (progress >= 100)
  • Match both render_generated_image and render_edited_image card types
  • Image URL constructed as https://assets.grok.com/{image_chunk.imageUrl}
  • Edit path uses disableMemory=false, temporary=false and browser-matching toolOverrides

Grok cardAttachment Types

type Purpose Key field
render_generated_image New image generation image_chunk.imageUrl
render_edited_image Image editing image_chunk.imageUrl
render_searched_image Web image search image.original

Test Plan

  • POST /v1/images/generations with grok-imagine-1.0 — returns image URL
  • POST /v1/chat/completions with grok-imagine-1.0 (stream) — returns markdown image in chat chunk
  • POST /v1/images/edits with grok-imagine-1.0-edit + image — returns edited image URL
  • Normal POST /v1/chat/completions with grok-3 — text chat unaffected

## Problem

Grok silently updated their web API:
1. Image generation via `app-chat` with explicit `modelName`/`modelMode`
   no longer triggers image generation - Grok returns text instead
2. Image results moved from `modelResponse.generatedImageUrls` to
   `cardAttachment.jsonData.image_chunk` format
3. Image editing uses a different card type (`render_edited_image`)
   than generation (`render_generated_image`)

This broke both `/v1/images/generations` and `/v1/images/edits`.

## Root Cause

Captured browser network requests and compared with grok2api payloads:
- Browser uses `modeId: "auto"` without `modelName`/`modelMode`
- Browser sends explicit `toolOverrides` disabling search tools
- Browser sends `disableMemory: false, temporary: false` for edits

## Fix

**app_chat.py**: `build_payload()` supports `model=None` — omits
`modelName`/`modelMode` fields, allowing `modeId`-based routing.

**image.py**: `_stream_app_chat()` and `_collect_app_chat()` now use
`model=None, mode=None` with `modeId: "auto"` in request_overrides.

**image_edit.py**:
- `ImageStreamProcessor` and `ImageCollectProcessor` parse new
  `cardAttachment` format with `image_chunk.imageUrl` (progress >= 100)
- Match both `render_generated_image` and `render_edited_image` types
- Image URL: `https://assets.grok.com/{image_chunk.imageUrl}`
- Edit path uses `disableMemory=false, temporary=false` and
  browser-matching `toolOverrides` to prevent search fallback
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant