Skip to content

duplicate of https://github.com/lightseekorg/smg/pull/1057#960

Open
TingtingZhou7 wants to merge 1 commit intolightseekorg:mainfrom
TingtingZhou7:tingzhou/add-img
Open

duplicate of https://github.com/lightseekorg/smg/pull/1057#960
TingtingZhou7 wants to merge 1 commit intolightseekorg:mainfrom
TingtingZhou7:tingzhou/add-img

Conversation

@TingtingZhou7
Copy link
Copy Markdown

@TingtingZhou7 TingtingZhou7 commented Mar 28, 2026

Please refer to #1057

@github-actions github-actions bot added grpc gRPC client and router changes model-gateway Model gateway crate changes openai OpenAI router changes labels Mar 28, 2026
@coderabbitai
Copy link
Copy Markdown

coderabbitai bot commented Mar 28, 2026

Note

Reviews paused

It looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the reviews.auto_review.auto_pause_after_reviewed_commits setting.

Use the following commands to manage reviews:

  • @coderabbitai resume to resume automatic reviews.
  • @coderabbitai review to trigger a single review.

Use the checkboxes below for quick actions:

  • ▶️ Resume reviews
  • 🔍 Trigger review
📝 Walkthrough

Walkthrough

Adds first-class image-generation tool support across protocols, gateway routing, MCP config/transformer, and streaming: new ResponseTool::ImageGeneration + ImageGenerationTool, ResponseOutputItem::ImageGenerationCall + status enum, wiring for built-in detection, serialization, MCP routing, transformer conversion, and tests.

Changes

Cohort / File(s) Summary
Protocol Definitions
crates/protocols/src/responses.rs
Add ResponseTool::ImageGeneration(ImageGenerationTool), ImageGenerationTool (flattened options), ResponseOutputItem::ImageGenerationCall { id, status, result }, and ImageGenerationCallStatus; add serde behavior and unit tests.
Gateway: gRPC / Harmony
model_gateway/src/routers/grpc/common/responses/utils.rs, model_gateway/src/routers/grpc/harmony/builder.rs
Treat image_generation as builtin: extend BUILTIN_TOOLS, update Tool::is_builtin and ResponseTool::is_builtin, include in tool-name collection and MCP connection detection.
Gateway: OpenAI serialization
model_gateway/src/routers/openai/responses/utils.rs
Serialize ResponseTool::ImageGeneration(_) in response_tool_to_value so restored tools can include image-generation tools.
MCP Config & Types
crates/mcp/src/core/config.rs, crates/mcp/src/transform/types.rs
Add BuiltinToolType::ImageGeneration and ResponseFormatConfig::ImageGenerationCall; map to ResponseFormat::ImageGenerationCall, update serde/display and tests.
MCP Transformer
crates/mcp/src/transform/transformer.rs
Add ResponseFormat::ImageGenerationCall handling and to_image_generation_call conversion that emits ResponseOutputItem::ImageGenerationCall with id ig_{tool_call_id}, status Completed, and extracted result; add unit test.
Gateway MCP utils & tests
model_gateway/src/routers/mcp_utils.rs
Include ImageGeneration in builtin routing/extraction, update docs, add image-server test config, and adjust tests to expect three builtin routes.
Gateway Streaming & Tool Loop
model_gateway/src/routers/grpc/common/responses/streaming.rs, model_gateway/src/routers/openai/mcp/tool_loop.rs
Add OutputItemType::ImageGenerationCall, map format strings and item types, allocate ig prefix, emit response.image_generation_call.* events, and skip intermediate searching event for image-generation.

Sequence Diagram(s)

sequenceDiagram
    participant Client
    participant Gateway as ModelGateway
    participant MCP
    participant Transformer as ResponseTransformer

    Client->>Gateway: Request with tool_choice "image_generation"
    Gateway->>Gateway: recognize builtin tool -> route decision
    Gateway->>MCP: Forward request to MCP image-generation server
    MCP->>Transformer: Return ResponseFormat::ImageGenerationCall payload
    Transformer->>Gateway: to_image_generation_call -> ImageGenerationCall output item
    Gateway->>Client: Respond with `image_generation_call` output item
Loading

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~20 minutes

Possibly related PRs

Suggested labels

protocols, mcp

Suggested reviewers

  • CatherineSue
  • key4ng
  • slin1237

Poem

🐰 I hopped through enums and serde streams,
Flattened options and added new dreams.
From gateway to MCP the ig calls run,
Bunny-approved, now image-gen is done! 🎨✨

🚥 Pre-merge checks | ✅ 2 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Title check ⚠️ Warning The PR title 'duplicate of #1057' is unrelated to the actual changeset, which implements image generation tool support across multiple crates and routers. Update the title to accurately reflect the main change, such as 'Add support for image generation tool across protocols and routing' or similar.
✅ Passed checks (2 passed)
Check name Status Explanation
Docstring Coverage ✅ Passed Docstring coverage is 100.00% which is sufficient. The required threshold is 80.00%.
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@TingtingZhou7 TingtingZhou7 changed the title Add Support for image generation tool [Not Ready For Review] Add Support for image generation tool Mar 28, 2026
@mergify
Copy link
Copy Markdown
Contributor

mergify bot commented Mar 28, 2026

Hi @TingtingZhou7, the DCO sign-off check has failed. All commits must include a Signed-off-by line.

To fix existing commits:

# Sign off the last N commits (replace N with the number of unsigned commits)
git rebase HEAD~N --signoff
git push --force-with-lease

To sign off future commits automatically:

  • Use git commit -s every time, or
  • VSCode: enable Git: Always Sign Off in Settings
  • PyCharm: enable Sign-off commit in the Commit tool window

Copy link
Copy Markdown

@gemini-code-assist gemini-code-assist bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request adds support for a new 'image_generation' tool across the protocol and gateway layers. I have noted that the new functionality lacks corresponding tests, which should be addressed to ensure reliability. Additionally, I recommend formatting the 'matches!' expression in 'model_gateway/src/routers/grpc/harmony/builder.rs' to span multiple lines for better consistency and readability.

matches!(
self.tool_type.as_str(),
"web_search_preview" | "code_interpreter" | "container"
"web_search_preview" | "code_interpreter" | "image_generation" | "container"
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

For consistency with other multi-line matches! expressions in this pull request and to improve readability, consider formatting this by putting each pattern on its own line.

            "web_search_preview"
                | "code_interpreter"
                | "image_generation"
                | "container"

@TingtingZhou7 TingtingZhou7 force-pushed the tingzhou/add-img branch 2 times, most recently from fb61255 to 3814457 Compare March 28, 2026 00:09
Copy link
Copy Markdown

@chatgpt-codex-connector chatgpt-codex-connector bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: d6e92d7239

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

Copy link
Copy Markdown

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 4

🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Inline comments:
In @.oca/custom_code_review_guidelines.txt:
- Around line 1-24: The "Sample guideline" content (the Java streams example /
"Use streams instead of a loop for better readability") appears unrelated to the
PR's image-generation scope; either remove this
.oca/custom_code_review_guidelines.txt change from the PR, or explicitly justify
its inclusion by updating the PR description and the file to state how these
review guidelines relate to the "Add Support for image generation tool" feature
(or move the guidelines to a separate documentation-only PR); ensure the file
text clearly references the image-generation context if kept.
- Line 22: Document that the example's use of Collection.toList() requires Java
16+ and either add a note stating the minimum Java version (Java 16) or change
the example to use Stream.collect(Collectors.toList()) for Java 8–15
compatibility; if you choose the latter, also note the mutability difference
(Collection.toList() returns an immutable List, while Collectors.toList()
returns a mutable list on older JDKs).
- Around line 9-22: The Java example in the template is inappropriate for this
Rust repo and contains API/type mismatches: replace the Java snippet
(references: List<Integer>, ArrayList, for (int number : numbers),
Arrays.stream(numbers), .toList()) with either a corrected Java variant that
preserves semantics (use numbers.stream() for collections and
collect(Collectors.toList()) if you need a mutable ArrayList) or—preferably—swap
the entire example for an equivalent Rust iterator sample using Vec and
iter().map(...).collect() so the example matches the repository language and
avoids the type and mutability problems.

In `@oracle-schema-config.local.yaml`:
- Line 1: The repo-wide config currently enables automatic DB DDL by setting
auto_migrate: true; change this key to auto_migrate: false in this committed
config so startup won't execute Oracle migrations by default, and instead enable
auto-migrate only in local/runtime overrides or explicit non-repo configs;
verify that the startup migration code paths (oracle.rs startup logic and
versioning.rs migration runner) will respect the config and that the default
safety check in schema.rs remains effective when the flag is false.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: ASSERTIVE

Plan: Pro

Run ID: d029e5cb-9b7c-444d-95d1-137b236a3f9f

📥 Commits

Reviewing files that changed from the base of the PR and between b011b77 and d6e92d7.

📒 Files selected for processing (6)
  • .oca/custom_code_review_guidelines.txt
  • crates/protocols/src/responses.rs
  • model_gateway/src/routers/grpc/common/responses/utils.rs
  • model_gateway/src/routers/grpc/harmony/builder.rs
  • model_gateway/src/routers/openai/responses/utils.rs
  • oracle-schema-config.local.yaml

Comment on lines +9 to +22
# ```java
# // Calculate squares of numbers
# List<Integer> squares = new ArrayList<>();
# for (int number : numbers) {
# squares.add(number * number);
# }
# ```
# Recommendation:
#
# ```java
# // Calculate squares of numbers
# List<Integer> squares = Arrays.stream(numbers)
# .map(n -> n * n) // Map each number to its square
# .toList();
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
# Description: Check Java version and stream usage patterns

# Check if Java version is documented anywhere
rg -i 'java.*(version|[0-9]{2})' --glob '*.md' --glob '*.txt' --glob 'pom.xml' --glob 'build.gradle*' -C2

# Check for existing .toList() usage (Java 16+ API)
rg '\.toList\(\)' --type=java -C2

# Check for Collectors.toList() usage (pre-Java 16 pattern)
rg 'Collectors\.toList\(\)' --type=java -C2

Repository: lightseekorg/smg

Length of output: 357


🏁 Script executed:

#!/bin/bash
# Explore repository structure and primary language
echo "=== Repository Structure ==="
git ls-files | head -30

echo -e "\n=== File Extensions ==="
git ls-files | sed 's/.*\.//' | sort | uniq -c | sort -rn | head -20

echo -e "\n=== Read the file being reviewed ==="
if [ -f .oca/custom_code_review_guidelines.txt ]; then
  cat -n .oca/custom_code_review_guidelines.txt
else
  echo "File not found"
fi

Repository: lightseekorg/smg

Length of output: 2505


Java example in template has type and API compatibility issues; consider replacing with Rust example matching repository language.

This is a template guideline file in a Rust repository. While the Java code example (lines 9-22) is intended as a sample pattern, it contains technical issues that would mislead developers:

  1. Type mismatch: The original loop for (int number : numbers) works with any Iterable, but the recommended Arrays.stream(numbers) requires an array type. If numbers is a List<Integer>, the recommendation won't compile.
  2. Mutability difference: ArrayList (original) is mutable; .toList() (recommended, Java 16+) returns an immutable list—a significant behavioral change.
  3. Language scope: A Rust project's review guidelines should ideally feature Rust examples, not Java, to match contributor context.

Consider either fixing the Java example to use numbers.stream() if it's a collection, or replacing it with a Rust iterator example that's more relevant to the project.

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In @.oca/custom_code_review_guidelines.txt around lines 9 - 22, The Java example
in the template is inappropriate for this Rust repo and contains API/type
mismatches: replace the Java snippet (references: List<Integer>, ArrayList, for
(int number : numbers), Arrays.stream(numbers), .toList()) with either a
corrected Java variant that preserves semantics (use numbers.stream() for
collections and collect(Collectors.toList()) if you need a mutable ArrayList)
or—preferably—swap the entire example for an equivalent Rust iterator sample
using Vec and iter().map(...).collect() so the example matches the repository
language and avoids the type and mutability problems.

# // Calculate squares of numbers
# List<Integer> squares = Arrays.stream(numbers)
# .map(n -> n * n) // Map each number to its square
# .toList();
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟡 Minor

🧩 Analysis chain

🌐 Web query:

What Java version introduced the Stream.toList() method and what are the differences between toList() and collect(Collectors.toList())?

💡 Result:

The Stream.toList method was introduced in Java 16. Key differences between Stream.toList and collect(Collectors.toList) are: - Mutability: toList returns an immutable List (throws UnsupportedOperationException on mutator methods). collect(Collectors.toList) returns a mutable List (typically ArrayList, modifiable). - Syntax: toList is more concise (no need for collect or Collectors import). collect requires stream.collect(Collectors.toList). - Null handling: Both allow null elements. - Type inference: collect(Collectors.toList) is more flexible with supertypes (e.g., Stream.collect(toList) to List); toList infers exact stream type, may need casts or explicit typing. - Performance: toList is slightly more optimized (direct array deposition, no Collector overhead). - Availability: toList since Java 16; Collectors.toList since Java 8. Note: In Java 21+, Collectors.toList was updated to return immutable lists matching toList behavior.

Citations:


Document Java version requirement for .toList().

The .toList() method was introduced in Java 16. If the project targets earlier Java versions, this example will not compile. For Java 8-15 compatibility, use .collect(Collectors.toList()) instead. Note: .toList() returns an immutable List, while Collectors.toList() returns a mutable one (though in Java 21+, Collectors.toList() also returns an immutable list).

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In @.oca/custom_code_review_guidelines.txt at line 22, Document that the
example's use of Collection.toList() requires Java 16+ and either add a note
stating the minimum Java version (Java 16) or change the example to use
Stream.collect(Collectors.toList()) for Java 8–15 compatibility; if you choose
the latter, also note the mutability difference (Collection.toList() returns an
immutable List, while Collectors.toList() returns a mutable list on older JDKs).

Copy link
Copy Markdown

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (2)
model_gateway/src/routers/grpc/common/responses/utils.rs (1)

25-29: 🧹 Nitpick | 🔵 Trivial

Update docstring to include image_generation.

The docstring mentions web_search_preview and code_interpreter as builtin tool types but doesn't include image_generation, which is now also handled.

📝 Suggested fix
 /// Ensure MCP connection succeeds if MCP tools or builtin tools are declared
 ///
-/// Checks if request declares MCP tools or builtin tool types (web_search_preview,
-/// code_interpreter), and if so, validates that the MCP clients can be created
+/// Checks if request declares MCP tools or builtin tool types (web_search_preview,
+/// code_interpreter, image_generation), and if so, validates that the MCP clients can be created
 /// and connected.
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@model_gateway/src/routers/grpc/common/responses/utils.rs` around lines 25 -
29, Update the docstring for the MCP connection check to also mention the
builtin tool type image_generation; locate the comment/block above the function
that "Ensure MCP connection succeeds if MCP tools or builtin tools are declared"
(the docstring referencing builtin tool types web_search_preview and
code_interpreter) and add image_generation to that list so the documentation
matches current behavior.
model_gateway/src/routers/openai/responses/utils.rs (1)

206-209: 🧹 Nitpick | 🔵 Trivial

Update docstring to include image_generation.

The function docstring mentions "MCP tools", "web_search_preview", and "code_interpreter" but should also include image_generation since it's now handled.

📝 Suggested fix
 /// Convert a single ResponseTool back to its original JSON representation.
 ///
-/// Handles MCP tools (with server metadata), web_search_preview, and code_interpreter.
+/// Handles MCP tools (with server metadata), web_search_preview, code_interpreter, and image_generation.
 /// Returns None for function tools and other types that don't need restoration.
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@model_gateway/src/routers/openai/responses/utils.rs` around lines 206 - 209,
Update the function docstring that begins "Convert a single ResponseTool back to
its original JSON representation." to also mention image_generation alongside
MCP tools, web_search_preview, and code_interpreter; explicitly state that
image_generation is handled and that the function still returns None for
function tools and other types that don't need restoration so readers see
image_generation is supported.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Inline comments:
In `@crates/protocols/src/responses.rs`:
- Around line 39-41: Add the missing ImageGeneration tool into the MCP routing
surfaces: update the BuiltinToolType enum in crates/mcp/src/core/config.rs to
include ImageGeneration and add the corresponding ImageGenerationCall variant to
ResponseFormatConfig; then modify extract_builtin_types() in
model_gateway/src/routers/mcp_utils.rs so it extracts ImageGeneration (and
ensure FileSearch is also extracted there), and update the response_format()
implementation for BuiltinToolType to handle the new ImageGeneration variant
(return the proper ResponseFormatConfig::ImageGenerationCall). Use the enum and
function names BuiltinToolType, ResponseFormatConfig, extract_builtin_types, and
response_format to locate the spots to change.

---

Outside diff comments:
In `@model_gateway/src/routers/grpc/common/responses/utils.rs`:
- Around line 25-29: Update the docstring for the MCP connection check to also
mention the builtin tool type image_generation; locate the comment/block above
the function that "Ensure MCP connection succeeds if MCP tools or builtin tools
are declared" (the docstring referencing builtin tool types web_search_preview
and code_interpreter) and add image_generation to that list so the documentation
matches current behavior.

In `@model_gateway/src/routers/openai/responses/utils.rs`:
- Around line 206-209: Update the function docstring that begins "Convert a
single ResponseTool back to its original JSON representation." to also mention
image_generation alongside MCP tools, web_search_preview, and code_interpreter;
explicitly state that image_generation is handled and that the function still
returns None for function tools and other types that don't need restoration so
readers see image_generation is supported.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: ASSERTIVE

Plan: Pro

Run ID: 5dfee3c3-70d3-451b-a54f-8bb74542dcde

📥 Commits

Reviewing files that changed from the base of the PR and between d6e92d7 and 3814457.

📒 Files selected for processing (6)
  • .oca/custom_code_review_guidelines.txt
  • crates/protocols/src/responses.rs
  • model_gateway/src/routers/grpc/common/responses/utils.rs
  • model_gateway/src/routers/grpc/harmony/builder.rs
  • model_gateway/src/routers/openai/responses/utils.rs
  • oracle-schema-config.local.yaml

Copy link
Copy Markdown

@chatgpt-codex-connector chatgpt-codex-connector bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: ec689199c9

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

Copy link
Copy Markdown

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Inline comments:
In `@crates/mcp/src/transform/transformer.rs`:
- Around line 396-420: Add additional unit tests for
ResponseTransformer::transform covering the edge cases for
ResponseFormat::ImageGenerationCall: one test that passes a direct JSON string
(to exercise the fallback path that treats the whole payload as a string), one
test that passes a non-string/non-object JSON value (to exercise the branch that
calls to_string() on the payload), and one test that asserts error payload
handling (if ResponseTransformer returns an error variant or Error payload
field) to verify the transformer produces the expected ImageGenerationCall
status and result; keep the same test structure as
test_image_generation_transform but vary the input payloads and expected id
("ig_req-..."), status, and result assertions.
- Around line 106-124: The to_image_generation_call function currently always
returns ResponseOutputItem::ImageGenerationCall with
ImageGenerationCallStatus::Completed and Some(result); update it to inspect the
MCP result for error/partial indicators (e.g., presence of "error" key, an
explicit "status" field like "failed"/"incomplete"/"generating", or non-2xx
error payload shapes) and map those to the appropriate ImageGenerationCallStatus
variants (InProgress/Generating/Incomplete/Failed) from the protocol, and set
result to None or include the error message only for Failed/Incomplete as
appropriate; locate and change logic in to_image_generation_call and the
construction of ResponseOutputItem::ImageGenerationCall (id, status, result) so
downstream consumers see correct status instead of always Completed.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: ASSERTIVE

Plan: Pro

Run ID: ca785e4b-e333-4866-b2ad-c5634359f518

📥 Commits

Reviewing files that changed from the base of the PR and between ec68919 and e03b4b7.

📒 Files selected for processing (4)
  • crates/mcp/src/core/config.rs
  • crates/mcp/src/transform/transformer.rs
  • crates/mcp/src/transform/types.rs
  • model_gateway/src/routers/mcp_utils.rs

Copy link
Copy Markdown

@chatgpt-codex-connector chatgpt-codex-connector bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: e03b4b7c90

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

Copy link
Copy Markdown

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (2)
model_gateway/src/routers/grpc/common/responses/streaming.rs (2)

476-484: 🧹 Nitpick | 🔵 Trivial

Consider defining a constant for the event type string.

The other variants use constants (e.g., WebSearchCallEvent::IN_PROGRESS). For consistency and maintainability, consider adding a similar constant like ImageGenerationCallEvent::IN_PROGRESS in the event_types module rather than using a hardcoded string.

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@model_gateway/src/routers/grpc/common/responses/streaming.rs` around lines
476 - 484, The match arm for ResponseFormat::ImageGenerationCall is using a
hardcoded string; instead add a constant like
ImageGenerationCallEvent::IN_PROGRESS in the event_types module and use that
constant here. Update the event_types module to define
ImageGenerationCallEvent::IN_PROGRESS (matching the naming/pattern of
WebSearchCallEvent::IN_PROGRESS), then change the match to return
ImageGenerationCallEvent::IN_PROGRESS so all ResponseFormat variants use the
same constant pattern and maintain consistency with emit_tool_event.

125-158: ⚠️ Potential issue | 🔴 Critical

Field mismatch: ImageGenerationCall uses id and result, not call_id and output.

Per crates/protocols/src/responses.rs (Context snippet 2), ResponseOutputItem::ImageGenerationCall has fields id, status, and result. However, this function:

  1. Looks up call_id (line 141) which doesn't exist on ImageGenerationCall
  2. Sets output (line 147) instead of result

As a result, ImageGenerationCall items will pass the is_tool_call check but never match, silently failing to update.

🐛 Proposed fix to handle ImageGenerationCall field differences
                     let is_tool_call = matches!(
                         item_type,
                         Some("mcp_call")
                             | Some("web_search_call")
                             | Some("code_interpreter_call")
                             | Some("file_search_call")
                             | Some("image_generation_call")
                     );
-                    if is_tool_call
-                        && item_data.get("call_id").and_then(|c| c.as_str())
-                            == Some(&tool_result.call_id)
-                    {
+                    // ImageGenerationCall uses `id` instead of `call_id`
+                    let call_id_field = if item_type == Some("image_generation_call") {
+                        "id"
+                    } else {
+                        "call_id"
+                    };
+                    if is_tool_call
+                        && item_data.get(call_id_field).and_then(|c| c.as_str())
+                            == Some(&tool_result.call_id)
+                    {
                         // Add output field
                         let output_str = serde_json::to_string(&tool_result.output)
                             .unwrap_or_else(|_| "{}".to_string());
-                        item_data["output"] = json!(output_str);
+                        // ImageGenerationCall uses `result` instead of `output`
+                        let output_field = if item_type == Some("image_generation_call") {
+                            "result"
+                        } else {
+                            "output"
+                        };
+                        item_data[output_field] = json!(output_str);
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@model_gateway/src/routers/grpc/common/responses/streaming.rs` around lines
125 - 158, The update_mcp_call_outputs function currently matches tool call
items by looking for "call_id" and writes an "output" field, which fails for
ImageGenerationCall items that use "id" and "result"; modify
update_mcp_call_outputs (and the loop that checks item_type ==
"image_generation_call") to treat image_generation_call specially: compare
item_data.get("id").and_then(|v| v.as_str()) to tool_result.call_id (instead of
"call_id") and write the tool_result.output into item_data["result"] (preserving
the same serialization strategy you use elsewhere), while keeping the existing
behavior for other tool types that use "call_id" and "output"; ensure you still
set item_data["status"] = "failed" when tool_result.is_error.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Outside diff comments:
In `@model_gateway/src/routers/grpc/common/responses/streaming.rs`:
- Around line 476-484: The match arm for ResponseFormat::ImageGenerationCall is
using a hardcoded string; instead add a constant like
ImageGenerationCallEvent::IN_PROGRESS in the event_types module and use that
constant here. Update the event_types module to define
ImageGenerationCallEvent::IN_PROGRESS (matching the naming/pattern of
WebSearchCallEvent::IN_PROGRESS), then change the match to return
ImageGenerationCallEvent::IN_PROGRESS so all ResponseFormat variants use the
same constant pattern and maintain consistency with emit_tool_event.
- Around line 125-158: The update_mcp_call_outputs function currently matches
tool call items by looking for "call_id" and writes an "output" field, which
fails for ImageGenerationCall items that use "id" and "result"; modify
update_mcp_call_outputs (and the loop that checks item_type ==
"image_generation_call") to treat image_generation_call specially: compare
item_data.get("id").and_then(|v| v.as_str()) to tool_result.call_id (instead of
"call_id") and write the tool_result.output into item_data["result"] (preserving
the same serialization strategy you use elsewhere), while keeping the existing
behavior for other tool types that use "call_id" and "output"; ensure you still
set item_data["status"] = "failed" when tool_result.is_error.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: ASSERTIVE

Plan: Pro

Run ID: 4aa4c854-ad76-461e-acc3-b967a6e7aed9

📥 Commits

Reviewing files that changed from the base of the PR and between e03b4b7 and 648d3a7.

📒 Files selected for processing (2)
  • model_gateway/src/routers/grpc/common/responses/streaming.rs
  • model_gateway/src/routers/openai/mcp/tool_loop.rs

Copy link
Copy Markdown

@chatgpt-codex-connector chatgpt-codex-connector bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 648d3a748f

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

@TingtingZhou7 TingtingZhou7 changed the title [Not Ready For Review] Add Support for image generation tool Add Support for image generation tool Mar 30, 2026
Copy link
Copy Markdown

@chatgpt-codex-connector chatgpt-codex-connector bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 6ca57cc806

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

ItemType::WEB_SEARCH_CALL => WebSearchCallEvent::COMPLETED,
ItemType::CODE_INTERPRETER_CALL => CodeInterpreterCallEvent::COMPLETED,
ItemType::FILE_SEARCH_CALL => FileSearchCallEvent::COMPLETED,
"image_generation_call" => ImageGenerationCallEvent::COMPLETED,
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Keep image tool event identity consistent in streaming

This new arm emits response.image_generation_call.completed, but in the same OpenAI streaming flow transform_streaming_event still maps non-web-search builtins to mcp_call with mcp_ IDs (model_gateway/src/routers/openai/responses/streaming.rs:146-149). For image-generation requests, clients will see response.output_item.added as mcp_call and then completion/done as image_generation_call for the same output index, which changes type/ID mid-stream and breaks consumers that correlate events by item identity.

Useful? React with 👍 / 👎.

Comment on lines +147 to +149
if item_type == Some("image_generation_call") {
let result_str = tool_result
.output
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Propagate image generation status from MCP payload

The new image-generation update path only extracts result from tool_result.output and never reads a returned status; afterwards status is only changed when tool_result.is_error is true. In the Harmony Responses streaming path (which calls this updater before response.completed), payloads such as {"status":"generating"} or {"status":"incomplete"} will be surfaced as completed calls, so clients lose the newly added status semantics and can treat unfinished generations as final.

Useful? React with 👍 / 👎.

Copy link
Copy Markdown

@chatgpt-codex-connector chatgpt-codex-connector bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 2d4662056a

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

| Some("web_search_call")
| Some("code_interpreter_call")
| Some("file_search_call")
| Some("image_generation_call")
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Set image_generation errors to incomplete status

By including image_generation_call in this shared updater, image tool items now pass through the generic error-status logic below, which writes "failed" when tool_result.is_error is true. However ResponseOutputItem::ImageGenerationCall uses ImageGenerationCallStatus (in_progress|completed|incomplete) and does not accept failed, so failed image items become schema-invalid; in finalize() they fail serde_json::from_value(...).ok() and get dropped from the persisted/final response. This occurs in the gRPC Harmony streaming path whenever an MCP image tool returns an error.

Useful? React with 👍 / 👎.

Copy link
Copy Markdown

@chatgpt-codex-connector chatgpt-codex-connector bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 79070250c7

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

Copy link
Copy Markdown

@chatgpt-codex-connector chatgpt-codex-connector bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 4a8cd870d8

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

@chatgpt-codex-connector
Copy link
Copy Markdown

Codex usage limits have been reached for code reviews. Please check with the admins of this repo to increase the limits by adding credits.
Repo admins can enable using credits for code reviews in their settings.

3 similar comments
@chatgpt-codex-connector
Copy link
Copy Markdown

Codex usage limits have been reached for code reviews. Please check with the admins of this repo to increase the limits by adding credits.
Repo admins can enable using credits for code reviews in their settings.

@chatgpt-codex-connector
Copy link
Copy Markdown

Codex usage limits have been reached for code reviews. Please check with the admins of this repo to increase the limits by adding credits.
Repo admins can enable using credits for code reviews in their settings.

@chatgpt-codex-connector
Copy link
Copy Markdown

Codex usage limits have been reached for code reviews. Please check with the admins of this repo to increase the limits by adding credits.
Repo admins can enable using credits for code reviews in their settings.

@github-actions github-actions bot added anthropic Anthropic router changes gemini Gemini router changes kv-index KV index crate changes mesh Mesh crate changes labels Apr 7, 2026
@github-actions github-actions bot removed documentation Improvements or additions to documentation tokenizer Tokenizer related changes python-bindings Python bindings changes dependencies Dependency updates ci CI/CD configuration changes wasm WebAssembly related changes benchmarks Benchmark changes tests Test changes tool-parser Tool/function call parser changes docker Docker configuration changes reasoning-parser Reasoning parser changes auth Authentication crate changes data-connector Data connector crate changes multimodal Multimodal crate changes workflow Workflow crate changes realtime-api Realtime API related changes anthropic Anthropic router changes gemini Gemini router changes kv-index KV index crate changes mesh Mesh crate changes labels Apr 7, 2026
Copy link
Copy Markdown

@chatgpt-codex-connector chatgpt-codex-connector bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 55942ab653

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

Comment on lines +429 to +434
pub enum ImageGenerationCallStatus {
InProgress,
Generating,
Completed,
Failed,
}
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Add missing incomplete image status variant

ImageGenerationCallStatus does not include an Incomplete variant, so any image_generation_call payload with "status":"incomplete" cannot deserialize into ResponseOutputItem. In the streaming path this is particularly risky because ResponseStreamEventEmitter::finalize() deserializes output items with serde_json::from_value(...).ok(), so the image item can be silently dropped from the final response when that status appears. Adding the missing enum variant keeps image status round-trippable and prevents losing output items.

Useful? React with 👍 / 👎.

@TingtingZhou7 TingtingZhou7 changed the title [Ready for Review] Add Support for image generation tool duplicate of https://github.com/lightseekorg/smg/pull/1055 Apr 7, 2026
@TingtingZhou7 TingtingZhou7 changed the title duplicate of https://github.com/lightseekorg/smg/pull/1055 duplicate of https://github.com/lightseekorg/smg/pull/1057 Apr 7, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

grpc gRPC client and router changes mcp MCP related changes model-gateway Model gateway crate changes needs-rebase PR has merge conflicts that need to be resolved openai OpenAI router changes protocols Protocols crate changes

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant