Area
Provider adapters
What are you trying to accomplish?
I need to route Gemini 3.7 Flash / 3.6 Flash multimodal video understanding requests from client agents (e.g. Hermes or custom AI agents) through the OpenCodeX local proxy, allowing requests that utilize Google's new Agentic Video Understanding protocol (processing: "agentic").
What prevents this today?
Google has recently released Agentic Video Understanding (Developer Guide). In this mode, video objects carry processing: "agentic" and pass lightweight URIs or YouTube URLs instead of pre-extracted Base64 frames.
Currently, proxy schema validation and sanitizers may not recognize the processing: "agentic" attribute or URI-based video pointers on Google/Antigravity providers, risking stripping the field or raising payload validation errors.
What should OpenCodex do?
When a client sends a request containing a video object with processing: "agentic" and a video URI/URL:
- The OpenCodeX proxy adapter for Google / Antigravity endpoints should preserve and forward the
processing property in the upstream payload without filtering.
- Support pass-through for video input items using file URIs and public YouTube URLs.
- Smoothly handle server-side iterative Think → Act tool deliberations in the streaming / response loop and return the final grounded timestamps and token usage properly.
Example usage or interface
{
"model": "google-antigravity/gemini-3.7-flash",
"messages": [
{
"role": "user",
"content": [
{
"type": "video_url",
"video_url": {
"url": "https://www.youtube.com/watch?v=example",
"processing": "agentic"
}
},
{
"type": "text",
"text": "Locate the timestamp where the robot arms pick up the gear and summarize the key actions."
}
]
}
]
}Alternatives or workarounds
Currently fallback to static 1 FPS frame decoding, which is token-heavy (up to 88% higher token consumption) and frequently hits proxy payload size limits on long clips.
Additional context
Checks
Area
Provider adapters
What are you trying to accomplish?
I need to route Gemini 3.7 Flash / 3.6 Flash multimodal video understanding requests from client agents (e.g. Hermes or custom AI agents) through the OpenCodeX local proxy, allowing requests that utilize Google's new Agentic Video Understanding protocol (
processing: "agentic").What prevents this today?
Google has recently released Agentic Video Understanding (Developer Guide). In this mode, video objects carry
processing: "agentic"and pass lightweight URIs or YouTube URLs instead of pre-extracted Base64 frames.Currently, proxy schema validation and sanitizers may not recognize the
processing: "agentic"attribute or URI-based video pointers on Google/Antigravity providers, risking stripping the field or raising payload validation errors.What should OpenCodex do?
When a client sends a request containing a video object with
processing: "agentic"and a video URI/URL:processingproperty in the upstream payload without filtering.Example usage or interface
{ "model": "google-antigravity/gemini-3.7-flash", "messages": [ { "role": "user", "content": [ { "type": "video_url", "video_url": { "url": "https://www.youtube.com/watch?v=example", "processing": "agentic" } }, { "type": "text", "text": "Locate the timestamp where the robot arms pick up the gear and summarize the key actions." } ] } ] }Alternatives or workarounds
Currently fallback to static 1 FPS frame decoding, which is token-heavy (up to 88% higher token consumption) and frequently hits proxy payload size limits on long clips.
Additional context
Checks