Skip to content

Support Google's fileAPI when sending large files to Google Models #1206

Description

@DESU-CLUB

Please describe the feature you want
Currently on Pochi, large video files that are attached to Pochi causes Pochi to OOM. This could be partially due to Vercel's AI SDK internally not garbage collecting the attachment properly in step, as seen in this issue.

A quick workaround for this is to support videos as URLs, which would allow for the model to not attach the raw data inline, but allow the model to fetch the video outside of AI SDK using supported APIs for file management, like Google's FileAPI.

This on paper should be supported by AI SDK, as seen here

To quote:

The AI SDK will automatically download URLs if you pass them as data, except for https://generativelanguage.googleapis.com/v1beta/files/ and YouTube URLs. You can use the Google Generative AI Files API to upload larger files to that location. YouTube URLs (public or unlisted videos) are supported directly

This prevents OOM by not forcing the download, and allow us to read large files using Google's FileAPI.

Logs:

bun run pochi -p "Video analyse please" -a ../Documents/projects/video-analysis/basketball_data/duke_cal_q1.mp4

Pochi 0.5.0-dev (update available 0.5.103)
────────────────────────────────────────────────────────────────────────────────────────────────────
[cli] Reading attachment: /Users/warrenlow/Documents/projects/video-analysis/basketball_data/duke_cal_q1.mp4
[cli] Attachment read: 160372819 bytes (152.94 MB)
[cli] MimeType: video/mp4
[cli] Calling fileToUri...
[store-blob] fileToUri called with file size=160372819 bytes (152.94 MB), type=video/mp4
[store-blob] fileToUri: arrayBuffer converted to Uint8Array
[store-blob] fileToUri: stored blob, uri=store-blob:43cdb50cc853f0553872dbd1b3425f788356a994967d44da0d0161634578d624
[cli] fileToUri returned: store-blob:43cdb50cc853f0553872dbd1b3425f788356a994967d44da0d0161634578d624
[cli] Attachment added to parts
[cli] Creating TaskRunner...
[cli] Parts count: 3
[cli]   - text part: <system-reminder>Attached file: ../Documents/proje...
[cli]   - file part: mediaType=video/mp4, url=store-blob:43cdb50cc853f0553872dbd1b3425f788356a994967d44da0d0161634578d624
[cli]   - text part: Video analyse please...
[cli] TaskRunner created
You
⠋ [cli] Calling runner.run()...
[task-runner] run() called
[task-runner] Starting step loop
[task-runner] Calling step()
[task-runner] step() called
[task-runner] step(): lastMessage role=user, parts count=3
[task-runner] step(): calling process()
[task-runner] step(): process() returned: retry
[task-runner] step(): calling chatKit.chat.sendMessage()
[flexible-chat-transport] sendMessages called
[flexible-chat-transport]   messages count: 1
[flexible-chat-transport]   - role=user, parts=3
[flexible-chat-transport]     - text: <system-reminder>Attached file: ../Documents/proje...
[flexible-chat-transport]     - file: mediaType=video/mp4, url=store-blob:43cdb50cc853f0553872dbd1b3425f788356a994967d44da0d0161634578d624
[flexible-chat-transport]     - text: Video analyse please...
[flexible-chat-transport]   llm.id: pochi/google/gemini-3-flash
[flexible-chat-transport] model created
[flexible-chat-transport] preparedMessages ready, calling convertToModelMessages...
[flexible-chat-transport]   Memory before convertToModelMessages: 260 MB
[flexible-chat-transport]   Memory after convertToModelMessages: 260 MB
[flexible-chat-transport] modelMessages ready, calling streamText...
[store-blob] makeDownloadFunction called
[store-blob] downloadFn called with 1 items
[store-blob]   - url=store-blob:43cdb50cc853f0553872dbd1b3425f788356a994967d44da0d0161634578d624, isUrlSupportedByModel=false
[store-blob] downloadFn: fetching from blobStore: store-blob:43cdb50cc853f0553872dbd1b3425f788356a994967d44da0d0161634578d624
Video analyse please ⠙ [store-blob] downloadFn: got blob with 160372819 bytes (152.94 MB)
2026-01-28 23:00:08.614	ERROR	/node_modules/ai/dist/index.mjs:9906	LiveChatKit	onError

 Error  Out of memory, 3254, 49, 3810, 54, /Users/warrenlow/pochi/node_modules/ai/dist/index.mjs
error stack:
  • index.mjs	<anonymous>
	/node_modules/ai/dist/index.mjs:3810
  • index.mjs	<anonymous>
	/node_modules/ai/dist/index.mjs:3427
  • index.mjs	<anonymous>
	/node_modules/ai/dist/index.mjs:7267
  • index.mjs	<anonymous>
	/node_modules/ai/dist/index.mjs:7265
  • index.mjs	processQueue
	/node_modules/ai/dist/index.mjs:7258
  • index.mjs	processQueue
	/node_modules/ai/dist/index.mjs:7252
  • index.mjs	<anonymous>
	/node_modules/ai/dist/index.mjs:7273
  • native	new Promise
	native:1
  • index.mjs	run
	/node_modules/ai/dist/index.mjs:7264
  • index.mjs	run
	/node_modules/ai/dist/index.mjs:7263
  • index.mjs	transform
	/node_modules/ai/dist/index.mjs:3427
  • index.mjs	transform
	/node_modules/ai/dist/index.mjs:3426
  • native	<anonymous>
	native:1
  • native	<anonymous>
	native:1
  • native	<anonymous>
	native:1
  • native	<anonymous>
	native:1
  • native	<anonymous>
	native:1
  • native	<anonymous>
	native:1
  • native	<anonymous>
	native:1
  • native	<anonymous>
	native:7
  • native	processTicksAndRejections
	native:7
[task-runner] step(): sendMessage() completed
[task-runner] step() returned: retry
Video analyse please ⠧

Additional context
Add any other context or screenshots about the feature request here.


Please reply with a 👍 if you want this feature.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

enhancementNew feature or request

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions