Skip to content

fix(vision): downscale oversized images and follow image URL redirects - #1

Open
xiaomaozjj666 wants to merge 1 commit into
whitelonng:mainfrom
xiaomaozjj666:fix/image-resize-and-redirects
Open

fix(vision): downscale oversized images and follow image URL redirects#1
xiaomaozjj666 wants to merge 1 commit into
whitelonng:mainfrom
xiaomaozjj666:fix/image-resize-and-redirects

Conversation

@xiaomaozjj666

Copy link
Copy Markdown

Summary

Two practical fixes for the vision tool, both hit real deployments (verified against the ModelScope API-Inference endpoint):

  1. Downscale images that exceed the vision endpoint's input limit. ModelScope API-Inference rejects images above 2048×2048 with HTTP 400 (input size exceed limit 2048x2048). This applies to any 2K/4K screenshot or large photo. The fix resizes (aspect-preserving, fit-inside) via sharp when it is resolvable, and falls back to the original bytes when sharp is not installed — so deployments without sharp keep exactly the previous behavior. Animated GIF inputs reduce to their first frame, which is acceptable for the description use case.

  2. Follow redirects when fetching image URLs. Many public image hosts (picsum.photos, imgur, …) answer with a 302 to their CDN. The tool previously used redirect: 'error', so those URLs failed with fetch failed. redirect: 'follow' keeps the fetch to http(s) targets only (undici rejects non-http(s) redirect targets), preserving the SSRF posture for non-http schemes.

Changes

  • src/index.ts: toImage() becomes async and passes bytes through fitImageWithinVisionLimit() (new helper, VISION_MAX_SIDE = 2048); loadImage() URL fetch follows redirects. The vision endpoint POST keeps redirect: 'error'.

Verification

  • Syntax: esbuild transform passes.
  • Behavior: the equivalent JS patch was applied locally and tested end-to-end: a 2400×1800 PNG was resized to 2048×1536 and described successfully by Qwen3-VL-235B; a https://picsum.photos/... URL (302 → fastly CDN) was fetched and described successfully. A sharp module is resolvable in the DSH host profile.

Notes

  • sharp is intentionally an optional, dynamic import — no new hard dependency.
  • Suggested follow-up: expose VISION_MAX_SIDE as a config option if other endpoints have different limits.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant