Skip to content

fix(canvas): size video nodes to the video's real aspect ratio - #106

Open
Luna Qiu (lunaqiu) wants to merge 1 commit into
microsoft:mainfrom
lunaqiu:fix/video-node-aspect-ratio
Open

fix(canvas): size video nodes to the video's real aspect ratio#106
Luna Qiu (lunaqiu) wants to merge 1 commit into
microsoft:mainfrom
lunaqiu:fix/video-node-aspect-ratio

Conversation

@lunaqiu

Copy link
Copy Markdown
Contributor

Fixes #105

Problem

Dragging or pasting a video onto the canvas created the node at the generic 400×300 (4:3) default, so a 720×1280 clip rendered letterboxed with large empty bars.

uploadFileToNodeInput uploaded the file without measuring it, so naturalDimensions was absent and resolveAddNodes fell back to the video type default. The image branch directly above already measured via getImageDimensionsFromBlob; the only code that ever probed a video was a private getVideoDimensions inside CanvasToolbar.tsx.

The wrong ratio was sticky: resolveGeometryEdit preserves the current box ratio for image/video, so every later resize faithfully preserved the incorrect 4:3.

Change

  • utils/io/media.ts — add getVideoDimensionsFromBlob, mirroring getImageDimensionsFromBlob (preload='metadata', revokes the object URL on both paths).
  • nodeInputBuilders.ts — the video branch now measures alongside the upload and passes naturalDimensions.
  • CanvasToolbar.tsx — drops its two private probes and reuses the shared helpers.

Probing is non-fatal. A codec the browser cannot decode (e.g. ProRes .mov) still yields a node at the default size; previously the rejected probe propagated through Promise.all and the toolbar path dropped the node entirely.

Not covered

Agent-created video nodes hit the same default — the aspect-ratio normalizers in apps/server/src/modules/canvas/canvas-executor.ts are all gated on nodeType === 'image'. Left out deliberately: there is no video-generation tool, so an agent can only point at a pre-existing video artifact, and correcting it server-side would need an MP4/WebM metadata parser. Existing mis-sized nodes are not corrected retroactively; re-adding the file fixes them.

Verification

  • Drag and Ctrl+V a 720×1280 .mp4 → node is 400×711 instead of 400×300.
  • Toolbar Upload file picker and all image paths unchanged.
  • pnpm typecheck clean; pnpm lint 0 errors; web unit tests 44 passing.

Dragging or pasting a video created the node at the generic 400x300 (4:3)
default: the drop/paste builder uploaded the file without measuring it, so
`naturalDimensions` was absent and `resolveAddNodes` fell back to the type
default. A 720x1280 clip rendered letterboxed inside a 4:3 box.

Add `getVideoDimensionsFromBlob` alongside `getImageDimensionsFromBlob` and
call it from the video branch of `uploadFileToNodeInput`. `CanvasToolbar`
now reuses the shared helpers instead of its private copies.

Probing is non-fatal: a codec the browser cannot decode (e.g. ProRes .mov)
still yields a node at the default size, where previously the rejected probe
propagated through `Promise.all` and dropped the node entirely.

Fixes microsoft#105
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug]: Dropped or pasted videos are sized 4:3 regardless of the video's real aspect ratio

1 participant