Export Remotion videos on an NVIDIA GPU and leave the CPU free. On a 75 s production video with a presenter and motion graphics, the layered render used 112 s of CPU instead of 460 s (4.1× less) and finished in 49 s instead of 111 s. The comparison is Remotion's own GPU path, <Video> + NVENC, on the same machine.

Português: LEIAME.md

Where this is going

The goal of this toolkit is to move the video work of a Remotion export (decode, crop, framing, compositing) out of Chrome and onto the GPU, leaving Chrome only the graphics, while you keep writing videos in React exactly as today. NVENC alone is not that; Remotion already offers it.

Today that works for one family of videos: footage with crop, position and zoom, plus graphics on top. Two things would widen it, and both need people outside Konecz: compatibility with richer compositions and evidence on other GPUs. See the roadmap.

How it works

In a Remotion render with footage, React is not the problem. Almost all of the CPU goes to decoding the footage and taking a screenshot of every frame.

So the footage leaves the browser. The same React code that frames the presenter (box, crop, slow zoom, punch-in, spring into a card) writes a framing plan. Chrome renders only the graphics, as PNG with alpha. ffmpeg does the rest on the GPU.

• The plan is compiled to piecewise-linear libplacebo expressions, with at most 0.05 px of error.
• libplacebo blends in linear light and Chrome blends in gamma. The inputs are tagged so that the blend matches Chrome's.
• 25 fps footage is mapped to 30 fps output the way Remotion picks frames (fps=30:round=up).
• Dedup (optional) hashes the page state before each capture. A frame whose state was already captured becomes a link to that PNG. On video R, 29% of the frames were reused, and every reused frame matched a fresh capture byte for byte.

Which path to use

Quick start

Requirements:

• Linux;
• an NVIDIA GPU with NVENC;
• ffmpeg 7 or newer, built with libplacebo and Vulkan;
• Node 20 or newer;
• Remotion 4.0.536.

npm install
npm run doctor            # checks ffmpeg, the NVIDIA driver, Vulkan and Remotion, path by path
npm run example:p1        # the example video through Remotion + <Video> + NVENC
npm run example:layers    # the same video through the layered render
npm test

The example makes its own synthetic footage on first use, so no media is downloaded. Outputs go to examples/presenter/out/.

Using it with Claude Code

This project was built with Claude Code, and it ships a skill for people who edit video with Claude.

• .claude/skills/remotion-gpu/SKILL.md teaches Claude to pick the path, write compositions that the layered render accepts, and measure before and after.
• docs/claude-code.md explains how to install the skill in your own Remotion project and what to ask for.

Benchmark, and how to send your results

Synthetic videos A to E cover typography, a presenter, several videos at once, heavy effects and a slide. The pilot below has N=1 and 4 s per cell, so it is indicative only. The quick run that followed was stopped halfway, and it ranks the paths the same way.

npm run bench:smoke       # N=1, 4 s: checks that every path runs on your machine
npm run bench:quick       # N=3, 15 s, 1080p
npm run bench:full        # N=5, 30 s, 1080p and 4K (hours; run it overnight)

Every run writes bench/results/<run>/ with the raw rounds, the environment, an analysis and a submission.json, the one file to attach to a benchmark-result issue. Read it before you send it. We only have numbers from one RTX 5090 on a busy server. Results from other cards, from AMD or Intel, from laptops and from idle machines are the most useful contribution. CONTRIBUTING.md explains how to send them.

What did not work

What is new and what is not

Much of this was done before, and the paper cites it:

• Remotion itself recommends <Video> from @remotion/media and supports NVENC output.
• HyperFrames already composites video outside the browser, in layers and on the CPU. It also skips repeated frames, by default.
• CasparCG composites HTML graphics over video on the GPU, in real time.
• Remotion's Experimental.Clipper tried partial capture and was removed because it made renders slower.

What we add:

• the framing written in the React component becomes a plan that runs from end to end on the GPU (NVDEC, libplacebo, NVENC);
• the result is measured against Chrome's render, next to the encoder's noise floor;
• the CPU numbers and the benchmark are published as open data.

Roadmap

Evidence on more hardware. Every number so far comes from one RTX 5090 on a busy server.

Hardware	Status
RTX 5090, Linux	measured (this repository)
RTX 3060, 4060, 4070, 4090	wanted: run bench:quick and send the result
Professional cards (RTX A-series, L4) and cloud T4	wanted
Laptop GPUs	wanted
AMD and Intel (VA-API)	not supported yet: P2 needs a VA-API path
CPU only	runs (--device cpu), as a baseline

Compatibility. What keeps a composition on P1 today, in the order we plan to tackle it:

• several takes on screen at once (picture-in-picture, grids);
• masks, rotation and blur on the footage, done on the GPU;
• transitions between takes;
• reading the framing plan from any <Video> in a composition, without a component written for it;
• dedup on Remotion versions after 4.0.536 (ideally through a public hook upstream instead of a patch).

Method. A full benchmark run (N≥5, idle machine, 4K), and HyperFrames as one more baseline.

Ideas and results go to Discussions; benchmark results to an issue with the Benchmark result template.

Paper

paper/paper.md (English) and paper/artigo.md (Portuguese) cover the method, the results, the threats to validity and the references. Every number comes from data/, and data/README.md gives the source, run count, clip length and machine load for each one.

Notices

• Remotion has its own license. Companies above a certain size need a paid license; see remotion.dev/license. This project does not change that.
• Konecz is not affiliated with Remotion.
• Dedup uses an internal part of Remotion (takeFrame in @remotion/renderer). It works only with version 4.0.536 and refuses to run on any other. The rest works without it.
• The numbers come from one machine: Ryzen 7 7700X, RTX 5090, with production sites and a resident LLM server sharing it, capped at 8 of 16 threads. Your ratios will differ.

License

MIT, see LICENSE. The fonts embedded in the figures (Schibsted Grotesk and IBM Plex Mono) are under the SIL Open Font License; see figures/build/fonts/.

Made by Konecz with Claude Code.