Skip to content

ndistreamer 2.0: reengineered real-time pipeline - #9

Merged
lowlevelmetal merged 3 commits into
mainfrom
v2-reengineer
Oct 1, 2026
Merged

lowlevelmetal merged 3 commits into
mainfrom
v2-reengineer

Conversation

@lowlevelmetal

Copy link
Copy Markdown
Owner

A ground-up rewrite. The goal is the same (stream media as an NDI source from Linux), but the architecture, timing model and feature set are new. See CHANGELOG.md for the full list.

Summary

  • One pipeline: demux → decode → process → present, connected by bounded queues and a shared presentation clock. This replaces the three duplicated software/VAAPI/CUDA apps.
  • Accurate A/V timing: frames are sent at their timestamps, audio is the master clock while it plays, and NDI timecodes come from media timestamps.
  • Hardware decoding: -t auto|vaapi|cuda|… accepts any FFmpeg device type. A stream the device can't decode falls back to software, and GPU frames are downloaded straight into NDI's buffer layout.
  • Lower CPU: the default -p auto passes through layouts NDI accepts. At 1080p60 H.264 with the real NDI 6.3 runtime, sender CPU is 33% of a core with software decode and 19% with VAAPI, versus 65% when forcing UYVY. All 720/720 frames were delivered with 0 dropped.
  • Graceful overload: late video is dropped, then late frames skip processing, then only keyframes are decoded. Audio is never starved. Live sources tolerate clock drift and mid-GOP joins.
  • New features: seamless --loop, --vf filters, any FFmpeg input or capture device, multichannel float audio, clean SIGINT/SIGTERM handling, --stats.
  • No NDI SDK needed to build: the runtime is loaded at start-up. The old /opt/ndi SDK path is still searched.
  • Compatible CLI: -i, -s and -t software|vaapi|cuda still work.

Fixes compared to 1.x

  • No longer crashes on files without audio.
  • The NDI frame rate is no longer 0/0.
  • The last frames of a file are no longer lost.
  • Builds with FFmpeg 8+ (avcodec_close was removed upstream).

Testing

  • 85 tests pass locally with GCC 16 and FFmpeg 9.0. The 83 that don't need the NDI SDK also pass under ASan/UBSan, and ThreadSanitizer is clean for project code.
  • The whole-pipeline tests stream generated clips into a fake libndi.so. They check frame counts, real-time pacing, A/V alignment, loop continuity, pixel layouts, async frame lifetime, and overload handling.
  • With the NDI SDK 6.3.2:
    • ndi_abi checks the project's NDI declarations against the official headers with static_asserts.
    • ndi_end_to_end streams to a real NDI receiver through a private discovery server.
    • Both pass.
  • VAAPI decoding was checked on an AMD GPU, with both software and hardware filters.
  • An independent code review found 11 issues. All were fixed and checked against the reviewer's reproductions, and regression tests were added.
  • This PR's CI covers Ubuntu 24.04 (FFmpeg 6.1) with GCC, Clang, and ASan/UBSan.

🤖 Generated with Claude Code

castusmatt and others added 3 commits September 30, 2026 19:29
Replace the three duplicated software/VAAPI/CUDA applications with one
threaded pipeline (demux -> decode -> process -> present) connected by
bounded queues and driven by a shared presentation clock.

- Frames are sent at their presentation timestamps; audio is the master
  clock while it plays, and NDI timecodes derive from media timestamps.
- Hardware decoding through any FFmpeg device type, with per-stream
  software fallback; GPU frames download straight into NDI's layout.
- Pixel formats NDI accepts pass through (new default `-p auto`), cutting
  sender CPU by 45-67% versus converting everything to UYVY.
- Graceful overload: drop late frames when newer ones are ready, skip
  processing of late frames, then decode keyframes only. Audio is never
  starved, and live sources tolerate drift and mid-GOP joins.
- Seamless looping, FFmpeg filters, any FFmpeg input or capture device,
  multichannel float audio, clean signal handling, stats and logging.
- The NDI runtime is loaded with dlopen, so building needs no NDI SDK.

Fixes 1.x crashes on files without audio, the 0/0 NDI frame rate, lost
final frames at end of input, and the build against FFmpeg 8+.

Tests stream generated clips through the whole pipeline into a fake
libndi.so, checking pacing, A/V alignment, loop continuity and frame
lifetime. With the NDI SDK configured, they also verify the ABI against
the official headers and stream end to end to a real receiver. CI builds
with GCC, Clang and ASan/UBSan on Ubuntu 24.04 (FFmpeg 6.1).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Older FFmpeg (6.1, found by CI) does not trim AAC end padding when
decoding, so at a loop point the last block of a pass ran 256 samples
past where the next pass begins. Receivers would play the extra samples
and drift a little further behind on every loop.

The audio processor now holds back one block (about 21 ms). When the
next block starts slightly before it ends, the held block's tail is
trimmed. Larger backward jumps are discontinuities and are left to the
presentation clock.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@lowlevelmetal
lowlevelmetal merged commit 3018a0b into main Oct 1, 2026
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants