Sender: fix long-session stutter — real-time PTS + move video pipeline off the main thread

[DrDavidL]

Summary

The actual fix for the long-session cursor/video stutter that #69 and #81 were chasing. Multiple independent causes were confirmed by live sample of the running sender; this PR addresses all of them.

Supersedes #69 and #81 — this branch contains their commits, so they've been closed and the whole fix lands here as one PR. Branches off the current main (bb457f3) and merges cleanly with no conflicts.

There are four parts, found in order:

1. Real-time PTS for the CGDisplayStream path (timestamp drift).
2. Move the video pipeline off the main thread (UI layout starving frame delivery).
3. Isolate the FPS readout (1 Hz full-window relayout).
4. Fix a CGDisplayStream teardown use-after-free surfaced by Part 2 (24 h soak crash).

---

Part 1 — Real-time PTS (timestamp drift)

The extended-desktop capture path uses CGDisplayStream, and stamped each frame's PTS from a frame counter scaled by the nominal frame rate:

displayStreamFrameSequence += 1
let pts = CMTime(value: displayStreamFrameSequence, timescale: Int32(capturePreset.expectedFrameRate))

CGDisplayStream delivers frames irregularly — event-driven on screen changes, not a fixed cadence. Stamping them as if exactly 1/fps apart makes the synthetic PTS clock drift from wall-clock time, compounding over the session. The receiver paces by PTS, so after a few hours the cursor/video progressively stutter. Cleared by a sender restart (counter resets), not a receiver restart; frames keep flowing throughout (so #81's no-frames watchdog correctly never fired).

Fix: derive PTS from the frame's actual capture time. displayTime is in mach-absolute units — the same host clock the SCStream path already uses — so both capture paths and the receiver agree on timestamp semantics:

var pts = displayTime != 0
    ? CMClockMakeHostTimeFromSystemUnits(displayTime)
    : CMClockGetTime(CMClockGetHostTimeClock())
if let last = lastEncodedDisplayPTS, CMTimeCompare(pts, last) <= 0 {
    pts = CMTimeAdd(last, CMTime(value: 1, timescale: 600))  // strictly increasing PTS for VTCompressionSession
}

---

Part 2 — Move the video pipeline off the main thread

Further field diagnostics (live sample while a screensaver ran on the extended display) showed a second cause the PTS fix didn't cover. TBDisplaySenderSession is @MainActor, and the whole capture→encode→send pipeline ran on the main thread — three main-thread hops per frame (intake, encode-complete, send-complete). With the Session Monitor window open, its SwiftUI hosting view re-laid-out continuously (~18% of the main thread, recursive AppKit constraint layout). Those layout bursts blocked the main thread; per-frame work queued behind them and the pendingVideoPackets >= 3 cap then dropped frames → stutter, worst during a screensaver (continuous frames).

Confirmed: closing the window made it smooth and raised encoder throughput ~40% — pinpointing the main thread as the bottleneck.

Fix: extract TBVideoPipeline, which owns the VTCompressionSession, the pending/in-flight counters, and packet building/sending, and runs entirely on a dedicated serial queue (fd.tbmonitor.sender.pipeline). Encoder setup/teardown are serialized on that queue, so a frame can never encode into an invalidated session. Both capture paths (CGDisplayStream + SCStream) feed the queue directly. The main actor keeps only UI/lifecycle state and reads the two values it polls (sentFrames, lastCaptureFrameAt) under a small lock — no per-frame hop to main. The Part 1 PTS logic is preserved unchanged.

Measured: encode now runs off-main (fd.tbmonitor.sender.pipeline, confirmed in sample); main-thread layout dropped from ~18% → ~1% with the window open.

---

Part 3 — Isolate the FPS readout (the relayout loop noted as out-of-scope on the first pass)

The earlier "continuous SwiftUI re-layout" note is now fixed. senderFPS was @Published on the session and rewritten every second by the FPS timer; via the manager's objectWillChange bubble-up this re-rendered the entire window ~1.8×/sec whenever the Session Monitor window was open.

Fix: move senderFPS to a dedicated TBSessionLiveMetrics observable, displayed by an isolated SessionMonitorFPSRow subview, so the 1 Hz tick re-renders just that row.

Measured (window open, instrumented with Self._printChanges()): card/window re-renders dropped from 1.82/sec → 0/sec, both at idle and during a screensaver.

---

Part 4 — Fix a CGDisplayStream teardown use-after-free

A 24 h soak crashed (EXC_BAD_ACCESS at 0x10): the faulting thread was on fd.tbmonitor.sender.pipeline inside SkyLight's _CGYDisplayStreamFrameAvailable, while the main thread was concurrently in softRestartCapture. autoRestartOnWake fires a soft restart on every display-sleep/wake, so the teardown path runs often during a screensaver and eventually raced a frame callback. Two lifetime bugs in TBDirectDisplayStreamCapture, both introduced by Part 2:

1. stop() released the CGDisplayStream immediately. CGDisplayStreamStop is asynchronous — in-flight frames keep arriving until the stream delivers a final .stopped frame — so a queued frame event fired into a freed stream.
2. The frame handler dereferenced the pipeline via an unretained pointer while teardown set pipeline = nil concurrently.

Fix: hold the pipeline with a strong reference so it (and its delivery queue) outlives every callback — combined with the running guard, a late in-flight frame after pipeline.stop() simply no-ops; and keep the capture object + stream alive from stop() until the .stopped frame arrives (self-retain released in the handler), honoring CGDisplayStream's documented async-stop contract so the stream is never freed with events still queued.

---

Test plan

• Builds clean via the TBDisplaySender scheme.
• sample confirms encode runs on fd.tbmonitor.sender.pipeline, not the main thread.
• Main-thread layout ~1% (was ~18%) and re-renders 0/sec (was 1.82/sec) with the window open + screensaver running.
• FPS readout still updates live; screensaver smooth with the Session Monitor window open.
• Multi-hour Extended Desktop (CGDisplayStream) soak — confirm no drift-into-stutter and no teardown crash (the Part 4 regression manifested only after ~24 h of repeated display-sleep/wake soft-restarts).
• Duplicate Desktop / SCStream path unchanged.
• Connect/disconnect, cable test, and soft-restart (display sleep/wake) cycles behave; no encoder errors after stop/restart.

🤖 Generated with Claude Code

[swellweb]

Hi @DrDavidL, is it possible to move this fix to feature/input-dockstation? Please, the next fix will be made on this branch until it's merged into the main branch. Thank you so much.