Repository navigation
Play TrueHD Atmos and DTS:X with their height audio on Apple TV - #3
Open
kristofferR wants to merge 36 commits into
Open
kristofferR wants to merge 36 commits into
kristofferR wants to merge 36 commits into
Conversation
The remux re-encoded TrueHD and DTS to FLAC, which keeps the channel bed but drops the Atmos and DTS:X objects. On tvOS, iOS and macOS 26, 48 kHz TrueHD and DTS-HD MA tracks now go to SpatialAudioBridge instead, and its 7.1.4 APAC fragments are served as a separate audio rendition on the muxer's timeline. A track without Atmos or DTS:X, or with metadata the library cannot render, fails the launch as decoding starts; the player restarts it at the same position on the FLAC path. AtmosOutputActivator keeps the receiver in Atmos while APAC plays.
…y state A source-switch candidate whose track the library refused aborted the switch, although a cold launch of the same source plays it through the FLAC transcode; retry the candidate there instead. A refusal can also arrive long after playback started, because the remux runs ahead of it, so the fallback restart now resumes or stays paused as the viewer left it instead of following the seek autoplay option.
The remux already falls back to the decode time for segment timing. Without the same fallback here, a stream that only carries decode times never started its APAC session and fell back to FLAC.
A launch that reached the end of the file before the first APAC fragment never became ready and never failed, so playback hung. Fail it as an APAC failure, which restarts it on the FLAC path.
Apple's HLS authoring specification requires the audio and video playlists of a variant to use the same target duration. Each playlist took the target from its own longest segment, so 3.2 s APAC fragments beside 2 s video segments announced 4 and 2. Write both together from the longest segment of either.
…e switch refusals A refusal that arrived before its launch became ready, such as after a track change, sampled the stopped replacement and paused playback; such a launch now restarts as the one before it would. A source-switch candidate refused after the inner player started loading it now abandons that load and retries on FLAC instead of failing the switch.
SpatialAudioBridge's manifest needs Swift 6 tools and its API the 26 SDKs, so Xcode 15 no longer builds Lumen.
1.3.0 turns TrueHD dialogue normalisation off by default, so TrueHD Atmos plays at Apple TV's native Atmos level.
A failure queued while the switch committed ran the pending handler after the pending state was cleared, which dropped it and left the committed candidate on a failed remux. One handler now serves the candidate before and after the commit.
…llback A source switch carried the current source's FLAC fallback to a different file, so its eligible track never tried APAC; the candidate now tries again and still retries on FLAC when refused. A play or pause while the FLAC fallback prepared was reversed when it became ready; such a command now replaces the state the fallback restores.
Packets held for the video timeline returned their backpressure permit, and packets skipped while waiting for a sync point were never counted, so a stream that never started the conversion could buffer or read the whole file before falling back. Past 8 MB of such audio the session now fails as APAC, which restarts it on FLAC.
- Only a decoding refusal from the library keeps a track on FLAC, per track of the source. Encoding, CoreMedia and start-up failures, and a rendition that ends without a fragment, fall back for that launch only, so the next seek tries APAC again. Hot source switches carry the same policy. - TrueHD converts only when FFmpeg labels it Atmos; every DTS-HD MA track is still offered, since FFmpeg labels some DTS:X as plain MA. - A late APAC failure no longer restarts a player that already failed. - The remux reads at most the buffer duration ahead of playback, as the decoding engine does, so starting or seeking no longer converts the rest of the film. ProAVPlayer feeds the playback position to the session. - An APAC fragment without a duration fails the conversion instead of writing EXTINF:0. - The muxer timeline for the first video packet is a static function, and the sync point check and refusal test are reachable from tests. - Releasing the Atmos activator detaches it, and multichannel content is set on Lumen's audio-session queue and reset when APAC stops.
The timeline cases include the missing decode time after a seek with B-frames and a decode time alone. The bound test feeds a real conversion 8 MB of TrueHD without a major sync and expects a fallback for that launch only.
1.5.0 ships universal macOS and iOS simulator slices, so universal Mac builds of apps that embed Lumen work again; generic Apple TV simulator builds still need ARCHS=arm64. The README now says so, and lists iOS as building with spatial audio on AirPods still to be tested.
…ld, skip apac on airplay - The read-ahead compared the source's own video times with a playback position counted from the start of the file, so a file whose times start late, such as a transport stream, paused the remux for good. The item now subtracts the format's start time. - Each launch seeds the playback position with its start, so a cold start, a restart or a source-switch candidate reads at most the buffer duration ahead before the player reports a position. - Multichannel content is a process-wide setting, so it is now held while any player plays APAC and returns to its previous value after the last. - An AirPlay receiver plays the stream itself, so launches leave the APAC rendition out while external playback is active, and a launch playing APAC restarts on FLAC when AirPlay starts.
…d and airplay - The silent Dolby Digital Plus Atmos stream exists to switch an HDMI receiver into Atmos, so the activator now runs on tvOS only and SpatialAudioPlayback is linked there alone. iOS and macOS still hold multichannel content while a launch plays APAC. - A player released without a shutdown gives up its multichannel hold. - When AirPlay starts while a source-switch candidate is preparing an APAC rendition, the candidate restarts without it.
A launch that needed more than the buffer duration of source to reach the readiness gate, such as one with long GOPs, paused before the player attached, and its seeded playback position never moved, so it stalled for good. The read-ahead now applies once the session has fired ready.
…ch stays When AirPlay started while both the active launch and a switch candidate played APAC, the candidate was rebuilt and then abandoned by the active launch's restart, as any restart abandons a pending switch. The active launch now restarts on its own in that case, and the layer completes the switch cold, without APAC while AirPlay lasts.
A launch restarted on FLAC for AirPlay, or started during it, stayed on FLAC after playback returned to the device. The session now records when it left APAC out for an eligible track the source has not refused, and the player restarts such a launch in place with APAC once external playback ends.
…t need Every new remux item runs KSOptions.isSpatialAudioEnabled for its audio tracks, which turns multichannel content off on an HDMI route. A seek keeps the player's hold set, so nothing turned it back on. The receiver still showed Atmos with it off; on tvOS the activator is what switches it, and on iOS the existing route check already turns it on for spatial outputs.
With AirPods, APAC sounded spatial with head tracking while the silent Atmos stream played and flatter without it, so iOS 26 attaches the activator with the same lifecycle as tvOS. It stops as soon as AirPlay starts, ahead of the restart without APAC.
1.6.1 fixes two DTS:X faults after a seek or a mid-stream start, both in the decoder Lumen's DTS:X path uses: the first frame or two came out distorted (an audible click) and now play as silence, and a dropped pending frame pulled the audio about 11 ms early.
2.0 reports tracks it will never render as unsupported rather than as a decoding error, so refusal memory now matches that class. Every other error, including a discontinuity, falls back for the launch only: the point lies ahead of playback, so the next seek starts a new session. Sync points come from the library's own check.
A track the library refuses, such as DTS-HD MA without DTS:X, used to start the APAC conversion and fail the launch, which then reopened on FLAC. The launch now decodes about a second of the track first and keeps a refused one on FLAC from the start.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Hi! This PR lets Lumen play TrueHD Atmos and DTS:X on Apple TV with their height and object audio intact, instead of flattening them.
Why this matters
Apple TV can't send TrueHD or DTS-HD MA to a receiver, so today the remux re-encodes them to FLAC. That keeps the channel bed lossless, but it throws away the Atmos and DTS:X objects: the overhead and moving sounds those formats exist for. On most 4K Blu-ray remuxes the TrueHD track is the only Atmos track, so anyone with an Atmos system loses exactly what they set it up for.
Apple's own spatial format, APAC, can carry that audio, and Apple TV plays it through its native spatial pipeline. This PR uses SpatialAudioBridge, an open-source library I wrote for this, to render the objects to 7.1.4 and encode them as APAC on the device. With a silent Dolby Digital Plus Atmos stream playing alongside, the receiver switches to Atmos and the height channels come back.
What changes
ProAVSpatialAudioTranscoder. It hands the demuxed packets to the library and serves the APAC fragments as a separate HLS audio rendition next to the video-only remux.ProAVPlayerrestart at the same position on the FLAC path, keep the play or pause state, and keep that track on FLAC. Other errors, including a jump in the track's timestamps, fall back for that launch only, and the next seek tries APAC again. Older systems never try.maxBufferDuration(30 s by default). Reliable declared audio-end metadata allows video-only pacing after audio ends. Unknown-duration audio tails retain the limitation described below.AtmosOutputActivatorplays the silent Atmos stream that switches the receiver over while APAC plays, and stops on stop, failure, launches without APAC and while AirPlay is active.SpatialAudioBridgeNotices.textin their acknowledgements, as the README says.Testing
Current validation (SpatialAudioBridge 2.0.0, commit
c62594a4ac5756bd7aafbe9268ecb5191ad0c72d): the fork cloud run passed with Xcode 26.6: macOS arm64 and x86_64 builds, all 117 tests on macOS and on the iOS and tvOS Simulators, and the resolved-library check. The validation ref differs from this source tree only in the manual build workflow. The exact candidate also received a clean Codex review in the isolated fork review.The earlier full-suite run exposed a pre-existing
KSPlayerLayerlifetime bug: releasing a layer before playback initialized its lazy timer during deinitialization. This candidate fixes that without changing play/pause timer behavior and adds a focused release-before-play test; the full suite then passed.On Apple TV 4K (2nd generation), tvOS 27.2, Jellyfin direct play, with this candidate:
Runtime limits: this candidate has not been retested on macOS, AirPods or AirPlay. AirPlay connect/disconnect during source opening, a queued source switch or an internal restart still needs a device check. Read-ahead uses the least advanced required audio/video stream and stops requiring audio only at a reliable declared track end. For an audio track with no reliable end metadata, the video-only tail can still read through to EOF. No EOF is inferred from a temporary absence of audio packets.
Earlier runtime checks:
ProAVPlayerandAVPlayer: TrueHD Atmos, DTS:X and an alternate DTS:X presentation that FFmpeg labels plain DTS-HD MA play as APAC from the start and after seeks. Plain TrueHD and DTS-HD MA fall back to FLAC. Both renditions put the same source time at the same timestamp. Unit tests cover the APAC rendition, its readiness gate, the track gate, the muxer timeline (including B-frames and missing decode times), which errors refuse a track, the read-ahead and the 8 MB bound.A note on the library: it's young (2.0), I maintain it, and it renders Atmos objects with the object panner from the public spec (ETSI TS 103 448). It isn't a licensed Dolby renderer. Happy to split this up, change anything, or answer questions!
Resumo em português: este PR faz o Lumen tocar TrueHD Atmos e DTS:X no Apple TV como APAC, o formato espacial da Apple, mantendo os canais de altura e os objetos que hoje se perdem na conversão para FLAC. Faixas sem Atmos ou DTS:X continuam no caminho FLAC de hoje, e não há mudanças na interface.