Skip to content

M6/J873g: enable Kopsia hardware video decoding - #2139

Draft
sithglan wants to merge 11 commits into
aurora-silicon:integration/aurorafrom
sithglan:thomas/j873-kopsia
Draft

sithglan wants to merge 11 commits into
aurora-silicon:integration/aurorafrom
sithglan:thomas/j873-kopsia

Conversation

@sithglan

@sithglan sithglan commented Nov 17, 2026 •

Copy link
Copy Markdown
Contributor

This enables V4L2 Request hardware decoding on the Mac mini M6 (J873g/T8152). With the request-enabled FFmpeg/mpv packages, H.264, HEVC/Main10, VP9 and AV1 decode through Kopsia, and mpv presents DRM PRIME frames without a download to the CPU.

Draft for review of the initial hardware bring-up: completion is polled and the decoder power domain stays on. Suspend and fault recovery are not qualified.

Scope and prerequisites

Based on integration/aurora at 0abc2f385e54. The first three commits carry the Gen3 DART binding, locked-stream support and VHE TLBI fix from #1800, with their source commits recorded. The remaining seven commits add:

  • PSD service 50 as the shared power domain of the decoder and its DART, waiting for ACK before reading the state that consumes it.
  • Cold initialization restricted to the inactive AVD Gen3 instance, using its TLBI aperture and a 64 GiB DMA window at 1 TiB.
  • A Kopsia host backend with the MCU halted, one frame in flight, and VP/PP completion polling while work is active. Existing codec programs and completion handling are reused; no external decoder firmware is needed.
  • Bindings and the J873g decoder/DART description.

The 346 decoder tunables match the MIT-licensed avd-fw v5/tier2 table, with its attribution retained. #2106 is a related Gen3 DART implementation to reconcile during review. There are no new aura, FFmpeg or mpv source changes in this series.

Hardware validation

Mac mini M6, apple,j873g / apple,t8152, Debian forky/sid. Tested kernel: 7.3.0-aurora-j873-avd+ (#6), built from the local M6 integration base 805d9fcc6812 plus this decoder implementation. The exact rebased PR branch has not been booted; its build/schema/host checks are listed separately below.

Both asahi,os-fw-version and asahi,system-fw-version report unknown; the installed aura Stage 1 version was not recorded. Userspace: FFmpeg 7:9.0.2-6+v4l2req2, mpv/libmpv2 0.42.0-1+v4l2req2, Mesa 26.3.0~aurora.wip.ac9d556248c-1, Sway 1.12-1, OpenGL renderer Apple M6 (G18G B1).

Repeated after the final reboot:

Codec Frames compared with software Result
H.264 60 Exact
HEVC 60 Exact
HEVC Main10 60 Exact
VP9 60 Exact
AV1 60 Exact

The tests force -hwaccel v4l2request -hwaccel_output_format drm_prime, download to NV12/P010, normalize to the software pixel format and compare nonempty frame hashes. Logs must also identify the AVD media driver, so software fallback cannot pass. Three simultaneous H.264/HEVC/VP9 processes also produced 60 exact frames each.

Additional hardware results:

  • 4K60 H.264: 120 exact frames; 300 frames decoded to a DRM PRIME null sink at 112 fps.
  • mpv zero-copy rendering visually checked; a 20-second fullscreen run had zero decoder drops and two presentation drops.
  • After the final reboot, exact seeks forward and backward kept hwdec=v4l2request and DRM PRIME interop, with zero decoder or presentation drops during that test.
  • /dev/video10 and /dev/media0 present; no AVD errors, kernel WARNING/Oops/SError, or failed systemd units in the final boot.

Select hwdec=v4l2request in mpv. FFmpeg needs -hwaccel v4l2request before the input; use -hwaccel_output_format drm_prime when retaining hardware frames. CPU filters need an explicit download.

Checks on this PR branch

  • Clang 22, arm64, W=1: complete AVD objects, PSD and DART objects, and the J873g DTB build. Existing DART unused-helper and GPU reserved-memory DT warnings remain.
  • All three affected bindings pass dt_binding_check; the J873g DTB passes validation against those schemas.
  • DART host checks: 15/15; AVD host checks: 12/12; hardware defaults: 1/1.
  • git diff --check passes. Checkpatch reports no errors; remaining warnings concern new-file MAINTAINERS coverage and British spelling of acknowledgment.
  • The hardware-tested local tree also completed an Image/modules/DT build.

Remaining work

PostOffice completion and runtime power gating are not implemented. The PSD domain remains powered. Gen3 fault interrupts remain disabled, and recovery reset is disabled for Kopsia: the existing watchdog parks DMA and retires the decoder rather than issuing an unqualified reset. Suspend/resume and fault injection have not been tested. Encoding and browser integration are outside this change.

The DARTs in front of the T8152 display coprocessors (DCP, DCPEXT) and
their display pipes (ADT "dart,gen3", version 0x300) use the T8110
register layout, but they belong to the SPTM, and the bootloader hands
them over locked. Their TLB command register at 0x80 answers translation
queries instead of flushing. A stream is invalidated the way iBoot does
it: a payload naming the DART and the stream is written to the SoC's TLB
invalidation aperture, which these DARTs share, and a broadcast TLBI
follows.

Add apple,t8152-dart-gen3 for them, without the apple,t8110-dart
fallback, since a T8110 driver would flush through the wrong register,
and a "tlbi" register entry for the DART's window in the aperture.

Signed-off-by: Thomas Glanzmann <thomas@glanzmann.de>
(cherry picked from commit 36cd293)
The T8152 display DARTs (ADT "dart,gen3", version 0x300) use the T8110
register layout but belong to the SPTM: their TLB command register at 0x80
answers translation queries instead of flushing. A stream is invalidated
the way iBoot does it, by writing a 128-bit payload (valid, route, SID)
to the SoC's shared TLBI aperture and broadcasting a TLBI, which the DART
snoops. The route is the DART's 0x4604 register.

Admit a Gen3 DART only when the bootloader hands it over locked (its
streams cannot be set up cold yet), map the aperture from a second "tlbi"
reg entry without claiming it, since every Gen3 DART shares it, flush the
whole stream where a range flush would go through 0x80, and do not claim
the fault interrupt or drain stale exceptions, whose handling acknowledges
through that register as well.

The range flush test fixture gains the new hw flag.

Signed-off-by: Thomas Glanzmann <thomas@glanzmann.de>
(cherry picked from commit 1ce2050)
The Gen3 DART snoops the broadcast TLBI that follows the aperture write.
iBoot and the aura bootloader issue TLBI VALE1OS at EL2 without VHE,
which targets the EL1&0 regime. On a VHE kernel the same instruction
targets EL2&0 while HCR_EL2.TGE is set, so clear TGE around it, as KVM
does to reach the EL1&0 regime.

The J873g display works with this in place; the effect of the change on
its own has not been isolated on hardware.

Signed-off-by: Thomas Glanzmann <thomas@glanzmann.de>
(cherry picked from commit d57b118)
@sithglan

Copy link
Copy Markdown
Contributor Author

Rebased onto integration/aurora 0abc2f3. It is now a544621 (was b782c27); the old branch no longer merged.

On 0abc2f3: Image, modules and the J873g DTB build (LLVM 22, the M6's config, W=1, no warnings in the touched files), apple-dt-check.sh: 0 new CHECK_DTBS warnings, and its 3 changed bindings pass dt_binding_check. run-host-tests.py: 427/427. #2178 carries these commits too and was booted on the M6 with them: /dev/video10 (Kopsia) probes as before.

@RyanTheTide

Copy link
Copy Markdown
Contributor

Thanks, Thomas. Bit-exact results for five codecs, with a check that rules out software fallback, are a strong start for the M6 decoder.

A few code asks, so this keeps working as more T8152 machines are added:

  • arch/arm64/boot/dts/apple/t8152-j873-avd.dtsi: ps_avd and avd are SoC blocks, and avd_dart is already in t8152-peripherals.dtsi (iommu@485020000). Please describe all three in t8152-peripherals.dtsi with status = "disabled" (the new DART compatible, tlbi reg, power domain and tunables too). The board then only sets status = "okay", and the J804 and J833ct get the same description.
  • drivers/iommu/apple-dart.c, cold Gen3 check: res->start != 0x485020000ULL hard-codes this instance's address. apple,t8152-avd-dart already selects the instance, and reg gives the address, so please drop the address test. The version and stream-count checks are fine.
  • avd-backend-kopsia.c avd_kopsia_poll() (the usleep_range(100, 200) loop) and avd-drv.c skipping the IRQ for Kopsia: if the ADT lists the decoder's interrupt, please describe it and complete VP/PP from the handler. Keep polling only as a fallback when the node has no interrupt, with a comment saying why.
  • Before this leaves draft: pmgr-psd.c sets GENPD_FLAG_ALWAYS_ON and has no power_off, and Gen3 DART faults are not reported (dart->irq = -1). Please add a power-off path and a Gen3 fault handler, or state the hardware reason for each in a comment.

Row: boot the exact PR head on the J873g and repeat your comparison for H.264, HEVC, HEVC Main10, VP9 and AV1, 60 frames each, all exact. Then run 4K60 H.264 to a null sink at 60 fps or more. With power-off added, /sys/kernel/debug/pm_genpd/pm_genpd_summary should show the AVD domain off after 10 s idle and on again for the next decode.

After that we will run a no-regression row for the shared DART changes on our A18 Pro (Neo) and M4 Pro.

@sithglan

Copy link
Copy Markdown
Contributor Author

Thanks. I've done the parts I can verify without the M6. The head is now ee96cb6538a7:

  • DT: ps_avd, avd and avd_dart are now described in t8152-peripherals.dtsi, all with status = "disabled". That includes the apple,t8152-avd-dart compatible, the tlbi reg, the power domain, apple,dma-range and the tunables (b967e8a1b335). t8152-j873-avd.dtsi only sets the three nodes to okay (ee96cb6538a7). I compared the preprocessed J873g tree before and after, with label references kept. The only difference is an explicit status = "okay" on ps_avd and avd. The J804 and J833ct now get the same description, disabled. dtbs_check reports nothing for the AVD nodes on any of the three.
  • DART: the cold Gen3 admission no longer tests res->start, so apple,t8152-avd-dart alone selects the instance. The version, stream-count, PSD domain, tunable and inactive-stream checks stay (fedf1864d853). apple-dart.o builds, with no new W=1 warnings.
  • Decoder interrupt: the ADT's avd node lists interrupts = [308, 268]. These follow the PostOffice pattern (299 + N, 259 + N) for its msgbox-mailbox-connection = 9, so they're that mailbox's send-empty/recv-not-empty pair, not a decoder completion line like the AVD's own mailbox lines on T8142/T6050. The backend halts the MCU, and nothing is known to raise either line then. For now there's no interrupt to describe, and I've added the comment you asked for in avd-backend-kopsia.c and at the IRQ skip in avd-drv.c (ae3336208dbd). On the M6 I'll check whether VP/PP completion raises 268 with the MCU halted. If it does, I'll describe it and complete VP/PP from the handler, keeping the poll only for a node without an interrupt.

Still open, and both need the M6:

I haven't booted this head yet. When the M6 is back on its tether, I'll run the row you listed: the five codecs × 60 frames compared exactly, 4K60 H.264 to a null sink, and the genpd summary once power-off exists.

@sithglan

Copy link
Copy Markdown
Contributor Author

Small follow-up: I rewrapped one overlong line in the DART commit message (checkpatch), so the head is now 49198181bd26. The trees are identical to ee96cb6538a7. The commits named above are now 201b537b6f28 (DART), 49b35d7d904b (Kopsia backend), b8abb2e9e37f and 49198181bd26 (DT).

@sithglan

Copy link
Copy Markdown
Contributor Author

J873g decode row. The exact head can't boot the J873g from its SSD by itself: integration has no J873g NVMe, which comes with #1797. So the kernel is #2178's head 11efadab4eb2, which carries this PR's commits as of 49198181bd26 on the #1797 base, plus #2330/#2333 for their rows. Results:

  • ffmpeg framemd5, v4l2request + DRM PRIME + hwdownload, against software decode, 60 frames each: H.264, HEVC, HEVC Main10, VP9 and AV1 are all exact, and every hardware log shows Using V4L2 media driver avd.
  • 4K60 H.264 to a null sink (3840×2160, 59.94 fps, 3016 frames): 20.6 s, i.e. 146 fps on average (2.44× real time), with no AVD or DART errors in dmesg.

The genpd row (AVD off after 10 s idle, on for the next decode) waits for the power-off path, which is what I'm doing next. It comes with the generic PSD provider discussed on #1797.

@RyanTheTide

Copy link
Copy Markdown
Contributor

Thanks, Thomas. The SoC description, the dropped address check and the comment on the ADT mailbox pair cover those points. The decode row on #2178 is fine given the NVMe dependency.

Still open before this leaves draft: the PSD power-off, and Gen3 DART fault reporting (apple-dart.c, dart->irq = -1, "for now"). Doing both on the M6 first is reasonable.

On the 346 decoder tunables: the table is avd-fw's v5/tier2 one, made for an earlier decoder. Does the J873g ADT avd node carry its own tunables, as the DART's do? If so, please describe them in DT the way avd_dart does. If not, please say in the commit how you checked that the table fits T8152, for example against a trace of macOS register writes on the M6.

Row once power-off lands: pm_genpd_summary shows AVD off after 10 s idle and on for the next decode, with the five codecs still exact.

On T8152 the video decoder, the USB4 routers and their PHYs and the USB
controllers are powered through PMGR2 PSD services instead of PMGR
power-state registers. Each service has a 0x100-byte block with a target
state, an acknowledge bit and the current state.

Describe one service per node as a power domain, with an optional
apple,always-on for a service whose consumers cannot restore what
another agent set up in its blocks yet.

Signed-off-by: Thomas Glanzmann <thomas@glanzmann.de>
Provide each T8152 PMGR2 PSD service as a power domain. Power-on
requests state 0 and power-off the cold state 6. The driver waits for the
acknowledge before it reads the state, because reading the state
consumes the acknowledge. A service marked apple,always-on is powered at
probe and kept on. Otherwise the domain starts in the state the
hardware reports, so genpd powers off a service that nothing uses.

The consumers include IRQ-safe devices, the DARTs, so the domains are
IRQ-safe and a transition is polled atomically, as the PMGR power states
are. On the J873g the AVD service (50) reaches state 0 in 390-500 us and
the cold state in about 240 us; the timeout is 10 ms.

The AVD and the USB4 routers' CIO services, which so far each had their
own variant of this, and the USB controllers (services 175 and 176) use
it.

Signed-off-by: Thomas Glanzmann <thomas@glanzmann.de>
The AVD Gen3 DART uses the shared TLBI aperture like the display DARTs,
but Linux initializes it from an unlocked state with inactive streams.
Require its PSD power domain, DMA aperture and initialization tunables.

Signed-off-by: Thomas Glanzmann <thomas@glanzmann.de>
Admit cold Gen3 setup only for the instance its apple,t8152-avd-dart
compatible selects, with version 0x300, 16 streams, its PSD domain, ADT
tunables and inactive streams 0..2. Clear those streams, apply their
tunables and use the Gen3 TLBI aperture before enabling translation.

Map the TLBI aperture before reset and select a two-level 16K page table
inside a single 64 GiB high DMA window. Keep the existing locked display
DART admission and all other unlocked Gen3 instances unchanged.

On J873g this maps all three decoder streams for bit-exact H.264, HEVC,
HEVC Main10, VP9 and AV1 decoding. Gen3 fault IRQ handling and suspend
recovery remain outside this initial bring-up.

Signed-off-by: Thomas Glanzmann <thomas@glanzmann.de>
Describe the Kopsia MCU, decoder, wrapper and PIODMA register apertures,
its three DART streams and its shared PSD power domain. The initial
transport submits from the host with the MCU halted and polls completion;
it does not use the PostOffice mailbox or the legacy decoder reset line.

Signed-off-by: Thomas Glanzmann <thomas@glanzmann.de>
T8152 Kopsia retains the T8142 codec program format but uses different
initialization and completion registers. Add a host backend that halts
the MCU, initializes the wrapper and PIODMA, and reuses the existing codec
submission and completion logic with one frame in flight. Poll VP/PP
status only while a frame is active.

Enable H.264, HEVC, VP9 and AV1 on this variant without an external
firmware image. Retain the 128-byte H.264 MV and HEVC v5 header quirks.
Disable recovery reset: the existing watchdog parks DMA and retires the
decoder instead of issuing an unqualified reset.

The 346 register tunables are unchanged from the MIT-licensed avd-fw
v5/tier2 table. On J873g, forced DRM PRIME decode and download compares
exactly with software for 60 frames of each codec, including HEVC Main10;
mpv also renders through DRM PRIME and seeks in both directions.

The decoder is runtime power-gated through its PSD domain: the backend
is set up again on every resume. PostOffice completion, suspend/resume
and fault-injection testing remain outstanding.

Link: https://github.com/aurora-silicon/avd-fw/blob/ea3a65cc9c0c41907e6ad2714b813e4be6858485/src/hw/tunables/tunables_v5t2.h
Signed-off-by: Thomas Glanzmann <thomas@glanzmann.de>
Describe the T8152 Kopsia decoder in the SoC peripherals, disabled, so
every T8152 board can enable it: its five register banks, PSD service 50
(apple,t8152-psd) as the power domain shared by the decoder and its DART,
powered off by genpd while both are idle, and the DART as
the apple,t8152-avd-dart instance with its Gen3 TLBI aperture, its
initialization tunables and a 64 GiB window at 1 TiB inside the native
1..4 TiB DMA aperture. The tunables are those of the J873g's ADT.

Signed-off-by: Thomas Glanzmann <thomas@glanzmann.de>
Enable the T8152 decoder, its PSD domain and its DART on the J873g.

This enables the request decoder node used by FFmpeg and mpv on the M6
Mac mini. The PSD provider supplies both AVD and DART so translation is
initialized only after their shared domain is powered.

Signed-off-by: Thomas Glanzmann <thomas@glanzmann.de>
@sithglan

Copy link
Copy Markdown
Contributor Author

The power-off path is in. The head is now 892664b3b581:

  • The AVD's PSD domain is now the generic apple,t8152-psd provider from pmdomain: apple: T8152 PSD services as power domains #2459, which replaces this PR's own PSD binding and driver commits. ps_avd no longer has apple,always-on, so genpd powers the decoder and its DART off when both are idle. The provider is IRQ-safe, because the DART is an IRQ-safe device; without that, genpd keeps the domain on.
  • On resume the DART takes its usual lost-state path (the cold Gen3 init, then the saved TTBR/TCR), and the decoder runs the whole Kopsia setup again (avd_backend_boot()).

genpd row on the J873g (#2178's stack, which carries these commits):

  • At boot the domain powered off 2 s after probe: avd off-0, PSD state register 6.
  • The five codecs, 60 frames each against software, were all exact across 292 on/off transitions during the run. 4K60 H.264 to a null sink ran at 150 fps.
  • 11 s after the last decode the domain was off again.
  • Transitions took 390–500 µs for on and about 240 µs for off. No DART or AVD errors.

That leaves the Gen3 DART fault handler, which I'm working on now on the M6.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants