Skip to content

arm64: dts: apple: t8142: Neural Engine, video encoder and JPEG on every M5 - #2325

Draft
RyanTheTide wants to merge 1 commit into
integration/aurorafrom
dtfill/t8142-media-default
Draft

RyanTheTide wants to merge 1 commit into
integration/aurorafrom
dtfill/t8142-media-default

Conversation

@RyanTheTide

@RyanTheTide RyanTheTide commented Nov 19, 2026 •

Copy link
Copy Markdown
Contributor

Problem

The Neural Engine, the video encoder and the two JPEG engines are T8142 SoC blocks, but only some boards enabled them. The J815 had all three, the J813 only the encoder (described twice), and the J704 (MacBook Pro 14-inch, M5) none. The J704, J813 and J815 ADTs give the same registers, interrupts and power gates for ane, dart-ane, ave, dart-ave, jpeg0, jpeg1 and dart-jpeg.

Change

  • Enable the ANE (three DARTs, ANE_SYS), the AVE (two DARTs, VENC power tree) and JPEG (DART, JPG) in t8142-peripherals.dtsi.
  • Drop the per-board enables from t8142-j813.dts and t8142-j815.dts.

The ANE and AVE run firmware that iBoot preloads. The loader hands the preload over by compatible and disables the node when the ADT has none. Every DART keeps its engine's power domain. JPEG needs no firmware.

Testing

Built J704, J813 and J815 with cpp + dtc before and after and compared the decompiled trees node by node:

  • J815: the only change is status = "okay" becoming implicit.
  • J813: gains the ANE, its three DARTs, ANE_SYS, both JPEG engines, their DART and JPG.
  • J704: gains all of the above plus the encoder, its two DARTs and the VENC power tree.

dt-validate -m against the processed bindings reports nothing new on any of the three boards.

Still to run

Hardware rows. I have no M5 machine.

  • @term-guy (J704): ls /dev/accel* and one ANE inference, a 10 s H.264 encode through the AVE video node, and one JPEG encode/decode. Check dmesg for DART faults.
  • J813 (@sithglan or @gig3m): ANE probe, one JPEG decode, and the encoder still works.
  • J815 (@m0d16l14n1): regression only, expect no change.
Details
  • ADT comparison: adtdiff of J704 against J815 for ane, dart-ane, ave, dart-ave, jpeg0, jpeg1, dart-jpeg, pmgr and /cpus gave no differences in reg, interrupts, compatible or clock/power gates. J704 ane is ane,t8132exclave, IRQs 751/764; jpeg0/jpeg1/dart-jpeg IRQs 1156/1159/1157, matching the nodes in t8142-peripherals.dtsi.
  • The J815 enables came from @m0d16l14n1 (33d8f13, c3c1417).
  • Evidence (dtc decompiles, dt-validate output) is on the lab disk: traces/kernel/20261119-dtfill-m5m6/.

Assisted-by: LLM (Claude:Opus 5.5)

…ery M5

The Neural Engine (with its three DARTs and ANE_SYS power state), the
video encoder (two DARTs, VENC power tree) and both JPEG engines (DART,
JPG power state) are SoC blocks: the J704, J813 and J815 ADTs give the
same addresses, interrupts and power gates for ane, dart-ane, ave,
dart-ave, jpeg0, jpeg1 and dart-jpeg. Only the J815 enabled all three
and the J813 only the encoder (twice); the J704 had none of them.

Turn them on in t8142-peripherals.dtsi and drop the per-board enables.
The ANE and the encoder run firmware that iBoot preloads: the boot
loader hands the preload over by compatible and disables the node when
the ADT has none, so a board without a preload keeps them off. Every
DART keeps its engine's power domain. The JPEG engines need no firmware.

J815: no change in the built tree beyond status "okay" becoming implicit.
J813: gains the Neural Engine and both JPEG engines. J704: gains all
three.

The ANE and JPEG enables on J815 come from m0d16l14n1's
33d8f13 and c3c1417.

Signed-off-by: Ryan Murray <ryan@aurorasilicon.org>
@gig3m

gig3m commented Nov 19, 2026

Copy link
Copy Markdown
Contributor

J813 (MacBook Air 13" M5), macOS 27.2 stub (26B123). All three engines work. On 27.2 they also need two open kernel PRs and one aura change that's already on aura main. They're listed per engine below.

Tree: #2135's merged head 408e272 plus #2328 (so #2325), #2218 (which includes #1900) and #1899, with our usual J813 PRs on top. aura is sithglan's J813 branch 8b9c92ea plus aura#251 and aura#358.

DT: the J813 DTB differs from the one before only in the ANE, its three DARTs, ANE_SYS, jpeg0/jpeg1, the JPEG DART and JPG going to okay, as you list.

Neural Engine: binds and /dev/accel/accel0 appears. On 27.2 it needs both of these:

  • aura#251 (the T8142 ANE preload hand-off, merged). Without it: H17 ANE: no loader hand-off (raw-boot marker).
  • accel/ane: T8142: admit the macOS 27.2 (26B123) firmware build #1899 (the 27.2 firmware build). Without it: no loader hand-off (firmware TEXT pin) and layout differs. The preload is TEXT 0xbc000 and DATA 0x2b0000 at 0x100000bc000, which is accel/ane: T8142: admit the macOS 27.2 (26B123) firmware build #1899's 27.2 layout.
  • With both:
    H17 ANE: firmware 27.2 26B123, TEXT code sha256 9b57486790ff4145... (code pin)
    H17 ANE: loader hand-off verified; starting the engine
    H17 ANE: accel node ready; firmware starts on the first request
    
  • I didn't send it a program, so the firmware itself hasn't run yet.

JPEG: both engines bind (Apple JPEG encoder, HWREV 0x110010, video10 and video11). The driver only encodes (NV12 in, JPEG out), so I ran encodes instead of a decode: 30 frames at 1080p through each engine (v4l2jpegenc, v4l2video11jpegenc). The output is valid 1920x1080 JPEG.

Encoder: on 27.2 it needs #1900. Without it, the probe stops with -ENODEV: unsupported firmware (text CRC 24eb4625) and there is no video node. With #2218:

  • H.264 (h264_v4l2m2m, 1080p30, 10 s, 8 Mb/s): 300 frames, PSNR 43.7 dB against the source.
  • HEVC (hevc_v4l2m2m, 5 s): 150 frames, 41.3 dB.
  • After two deep sleeps, another H.264 encode (90 frames) and a JPEG encode both work.

No DART faults in dmesg.

🤖 Generated with Claude Code

@sithglan

Copy link
Copy Markdown
Contributor

J813 row (MacBook Air 13" M5, macOS 27.2 stub, enrolled aura 8b9c92ea).

The kernel is this head (1f4da1ec68ee) plus #1899 and #1900, the 27.2 firmware admissions for the ANE and the encoder, without which neither starts on 27.2. It also has CONFIG_DRM_ACCEL_ANE=m and CONFIG_VIDEO_APPLE_JPEG=m, which the J813 config didn't build.

  • ANE: the three ANE DARTs probe as cold ANE DARTs and ane_t6021 binds 420000000.ane, so the node and power state are right. The driver then leaves the engine off: H17 ANE: no loader hand-off (raw-boot marker), so there's no /dev/accel. gig3m's /dev/accel/accel0 on arm64: dts: apple: t8142: Give J813 and J815 the same SoC nodes #2383 was with aura#251 and aura#358, which this machine's enrolled aura doesn't have, so this part is loader-side, not this DT.
  • JPEG: both engines bind (apple-jpeg 301000000.jpeg and 301060000.jpeg: JPEG encoder, HWREV 0x110010). The driver is encode-only (NV12 in, JPEG out), so I can't run a JPEG decode. A 1280×720 frame through v4l2jpegenc gave a valid JPEG that decodes in software.
  • Encoder: apple-ave binds (firmware build 26B123). 10 s of 1080p30 (300 frames) through v4l2h264enc took 1.29 s, and the stream decodes in software without errors. On this firmware the driver admits only its qualified 30 fps / 3 B-frame profile: a 60 fps session is refused with -EOPNOTSUPP and the log says why.
  • No DART faults or SErrors at probe or during the runs.

@term-guy

Copy link
Copy Markdown

J704 (MacBook Pro 14" M5), macOS 27.0 stub (26A428). All three engines bind and run; no DART faults.

Setup: our J704 Fedora kernel (7.3.0, integration/aurora be8d4cd base + J704 GPU/DCP commits), booted through the installed aura (which has aura#251). The DTB is our J704 tree with this PR's J704 changes applied: the 18 nodes it enables, and our older tree's missing jpeg0/jpeg1 and ps_jpg added from this branch, checked property by property against this branch's J704 tree. apple-jpeg needed 74213b0 (the T8142 match), which our kernel predated.

Neural Engine: binds, /dev/accel/accel0 appears:
H17 ANE: loader hand-off verified; starting the engine
H17 ANE: accel node ready; firmware starts on the first request
No program sent (no userspace path here either), so the firmware itself hasn't run.

Encoder: /dev/video12. On 26A428 the driver only qualifies 30 fps with 3 B-frames, so ffmpeg's h264_v4l2m2m (which forces 0 B-frames) is refused with -EOPNOTSUPP ("26A428 layout: only 30 fps with 3 B-frames is qualified"). With gstreamer v4l2h264enc (driver default 3 B-frames), 1080p30, 10 s, 8 Mb/s: 300 frames, Main profile; decoded on the J704's own AVD (v4l2slh264dec), 45.4 dB average PSNR against the source (worst frame 42.5 dB).

JPEG: both engines bind (Apple JPEG encoder, HWREV 0x110010, video10/video11). Encode only, as on J813: 30 frames of 1080p NV12 through each engine, valid 1920x1080 baseline JPEG, identical output from both, interrupts rising on both.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants