Skip to content

soc: apple: suspend-to-RAM on the M4 Max (T6041, J616c) - #1664

Draft
AxxzyWasTaken wants to merge 39 commits into
aurora-silicon:integration/aurorafrom
AxxzyWasTaken:t6041-s2r
Draft

AxxzyWasTaken wants to merge 39 commits into
aurora-silicon:integration/aurorafrom
AxxzyWasTaken:t6041-s2r

Conversation

@AxxzyWasTaken

@AxxzyWasTaken AxxzyWasTaken commented Nov 12, 2026 •

Copy link
Copy Markdown
Contributor

Real suspend-to-RAM on the MacBook Pro 16" M4 Max (T6041, J616c) under Linux at EL2: entry, wake and a full resume of the running system, from the lid or systemctl suspend. It uses the sleep core from #1445; the T6041 side has its own platform driver (t6041-sleep.c), because the M4 Max commits through aura's S2R mailbox rather than the T8140 path.

Draft: the sleep works on my machine and the bring-up diagnostics are gone; see "Not working" for what is left.

Tested on

One machine: J616c, 26.6.2 firmware, booted at EL2 through aura.

This series, on #1445 as it was on 11 Nov (b8f5b1e7f750), 12 Nov:

  • 3 deep sleeps in a row in one boot (RTC alarm, 20 s each), each one back on its own: Wi-Fi (the SSH session I ran the test from stayed up through all three), NVMe, Bluetooth, no GPU errors. Kernel return path 16 ms.
  • Earlier versions of the same series, before the SPMI fix below: 4 more wakes (the RTC driver had to be rebound between them).
  • Rebased onto current integration/aurora (9219565939723cd5) for this PR; it rebased without conflicts, and two new AOP configs (T8152, T8142) needed the new ed0p field. With the J614C fixes and the cleanup below, 12 Nov evening: 3 deep sleeps in a row (RTC alarm, 20 s each), each one back on its own: Wi-Fi with the SSH session up, NVMe, Bluetooth, suspend_stats 3 ok and 0 failed. With the display gate fix below as well: the internal panel on appledrm from boot, then a lid close and open, deep sleep, and the picture back after the wake.

The same T6041 code on my older bring-up tree (7.3 daily kernel, 10 to 12 Nov):

  • 7 sleeps in a row, lid close and open, key wake, with a Hyprland session open.
  • Back after a wake: internal panel and backlight, GPU (firmware warm restart), NVMe, Wi-Fi, SD reader, Bluetooth (the driver rebinds itself), login and logout.
  • Timing: about 1.7 s from suspend entry to the sleep commit; panel lit about 0.3 to 0.5 s after the kernel gets control back.
  • Power: one unplugged deep sleep of 8 h 27 min (12 Nov, lid closed, then a lid wake) cost 7 mAh on the battery gauge (7124 to 7117 mAh of 7517, 95% before and after). That is close to what the gauge can resolve, so I'd read it as "well under 1% of the battery over a working day" rather than a watt figure.

Found while porting onto #1445

  • Parked CPUs: Apple SoC suspend-to-RAM on T8140 #1445's spin table only stops a parked CPU that answered spin_table_hold_parked() first. The T6041 commit called spin_table_stop_parked() without the hold, so the other cores stayed on and the watchdog reset the SoC 5 s after the commit. The T6041 enter now asks for the hold while it can still refuse.
  • SPMI replies at the wake: aura writes the PMU on the same SPMI queue at the wake, and its reply was still waiting when Linux came back. The RTC's acknowledge of the alarm took it as its own ("unexpected SPMI response 0x8b30"), and the RTC then refused every later suspend. spmi-apple-controller now drops replies left over at a firmware-driven resume, and the T6041 driver drains the FIFO around its own PMU writes. The SPMI change applies to any firmware-driven suspend, so T8140 too; it would be good to check it there.

Found on a J614C (joske's test, 12 Nov)

  • SEP DART sleep lists: with the in-tree device tree the suspend hung in the noirq phase. The SEP is told to sleep at device suspend, and its DART is saved first only if the DART's node has apple,sleep-restore; the J616C's bring-up tree has the lists, the in-tree one didn't. Without them the DART's noirq suspend read its registers after the SEP's domain went off. t6041.dtsi now has the lists (the same writes as T8140's), and the SEP refuses that sleep with a message instead of hanging when they're missing.
  • 40-core GPU: CONFIG_25G83 in hw/t6041.rs had no sleep_restart, so the GPU refused every sleep there; it has it now.
  • The J614C and J616C trees now have the apple,t6041-sleep node, and the driver doesn't bind without aura's S2R mailbox (before, it set its suspend ops and refused every sleep).
  • Booted on my J616C (the sleeps above), which already slept with the SEP DART lists; not yet run on a J614C.

Found on a J614C (joske, 17 Nov)

  • dcs_27 on the full part: the full T6041 reset in the sleep. joske's dump of all 32 AMCC and 32 DCS power words found one forced on, dcs_27 at main +0x530, past the other DCS words. soc: apple: t6041-sleep: dcs_27 to AUTO at the commit on the full part puts it on AUTO with the others. With that line, the rebased head (cd59f6bf5d on integration/aurora 59b02d5) wakes on the J614c from rtcwake -m mem -s 20 and from a lid close: panel, Wi-Fi, speakers and the SD reader are back. An earlier head with the same line slept 9 h 30 min unplugged for -18 mAh. The commit itself is build-tested only.
  • EDAC: apple_soc_edac event-log-handler: hardware error: event-log source 9 a few times around each wake on the J614c. Nothing visibly breaks.

Found on the J616C with integration/aurora

  • Display gate with the bring-up EL2 tree: that tree gives the external chains' DARTs, mailbox and DCP power-domains = <&dispextN_fe>, disabled like the PMP report entries above it; the in-tree trees describe no DISPEXT domain. The M4 Max gate enabled the chain nodes only, so they waited for domain providers that never came, failed at the deferred probe timeout (the mailboxes with -110), and the display subsystem never bound: the panel stayed on simpledrm. soc: apple: display-gate: M4 Max: chains from a tree that describes DISPEXT drops the property where the domain stays disabled; nothing changes with the in-tree trees.
  • Freeze at logout after a wake: logging out, or restarting sddm, after a deep sleep stopped the machine without a word in the log. Breadcrumbs in PMU scratch named the last access: a TLB flush on iommu@3a2300000, the HDMI chain's display DART. That chain's DCP fails its probe here, so it is not bound to the DCP driver and the sleep's stop never marked its DART dark, while the power word step powers the chain off. drm/apple: T6041 S2R: the external chains' DARTs dark by node marks both external chains' display and DCP DARTs dark by node before their power words go down. After it: 2 deep sleeps, each followed by an sddm restart, in one boot, with the machine and the panel fine. Then on my daily build of this series (latest integration/aurora + this PR + clpc/g17: busy-share memory floor from the DT + light effort on every G17 #1369), a lid close and open followed by a desktop logout: fine.
  • With the external DCPs bound, dcp_platform_suspend() warns twice (workqueue.c:4409, flush_work and cancel_delayed_work_sync on work that was never set up), apparently for the external DCP that is skipped as unusable. Harmless. That code isn't from this series, the gate fix only makes it reachable here; I left it alone.

Not working / not tested

  • External displays: I have no external monitor to test with. The current code shuts the external display controllers down at the first sleep and they stay down until a reboot (as J616s (M4 Pro) at EL2: hibernation and suspend-to-RAM #1477 does).
    @astaff, you are the other person doing M4 Max work: if you have a display and want to try or fix this part, that would be very welcome.

  • Touch ID after a wake, bring-up tree: works (a lid close and open, then a fingerprint unlock at the lock screen). A deep sleep is refused, cleanly, while the enclave is not confirmed asleep (soc: apple: sep: refuse a deep sleep at once over a stuck enclave).

  • Speakers after a wake: the amplifiers keep their configuration and report no fault; playback not checked by ear.

  • Camera: works after a wake on the J614c, including after a 9.5 h sleep (joske). Not checked on the J616c.

  • Panel dim after a wake (J614c): the brightness level is right, but the picture is at about half brightness until a DPMS off and on from the compositor; the same cycle inside the resume doesn't fix it (joske, 17 Nov). Cause not found yet.

  • Touch ID after a wake on this tree: works (a lid close and open, then an unlock at the lock screen). One earlier try failed 5 times and locked Touch ID; the integration driver keeps that lockout across restarts until SEP_BIO_REARM gets the password, which nothing on my install calls yet.

  • pm_test=platform and the display: the display stop assumes a real sleep follows. Under pm_test the firmware is stopped but never power-cycled, the restart expects a power cycle, and the internal panel stays dark (joske, J614c). Not supported yet.

  • Wi-Fi WoWLAN: with WoWLAN enabled, brcmfmac refuses deep with -95 (brcmf_pcie_pm_enter_D3) and the suspend is aborted; WoWLAN is s2idle only. With deep as the default, that is worth knowing (joske).

  • More than two external DCPs: the display stop used to handle only the first two external DCPs; drm/apple: T6041 S2R: every external DCP, not only the first two handles every enabled apple,t6041-dcpext. Build-tested only.

Notes for review

  • Boot requirements: EL2 through aura (it passes the S2R mailbox). The in-tree J614C and J616C trees now have the apple,t6041-sleep node and the SEP DART lists; I have only slept with the bring-up EL2 DTB on the J616C so far.
  • T6041's SEP gets SLEEP at device suspend, seconds before the commit, which is the order that has been tested on this machine. On T8140, Apple SoC suspend-to-RAM on T8140 #1445's commit-time SLEEP is unchanged; sep_pm_platform_commit() picks the path.
  • Overlap with J616s (M4 Pro) at EL2: hibernation and suspend-to-RAM #1477: the AIC save/restore, NVMe sleep flags, PMP SRAM backup, SMC WKTP/pmVC and conventional PCIe port re-setup commits do the same thing as J616s (M4 Pro) at EL2: hibernation and suspend-to-RAM #1477. Whichever lands first, the other can drop them.
  • Outside T6041-only code: spmi: apple: drop replies left by firmware at a firmware-driven resume (above), pinctrl: apple: ack a level interrupt again after unmasking it (all Apple SoCs; it stopped an interrupt storm from the speaker amps after a wake here), PCI: apple, nvme-apple: resume asynchronously, and the coprocessor stop helpers in sleep.c.
  • The bring-up diagnostics are gone: the last 3 commits drop them from t6041-sleep.c (the dry, diag and serror_go parameters, PMU scratch marks and notes, register snapshots), the DCP driver (the s2r_* parameters, the step-by-step test hook, S2RSTOP/S2RSTART prints, the panic dumper) and apple_dart_display_keep() (its register dump). The commit and wake do the same hardware writes, delays and polls as before; the defaults the parameters had are now fixed. They are separate commits so the change is easy to review, and can be folded into the commits they clean up. The 4 BRING-UP ONLY commits are dropped.

@joske

joske commented Nov 12, 2026

Copy link
Copy Markdown
Contributor

Tried this on a J614c (MacBook Pro 14" M4 Max, 40-core GPU), os-fw 26.6.2, EL2 through aura main d5e180ba. The kernel was integration/aurora 9219565 with this PR merged on top, plus an apple,t6041-sleep node in t6041-j614c.dts (the J614c's ADT has the same pmgr and apcie0 as the J616c's, so the driver's tables should fit). It probes: "T6041 deep sleep at EL2 through aura's S2R mailbox", and mem_sleep offers deep.

GPU: the 40-core part runs CONFIG_25G83 in hw/t6041.rs, which has no sleep_restart, so the first try was refused ("no firmware restart on this chip, refusing sleep"). Setting sleep_restart: true there, as t6041_25g83::CONFIG has, got past that.

Then it hangs:

  • dry=1 + rtcwake -m mem -s 20: never comes back; only a forced power-off helps.
  • pm_test=devices: comes back fine.
  • pm_test=platform: hangs. So it's in a late or noirq callback, before .enter.

PMU notes (diag=1, read after the forced reset from /sys/bus/nvmem/devices/spmi_nvmem0/nvmem at 0xa200; the 0x8308 mark is 0x10 both times):

  • With everything bound: last device 0000:01:00.0 (brcmfmac), callback count 0x195.
  • With 0000:01:00.0 and 0000:01:00.1 removed from the bus: last device 404c80000.iommu (the SEP's DART), count 0x18b.

The count drops by 10, the two removed devices' callbacks, so the hang comes at the same point in the sequence. The noted device looks like whatever is next, not the cause. My guess: something is already stuck by then, possibly around the SEP going to sleep at device suspend.

One more J614c data point, from #1477's hibernation dry runs on this machine: at every pm_test level from platform up, apple-pmgr-pwrstate ...power-controller@c00: PS sep: Failed to reach power state 0xf (requested 0xf, now: 0x1000030f) on the way back. I can run more tests or provide a different log if it helps.

@AxxzyWasTaken

Copy link
Copy Markdown
Contributor Author

Thanks for trying it on the J614c, this helps a lot.

I think the hang is the SEP's DART. On my J616c the bring-up device tree gives iommu@404c80000 three sleep lists (apple,sleep-prepare, apple,sleep-restore, apple,sleep-restore-late), and the in-tree t604x.dtsi has none. On T6041 the SEP is told to sleep at device suspend, and the SEP driver saves its DART with those lists first, after which the DART's own callbacks leave it alone. Without them nothing is saved, the enclave sleeps, its power domain drops (your 0x1000030f), and then the DART's noirq suspend reads its registers and the bus hangs. That fits both your runs: the hang is at the same point in the order, and the noted device is just whatever started last.

I've pushed four commits for it:

  • the lists on &sep_dart in t6041.dtsi, and the apple,t6041-sleep node in t6041-j614c-j616c.dtsi, so the in-tree tree should be enough now
  • the SEP refuses the sleep with a message, instead of hanging, when its DART has no lists
  • sleep_restart: true for CONFIG_25G83, as you did
  • the driver doesn't bind without aura's mailbox

To try it on your current build, this in your DTS should be enough:

&sep_dart {
	apple,sleep-prepare = <0x80 0xa102>, <0x704 0xffffffff>;
	apple,sleep-restore = <0x210 0x0>, <0x20c 0x800005f5>,
			      <0x220 0xf0f0f>, <0x224 0x80808>,
			      <0x300 0x1>, <0x318 0x1>;
	apple,sleep-restore-late = <0x80 0x8000>, <0x228 0x3f3f3f3f>,
				   <0x22c 0x1010101>;
};

dry=1 only stops at .enter, so it can't catch anything earlier. If it gets further and then resets or hangs, the 0x8308 mark after the forced reset says how far it got (6: it reached the commit, 0x67: Linux got all the way back), and the notes at 0xa200 work as before. The new commits are only build-tested; my J616c already slept with the lists, so nothing changes on it.

@joske

joske commented Nov 12, 2026

Copy link
Copy Markdown
Contributor

Thanks, the SEP DART lists were it: the hang at pm_test=platform is gone on the J614c. A real sleep now gets to the commit, but the Mac resets instead of waking.

Same J614c (MacBook Pro 14" M4 Max, 40-core GPU), os-fw 26.6.2, aura 1.0.0b12 + aura#166 + a local dcpext3 entry. Kernels: integration/aurora fa59399 + #1667/#1684 and a few J614c DT patches, with this PR merged at two points:

  • 35.7: PR head 128743e, no local sleep patches. The in-tree apple,t6041-sleep node probes ("T6041 deep sleep at EL2 through aura's S2R mailbox") and mem_sleep defaults to deep.
  • 35.6: PR at fe530fa, to still have diag and the PMU marks.

pm_test=platform (35.7): comes back. The PMP refuses ("firmware never stopped"), and so do the SIO and the GPU. The SEP logs "the enclave stays asleep until the next real sleep or a reboot". Two problems after it:

  • The internal panel stays black. The system is fine over SSH and GNOME is running, but apple-panel-bl reads brightness 54 and actual_brightness 0. Writing a new brightness, or display power-save off and on through Mutter, gets no reaction from the driver (nothing in the log). On the way down, dcp@402e00000 logged Endpoint 0x37 is not running, cannot send message, iomfb_poweroff_v26_6_desktop: clear swap timed out and setPowerState(0) timeout 1000 ms.
  • The next reboot hangs after SSH is gone; only a forced power-off helped.

Real sleep, rtcwake -m mem -s 20: the Mac resets (35.7, and 35.6 with diag=1). The journal ends at "PM: suspend entry (deep)".

PMU scratch after the 35.6 reset (diag=1; the 0x8308 mark was 0x10 before the run):

  • 0x8308 = 0x06: it reached the commit, but not 0x60, so Linux never ran again.
  • Bit 11 counts all 0: 0x8314, 0x831c-0x831f and 0x8311; 0x8316-0x8318 and 0x8384+ are 0xff.
  • 0x830a = 0x10, 0x830b = 0x0f, 0x830c = 0x4d.
  • 0x830d = 0x80: no USB-C port domains left on.
  • 0x830f = 0x18, 0x8312 = 0x98, 0x8315 = 0x80.
  • 0x8310 = 0xc0: all 64 keep-alive words.
  • 0x8319 = 0x80, 0x831a/b = 0x4f/0xff.
  • 0x8342/3 = 0x06/0x03, 0x8344/5 = 0/0.
  • SEP: 0x8346/7 = 0x07/0x07, 0x8359/5a = 0x3f/0x00, 0x835b = 2 ms.
  • 0x8348 = 0x80, 0x8349/4a = 0xff/0x81, 0x834b/c = 0x4f/0x01.
  • 0x8350/1 = 3 non-PS and 52 PS rows differing from macOS. The PS bytes from 0x83a2: ff, 13 × 4f, 15 × 00, 14 × 4f, 0f ff ff ff 00 00 ff 00 00.
  • 0xa200 note: 500700000.pmp, count bytes ba 02.

So the commit looks clean from here: nothing busy, the SEP asleep, the USB-C domains down. The failure is after it: the RTC wake doesn't come, or the wake path resets before it reaches Linux.

One more thing: with Wi-Fi WoWLAN enabled (iw phy phy0 wowlan enable magic-packet, or whatever re-enabled it at boot here), brcmfmac refuses deep with -95 (brcmf_pcie_pm_enter_D3), so the suspend is aborted. That's by design, WoWLAN being s2idle only, but it may be worth a hint when deep is the default.

I'm happy to run more: a different wake source, a debug build, or other PMU offsets.

@AxxzyWasTaken

Copy link
Copy Markdown
Contributor Author

Thanks, that's a big step: the commit really is clean now. Your 35.6 record matches the commit record of my J616c when it wakes fine almost byte for byte: 3 non-PS and 52 PS rows differing from macOS, the same 0x8342-0x834c, the same PS bytes, nothing busy, the SEP asleep, the USB-C domains down. Only the first differing PS row differs: ff on yours, 0f on mine.

My best guess for the reset: your kernel has #1667, #1684 and a local dcpext3 entry, so it runs more external DCPs than mine. The display stop before the sleep kept one "already stopped" slot per external DCP by name: dcp@3a2e00000 got one, and every other external DCP shared the second. So with three or four external DCPs, the second one stopped took that slot and the rest were skipped, their firmware still running at the commit. Only the first two chains' DARTs were made dark, too. I've pushed drm/apple: T6041 S2R: every external DCP, not only the first two, which keeps the mark in each DCP and makes the DARTs of every enabled apple,t6041-dcpext node dark.

Two things that would tell us more:

The pm_test=platform black panel is a real gap: the display stop assumes a real sleep follows. Under pm_test the firmware is stopped but never power-cycled, and the restart path expects a power cycle. So pm_test=platform with the display isn't supported yet. I'll put that in the PR text, along with your WoWLAN note (brcmfmac refuses deep with -95 while WoWLAN is enabled).

@joske

joske commented Nov 12, 2026

Copy link
Copy Markdown
Contributor

Thanks! The raw PMU scratch 0x8300-0x83ff from that 35.6 run (diag=1) is below; it was already read before the next boot. Decoded with fe530fa's t6041_macos_regs, the bitmap at 0x8352 puts the ff on row 9, fab3_soc (0x502280118): macOS has 0x4f there at the commit, this J614c 0xff, your J616c 0x0f. The non-PS diff that's still readable is dcs_perf (0x8666, macOS 0x7666). The rest of that area is overwritten by the 0x8384+ busy rows.

Correction to my last comment: 0x8359/5a fall inside the bitmap, which is written after them, so the 3f 00 there is bitmap and not the SEP state. 0x8346/7 (0x07/0x07) still say the SEP is asleep.

8300: 9c 41 a5 ce 48 84 92 14 06 60 10 0f 4d 80 18 18
8310: c0 00 98 0c 00 80 ff ff ff 80 4f ff 00 00 00 00
8320: d8 29 7e 42 88 12 43 3c 78 a5 98 88 ca 07 15 8d
8330: 18 22 30 66 72 47 51 69 c3 28 27 e4 5a 32 1f 0d
8340: 5f 5f 06 03 00 00 07 07 80 ff 81 4f 01 0f 00 00
8350: 03 34 10 02 00 00 00 00 fc 3f 00 02 00 f0 de 57
8360: 00 80 54 d5 d5 00 00 01 42 08 00 00 40 01 00 00
8370: 40 08 24 02 00 00 00 00 00 00 00 00 00 00 04 00
8380: 66 86 00 00 ff ff ff ff ff ff ff ff ff ff ff ff
8390: 69 63 5f 72 75 6e 74 69 6d 65 5f 73 75 73 70 65
83a0: 6e 64 ff 4f 4f 4f 4f 4f 4f 4f 4f 4f 4f 4f 4f 4f
83b0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 4f
83c0: 4f 4f 4f 4f 4f 4f 4f 4f 4f 4f 4f 4f 4f 0f ff ff
83d0: ff 00 00 ff 00 00 72 71 7c 64 70 6d 5f 72 75 6e
83e0: 5f 63 61 6c 6c 62 61 63 6b 7c 64 65 76 69 63 7c
83f0: 1b 00 00 00 1b 00 00 00 1b 00 00 00 1b 00 00 00

fac697f: still resets. Built as fac697f with fe530fa's t6041-sleep.c, sleep.c and sleep.h put back for the marks, all four external DCPs enabled (dcpext0-3), WoWLAN off, diag=1, rtcwake -m mem -s 20. The record is the same as before: mark 0x06, nothing busy, the same bytes at 0x8308-0x834c, fab3_soc still 0xff. The one difference is a 53rd PS row, nub_spmi4 (macOS 0x4f, here 0xff):

8350: 03 35 10 02 00 00 00 00 fc 3f 00 02 00 f0 de 57
8370: 40 08 26 02 00 00 00 00 00 00 00 00 00 00 04 00
83d0: ff 00 00 ff ff 00 00 71 7c 64 70 6d 5f 72 75 6e

(the other lines are as above.)

Next I'll try your other suggestion, only dcpext0/1 enabled, by booting an aura without the dcpext2/3 fill so the gate leaves those chains off.

@joske

joske commented Nov 13, 2026

Copy link
Copy Markdown
Contributor

Only dcpext0/1: still resets. Same 35.8 kernel (fac697f + fe530fa's sleep files for the marks), with an aura from main just before #166, so t6041_exts has only dcpext0/1. The gate enabled /dcp@3a2e00000 and /dcp@602e00000 and left /dcp@606e00000 and /dcp@306e00000 off ("no firmware memory"). No external displays, WoWLAN off, diag=1, rtcwake -m mem -s 20.

The record is the same again: mark 0x06, nothing busy, the same 0x8308-0x834c, and fab3_soc 0xff. The 53rd PS row is now dcs_10 (macOS 0x4f, here 0xff) instead of nub_spmi4, so that row seems to vary between runs.

8360: 00 80 54 f5 d5 00 00 01 42 08 00 00 40 01 00 00
8370: 40 08 24 02 00 00 00 00 00 00 00 00 00 00 04 00
83c0: 4f 4f 4f 4f 4f 4f ff 4f 4f 4f 4f 4f 4f 4f 0f ff
83d0: ff ff 00 00 ff 00 00 71 7c 64 70 6d 5f 72 75 6e

So with the same external DCPs as your J616c it still resets, and across the three runs fab3_soc at 0xff is the only difference from your record that stays. Is there anything on fab3 that the J614c has and the J616c doesn't, or that I could take out to test?

@joske

joske commented Nov 13, 2026

Copy link
Copy Markdown
Contributor

Your exact branch: still resets. Built fac697f as it is (base integration/aurora 9219565), with only fe530fa's t6041-sleep.c, sleep.c and sleep.h on top for the marks, and aura main 5ebc72f (1.0.0b13-7). No J614c patches of mine: only dcpext0/1 enabled, no external displays, and no WoWLAN on this base. diag=1, rtcwake -m mem -s 20.

The record is byte for byte the one from my first 35.6 run: mark 0x06, nothing busy, 3 non-PS and 52 PS rows differing from macOS with the same bitmap and bytes, fab3_soc 0xff.

So on the J614c it resets in every configuration I tried (4 or 2 external DCPs, my tree or yours) with the same clean commit record, and fab3_soc is the one row that differs from your J616c every time. Happy to try a patch that does something to fab3 before the commit, or to read anything else after the reset.

@AxxzyWasTaken

Copy link
Copy Markdown
Contributor Author

Thanks for running all of those. Same record in every configuration, with fab3_soc on, rules out the external DCPs.

One difference between our setups I can name: your kernel boots the in-tree device tree, mine boots my bring-up one, and they describe fab3_soc's children differently (in-tree: sio, ans, sep; mine: sio, ans, apcie_sys_st1, apcie_sys_gp). That only changes Linux's genpd bookkeeping, not what the hardware has on at the commit. Something under fab3 still reads on there, and the macOS table that 0x8350.. is checked against doesn't have it.

So, to see which domain it is, a small patch on top of fe530fa's t6041-sleep.c (the build you have for the marks). Right after MARK 6 it writes a bitmap of every PS word that reads on (ACTUAL != 0) to PMU SRAM: 0xa300 main PMGR (0x502800000, +0x000..+0x4f8, 20 bytes), 0xa314 East (0x502280000, +0x000..+0x3f8, 16 bytes), 0xa324 AON (0x508300000, +0x000..+0x178, 6 bytes), and 0x50 at 0xa32a once it's done. Bit n of byte n/8 is the word at +8n.

diff --git a/drivers/soc/apple/t6041-sleep.c b/drivers/soc/apple/t6041-sleep.c
index 15130a0..bd61e84 100644
--- a/drivers/soc/apple/t6041-sleep.c
+++ b/drivers/soc/apple/t6041-sleep.c
@@ -404,6 +404,32 @@ static int t6041_ps_match_all(void)
  * 0x837e..0x83a1 the first 6 non-PS diffs as u16 index + u32 value;
  * 0x83a2..0x83ff each differing PS row's [7:0] (actual, target), in row order, 94 of them.
  */
+/*
+ * DIAG: every PS word that reads on (ACTUAL != 0) at the commit, as bitmaps in PMU SRAM, bit n
+ * of byte n / 8 = the word at +8 n: 0xa300 the main PMGR (0x502800000, +0x000..+0x4f8, 20
+ * bytes), 0xa314 East (0x502280000, +0x000..+0x3f8, 16 bytes), 0xa324 AON (0x508300000,
+ * +0x000..+0x178, 6 bytes); 0xa32a = 0x50 when done.
+ */
+static void t6041_ps_on_map(struct t6041_sleep *s)
+{
+	static const struct { u16 at, words; } blk[3] = { { 0xa300, 160 }, { 0xa314, 128 }, { 0xa324, 48 } };
+	void __iomem *base[3] = { s->ps, s->east, s->ps_aon };
+	unsigned int b, i;
+
+	for (b = 0; b < 3; b++) {
+		for (i = 0; i < blk[b].words; i += 8) {
+			u8 byte = 0;
+			unsigned int j;
+
+			for (j = 0; j < 8; j++)
+				if (FIELD_GET(PS_ACTUAL, readl(base[b] + 8 * (i + j))))
+					byte |= BIT(j);
+			t6041_pmu_write(s, blk[b].at + i / 8, byte);
+		}
+	}
+	t6041_pmu_write(s, 0xa32a, 0x50);
+}
+
 static void t6041_reg_diff(struct t6041_sleep *s)
 {
 	unsigned int i, n[2] = { 0, 0 };
@@ -1210,6 +1236,7 @@ static int t6041_sleep_commit(unsigned long unused)
 		    busy_n < ARRAY_SIZE(busy_rows))
 			busy_rows[busy_n++] = i;
 	MARK(s, 6);
+	t6041_ps_on_map(s);
 	t6041_commit_t = read_sysreg(cntpct_el0);
 	for (i = 0; i < ARRAY_SIZE(t6041_macos_regs); i++)
 		t6041_reg_now[i] = readl(t6041_reg_map[i]);

After the reset: dd if=/sys/bus/nvmem/devices/spmi_nvmem*/nvmem bs=1 skip=$((0xa300)) count=48 2>/dev/null | xxd (the spmi_nvmem whose parent is 0-0b). I'll take the same map on my J616c, and the words on yours that aren't on mine are the candidates.

@AxxzyWasTaken

Copy link
Copy Markdown
Contributor Author

For comparison, the same map from my J616c: one RTC deep sleep (20 s) at the greeter on a kernel with fe530fa's sleep files plus the patch above. It woke normally (0x8308 = 0x67).

a300: 10 00 02 00 7f 80 fb ff ff 01 02 00 00 00 84 10 0e 07 00 00   main  0x502800000 +0x000..+0x4f8
a314: 00 00 00 00 50 00 78 c0 3f f0 ff f0 ff 00 00 00               East  0x502280000 +0x000..+0x3f8
a324: 01 f8 06 e1 05 00                                             AON   0x508300000 +0x000..+0x178
a32a: 50

Bit n of byte n/8 is the word at +8n. fab3_soc is East +0x118 (byte 4, bit 3): off here, as the 0x0f in the PS rows says. Any bit that is set in your map and clear in mine is a domain still on at your commit, and the likely holder of fab3. This run's 0x8350/1 read 03 35 (53 PS rows), so the 53rd row varies on mine too.

@joske

joske commented Nov 13, 2026

Copy link
Copy Markdown
Contributor

The map from the J614c, same build as my exact-branch run (fac697f + fe530fa's sleep files) plus your patch, rtcwake -m mem -s 20, reset as before. 0xa32a = 0x50, so it's complete:

a300: 10 00 02 00 7f 80 fb ff ff 7f 03 00 00 00 84 10 0e 07 00 00   main
a314: 00 00 00 00 58 00 78 c0 3f f0 ff ff ff 0f 00 00               East
a324: 01 f8 06 e1 05 00                                             AON
a32a: 50

Nothing is on in yours that's off in mine. On here and off on your J616c, named from the J614c ADT's pmgr devices:

Block Offsets Domains
main +0x248..+0x280 AMCC24, DCS_24, AMCC25, DCS_25, AMCC26, DCS_26, AMCC27
East +0x118 FAB3_SOC
East +0x2c0..+0x2d8 AMCC28..AMCC31
East +0x340..+0x358 DCS_28..DCS_31

So at the commit this machine has all 32 memory controllers on (AMCC0-31), and yours has 24 (AMCC0-23). This one is the full M4 Max (16 CPU cores, 64 GB, 512-bit). Going by the 24, I guess yours is the binned one (14 cores, 36 GB, 384-bit)? AMCC28-31 and DCS_28-31 sit in the East block with FAB3_SOC, so fab3 looks like the fabric for that last quarter of DRAM. If that's right, it isn't a leaked domain but the extra channels, and whatever the sleep path does per memory channel (or the macOS table it's checked against) only covers 24 of them.

If you have a macOS table or sequence from a 512-bit M4 Max, or want me to try something per channel for AMCC24-31, I can run it. One aside: this boot came from the J614c's aura build with the dcpext3 entry rather than stock main (update-m1n1 didn't switch it), but the gate still only enabled dcpext0/1, as in the exact-branch run, and the commit record is the same as before (mark 0x06, nothing busy, 3 non-PS and 52 PS diffs, fab3_soc 0xff).

🤖 Generated with Claude Code

@AxxzyWasTaken

Copy link
Copy Markdown
Contributor Author

That's it, thanks. Yes, mine is the binned one: 14 cores, 36 GB, 384-bit. On it, the words you list read target and actual 0 live (amcc24 +0x248 = 0x0f000300, amcc28 East +0x2c0 = 0x0f000300), so those channels are fused off.

The sleep path is written for 24 channels throughout, not only the macOS table:

  • the AMCC error-enable quiesce runs on 24 planes;
  • the cache flush and allocation-off (t6041_mcc_macos) and the way restore after the wake run on 6 AMCCs;
  • DCS dynamic power gating runs on 24 DCS blocks;
  • the table matched against macOS comes from my J616c, so it has no words for AMCC24-31 or DCS_24-31.

So on yours, the last eight channels are never flushed, and their PS words stay forced on. That holds fab3_soc at ff.

I've pushed soc: apple: t6041-sleep: the full part's 32 memory channels:

  • At probe, the driver counts 32 channels when amcc24's and amcc28's words aren't both zero; it logs t6041-sleep ...: 32 memory channels.
  • It quiesces, flushes and power-gates all eight AMCCs and 32 DCS blocks, and puts the 15 extra words on AUTO as macOS has their twins (an AMCC slice 0x1000030f, a DCS channel 0x1400034f). They're saved first and put back on wake, before the cache restore.

One check before you run it: the AMCC and DCS addresses for groups 6 and 7 are extrapolated from the first six: AMCC block 0x22c000000 and 0x22e000000, DCS 0x22c200000 + 0x200000 * c and 0x22e200000 + 0x200000 * c. Could you confirm them against your ADT's mcc reg list (reg 12+ the AMCC blocks, reg 20+ the DCS channels)? If they differ, it would fault on the first sleep.

If it still resets, the same map with the new commit would show whether fab3_soc came down.

@joske

joske commented Nov 13, 2026

Copy link
Copy Markdown
Contributor

f3fcc17: still resets. The addresses match the J614c ADT first: arm-io ranges maps 0 to 0x200000000, and mcc has amcc-count = 8, dcs-count-per-amcc = 4. Reg 18/19 are 0x2c000000/0x2e000000 (0x200000 each), so 0x22c000000/0x22e000000. Reg 44-47 are 0x2c200000-0x2c800000 and reg 48-51 are 0x2e200000-0x2e800000 (0x8000 each), so 0x22c/0x22e200000 + 0x200000 * c. The error words (+0x25000 + 0x40000 * p) and the all-planes window (+0x1c0000) fall inside those blocks.

  • f3fcc17 as it is (on fac697f, base 9219565, dcpext0/1 only): the probe logs t6041-sleep t6041-sleep: 32 memory channels, then rtcwake -m mem -s 20 resets as before.
  • The same + your map, with the call placed just before t6041_spin_s(20) (after t6041_atc_commons_on()), so the SPMI-quiet gap stays as it is. 0xa300..0xa32f was zeroed before the run. After the reset it reads:
a300: 10 00 02 00 7f 80 fb ff ff 7f 03 00 00 00 84 10 0e 07 00 00   main
a314: 00 00 00 00 58 00 78 c0 3f f0 ff ff ff 0f 00 00               East
a324: 01 f8 06 e1 05 00                                             AON
a32a: 50

So it's fresh, and it's bit for bit the map from before the change: AMCC24-27, DCS_24-26 (main +0x248..+0x280), AMCC28-31 and DCS_28-31 (East +0x2c0..+0x2d8, +0x340..+0x358), and FAB3_SOC (East +0x118) still read on at that point. Note that on your J616c map AMCC0-23 read on at the commit too, so on AMCC/DCS words seem expected there. What's still different is fab3_soc, which stays on here alongside the extra channels even with them on AUTO.

If it helps, I can dump the raw words (target, AUTO, flags, actual) of those 15 plus fab3_soc at the same point, or try whatever you'd like done to fab3 or the extra channels before the commit.

🤖 Generated with Claude Code

@AxxzyWasTaken

Copy link
Copy Markdown
Contributor Author

Thanks for checking the addresses and for the run. Agreed: AMCC/DCS reading on at that point is expected, because their AUTO state only drops them once the commit has gone quiet. fab3_soc is the one real difference.

Yes, the raw words would help most. Could you dump these 16 at two points: where the map runs now, and right after t6041_ps_match_all() (the last PS writes before the commit)?

  • amcc24-27 (main +0x248, +0x258, +0x268, +0x280)
  • dcs_24-26 (main +0x250, +0x260, +0x270)
  • amcc28-31 (East +0x2c0..+0x2d8)
  • dcs_28-31 (East +0x340..+0x358)
  • fab3_soc (East +0x118)

Writing them as 32-bit words from 0xa340 (16 x 4 bytes, then the second set from 0xa380) keeps them clear of the map.

Those words would tell us three things:

  • whether the AUTO bits and auto states took (bits 28 and 27:24);
  • whether a target was left at f;
  • whether fab3_soc's own word still has a target of f, or AUTO 4 with a child keeping it up.

If fab3_soc is at AUTO 4 with the extra channels on AUTO, then the next suspect is the fab3 child the J614c has and mine doesn't. In your bring-up DTB, fab3's children are sio, ans, apcie_sys_st1 and apcie_sys_gp.

@joske

joske commented Nov 14, 2026

Copy link
Copy Markdown
Contributor

The raw words, f3fcc17 + your map + a dump of the 16 words at both points (u32 LE, in your order: amcc24-27, dcs_24-26, amcc28-31, dcs_28-31, fab3_soc). 0xa300..0xa3bf was zeroed first; after the reset the done marks 0xa32a/b/c read 50 51 52, so all three sets are from this run. It reset as before.

a300: 10 00 02 00 7f 80 fb ff ff 7f 03 00 00 00 84 10 0e 07 00 00   map (unchanged)
a314: 00 00 00 00 58 00 78 c0 3f f0 ff ff ff 0f 00 00
a324: 01 f8 06 e1 05 00 50 51 52
Word right after t6041_ps_match_all() (0xa380) where the map runs (0xa340)
amcc24 0x1f0002ff 0x1400024f
amcc25-27, dcs_24-26, amcc28-31, dcs_28-31 0x1400024f (all 14) 0x1400024f (all 14)
fab3_soc 0x1f0000ff 0x1f0000ff
  • AUTO took on all 15 (bit 28). amcc24 hadn't settled yet right after the match (auto state f, actual f), and by the map point it's like the others.
  • The auto states: all 15 end at auto state 4, actual 4, target f. That's macOS's DCS value (0x1400034f, apart from bit 8), but for the eight AMCC words macOS has auto state 0 and actual 0 (0x1000030f). The AMCC ones read 4 here, so either the 0 didn't take or the field reads back the state it's at.
  • fab3_soc is on its own word: 0x1f0000ff at both points, AUTO 1, auto state f, actual f, target f. So not AUTO 4 with a child holding it up.

Raw dump of 0xa340..0xa3bf for reference:

a340: 4f 02 00 14 4f 02 00 14 4f 02 00 14 4f 02 00 14   (x3 lines)
a370: 4f 02 00 14 4f 02 00 14 4f 02 00 14 ff 00 00 1f
a380: ff 02 00 1f 4f 02 00 14 4f 02 00 14 4f 02 00 14
a390: 4f 02 00 14 4f 02 00 14 4f 02 00 14 4f 02 00 14   (x2 lines)
a3b0: 4f 02 00 14 4f 02 00 14 4f 02 00 14 ff 00 00 1f

🤖 Generated with Claude Code

@joske

joske commented Nov 14, 2026

Copy link
Copy Markdown
Contributor

One more thought on those words, and a question.

t6041_ps_match_all() already writes fab3_soc with your table's 0x1400034f (AUTO, auto state 4), and the word reads 0x1f0000ff right after. The AMCC words were written with auto state 0 and read 4, and amcc24 went from f to 4 between the two dumps. So [27:24] looks like the level the hardware grants rather than the one we write: at least what we ask for, more if something downstream needs it. On this J614c something needs fab3 at f.

The children both machines have (sio, ans, sep) read the same in both maps. The only difference is the eight extra channels, and AMCC28-31/DCS_28-31 sit in the East block with fab3. On your binned part they're fused off and fab3 goes to 0. So I suspect fab3 is the fabric for that last quarter of DRAM, and with those channels at auto state 4 (which macOS allows for the DCS words too), fab3 has to stay up on a full part. If so, fab3 at f is expected here, and the reset comes from something a binned-J616c reference can't show.

So, how did you capture the macOS commit state on your J616c? If it's something I can run on this J614c under macOS (26.6.2), a full-part table would show what macOS really does with fab3_soc and AMCC24-31 at the commit, and what else differs. I'd rather get that than guess at memory fabric power before the commit.

If you'd like the theory confirmed first, I can do one more run that rewrites fab3_soc to auto state 4 and the extra AMCCs to 0 at the map point, waits, and dumps the words again.

🤖 Generated with Claude Code

@joske

joske commented Nov 15, 2026

Copy link
Copy Markdown
Contributor

Same result on the new head (7288144, with the integration/aurora merge) plus the same two diag commits: 32 memory channels, rtcwake -m mem -s 20, reset. The dump is fresh (0xa300..0xa3bf zeroed first, marks 50 51 52) and the map is byte for byte the same. The words too: the 14 channel words at 0x1400024f and fab3_soc at 0x1f0000ff at both points. The only change is that amcc24 had already settled by the t6041_ps_match_all() point this time (0x1400024f instead of 0x1f0002ff), which looks like timing.

🤖 Generated with Claude Code

@joske

joske commented Nov 17, 2026

Copy link
Copy Markdown
Contributor

I tried one more idea on the J614c, from the M5 Max sleep in #2119. There the commit dropped all the CPU clusters together, the P cluster's power-down queued behind the boot cluster's and never finished, and the engine's watchdog reset the chip 5 s after the commit. The fix was to take the secondary clusters down first and poll until they're done. Our symptom is the same, so I checked whether it applies here.

Diag on 7288144: the cluster block (0x502820000, +0x00..+0x38) dumped to PMU SRAM at three points in t6041_sleep_commit(). The boot CPU is 0x10100, so P0 is the boot cluster.

E +0x0 P0 +0x4 (boot) P1 +0x8 AVE +0xc/+0x10 +0x14..+0x38
before writel(0, cluster + 0x0) 0xf 0xf 0xf 0 0
right after writel(0, cluster + 0x8) 0x10 0xf 0x10 0 0
after t6041_atc_commons_on(), before the quiet gap 0 0x10 0 0 0

E and P1 change at once and read 0 by the last point, the same as the AVE complexes. Only the boot cluster is still at 0x10. That is the order #2119 needed, so I don't think this is our reset. T6041 has no T6050-style sequencer status words at +0x18/+0x1c (everything past the AVE words reads 0), so this is based on the cluster words only. Same reset as before.

So I'm back to the AMCC words. Your comment in t6041_mcc_macos() says macOS has every slice power-gated (0x1000030f) at its commit, while here they're only clock-gated (0x1400024f). I'm happy to run a dump of all 32 AMCC/DCS words at the commit, or anything else that would help. And the earlier question still stands: how did you capture the macOS trace? If I can take the same trace on the full part, we can compare it word for word with yours.

🤖 Generated with Claude Code

@joske

joske commented Nov 17, 2026

Copy link
Copy Markdown
Contributor

Found it: on the full T6041 the reset is dcs_27, and with one line added the J614c sleeps and wakes.

A diag kernel on 7288144 dumped the PS words of all 32 AMCC slices and all 32 DCS channels right after t6041_atc_commons_on(). 63 of the 64 read 0x1400024f (AUTO, auto state 4). The one exception is dcs_27, main +0x530 (DCS_27 in the J614c's ADT pmgr:devices), which reads 0x0f0002ff: forced on, no AUTO. t6041_full_ps leaves it out ("dcs_27 isn't among the words the J614c has on there"), but that conclusion came from my PS-on map, which only covered main +0x000..+0x4f8. 0x530 is past its end, so the map couldn't show it, sorry about that. dcs_27 is the only memory channel outside the main DCS range, and it only exists on the full part, which fits the binned/full split. It also matches what #1910 says for T6040: the engine expects every DCS to gate itself and waits for good if one isn't in automatic mode.

The fix, with the value your table has for the other DCS words:

 	{ 0, 0x280, 0x1000030f },	/* amcc27 */
+	{ 0, 0x530, 0x1400034f },	/* dcs_27 */
 	{ 1, 0x2c0, 0x1000030f },	/* amcc28 */

(and t6041_full_ps[15] → [16], plus the comment above it).

On 7288144 plus that line, J614c (16/40, 32 channels, os-fw 26.6.2), one boot:

  • 3 deep sleeps with rtcwake -m mem -s 20, each back after about 22 s: panel, Wi-Fi with my SSH session still up, Bluetooth (rebinds itself after "power lost in the sleep"), SEP, ANE.
  • Then a lid close of about 4.5 minutes and a lid open: back the same way. suspend_stats: 4 ok.

Still open on the J614c:

  • SD reader: after the first wake, sdhci-pci 0000:02:00.0 fails its resume (mmc0: enable PCI MSI failed, error=-22, pci_pm_resume returns -22). Then mmc0 times out a command every 10 s, which keeps a workqueue busy, so the next suspend fails at freeze after 20 s (wq_busy=1, last_failed_dev 0000:02:00.0). With sdhci_pci unloaded, the later sleeps were fine. No card inserted.
  • EDAC: apple_soc_edac event-log-handler: hardware error: event-log source 9 (IRQ 249) (AIC 476) at the first wake, a few times after it, and not at the later ones. Nothing visibly broken.
  • Bluetooth logs failed to destroy transfer ring 6 / reprobe after command timeout failed at the wakes, but is powered and works after.

🤖 Generated with Claude Code

@joske

joske commented Nov 17, 2026

Copy link
Copy Markdown
Contributor

Two more things from the J614c, one bad and one good.

Merged with a newer integration/aurora, the wake hangs. I merged 7288144 (plus the dcs_27 line) into integration/aurora c0a2a75 (15 Nov) for my daily kernel. Five conflicts, all resolved by keeping both sides: macsmc.c/macsmc.h, pcie-apple.c, soc/apple/sleep.c, g17/sleep.c. On that kernel the J614c never comes back from a deep sleep, with a lid close or rtcwake -m mem -s 20. The screen stays black, there's no watchdog reset 5 s later, and only a forced power-off gets it back. The journal ends at PM: suspend entry (deep), so I can't say whether it dies before the commit or in the wake. The same tree without my one PCIe change below hangs the same way.

The same set of extra commits cherry-picked onto your branch instead (7288144 + dcs_27) wakes every time. So it's something in integration between your base (fc05050) and c0a2a75. That range is 19 PRs, and the one that touches the sleep path is #1963 (T6050 S2R: sleep.c +1258, plus macsmc.c, pcie-apple.c, spmi-apple-controller.c and RTKit). I haven't bisected it, so #1963 is my guess, not a result. Worth knowing before your next rebase.

The SD reader after a wake. I found the cause of the enable PCI MSI failed, error=-22 from my last comment. After a real sleep, the PCI core's noirq power-up of the functions runs while the ports are still down (Unable to change power state from D3hot to D0, device inaccessible), so current_state stays at D3. apple_pcie_resume() then retrains the port and restores the saved config, but the function is still marked D3. The GL9755 driver's resume frees and re-allocates its MSI vector, and __pci_enable_msi_range() refuses a device that isn't in D0 (drivers/pci/msi/msi.c, -EINVAL). There's no interrupt after that, every command times out, and the busy MMC workqueue fails the next suspend at freeze. The fix is to record D0 before the restore:

@@ static int apple_pcie_resume(struct device *dev)
 		if (!port || !port->hib_cold)
 			continue;
+		/*
+		 * The PCI core's noirq power-up ran with the link down and left the function
+		 * marked D3: the reset brought it up in D0, so record that before the
+		 * restore, or its driver's resume can't enable MSI again (-EINVAL).
+		 */
+		pci_set_power_state(fn->pdev, PCI_D0);
 		if (pci_load_saved_state(fn->pdev, fn->state))

With it, on 7288144 + dcs_27: two rtcwake sleeps in a row with the reader bound, no MSI error, no mmc0 timeouts, the reader in D0 after each wake. Before, the second sleep always failed. Wi-Fi and Bluetooth hit the same failed power-up but recover through their own restart.

Still open here: apple_soc_edac event-log-handler: hardware error: event-log source 9 (AIC 476), a few times around each wake. Nothing visibly breaks.

🤖 Generated with Claude Code

@AxxzyWasTaken

Copy link
Copy Markdown
Contributor Author

Rebuilt the series without the merge commit: the 37 commits now sit directly on fc05050, the merge's integration/aurora side. The four files the merge resolved got the same resolutions, and the final tree is identical to the previous head 7288144.

It still conflicts with current integration/aurora, in drivers/soc/apple/sleep.c, drivers/gpu/drm/asahi/g17/sleep.c, drivers/pci/controller/pcie-apple.c, drivers/mfd/macsmc.c and include/linux/mfd/macsmc.h. Upstream changed those a lot since (about 10k lines in sleep.c alone). dt-checks still fails: /t6041-sleep (apple,t6041-sleep) has no binding schema. Both need a proper rebase and a binding, which I'll do separately.

Disclosure: prepared with an AI tool (the commits carry Assisted-by: LLM); tested on my M4 Max as described.

@joske

joske commented Nov 17, 2026

Copy link
Copy Markdown
Contributor

An overnight result from the J614c (16/40), on 7288144 + dcs_27 + the SD fix and a few unrelated cherry-picks, now my daily kernel.

Battery: one lid-closed deep sleep of 9 h 30 min, unplugged. charge_now went from 5825 to 5807 mAh (−18 mAh, −205 mWh by energy_now), and capacity stayed at 99%. That figure also covers about 17 s awake before the suspend and about 1.5 min awake after the wake, so the sleep itself was well under 2 mA on average. That's in line with your ~7 mAh in 8.5 h, and better than macOS's "under 1% a night" on this machine.

After that wake: panel, Wi-Fi (I'm typing this over it), speakers, Bluetooth and the camera all work. The camera also worked after earlier wakes, so the description's note that the ISP doesn't come back is out of date, at least for T6041. No EDAC event-log source 9 at this wake.

Brightness after a lid wake: two parts.

  • Before, the panel came back at 5 to 7 nits whatever the level, and only a DPMS on (lock/unlock) fixed it. dcp_backlight_s2r_restore() asks for a fade (dcp_bl_request()), but after the S2R restart nothing ever runs it: no fade_start, no fade work. Its first present can also reach the firmware before the backlight is enabled (Received brightness update when backlight is disabled). In my test change, the restore now:

    • jumps straight to the target (dcp_bl_jump());
    • clears a dpms_off left from the lid close, since the mode set has lit the panel;
    • sends the level once more 1.5 s later.

    With that, the level is right after a lid wake (S2R brightness restore: bd 153 target 153 level 153 actual 153 ...).

  • Still open: even at the right level, the picture looks clearly dimmer (about half) than the same level after a lock/unlock, and a lock/unlock still brings it back. So something besides the brightness value isn't restored by the S2R re-commit. My guess is color/gamma or another panel setting that the DPMS-on path sends and the S2R path doesn't, but I haven't found it yet. The rtcwake sleeps don't show this; only lid sleeps do.

I can push the brightness change somewhere if it's useful, but it's a workaround for the level, not the whole fix.

🤖 Generated with Claude Code

The probe line and the header comment still said "test, entry only"
and "no resume yet". The driver resumes: the commit writes cpu_resume
into aura's S2R mailbox, and the wake comes back through it. No code
change.

Assisted-by: LLM
Signed-off-by: AxxzyWasTaken <axxzytte@gmail.com>
The spin table only stops a parked CPU that answered a hold first:
smp_rvbar_spin_table_cpu_die() leaves the park for the stop only while
spin_table_hold_parked() has asked for it. The T6041 commit called
spin_table_stop_parked() without the hold, so every other core stayed
awake in the park, with its power state still on. The sleep engine then
waited for the cores, and the watchdog reset the SoC about 5 s after
the commit.

Ask for the hold in .enter while the suspend can still be refused, as
the sleep core does for T8140, and refuse the sleep if a parked CPU
does not answer.

Assisted-by: LLM
Signed-off-by: AxxzyWasTaken <axxzytte@gmail.com>
The driver writes the PMU over the SPMI controller by itself and gave
each command's reply a fixed 50 us before draining the FIFO. A reply
that arrived later stayed there. The controller keeps its power
through the sleep, so after the wake the SPMI driver's next command,
the RTC's acknowledge of the alarm that woke the machine, read that
reply ("unexpected SPMI response 0x8b30"), the RTC marked its I/O as
failed and refused every later suspend.

Poll for the reply for up to 1 ms, then drain the FIFO.

Assisted-by: LLM
Signed-off-by: AxxzyWasTaken <axxzytte@gmail.com>
While Linux is away the queue keeps being used: aura's wake path and
the platform's sleep code write the PMU through it. Every command the
driver sent was answered before the suspend, so a reply still waiting
at the resume is someone else's, and the next command takes it as its
own. On the M4 Max that was the RTC's acknowledge of the alarm that
woke the machine ("unexpected SPMI response 0x8b30"), after which the
RTC refused every later suspend.

Read such replies out at resume_noirq after a firmware-driven suspend.

Assisted-by: LLM
Signed-off-by: AxxzyWasTaken <axxzytte@gmail.com>
aura writes the PMU on the same queue at the wake. Waiting for the
first reply after a write could take one of its replies for ours and
leave ours behind. Empty the FIFO before the write as well.

Assisted-by: LLM
Signed-off-by: AxxzyWasTaken <axxzytte@gmail.com>
… lists

On T6041 the enclave is told to sleep at device suspend. Its DART is
saved first with apple_dart_sleep_prepare(), which only saves a DART
whose node has apple,sleep-restore, and only such a DART is then left
alone by its own callbacks. Without the property nothing was saved, the
enclave slept, its power domain went off, and the DART's noirq suspend
read its registers: the suspend hung there. This was seen on a J614C
booted with the in-tree device tree, which has no lists for that DART.

Refuse the sleep at suspend instead when the DART has no list.

Assisted-by: LLM
Signed-off-by: AxxzyWasTaken <axxzytte@gmail.com>
Without the mailbox the driver set its suspend ops anyway and refused
every sleep in .enter. Deep sleep is then the default, so a kernel
booted at EL2 by another loader could not suspend at all. Don't bind
instead.

Assisted-by: LLM
Signed-off-by: AxxzyWasTaken <axxzytte@gmail.com>
The T6041 sleep driver binds to an "apple,t6041-sleep" node, which the
J614c and J616c device trees get at their root. The node has no
registers and no properties: the driver maps the blocks it writes
itself, and finds its way back from a sleep through the boot loader's
S2R mailbox in /chosen. The other Apple sleep bindings (apple,t8140-sleep
and apple,t6050-sleep) require the registers and the display
coprocessors, so this node gets a binding of its own.

Without it, CHECK_DTBS reports /t6041-sleep in t6041-j614c.dtb and
t6041-j616c.dtb as matching no schema.

Assisted-by: LLM
Signed-off-by: AxxzyWasTaken <axxzytte@gmail.com>
Give the SEP's DART the writes around a sleep that T8140's has, as the
J616C's bring-up device tree does: the SEP driver saves the DART with
them before the enclave sleeps at suspend, and refuses that sleep
without them.

Add the apple,t6041-sleep node to the J614C and J616C, so the in-tree
device tree is enough to test suspend-to-RAM there. The driver stays out
unless aura passes its S2R mailbox.

Assisted-by: LLM
Signed-off-by: AxxzyWasTaken <axxzytte@gmail.com>
The 40-core J614C on 25G83 firmware runs CONFIG_25G83, which had no
sleep_restart, so its GPU refused every deep sleep. Set it as the
32-core profile has.

Assisted-by: LLM
Signed-off-by: AxxzyWasTaken <axxzytte@gmail.com>
Remove what was only there to find the sleep's problems on hardware:
the dry, diag and serror_go parameters, the progress marks and notes in
PMU scratch and SRAM, the register snapshot against the macOS table at
the commit, the before and after windows with their S2RDIFF and S2RAMCC
lines, the console, die and panic notes, and code nothing called any
more (the MCC stop, the AXI2AF teardown, the core snapshot).

The writes, delays and polls of the commit and of the wake are the same
as before. The AMCC cache words and the display's PMC votes, which the
wake puts back, are saved on their own now instead of through the
comparison windows. The clkgen write-back that worked from those windows
only ran with diag set, and goes with them. What the wake could not
bring back is reported at .end.

apple_sleep_coprocs_count() had no other user and goes too.

Assisted-by: LLM
Signed-off-by: AxxzyWasTaken <axxzytte@gmail.com>
apple_dart_display_keep() read the whole register block of a display
DART before a sleep and printed what differed after the wake, with each
stream's TCR and TTBR. That was only there to look for what the wake
took. Keep the TCR and TTBR save and write-back, and declare the helper
in <linux/soc/apple/dart.h> instead of in its caller.

Assisted-by: LLM
Signed-off-by: AxxzyWasTaken <axxzytte@gmail.com>
The display stop and restart for T6041's deep sleep ran as test steps
first: an s2r_stop parameter took them one at a time, and each step
wrote progress marks to PMU scratch, logged S2RSTOP, S2RSTART and
S2RTIME lines and kept result words. A kmsg dumper put a panic's call
trace into PMU scratch, and s2r_bl showed the backlight state.

Remove all of that, and the parameters with it. The waits are constants
with the values that ran (settle 200 ms, words 100 ms, stop 300 ms,
down 100 ms), and each switch keeps the behaviour it had by default: the
display power-off without the sleep transition (s2r_hard = 2), bridge 4
left as boot has it, the GAPF word not written back, the backlight and
the boot power words restored, the steps run from the display's own
suspend and resume. What fails is still reported. The power word step
now unmaps what it maps.

dcp_sleep() has no caller again and gets its __maybe_unused back.

Assisted-by: LLM
Signed-off-by: AxxzyWasTaken <axxzytte@gmail.com>
…ISPEXT

The M4 Max gate enables each external display chain as the in-tree
trees describe it: no DISPEXT power domain, the domains are on as iBoot
left them. The J616c's bring-up EL2 tree does describe them: the chains'
DARTs, mailbox and DCP point at dispextN_fe, which is disabled like the
PMP report entries above it. Enabled as they were, those devices waited
for power domain providers that never came and failed at the deferred
probe timeout (the mailboxes with -110), and the display subsystem never
bound: the internal panel stayed on simpledrm.

Drop the power-domains property of a chain node whose domain stays
disabled, so such a tree ends up as the in-tree ones do. Nothing changes
with the in-tree trees.

Assisted-by: LLM
Signed-off-by: AxxzyWasTaken <axxzytte@gmail.com>
The power word step forces both external display chains down, and the
display subsystem maps every framebuffer through their display DARTs.
Only the DART of an external DCP the stop visited was marked dark, but
a DCP whose probe failed (the display subsystem skips it as unusable)
is not bound to the DCP driver, and the stop never saw it. After the
wake, an sddm restart or a logout unmapped a framebuffer, flushed that
DART behind its dead block, and the machine stopped: on the J616c the
HDMI chain's display DART at 0x3a2300000.

Mark the external chains' display and DCP DARTs dark by device tree
node right before their power words go down, while they can still be
read, whether or not their DCP bound.

Assisted-by: LLM
Signed-off-by: AxxzyWasTaken <axxzytte@gmail.com>
The display stop for a T6041 sleep kept one "stopped already" slot per
external DCP by name: dcp@3a2e00000 got one, every other external DCP
shared the second. With more than two external chains enabled (dcpext2,
dcpext3) the second external DCP to be stopped marked that slot, and
any after it was skipped: its firmware still ran at the commit. The
DARTs made dark before the power word step were also only those of the
first two chains.

Keep the mark in each DCP, and make the display and DCP DARTs of every
enabled apple,t6041-dcpext node dark, found through the device tree.

Assisted-by: LLM
Signed-off-by: AxxzyWasTaken <axxzytte@gmail.com>
The full T6041 has eight AMCCs and 32 memory channels, the binned one
six and 24. The sleep path only handled 24: the AMCC quiesce, the cache
flush and way restore, and DCS dynamic power gating stopped at the sixth
AMCC, and the macOS commit table (from a binned J616c) has no words for
amcc24-31 or dcs_24-31. On a full J614c those words stayed forced on at
the commit, holding fab3_soc on, and the chip reset instead of sleeping.

Count the channels at probe from amcc24's and amcc28's PS words, which
read target and actual 0 on the binned part. On the full part, run the
quiesce, flush and restore on all eight AMCCs. Put the 15 extra words on
AUTO as macOS has the same kind of word, save them first, and put them
back on the wake before the cache restore.

Assisted-by: LLM
Signed-off-by: AxxzyWasTaken <axxzytte@gmail.com>
The sleep commit writes WKTP = 2 on T6041, but nothing wrote it back.
With the key left at 2 the SMC treats all the time awake after the first
sleep as a power nap: it keeps its own fan policy against a fan request
(on the J616c every fan target write timed out after about 10 s and the
fans stayed at 0 rpm under load), and under thermal load it requests a
sleep once a second ("SMC Thermal pressure requested Sleep during power
nap" in its syslog), slowing the machine down. Only a reboot cleared it.

Write WKTP = 0 after a full wake, and when the sleep ends in a refusal
(apple_smc_complete), as the M4 Pro sleep series in aurora-silicon#1910 does. A dark
wake keeps the key: it sleeps again, and the next commit writes it anyway.

Assisted-by: LLM
Signed-off-by: AxxzyWasTaken <axxzytte@gmail.com>
… EIO again

integration/aurora now requires request-path messages in apple_sep to
be rate-limited (tools/aurora-sep/tests/test_log_ratelimit.py). Use the
rate-limited macros for the T6041 suspend-time sleep path and the
mailbox check.

On platforms with a sleep core commit, sleep_enter() refused a Lost gate
with EBUSY and a message; integration/aurora refuses it with EIO, which
sep-suspend-errors-tests checks. Take that back: T6041 never reaches
it, it sleeps through the suspend-time path.

Assisted-by: LLM
Signed-off-by: AxxzyWasTaken <axxzytte@gmail.com>
…ests

The bcm4377 resume and SEP suspend tests compile extracted driver
functions against fixtures. The T6041 S2R code those functions now call
is not in the fixtures, so neither test built.

bcm4377: dma_window_base, pm_resume_via_firmware() and
bcm4377_queue_rebind(). The controls model resumes after s2idle, so
pm_resume_via_firmware() is false and the rebind is never reached.

SEP: sleep_at_suspend, sleep_enter_at_suspend(), sleep_exit_at_suspend(),
mailbox_drains() and pm::platform_commit(). Every platform the test
models has a sleep core commit, so platform_commit() is true, the
suspend-time path is never taken and the mailbox always drains.

All controls and mutations pass, as does the whole run-host-tests.py
suite (321/321).

Assisted-by: LLM
Signed-off-by: AxxzyWasTaken <axxzytte@gmail.com>
@AxxzyWasTaken

Copy link
Copy Markdown
Contributor Author

Rebased onto current integration/aurora (59b02d5), from fc05050. The PR now has 38 commits: the same 37 plus a binding for the sleep node.

Conflicts

  • soc: apple: sleep (coprocessor stop helpers): the helpers sit next to the reworked coprocessor stop. apple_sleep_coproc_run_off() now reads the sleep core's SoC data, which T6041 doesn't have, so it gets a NULL check. The ASC status is still read as before.
  • drm/asahi: g17: sleep on T6041: the PMP-report-entry lookup of the GPU's power word stays in the non-M5 branch. T6050's asahi_g17_m5_nodes() path is untouched.
  • PCI: apple (conventional ports): kept next to the new T6050 paths. Those return before the conventional-host hooks, so the restore still sees only conventional hosts.
  • mfd: macsmc (WKTP back to 0): wake_silent goes after the T6050 saved-context fields. .complete writes WKTP after clearing sleep_policy.
  • API change, not a textual conflict: the T8142 DPMS work replaced the fixed DCP_S2R_*_STREAMS masks with per-SoC layouts. The T6041 display stop/restart now takes T8140's masks from dcp_s2r_t8140, so the values are unchanged.

Binding
New commit dt-bindings: soc: apple: add the T6041 sleep node, placed right before the DT commit. apple,t6041-sleep has a compatible and nothing else: the driver maps its own blocks and finds the way back through the S2R mailbox in /chosen. apple,t8140-sleep and apple,t6050-sleep both require registers and display coprocessors, so this node gets its own schema. This fixes the dt-checks failure.

Duplicates
I found none to drop. Upstream has no WKTP reset (#1910 is still open) and no conventional-port restore (#1477 was closed). The new pinctrl irq_mask_ack, the apple_dart_m5_display_* helpers and the T6050 PMP apple,sram-keep cover other cases or only T6050/T8142.

Tested (compile and checks only)

  • Each commit builds the objects it touches (LLVM=1). That covers soc/apple including the Rust drivers, asahi, drm/apple, pcie-apple, macsmc, AIC, nvme-apple, SPMI, pinctrl, bcm4377 and DART. The head adds no warnings in these files.
  • dt_binding_check passes for the new schema.
  • scripts/dtc/apple-dt-check.sh, base vs head as in CI: 0 new warnings, binding ok.
  • tools/asahi/run-host-tests.py: 427/427.

The rebased series has not been booted. What ran on the J616c is the previous head on fc05050.

Disclosure: prepared with an AI tool (the commits carry Assisted-by: LLM); tested as described.

@joske

joske commented Nov 17, 2026

Copy link
Copy Markdown
Contributor

I booted the rebased head (cd59f6b on integration 59b02d5) on the J614c, with the dcs_27 line added. It wakes, from rtcwake -m mem -s 20 and from a lid close of a few minutes: panel, Wi-Fi, speakers and the SD reader all back. So the hang I had with my own merge onto c0a2a75 doesn't show up in your rebase. dcs_27 is still missing from the head; without it the full T6041 resets in the sleep (see my earlier comment).

On the dim panel after a wake, a correction and a finding.

Correction: I wrote that only lid sleeps showed it. On the rebased head the rtcwake sleeps show it too: the right level (S2R brightness restore: bd 79 target 79 level 79 ...), but the picture at about half its brightness.

Finding: a DPMS off and on from the compositor fixes it every time, even with only 0.1 s off (GNOME: busctl --user set-property org.gnome.Mutter.DisplayConfig /org/gnome/Mutter/DisplayConfig org.gnome.Mutter.DisplayConfig PowerSaveMode i 1, then i 0). The same cycle done in the driver doesn't, as long as it runs inside the wake:

  • The wake's mode set in dcp_t6041_recommit() is already a full off and on (dcp_poweroff(), then dcp_poweron() 30 ms later).
  • An extra active=false/active=true commit right after it, still in the resume, didn't help.
  • Marking the backlight level lit before the mode set (dcp_backlight_wake_lit(), as the firmware 27 wake does) and sending the colour matrix, brightness correction and gamma with it (color_mgmt_changed) didn't help either.
  • The same off and on from a delayed work, about 0.9 s after PM: suspend exit, fixes it.

In the DCP syslog the bad and the good power-on look the same:

  • the BLM and LTM M3 loads, handle_stage1_enable blm_en 1 ltm_en 1 and enable_backlight_gated: enabling backlight;
  • the same brightness_changed ramp, ending at the same value.

The differences:

  • Only the first power-on after the restart loads the LED M3 (load LED M3 IMem/DMem).
  • Only that one runs inside the resume.

So something in the mini-LED path that the first power-on after the restart sets up seems to need either time or the end of the resume. I don't know which yet. A delayed off/on works as a stopgap in my daily, but I don't think it's the right fix for the PR.

Still seen on the J614c: event-log source 9 (EDAC) a few times after each wake. Nothing visibly breaks.

🤖 Generated with Claude Code

The full T6041 (J614c, 32 channels) reset in the sleep. joske's dump of
all 32 AMCC and 32 DCS power words right after t6041_atc_commons_on()
found 63 of them on AUTO and one forced on: dcs_27 at main +0x530
(DCS_27 in the J614c's ADT pmgr:devices) read 0x0f0002ff. It isn't next
to dcs_26, so the list of the full part's words missed it.

Add it to t6041_full_ps with the value the other DCS words get,
0x1400034f. With this line the J614c sleeps and wakes (rtcwake and lid,
on the previous head and on this one rebased; joske). The list is only
used on the full part (eight AMCC groups); on the binned J616c dcs_27 is
fused off and reads 0x0f000300.

Assisted-by: LLM
Signed-off-by: AxxzyWasTaken <axxzytte@gmail.com>
@AxxzyWasTaken

Copy link
Copy Markdown
Contributor Author

@joske thanks for all of this, and sorry for the slow answer.

dcs_27 is in a new commit at the head, "soc: apple: t6041-sleep: dcs_27 to AUTO at the commit on the full part". It is your line, { 0, 0x530, 0x1400034f } after amcc27, with t6041_full_ps[16] and full_pre[16]. The comment above the list no longer says the J614c doesn't have the word there. The commit message credits your dump and your J614c sleeps. I've built it but not booted it. The list only runs on the full part anyway: on my binned J616c, dcs_27 reads 0x0f000300 (#2175) and the list is skipped.

Camera: thanks. The description now says the camera works after a wake on the J614c, and drops the old note.

Dim panel after a wake: I don't know the cause yet, and I haven't looked for it on my J616c. From your findings, the first power-on after the restart differs in two ways: it loads the LED M3, and it runs inside the resume. A later off and on fixes the picture. These are the checks I'd run next, in this order:

  1. Time, or the end of the resume? Try two runs.

    • (a) Do the extra off and on still inside the resume, but after a msleep(1000) at the end of dcp_t6041_start_one().
    • (b) Do it from a PM_POST_SUSPEND notifier, with no delay.

    If (a) fixes it, the first power-on needs time, probably because the LED M3 is still starting when the power-on sends its settings. If only (b) fixes it, something that resumes after the display is what matters.

  2. If it's (b): get the resume order after the DCP with pm_print_times. The candidates are the SMC (the panel and backlight keys), the AOP (ALS) and the backlight class device. If one of them feeds the LED driver, the one that resumes before your delayed off and on is the difference.

  3. The DCP syslog of the first power-on. Look at where the LED M3's own start-up lines fall relative to enable_backlight_gated and the last brightness_changed. If the backlight is enabled before the LED M3 says it's running, that would fit a picture at half brightness. A cold boot's first power-on loads the LED M3 too. If the panel is fine after a cold boot, comparing its order with the wake's should show what differs.

  4. The calls of the bad and the good power-on. Compare the property calls our driver logs for each (setProperty/applyProperty and the swap's brightness fields). Look for anything the second sends that the first doesn't, such as an LED or local dimming setting.

I agree that the delayed off and on shouldn't go into this PR as the fix. If none of the checks above finds the cause, it could go in as a separate commit marked as a workaround, so it's easy to drop later.

Disclosure: prepared with an AI tool (the commits carry Assisted-by: LLM); tested as described.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants