Skip to content

soc: apple: sleep: T8142: start the cores and write the cluster words as XNU (on #2192) - #2193

Open
m0d16l14n1 wants to merge 2 commits into
aurora-silicon:integration/aurorafrom
m0d16l14n1:j815/t8142-sleep-xnu-wake
Open

m0d16l14n1 wants to merge 2 commits into
aurora-silicon:integration/aurorafrom
m0d16l14n1:j815/t8142-sleep-xnu-wake

Conversation

@m0d16l14n1

Copy link
Copy Markdown
Contributor

Stacked on #2192 (the pen fix, the first commit here). The burst below needs it.

In a traced macOS 27.2 sleep on J813 (boot, a 40 s sleep, the wake and three more entries), XNU differs from us in three places. This PR does the same as XNU, each behind a parameter:

XNU (trace) Before Parameter (default)
Wake 16.5 ms after the wake, the boot core starts all nine other cores in one 0.23 ms burst, three words each (+4 the core, +8 the E core or 0, +0xc the P core or 0) each core later, at its own bring-up, two words apple_sleep.cpu_burst (Y): burst for the cores online at .begin; they wait in the holding pen and each bring-up only releases its core
Commit clusters' words read-modify-written: 0xf → 0 after boot, 0x2f → 0x20 after every wake 0 every time apple_sleep.cluster_rmw (Y)
Perf states the SoC/DCS perf-state page is never touched lowered before the memory stop apple_sleep.perf_down (N)

T8142's data has no cpu_clusters, so the burst takes the cluster from the MPIDR.

Tested on J815 (MacBook Air 15" M5):

  • Two real deep sleeps (256 s and 172 s), woken by the keyboard/trackpad. Each had one "9 cores started in one burst", nine "started in the wake burst" and nine "Booted secondary processor".
  • After each wake: all CPUs online, display, NVMe, Wi-Fi and Bluetooth back.
  • An earlier 366 s sleep ran with cluster_rmw and perf_down=N alone.
  • checkpatch --strict is clean, and the change builds with W=1 on integration/aurora.

To go back for one sleep: apple_sleep.cpu_burst=0 cluster_rmw=0 perf_down=1.

@sithglan, @gig3m: it comes from your g01 trace and runs on J813 too. Could you give it a few deep sleeps there? If the wake misbehaves, try cpu_burst=0 first.

Part of the J815 bring-up; board results are summarised in #135.

🤖 Generated with Claude Code

m0d16l14n1 and others added 2 commits November 18, 2026 16:02
secondary_holding_pen_release is written with a CPU's MPIDR to let it
out of the pen and never reset, so after boot it still names the last
CPU released. secondary_data.task likewise keeps that CPU's idle task:
__cpu_up() only clears it on failure.

With the restartable spin table (aurora,rvbar-spin-table) a CPU comes
back through the pen whenever the SoC restarts it at its reset vector:
after a system sleep's commit, and at the wake. The CPU whose MPIDR is
still in the pen walks straight through into secondary_startup, ahead of
its own bring-up, and runs secondary_start_kernel() as that CPU with
the stale task. The hotplug state machine has not started it, so no
STARTING callback runs (no arch timer, no tick device), yet it marks
itself online and completes cpu_running, so the real bring-up returns
at once. It idles quietly while cpuidle is paused and faults at the
first menu_select() after cpuidle_resume():

  Unable to handle kernel NULL pointer dereference at virtual address 0000000000000020
  CPU: 9 UID: 0 PID: 0 Comm: swapper/9
  pc : tick_nohz_get_sleep_length+0x68/0xd4
  Call trace:
   tick_nohz_get_sleep_length+0x68/0xd4 (P)
   menu_select+0x414/0x774
   cpuidle_select+0x1c/0x28
   do_idle+0x1e8/0x310
   cpu_startup_entry+0x34/0x38
   secondary_start_kernel+0x198/0x1bc
  Kernel panic - not syncing: Attempted to kill the idle task!

Seen on a J815 (MacBook Air 15" M5) when every core was started at the
end of the wake, as macOS does, instead of one by one at its bring-up.

Reset the release to INVALID_HWID in the released CPU's cpu_postboot,
which runs before complete(&cpu_running), so it can't race with the
next bring-up; and again before a suspend's hold and after the wake,
before any CPU is started. Clear secondary_data.task after a successful
bring-up too, as kernels before 5.x did, so a core that still reaches
__secondary_switched without a bring-up of its own waits in
__secondary_too_slow instead of running as another CPU.

Tested on J815: two real deep sleeps (256 s and 172 s) with all nine
secondary cores started together at the wake, all back online, where
the same kernel without this change panicked on the first wake.

Signed-off-by: Nick <nekreshen@yandex.ru>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Nick <nekreshen@yandex.ru>
… as XNU

A traced macOS 27.2 sleep on a J813 (boot, a 40 s sleep, the wake and
three more entries) differs from ours in three places:

- The wake: 16.5 ms after it, right after the memory side and the WD2
  stop, the boot core starts all nine other cores in one 0.23 ms burst,
  three words each (+4 the core's bit, +8 the E core's bit or 0, +0xc
  the P core's bit or 0). We started each core later, from its own
  bring-up, with two words.
- The commit: the clusters' words are read-modify-written. The first
  sleep after boot reads 0xf and writes 0; every commit after a wake
  reads 0x2f and writes 0x20, keeping bit 5. We wrote 0 every time.
- The SoC/DCS perf-state page is never accessed.

Do the same, each behind a parameter:

- apple_sleep.cpu_burst (default Y): at the end of the wake, start the
  cores that were online when the sleep began, in one burst with XNU's
  three words. They come up through the boot object into the kernel's
  holding pen and wait there; each bring-up only releases its core.
  T8142's data has no cpu_clusters, so the cluster comes from the MPIDR.
- apple_sleep.cluster_rmw (default Y): clear only the clusters' on bits
  at the commit.
- apple_sleep.perf_down: default N.

The burst depends on the holding pen being closed behind each released
CPU ("arm64: spin table: close the holding pen behind each released
CPU"); without it the last CPU booted runs ahead of its bring-up.

Tested on J815 (MacBook Air 15" M5): two real deep sleeps (256 s and
172 s), each with "9 cores started in one burst", nine "started in the
wake burst" and nine "Booted secondary processor", all CPUs back,
display, NVMe, Wi-Fi and Bluetooth up; cluster_rmw and perf_down=N in a
third sleep of 366 s before that. To go back for one sleep:
apple_sleep.cpu_burst=0 cluster_rmw=0 perf_down=1.

Signed-off-by: Nick <nekreshen@yandex.ru>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Nick <nekreshen@yandex.ru>
@sithglan

Copy link
Copy Markdown
Contributor

Tested the head (8e270f6, on integration/aurora 0abc2f3, so #2192 and #2177 are in) on my J813 with no other patches and the config of my daily kernel. Setup: macOS 27.2 (26B123) stub, the enrolled EL2 aura 8b9c92ea, apple_sleep.deep=1 nvme_apple.sleep_untested=1 appledrm.s2r_untested=1 idle=nop arm64.nowfxt maxcpus=1. The other nine CPUs are onlined from userspace after boot, and SEP is attached.

Eight real deep sleeps on one boot, woken by the PMU alarm (t8142_wake_s), Sway on the internal panel:

  • Defaults (cpu_burst=Y cluster_rmw=Y perf_down=N): 8, 20, 60 and 180 s. Each wake logged one "9 cores started in one burst", nine "started in the wake burst, released from the pen" and nine "Booted secondary processor".
  • Then the old behaviour on the same boot (cpu_burst=0 cluster_rmw=0 perf_down=1 written to /sys/module/apple_sleep/parameters): 8 and 20 s, cores started one by one.
  • Then the defaults again: 8 and 30 s.
  • All eight: suspend_stats success 0 -> 8, fail 0, boot object resumes 0 -> 8, wake source NUB.SPMI0.SW3, same boot ID, elapsed within 0.9 s of the alarm.
  • After each wake, all of these passed:
    • CPUs 0-9 online, and each one ran a task;
    • the panel enabled;
    • a GPU clear and readback;
    • Bluetooth powered, with the factory address;
    • a Wi-Fi ping to the gateway;
    • a 32 MiB direct NVMe read with the expected checksum.
  • The first ping, 6 s after the wake, sometimes lost a packet (3 of 8 wakes, in both modes); Wi-Fi was clean right after.
  • dmesg: no oops, warning or "failed to come online". Only the messages the boot already has (IRQ affinity -38, DCP syslog lines).
  • The nine cores came up in 2.8-2.9 ms with the burst and 3.3-3.4 ms one by one.

Not covered: a lid wake.

@sithglan

Copy link
Copy Markdown
Contributor

Lid wake too, on the same boot with the defaults (cpu_burst=Y cluster_rmw=Y perf_down=N) and the normal lid policy (untested_commit=1, t8142_wake_s=0). I closed the lid, waited about 10 s and opened it:

  • It was a real sleep: boot object resumes 8 -> 9, and suspend_stats success went to 9 with fail 0.
  • Wake source MTP.DOCK.CHANNELS.AP0.IRQ, with macsmc-input logging lid open ... at +11857 ms in sleep.
  • The wake started the nine cores in one burst, and the pen released each one.
  • After the wake, all of these passed:
    • CPUs 0-9;
    • the panel back on in Sway, eDP-1 active and powered at 2560x1600;
    • a GPU clear and readback;
    • Bluetooth with the factory address;
    • Wi-Fi 3/3;
    • the 32 MiB direct NVMe read.
  • No oops. dmesg has the same messages as on the alarm wakes.

@gig3m

gig3m commented Nov 18, 2026

Copy link
Copy Markdown
Contributor

Tested on my J813 (macOS 27.2 / 26B123 stub, EL2 aura 8b9c92ea) with this PR's two commits picked onto #2135's merged head 408e27268fd6. Also on top, as in my #2135 comments: #1732's Touch ID node, #2019, #2142's rear USB4 commits, the J813 palm change, a local XRGB8888 plane patch and #2195. Boot arguments: appledrm.s2r_untested=1 apple_sleep.deep=1 nvme_apple.sleep_untested=1 apple_centauri.alpha_sleep_untested=1 idle=nop arm64.nowfxt, with no maxcpus, so all ten CPUs start at boot through the closed pen (nine "Booted secondary processor").

Six real deep sleeps on one boot, Hyprland on the internal panel:

  • Four woken by the alarm, through systemctl suspend:
    • defaults (cpu_burst=Y cluster_rmw=Y perf_down=N) for 30, 60 and 120 s;
    • the old behaviour (cpu_burst=0 cluster_rmw=0 perf_down=1) for 30 s.
    • Each woke 1-2 s after its alarm with wake source: NUB.SPMI0.SW3.
  • Two woken by the lid, with the defaults and untested_commit=1: 3 min 56 s and 47 s (lid open ... at +235190 ms in sleep and +46546 ms).
  • Boot object resumes went 0 -> 6, and suspend_stats ended at success 6, fail 0.
  • Each default-mode wake (three alarm, two lid) logged one "9 cores started in one burst", nine "released from the pen" and nine "Booted secondary processor". The old-mode wake had no burst and nine one-by-one starts.
  • From the first to the ninth "Booted secondary processor", the burst took 7.7 ms and the one-by-one start 8.3 ms.
  • After each alarm wake, a script checked these 15 s after the wake, and all passed:
    • CPUs 0-9 online;
    • Wi-Fi gateway ping 3/3 (no first-ping loss);
    • Bluetooth Powered: yes;
    • a 32 MiB direct NVMe read;
    • a VP9 decode through the AVD, bit-exact against libavcodec.
  • After the two lid wakes, the panel and the keyboard backlight came back on. I checked the rest once after the second: CPUs 0-9, ping 3/3, Bluetooth powered.
  • dmesg: no oops, warning or "failed to come online". Only the IRQ affinity -38 warnings and the DCP syslog lines the boot already has.

The machine was on AC for these, so there's no power number yet. A lid-closed night on battery with t8142_wake_s=3600 follows, to compare with the same stack without this PR (about 0.21 W over 8.9 h, in my #2135 comment).

🤖 Generated with Claude Code

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants