Skip to content

arm64: spin table: close the holding pen behind each released CPU (deep-sleep wake panic) - #2192

Open
m0d16l14n1 wants to merge 1 commit into
aurora-silicon:integration/aurorafrom
m0d16l14n1:j815/spin-table-pen-release
Open

m0d16l14n1 wants to merge 1 commit into
aurora-silicon:integration/aurorafrom
m0d16l14n1:j815/spin-table-pen-release

Conversation

@m0d16l14n1

Copy link
Copy Markdown
Contributor

The spin-table holding pen is never closed after a CPU is released. After boot, secondary_holding_pen_release still holds the last CPU's MPIDR, and secondary_data.task still holds its idle task (__cpu_up() only clears it on failure).

With the restartable spin table (aurora,rvbar-spin-table), a CPU comes back through the pen whenever the SoC restarts it at its reset vector: after a sleep's commit, and at the wake. The CPU whose MPIDR is still in the pen goes straight through it, ahead of its own bring-up:

  • It runs secondary_start_kernel() as that CPU, with no STARTING callbacks run, so it has no tick device.
  • It marks itself online, so the real bring-up returns at once.
  • It then faults at the first menu_select() after cpuidle_resume().
Unable to handle kernel NULL pointer dereference at virtual address 0000000000000020
CPU: 9 UID: 0 PID: 0 Comm: swapper/9
pc : tick_nohz_get_sleep_length+0x68/0xd4
 menu_select -> cpuidle_select -> do_idle -> cpu_startup_entry -> secondary_start_kernel
Kernel panic - not syncing: Attempted to kill the idle task!

The J815 hit this when every core was started together at the end of the wake, as macOS does (the next PR in this series). The same stale value can let a core through after the commit too, because the pen checks the release before secondary_pen_deep.

Change

  • The released CPU resets the release to INVALID_HWID in cpu_postboot. That runs before complete(&cpu_running), so it can't race with the next bring-up.
  • The release is reset again before a suspend's hold and after the wake, before any CPU is started.
  • secondary_data.task is cleared after a successful bring-up, as before 5.x, so a stray core parks in __secondary_too_slow.

Tested on J815 (MacBook Air 15" M5):

  • Two real deep sleeps (256 s and 172 s) with all nine secondary cores started together at the wake: all back online, no oops.
  • The same kernel without this change panicked on the first wake (trace above).
  • checkpatch --strict is clean, and the change builds with W=1 on integration/aurora.

@sithglan, @gig3m: this is in the pen your T8142 sleep path uses. Could you run a few deep sleeps on J813 with it? It's a no-op for the current one-by-one start, apart from closing the post-commit window.

Part of the J815 bring-up; board results are summarised in #135.

🤖 Generated with Claude Code

secondary_holding_pen_release is written with a CPU's MPIDR to let it
out of the pen and never reset, so after boot it still names the last
CPU released. secondary_data.task likewise keeps that CPU's idle task:
__cpu_up() only clears it on failure.

With the restartable spin table (aurora,rvbar-spin-table) a CPU comes
back through the pen whenever the SoC restarts it at its reset vector:
after a system sleep's commit, and at the wake. The CPU whose MPIDR is
still in the pen walks straight through into secondary_startup, ahead of
its own bring-up, and runs secondary_start_kernel() as that CPU with
the stale task. The hotplug state machine has not started it, so no
STARTING callback runs (no arch timer, no tick device), yet it marks
itself online and completes cpu_running, so the real bring-up returns
at once. It idles quietly while cpuidle is paused and faults at the
first menu_select() after cpuidle_resume():

  Unable to handle kernel NULL pointer dereference at virtual address 0000000000000020
  CPU: 9 UID: 0 PID: 0 Comm: swapper/9
  pc : tick_nohz_get_sleep_length+0x68/0xd4
  Call trace:
   tick_nohz_get_sleep_length+0x68/0xd4 (P)
   menu_select+0x414/0x774
   cpuidle_select+0x1c/0x28
   do_idle+0x1e8/0x310
   cpu_startup_entry+0x34/0x38
   secondary_start_kernel+0x198/0x1bc
  Kernel panic - not syncing: Attempted to kill the idle task!

Seen on a J815 (MacBook Air 15" M5) when every core was started at the
end of the wake, as macOS does, instead of one by one at its bring-up.

Reset the release to INVALID_HWID in the released CPU's cpu_postboot,
which runs before complete(&cpu_running), so it can't race with the
next bring-up; and again before a suspend's hold and after the wake,
before any CPU is started. Clear secondary_data.task after a successful
bring-up too, as kernels before 5.x did, so a core that still reaches
__secondary_switched without a bring-up of its own waits in
__secondary_too_slow instead of running as another CPU.

Tested on J815: two real deep sleeps (256 s and 172 s) with all nine
secondary cores started together at the wake, all back online, where
the same kernel without this change panicked on the first wake.

Signed-off-by: Nick <nekreshen@yandex.ru>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Nick <nekreshen@yandex.ru>
@sithglan

Copy link
Copy Markdown
Contributor

Tested on J813 as the first commit of #2193 (head 8e270f6 on integration/aurora 0abc2f3). Details are in #2193 (comment).

  • The J813 boots with maxcpus=1 and onlines the other nine CPUs from userspace. That bring-up works with the pen closed: CPUs 0-9 online.
  • Eight real deep sleeps (8 to 180 s) on one boot, no oops. All ten CPUs were back after every wake.
  • Two of them ran with soc: apple: sleep: T8142: start the cores and write the cluster words as XNU (on #2192) #2193's switches back at the old behaviour (cpu_burst=0 cluster_rmw=0 perf_down=1), which leaves this commit's one-by-one start.
  • The other six ran with the wake burst, so cores waited in the pen until their bring-up released them.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants