Repository navigation
Conversation
This was referenced Nov 15, 2026
The M5 Ultra (T6050, J775D) numbers its 30 CPUs in the ADT 0 to 34: 6 * cluster + core, 18 per die, so cpu-ids 5, 11, 17, 23 and 29 are unused. smp_init() indexes every per-CPU table by cpu-id and stops at MAX_CPUS (32), so it logs "cpu-id 32 exceeds max CPU count 32" for cpu-ids 32 to 34, and Linux gets 27 of the 30 CPUs. MAX_CPUS goes to 64, the width of the u64 CPU masks (smp_started_mask, hv_cpus_in_guest, s2r_twi_mask). It also sized the secondary stacks, one 64 KiB stack per cpu-id, so on its own it would add 2 MiB to every Stage 2's .bss (and 2 MiB to a Stage 1's heap), as dj noted in aurora-silicon#178. The stacks keep their own count instead, SECONDARY_STACKS (32, as now), and go to the CPUs present in cpu-id order: smp_init() records each CPU's slot in stack_slot[], next to smp_reset_stacks in .data.smp_shared because a secondary reads it with the MMU off. They stay in .bss (svenpeter's e327b97, for PSCI) or Stage 1's heap (Ryan's f545f79). A CPU past the 32nd present would get no stack and is skipped, as a cpu-id past MAX_CPUS still is. Earlier raises took the same path for each larger Mac: 20 to 24 for the M2 Ultra (dabad98), 24 to 32 for the M3 Ultra (1a0e368). Every SoC runs this code, since MAX_CPUS is compile-time: - The per-CPU tables double. In a RELEASE=1 CHAINLOADING=1 ESP_STAGE2=1 build the .bss objects grow by 12.6 KiB (hv_sprr_cpus 6.5 KiB, pcpu 2 KiB, hv_secondary_regs 1 KiB, the rest 3.1 KiB), which the alignment padding absorbs: .bss ends 0x1200 lower and _end is unchanged. .data.smp_shared grows by 6.6 KiB (kernel_park, spin_table_image, smp_reset_stacks, stack_slot) inside its 64 KiB padding, so _file_vital_end and _file_end are unchanged. The two configurations CI's esp-stage2-size job builds report the same vital size as main. - The stacks: same count, same place. Where the ADT's cpu-ids run 0 to n-1 with no gap a CPU keeps the stack it had. Where they skip (as on J775D's first die, and J616s, which has no cpu9), a CPU after the gap takes a lower one of the same 32. T8142 keeps its own block, indexed by cpu-id as before. T8152's twelve cpu-ids are 0 to 11, so its slots equal them, as its cpu_reset in start.S assumes. - Development builds only: the NV2 trace buffer (NV2_TRACE=1) grows from 2 to 4 MiB of heap, the SPTM-virt GL2 shadow (SPTM_VIRT=1) from 256 to 512 KiB, and an SPTM_VIRT=1 image by 64 KiB (its per-CPU tables in .bss). Default and release images keep their size. - The cpu-id loops (start.S's reset-stack search, kboot's CPU pruning, hv, nv2, sptm_virt) run to 64 instead of 32. Tested on J775D (M5 Ultra) only, with main at eb66d6c plus this commit and the M5 Ultra GPU firmware-build commit, the kernel uploaded over USB, booting integration/aurora at 7c71d6df6563: the loader starts all 29 secondaries, cpu-ids 32 to 34 included, with no "exceeds" line; Linux brings up 30 CPUs (online 0-29, six clusters) and runs a task on each, probes the GPU and passes its compute, chained-submit and 46-client checks, with no error-level kernel message. Stage 1, ESP Stage 2 and SPTM_VIRT builds compile with main's warnings, the host pytest suite gives main's 707 passed and 56 skipped, and clang-format is clean. Signed-off-by: Jeff Queisser <funkzy@users.noreply.github.com>
funkzy
force-pushed
the
t6050-ultra-cpus
branch
from
November 18, 2026 04:44
cd798be to
bf986d8
Compare
Contributor
|
No-regression rows with this PR stacked on current main (aura-try Stage 2 trials, production kernels): J616s (T6040, cpu9 absent from the ADT) 14/14 CPUs online, s2idle 3/3; Neo (T8140, J700) 6/6 CPUs, image fits, s2idle 3/3; M2 8/8 and M1 Pro 10/10 CPUs, s2idle 3/3. 0 kernel errors and 0 DART faults on all four. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What is missing
The M5 Ultra (Mac Studio, J775D) numbers its 30 CPUs 0 to 34 in the ADT: 6 * cluster + core, 18 per die, so 5, 11,
17, 23 and 29 are unused.
smp_init()indexes every per-CPU table by cpu-id and stops at MAX_CPUS (32). On main theloader logs "cpu-id 32 exceeds max CPU count 32: increase MAX_CPUS" for cpu-ids 32 to 34 and Linux gets 27 of the 30
CPUs.
What this adds
smp_started_mask,hv_cpus_in_guest,s2r_twi_mask).Stage 2's .bss and 2 MiB to a Stage 1's heap (dj's note in M5 Ultra multi-die groundwork: AIC IRQ numbers, CPU die, PMGR die stride, MIDR parts #178). The stacks get their own count instead,
SECONDARY_STACKS(32, as now), and go to the CPUs present in cpu-id order:smp_init()records each CPU's slotin
stack_slot[], in.data.smp_sharedbesidesmp_reset_stacks, because a secondary reads it with the MMU off.Earlier raises took the same path for each larger Mac: 20 to 24 for the M2 Ultra (marcan's dabad98), 24 to 32 for
the M3 Ultra (jannau's 1a0e368).
Who runs the changed code
Every SoC: MAX_CPUS is compile-time.
RELEASE=1 CHAINLOADING=1 ESP_STAGE2=1build the .bss objects grow by 12.6 KiB(
hv_sprr_cpus6.5 KiB,pcpu2 KiB,hv_secondary_regs1 KiB, the rest 3.1 KiB), which the alignment paddingabsorbs: .bss ends 0x1200 lower and
_endis unchanged..data.smp_sharedgrows by 6.6 KiB (kernel_park,spin_table_image,smp_reset_stacks,stack_slot) inside its 64 KiB padding, so_file_vital_endand_file_endare unchanged.had. Where they skip (as on J775D's first die, and J616s, which has no cpu9), a CPU after the gap takes a lower one
of the same 32.
cpu_resetin start.S assumes.NV2_TRACE=1) grows from 2 to 4 MiB of heap and the SPTM-virtGL2 shadow (
SPTM_VIRT=1) from 256 to 512 KiB.Size
clang/lld 23.1.4,
RELEASE=1 CHAINLOADING=1 ESP_STAGE2=1:aura.bin3997696 bytes as on main;_file_vital_end0x11b6c8 and
_end0x3d0000 unchanged; .bss 0x25bf30 to 0x25ad30. Code: +0x88 forsmp_assign_stacks(), a fewinstructions in the callers. A plain
MAX_CPUS 64would take .bss to 0x45ed30 and_endto 0x5d0000. CI'sesp-stage2-size configurations report main's vital sizes (1095728 and 744376 bytes). The exception is a development
build with
SPTM_VIRT=1, whose .bss objects add 60 KiB (nv2_cpus32 KiB,sptm_world12.75 KiB) and cross a64 KiB boundary, so
_bss_end,_endandaura.bingrow by 64 KiB (CHAINLOADING=1 SPTM_VIRT=1:_bss_end0x190000 to 0x1a0000, 1769472 to 1835008 bytes; with
ESP_STAGE2=1, 4128768 to 4194304 bytes).Validation
RELEASE=1 CHAINLOADING=1(Stage 1, heap stacks),ESP_STAGE2=1 SPTM_VIRT=1and CI's twoesp-stage2-size configurations: all compile with main's warnings, none in smp.
tests/python, pytest 9.1.1): 707 passed, 56 skipped, as on main.unchanged. There the build above has main's warnings,
aura.bin3997696 bytes and_file_vital_end0x11b890, bothas on main (the figures under Size are from the earlier base), and the host pytest suite as CI runs it gives
750 passed, 56 skipped, as on main.
Hardware
booting integration/aurora at 7c71d6df6563 with the M5 Ultra linux pull request (and a compute-timestamp change
that is no longer in this set): no "exceeds" line; the loader starts all 29 secondaries, cpu-ids 32 to 34
included, and passes them to the device tree. Linux: "smp: Brought up 1 node, 30 CPUs", online 0-29, six
clusters, a task run on each CPU. The GPU probes and passes its compute, chained-submit and 46-client checks, with
no recovery, fault or timeout, and the kernel log has no error-level line.
fc050503ffcd with only the linux pull requests below, one boot: "smp: Brought up 1 node, 30 CPUs", online 0-29;
the GPU passes its compute, chained-submit and 46-client checks and a 628-case render CTS subset (571 pass, 57 not
supported), with no recovery, fault or timeout.
integration/aurora at 1f38ff8f4dc9 with the linux pull request below: "smp: Brought up 1 node, 30 CPUs", online
0-29, on the 27.0 stub (the GPU checks and the CTS subset pass as above) and on the 27.2 stub (26B123) alike.
assembly path that reads a stack by cpu-id, and one on a T8140 would check its .bss limit.
Question
64 follows the u64 masks and #178's note. J775D needs 35; MAX_CPUS 36 (two dies of 18) would add 1.6 KiB of .bss
objects instead of 12.6 KiB. Which would you rather have?
Part of the M5 Ultra platform pack
Each pull request below stands on its own; together they bring the Mac Studio (M5 Ultra, J775D) up from public
branches to the desktop.