Skip to content

PCI: apple: Allow L1 substates on J514S internal links - #489

Open
ilyasakin wants to merge 1 commit into
omacom:integration/aurorafrom
ilyasakin:pcie-apple-l1ss
Open

ilyasakin wants to merge 1 commit into
omacom:integration/aurorafrom
ilyasakin:pcie-apple-l1ss

Conversation

@ilyasakin

@ilyasakin ilyasakin commented Nov 16, 2026 •

Copy link
Copy Markdown

What this changes and why

On the J514S (MacBook Pro 14", M3 Pro, T6030), the internal PCIe links only ever enter ASPM L1:

  • Wi-Fi/Bluetooth: BCM4388 at 01:00.0 and 01:00.1
  • SD reader: GL9755 at 02:00.0

Both ends of each link, and their root ports, advertise ASPM and PCI-PM L1.1 and L1.2 (L1SubCap: PCI-PM_L1.2+ PCI-PM_L1.1+ ASPM_L1.2+ ASPM_L1.1+). But L1SubCtl1 is all off: the boot loader does not set them, and the default ASPM policy keeps what it found. Writing link/l1_2_aspm is accepted and changes nothing under the default policy.

This enables L1 and its substates on the links of a non-tunneled Apple PCIe controller once its bus is scanned, as pcie-qcom does: each device is put in D0 first (PCIe r6.0 §5.5.4), then pci_enable_link_state_locked() runs. That call replaces the link's ASPM and Clock PM defaults, so the mask also keeps L0S (masked by what the link supports; these root ports advertise L1 only), and keeps CLKPM only where the ASPM core left CLKREQ# on. Clock PM is not forced on DT platforms; on the J514S it stays off, as iBoot leaves it.

Only the J514S (apple,j514s), where this was measured, is listed. The J516S has the same T6030 links but has not been measured. Tunneled ports are never touched. The substates are set at probe only, so after a remove and rescan of a device they stay off until reboot.

What was tested

  • Mac model and chip: J514S, MacBook Pro 14" M3 Pro (T6030), 11-core.
  • Power, as shipped: this commit, no pcie_aspm parameter, substates turned off and on again through the link/ sysfs attributes. Display off, 120 one-second samples of the SMC's total system power (macsmc_hwmon "Total System Power") per phase, on AC at 100%, Wi-Fi power save on:
Median Mean
L1 + L1.1 + L1.2 (this commit) 3.06 W 3.10 W
L1 only (substates off at runtime) 3.36 W 3.43 W
L1 + L1.1 + L1.2 again 3.02 W 3.04 W

About 0.3 W. An earlier run with pcie_aspm.policy=powersupersave (which also turns on L0s and Clock PM) gave 3.99 → 3.64 W.

  • Latency: 60 pings to the router: 2.8 / 5.1 ms (avg / max) without the substates, 3.1 / 4.1 ms with them.
  • Throughput: a 400 MB transfer from another Mac on the same LAN, alternating the two policies three times. 308 / 330 / 334 Mbit/s without, 318 / 331 / 315 Mbit/s with.
  • Bluetooth: stayed powered and working. No PCIe, AER, brcmfmac or sdhci messages in dmesg.
  • This commit: booted on my daily branch, cherry-picked unchanged, in a kernel built with the omarchy-pkgs linux-aurora recipe and without any pcie_aspm parameter. The ASPM policy stays default, yet:
    • link/l1_1_aspm, l1_2_aspm, l1_1_pcipm and l1_2_pcipm read 1 on both 01:00.0 and 02:00.0, link/clkpm reads 0 on all three functions, and lspci shows L1SubCtl1: PCI-PM_L1.2+ PCI-PM_L1.1+ ASPM_L1.2+ ASPM_L1.1+;
    • Wi-Fi associated at 160 MHz HE and Bluetooth came up;
    • after a 20 s suspend-to-idle cycle the substates were still on and Wi-Fi still worked.
  • Build: make W=1 drivers/pci/controller/pcie-apple.o (clang) is clean. checkpatch is clean apart from the trailers.

What was NOT tested

  • Other machines: J516S, T6031/T6034 boards and M1/M2 machines. They are not listed, so nothing changes for them.
  • Remove and rescan: not exercised; per the commit message the substates stay off after one until reboot.
  • SD card traffic: I did not move data through the SD reader with the substates on.
  • Long runs: not tested over days. I can report if anything shows up.

Risk

  • J514S only: the internal links can now enter L1.1/L1.2, so the risk is a device that misbehaves there. The BCM4388 and the GL9755 both advertise support, and neither showed a problem.
  • Everything else: unchanged.

AI assistance

  • I used an AI tool or agent to write or draft part of this change: Claude (claude-opus-5-5), via Claude Code. It found the disabled substates, measured them, wrote the patch and this text, with me running the machine and reviewing.
  • A human has read the whole diff and takes responsibility for it.

🤖 Generated with Claude Code

@scottjones scottjones left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for this, and for the careful measurements. The approach looks right to me, and the locking and the D0 step match pcie-qcom. A few things before it goes in:

  1. pci_enable_link_state_locked() replaces the link's defaults rather than adding to them (aspm.c sets aspm_default and clkpm_default from the mask). Without PCIE_LINK_STATE_L0S in the mask, L0s goes off on any link that supports it, since the core enables L0s and L1 by default on DT platforms. Without PCIE_LINK_STATE_CLKPM, Clock PM goes off if iBoot had set CLKREQ_EN. Please add L0S, which is safe because it's masked by what the link supports. For Clock PM, could you check today's lspci -vv LnkCtl ClockPM line and decide from that? Upstream deliberately stopped forcing Clock PM on DT platforms in df5192d, so I'd rather not pass PCIE_LINK_STATE_ALL blindly.
  2. The 0.35 W figure came from pcie_aspm.policy=powersupersave, which also turns on L0s and Clock PM. Could you re-measure on the patched kernel with no pcie_aspm parameter, so the number matches what ships?
  3. The match is on apple,t6030, which the J516S also carries (t6030-j516s.dts), so the 16" gets this too. Either match apple,j514s only or update the description. Its Wi-Fi and SD setup is the same, so I don't expect trouble.

A minor note, no change needed: after a remove and rescan, the substates stay off until reboot. A line in the commit message would cover it.

On a J514S (MacBook Pro 14", M3 Pro), both ends of the Wi-Fi/Bluetooth
link (BCM4388) and of the SD reader link (GL9755) advertise ASPM and
PCI-PM L1.1 and L1.2, as do their root ports, but the links only ever
enter L1: the boot loader leaves the substates off, and the default ASPM
policy keeps what it found.

Enable L1 and its substates on the links of a non-tunneled controller of
a listed machine once its bus is scanned, as pcie-qcom does.
pci_enable_link_state_locked() replaces the link's ASPM and Clock PM
defaults, so the mask keeps L0s (masked by what the link supports; these
root ports advertise L1 only), and keeps Clock PM only where the ASPM
core left CLKREQ# enabled: it is not forced on DT platforms. On the J514S
Clock PM stays off.

With this patch and no pcie_aspm parameter, display-off idle power was
3.06 W with the substates on, 3.36 W after turning them off through the
link sysfs attributes and 3.02 W after turning them back on (medians of
120 one-second samples of the SMC's total system power): about 0.3 W.
Ping to the router stayed at about 3 ms with no sample above 5 ms, and a
LAN transfer ran at 315-331 Mbit/s against 308-334 Mbit/s without them.

Only the J514S, where this was measured, is listed. The J516S (16" M3
Pro) has the same T6030 links but has not been measured. Tunneled ports
are never touched. The substates are set at probe only: after a remove
and rescan of a device they stay off until reboot.

Assisted-by: Claude:claude-opus-5-5
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Ilyas Akin <ashnwor@gmail.com>
@ilyasakin ilyasakin changed the title PCI: apple: Allow L1 substates on T6030 internal links PCI: apple: Allow L1 substates on J514S internal links Nov 16, 2026
@ilyasakin

Copy link
Copy Markdown
Author

Thanks. I've force-pushed changes for all three points:

  1. Mask. It now includes L0S. These root ports only advertise L1, so on the J514S this changes nothing, but the default no longer drops it. I didn't pass Clock PM blindly. The walk adds PCIE_LINK_STATE_CLKPM only when the device's LnkCtl still has CLKREQ_EN set at the time the walk runs, so whatever the ASPM core kept from the boot loader stays. With that change on the J514S, link/clkpm reads 0 on all three functions. The endpoints support Clock PM and the root ports don't, so iBoot leaves it off and it stays off.

  2. Measurement. I re-measured on the patched kernel with no pcie_aspm parameter, switching the substates through the link/ sysfs attributes: on, then off, then on again. Display off, 120 one-second samples per phase:

    • substates on: 3.06 W
    • substates off: 3.36 W
    • on again: 3.02 W

    That's about 0.3 W. The commit text and the code comment now say 0.3 W.

  3. Match. It's apple,j514s only now. The J516S is noted as having the same links but not measured, and the title says J514S.

I also added the remove/rescan note to the commit message.

@scottjones scottjones left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for turning this around so quickly. All three points are covered: L0s is in the mask, Clock PM is no longer forced and only follows what CLKREQ_EN already says, and the match is J514S only. Thanks too for re-measuring without pcie_aspm, so the 0.3 W is what actually ships. Approving.

One comment, no change needed for this PR. Clock PM is decided per function here, but the ASPM core takes the worst case across all functions on the link (pcie_clkpm_cap_init()). The walk calls pci_enable_link_state_locked() once per function and the last call wins, so whichever function is visited last decides Clock PM for the whole link. If one function had CLKREQ_EN set and the other didn't, this would turn Clock PM on where the core left it off. On J514S clkpm reads 0 on every function, so it can't happen there, but it does mean "only where the ASPM core left CLKREQ# on" isn't strictly true in that mixed case. If the list grows past J514S, it might be worth ANDing CLKREQ_EN across pdev->bus->devices first.

@ilyasakin

Copy link
Copy Markdown
Author

For the record, two things a later upstream reviewer would likely ask about.

What macOS does on these links. From IORegistry on a J514S running macOS:

  • Wi-Fi/BT root port: pci-aspm-default = 2 and pci-l1pm-control = 0x4055190f. That's all four L1 substates enabled (LTR threshold about 87 µs), so on this link we match Apple's setting, and CLKREQ# is wired.
  • SD reader root port: pci-aspm-default = 0. macOS keeps ASPM off there, presumably because it powers the idle reader down entirely. Linux keeps the GL9755 in D0 with runtime PM off.

Where the saving comes from, toggling per link at runtime on the patched kernel, display off:

Power
both links 2.46 W / 2.41 W
Wi-Fi link only 2.68 W
neither 2.78 W

So about 0.10 W comes from the Wi-Fi link and about 0.24 W from the SD reader link.

SD card I/O with the substates on, off, on (UHS-I SDR104 card, 1 GiB O_DIRECT write and read, 20 s of 4k QD1 random reads):

  • SHA-256 matched every round;
  • no PCIe, AER, sdhci or mmc messages;
  • removing and reinserting the card is detected.

The cost is in QD1 reads: about +30 µs at p50, 2861/2397 IOPS on against 3078 off. Sequential throughput was within the card's own variation.

@ilyasakin

Copy link
Copy Markdown
Author

Thanks for the review. Good point on Clock PM: the last function visited sets it for the whole link. If the list grows past the J514S, I'll AND CLKREQ_EN across pdev->bus->devices first, as you suggest.

@c4software

Copy link
Copy Markdown

Tested on a second J514S. On 01:00.0, 01:00.1 and 02:00.0: l1_aspm=1 l1_1_aspm=1 l1_2_aspm=1 l1_1_pcipm=1 l1_2_pcipm=1, clkpm=0, policy default. Wi-Fi, Bluetooth and the SD reader all work, and Wi-Fi came back after each of the three suspends.

Not tested: power on its own, and battery runtime.

Test setup

Testing and write-up assisted by Claude Code.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants