Skip to content

Apple M6 (T8152) PCIe-C tunnel on the rear port (on top of #191 and the USB4 PR) - #198

Open
joshuaswarren wants to merge 20 commits into
omacom:integration/aurorafrom
joshuaswarren:m6-pcie
Open

joshuaswarren wants to merge 20 commits into
omacom:integration/aurorafrom
joshuaswarren:m6-pcie

Conversation

@joshuaswarren

@joshuaswarren joshuaswarren commented Oct 23, 2026 •

Copy link
Copy Markdown

Summary

The PCIe-C tunnel of the Apple M6 (T8152) Mac mini, rear USB-C port (ACIO3): the binding, the start order and the shifted reset registers in pcie-apple, and the tunnel device tree. Three commits.

The first fourteen commits of this branch are #191 (the M6 base, ten commits) and #197 (the USB4 PR, four commits). Only the last three commits are new here: the diff shows the rest too until those merge, then I rebase. Review the last three.

It is not needed for the release candidate, and there is no rush to review or merge it.

Commits

# Commit Notes
15 dt-bindings: pci: apple,pcie: add apple,t8152-pciec and preinit-status its own commit; apple,t8103-pciec fallback; documents apple,pciec-preinit-status, which the driver already read
16 PCI: apple: add the T8152 PCIe-C tunnel port start order and reset registers new t8152_pciec_hw with a cold start order and a tunnel register delta of 0x700
17 arm64: dts: apple: t8152-j873g: add the ACIO3 PCIe-C tunnel usb4-pcie-tunnel@0 with an identity ranges; the LTSSM window is from the ADT register list of the controller

Measured, on an M6 Mac mini (an older tree, the same logic in a copy of the driver)

  • Three boots with a Thunderbolt 3 dock on ACIO3: the tunnel link comes up on the first attempt, the tunneled root port (106b:101b) enumerates, and the dock's two Intel bridges (8086:15da) appear behind it. Before the two driver changes the host bridge hung the machine, and then probed but never linked.

Not measured

  • This branch on hardware.
  • An endpoint behind the dock: the dock's slot link did not train with either of two cards, also on other hosts.
  • Suspend and cable removal with the shifted registers; MSI delivery.
  • The kernel's own make dtbs_check / dt_binding_check (a hand-run dt-validate of the board file is in the first comment).

Not included

  • The native PCIe ports 0 and 2 of the same machine (Apple N1 Wi-Fi and Bluetooth, an RTL8127 10 GbE card) need a PIODMA change for their config space. It is not included, and neither are their device tree nodes. 10 GbE on port 2 was measured working (a 962 MB download at a negotiated 10 Gb/s) on a different tree with that change; it is not a result of this PR.

Receipts

arm64 clang, W=1, 16K pages, PCIE_APPLE=m, USB4_APPLE_SOC=m: drivers/pci/controller/pcie-apple.o, drivers/thunderbolt/apple.o, the eight objects of #191 and t8152-j873g.dtb, all rc 0, 0 warnings. check-commits: Signed-off-by on all 17 commits; no checkpatch errors (warnings: MAINTAINERS notice).

The T8152 GPIO block has the same layout as the T8140 one. Match
"apple,t8152-pinctrl" with the T8140 hardware data.

Signed-off-by: Joshua Warren <816217+joshuaswarren@users.noreply.github.com>
T8152 has a SART with a new register layout: the per-entry config,
physical address and size registers sit at 0x00, 0x60 and 0xc0. Add the
operations for it and match "apple,t8152-sart". The binding lists the
new compatible as a stand-alone entry.

Measured on an M6 Mac mini with an earlier form of this change: the
machine boots and runs its root file system from NVMe, which uses this
SART. This version is compile-checked only.

Signed-off-by: Joshua Warren <816217+joshuaswarren@users.noreply.github.com>
On T8152 the queue setup registers (admin queue, IO queues, the linear
submission queue control) live at different offsets, and a few of them
accept writes but read back zero. Add a register translation for that
layout, skip the read-back checks that cannot work, and describe the
controller with a new hardware data block matched by
"apple,t8152-nvme-ans2".

The ANS is a virtual PMGR device on this SoC and has no reset line, so
the hardware data block carries a no_reset_control flag. Only T8152 skips
the reset lookup. Every other SoC still requires a reset control and
fails the probe without one, as before.

The binding gets the new compatible. On T8152 the resets, power-domains
and power-domain-names properties are not required, and the controller
has two register regions (the NVMMU registers live in the first one).

Measured on an M6 Mac mini with an earlier form of this change: the
machine boots and runs its root file system from NVMe. This version is
compile-checked only.

Signed-off-by: Joshua Warren <816217+joshuaswarren@users.noreply.github.com>
@joshuaswarren

Copy link
Copy Markdown
Author

Receipts at head 21ef6dfee4b83fc540107bb320bac8139ff19453:

  • Compile: arm64 clang W=1, 16K pages, USB4_APPLE_SOC=m, PCIE_APPLE=m: the eight objects of Apple M6 (T8152) Mac mini base: PMGR, USB2, DART, NVMe, board fragments #191, drivers/thunderbolt/apple.o, drivers/pci/controller/pcie-apple.o and apple/t8152-j873g.dtb, all rc 0, 0 warnings (offload-build, checkH.log).
  • dtcheck.sh (dtschema 2026.9, by hand) on t8152-j873g.dts: cpp rc 0, dtc -@ rc 0 with 0 warning lines, dt-validate 0 lines. The 72 binding examples print one message, display/apple,dcp-typec-routes.yaml, which is in the baseline on any tree.
  • check-commits: Signed-off-by on all commits; no checkpatch errors.
  • Not run: a boot, the kernel's own make dtbs_check / dt_binding_check.

@greptile-apps

greptile-apps Bot commented Oct 23, 2026 •

Copy link
Copy Markdown

RetriggerView in GreptileConfidence Score: 5/5

[Medium impact] No new blocking issue was established; the PR appears safe to merge within its stated bring-up limits.

Summary

Adds the T8152 rear-port PCIe-C tunnel, its binding, cold-start order, and shifted reset registers. The branch also carries the M6 base and USB4 support.

  • The rear ACIO3 port can connect PCIe devices through a T8152 tunnel.
  • ACIO3 runs its USB4 router and sends tunneled PCIe traffic through a T8152 DART.
  • T8152 USB-C ports can run fixed USB2 host or device connections.
  • T8152 power domains and platform devices are described for Linux.

Diagram

%%{init: {'theme': 'neutral'}}%%
flowchart TD
  A[ACIO checks tunnel support] --> B[T8152 cold-start path accepted]
  B --> C[Prepare port and start APPCLK]
  C --> D[Create DART and PCIe children]
  D --> E[Initialize DART on first use]
  D --> F[Probe PCIe host]
  E --> G[Start tunneled link]
  F --> G
Loading

Reviews (7) · Last reviewed commit: "arm64: dts: apple: t8152-j873g: add the ..." · Reviewed by Greptile

@joshuaswarren

Copy link
Copy Markdown
Author

GREEN at 21ef6df for the three new commits (9455ab1 binding, ee003c0 PCI driver, 21ef6df dts). The four ACIO commits below them are #197 a70664c, where I posted my review.

Read: the three diffs and pcie-apple.c at this head (tunnel release_reset, reinitialize, stop, preinit parsing around line 1700).

Checked:

  • ee003c0: PORT_TUNCTRL and PORT_TUNSTAT become function-like macros over hw->tun_delta; 0x988 + 0x700 = 0x1088 and 0x98c + 0x700 = 0x108c as the comment says, and every use is converted (a bare use would not compile). Other hardware has tun_delta 0, so offsets do not move for them. cold_order is read only in apple_pcie_tunnel_reinitialize: PERST release and port clock enable, wait for READY, then clear the clock-gate disable, apply the rc tunable (first config-space access), counters, INTSTAT/LINKCMDSTS clears, intr2axi pulse, then the common link-idle wait. The other path clears both bits first, applies the tunable, then sets them. That matches the stated reason (clock running with READY before the first config access). The goto only skips the code after it in the same function.
  • Binding: compatible entry and apple,pciec-preinit-status are consistent with the driver, which already reads the property.
  • dts: the tunnel node's identity window (0 to 0x2000000000) contains the config space at 0x1c80000000 and the DART at 0x414000000. The DART's doorbell reg is the same page as the ACIO one (0x200000280), safe because the driver does not claim it. iommu-map covers 0x40 RIDs with mask 0x3f, equal to max_rid2sid 64. The three PCI ranges tile without overlap: 64-bit prefetch 0x1400000000 + 8 GiB, then 0x1600100000 + 0x3ff00000 meeting 0x1640000000 + 0x40000000.

Findings:

  1. (should fix, small) apple,pciec-preinit-status = <1> states a loader handoff that the node's own comment says does not exist ("m1n1 does not hand off a PCIe-C init"). It is there only to pass the check at pcie-apple.c:1710-1723, and the binding text says 1 means the loader left the port initialized. The new hw flag already says what is true for this part; let the check accept hw->cold_order, and drop the property from the dts.
  2. (nit) The comment on apciec3 says the t8103 fallback binds t8103_pciec_hw. With "apple,t8152-pciec" first in the compatible list, the new t8152_pciec_hw entry matches, so the comment is stale. That entry still uses the t8103 MSIADDR/PERST/RID2SID offsets; the comment calls them unverified, fine.
  3. (nit) msi-ranges 1975 x32 is described as the next free slice of the shared 1879 block, but no apcie node exists in this tree to check against. The comment says unverified, and the PR says MSI is not needed to enumerate; fine as is.
  4. (nit) This dtsi has no note that the loader fills the apple,tunable-* properties of the PCIe node (the USB4 dtsi says so for ACIO).

Not checked: addresses, interrupts and ranges against the ADT; compile and dt check receipts are the author's, not reproduced; nothing run on hardware.

@joshuaswarren

Copy link
Copy Markdown
Author

New head 92276804bcef2f01855d8a2c0be29e1d86b8b7aa, three new commits on top of #197's updated head df61647fa3fd. Per the review at 21ef6dfe (w7Q):

  • Should-fix, apple,pciec-preinit-status. pcie-apple.c now has preinit_ok = (!ret && preinit_status == 1) || pcie->hw->cold_order, with a comment that T8152 has no loader handoff. The property is gone from the dtsi. It stays documented in the binding, because other SoCs use it, and the binding commit text no longer says the T8152 tree sets it. The driver commit text says why.
  • Nits: the stale t8103-fallback comment is replaced (the driver selects the T8152 table from the first compatible). The MSI slice is called UNVERIFIED in the dtsi comment and the commit text. The dtsi header and the commit text say the loader supplies the tunables.
  • Hardware caveat for the should-fix: the lab driver ran with apple,pciec-preinit-status = <1>. The no-property path was not booted. It reaches the same preinit_ok true branch, but through a different device tree.

Receipts at this head: arm64 clang W=1, 16K pages, PCIE_APPLE=m, USB4_APPLE_SOC=m: pcie-apple.o, thunderbolt/apple.o, the eight objects of #191 and t8152-j873g.dtb, all rc 0, 0 warnings. dtcheck.sh: dtc -@ 0 warning lines, dt-validate of the board file 0 lines, the 72 examples print only the baseline message. Not run: a boot.

@joshuaswarren

Copy link
Copy Markdown
Author

GREEN at 9227680

Read: the seven commits above the #191 head against what I reviewed (the four ACIO commits at a70664c in #197, the three PCIe-C commits at 21ef6df here), compared as diffs with the hash lines removed. The four ACIO commits are the same as #197 df61647 (comment changes only). The PCI binding commit is identical.

Resolved from my review:

  • Item 1: pcie-apple.c now has preinit_ok = (!ret && preinit_status == 1) || pcie->hw->cold_order, with the comment saying T8152 has no handoff. This gives the same result on T8152 as the old property = <1>, so behaviour is unchanged. The dtsi no longer carries the property. I checked the other readers of that property: apple_pcie_tunnel_needs_cold_init() returns false first on the compatible check ("apple,t6000-pciec"), so its result on this node is the same with or without the property.
  • Items 2 to 4: the stale t8103-fallback comment, the MSI slice comment (now says unverified and unmeasured) and the loader-tunables note are fixed in the dtsi.

Not run: build. Compile and dt check receipts are the author's, not reproduced; addresses are not checked against the ADT. Merge still needs CI and a second non-author GREEN.

Comment thread drivers/usb/dwc3/dwc3-apple.c Resolved
@joshuaswarren

Copy link
Copy Markdown
Author

GREEN at 9227680

Read: the three commits above the #197 head (57e9e19 binding, c8df94c PCI driver, 9227680 dts); drivers/pci/controller/pcie-apple.c and t8152-j873g-usb4-pcie.dtsi at this head. The four ACIO commits below them are #197 df61647f, reviewed there.

Checked

  • Addresses and numbers against the M6 ADT dump (apciec3, dart-apciec3):
    • reg entries 0 to 6 match in order: config 0x1c80000000 (0x10000000), rc 0x214034000, port0 0x214020000 (0x8000), fabric 0x214038000, debug 0x214060000, ltssm 0x210020000, intr2axi 0x21402c000, each plus 0x200000000 except the config space. The eighth ADT range (0x214028000) is not used, as the comment says.
    • Interrupt 1822, msi-vector-offset 1879, power slot 89 (second power-gates entry).
    • The three PCI ranges equal the ADT tuples: 0x43000000 to 0x1400000000 (8 GiB), 0x02000000 0x100000 to 0x1600100000 (0x3ff00000), 0x42000000 0x40000000 to 0x1640000000 (0x40000000).
    • DART: reg 0x214000000, interrupt 1823, 256 SIDs, vm-base 0x10080000000, vm-size 0x7ff00000, piodma SID 254, and all 7 apple,tunable tuples equal dart-tunables-instance-0.
  • dt check (dtschema 2026.9) on this head's board file: dtc -@ 0 warning lines, dt-validate 0 lines, 72 examples with only the baseline message. dtc with the W=1 warning set (unit_address_vs_reg, simple_bus_reg, pci_device_reg, pci_device_bus_num, unique_unit_address, interrupt_provider) reports nothing for the new nodes.
  • Driver.
    • PORT_TUNCTRL and PORT_TUNSTAT are function-like macros over hw->tun_delta; every use (release reset, the status print, the PERST-on and ACK-request sequence, apple_pcie_reset_for_resume) passes the port, and tun_delta is 0 for every other hardware table.
    • preinit_ok = (!ret && preinit_status == 1) || hw->cold_order: on T8152 a port that is not READY after the 250 ms poll goes to apple_pcie_tunnel_reinitialize(), and with cold_order that function runs PERST off, APPCLK on, wait for READY, clear CGDIS, apply the rc tunable, then the link-idle wait. A port that is already READY skips the replay, as for the other tunneled controllers.
  • Binding. The apple,t8152-pciec entry with the t8103 fallback is the same shape as t8112; dt-validate accepts the seven named reg entries.

Findings (not blocking)

  1. The cold_order branch is a third copy of the start sequence in apple_pcie_tunnel_cold_init(); the copies differ only in where the READY wait and the CGDIS clear sit. One table-driven order would keep one copy.
  2. The link-idle failure message in apple_pcie_tunnel_reinitialize() says "after hot reconnect", and the cold-order path reaches it too.
  3. The dtsi comments still use "PROVEN" and "UNVERIFIED" as tags (reg and ranges comments, MSI slice). State the fact or drop the tag.

Not checked

Compile receipts, checkpatch beyond the UNDOCUMENTED_DT_STRING warning on apple,t8152-pciec (documented in the first commit of this PR), any hardware behavior. The author's note that the path without apple,pciec-preinit-status was not booted stands. Merge still needs CI and the owner's word.

@joshuaswarren

Copy link
Copy Markdown
Author

New head fb0d0816bf2493a0f16ce63252d103c05afda686 (rebased on the same base as the other M6 PRs, plus the fixes for the Greptile and reviewer findings; every Greptile thread of the previous heads has a reply with the commit and is resolved).

Receipts at this head (CT-local, arm64 clang W=1, 16K pages, PCIE_APPLE=m, USB4_APPLE_SOC=m): pcie-apple.o, thunderbolt/apple.o, the eight objects of #191 and t8152-j873g.dtb, rc 0, 0 warnings. dtcheck.sh (dtschema 2026.9) on the board file: dtc -@ 0 warning lines, dt-validate 0 lines, the binding examples print only the baseline message of display/apple,dcp-typec-routes.yaml. Changes since the last GREEN: a failing phy_init() or phy_power_on() in dwc3_apple_core_start() now goes through dwc3_apple_core_stop(); the PSD parents (ps_atcN_usb_aon, ps_atcN_usb); iommus exactly four; the host-only lists skipped for a fixed peripheral; the Gen3 doorbell guarded for non-arm64 builds; a locked Gen3 DART no longer takes the range flush through TLB_CMD; the PSD group power-off restores the stopped slots. Not run: a boot, the kernel's own make dtbs_check.

Comment thread drivers/pci/controller/pcie-apple.c Resolved
Comment thread drivers/iommu/apple-dart.c Resolved
@joshuaswarren

Copy link
Copy Markdown
Author

Rebased on the new #197 head dfde2fd2c42f8e5d55b78661223d1876b3546a33 (the A series gained a mailbox binding commit); new head f877c81de56bce49ea75029e825c6204cf230ec7, the four commits of this PR unchanged. Receipts: arm64 clang W=1, 16K pages: the eight objects of #191, thunderbolt/apple.o, mailbox.o, pcie-apple.o and t8152-j873g.dtb, all rc 0, 0 warnings; dtcheck.sh board file 0 messages. Not run: a boot.

@joshuaswarren

Copy link
Copy Markdown
Author

GREEN at f877c81

Read: the nineteen commits against the ones I reviewed at 9227680, compared per commit as diffs with the hash lines removed.

  • The ten base commits are the Apple M6 (T8152) Mac mini base: PMGR, USB2, DART, NVMe, board fragments #191 head b8943e8; the ACIO start, NHI pdf, ACIO binding, mailbox binding and ACIO3 dts commits are the ones I cleared in Apple M6 (T8152) USB4/Thunderbolt host router, rear port ACIO3 (on top of #191) #197 at dfde2fd (see that review). The PCI binding commit is identical.
  • New commit, iommu/apple-dart: a DART is recognised as tunneled when its parent also holds a child with an apple,t8103-pciec or apple,t6000-pciec compatible (the old "usb4-pcie-tunnel-" name test is kept as a second condition, so existing trees behave as before). The loop does not break early, so no child reference leaks; the parent reference is dropped.
  • PCI driver: the walker that finds the tunnel's DARTs now also accepts apple,t8152-dart; the rest is hunk offsets.
  • PCIe-C dts: apple,dma-range is now the endpoint-visible window 0x80000000 + 0x7ff00000 (top MiB left for the MSI doorbell at 0xfffff000) with apple,dma-offset 0x10000000000 for the 1 TiB alias; the driver already applies dma_offset in its iova math (lines around 1538, 1725, 1750, 1863 of apple-dart.c at this head). Comments cleaned.
    Not run: build; the author's compile and dt receipts are not reproduced; addresses are not checked against the ADT. Merge still needs CI and a second non-author GREEN.

@joshuaswarren
joshuaswarren force-pushed the m6-pcie branch 2 times, most recently from 00beab3 to 0db54fd Compare October 23, 2026 23:14
Comment thread drivers/pmdomain/apple/pmgr-psd.c Resolved
On T8152 the ATC USB blocks (and the GPU) are behind a per-slot service
request in the power state director, on top of the normal PMGR power
states. Expose each slot, or a group of slots that must come up together,
as a generic power domain. Power-on writes the slot request and waits for
the acknowledge; power-off requests state 6 in reverse order.

Add the devicetree binding.

When a power-on request times out, the slot that timed out is put back to idle with the slots before it, because its acknowledge was not seen but it may still start.

When a group power-off fails, every slot is put back to its active state, including a slot whose off request timed out and may still finish stopping, because genpd keeps the domain marked on after a failed power-off and would skip power_on at the next resume.

Signed-off-by: Joshua Warren <816217+joshuaswarren@users.noreply.github.com>
The fixed host ports on T8152 need only the USB2 part of the PHY. Add a
driver that applies the bootloader-supplied calibration, handles host and
device mode, and leaves the USB3 and USB4 lanes alone.

Add the devicetree binding.

Signed-off-by: Joshua Warren <816217+joshuaswarren@users.noreply.github.com>
A hub with the Apple ID 05ac:800f sits behind the front ports of the M6
Mac mini. Add that ID to the hub quirk table with
HUB_QUIRK_DISABLE_AUTOSUSPEND so the kernel keeps it powered.

The match is on the vendor and product ID alone. Every hub with this ID
loses autosuspend, not only the one on this machine. The constant is
named after the ID, USB_PRODUCT_APPLE_HUB_800F, because the ID is
Apple's and the part behind it was not identified from the descriptor.

This quirk was not measured on its own: it only keeps the hub awake.

Signed-off-by: Joshua Warren <816217+joshuaswarren@users.noreply.github.com>
Add a peripherals fragment to the M6 Mac mini board file: the always-on
GPIO controller with the reset hog for the front-port USB hub, and the
four PSD service slots for the USB ports.

The NVMe controller, its mailbox and its address filter are not described
here. The NVMe node needs a mailbox driver change that is not in this
tree yet.

The fragment holds only addresses, interrupt numbers and wiring. The boot
loader adds the register tunables that some blocks need. The PMGR power
domains are not described yet, so nodes that need them are left out.

Signed-off-by: Joshua Warren <816217+joshuaswarren@users.noreply.github.com>
Add the DART of the Apple M6 (T8152), a third-generation DART whose
topology is fixed by the hardware. Its streams are described by static
parameters instead of being probed, and it takes no MMIO access until the
first use, because the block can be power gated behind the ATC USB
domains.

The binding requires a second register region (the TLB invalidate
doorbell) for the T8152 compatible only.

The change adds:
- the apple,t8152-dart compatible and its hardware description;
- static topology and "first use" initialisation of every DART;
- the Gen3 TLB invalidate and the fault handler;
- the reset path for DARTs that the boot loader leaves locked.

A DART that the boot loader leaves locked is held powered for its
lifetime. That applies to Gen3 hardware only. The other generations keep
their behaviour. The first-use initialisation re-reads the parameters and fails if the
page size differs from the one assumed at probe. It runs under a spinlock, and
every function it calls is non-sleeping (register accesses and an atomic
poll).

Measured on an M6 Mac mini with the earlier form of this change: the
display controller DART attaches locked on every boot. This version is
rebased on current integration/aurora and was compile-checked only. The
USB DART paths need the T8152 PSD power domains and were not run on
their own.

The doorbell store and the TLBI of the Gen3 invalidate are arm64 instructions: they are built only with CONFIG_ARM64, and other architectures (COMPILE_TEST) get a stub that returns -EOPNOTSUPP. A locked Gen3 DART no longer takes the range flush through TLB_CMD, which Gen3 does not accept; it uses the full-stream Gen3 callback.

Gen3 does not take commands through TLB_CMD, so the unlock after a lock-on-error exception is not sent there: the DART logs once that this is not supported. The stale-exception drain of a reset does not read 0x704 on Gen3, like the fault handler.

Signed-off-by: Joshua Warren <816217+joshuaswarren@users.noreply.github.com>
Add the apple,t8152-dwc3 compatible to the Apple dwc3 glue. The ATC ports
of the M6 (T8152) controller never switch roles through the Type-C
controller. The glue starts the controller as a fixed host, with or
without a hub child, or as a fixed peripheral when the node sets
dr_mode = "peripheral".

The T8152 profile is USB2 only: it needs maximum-speed = "high-speed",
takes no reset line, and its USB3 PHY is optional. The calibration lists
come from the device tree (apple,tunable-* properties that the boot loader
fills in at runtime; the binding does not require them in the board file) and are applied to the register windows in a fixed order after
the core is reset. The pipe handler is set up for the selected role.

The apple,t8103-dwc3 paths keep their behaviour. The controller stays a
syscore device while it runs, as before: no separate suspend or resume
path is added for T8152.

The earlier form of this change ran on an M6 Mac mini USB2 front port.
This version is rebased on current integration/aurora and was
compile-checked only. SuperSpeed does not train yet and is not part of
this change.

A fixed peripheral does not require the host-only calibration lists at probe, since it never applies them. The binding lists exactly four iommus entries, two stream IDs on each of two DARTs.

If the T8152 USB2 PHY fails to initialise or power on during core start, the failure goes through the same shutdown as a failed core init, so the reset and syscore flags set earlier are cleared again.

Signed-off-by: Joshua Warren <816217+joshuaswarren@users.noreply.github.com>
Describe the power management blocks and the USB controllers of the Mac
mini (M6, J873) next to the peripherals fragment:
- the two PMGR blocks and their power domains, in t8152-j873-pmgr.dtsi;
- for each of the four USB ports (usb0, usb1, usb2 and usb3 of the SoC):
  two DART instances, the USB2 PHY and the dwc3 controller.

The register windows, interrupts and DART tunables are the values of the
Apple device tree of the machine. The boot loader adds the apple,tunable-*
lists of the USB2 PHY and the controller at runtime.

The bindings list the apple,t8152-pmgr and apple,t8152-pmgr-pwrstate
compatibles, and the dwc3 and USB2 PHY bindings take the tunable lists as
optional, because the boot loader adds them.

The fragment is a plain description of the hardware. The ports that the
change describes run USB2 only: SuperSpeed and USB4 are not part of it.

This commit compiles to a device tree blob with dtc without warnings. It
was not booted. The bring-up it comes from enumerated the front-port USB2
hub and a USB disk at 480 Mb/s.

The four PSD service slots get their PMGR domains (the aon and the port domain of their port) as parents, so genpd powers them before the slot request.

Signed-off-by: Joshua Warren <816217+joshuaswarren@users.noreply.github.com>
@joshuaswarren

Copy link
Copy Markdown
Author

NOT GREEN at 0db54fd

The four PCIe-C commits are fine; the PR is blocked by the mailbox compatible in the #197 commits below it (details in my comment on #197): apple,t8152-acio-mailbox falls back to apple,asc-mailbox-v4, which selects the classic ASC register layout in mailbox.c while the node describes a counted-FIFO block with a 0x80-byte window. This PR passes that node on to the board, so the finding applies here until #197 is fixed.

Checked in the four PCIe-C commits

  • Binding. apple,t8152-pciec with the apple,t8103-pciec fallback and apple,pciec-preinit-status; unchanged since 92276804.
  • DART tunnel detection (apple_dart_is_tunneled). A DART whose parent also holds an apple,t8103-pciec or apple,t6000-pciec child is tunneled; the old name-prefix test stays as a second test. I checked the shared-parent risk: in every device tree of this tree that has a PCIe-C node (t8103, t8112, t600x, t602x, t6030, t8152) the PCIe-C node sits in a usb4-pcie-tunnel-N container together with its own DART, so no ordinary DART shares a parent with one and no existing device tree changes behavior.
  • PCI driver.
    • apple_pcie_tunnel_prepare() now also selects a port whose hardware table has cold_order (apple_pcie_tunnel_has_cold_order() through of_match_node), so the T8152 tunnel clocks the port before its DART child probes. The commit text says this path is not measured on hardware.
    • In apple_pcie_tunnel_cold_init() the cold_order ports wait for READY, clear the clock-gate disable, then apply the root-complex tunables, the same order as the replay path; the late CGDIS clear is skipped for them. T8103 and T600x ports are unchanged.
    • preinit_ok = (...) || hw->cold_order, PORT_TUNCTRL and PORT_TUNSTAT with tun_delta, and the apple,t8152-dart visit in the walker are as before.
  • dt check (dtschema 2026.9) on this PR's tree: dtc -@ 0 warning lines, dt-validate of the board file 0 lines, 73 examples with only the baseline message. Every address, interrupt, range and DART tunable of the PCIe-C node and its DART equals the ADT dump (as in my earlier review).

Not checked

Compile receipts (the author's), hardware (the prepare path is stated as unmeasured), and any T6000 or T8103 behavior beyond reading the device trees.

@joshuaswarren

Copy link
Copy Markdown
Author

New head 790270410265ca05665b409bb8f0f68fd3f14352: rebased on #197 at bdb5d361; its four commits are unchanged. It carries the ACIO node of #197 and so waits for the same mailbox fix.

On T8152 the macOS driver starts the ACIO CPU through the coprocessor
wrapper block (the "cpu" region, acio-cpu reg[1]): write 0x10 to
+0x28, idle state is +0x44 bit 1. That is the block t6030 already
uses, so add a T8152 entry to the per-SoC table with mxwrap set and
no rc M3_CTRL start (whether T8152 has that bit at all is not
decoded). The apple,t8152-usb4-acio compatible selects it.

T8152 never shows M3_STAT ready, so this entry does not wait for it (a
500 ms poll would time out on every start): the RTKit boot of the
co-processor is the readiness check instead.

Measured on an Apple M6 (T8152) Mac mini, rear USB-C port (ACIO3),
with this start sequence in an older tree: M3_STAT never becomes
ready, the NHI answers the first HOP_COUNT read (12 hops), and the
USB4 domain plus host router enumerate. Not measured: this patch on
this base (compiled only), the non-posted mapping of the "cpu" region
on T8152 (the driver maps it non-posted as for t6030), and any other
SoC; their table entries and code paths are unchanged.

Signed-off-by: Joshua Warren <816217+joshuaswarren@users.noreply.github.com>
T8152 has no "pdf" range: the ACIO+0xdb0000 block that M1/M2 device
trees map there aborts on a read, and macOS never writes the RX ring
options+4 word either. Treat the range as optional: map it only when
the node describes it, and when it is absent program the ring options
without the RX options+4 and PDF writes.

Measured on an Apple M6 (T8152) Mac mini, rear USB-C port (ACIO3): a
read of the M1/M2 pdf offset takes a synchronous external abort; with
the ring options programmed and both the options+4 and PDF writes
skipped, all rings start, the first TX and RX ring interrupts fire and
the USB4 domain enumerates. The location of the RX sof/eof mask on
T8152 is unknown; receive frames arrive without it. M1/M2 keep the
existing behaviour whenever the node carries the range.
A M1/M2 device tree that loses its "pdf" range now probes without the
PDF writes and without an error or a message.

Signed-off-by: Joshua Warren <816217+joshuaswarren@users.noreply.github.com>
Add the apple,t8152-usb4-acio and apple,t8152-usb4-nhi compatibles,
describe the co-processor wrapper ("cpu") register range of the acio
node, and make the NHI "pdf"
range optional: T8152 has no PDF range at all. T8152 binds one DART
stream per ring like t6030/t6031/t8122, so it takes the 24-iommu
case.

The "cpu" range is required for apple,t6030-usb4-acio and
apple,t8152-usb4-acio, whose driver does not probe without it, and is
not allowed for the other compatibles. The t6030 device trees already
carry it as a fourth range after the PCIe adapter, which the binding
did not allow; T8152 has no PCIe adapter range and lists rc, sram and
cpu.

Existing boards are unaffected: their reg entries stay valid as they
are, and they still require the reset. apple,t8152-usb4-acio does not
require it: its reset driver is separate work and the board tree does
not describe the reset yet.

Measured: the device tree built from these bindings enumerates the
USB4 host router on the Mac mini (M6, T8152) on an older tree. Not
measured: dt-schema validation of this patch.

Signed-off-by: Joshua Warren <816217+joshuaswarren@users.noreply.github.com>
The ACIO (USB4/Thunderbolt) co-processor of the T8152 talks to the
application processor through a mailbox with its own register layout:
a 0x80-byte window with the receive FIFO at +0, the transmit FIFO at
+0x10 and a status word at +0x20 that counts the receive occupancy and
the free transmit slots. That is neither the classic ASC layout of
apple,asc-mailbox-v4 nor the older M3 layout, so the compatible has no
fallback. Only two of its interrupts are known, the receive-not-empty
and the send-empty line, as for the ANE mailboxes; the interrupt
constraints are the ones of the ANE mailboxes.

The layout was read from the macOS driver and ran on hardware in an
older tree: it carried the RTKit boot of the co-processor. Not measured:
this binding on this base. Measured: the device tree that uses it passes
the schema check.

Signed-off-by: Joshua Warren <816217+joshuaswarren@users.noreply.github.com>
apple,t8152-acio-mailbox has a different register interface than the
ASC mailboxes: the status word at +0x20 holds the receive occupancy in
bits 3:0 and the number of free transmit slots in bits 11:8, the receive
FIFO is at +0, the transmit FIFO at +0x10, and each 128-bit message is
moved with one paired 64-bit access. Reading or writing the classic ASC
offsets (control at +0x110, send at +0x800, receive at +0x830) from this
window would touch unrelated registers of the wrapper block.

Add a counted_fifo flag to the hardware table, test "transmit full" and
"receive empty" through two helpers that honour it, and move the message
read and write into helpers that use one load or store pair for the
counted layout (arm64 only; the entry is refused elsewhere). Every
other hardware table keeps its flags unchanged and its two separate
64-bit accesses.

Measured here: arm64 compile. Run on hardware in an older tree: this
layout and the paired access carried the RTKit boot of the ACIO
co-processor. Not measured: this patch on this base, the send-empty
interrupt path of this mailbox.

Signed-off-by: Joshua Warren <816217+joshuaswarren@users.noreply.github.com>
Describe the free rear USB-C port (ATC3, ACIO3) of the Mac mini (M6,
T8152): four ATC3 power-state-director slots, the ACIO coprocessor
mailbox, the acio node with its rc, sram and coprocessor-wrapper
("cpu") registers, and the NHI with its 24 ring interrupts and all 24
ring DART stream ids bound (tx ring N = SID N, rx ring N = SID 16+N).
Behind the acio node sits its own Gen3 DART with a static topology.

The acio node carries no "pdf" range, no reset and no Type-C graph:
the ATC3 PHY CIO-mode link and the hpm3 wiring are separate work. The
PCIe-C tunnel container is not part of this patch.

Addresses are physical: ADT bus address + 0x200000000 for every ADT
address below 0x400000000. The boot loader fills the apple,tunable-*
properties and apple,thunderbolt-drom through the usb4-3 aliases.

Measured on the Mac mini (M6, T8152) with these nodes on an older
tree, plus the power-domain, DART, reset and mailbox support they
need: the ACIO firmware boots, the NHI answers, rings start, and the
USB4 domain and host router enumerate. Not measured: this patch on
this base (device tree compile only); ACIO0 and ACIO1 (NCM tether,
Studio Display) are not described and there is no ACIO2; the
interrupt order of the ring interrupts was confirmed for rings 0 of
both directions only.

Depends on support outside this patch: apple,t8152-dart (in this
tree), the apple,t8152-psd domains with the ps_atc3_usb_aon parent for
the three CIO slots (the parent is a guess), and a reset driver for the
CIO reconfigure pair. The mailbox compatible is described by the
previous patch. Without that reset the ACIO cannot be powered, so this
node describes hardware and does not boot it by itself; the binding does
not require resets for apple,t8152-usb4-acio until that driver exists.

Signed-off-by: Joshua Warren <816217+joshuaswarren@users.noreply.github.com>
@joshuaswarren

Copy link
Copy Markdown
Author

Review at 7902704: I withdraw my GREEN at f877c81 until the mailbox finding on #197 is fixed, since this PR carries the same ACIO and mailbox commits. The three PCIe-C commits of this PR are unchanged from what I cleared, and the base is the #191 head 9fbb356 (delta review posted there: PSD power-on rollback includes the failed slot, PSD power-off restores every slot, Gen3 DART drain and unlock changes). I will re-read the mailbox change by exact head when it lands.

@joshuaswarren

Copy link
Copy Markdown
Author

New head e21d7e6ff6fe93843505bd11032e79a2ed3c6f95: rebased on #197 at f40f1bb135828837d789875d8f930401e5c1eab5, its four commits unchanged. Receipts: the same objects plus pcie-apple.o, rc 0, 0 warnings; dtcheck board file 0 messages. Not run: a boot.

@joshuaswarren

Copy link
Copy Markdown
Author

GREEN at e21d7e6

Read: the twenty commits against the ones I reviewed at 7902704, compared per commit as diffs with the hash lines removed. The four PCIe-C commits of this PR (binding, tunneled-DART detection, PCI driver, PCIe-C dts) and the ACIO start, NHI pdf and ACIO binding commits are identical. The mailbox binding commit, the new counted-FIFO mailbox driver commit and the ACIO3 dts compatible are the ones I reviewed in #197 at f40f1bb (see that comment); the base is the #191 head 9fbb356. This restores my GREEN that I withdrew for the mailbox finding. Not run: build; the author's compile and dt receipts are not reproduced. Merge still needs CI and a second non-author GREEN.

@joshuaswarren

Copy link
Copy Markdown
Author

GREEN at e21d7e6

The four PCIe-C commits are identical to the ones I reviewed at 0db54fde (compared as diffs with the hash lines removed); the head changed because #197 below it now carries the mailbox fix (own compatible apple,t8152-acio-mailbox, counted-FIFO hardware table in mailbox.c, reviewed and cleared in #197 at f40f1bb1). The blocking finding of my comment at 0db54fde is therefore gone.

dtcheck.sh (dtschema 2026.9) on this head: dtc -@ 0 warning lines, dt-validate of the board file 0 lines, 72 binding examples with only the baseline message. The PCIe-C review items stand as written there: the DART tunnel detection against every PCIe-C container in the tree, the cold_order prepare and cold-init order, and the ADT comparison of every address, interrupt, range and DART tunable.

Not checked: compile receipts (the author's), hardware (the cold-start prepare path is stated as unmeasured). Merge still needs CI and the owner's word.

The PCIe-C tunnel port of the T8152 needs its own compatible: its
tunnel reset registers sit at different offsets than on t8103 and its
start order differs (see the driver patch that follows). Like the
t8112 entry it falls back to apple,t8103-pciec, so the register,
interrupt and iommu constraints of that compatible apply unchanged.

Also document apple,pciec-preinit-status, which the driver already reads
and which the boot loader sets on SoCs that have a handoff.

Measured: the schema check of the board tree with the tunnel node
(see the device tree patch). Not measured: the binding alone.

Signed-off-by: Joshua Warren <816217+joshuaswarren@users.noreply.github.com>
apple_dart_probe() decided that a DART belongs to a PCIe-C tunnel by
the prefix of its parent's node name ("usb4-pcie-tunnel-"). That
treats a tunneled DART as an ordinary one as soon as a board names the
container differently (a container with a non-empty ranges property
needs a unit address, and the schema checks of the T8152 tree
require one): the registers are then mapped posted and without the
non-posted ordering, and the first reset of the DART can end in an
asynchronous external abort.

Look for the PCIe-C root complex instead: a DART whose parent also
holds an apple,t8103-pciec or apple,t6000-pciec node (every PCIe-C
compatible in the tree falls back to one of the two) is tunneled. The
old name test stays as a second, additional test, so no existing device
tree changes its behaviour.

Measured: arm64 compile of apple-dart.c. Not measured: on hardware; the
existing tunnel containers (t8103, t8112, t600x, t602x, t6030) were
checked by reading the device trees, not by booting them.

Signed-off-by: Joshua Warren <816217+joshuaswarren@users.noreply.github.com>
…gisters

The PCIe-C port of the T8152 differs from the t8103 one in two ways
that matter for bringing up a tunneled root port:

 - The tunnel reset control and status registers sit 0x700 higher
   (+0x1088 and +0x108c instead of +0x988 and +0x98c). The t8103
   offsets read as zero on T8152, so releasing the tunnel reset
   through them has no effect and the link never trains.
 - The first config-space access has to come after the port clock is
   running and PORT_STATUS reports READY. The t8103 replay order
   configures the root complex first and hangs the machine on T8152.

Add apple,t8152-pciec with a hw_info that carries the register offset
delta (tun_delta) and selects the start order (cold_order): PERST off,
APPCLK on, wait for READY, clear the clock-gate disable, then the root
complex tunables. The other hardware tables keep tun_delta = 0 and
cold_order = false, so their register offsets and order do not change.
PORT_TUNCTRL and PORT_TUNSTAT now take the port to find the delta.

T8152 has no boot-loader handoff for this port, so cold_order also
counts as a valid preinit status: the board tree does not have to claim
a handoff that does not exist, and the start path replays the port from
its stopped state.

Measured on the Mac mini (M6, T8152), with this logic in a copy of the
driver built on an older tree: three boots, each with a Thunderbolt 3
dock on the ATC3 port. The link comes up on the first attempt, the
tunneled root port (106b:101b) enumerates and the dock's two Intel
bridges (8086:15da) appear behind it. Before the two changes the
host bridge hung the machine, and then probed but never linked.

Not measured: this patch on this base (compiled only); an endpoint
behind the dock (the dock's slot link did not train with either of two
cards, also on other hosts); the stop and suspend paths with the
shifted registers; cable removal and reconnect.

ACIO starts the tunnel only when apple_cio_pcie_tunnel_is_preinitialized()
accepts the PCIe-C node, which asks apple_pcie_tunnel_needs_cold_init()
for nodes without a handoff property. That guard accepted only
apple,t6000-pciec on the T600x/T602x machines, so a T8152 node (with
no handoff property, as it has no handoff) was refused: the tunnel
stayed off, and the cold start below was never reached from ACIO.
The guard now also accepts apple,t8152-pciec on an apple,t8152
machine. The tunnel_kernel_init parameter and the check for DART
tunables still apply, and no bootloader handoff is claimed. The
ACIO side already looks at nodes with the apple,t8103-pciec
fallback, which the T8152 node lists, so it needed no change. The
node now also takes the T600x cold-init paths of suspend and resume,
which are not measured on T8152.

Cold start: the tunnel DART is the first child and probes as soon as it
is created, so apple_pcie_tunnel_prepare() has to clock the port before
the children are populated. It selected only ports with
apple,pciec-kernel-init and the supported T600x and T602x ports; a T8152
tunnel matched neither, so its DART could probe before the port clock
ran. A port whose hardware table has cold_order is now selected too,
and its cold init uses the same order as the replay path: READY, then
the clock-gate disable, then the root complex tunables. The T8152 DART
defers its register access to the first use, which is why this did not
show up in the runs on hardware (those used the replay path). Not
measured: the prepare path on T8152 hardware.

apple_pcie_walk_tunnel_darts() now also visits apple,t8152-dart, so
apple_pcie_tunnel_quiesce() gates the commands of the T8152 tunnel DART
before APPCLK stops; without it the walker skipped that DART. The save
and restore steps still return -EOPNOTSUPP for it (it is not a
DART_T8020 hardware table), so suspend with a tunnel remains
unsupported on T8152; only the command gating is new. Not measured on
hardware.

Signed-off-by: Joshua Warren <816217+joshuaswarren@users.noreply.github.com>
Describe the PCIe-C tunnel of the rear USB-C port (ATC3, ACIO3) of
the Mac mini (M6, T8152): the tunnel's own Gen3 DART with 256 streams
and a static topology, and the PCIe-C root complex with its ECAM,
rc, port0, fabric, debug and intr2axi register windows, read from the
ADT. The acio node gets its apple,pcie-tunnel link. The tunnel
address ranges are identity per port (PCI address = CPU address): a
device behind the tunnel decodes the address it is given.

Measured on the Mac mini (M6, T8152) with these nodes on an older
tree and a Thunderbolt 3 dock on ACIO3: three boots, the tunnel link
comes up on the first attempt, the tunneled root port (106b:101b)
enumerates and the dock's two Intel bridges (8086:15da) appear behind
it. Identity ranges were also measured on an M2 Max (another board,
by another contributor): with a GPU in a USB4 enclosure, BAR0
returned data. Not
measured: this patch on this base (device tree compile and schema
check only); an endpoint behind the dock on the M6 (the dock's slot
link did not train with either of two cards, also on other hosts);
MSI delivery (the vector slice 1975 is unverified; enumeration does
not use MSI); the interrupt number of the PCIe-C port.

The container has a unit address so that dtc and the schema accept its
non-empty identity ranges; the DART driver finds a tunneled DART by its
PCIe-C sibling (the previous iommu patch) and no longer by the
container name.

DMA window: the ADT gives the tunnel DART vm-base 0x10080000000 and
vm-size 0x7ff00000. That is the 1 TiB alias of the endpoint-visible
window 0x80000000..0xfff00000, the upper 2 GiB of the 32-bit bus space
minus the top MiB (the MSI doorbell is at 0xfffff000); the T602x PCIe-C
DARTs are described the same way. So apple,dma-range is the
endpoint-visible window and apple,dma-offset is the alias: endpoints see
32-bit bus addresses and a 32-bit-DMA card gets an address it can use.
This is inferred from the ADT numbers and the T602x description; no
endpoint DMA through this tunnel was measured on T8152.

The boot loader supplies the register tunables of the tunnel nodes.
The node carries no apple,pciec-preinit-status: the driver treats the
T8152 hardware table as its own start path. The MSI vector slice at 1975
is not checked against the other PCIe controllers of the SoC.

The ltssm range (0x410020000, 0x4000) is the sixth register range of
this port's ADT node; the driver does not map it.

Depends on the ACIO3 nodes (series A) and on the PCIe-C driver patch
for apple,t8152-pciec.

Signed-off-by: Joshua Warren <816217+joshuaswarren@users.noreply.github.com>
@joshuaswarren

Copy link
Copy Markdown
Author

New head f4d8c0081e64cc48d4808654fcfa20374c49b5e0: fixes the Greptile finding on the tunnel-start check. apple_pcie_tunnel_needs_cold_init() now accepts apple,t8152-pciec on an apple,t8152 machine as well as apple,t6000-pciec on the T600x/T602x list; the tunnel_kernel_init and DART-tunables guards still apply and no bootloader handoff is claimed. The T8152 node lists the apple,t8103-pciec fallback, so ACIO's check reaches this function and the tunnel starts through the prepare path. Side effect, in the commit text: the node now takes the T600x cold-init suspend/resume paths (not measured on T8152). Receipts: pcie-apple.o, thunderbolt/apple.o, mailbox.o, t8152-j873g.dtb rc 0, 0 warnings (arm64 clang W=1, 16K pages). Not run: a boot.

@joshuaswarren

Copy link
Copy Markdown
Author

GREEN at f4d8c00

Compared with my GREEN at e21d7e6f: only the PCI driver commit changed, in apple_pcie_tunnel_needs_cold_init() (the rest of the diff is hunk offsets); the other three commits and the six below are identical.

  • The guard now reads: for apple,t6000-pciec the machine must match the T600x/T602x list as before; otherwise the node must be apple,t8152-pciec on an apple,t8152 machine. tunnel_kernel_init=0 still disables it, a handoff apple,pciec-preinit-status = <1> still refuses, and the DART apple,tunable check still applies. The J873G root compatible is "apple,j873g", "apple,t8152", ..., the node has no handoff property, iommu-map reaches the tunnel DART and that DART carries apple,tunable, so the guard accepts it and ACIO starts the tunnel (its own check follows the apple,t8103-pciec fallback, as the commit says).
  • The commit text now says the T8152 node takes the T600x cold-init suspend and resume paths, which are not measured on T8152.
  • Device tree and binding files are unchanged, so the dtcheck.sh result at e21d7e6f stands (0 warning lines, 0 lines, 72 examples with only the baseline message).

Not checked: compile receipts (the author's) and hardware. Merge still needs CI and the owner's word.

@joshuaswarren

Copy link
Copy Markdown
Author

Status at head f4d8c0081e64cc48d4808654fcfa20374c49b5e0: GREEN from two independent reviews at this exact head, all Greptile threads replied and resolved, arm64 clang W=1 compile of the touched objects rc 0 with 0 warnings, device-tree schema check 0 messages. Not done: a boot on hardware, CI. Needs a maintainer's CI approval and merge. Not needed for the release candidate.

@joshuaswarren

Copy link
Copy Markdown
Author

GREEN at f4d8c00

Correction first: my comment at e21d7e6 said the PCI driver commit was identical to what I had cleared. I compared it against 7902704, which I had not reviewed, instead of against f877c81. This review compares the PCI driver commit at this head (676f026) directly against the one I read at f877c81 (37275f5); the other three PCIe-C commits and the base layers are identical per commit (the #197 f40f1bb layers were cleared in #197).
The real delta of the PCI driver commit:

  • apple_pcie_tunnel_needs_cold_init: still true for T6000 on the listed machines; now also true for an apple,t8152-pciec node on an apple,t8152 machine, when tunnel_kernel_init is on, the node carries no preinit status of 1 (the dtsi no longer has the property) and its DART has tunables.
  • apple_pcie_tunnel_cold_init: for cold_order hardware the clock-gate disable is cleared right after READY and before the root-complex tunables, and not again afterwards; every other hardware keeps the old order (the second clear is now "!kernel_init && !cold_order").
  • apple_pcie_tunnel_prepare (the clocking before the DART probes): its search for a port to clock now also stops at a node whose hw table has cold_order, via a small helper that matches the node against the driver's table.
    Read through: with this, T8152 gets tunnel_cold_init set, so the PCI notifier registration and the other tunnel_cold_init paths now run on T8152, while setup_port still takes the "handoff" branch (preinit_ok is forced true by cold_order): it polls READY and replays reinitialize when the port is stopped, which is the cold_order path I read earlier. That combination is consistent on paper; it is new for T8152 and I have no board evidence for the notifier and resume paths.
    Not run: build; the author's compile and dt receipts are not reproduced. Merge still needs CI and a second non-author GREEN.

@joshuaswarren

Copy link
Copy Markdown
Author

[Joshua's agent] This PR is awaiting maintainer review. Is anything else needed from us?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant