Repository navigation
Conversation
c2a638b to
f24d520
Compare
scottjones
left a comment
There was a problem hiding this comment.
Thanks, this is a good saving, and I like that it only touches the DISPEXT entries the T6030 gate marks. A few things before it can go in:
- Host test.
tools/asahi/t6030-pmp-graph-tests.pyno longer builds at this head (8 compile errors), but passes at the base. The new forward declaration ofapple_pmp_report_set_state()throws off the test's extractor, and the test's copy of the struct has noidle_releasefield and doesn't pick up the new release function. The suite is blocking in host-tests.json. It's a small fix in the harness. - Re-assert on attach.
apple_pmp_report_use()re-asserts throughset_state(), which polls once for 50 ms, ignores the timeout passed in and never retries. If the ack is late,dcpext_bringupmarks the DCP-ext FAILED with no retry until reboot. The tree already treats a late DISPEXT ack as possible (the resume retry,defer-startup-ack, the T8122 wait loop). Could it poll or retry up to@timeout? - Timeout paths.
set_state()writes before it polls, so a timeout still leaves the request withdrawn in hardware. At boot the bit isn't recorded as released, so the first attach gets -EIO. On idle the display side still thinks it's asserted, so the next attach skips the re-assert. The sleep path already records the release even on timeout. Doing the same here, and logging when the idle withdraw fails, would cover it. - Attach race.
external_ready()reads the released flag beforehpd_mutexis taken, so a display plugged in during the withdraw can train its link with the request withdrawn. The window is small, but nothing closes it.
Smaller things: use disable_delayed_work_sync() at unbind; use() returns success without an ack when the status isn't READY; the comment says 0.35 W where your measurement shows about 0.25 W; and the subject prefix would usually be pmdomain: apple: pmp-report:.
On testing, could you also try an attach after a few hours idle, and one USB-C or Thunderbolt display? #401 saw a withdrawn DISPEXT request leave a display dark on T600x. T6030's CPU floor should prevent that here, but this state can last much longer than sleep.
(#401 merges cleanly with this one, so whichever lands second only needs a small harness rebase.)
f24d520 to
b6ac3d2
Compare
|
Thanks for the thorough review. I've force-pushed these changes:
I also made the smaller changes:
One more gap, found while testing with a TS4. A Thunderbolt display starts the processor its port is wired to, then gets routed to dpin0 of the other processor. Nothing ever disconnects from the first one, so it kept its request until reboot. The idle check now also runs once a processor's firmware is running. Tested on the J514S:
No ack timed out in any of these runs. Two things that look odd in the logs aren't from this change: the firmware's The attach after a few hours idle runs tonight, and I'll report back. |
scottjones
left a comment
There was a problem hiding this comment.
Thanks for working through all of that. Most of the earlier points are fixed, and the graph host test passes again. Moving the re-assert under hpd_mutex in connect_raw() closes the attach race nicely. There's one small thing left that I'd like fixed before it goes in.
use() skips the re-assert when the ack is late
apple_pmp_report_use() only re-asserts when wait_ready() returns -EIO. If a withdraw times out and the ack bit is still set, wait_ready() returns 0, so use() returns 0 without writing the request. The link then trains with the request withdrawn, and because st->released still holds the bit, nothing asserts it again until reboot. This also hits the first attach after boot if apple_pmp_start_release_idle() timed out. The new comments say the first use asserts it again, so I think this is just a missed case.
I think the fix is one line, if (ret && ret != -EIO) return ret;, since the loop already checks st->released. I've only checked that by reading the code, not by running it.
Could you add one host test with it: the released bit set with the ack still set, and use() must write the request again. use() and idle() aren't in the harness at the moment, so none of the new behaviour is covered yet.
Comments, not blocking
- The new ack poll in
use()can holdapple_pmp_start_mutex,hpd_mutexand, on HDMI, the fabric lock for up to 30 s. That happens with a late ack, or when the PMP status isn't exactlyPMP_REPORT_READY, where the poll can never succeed. Polling after dropping the lock, or a shorter timeout for the re-assert, would avoid stalling everything else. - On HDMI, a re-attach whose re-assert fails returns -EAGAIN, and nothing retries it once the processor is already running, so the screen stays dark until replug. Returning the real error (say -ETIMEDOUT) would let
dcp_external_retry()pick it up. Type-C is fine. - If an idle withdraw's ack is late and the system sleeps before it drops, resume re-asserts the request but
st->releasedstill says withdrawn, so it stays asserted until the next attach. Power only. - The
external_idle_triesincrement and its reset indcp_external_idle_schedule()run outsidehpd_mutex, so a lost reset can stop idle retries early. Also power only. - The commit message says the next use "only waits". After a re-assert whose ack is late,
st->releasedis already clear, so the nextuse()returns -EIO straight away. Either the wording or the code needs adjusting.
b6ac3d2 to
f01cd08
Compare
|
Thanks. You were right about Blocking: the late withdraw ack. It's your one-liner, A second case of the same bug: the late re-assert ack. Before, Host test.
Results:
Your comments:
Retest on the J514S, booted with this commit on my daily branch:
There were no "not acknowledged" or "withdrawn again" messages in the whole boot. W=1 and checkpatch are clean. Every The attach after a few hours idle is still pending; I'll report it here. |
scottjones
left a comment
There was a problem hiding this comment.
Thanks for going through the whole set of late-ack states rather than just the one case I pointed at. Rolling back in use() when a re-assert's ack is late was a good find of your own, too.
Both blockers are in: the one-line fix in use(), and a host test that fails when that fix is reverted. Having use() and idle() in the harness now makes all of this much easier to trust. The HDMI retry, the resume after a late idle withdraw, the idle counter race and the commit message are all addressed. Approving.
Comments, not blocking (a follow-up is fine)
apple_pmp_report_use()polls for the ack (read_poll_timeoutinapple_pmp_report_request()) while holding the globalapple_pmp_start_mutex, for the caller's whole timeout. On a dcpext first start that's 30 s. If the PMP doesn't ack, the other processor's re-assert waits behind it inwait_ready()while holding its ownhpd_mutex(and the fabric lock on HDMI), and the GPU and internal DCP waits queue up behind it too. Before this patchwait_ready()waited with the mutex released. I'd suggest giving the first-start ack poll a short budget like the re-assert's, or polling after dropping the mutex and recheckingreleased. Either one also removes the doubled timeout: a first start can now take up to 60 s and return -ETIMEDOUT, where a PMP that isn't READY used to get -EIO straight away.- Commit message: "500 ms at most" isn't true in that case. Also, the retries come from
dcp_external_retry()and the Type-C reconnect, not from*retryable, which is NULL for every T6030 caller. - Small one: the idle check reuses
dcp_external_sleep_vetoed(), which logs "sleeping with it as it is" after a session failure even when nothing is suspending.
One more note: the 500 ms re-assert budget isn't backed by a measured ack time yet. If the ack turns out slower in practice, re-plugs will go through the retry path, so it'd be worth logging the real latency when you next have it on hardware.
…quest
On a J514S (MacBook Pro 14", M3 Pro), the PMP DISPEXT requests of both
external display processors are seeded at the PMP's start and held from
then on, although a processor's firmware only starts when a display is
first attached, and keeps running after the display goes away. Held, they
keep the PMP's external display state up: display-off idle power is
3.66 W with them and 3.41 W with both withdrawn. macOS keeps an unused
processor's floor off (IOReport: DCPEXT1 IOP state Sleep, PMP DISP_EXT
floor OFF).
Keep seeding them, as the start checks those requests against the PMP's
tunables, but withdraw a request marked apple,release-until-used right
after the PMP has started. apple_pmp_report_use() asserts it again and
polls for its acknowledgment for up to the caller's timeout, from a
running PMP only; the external processor's start calls it in place of
apple_pmp_report_wait_ready(). The T6030 display gate marks the DISPEXT
requests.
A running processor withdraws its request again once its display is gone
(apple_pmp_report_idle()): a few seconds after a disconnect, under the
same test system sleep uses before it withdraws that request, and its
firmware keeps running as it does in sleep. The next attach asserts the
request under hpd_mutex, which it then holds up to the link, so an idle
withdraw cannot come in between. The same check also runs once a
processor's firmware is running: a Thunderbolt display starts the
processor its port is wired to, then is routed to dpin0 of the other one,
and no display ever disconnects from the first. A withdrawn request is
not acknowledged, so system sleep has nothing to release for it and
resume does not assert it.
A request is written before its acknowledgment is awaited, and the
request, its record in the report and the display side's view are kept
in step when that acknowledgment is late:
- a withdraw acknowledged late is recorded as withdrawn and logged;
the next use writes the request again even if the PMP still shows
it acknowledged;
- an assert not acknowledged within the caller's timeout is withdrawn
again and stays recorded so, and the attach fails with that error;
the next use writes it and waits again;
- a request recorded as withdrawn is withdrawn in the hardware, so an
idle withdraw writes nothing more, and resume does not assert one
that system sleep found still acknowledged.
An attach that re-asserts the request holds hpd_mutex, and on HDMI the
fabric lock, so it gives the acknowledgment 500 ms; the first start
gives it 30 s. A failed re-assert fails the attach with its error, and
dcp_external_retry() on HDMI, or the Type-C reconnect, tries again.
The PMP graph host test extracts these functions: its extractor now skips
prototypes, and its copy of struct apple_pmp_start and its kernel API
stubs cover the new fields and helpers. It now also runs use() and
idle() with apple,release-until-used: a case for each late
acknowledgment above, each with a mutant that the case catches.
Assisted-by: Claude:claude-opus-5-5
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Ilyas Akin <ashnwor@gmail.com>
dcp_external_sleep_vetoed() logs "external display processor failed earlier: sleeping with it as it is" when it lets system sleep through for a failed processor. The idle check of an unused external processor uses the same test, so the line also showed up after a session failure with nothing suspending. Keep the test silent and log from dcp_platform_prepare() and dcp_platform_suspend(), under the same conditions as before, so the idle check no longer logs it. Assisted-by: Claude:claude-opus-5-5 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: Ilyas Akin <ashnwor@gmail.com>
apple_pmp_report_use() polled for the acknowledgment of a request it asserted again while holding apple_pmp_start_mutex, for the caller's whole timeout: 30 s on an external display processor's first start. Every other report's waiters take that mutex, so a PMP that did not acknowledge stalled the other processor's re-assert (holding its hpd_mutex, and on HDMI the fabric lock), the GPU and the internal DCP behind it. A PMP that was not running could never acknowledge, yet was polled for the whole timeout after the start wait had used it already, so a first start could take 60 s and return -ETIMEDOUT where it used to get -EIO at once. Write the request under the mutex, mark it as being asserted and hold a waiter on the report, then poll with the mutex dropped for what is left of the caller's timeout, and take the mutex again to record the request as asserted or withdraw it again. The request stays recorded as withdrawn until its acknowledgment is in, so system sleep and resume treat it as before; idle() and another use() of it get -EBUSY meanwhile and write nothing. A report torn down meanwhile withdraws it again and returns -ENODEV, and its drain waits for the waiter. A PMP that is not running gets -EIO with nothing written. The time left is at least one jiffy, as a zero timeout would let the poll run forever. No in-tree caller can reach -EBUSY: a processor's re-assert and its idle withdraw both run under its hpd_mutex once it is running, and its first start runs before that. Each request belongs to one processor. Log how long the acknowledgment took. On a J514S it was 0.6 to 1.3 ms in five asserts, HDMI and Thunderbolt (TS4), first starts and re-attaches, including both processors asserting at the same time. The re-assert asks for it within 500 ms. The PMP graph host test runs a hook inside the ack poll, as another caller would, and counts request writes. New cases, each with a mutant that it catches: the PMP not running (nothing written, nothing polled); the mutex not held during the poll; idle() and use() mid-poll get -EBUSY and write nothing; the poll budget is what is left of the timeout, at least one jiffy; and a teardown mid-poll withdraws the request, returns -ENODEV and releases the waiter. Assisted-by: Claude:claude-opus-5-5 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: Ilyas Akin <ashnwor@gmail.com>
f01cd08 to
1519058
Compare
|
Thanks for approving. I'd rather not leave comment 1 for a follow-up, since this PR introduced it, so I've added two commits and reworded the first one's message: 1. The ack poll under
I first tried your other option, a 500 ms cap with retries for the first start. It didn't hold up. A PMP start that had failed with -ETIMEDOUT made it spin for 30 s, and rolling the request back and rewriting it every 500 ms could take a stale ack as the new one. So I went with the unlocked poll. Ack latency, as you asked. The re-assert's success line now logs how long the ack took. I measured five asserts on the J514S with this build, and every ack took 0.6 to 1.3 ms:
The poll steps 1 ms at a time, so these are upper bounds. That gives the 500 ms budget a margin of a few hundred times. No ack timed out and nothing went through the retry path. In the TS4 row both processors asserted at the same moment, and each got its ack without waiting on the other; before this change the second would have waited on the mutex. Host test. The graph test runs a hook inside the ack poll that acts like another caller, and it counts writes to the request register. The new cases are:
Each has a mutant, and each mutant dies at the case meant for it.
W=1 and checkpatch are clean on the driver code. Every The attach after a few hours idle is still pending. |
|
The attach after a few hours idle is now tested too. HDMI (2560×1440 @ 100) was plugged in 3 h 2 min after its processor withdrew its request. The system stayed awake the whole time. The request was asserted again with the ack in 926 µs, and the display came up. There were no timeouts and no retries. The firmware's |
scottjones
left a comment
There was a problem hiding this comment.
Thanks for taking comment 1 on now rather than leaving it for later. The unlocked poll looks right to me. The waiter keeps the report alive through the poll, the rollback keeps "released" meaning withdrawn, and the first start and the ack poll now share one deadline. The latency numbers settle the 500 ms question too. Approving.
One question, not blocking: on the TS4 attach, does the processor the port is wired to log "no display attached: PMP request withdrawn" a few seconds after its firmware starts? Your table shows the asserts but not that withdraw, and that is where the TS4 power saving comes from.
|
Yes. On the first TS4 attach, dcpext1 (the processor the port is wired to) starts, the display is routed to dcpext0's dpin0, and dcpext1 then withdraws its request 5.2 s after its firmware comes up: dcpext0 withdraws its own about 5 s after unplug (372.7). On the re-attach, the display was routed to dcpext1 itself, so only request 9 was asserted, and dcpext0 stayed withdrawn. |
What this changes and why
On the J514S (MacBook Pro 14", M3 Pro, T6030), the PMP DISPEXT requests of both external display processors are seeded when the PMP starts and held from then on. That happens even though:
Held, the requests keep the PMP's external display state up. macOS keeps an unused processor off: on another J514S, IOReport shows
DCPEXT1IOP stateSleepand the PMPDISP_EXT Floorfor DISPEXT1 atOFF.This withdraws a DISPEXT request whenever its processor has no display:
At boot: the requests are still seeded, because the start checks them against the PMP's tunables. A request marked
apple,release-until-used(the T6030 display gate marks the DISPEXT ones) is withdrawn right after the PMP has started.On attach: the processor's start calls the new
apple_pmp_report_use()instead ofapple_pmp_report_wait_ready(). It asserts the request again and polls for the PMP's acknowledgment, up to the caller's timeout and from a running PMP only, before the CPU runs. A later attach to a running processor does the same indcp_dptx_connect_raw(), underhpd_mutex, which it then holds up to the link, so an idle withdraw cannot come in between.On detach: a few seconds after the display goes away,
apple_pmp_report_idle()withdraws the request again. It uses the same test system sleep already uses before it withdraws that request (dcp_external_sleep_vetoed()). The firmware keeps running, as it already does with the request withdrawn in sleep. If the display is still being taken down, this is retried for about a minute. The same check also runs once a processor's firmware is running: a Thunderbolt display starts the processor its port is wired to and is then routed to dpin0 of the other one, so nothing ever disconnects from the first.Late acknowledgments: the request, its record in the report and the display side's view are kept in step:
An attach waits at most 500 ms for the re-assert, since it holds
hpd_mutex. The PMP graph host test covers each of these cases with a mutant.System sleep: a withdrawn request is not acknowledged, so sleep has nothing to release for it and resume does not assert it. Otherwise sleep is unchanged.
What was tested
linux-aurorarecipe viatools/aurora-ci/prepare-recipe.py.macsmc_hwmon"Total System Power"), on AC at 100%, with PCIe L1 substates on (PCI: apple: Allow L1 substates on J514S internal links #489):For comparison, refusing both processors at boot (
apple_t6030_display.dcpext=0) saved 0.35 W. The rest of that is the processors' power domains, which stay on so the next attach works.request 8 asserted for useand starts the processor;no display attached: PMP request withdrawn until the next attachabout 5 s later;display attached: PMP request asserted again, and the display comes back.Modeset done, but pipe not enabledline on attach also shows on 1.15 kernels without this change.withdrew unused requests 0x300 until first use. The internal panel starts as before.make W=1of the touched objects (clang) is clean, withCONFIG_DRM_APPLE_KUNIT_TESTon. checkpatch is clean apart from the trailers.t6030-pmp-graph-tests.py --mutationspasses (27 cases, every mutant caught), and also with--refat the base. The late-acknowledgment cases fail on the previous head (--ref b6ac3d2e7). Everytoolshost suite was run at the base and at this head: this PR changes no other result.What was NOT tested
Risk
AI assistance
🤖 Generated with Claude Code