Repository navigation
soc: apple: show the PMP request words and AGX vote state in the report status (stacked on #334) - #338
Closed
joshuaswarren wants to merge 6 commits into
Closed
soc: apple: show the PMP request words and AGX vote state in the report status (stacked on #334)#338joshuaswarren wants to merge 6 commits into
joshuaswarren wants to merge 6 commits into
Conversation
…n t600x On the M1 Max (t6001) the PMP keeps memory at its idle floor for a busy GPU until the AGX power request, bit 9 of the PMP report's PS-REQ word, is set. Nothing sets it on Linux, so GPU reads top out at 241 GB/s. The same chip reads 361 GB/s on macOS. Report GPU activity to the PMP. The GPU driver calls apple_pmp_agx_activity_begin() when a submission starts and apple_pmp_agx_activity_end() when it finishes. pmp-report sets bit 9 when the first operation starts and clears it 100 ms after the last one ends. It also clears it at once when the GPU driver unbinds or the report goes away, so the request is never left set past the GPU's power-domain release. The register sequence is the one apple_pmp_report_set_state() already uses for report entries: read 0xf80, write 0x107c0, poll 0x1000, status 0x10, 50 ms acknowledge. A set that is not acknowledged is cleared again and not retried for a second. The vote is off by default. pmp_report.agx_vote=1 turns it on, and pmp_report.agx_vote_idle_ms (default 100) sets the hold. With it off nothing is written. It applies to apple,t6001 only; t6000 is untested. The decisions (set, clear, hold, retry, wrap of the 32-bit millisecond clock) are plain functions in pmp-agx-vote.h with no kernel dependency. Setting the bit by hand with a test module on a 16-inch M1 Max moved a GPU read microbenchmark from 241 to 368 GB/s. This patch has not yet been run on hardware. Signed-off-by: Joshua Warren <816217+joshuaswarren@users.noreply.github.com>
…one power request bit The PS-REQ word of the PMP report is shared by every client of the PMP: the report entries (display, ISP, video decode, the AFNC IOAs), the T6001 AGX vote (bit 9) and, out of tree, the ANE clock level of omarchy-ane (bit 10). The report entries and the AGX vote read-modify-write the word under rep->lock. A module outside this driver could only map the window and read-modify-write it without that lock, so a concurrent write of another bit could be lost. apple_pmp_report_set_request(report_np, id, enable, no_ack) is the one writer a client outside this file may use. It calls the same apple_pmp_report_set_state() the entries use, so it takes the same lock and does the same acknowledge poll (at most 50 ms). The caller owns the bit it asks for. Signed-off-by: Joshua Warren <816217+joshuaswarren@users.noreply.github.com>
…ment clearing apple_pmp_report_set_request() lets a client outside pmp-report update one bit of the PS-REQ word. On the T6001 bit 9 is the AGX request, which the GPU vote owns: a second writer could clear it while the GPU works, or set it and leave it set. Refuse it from the export with -EBUSY on a t600x report. The vote itself writes through apple_pmp_report_set_state() and is not affected. Document what a caller must do after -ETIMEDOUT: the request was written but not acknowledged and stays in the word, so the caller clears it with @enable false and @no_ack true before giving up. Signed-off-by: Joshua Warren <816217+joshuaswarren@users.noreply.github.com>
…st operation A 9B model at 8192 tokens of context had one gap of up to 111.5 ms between two decoded tokens in every process, once per run, with the GPU idle for a good part of it. The 100 ms default hold dropped the request in that gap, which costs one slow token while the PMP raises memory again. The longest gap in any other configuration measured was 64 ms. Raise the default to 250 ms, about 2.2 times the longest gap seen. The cost is a few hundred milliseconds of the higher memory state after the last job. pmp_report.agx_vote_idle_ms still overrides it at run time. The default is one constant, AGX_VOTE_IDLE_MS_DEFAULT, used for both the parameter and the initial state. The userspace test has a case that rides through a 112 ms gap with the default and still drops the request after a long idle. Signed-off-by: Joshua Warren <816217+joshuawarren@users.noreply.github.com>
…ad only) Two read-only debugfs views for sampling next to a timing: apple-pmgr-misc exposes the whole clock-generator word of the fabric and DCS system devices, and the PMP report status gains the request and actual words and, on the T6001, the AGX vote state. They answer which hardware state a measurement ran in without a separate investigation. Nothing is written and nothing changes with them absent. Signed-off-by: Joshua Warren <816217+joshuaswarren@users.noreply.github.com>
The status file was created only for reports with fast-die entries or a PMP this driver starts. It now exists for every report: it prints running, and only while the PMP runs the request and actual words (and on the T6001 the AGX vote state). This also keeps the probe free of a reference to the T6001 offsets, which the host test of the T6030 PMP graph extracts and compiles without them. Signed-off-by: Joshua Warren <816217+joshuaswarren@users.noreply.github.com>
Author
Author
|
Closed: superseded upstream. The AGX vote landed on aurora-silicon/linux as #1290; the AGX-hold mechanism was replaced by scoped performance requests (996ef2a); the pmp-report status-file direction was overtaken when the raw DVFS dump was removed (9d5b025); the debugfs views for the fabric and DCS words live on in aurora-silicon/linux#2143. Kernel work continues on the aurora-silicon tree. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
[Joshua's agent] Stacked on #334: the first four commits are #334's; only the top commit (78ebe0e,
soc: apple: show the fabric, DCS and PMP request words in debugfs (read only)) is new, and this PR is rebased ontointegration/aurorawhen #334 merges. Review that one commit.What it adds to
pmp-report: the report status debugfs file now also exists for the T6001 report and, when the PMP is running, prints the request word (PS-REQ), the actual word (PS-ACK) and the AGX vote state (agx_vote: enabled voted users idle_ms failures, copied under the vote's own lock). Read only: tworeadq()at the offsets the driver already reads on every set, and the vote state copy. Nothing is written.Why: a performance A/B on the 16-inch M1 Max (decode +25 to +29 percent with the AGX request set) needed a separate probe module to show which request bits and memory states each run had. With this file and the fabric and DCS words of the companion
apple-pmgr-miscchange (#336) the state sits next to every timing.Checked: the same change applied to the 7.1.12 lab tree builds with its flags (object compiled) and a separate review of that patch found no write, no lifetime problem and the file created only for the T6001 report with the PMP ready. This commit applies cleanly on #334's head (
9f2a319c). Not booted yet.