diff --git a/devices/j72t/firmware.json b/devices/j72t/firmware.json index 0d25e450..cf1806b3 100644 --- a/devices/j72t/firmware.json +++ b/devices/j72t/firmware.json @@ -313,6 +313,16 @@ "coprocessor": "AOP_ext" } ] + }, + { + "destination": "lib/firmware/apple/gfx-s8003.bin", + "sha256": "33ae20cbbe2498e4f119ba4e9a55467e4e0192d4c86d5bf1903adce63ffa5dca", + "steps": [ + { + "step": "agx-g5p-ldr-from-kernelcache@1", + "input": "kernelcache" + } + ] } ] } diff --git a/docs/bring-up/gpu/a9-g5p.md b/docs/bring-up/gpu/a9-g5p.md new file mode 100644 index 00000000..bcf20c47 --- /dev/null +++ b/docs/bring-up/gpu/a9-g5p.md @@ -0,0 +1,702 @@ +# A9 GPU (G5P, iPad 5): what the powervr backend took and what it left + +The iPad 5's GPU (S8003/A9, PowerVR GT7600, Apple's "G5P") was brought up by +Berkay Kösebay in his own driver, `apple-g5p` (orchard #74, with #119, #120 +and #125 on top). Orchard runs the GPU from the upstream `powervr` driver +instead, as a second chip next to the A10 (`pvr_apple_s8003.c`). This page +records everything of his boot work that the backend does **not** carry, with +the reason it existed and what he measured, so none of it has to be found +again. It also says where the ported parts live now, and it defines what +userspace hands the kernel for a job on the A9 ("The job interface for +userspace"): the Mesa port is built from that section. + +Everything below about the hardware and the firmware is his finding, read from +his code and its comments (`drivers/gpu/apple-g5p/` and +`drivers/gpu/drm/apple-g5p/` in #74, with #119 and #125). Firmware +addresses (`0x1000....`) are the META image's, kext addresses (`c9....`, +`6c9....`) are the iOS 16.7.10 (20H350) AGX kext's, as he cites them. Nothing +here was re-measured: the backend has not run on hardware yet. + +State of the backend: it boots Apple's unmodified firmware, attaches VMs, +writes geometry, fragment and compute jobs, completes them from the +firmware's stamps, restarts the firmware when a job hangs, and stops it when +idle and for sleep. The core counts as supported, so the GPU starts on every +boot of a board whose device tree enables it. None of it has run on hardware: +the first run is Berkay's test of `main` on his J71t. + +## Where his boot code lives now + +| apple-g5p | powervr backend | +|---|---| +| `ldr.h` `g5p_ldr_walk()` (own LDR parser) | not ported: `pvr_meta_ldr_process()` in `pvr_fw_meta.c`, the driver's own walker, with a target per firmware | +| `pack.h` `g5p_image_offset()`, `g5p_pack_record()` | `s8003_ldr_mem()`, `s8003_ldr_config()`, `s8003_ldr_start_threads()` | +| `boot-layout.h` `g5p_plan_layout()` | `s8003_layout_plan()` (objects by their CPU-side kext number) | +| `boot-layout.h` `g5p_make_boot_config()` | `s8003_image_configure()` | +| `boot-layout.h` `g5p_make_init()` | `s8003_init_build()`; the register addresses and the interrupt come from the device tree (`s8003_parse_dt()`) | +| `boot-layout.h` `g5p_meta_address()`, entry encoders | `s8003_meta_addr()`; the page-table entries are built by `pvr_mmu.c` | +| `boot.inc` `g5p_boot_map_page()`, `g5p_boot_region()` (private page tables) | the kernel VM context: `s8003_map()`, `pvr_vm_map()`, `pvr_vm_map_io()` | +| `perf-envelope.h` `g5p_perf_install()`, `g5p_perf_seed()` | `s8003_init_perf()` | +| `core.inc` `g5p_fender_wake_once()` | `pvr_apple_s8003_boot()` | +| `fender.h` idle / hold / start, `boot.inc` `g5p_boot_program_stream()` | `s8003_core_off()`, `s8003_release()` | +| `boot.inc` `g5p_boot_drain()`, `boot-r73b.inc` `g5p_r73b_aux_pass()` | `s8003_drain()`, on a timer instead of the interrupt | +| `boot.inc` `g5p_boot_events()` (ALIVE wait) | `s8003_wait_alive()` | +| `lifecycle.inc` control prepare / submit / wait, `kick.inc` | `s8003_first_command()`, `s8003_device_control()`, `s8003_doorbell()` | +| `lifecycle.inc` `g5p_boot_park()` | `pvr_apple_s8003_stop()` | +| `boot.inc` `g5p_boot_sink_init()` (zero page at GPU address 0) | `s8003_map()`: `zero`, in the kernel VM, plain leaf flags as his | + +## The boot, as ported + +Registers: R0 is the Rogue core (`rgx`), R1 is Fender (`fender`). + +1. **Wake** (`c93fe8..c93ff8`): R1+0x10000 = 0xffff0000 (an empty boot stream), + R1+0 = 0x40000000, R1+0x1000 = 0x11 (power state 1), wait for bit 4 of + R1+0x1000 to clear. R0 answers afterwards; the core ID is R0+0x18 = + 0x011b0602, revision R0+0x20 = 0x00090f00 (BVNC 9.15.6.2). +2. **Idle** (`c9fa10`): R1+0x20000 = 0, wait R1+0x1000 bit 4 clear, + R1+0x1000 = 0x10 (state 0), wait for state 0. +3. **Hold** (`c9fc10`): R1+0x20000 = 0x20, wait for it to read back, state + still 0. +4. **Boot stream** (`c9feb4`): R1+0x10 = 0, R1+0 = 0x40000000, then pairs of + words from R1+0x10000 (at most 0x200 bytes): a Rogue register offset with a + flag byte above it (0xff on the last or only half of a register) and its + value. The writes: SOFT_RESET 0x00e7fffffffffc1d, then + 0x00e7fe071ffffc1d, then 0x100000000; META_BOOT = 1; + MMU_CBASE_MAPPING_CONTEXT = 0; MMU_CBASE_MAPPING = catalogue >> 12; + SLC3_CTRL_MISC = 3; SLC3_SCRAMBLE and SCRAMBLE2 = 0xb42dd24b4bd22db4; + R0+0xe160 = the address of a spare page; GARTEN_SLC = 0; SOFT_RESET = 0; + the end marker 0xffff0000. Between them R1+0x2010 = 0xbf and + R1+0x2014 = 0x233. Each stream word reads back as written; that is no + proof that Fender ran it. +5. **Release** (`c9f954`): R1+0x20000 = 0, state 0 confirmed, + R1+0x1000 = 0x10 | q, wait for state q. Fender replays the stream; its + last write releases the reset and the META boots itself from the image + mapped at GPU address 0xe1c0000000. +6. **Alive**: the firmware posts event type 0 to the event ring. +7. **First command**: device-control command 8 with no payload + (control+0x1020 = 0, ring entry {8, 0, 0, 0, 0}), R1+0x14 = 7 and its + read-back, R0+0xb00 (MTS_SCHEDULE) = 0x10; the firmware advances the ring's + consumer index. + +Not known: what R1+0x2010 / R1+0x2014 and R0+0xe160 mean, what distinguishes +the two SOFT_RESET masks (the first is the other register layout's full +mask, see "R46" below), and what command 8 does beyond being consumed. + +The META slave port is not used for the boot. Polling MSLVCTRL1 while the +reset is held locked the SoC; his driver forbids slave-port access from the +first boot stream on for the rest of the boot. + +### Memory + +| GPU address | size | contents | leaf attributes | +|---|---|---|---| +| 0xe1c0000000 | 0x30000 | the unpacked image | PM/META protect | +| 0xe1c0200000 | 0x60000 | "shared": control, rings | protect, cache coherent | +| 0xe1c3200000 | 0x8000 | "root": init, channels, policy, platform | protect, cache coherent | +| 0xe1c4200000 | 4 pages | Fender (2), AIC+0 (time), AIC+0x4000 (software interrupts) | protect, SLC bypass | +| 0 | 1 page | zeroes ("sink", his round 12): catches an access at address 0 while the firmware starts; iOS can map a dummy page for every unmapped address | none | + +The image: LDR address 0x10000000 (0x18000) at +0x4000, 0x20000000 (0x4000) at ++0x1c000, 0xc0000000 (0x4000) at +0, 0xe0001000 (0x4000) at +0x20000, core +memory 0x80000080 (0x55dc) at +0x24000; ZEROMEM commands for +0x80000000..0x88000000 are not applied by the host. The bootloader's +configuration starts at +0x200 (13 segment-MMU words, the image's 27 register +writes, a zero pair, the address of +0x3ff8); +0x3ff8 and +0x3ffc hold the +META addresses of the init object and the control object. + +The rings (256 entries each; indices in a 0x20-byte control object, consumer +at +0, producer at +0x10): events (0x70/0x74, 44-byte entries), four log lanes +(0x71/0x75), statistics (0x72/0x76, 44 bytes), trace (0x73/0x77, 36 bytes), +device control (0x6c/0x6d, 20 bytes), three work rings (0x60/0x61, 0x64/0x65, +0x68/0x69, 20 bytes). Apple's order per interrupt: acknowledge it, the four +log lanes, events, statistics, trace; the consumer index goes back only after +the entries were read, with a full barrier. + +Events: type 0 alive (word 1 was 0x8000a120 in his logs), type 1 a job's +stamp (two per job), recovery (the firmware stopped; iOS restarts the GPU), 7 +page request, 8 channel error. A healthy firmware posts about 290 alive events +and raises its interrupt about 575 times a second, idle or not. + +### The governor ("policy", object 0x4a) and why it is pinned + +The firmware has its own DVFS governor. Its inputs: the clock table at +platform+0x710 (highest index), +0x714 (count), +0x718.. (MHz per Fender state: +0, 340, 474, 550, 723, iOS's values on J71t, captured 2026-11-01); limits at +control+0x1028 (3348, the power envelope; iOS has 3851), +0x102c (floor), ++0x1030 (ceiling 400); overrides control+0x1038 / +0x1040, which the firmware +copies to policy+0x68c / +0x6f4 on every pass. It requests state +`min(policy+0x6f0, policy+0x688, control+0x102c) / 100`. + +The backend uses his "pinned" record, the only one every boot of his consumed +command 8 with: floor = override = 100 * q, ceilings 400, the sampler off +(policy+4 = 0), and for q > 1 the record pre-set to the target (policy+0x28 = +policy+0x2c = q, +0x30 = 0, +8 = 0) so the firmware's boot sampler finds the +core where it wants it; the host powers the core up in state q itself. +Pinned, the firmware never changes the state and never powers the core off. + +What he measured on the way: + +- The firmware copies the clock table at boot and its scheduler reloads its + timer from the entry of the current state; with the table left zero that + entry is 0. iOS has 340 for state 1 (platform+0x71c), and his default + boots install it ("latency114"). +- platform+0x784 is added to the AIC's time to form the firmware's clock. iOS + writes its abstime offset; an absolute counter value put the firmware's + clock an uptime ahead of every host stamp. Linux writes 0. +- policy+0x8f8 is the governor's last-sample time. With policy+0 = 1 and + policy+4 = 16 (iOS's persisted record) and a stale stamp, the boot sampler + ran the load governor on empty accumulators and went to state 0: no ALIVE + ("seed 2", 2026-11-03). Re-stamping just before the release fixed the path + but is not needed with the sampler off. +- policy+4 = 16 with control+0x20 = 0 wedged the first TA ("latency123" + T2/T3); installing iOS's whole record at map with the release at state 1 + came ALIVE but never consumed command 8 ("integration-g", 2026-11-11). +- platform+0x6c8 must be 1 on this path (with 0 the firmware skips its policy + loop, "dc variant 7"); platform+0x6d4 = 1 turns on per-tick cache + maintenance and is left 0. + +## Not ported + +### The phase runner and its stages + +`apple-g5p` runs the boot as named phases (`validate`, `prepare`, `wake`, +`id`, `sample`, `map`, `arm`, `load`, `release`, `events`, `control-*`) from a +sysfs `command` file or its autoboot work, each latching a result. The backend +has one function per step and no runner. Dropped with it: + +- **`sample`**: reads T0/T1 TXENABLE/TXSTATUS (0x04800000/10, 0x04801000/10) + through the slave port before the first boot stream. Diagnostic. +- **`arm`**: stores the stream without its last write and powers up with the + reset still held, then reads SOFT_RESET back as 0x100000000 to prove Fender + executes the stream. Proven; the backend releases directly. +- **`load`** as a stage: a memcmp of the mapped image against the packed one. +- Snapshots of Fender (0, 8, 0x10, 0x14, 0x1000, 0x1004, 0x2010, 0x2014, + 0x20000) and of R0 around each stage, the PMGR gfx word decode, the FIFO + dump, every `G2 ...` log line. +- The SHA-256 of the LDR and of the packed image checked in the kernel, and + the parser's pins (exactly 119112 bytes, 94 records, one start record of 32 + bytes). The firmware recipe pins the file's digest when it extracts it. + +### Alternate boot paths and their knobs + +All of these were boot-time switches ("latched at map"); the defaults were +r73b case 1, fast 1, r73e 1, r74 1, r74f 1, r74g 1, r74h 0, r87b policy 1, +"nopatch" 1. The backend has no switch. + +- **r73b** (crash-first interrupt diagnostics): serves all rings from the + interrupt thread with breadcrumb rows kept in RAM (256 rows), an interrupt + cap (8192, later 1048576) and an optional watchdog. Its functional part, + the ring order and the acknowledge-first rule, is ported; the rows, caps + and watchdog are not. +- **r73c / r73e** ("fast control", "first-ALIVE submit"): command 8 is + prepared before the release and published from the interrupt at the first + ALIVE, to measure release-to-ALIVE and release-to-submit times and to close + the window before the firmware's first power decision. The backend + publishes it from process context after its ALIVE poll (1 ms steps); the + staged phases before r73c did the same with longer gaps and the firmware + consumed it. +- **r74e and "wake112"** (the power proof): the release polls R1+0x1000 for + the requested state for 1.8 ms (10 ms above state 1) and stops as soon as + ALIVE is seen, because after ALIVE only one owner may read R1. On warm + boots ALIVE could arrive before the first read (off 1500 ns, on 375 ns, + nothing sampled); that case and "request still in flight" are deferred to + the first-ALIVE handler instead of counted as a failed power-on. See the + second race fix below. The backend's poll simply continues for up to + 100 ms regardless of ALIVE. +- **r74 / r74b** ("forced-ready epoch", RAM-only waits): raises the caps and + makes the wait loops read RAM only. +- **r74f** (deferred doorbell): if the power state is not "on" when a DC3 + attach pair or a TA is published, the host repeats the off/on transition + (bounds 1000 us off, 1800 us on) and the first ALIVE afterwards owns one + doorbell attempt (bound 250 ms). The backend does not repeat a power + transition: it rings the doorbell when no request is in flight and waits + for the firmware either way (`s8003_doorbell()`). +- **r74g** (firmware-owned MMU pass): the firmware's DC3 handler + (0x1000218c..0x10002218) writes R0+0xe140/0xe148 for the slot, copies the + host's generation to its cached one and issues the catalogue invalidation + R0+0xe138 = (N << 3) | 4; a consumed DC3 pair is the receipt, so the host + does no MMU register access. The TA is then published without a doorbell + and given a grace of four ALIVEs or 20 ms to be taken; root+0x3980 = 2 + means the firmware entered recovery. This is how a client's page tables + reach the firmware; in the backend `s8003_slots_load()`, with a doorbell + and without the grace. +- **r74h, r80..r96**: observers (trace tail, register banks read from the + interrupt). Diagnostic. +- **r87b** (attach policy): in two campaigns the DC3 attach read the power + state as 0x10/0x11, the host's off/on repair timed out (-110) and the + machine froze, while every retained case shows the firmware consuming the + pair by itself. Policy 1 ("self-drain"): no repair, no bell, wait for the + consume in RAM for 3 s. The backend's attach waits 3 s the same way and + rings the doorbell first unless a power request is in flight. +- **heartbeat116**: replaced the lifetime interrupt/event caps (which made an + idle, healthy GPU fail after about 30 minutes) by per-window rates. The + backend has no cap. +- **r73f cases 1..4, r75, r79** (patched firmware images) and **firmware + patching** as such: `apple-g5p` could boot a modified image; "nopatch" + (Apple's original image on the r74/r74f/r74g path) became the default + after its device pass on 2026-11-03. Only the names and the refusal logic + are in #74; the patches themselves are not, so their contents are not + recorded here. The backend boots Apple's image only and checks nothing + that a patched one would need. +- **R46**: the first SOFT_RESET value with bits 56..59 added (VBS/VCE/TE3/ + TILING_CORE in the other register layout). Expected to be a no-op; off. +- **Release ACK**: after the release, clear EVENT_STATUS bit 9 (0x200) once + through R0+0x138. A notification acknowledge only; off by default. +- **dc variants 1, 5, 6, 8, 9** of the init data (cache-bypass aliases for + the device-control pointers, control+0x30 = 0, iOS's DVFS words). Only + variant 7 flew by default, and its one change (platform+0x6c8) is undone + again by the default path: the init data is as `g5p_make_init()` builds it. + +### The overclock + +`gpu-oc.h`, `boot_gpu_oc_mhz`: PMGR's PSINFO table (PMGR+0x70000 + 0x10 * n) +holds what Fender applies for power state n; LLB writes slots 0..4 once per SoC +reset. Word 0 = core-rail DAC code [31:24], PLL valid [21], multiplier +[20:12], predivider [8:4], postdivider-1 [3:0]; word 1 [7:0] = buck8 code; +word 2 = timing. f = 24 MHz * mult / prediv / postdiv. Record 4 on this chip +is 0x8a2f1041 (723 MHz, core 881.25 mV, buck8 987.5 mV). His overclock wrote +a record 5 from record 4 with only the multiplier moved (726..1080 MHz, VCO +at most 2160 MHz, Apple's own 804/850/900 MHz descriptors preferred), +extended the firmware's table to state 5 and powered up there, with a +thermal stop that rebooted the firmware at state 4. + +Not ported: it changes a volatile PMGR table outside the GPU, and the product +bar is the stock clock. The table decode is the useful part if the GPU's OPP +table is ever filled from PSINFO. + +### The firmware's own idle park ("self-park") + +With control+0x20 = 1 (iOS: platform word 0x00200545 & 1) the firmware parks +itself when idle (0x1001cec8: drain, R1+0x1004 = wake target, R1+0 |= bit 29, +then bit 30, state 0) and reboots on the next doorbell; the kext then only +kicks (R1+0x14 = 7, R0+0xb00 at any state) instead of power-cycling. A parked +firmware wakes into its sampler, which divides by policy+4: the pinned record +(policy+4 = 0) must not be parked. His driver armed the park only after a +complete boot, writing iOS's coefficient block (47 words of policy), the +3851 mW envelope, policy+4 = 16 and then control+0x20 = 1, in that order. +R1+0[30:29] = 11 with state 0 and R1+8 = 0 is what Apple's kext accepts as +quiescent before sleep. + +Not ported: the backend stops the firmware from the host when idle (runtime +PM), as on the A10. If that proves too slow or does not repeat, this is the +alternative, and the coefficient block is in `perf-envelope.h` +(`g5p_park_policy[]`). + +### Power cycling, recovery, dirty start + +- **r33**: hold the gfx power gate (PMGR 0x80340) off for a while before the + first Fender write, because R0+0x418/0x41c keep the previous boot's value + through a 10 us cycle. Auto-PM (bit 28) is cleared with the off target and + set again after the on target is reached, as pmgr-pwrstate does. +- **In-boot recovery** (`recover.inc`): freeze, mask the interrupt, cycle the + gfx gate (10 ms off), then wake / map / ... / control again with a new set + of firmware memory. Measured to bring back a wedged device. iOS does the + same: stop, platform power cycle, new boot. In the backend a hang + recovery (`pvr_apple_reset()`) stops the firmware, runtime-suspends the + GPU, which turns the gate off, waits 10 ms and leaves the boot to the next + job. If another job holds the GPU powered at that moment the gate stays + on and the firmware is restarted in place. An idle runtime-PM cycle + usually lasts far longer than 10 ms, but nothing enforces it: a job that + arrives right after the GPU went idle boots the firmware after a shorter + gate-off time. +- **dirtystart108**: a state machine that classifies how the GPU was left + (cold, kexec, gate seen off) and refuses a release or a DRM claim on bad + evidence. Dropped with kexec. +- **Teardown / kexec quiesce** (`teardown.inc`, #125): leaves a known state + for the next kernel. Dropped unless kexec becomes a product feature. +- **r41**: replay of PMGR bridge settings (0x201f00000, 0x201f80000). Off by + default; an experiment. + +### The two lost-GPU races of #125 + +1. **A phase's doorbell skipped** ("3d-silent"). The publish-time doorbell of + a render phase was skipped when the engine's completion-event count had + moved since the job was armed. With two jobs in flight the previous job's + 3D completes during this job's TA, the count moves, and this job's 3D was + taken for retired: no doorbell, the firmware did not consume the command + until something else woke its scheduler, and after a second of silence the + job and its context were lost (glmark2 terrain: 2 of about 11 runs). + Apple rings after every ring entry without any check, and a doorbell for a + consumed entry adds no work. His fix: skip only when the phase's own stamp + shows it retired. The backend does the same (`s8003_publish()`). +2. **A power-on request that ALIVE overtook.** The firmware boots without + waiting for the cores' power-on, so ALIVE can arrive while R1+0x1000 still + has bit 4 set next to the target (0x14 for state 4). Counting that as a + failed power-on left the GPU unusable for the boot (twice in a day on + J71t, cold and warm; last reads 0x14 after 43 and 52 polls, good power-ons + settle after 13 to 54 polls of 10 us). Apple's setGPCPerformanceState and + setGPUCorePower wait for bit 4 for up to 30 s. In the backend: the release + waits for the state up to 100 ms whether or not the firmware is alive, and + the doorbell waits for bit 4 (`s8003_core_ready()`). + +### Register access rules ("MMIO licence") + +`mmio-licence.h`: iOS allows an R0 access when the gfx gate is on, the wrapper +is up, R1+0 bit 29 is clear (the firmware raises it before it powers the core +down) and the state is 1..max. His driver adds: no request in flight (bit 4) +and R1+0 != 0. R0 must not be touched with the core off or in transition, and +no Fender register with the gfx gate off. The backend touches R0 only for the +core ID after the wake and for the doorbell after the state settled, and +Fender only while the device is runtime-active; it has no general accessor, +because it reads no GPU register while the firmware runs. + +### Smaller things + +- **Cleaning the page tables' cache lines** (`dcache_clean_poc` on catalogue, + directory and leaves before the release): not ported as a step of its own, + because `pvr_mmu.c` does it for every table it changes + (`dma_sync_single_for_device()`, which is that clean on arm64). +- **The firmware's interrupt** (AIC 174, the fifth of the node): the firmware + sets it through the AIC's software-set register (AIC+0x4014, bit 14) and + the host clears it at AIC+0x4094 before draining. The backend does not take + the interrupt yet (the AIC driver cannot clear a software-set interrupt) + and reads the rings whenever it looks for a finished job (every 100 us + while one is in flight) and every 100 ms otherwise. +- **Interrupt self-test** (host sets the software interrupt itself): dropped. +- **The device PM** (control page at shared+0x5c000, list and free list in + the two pages after the shared memory): the first context's PM in his + driver. The backend has PM sets only, starting right after the shared + memory. +- **Events beyond "alive" and a job's stamp**: recovery (the firmware + stopped), page request (type 7), channel error (type 8) with his fault + bookkeeping per context. The backend logs such an event and goes on + reading; a job the firmware gave up times out into the reset. +- **Timing values**: Fender polls 10 us / 100 ms; ALIVE wait 3 s; command 8 + wait 3 s; attach consume wait 3 s; first-ALIVE settle 2 ms; render waits + 20 s soft, 45 s, 60 s hard; quiesce lock 2 s. + +## The job path, as ported + +His compute and render paths are not the A10's ring-of-commands model. In the +backend the A9 therefore submits and completes its own jobs +(`pvr_apple_s8003_submit()`, `pvr_apple_s8003_engine_done()`); the fences and +their polling, the scheduler queues and their timeout, the order of scenes +across VMs, runtime PM, sleep and the reset are `pvr_apple.c`'s, shared with +the A10. + +| apple-g5p | powervr backend | +|---|---| +| `render-user-layout.h` `g5p_ru_channel_init()`, `compute-user-layout.h` `g5p_cu_channel_init()` | `s8003_channel_init()` | +| `render-user-layout.h` `g5p_ru_fill_ta()`, `g5p_ru_fill_ta_v3()`, the TA half of `g5p_ru_rt_fill()` | `s8003_fill_ta()` | +| `g5p_ru_fill_3d()`, `g5p_ru_fill_3d_v2()`, `g5p_ru_fill_3d_width()`, the 3D half of `g5p_ru_rt_fill()`, `render-user-msaa.h` `g5p_ru_fill_msaa()` | `s8003_fill_3d()`, `s8003_fill_output()` | +| `render-user-geometry.h` `g5p_ru_rt_derive_samples()` | `s8003_rt_derive()`, from the HWRT dataset's width, height and samples | +| `render-user-layout.h` `g5p_ru_job_at()` (record links, microsequence, command, pointer ring) | `s8003_job_write()` | +| `compute-user-layout.h` `g5p_cu_job()`, `g5p_cu_sku()` | `s8003_submit_compute()` | +| `drm-render-v3.inc` `g5p_rv3_publish_ta()`, `g5p_rv3_publish_3d()`, `drm-compute-v1.inc` publish | `s8003_publish()` | +| `render-user-wake.h` `g5p_wk_ring()`, `drm-render-v3.inc` `g5p_rv3_dc_kick()`, `kick.inc` | `s8003_doorbell()` | +| `render-user-layout.h` `g5p_ru_retired_pl()`, `g5p_cu_retired()` | `pvr_apple_s8003_engine_done()` | +| `drm-render-v3.inc` `g5p_rv3_fw_touch()` | `s8003_touch()`, at most once a millisecond while a job is waited for | +| `render-user-bind.h`, `drm-render-v3.inc` `g5p_rv3_bind()`, `g5p_rv3_unpark()`, `g5p_rv3_dc_wait()` | `s8003_slots_load()`, `s8003_vm_attach()`, `s8003_device_control()` | +| `render-user-mmu.h`, `drm-render.inc` `g5p_render_invalidate()` | `pvr_apple_s8003_mmu_flush()` | +| `render-user-window.h`, `drm-render.inc` `g5p_render_fw_entry()` (firmware window) | `pvr_apple_s8003_vm_init()`: the PM sets mapped in every VM at the firmware's addresses | +| `render-user-pmset.h`, `render-user-layout.h` `g5p_ru_pm()`, `drm-render-v3.inc` `g5p_rv3_pmctx_build()` | `s8003_pm_build()`, `pvr_apple_s8003_vm_scenes()` | +| `render-user-pmset.h` park rule, `render_user_close_park` | `pvr_apple_s8003_vm_fini()` and the park VM | +| `recover.inc`, `power-r33.inc` (gate cycle, 10 ms) | `pvr_apple_reset()` with the chip's `gate_off_ms` | +| #119 `render_user_pipeline_depth` 2, `render_user_joint`, job slots | two scenes per VM (`scenes`, `vm_scenes()`), the fragment job submitted with its geometry job, two record slots per channel | +| #125 "ring a phase's doorbell unless its own stamp ..." | `s8003_publish()` | + +### Channels and records + +Each engine has one channel in the shared memory: geometry at shared+0x54000, +fragment at +0x58000 (a page each), compute at +0x5b000 (0x1000 bytes). In a +channel: control words at +0x100 (five watermarks 16 bytes apart, the host's +at +0x140; the ring's size, 16, at +0x150), a ring of 16 pointers at +0x200, +a timestamp queue (+0x1800; compute +0x900), two stamp words (+0x1840; +compute +0x940), a dependency object (+0x1900; compute +0x980). + +A render job takes two pointer-ring entries, a 0x40-byte command and its work +record W (geometry 0x560 bytes, fragment 0x8c0), and has a microsequence +(geometry 0xc0 bytes, fragment 0x180). Odd jobs have command, W and +microsequence at +0x300, +0x400 and +0xa00 (fragment +0xd00), even jobs +packed from +0x1c00, so a job never rewrites what the firmware still reads +for the job before it. A compute job takes one entry: W (0x290 bytes) at ++0x400, microsequence at +0x800. + +The command of a geometry job is `{6, slot, PM id, PM set}`: the firmware +takes the parameter manager's record from the set and rebuilds its free +list. His driver sends that only when the PM state is not known clean and +otherwise `{4, own stamp, seq - 1}` (`render_user_pm_record`); the backend +always sends it, his older and longer-flown form. The command of a fragment +job is `{4, address of the geometry channel's second stamp, n}`: wait until +that stamp reached n, the number of its geometry job. The firmware compares +signed, so a later geometry job's stamp does not block it. + +To publish: an entry `{engine, channel, end}` in the engine's work ring +(0x61, 0x65, 0x69; `end` is how far the pointer ring is filled), the host's +watermark = `end`, the work ring's producer index, then the doorbell: +Fender+0x14 = 7 and its read-back, wait for no power request in flight, +R0+0xb00 = 0x10 (geometry, compute, device control) or 0x11 (fragment). The +doorbell is skipped only if both of the channel's stamps already reached the +job's number. His compute path rang no doorbell; the backend rings the +geometry engine's, which wakes the firmware's scheduler. + +A job is done when both stamps reached its number and every watermark lies +between its end of the pointer ring and that of the last job submitted. His +driver also looked at the engine's idle words (channel object +0x724..), the +completion events and a 500 ms hold, to tell a job the firmware finished from +one it gave up; the backend takes the stamps alone, and a job whose stamps +never come times out after 5 s, which restarts the firmware. + +### VMs: slots and tags + +A VM gets two of the firmware's slots (7 to 255) with its first job, one for +geometry and compute jobs and one for fragment jobs: the page-table root's +address goes to object 0x86[slot] (64 bits), a tag to 0x84[slot], then two +device-control commands `{3, slot}` (the fragment slot's first). The +firmware's handler loads the root and copies the tag to 0x85[slot]; the +commands being consumed and the tag copied is the receipt. No job runs while +a slot is loaded. + +The tag's top ten bits changing make the firmware drop the slot's +translations from the catalogue down when the slot's next job starts +(bits 21:12 from the directory, 11:0 the tables only). The backend adds +0x400000 to the tag of every loaded slot whenever powervr flushes the MMU +(any VM's page tables changed); his driver kept a counter per VM and level. + +When a VM goes its slots are not emptied but loaded with the park VM's root +(a VM that maps only what every VM maps) and tag 0x500000, and slots and PM +sets are handed out in turn: the PM kept writing the free list of a context +through its geometry slot after the context was closed, two job switches +later at the latest (his release8 runs, page faults in the closed slot). +A parked slot is emptied (root 0, tag 0, `{3, slot}`) before a new VM takes +it. + +After every start of the firmware all of this is told again: slots with the +VM's next job, PM sets with its next geometry job. + +### The parameter manager's memory + +A PM set is three pages in the firmware's GiB, behind the shared memory +(GPU address 0xe1c0260000 + 0xc000 * set; 12 sets, his driver had 8 and a +"device PM" the backend does not have): + +- control page: +0x10 the META address of the last block's list entry, +0x14 + the free list's GPU address, +0x1c its bytes, +0x20 its entries, +0x24 + entries - 1, +0x34 0x4000, +0x40 the list page's META address, +0x54 and + +0x58 the entries again. At +0x80 the scene's record (0xc0 bytes): +0xa8 + macrotile array, +0xa0 MList, +0xb0 VHEAP (GPU addresses in the VM), +0xb8 + the META address of list+0x40; the firmware saves the PM's state in the + rest between a geometry job and its fragment job. +- list page: +0x60 = 1; from +0x80 one entry of 0x24 bytes per block: the + META address of the entry before it, then the block's eight 16 KiB pages + as 4 KiB page numbers. +- free list page: four consecutive 4 KiB page numbers per pool page. + +A set's PM id is its number + 1 (W+0x10 of the geometry record, both +microsequences, the command). + +The pool is `blocks` blocks of 128 KiB, kernel pages that are mapped nowhere. +The PM's own tables (5 pages: macrotile array 0x80 bytes, MList 0x10000, +VHEAP 0x600) are kernel pages mapped in the VM at +0x101000000 + 0x20000 * set, protected (PM/firmware only, bit 62) and without +a CPU mapping for userspace. His driver mapped them at 0x101000000 without +bit 62; see "Open on hardware". + +All sets' three pages are mapped in every VM at the firmware's addresses +with the firmware's attributes (protected, cache coherent): the PM reaches +them through the job's slot. His driver shared one kernel-built directory +under the firmware's catalogue entry for that; the backend maps the same +pages through the VM's own tables. + +### What is not ported from the job path + +- His own UAPI, VM, buffer objects, resource lists and their validation + (`drm-vm.inc`, `render-user-validate.h`, `compute-user-validate.h`): the + backend takes addresses from the stream unchecked, as on the A10. An + address userspace gets wrong faults in its own slot. +- The serial executor with its phases, the r74f/r74g/r87b attach machinery, + job logs, latency and job-time accounting, receipts, PM snapshots, fault + readers, the wake A/B, `job_slots`/`pm_once`/`pm_ctx` switches, spare + pairs, the per-boot fault budget and frozen contexts. +- The render target's "main ring" of sixteen slices: the HWRT dataset has a + region header buffer per RT data, which userspace alternates. +- Typed profiles V1 and V2 (allocator pools, `vdm_control`). +- The firmware's self-park and the retained-RAM sleep of #120: the firmware + is stopped for runtime suspend and for sleep and booted again. + +## The job interface for userspace + +The A9 uses powervr's UAPI (`include/uapi/drm/pvr_drm.h`) unchanged: VM +contexts and maps, buffer objects, contexts of type `RENDER` and `COMPUTE`, +`CREATE_FREE_LIST`, `CREATE_HWRT_DATASET`, `SUBMIT_JOBS`, sync objects. +What differs from the A10 is listed here. The structures are in +`drivers/gpu/drm/imagination/pvr_apple.h`, the header values in +`pvr_rogue_fwif_stream.h`, the stream definitions (order of the fields in the +stream) at the end of `pvr_stream_defs.c`. + +### Device + +- The device query's GPU info reports BVNC 9.15.6.2. The heaps are the A10's + (`pvr_apple_heaps`): general at 0x0800000000, PDS at 0x2a00000000, USC at + 0x2c00000000, region headers at 0x2ef0000000, all below 2^38. +- PDS code addresses in the firmware's program words are relative to the PDS + heap's base, USC code addresses to the USC heap's base: the firmware is + given those bases at boot. (apple-g5p used one base, 0x300000000, for + both, and a 4 GiB window from it.) +- Reserved in every VM, outside the heaps: 0 (a zeroed page), + 0x200000 (a kernel page, unused on the A9), 0x101000000 to 0x101180000 + (the PM's tables), 0xe1c0260000 to 0xe1c02f0000 (PM sets). +- Context creation: the static context state is optional and only + validated, as on the A10. + +### Free list + +`CREATE_FREE_LIST` is validated as on the A10 (the free-list buffer must be +a `PM_FW_PROTECT` object that userspace cannot map), but the list holds no +pages on the A9 and its buffer is not written. `initial_num_pages` (4 KiB +pages) is the size of the parameter manager's pool for the VM's renders: +`blocks = max(initial_num_pages / 32, (50 + tiles) / 32)`, both rounded up, +at most 128 (16 MiB); `tiles` is the render target's 32x32 tiles. A VM pins +one pool with its first render and a second one with its first render while +a scene of its own is still in the firmware, if the firmware has more than +half of its 12 PM sets left. There is no growth and no partial render: a +scene that needs more than the pool stops the firmware and the job times +out. + +### HWRT dataset + +`CREATE_HWRT_DATASET` is validated as on the A10 (MList, macrotile array and +VHEAP in protected objects), single layer, 1 or 4 samples, at most +8192x8192. Used on the A9: + +- `width`, `height`, `samples`: the kernel derives the geometry words; +- `rt_data_args[i].region_header_dev_addr`: the region headers of RT data i + (geometry W+0x28, fragment W+0x20). The buffer holds `px * py * 5` bytes; + the kernel cannot check that; +- `region_header_size`: as `pvr_drm.h` defines it for this parameter format + and as Mesa passes it for every other device, the ISP region size in + dwords: the region headers of one of the 16 macrotiles, + `mx * my * s * s * 5 / 4` (so `px * py * 5 / 64`). The kernel refuses a + job on a dataset that gave less; it writes its own derived value to the + record (fragment W+0xa8), not the dataset's; +- `geom_data_args.tpc_dev_addr`, `tpc_size`: the tail pointer cache, at + least `p2 * p2 * 8` bytes; + +with `tx = ceil(width / 32)`, `ty = ceil(height / 32)`, +`mx = align(ceil(tx / 4), 4)`, `my` alike, `s = 2` for 4 samples else 1, +`px = mx * s * 4`, `py = my * s * 4`, `p2 = roundup_pow_of_two(max(px, py))`. +A job on a dataset with a smaller `region_header_size` or `tpc_size` fails +with `-EINVAL`. + +Not used on the A9: `pm_mlist_dev_addr`, `macrotile_array_dev_addr`, +`vheap_table_dev_addr`, `rtc_dev_addr`, the ISP merge values, the free +list's own buffer. The PM's tables are the kernel's (above). + +Userspace alternates the two RT datas between consecutive renders to one +target, as upstream does: the kernel may hold two scenes of a VM, and each +needs its own region headers. + +### Jobs + +A render is a geometry job and a fragment job on the same HWRT data, in one +`SUBMIT_JOBS` call or two. The kernel submits the fragment job to the +firmware right after its geometry job. A fragment job whose geometry job was +not submitted (or failed) ends with `-ECANCELED`. Partial-render fragment +jobs are refused (`-EOPNOTSUPP`), transfer contexts do not exist, as on the +A10. A compute job needs no render before it. + +Each command stream is the Rogue command's stream followed by one extension +header and its data. The header word is `type << 29 | BIT(0)`; all three +A9 extensions have only bit 0. The kernel does not insist on the extension: +without it those words of the record are zero. The A10's extension types (1 +for geometry and fragment) are refused on the A9. + +**Geometry**: `PVR_STREAM_EXTHDR_TYPE_GEOM_G5P` = 2, +`struct pvr_apple_g5p_geom_regs`. From the Rogue command the kernel reads +only `regs.vdm_ctrl_stream_base` (W+0x98, low two bits cleared). + +| field | drm_g5p_render_pass(_v3) | goes to (geometry W) | +|---|---|---| +| `u64 scratch_base` | `scratch_va` | +0x90, low three bits cleared | +| `u64 scratch_ptr[6]` | `scratch_ptr[6]` | [0..3] +0x468.., [4] +0x48 and +0x488, [5] +0x50 | +| `u64 ta_program` | `ta_program_va` - PDS base | +0x88 (R0+0x470): byte offset of the TA state PDS program from the PDS heap's base | +| `u64 sample_positions` | `sample_positions` | +0x40; 0 = the standard pattern (0x88, or 0xeaa26e26 for 4 samples) | +| `u32 context_pds[2][6]` | `context_pds` | +0x58, +0x70 | +| `u32 resume_pds[2][4]` | `resume_pds` | +0xa0, +0xb0 | +| `u32 ta_pds[4]` | `ta_pds` | +0xc0 (three words; the fourth is zero) | + +**Fragment**: `PVR_STREAM_EXTHDR_TYPE_FRAG_G5P` = 2, +`struct pvr_apple_g5p_frag_regs`. From the Rogue command the kernel reads +`regs.isp_scissor_base` (W+0x60 with the low two bits cleared, +0x4a0), +`regs.isp_dbias_base` (+0x68, +0x490), `regs.isp_oclqry_base` (+0x70, ++0x78), `regs.isp_bgobjdepth` (+0x3a4, +0x6a8) and `regs.isp_bgobjvals` +(+0x3a8, +0x6ac); these were `scissor_va`, `dbias_va`, +`occlusion_query_va`, `bgobj_depth` and `bgobj_vals` of +`drm_g5p_render_pass_ext`. A zero depth or values word is taken as given +(apple-g5p's defaults for a pass without them were 1.0f and 0x300). + +| field | drm_g5p_render_pass(_ext) | goes to (fragment W) | +|---|---|---| +| `u64 background_texture` | `background_texture` | +0xb8, +0x5f0 | +| `u64 zls[6]` | `zls[6]` | +0x80/+0x618, +0x88/+0x628, +0x90/+0x630, +0x98/+0x648, +0xa0/+0x650, +0x138/+0x6d0 | +| `u64 sample_positions` | `sample_positions` | +0x58; 0 = the standard pattern | +| `u32 background[8]` | `background` | +0x5b8..; [0], [1] also +0xb0, +0xb4; [4..7] also +0xc0.. | +| `u32 eot[4]` | `eot` | [1], [0], [2] at +0x3b4..; [0..2] at +0x680.. | +| `u32 eot_resume[4]` | `eot_resume` | [0..2] at +0x694.. | +| `u32 restore[4]` | `restore` | +0x600, +0x604, +0x610, +0x614 | +| `u32 pixel_output_width` | `pixel_output_width` | 2, 4 or 8 (0 = 2): +0x6b0 and the partition words | +| `u32 flags` | `msaa_flags` | bit 0 `PVR_APPLE_G5P_FRAG_MSAA_RESOLVE`, with 4 samples only | + +The sample count is the HWRT dataset's; there is no `samples` field. With 4 +samples the kernel writes the AA mode, ISP_CTL and the output partitions as +`g5p_ru_fill_msaa()` did. + +**Compute**: `PVR_STREAM_EXTHDR_TYPE_COMPUTE_G5P` = 1, +`struct pvr_apple_g5p_compute_regs`. From the Rogue command the kernel reads +only `regs.cdm_ctrl_stream_base` (W+0x70). Compute jobs are submitted on a +`COMPUTE` context; they run on the VM's geometry slot. + +| field | drm_g5p_compute_pass | goes to (compute W) | +|---|---|---| +| `u64 state[4]` | `state_va[4]` | +0x1ec.. | +| `u64 tail` | `state_tail_va` | +0x60, +0x20c | +| `u64 terminate` | `cdm_va + cdm_end - 4` | +0x214, +0x23c: the address of the stream's terminate word | +| `u64 pds_store` | `store.code \| store.data << 32` | +0x21c | +| `u64 pds_resume` | `resume.code \| resume.data << 32` | +0x22c | +| `u64 pds_terminate` | `terminate.code \| terminate.data << 32` | +0x68 | +| `u32 config[2]` | `config` | +0x10, +0x14 | +| `u32 pds_store1` | `store.control` | +0x224 | +| `u32 pds_resume1` | `resume.control` | +0x234 | + +Fields in the stream follow the order of the tables, each naturally aligned; +an array is copied as bytes, aligned to 8. + +What apple-g5p's passes had and the stream does not: `vdm_va` (the Rogue +command), `vdm_size`, `vdm_end`, `ta_program_size`, `scratch_size`, the +pools and `vdm_control` of the older profiles, `samples`, the resource list +with its access flags, `zls_address_mask`, `occlusion_query_size`, +`scissor_count`, `dbias_count`: the kernel does not validate ranges. + +### Errors and loss + +A job the firmware does not finish in 5 s ends with `-ETIMEDOUT`, as does +every other job in flight; the context it was on takes no more jobs (the +A10's rule), the firmware is stopped, the GPU's power domain goes off for at +least 10 ms if no other job holds it on, and the next job boots the firmware +again. Other contexts go on. + +## Open on hardware + +- Does the boot work at all in this shape (kernel VM page tables, shmem + pages instead of DMA-coherent ones, polling)? +- Does stop, gate off, boot repeat (runtime PM), and how long does it take? +- Does the firmware mind that its interrupt is never acknowledged? +- Which of the first four interrupts fire, and for what? +- Does a render complete with the PM's tables mapped protected (bit 62)? His + driver mapped them without it. If not, the fallback is his attribute for + those five pages; a compute shader writing them must then be shown to + fault or the pages need another guard. +- Does the firmware take PDS and USC code bases that differ (powervr's + heaps, 0x2a00000000 and 0x2c00000000), and data addresses up to 2^38? His + driver ran with one base at 0x300000000 and low addresses. +- Does a geometry job's `{6, ...}` command on every job, the stamps alone as + completion, and a doorbell for compute behave as assumed above? +- Do two scenes of one VM in flight work with one PM set each from the + start (his second set came with the first pipelined job)? +- Is 3 MiB at platform+0x77c the limit for PM sets (12 used, he had 8)? +- Does a reset's gate cycle through runtime PM bring a hung firmware back + as his direct PMGR cycle did, and what happens when another job keeps the + domain on (then the firmware is restarted without the cycle)? +- The cost of the page reads (`s8003_touch()`) per millisecond while jobs + run, and whether they are needed with the backend's memory (his was + DMA-coherent memory mapped non-cached; the backend's is shmem mapped + write-combined). diff --git a/kernel/patches/devices/s8000/0014-arm64-dts-apple-iPad-5-J71t-Enable-the-GPU.patch b/kernel/patches/devices/s8000/0014-arm64-dts-apple-iPad-5-J71t-Enable-the-GPU.patch new file mode 100644 index 00000000..bbaf3015 --- /dev/null +++ b/kernel/patches/devices/s8000/0014-arm64-dts-apple-iPad-5-J71t-Enable-the-GPU.patch @@ -0,0 +1,37 @@ +From 0000000000000000000000000000000000000000 Mon Sep 17 00:00:00 2001 +From: Henrik Klee +Date: Sat, 14 Nov 2026 23:28:48 +0800 +Subject: [PATCH] arm64: dts: apple: iPad 5 (J71t): Enable the GPU +MIME-Version: 1.0 +Content-Type: text/plain; charset=UTF-8 +Content-Transfer-Encoding: 8bit + +The powervr driver takes the A9's GPU node now and starts its firmware. +Enabled on the J71t, the board Berkay Kösebay's driver runs this GPU on +and the one whose firmware recipe has the GPU's image. The J72t and the +S8003 phones keep it disabled until theirs have it: without the file the +driver would only fail to load it at every boot. + +The driver binds only with powervr.exp_hw_support=1 as long as the GPU +does not render. + +Assisted-by: LLM +--- + arch/arm64/boot/dts/apple/s8003-j71t.dts | 7 +++++++ + 1 file changed, 7 insertions(+) + +diff --git a/arch/arm64/boot/dts/apple/s8003-j71t.dts b/arch/arm64/boot/dts/apple/s8003-j71t.dts +index 733cefc..4775055 100644 +--- a/arch/arm64/boot/dts/apple/s8003-j71t.dts ++++ b/arch/arm64/boot/dts/apple/s8003-j71t.dts +@@ -24,3 +24,10 @@ + &wifi { + brcm,board-type = "apple,feara"; + }; ++ ++/* ++ * The GPU: the powervr driver starts its firmware. It does not render yet. ++ */ ++&gpu { ++ status = "okay"; ++}; diff --git a/kernel/patches/devices/s8000/0015-arm64-dts-apple-iPad-5-J72t-Enable-the-GPU.patch b/kernel/patches/devices/s8000/0015-arm64-dts-apple-iPad-5-J72t-Enable-the-GPU.patch new file mode 100644 index 00000000..481e6b79 --- /dev/null +++ b/kernel/patches/devices/s8000/0015-arm64-dts-apple-iPad-5-J72t-Enable-the-GPU.patch @@ -0,0 +1,50 @@ +From 0000000000000000000000000000000000000000 Mon Sep 17 00:00:00 2001 +From: Henrik Klee +Date: Sun, 15 Nov 2026 00:12:59 +0800 +Subject: [PATCH] arm64: dts: apple: iPad 5 (J72t): Enable the GPU + +The cellular iPad 5 has the same GPU as the J71t and, in the same iPadOS +16.7.10 restore image, the same firmware for it: its kernelcache's +AGXFirmwareKextG5P section is byte for byte the J71t's (checked with +orchard.apple.remote_ipsw; sha256 33ae20cb...5dca). Its firmware recipe +installs the file now, so its node is enabled too. + +The S8003 phones stay as they are: iOS 15.8.8's kernelcaches for the +N71m, N66m and N69 carry a firmware image of another size, which the +recipe step refuses and whose layout the driver has not been checked +against. + +Assisted-by: LLM +--- + arch/arm64/boot/dts/apple/s8003-j71t.dts | 4 +--- + arch/arm64/boot/dts/apple/s8003-j72t.dts | 5 +++++ + 2 files changed, 6 insertions(+), 3 deletions(-) + +diff --git a/arch/arm64/boot/dts/apple/s8003-j71t.dts b/arch/arm64/boot/dts/apple/s8003-j71t.dts +index 4775055..018ad6c 100644 +--- a/arch/arm64/boot/dts/apple/s8003-j71t.dts ++++ b/arch/arm64/boot/dts/apple/s8003-j71t.dts +@@ -25,9 +25,7 @@ + brcm,board-type = "apple,feara"; + }; + +-/* +- * The GPU: the powervr driver starts its firmware. It does not render yet. +- */ ++/* The GPU: its firmware file comes from this board's firmware recipe */ + &gpu { + status = "okay"; + }; +diff --git a/arch/arm64/boot/dts/apple/s8003-j72t.dts b/arch/arm64/boot/dts/apple/s8003-j72t.dts +index bc36459..a3e0dea 100644 +--- a/arch/arm64/boot/dts/apple/s8003-j72t.dts ++++ b/arch/arm64/boot/dts/apple/s8003-j72t.dts +@@ -24,3 +24,8 @@ + &wifi { + brcm,board-type = "apple,fearb"; + }; ++ ++/* The GPU: its firmware file comes from this board's firmware recipe */ ++&gpu { ++ status = "okay"; ++}; diff --git a/kernel/patches/drivers/gpu/0017-drm-imagination-Let-the-caller-place-a-META-LDR-imag.patch b/kernel/patches/drivers/gpu/0017-drm-imagination-Let-the-caller-place-a-META-LDR-imag.patch new file mode 100644 index 00000000..71b8b049 --- /dev/null +++ b/kernel/patches/drivers/gpu/0017-drm-imagination-Let-the-caller-place-a-META-LDR-imag.patch @@ -0,0 +1,387 @@ +From 0000000000000000000000000000000000000000 Mon Sep 17 00:00:00 2001 +From: Henrik Klee +Date: Sat, 14 Nov 2026 23:28:12 +0800 +Subject: [PATCH] drm/imagination: Let the caller place a META LDR image's + contents + +The LDR walker put an image's memory into the four sections of an IMG +firmware and nothing else. Apple's GPU firmware for the A9 is a META LDR +image as well, laid out differently: its sections go into one block at +offsets Apple's bootloader expects, its register writes are needed one +by one, and so are the stack pointer and entry of the thread it starts. + +Give the walker a target: a function that says where the bytes at a META +address go (or that the command is left out), and optionally one for +each register write and one for the START_THREADS command. IMG's +firmware loads through a target that does what the walker did before: +the same aliasing of the global range, the same core memory rules, the +same sections. + +A chain of blocks that never ends is refused: more blocks than the image +has room for cannot be a list. + +Assisted-by: LLM +--- + drivers/gpu/drm/imagination/pvr_fw_meta.c | 205 +++++++++++++++------- + drivers/gpu/drm/imagination/pvr_fw_meta.h | 23 +++ + 2 files changed, 165 insertions(+), 63 deletions(-) + +diff --git a/drivers/gpu/drm/imagination/pvr_fw_meta.c b/drivers/gpu/drm/imagination/pvr_fw_meta.c +index 9ff03bc..30022e4 100644 +--- a/drivers/gpu/drm/imagination/pvr_fw_meta.c ++++ b/drivers/gpu/drm/imagination/pvr_fw_meta.c +@@ -105,15 +105,57 @@ add_boot_arg(u32 **boot_conf, u32 param, u32 data) + *(*boot_conf)++ = data; + } + ++/* ++ * The sections an IMG firmware image is loaded into, as the target of ++ * pvr_meta_ldr_process(). ++ */ ++struct pvr_meta_fw_sections { ++ struct pvr_device *pvr_dev; ++ u32 coremem_size; ++ u8 *fw_code_ptr; ++ u8 *fw_data_ptr; ++ u8 *fw_core_code_ptr; ++ u8 *fw_core_data_ptr; ++}; ++ ++static int ++pvr_meta_fw_sections_mem(void *data, u32 offset, u32 size, bool zero, void **write_addr) ++{ ++ struct pvr_meta_fw_sections *sections = data; ++ ++ if (zero) { ++ if (ROGUE_META_IS_COREMEM_DATA(offset, sections->coremem_size)) { ++ /* cannot zero coremem directly */ ++ *write_addr = NULL; ++ return 0; ++ } ++ ++ /* Global range is aliased to local range */ ++ offset &= ~META_MEM_GLOBAL_RANGE_BIT; ++ } else if (!ROGUE_META_IS_COREMEM_CODE(offset, sections->coremem_size) && ++ !ROGUE_META_IS_COREMEM_DATA(offset, sections->coremem_size)) { ++ /* Global range is aliased to local range */ ++ offset &= ~META_MEM_GLOBAL_RANGE_BIT; ++ } ++ ++ return pvr_fw_find_mmu_segment(sections->pvr_dev, offset, size, sections->fw_code_ptr, ++ sections->fw_data_ptr, sections->fw_core_code_ptr, ++ sections->fw_core_data_ptr, write_addr); ++} ++ ++static const struct pvr_meta_ldr_target pvr_meta_fw_sections_target = { ++ .mem = pvr_meta_fw_sections_mem, ++}; ++ + static int + meta_ldr_cmd_loadmem(struct drm_device *drm_dev, const u8 *fw, +- struct rogue_meta_ldr_l1_data_blk *l1_data, u32 coremem_size, u8 *fw_code_ptr, +- u8 *fw_data_ptr, u8 *fw_core_code_ptr, u8 *fw_core_data_ptr, const u32 fw_size) ++ struct rogue_meta_ldr_l1_data_blk *l1_data, ++ const struct pvr_meta_ldr_target *target, void *target_data, ++ const u32 fw_size) + { + struct rogue_meta_ldr_l2_data_blk *l2_block = + (struct rogue_meta_ldr_l2_data_blk *)(fw + + l1_data->cmd_data[1]); +- struct pvr_device *pvr_dev = to_pvr_device(drm_dev); + u32 offset = l1_data->cmd_data[0]; + u32 data_size; + void *write_addr; +@@ -130,14 +172,7 @@ meta_ldr_cmd_loadmem(struct drm_device *drm_dev, const u8 *fw, + ((((u8 *)l2_block->block_data) + data_size) - fw) >= fw_size) + return -EINVAL; + +- if (!ROGUE_META_IS_COREMEM_CODE(offset, coremem_size) && +- !ROGUE_META_IS_COREMEM_DATA(offset, coremem_size)) { +- /* Global range is aliased to local range */ +- offset &= ~META_MEM_GLOBAL_RANGE_BIT; +- } +- +- err = pvr_fw_find_mmu_segment(pvr_dev, offset, data_size, fw_code_ptr, fw_data_ptr, +- fw_core_code_ptr, fw_core_data_ptr, &write_addr); ++ err = target->mem(target_data, offset, data_size, false, &write_addr); + if (err) { + drm_err(drm_dev, + "Addr 0x%x (size: %d) not found in any firmware segment", +@@ -145,32 +180,23 @@ meta_ldr_cmd_loadmem(struct drm_device *drm_dev, const u8 *fw, + return err; + } + +- memcpy(write_addr, l2_block->block_data, data_size); ++ if (write_addr) ++ memcpy(write_addr, l2_block->block_data, data_size); + + return 0; + } + + static int + meta_ldr_cmd_zeromem(struct drm_device *drm_dev, +- struct rogue_meta_ldr_l1_data_blk *l1_data, u32 coremem_size, +- u8 *fw_code_ptr, u8 *fw_data_ptr, u8 *fw_core_code_ptr, u8 *fw_core_data_ptr) ++ struct rogue_meta_ldr_l1_data_blk *l1_data, ++ const struct pvr_meta_ldr_target *target, void *target_data) + { +- struct pvr_device *pvr_dev = to_pvr_device(drm_dev); + u32 offset = l1_data->cmd_data[0]; + u32 byte_count = l1_data->cmd_data[1]; + void *write_addr; + int err; + +- if (ROGUE_META_IS_COREMEM_DATA(offset, coremem_size)) { +- /* cannot zero coremem directly */ +- return 0; +- } +- +- /* Global range is aliased to local range */ +- offset &= ~META_MEM_GLOBAL_RANGE_BIT; +- +- err = pvr_fw_find_mmu_segment(pvr_dev, offset, byte_count, fw_code_ptr, fw_data_ptr, +- fw_core_code_ptr, fw_core_data_ptr, &write_addr); ++ err = target->mem(target_data, offset, byte_count, true, &write_addr); + if (err) { + drm_err(drm_dev, + "Addr 0x%x (size: %d) not found in any firmware segment", +@@ -178,14 +204,44 @@ meta_ldr_cmd_zeromem(struct drm_device *drm_dev, + return err; + } + +- memset(write_addr, 0, byte_count); ++ if (write_addr) ++ memset(write_addr, 0, byte_count); + + return 0; + } + ++static int ++meta_ldr_cmd_start_threads(const u8 *fw, struct rogue_meta_ldr_l1_data_blk *l1_data, ++ const struct pvr_meta_ldr_target *target, void *target_data, ++ const u32 fw_size) ++{ ++ struct rogue_meta_ldr_l2_data_blk *l2_block = ++ (struct rogue_meta_ldr_l2_data_blk *)(fw + ++ l1_data->cmd_data[0]); ++ u32 data_size; ++ ++ /* Verify block header is within bounds. */ ++ if (((u8 *)l2_block - fw) >= fw_size || ((u8 *)(l2_block + 1) - fw) >= fw_size) ++ return -EINVAL; ++ ++ if (l2_block->length < 6) ++ return -EINVAL; ++ ++ data_size = l2_block->length - 6 /* L2 Tag length and checksum */; ++ ++ /* Verify data is within bounds. */ ++ if (((u8 *)l2_block->block_data - fw) >= fw_size || ++ ((((u8 *)l2_block->block_data) + data_size) - fw) >= fw_size) ++ return -EINVAL; ++ ++ return target->start_threads(target_data, l2_block->block_data, ++ data_size / sizeof(u32)); ++} ++ + static int + meta_ldr_cmd_config(struct drm_device *drm_dev, const u8 *fw, + struct rogue_meta_ldr_l1_data_blk *l1_data, ++ const struct pvr_meta_ldr_target *target, void *target_data, + const u32 fw_size, u32 **boot_conf_ptr) + { + struct rogue_meta_ldr_l2_data_blk *l2_block = +@@ -211,6 +267,14 @@ meta_ldr_cmd_config(struct drm_device *drm_dev, const u8 *fw, + if (config_command->type != ROGUE_META_LDR_CFG_WRITE) + return -EINVAL; + ++ if (target->config) { ++ int err = target->config(target_data, config_command->block_data[0], ++ config_command->block_data[1]); ++ ++ if (err) ++ return err; ++ } ++ + /* + * Only write to bootloader if we got a valid pointer to the FW + * code allocation. +@@ -238,43 +302,44 @@ meta_ldr_cmd_config(struct drm_device *drm_dev, const u8 *fw, + } + + /** +- * process_ldr_command_stream() - Process LDR firmware image and populate +- * firmware sections +- * @pvr_dev: Device pointer. ++ * pvr_meta_ldr_process() - Process a META LDR firmware image ++ * @drm_dev: DRM device, for messages. + * @fw: Pointer to firmware image. +- * @fw_code_ptr: Pointer to FW code section. +- * @fw_data_ptr: Pointer to FW data section. +- * @fw_core_code_ptr: Pointer to FW coremem code section. +- * @fw_core_data_ptr: Pointer to FW coremem data section. +- * @boot_conf_ptr: Pointer to boot config argument pointer. ++ * @fw_size: Size of the firmware image in bytes. ++ * @target: Where the image's memory goes. ++ * @target_data: Passed to the functions of @target. ++ * @boot_conf_ptr: Pointer to boot config argument pointer; each register write ++ * of the image's CONFIG commands is appended as a pair of ++ * words. May be %NULL. + * + * Returns : + * * 0 on success, or +- * * -EINVAL on any error in LDR command stream. ++ * * -EINVAL on any error in LDR command stream, or ++ * * Any error returned by the functions of @target. + */ +-static int +-process_ldr_command_stream(struct pvr_device *pvr_dev, const u8 *fw, u8 *fw_code_ptr, +- u8 *fw_data_ptr, u8 *fw_core_code_ptr, +- u8 *fw_core_data_ptr, u32 **boot_conf_ptr) ++int ++pvr_meta_ldr_process(struct drm_device *drm_dev, const u8 *fw, u32 fw_size, ++ const struct pvr_meta_ldr_target *target, void *target_data, ++ u32 **boot_conf_ptr) + { +- struct drm_device *drm_dev = from_pvr_device(pvr_dev); + struct rogue_meta_ldr_block_hdr *ldr_header = + (struct rogue_meta_ldr_block_hdr *)fw; +- struct rogue_meta_ldr_l1_data_blk *l1_data = +- (struct rogue_meta_ldr_l1_data_blk *)(fw + ldr_header->sl_data); +- const u32 fw_size = pvr_dev->fw_dev.firmware->size; ++ struct rogue_meta_ldr_l1_data_blk *l1_data; ++ u32 blocks = 0; + int err; + + u32 *boot_conf = boot_conf_ptr ? *boot_conf_ptr : NULL; +- u32 coremem_size; + +- err = PVR_FEATURE_VALUE(pvr_dev, meta_coremem_size, &coremem_size); +- if (err) +- return err; ++ if (fw_size < sizeof(*ldr_header)) ++ return -EINVAL; + +- coremem_size *= SZ_1K; ++ l1_data = (struct rogue_meta_ldr_l1_data_blk *)(fw + ldr_header->sl_data); + + while (l1_data) { ++ /* More blocks than the image has room for: its chain is a loop. */ ++ if (blocks++ > fw_size / sizeof(*l1_data)) ++ return -EINVAL; ++ + /* Verify block header is within bounds. */ + if (((u8 *)l1_data - fw) >= fw_size || ((u8 *)(l1_data + 1) - fw) >= fw_size) + return -EINVAL; +@@ -286,32 +351,33 @@ process_ldr_command_stream(struct pvr_device *pvr_dev, const u8 *fw, u8 *fw_code + + switch (l1_data->cmd & ROGUE_META_LDR_CMD_MASK) + case ROGUE_META_LDR_CMD_LOADMEM: { +- err = meta_ldr_cmd_loadmem(drm_dev, fw, l1_data, +- coremem_size, +- fw_code_ptr, fw_data_ptr, +- fw_core_code_ptr, +- fw_core_data_ptr, fw_size); ++ err = meta_ldr_cmd_loadmem(drm_dev, fw, l1_data, target, ++ target_data, fw_size); + if (err) + return err; + break; + + case ROGUE_META_LDR_CMD_START_THREADS: +- /* Don't process this block */ ++ /* Only for a target that starts the threads itself */ ++ if (!target->start_threads) ++ break; ++ ++ err = meta_ldr_cmd_start_threads(fw, l1_data, target, ++ target_data, fw_size); ++ if (err) ++ return err; + break; + + case ROGUE_META_LDR_CMD_ZEROMEM: +- err = meta_ldr_cmd_zeromem(drm_dev, l1_data, +- coremem_size, +- fw_code_ptr, fw_data_ptr, +- fw_core_code_ptr, +- fw_core_data_ptr); ++ err = meta_ldr_cmd_zeromem(drm_dev, l1_data, target, ++ target_data); + if (err) + return err; + break; + + case ROGUE_META_LDR_CMD_CONFIG: +- err = meta_ldr_cmd_config(drm_dev, fw, l1_data, fw_size, +- &boot_conf); ++ err = meta_ldr_cmd_config(drm_dev, fw, l1_data, target, ++ target_data, fw_size, &boot_conf); + if (err) + return err; + break; +@@ -459,6 +525,7 @@ pvr_meta_fw_process(struct pvr_device *pvr_dev, const u8 *fw, + u32 core_code_alloc_size) + { + struct pvr_fw_device *fw_dev = &pvr_dev->fw_dev; ++ struct pvr_meta_fw_sections sections; + u32 *boot_conf; + int err; + +@@ -471,8 +538,20 @@ pvr_meta_fw_process(struct pvr_device *pvr_dev, const u8 *fw, + configure_seg_mmu(pvr_dev, &boot_conf); + + /* Populate FW sections from LDR image. */ +- err = process_ldr_command_stream(pvr_dev, fw, fw_code_ptr, fw_data_ptr, fw_core_code_ptr, +- fw_core_data_ptr, &boot_conf); ++ sections.pvr_dev = pvr_dev; ++ sections.fw_code_ptr = fw_code_ptr; ++ sections.fw_data_ptr = fw_data_ptr; ++ sections.fw_core_code_ptr = fw_core_code_ptr; ++ sections.fw_core_data_ptr = fw_core_data_ptr; ++ ++ err = PVR_FEATURE_VALUE(pvr_dev, meta_coremem_size, §ions.coremem_size); ++ if (err) ++ return err; ++ ++ sections.coremem_size *= SZ_1K; ++ ++ err = pvr_meta_ldr_process(from_pvr_device(pvr_dev), fw, pvr_dev->fw_dev.firmware->size, ++ &pvr_meta_fw_sections_target, §ions, &boot_conf); + if (err) + return err; + +diff --git a/drivers/gpu/drm/imagination/pvr_fw_meta.h b/drivers/gpu/drm/imagination/pvr_fw_meta.h +index 911ad70..942e470 100644 +--- a/drivers/gpu/drm/imagination/pvr_fw_meta.h ++++ b/drivers/gpu/drm/imagination/pvr_fw_meta.h +@@ -9,6 +9,29 @@ + /* Forward declaration from pvr_device.h */ + struct pvr_device; + ++struct drm_device; ++ ++/** ++ * struct pvr_meta_ldr_target - Where the contents of a META LDR image go ++ * @mem: For a LOADMEM or ZEROMEM (@zero) command: store in @write_addr where ++ * the @size bytes at META address @addr are kept, or %NULL to leave the ++ * command out. Returns 0 or a negative error for an address the target ++ * has no place for. ++ * @config: For each register write of a CONFIG command. Returns 0 or a ++ * negative error. May be %NULL. ++ * @start_threads: For a START_THREADS command, with its @count words. Returns ++ * 0 or a negative error. May be %NULL: the command is skipped. ++ */ ++struct pvr_meta_ldr_target { ++ int (*mem)(void *data, u32 addr, u32 size, bool zero, void **write_addr); ++ int (*config)(void *data, u32 reg, u32 value); ++ int (*start_threads)(void *data, const u32 *words, u32 count); ++}; ++ + int pvr_meta_cr_read32(struct pvr_device *pvr_dev, u32 reg_addr, u32 *reg_value_out); + ++int pvr_meta_ldr_process(struct drm_device *drm_dev, const u8 *fw, u32 fw_size, ++ const struct pvr_meta_ldr_target *target, void *target_data, ++ u32 **boot_conf_ptr); ++ + #endif /* PVR_FW_META_H */ diff --git a/kernel/patches/drivers/gpu/0018-drm-imagination-Map-cache-coherent-objects-and-regis.patch b/kernel/patches/drivers/gpu/0018-drm-imagination-Map-cache-coherent-objects-and-regis.patch new file mode 100644 index 00000000..ba298ba3 --- /dev/null +++ b/kernel/patches/drivers/gpu/0018-drm-imagination-Map-cache-coherent-objects-and-regis.patch @@ -0,0 +1,181 @@ +From 0000000000000000000000000000000000000000 Mon Sep 17 00:00:00 2001 +From: Henrik Klee +Date: Sat, 14 Nov 2026 23:28:12 +0800 +Subject: [PATCH] drm/imagination: Map cache-coherent objects and registers for + a firmware + +Two things Apple's A9 firmware needs in its page tables, which are the +kernel VM context's: + +Its shared memory is mapped cache coherent (the CC bit of the page table +entries), as Apple's driver maps it. A kernel-only object flag, +PVR_BO_DEVICE_CACHE_COHERENT, asks for it; userspace cannot set it, and +no existing object has it. + +It reaches registers outside the GPU (the chip's power block, the +interrupt controller) through its own address space. pvr_vm_map_io() +maps such a range into the kernel VM context, for the firmware only and +past the device cache; pvr_vm_unmap_io() removes it. No memory object +stands behind it, so the VM does not track it and the caller removes it +before the context goes. + +Assisted-by: LLM +--- + drivers/gpu/drm/imagination/pvr_gem.h | 11 +++- + drivers/gpu/drm/imagination/pvr_mmu.c | 2 +- + drivers/gpu/drm/imagination/pvr_vm.c | 80 +++++++++++++++++++++++++++ + drivers/gpu/drm/imagination/pvr_vm.h | 2 + + 4 files changed, 92 insertions(+), 3 deletions(-) + +diff --git a/drivers/gpu/drm/imagination/pvr_gem.h b/drivers/gpu/drm/imagination/pvr_gem.h +index 4286e04..efcd270 100644 +--- a/drivers/gpu/drm/imagination/pvr_gem.h ++++ b/drivers/gpu/drm/imagination/pvr_gem.h +@@ -52,14 +52,21 @@ struct pvr_file; + * Firmware options + * :PVR_BO_FW_NO_CLEAR_ON_RESET: By default, all FW objects are cleared and reinitialised on hard + * reset. Set this flag to override this behaviour and preserve buffer contents on reset. ++ * ++ * Device mapping options ++ * :PVR_BO_DEVICE_CACHE_COHERENT: Map the object's pages as cache coherent on the device (the ++ * CC bit of their page table entries). By default it is clear. + */ + #define PVR_BO_CPU_CACHED BIT_ULL(63) + + #define PVR_BO_FW_NO_CLEAR_ON_RESET BIT_ULL(62) + +-#define PVR_BO_KERNEL_FLAGS_MASK (PVR_BO_CPU_CACHED | PVR_BO_FW_NO_CLEAR_ON_RESET) ++#define PVR_BO_DEVICE_CACHE_COHERENT BIT_ULL(61) ++ ++#define PVR_BO_KERNEL_FLAGS_MASK (PVR_BO_CPU_CACHED | PVR_BO_FW_NO_CLEAR_ON_RESET | \ ++ PVR_BO_DEVICE_CACHE_COHERENT) + +-/* Bits 61..3 are undefined. */ ++/* Bits 60..3 are undefined. */ + /* Bits 2..0 are defined in the UAPI. */ + + /* Other utilities. */ +diff --git a/drivers/gpu/drm/imagination/pvr_mmu.c b/drivers/gpu/drm/imagination/pvr_mmu.c +index 72fe583..a4996b9 100644 +--- a/drivers/gpu/drm/imagination/pvr_mmu.c ++++ b/drivers/gpu/drm/imagination/pvr_mmu.c +@@ -2607,7 +2607,7 @@ int pvr_mmu_map(struct pvr_mmu_op_context *op_ctx, u64 size, u64 flags, + + memcpy(&ptr_copy, &op_ctx->curr_page, sizeof(ptr_copy)); + +- flags_raw = pvr_page_flags_raw_create(false, false, ++ flags_raw = pvr_page_flags_raw_create(false, flags & PVR_BO_DEVICE_CACHE_COHERENT, + flags & DRM_PVR_BO_BYPASS_DEVICE_CACHE, + flags & DRM_PVR_BO_PM_FW_PROTECT); + +diff --git a/drivers/gpu/drm/imagination/pvr_vm.c b/drivers/gpu/drm/imagination/pvr_vm.c +index 7353dbd..bad2742 100644 +--- a/drivers/gpu/drm/imagination/pvr_vm.c ++++ b/drivers/gpu/drm/imagination/pvr_vm.c +@@ -26,6 +26,7 @@ + #include + #include + #include ++#include + #include + + /** +@@ -940,6 +941,85 @@ pvr_vm_unmap_obj(struct pvr_vm_context *vm_ctx, struct pvr_gem_object *pvr_obj, + return err; + } + ++/** ++ * pvr_vm_map_io() - Map device registers into the kernel VM context ++ * @vm_ctx: The kernel VM context. ++ * @io_addr: Bus address of the registers. ++ * @device_addr: Virtual device address at the start of the mapping. ++ * @size: Size of the mapping. ++ * ++ * For a firmware that reaches registers outside the GPU through its own page ++ * tables. The pages are mapped for the firmware only and bypass the device ++ * cache. No memory object stands behind the mapping, so it is not tracked: ++ * the caller keeps the range free of other mappings and removes it with ++ * pvr_vm_unmap_io() before the context goes. ++ * ++ * Return: ++ * * 0 on success, ++ * * -%EINVAL for a userspace context or an address or size that is not ++ * device page-aligned, or ++ * * Any error encountered while building the page tables. ++ */ ++int ++pvr_vm_map_io(struct pvr_vm_context *vm_ctx, phys_addr_t io_addr, u64 device_addr, u64 size) ++{ ++ struct pvr_mmu_op_context *op_ctx; ++ struct sg_table sgt; ++ int err; ++ ++ if (vm_ctx->is_userspace_context || !size || ++ ((io_addr | device_addr | size) & ~PVR_DEVICE_PAGE_MASK)) ++ return -EINVAL; ++ ++ err = sg_alloc_table(&sgt, 1, GFP_KERNEL); ++ if (err) ++ return err; ++ ++ sg_dma_address(sgt.sgl) = io_addr; ++ sg_dma_len(sgt.sgl) = size; ++ ++ op_ctx = pvr_mmu_op_context_create(vm_ctx->mmu_ctx, &sgt, 0, size); ++ if (IS_ERR(op_ctx)) { ++ err = PTR_ERR(op_ctx); ++ goto err_free_table; ++ } ++ ++ mutex_lock(&vm_ctx->lock); ++ err = pvr_mmu_map(op_ctx, size, ++ DRM_PVR_BO_BYPASS_DEVICE_CACHE | DRM_PVR_BO_PM_FW_PROTECT, ++ device_addr); ++ mutex_unlock(&vm_ctx->lock); ++ ++ pvr_mmu_op_context_destroy(op_ctx); ++ ++err_free_table: ++ sg_free_table(&sgt); ++ ++ return err; ++} ++ ++/** ++ * pvr_vm_unmap_io() - Remove a mapping made with pvr_vm_map_io() ++ * @vm_ctx: The kernel VM context. ++ * @device_addr: Virtual device address at the start of the mapping. ++ * @size: Size of the mapping. ++ */ ++void ++pvr_vm_unmap_io(struct pvr_vm_context *vm_ctx, u64 device_addr, u64 size) ++{ ++ struct pvr_mmu_op_context *op_ctx; ++ ++ op_ctx = pvr_mmu_op_context_create(vm_ctx->mmu_ctx, NULL, 0, 0); ++ if (WARN_ON(IS_ERR(op_ctx))) ++ return; ++ ++ mutex_lock(&vm_ctx->lock); ++ WARN_ON(pvr_mmu_unmap(op_ctx, device_addr, size)); ++ mutex_unlock(&vm_ctx->lock); ++ ++ pvr_mmu_op_context_destroy(op_ctx); ++} ++ + /** + * pvr_vm_unmap() - Unmap an already mapped section of device-virtual memory. + * @vm_ctx: Target VM context. +diff --git a/drivers/gpu/drm/imagination/pvr_vm.h b/drivers/gpu/drm/imagination/pvr_vm.h +index d3b3f3f..f7a4f1c 100644 +--- a/drivers/gpu/drm/imagination/pvr_vm.h ++++ b/drivers/gpu/drm/imagination/pvr_vm.h +@@ -59,6 +59,8 @@ int pvr_vm_unmap_obj(struct pvr_vm_context *vm_ctx, + struct pvr_gem_object *pvr_obj, + u64 device_addr, u64 size); + int pvr_vm_unmap(struct pvr_vm_context *vm_ctx, u64 device_addr, u64 size); ++int pvr_vm_map_io(struct pvr_vm_context *vm_ctx, phys_addr_t io_addr, u64 device_addr, u64 size); ++void pvr_vm_unmap_io(struct pvr_vm_context *vm_ctx, u64 device_addr, u64 size); + void pvr_vm_unmap_all(struct pvr_vm_context *vm_ctx); + + dma_addr_t pvr_vm_get_page_table_root_addr(struct pvr_vm_context *vm_ctx); diff --git a/kernel/patches/drivers/gpu/0019-drm-imagination-apple-Allow-a-chip-whose-firmware-ta.patch b/kernel/patches/drivers/gpu/0019-drm-imagination-apple-Allow-a-chip-whose-firmware-ta.patch new file mode 100644 index 00000000..6d796708 --- /dev/null +++ b/kernel/patches/drivers/gpu/0019-drm-imagination-apple-Allow-a-chip-whose-firmware-ta.patch @@ -0,0 +1,193 @@ +From 0000000000000000000000000000000000000000 Mon Sep 17 00:00:00 2001 +From: Henrik Klee +Date: Sat, 14 Nov 2026 23:28:12 +0800 +Subject: [PATCH] drm/imagination: apple: Allow a chip whose firmware takes no + jobs yet + +For a chip that is brought up in steps: its firmware is started and +stopped before its commands are written. + +Such a chip leaves out the job functions. No context is created on it +(-EOPNOTSUPP), so nothing can be submitted and no client takes it for a +GPU that renders. + +A chip may also have a fini(): called when the device goes, after the +firmware stopped and while the kernel VM context still exists, and after +a first start that failed. A chip whose firmware lives in the kernel VM +unmaps it there. + +The power state the core starts in gets its own function: a chip that +tells its firmware the state before it powers the core asks for it. + +The A10 has every job function and no fini(): nothing changes for it. + +Assisted-by: LLM +--- + drivers/gpu/drm/imagination/pvr_apple.c | 47 +++++++++++++++----- + drivers/gpu/drm/imagination/pvr_apple.h | 2 + + drivers/gpu/drm/imagination/pvr_apple_chip.h | 9 ++++ + drivers/gpu/drm/imagination/pvr_context.c | 4 ++ + 4 files changed, 52 insertions(+), 10 deletions(-) + +diff --git a/drivers/gpu/drm/imagination/pvr_apple.c b/drivers/gpu/drm/imagination/pvr_apple.c +index 2039928..b64ff4e 100644 +--- a/drivers/gpu/drm/imagination/pvr_apple.c ++++ b/drivers/gpu/drm/imagination/pvr_apple.c +@@ -88,6 +88,19 @@ int pvr_apple_request_power_state(struct pvr_apple *apple, u32 state) + return err; + } + ++/** ++ * pvr_apple_start_power_state() - The power state the core starts in ++ * @apple: Apple firmware state. ++ * ++ * Return: the fixed state if one is set, else the fastest. Never state 0: ++ * that is off, and the firmware is about to start. ++ */ ++u32 pvr_apple_start_power_state(struct pvr_apple *apple) ++{ ++ return clamp(pvr_apple_perf_state ?: apple->perf_state_max, 1u, ++ apple->perf_state_max ?: 1u); ++} ++ + /** + * pvr_apple_fender_power_on() - Power the Rogue core up + * @apple: Apple firmware state. +@@ -114,9 +127,7 @@ int pvr_apple_fender_power_on(struct pvr_apple *apple) + * from there. + */ + mutex_lock(&apple->perf_lock); +- /* never state 0 here: that is off, and the firmware is about to start */ +- err = pvr_apple_request_power_state(apple, clamp(pvr_apple_perf_state ?: apple->perf_state_max, +- 1u, apple->perf_state_max ?: 1u)); ++ err = pvr_apple_request_power_state(apple, pvr_apple_start_power_state(apple)); + mutex_unlock(&apple->perf_lock); + return err; + } +@@ -1260,6 +1271,8 @@ int pvr_apple_fw_init(struct pvr_device *pvr_dev) + return err; + err = pvr_apple_fw_start(apple); + if (err) { ++ if (apple->chip->fini) ++ apple->chip->fini(apple); + pvr_apple_gpu_pages_fini(apple); + return err; + } +@@ -1455,13 +1468,15 @@ void pvr_apple_fw_fini(struct pvr_device *pvr_dev) + pvr_gem_object_put(apple->pm_pages); + apple->pm_pages = NULL; + } +- if (!apple->booted) +- return; +- apple->chip->stop(apple, false); +- /* the firmware's messages until its stop queued completion checks */ +- cancel_delayed_work_sync(&apple->poll_work); +- hrtimer_cancel(&apple->poll_timer); +- apple->booted = false; ++ if (apple->booted) { ++ apple->chip->stop(apple, false); ++ /* the firmware's messages until its stop queued completion checks */ ++ cancel_delayed_work_sync(&apple->poll_work); ++ hrtimer_cancel(&apple->poll_timer); ++ apple->booted = false; ++ } ++ if (apple->chip->fini) ++ apple->chip->fini(apple); + } + + /* +@@ -1488,6 +1503,18 @@ void pvr_apple_device_info_init(struct pvr_device *pvr_dev) + pvr_dev->features.isp_samples_per_pixel = 2; + } + ++/** ++ * pvr_apple_takes_jobs() - Whether the chip's firmware is given jobs ++ * @pvr_dev: Target device. ++ * ++ * Return: %false for a chip whose firmware is started and stopped, but whose ++ * commands are not written yet: it gets no contexts. ++ */ ++bool pvr_apple_takes_jobs(struct pvr_device *pvr_dev) ++{ ++ return pvr_dev->device_data->apple_chip->write_compute != NULL; ++} ++ + /** + * pvr_apple_cmd_stream() - How a render job's command stream is decoded + * @pvr_dev: Target device. +diff --git a/drivers/gpu/drm/imagination/pvr_apple.h b/drivers/gpu/drm/imagination/pvr_apple.h +index 90aeaae..356d930 100644 +--- a/drivers/gpu/drm/imagination/pvr_apple.h ++++ b/drivers/gpu/drm/imagination/pvr_apple.h +@@ -93,6 +93,7 @@ int pvr_apple_reset(struct pvr_device *pvr_dev); + int pvr_apple_suspend(struct pvr_device *pvr_dev); + int pvr_apple_resume(struct pvr_device *pvr_dev); + void pvr_apple_device_info_init(struct pvr_device *pvr_dev); ++bool pvr_apple_takes_jobs(struct pvr_device *pvr_dev); + struct dma_fence *pvr_apple_submit_compute_job(struct pvr_device *pvr_dev, + struct pvr_vm_context *vm_ctx, u64 stream, + u64 ctx_state); +@@ -123,6 +124,7 @@ static inline int pvr_apple_reset(struct pvr_device *pvr_dev) { return -ENODEV; + static inline int pvr_apple_suspend(struct pvr_device *pvr_dev) { return 0; } + static inline int pvr_apple_resume(struct pvr_device *pvr_dev) { return 0; } + static inline void pvr_apple_device_info_init(struct pvr_device *pvr_dev) {} ++static inline bool pvr_apple_takes_jobs(struct pvr_device *pvr_dev) { return false; } + static inline int pvr_apple_vm_init(struct pvr_device *pvr_dev, struct pvr_vm_context *vm_ctx) + { + return -ENODEV; +diff --git a/drivers/gpu/drm/imagination/pvr_apple_chip.h b/drivers/gpu/drm/imagination/pvr_apple_chip.h +index 9f399ea..168338a 100644 +--- a/drivers/gpu/drm/imagination/pvr_apple_chip.h ++++ b/drivers/gpu/drm/imagination/pvr_apple_chip.h +@@ -172,6 +172,9 @@ struct pvr_apple { + * the processor ready for the next @boot regardless. + * @idle: After @stop for runtime suspend: give up what would keep the GPU's + * power domain on. ++ * @fini: When the device goes, after @stop, with the kernel VM still there: ++ * give up what @start made in it. Also after a first @start that failed. ++ * Optional. + * @restore_image: Before @boot, when the firmware ran before: put its memory + * back as @init left it. + * @kick: Make the firmware look at its work rings. +@@ -184,6 +187,10 @@ struct pvr_apple { + * @write_frag: Write fragment job @seq and its work-ring entry. + * @report_hang: Log what else the firmware recorded about a job that timed out. + * ++ * A chip without @write_compute takes no jobs: its firmware is started and ++ * stopped, no context is created on it (pvr_apple_takes_jobs()), and @kick, ++ * the job functions and the streams are not used. ++ * + * pvr_apple.c calls @boot, @start, @stop, @restore_image, @kick and the + * write functions with &pvr_apple.submit_lock held, except @boot and @start + * at probe and @stop when the device goes. It publishes the ring's producer index +@@ -202,6 +209,7 @@ struct pvr_apple_chip { + int (*start)(struct pvr_apple *apple); + void (*stop)(struct pvr_apple *apple, bool recover); + void (*idle)(struct pvr_apple *apple); ++ void (*fini)(struct pvr_apple *apple); + void (*restore_image)(struct pvr_apple *apple); + int (*kick)(struct pvr_apple *apple); + void (*write_compute)(struct pvr_apple *apple, u32 seq, u64 stream, u64 ctx_state); +@@ -225,6 +233,7 @@ static inline struct device *apple_dev(struct pvr_apple *apple) + + int pvr_apple_fw_load(struct pvr_apple *apple, size_t max_size); + int pvr_apple_request_power_state(struct pvr_apple *apple, u32 state); ++u32 pvr_apple_start_power_state(struct pvr_apple *apple); + int pvr_apple_fender_power_on(struct pvr_apple *apple); + + #endif /* PVR_APPLE_CHIP_H */ +diff --git a/drivers/gpu/drm/imagination/pvr_context.c b/drivers/gpu/drm/imagination/pvr_context.c +index 8e4a7fa..508a1cd 100644 +--- a/drivers/gpu/drm/imagination/pvr_context.c ++++ b/drivers/gpu/drm/imagination/pvr_context.c +@@ -296,6 +296,10 @@ int pvr_context_create(struct pvr_file *pvr_file, struct drm_pvr_ioctl_create_co + args->type != DRM_PVR_CTX_TYPE_RENDER) + return -EOPNOTSUPP; + ++ /* ... on the chips whose commands the driver writes. */ ++ if (pvr_device_is_apple(pvr_dev) && !pvr_apple_takes_jobs(pvr_dev)) ++ return -EOPNOTSUPP; ++ + ctx_size = get_fw_obj_size(args->type); + if (ctx_size < 0) + return ctx_size; diff --git a/kernel/patches/drivers/gpu/0020-drm-imagination-apple-Boot-the-A9-S8003-GPU-firmware.patch b/kernel/patches/drivers/gpu/0020-drm-imagination-apple-Boot-the-A9-S8003-GPU-firmware.patch new file mode 100644 index 00000000..a7fc9570 --- /dev/null +++ b/kernel/patches/drivers/gpu/0020-drm-imagination-apple-Boot-the-A9-S8003-GPU-firmware.patch @@ -0,0 +1,1350 @@ +From 0000000000000000000000000000000000000000 Mon Sep 17 00:00:00 2001 +From: =?UTF-8?q?Berkay=20K=C3=B6sebay?= +Date: Sat, 14 Nov 2026 23:28:38 +0800 +Subject: [PATCH] drm/imagination: apple: Boot the A9 (S8003) GPU firmware + +The iPad 5's GPU is a Rogue GT7600 whose firmware, Apple's, runs on the +core's own META processor. The powervr driver now takes the A9's node +(apple,s8003-gpu) next to the A10's and starts that firmware: + +- It unpacks the device's own firmware file, a META LDR image, with the + driver's LDR walker into the block Apple's bootloader expects, and adds + the bootloader's configuration. +- The firmware's page tables are the kernel VM context's: the image, its + root and shared memory and the registers it uses are mapped there, at + the firmware's addresses. The bus addresses of those registers (Fender, + the Rogue core, the interrupt controller) and the number of its + interrupt come from the device tree. +- The registers of a META boot go into Fender's boot stream; the core is + powered up through Fender, and the driver waits for the firmware's + "alive" event, then gives it an empty device-control command and waits + until the firmware took it. +- Stopping is Fender's power state 0; runtime PM stops the firmware a + second after its last use and starts it again from the kept image, as + on the A10. + +One boot path: Apple's unmodified image, the governor pinned to the +state the core is powered up in (the fastest, 4, unless the +apple_perf_state parameter names another). + +The firmware's commands are not written yet, so no context can be +created and nothing renders. The core counts as experimental: the driver +binds only with powervr.exp_hw_support=1. Without it the probe wakes +Fender, reads the core's ID and powers the GPU off again. + +The firmware's interrupt to the host is not taken; the driver reads the +firmware's rings on a timer while it runs. + +[Orchard: ported from orchard PR #74 (apple-g5p boot.inc, boot-layout.h, +pack.h, fender.h, perf-envelope.h, lifecycle.inc), reshaped as a chip of +the powervr driver's Apple firmware path. Not ported: see +docs/bring-up/gpu/a9-g5p.md in orchard.] + +Assisted-by: LLM +--- + drivers/gpu/drm/imagination/Kconfig | 6 +- + drivers/gpu/drm/imagination/Makefile | 1 + + drivers/gpu/drm/imagination/pvr_apple.c | 3 +- + drivers/gpu/drm/imagination/pvr_apple.h | 1 + + drivers/gpu/drm/imagination/pvr_apple_s8003.c | 1189 +++++++++++++++++ + drivers/gpu/drm/imagination/pvr_device.c | 4 + + drivers/gpu/drm/imagination/pvr_drv.c | 10 + + 7 files changed, 1211 insertions(+), 3 deletions(-) + create mode 100644 drivers/gpu/drm/imagination/pvr_apple_s8003.c + +diff --git a/drivers/gpu/drm/imagination/Kconfig b/drivers/gpu/drm/imagination/Kconfig +index d7f65a9..515a31b 100644 +--- a/drivers/gpu/drm/imagination/Kconfig ++++ b/drivers/gpu/drm/imagination/Kconfig +@@ -20,14 +20,16 @@ config DRM_POWERVR + If "M" is selected, the module will be called powervr. + + config DRM_POWERVR_APPLE +- bool "Apple A10 (T8010) GPU support" ++ bool "Apple A9 (S8003) and A10 (T8010) GPU support" + depends on DRM_POWERVR + depends on APPLE_RTKIT && (ARCH_APPLE || COMPILE_TEST) + depends on DRM_POWERVR=m || APPLE_RTKIT=y + help + Support the Rogue-based GPU of the Apple A10 (T8010), which runs + Apple's GFX firmware on an RTKit coprocessor instead of +- Imagination's firmware. ++ Imagination's firmware, and start the firmware of the Apple A9 ++ (S8003), which runs on the GPU's own processor. The A9's GPU does ++ not render yet. + + If in doubt, say "N". + +diff --git a/drivers/gpu/drm/imagination/Makefile b/drivers/gpu/drm/imagination/Makefile +index e991b67..08181ce 100644 +--- a/drivers/gpu/drm/imagination/Makefile ++++ b/drivers/gpu/drm/imagination/Makefile +@@ -33,6 +33,7 @@ powervr-$(CONFIG_DRM_POWERVR_APPLE) += \ + pvr_apple.o \ + pvr_apple_layout.o \ + pvr_apple_queue.o \ ++ pvr_apple_s8003.o \ + pvr_apple_t8010.o + + powervr-$(CONFIG_DEBUG_FS) += \ +diff --git a/drivers/gpu/drm/imagination/pvr_apple.c b/drivers/gpu/drm/imagination/pvr_apple.c +index b64ff4e..0d9c2e5 100644 +--- a/drivers/gpu/drm/imagination/pvr_apple.c ++++ b/drivers/gpu/drm/imagination/pvr_apple.c +@@ -8,7 +8,8 @@ + * and fragment jobs, the GPU's power and performance state (Fender), runtime + * PM, sleep and recovery. The processor the firmware runs on, its memory + * layout and its commands belong to the chip (struct pvr_apple_chip, +- * pvr_apple_chip.h): pvr_apple_t8010.c for the A10. ++ * pvr_apple_chip.h): pvr_apple_t8010.c for the A10, pvr_apple_s8003.c for the ++ * A9. + */ + + #include "pvr_apple.h" +diff --git a/drivers/gpu/drm/imagination/pvr_apple.h b/drivers/gpu/drm/imagination/pvr_apple.h +index 356d930..a7f697c 100644 +--- a/drivers/gpu/drm/imagination/pvr_apple.h ++++ b/drivers/gpu/drm/imagination/pvr_apple.h +@@ -22,6 +22,7 @@ struct pvr_stream_cmd_defs; + struct pvr_vm_context; + + extern const struct pvr_power_sequence_ops pvr_power_sequence_ops_apple; ++extern const struct pvr_apple_chip pvr_apple_chip_s8003; + extern const struct pvr_apple_chip pvr_apple_chip_t8010; + + /* +diff --git a/drivers/gpu/drm/imagination/pvr_apple_s8003.c b/drivers/gpu/drm/imagination/pvr_apple_s8003.c +new file mode 100644 +index 0000000..9b84009 +--- /dev/null ++++ b/drivers/gpu/drm/imagination/pvr_apple_s8003.c +@@ -0,0 +1,1189 @@ ++// SPDX-License-Identifier: GPL-2.0-only OR MIT ++/* ++ * Apple A9 (S8003): the GFX firmware on the Rogue core's own META processor. ++ * ++ * The A9 has no coprocessor for its GPU firmware: Apple's firmware is a META ++ * LDR image, as Imagination's is, and runs on the META inside the GT7600. The ++ * host does what Apple's kext does: ++ * ++ * - unpack the image into one block of GPU memory, with the bootloader's ++ * configuration (the segment MMU and the image's own register writes) at ++ * its fixed place in it; ++ * - map that block, the memory shared with the firmware ("root" and ++ * "shared": its init data, rings and control words) and the registers the ++ * firmware uses (Fender, the interrupt controller) at the firmware's ++ * addresses, in the kernel VM context: the firmware's page tables are the ++ * kernel VM's, and the image and the shared memory lie where powervr puts ++ * IMG's firmware heap; ++ * - store the Rogue registers of a META boot (reset, boot mode, page-table ++ * root, cache setup, reset release) in Fender's boot stream: Fender ++ * replays it into the core when the core is powered up; ++ * - power the core up and wait until the firmware posts its first event. ++ * ++ * The META slave port is not used: the firmware boots itself from the memory ++ * mapped for it. ++ * ++ * Not here yet: the firmware's commands. No job is written, so the chip takes ++ * none (pvr_apple_takes_jobs()), and the firmware's interrupt to the host is ++ * not taken: its rings are read on a timer while it runs. ++ * ++ * The boot sequence, the layout of the shared memory and every value in it ++ * are from Berkay Kösebay's G5P driver, which runs the GPU on the iPad 5 ++ * (J71t); the names of the firmware's objects are his readings of Apple's ++ * kext and firmware. ++ */ ++ ++#include "pvr_apple.h" ++#include "pvr_apple_chip.h" ++#include "pvr_device.h" ++#include "pvr_fw_meta.h" ++#include "pvr_gem.h" ++#include "pvr_mmu.h" ++#include "pvr_rogue_cr_defs.h" ++#include "pvr_rogue_meta.h" ++#include "pvr_vm.h" ++ ++#include ++#include ++#include ++#include ++#include ++#include ++#include ++#include ++#include ++#include ++#include ++#include ++#include ++#include ++ ++/* ++ * Fender beyond the power state. The boot stream is a list of Rogue register ++ * writes, two words each; Fender runs it when the core's power state leaves 0. ++ */ ++#define FENDER_CTRL 0x0000 ++#define FENDER_CTRL_BOOT 0x40000000 /* as iOS sets it before a boot */ ++#define FENDER_STREAM_STATE 0x0010 /* cleared before the stream is stored */ ++#define FENDER_WAKE 0x0014 ++#define FENDER_WAKE_ALL 0x7 /* before every doorbell */ ++#define FENDER_BOOT_2010 0x2010 /* iOS's values before a boot; */ ++#define FENDER_BOOT_2014 0x2014 /* their meaning is not known */ ++#define FENDER_STREAM 0x10000 ++#define FENDER_STREAM_SIZE 0x200 ++#define FENDER_STREAM_END 0xffff0000 ++#define FENDER_CORE 0x20000 ++#define FENDER_CORE_HOLD BIT(5) /* keep the core in reset */ ++ ++#define S8003_PERF_STATE_MAX 4 ++ ++/* Rogue registers of the boot stream that powervr has no name for */ ++#define S8003_CR_E160 0xe160 /* takes the address of a spare page */ ++ ++/* SOFT_RESET's first two values, as iOS sequences them */ ++#define S8003_SOFT_RESET_ALL 0x00e7fffffffffc1dull ++#define S8003_SOFT_RESET_CORE 0x00e7fe071ffffc1dull ++#define S8003_SLC3_SCRAMBLE 0xb42dd24b4bd22db4ull ++ ++/* The interrupt controller's registers the firmware uses, from its base */ ++#define AIC_TIME 0x1020 ++#define AIC_SW_SET 0x4000 ++ ++/* ++ * The firmware's GPU addresses: all in the GiB powervr's kernel VM keeps for ++ * a firmware (IMG's bootloader is at the same address, ++ * ROGUE_FW_BOOTLDR_DEVV_ADDR). ++ */ ++#define S8003_IMAGE_VA 0xe1c0000000ull ++#define S8003_SHARED_VA 0xe1c0200000ull ++#define S8003_ROOT_VA 0xe1c3200000ull ++#define S8003_PIO_VA 0xe1c4200000ull ++ ++#define S8003_IMAGE_SIZE 0x30000 ++#define S8003_ROOT_SIZE 0x8000 ++#define S8003_SHARED_SIZE 0x60000 ++#define S8003_PIO_FENDER 0x0000 /* offsets from S8003_PIO_VA */ ++#define S8003_PIO_AIC 0x8000 ++#define S8003_PIO_SIZE 0x8000 /* of each */ ++ ++/* The unpacked image: the bootloader's configuration and what it hands over */ ++#define S8003_IMAGE_CONF 0x200 ++#define S8003_IMAGE_HANDOVER 0x3ff8 /* META addresses of INIT and CONTROL */ ++#define S8003_IMAGE_COREMEM 0x24000 /* core memory's contents */ ++#define S8003_IMAGE_COREMEM_SIZE 0x55dc ++#define S8003_LDR_CONF_MAX 27 /* register writes in Apple's image */ ++ ++/* ++ * The objects of the root and shared memory, by the number Apple's kext ++ * gives each (it numbers every object twice, for its CPU and its GPU address; ++ * these are the CPU-side numbers). Those without a name here are placed and ++ * announced to the firmware, and left zero. ++ */ ++enum { ++ S8003_INIT = 0x44, /* what the firmware reads first */ ++ S8003_CHANNEL = 0x47, /* three: 0x47, 0x48, 0x49 */ ++ S8003_POLICY = 0x4a, /* the power governor's record */ ++ S8003_PLATFORM = 0x4d, /* registers, clocks, interrupt, thread */ ++ S8003_CONTROL = 0x5e, ++ S8003_WORK_CTL = 0x60, /* work rings: indices 0x60, 0x64, 0x68, */ ++ S8003_WORK_RING = 0x61, /* entries 0x61, 0x65, 0x69 */ ++ S8003_DC_CTL = 0x6c, /* device-control ring */ ++ S8003_DC_RING = 0x6d, ++ S8003_EVENT_CTL = 0x70, ++ S8003_LOG_CTL = 0x71, /* four lanes */ ++ S8003_STATS_CTL = 0x72, ++ S8003_TRACE_CTL = 0x73, ++ S8003_EVENTS = 0x74, ++ S8003_LOGS = 0x75, ++ S8003_STATS = 0x76, ++ S8003_TRACE = 0x77, ++ S8003_MMU_ROOT = 0x86, /* the address of the page catalogue */ ++ S8003_OBJECTS = 0x90, ++}; ++ ++/* A ring's indices: the consumer's, 16 bytes on the producer's */ ++#define S8003_RING_CONSUMER 0x00 ++#define S8003_RING_PRODUCER 0x10 ++#define S8003_RING_CTL_SIZE 0x20 /* one lane's */ ++#define S8003_EVENT_SIZE 44 ++#define S8003_DC_SIZE 20 ++ ++/* Events: the first word */ ++#define S8003_EVENT_ALIVE 0 ++#define S8003_EVENT_STAMP 1 ++ ++/* The device-control command that carries nothing, and the ring's doorbell */ ++#define S8003_DC_NONE 8 ++#define S8003_MTS_DEVICE_CONTROL (ROGUE_CR_MTS_SCHEDULE_TASK_COUNTED | ROGUE_CR_MTS_SCHEDULE_DM_DM0) ++ ++#define S8003_ALIVE_TIMEOUT_US 3000000 ++#define S8003_DC_TIMEOUT_US 3000000 ++/* ++ * The firmware posts an "alive" event each time its scheduler runs, about 290 ++ * a second, idle or not (measured on the iPad 5): read the rings well before ++ * their 256 entries are used. ++ */ ++#define S8003_DRAIN_MS 100 ++ ++struct s8003_layout { ++ u32 offset[S8003_OBJECTS]; ++ u8 shared[S8003_OBJECTS]; /* in the shared memory, not the root */ ++}; ++ ++struct s8003_mem { ++ struct pvr_gem_object *obj; ++ u8 *map; ++ u64 device_addr; ++ size_t size; ++ bool mapped; ++}; ++ ++/* ++ * What pvr_apple.c reads from the memory shared with the firmware: the ++ * engines' stamps and ring indices, GPU context 0's MMU words, the event ++ * ring's indices. Where the A9's firmware keeps them is settled with its ++ * commands; until then they are here, zero: nothing submitted, nothing done. ++ */ ++struct s8003_idle { ++ __le32 stamp[APPLE_ENGINES]; ++ __le32 producer[APPLE_ENGINES]; ++ __le32 frag_ring; ++ __le32 frag_retired; ++ __le64 mmu_root; ++ __le32 mmu_gen; ++ __le32 event_consumer; ++ __le32 event_producer; ++}; ++ ++struct pvr_apple_s8003 { ++ struct pvr_apple *apple; ++ struct delayed_work drain_work; /* reads the rings while the firmware runs */ ++ ++ /* Bus addresses the firmware is told: its registers' and ours */ ++ phys_addr_t fender_phys; ++ phys_addr_t rgx_phys; ++ phys_addr_t aic_phys; ++ u32 irq; /* the firmware's interrupt to the host, as the AIC numbers it */ ++ ++ struct s8003_layout layout; ++ u8 *image; /* the unpacked image, as the bootloader finds it */ ++ u32 conf[S8003_LDR_CONF_MAX][2]; ++ u32 conf_count; ++ u32 thread_sp; ++ u32 thread_pc; ++ ++ struct s8003_mem fw; /* the image the firmware runs in */ ++ struct s8003_mem root; ++ struct s8003_mem shared; ++ struct s8003_mem zero; /* at GPU address 0 */ ++ struct pvr_gem_object *spare; ++ bool pio_mapped; ++ ++ struct s8003_idle idle; ++}; ++ ++static void put32(u8 *p, u32 value) ++{ ++ put_unaligned_le32(value, p); ++} ++ ++static void put64(u8 *p, u64 value) ++{ ++ put_unaligned_le64(value, p); ++} ++ ++/* ++ * The firmware's 32-bit address of a GPU address in its GiB. The root memory ++ * is reached through META's uncached range, the rest through its cached one. ++ */ ++static u32 s8003_meta_addr(u64 va) ++{ ++ u32 range = va >= S8003_ROOT_VA && va < S8003_ROOT_VA + SZ_16M ? 0x40000000 : 0xc0000000; ++ ++ return (lower_32_bits(va) + 0x40000000) | range; ++} ++ ++/* Apple's kext packs each group in this order, every object on 64 bytes. */ ++static int s8003_layout_plan(struct s8003_layout *layout) ++{ ++ static const struct { ++ u8 object; ++ bool shared; ++ u32 size; ++ } objects[] = { ++ { S8003_INIT, false, 0x1490 }, ++ { S8003_CHANNEL, false, 0x990 }, ++ { S8003_CHANNEL + 1, false, 0x8e0 }, ++ { S8003_CHANNEL + 2, false, 0x8c0 }, ++ { S8003_POLICY, false, 0x938 }, ++ { 0x4b, false, 0x28 }, ++ { 0x4c, false, 8 }, ++ { S8003_PLATFORM, false, 0x78c }, ++ { 0x4e, false, 0x84 }, ++ { S8003_CONTROL, true, 0x2058 }, ++ { S8003_EVENT_CTL, true, S8003_RING_CTL_SIZE }, ++ { S8003_LOG_CTL, true, 4 * S8003_RING_CTL_SIZE }, ++ { S8003_STATS_CTL, true, S8003_RING_CTL_SIZE }, ++ { S8003_TRACE_CTL, true, S8003_RING_CTL_SIZE }, ++ { S8003_EVENTS, true, 0x2c00 }, ++ { S8003_LOGS, true, 0xb000 }, ++ { 0x4f, true, 0x36000 }, ++ { S8003_STATS, true, 0x2c00 }, ++ { S8003_TRACE, true, 0x2400 }, ++ { S8003_DC_CTL, true, S8003_RING_CTL_SIZE }, ++ { S8003_DC_RING, true, 0x1400 }, ++ { S8003_WORK_CTL, true, S8003_RING_CTL_SIZE }, ++ { S8003_WORK_CTL + 4, true, S8003_RING_CTL_SIZE }, ++ { S8003_WORK_CTL + 8, true, S8003_RING_CTL_SIZE }, ++ { S8003_WORK_RING, true, 0x1400 }, ++ { S8003_WORK_RING + 4, true, 0x1400 }, ++ { S8003_WORK_RING + 8, true, 0x1400 }, ++ { 0x5c, true, 8 }, ++ { 0x85, true, 0x400 }, ++ { 0x84, true, 0x400 }, ++ { S8003_MMU_ROOT, true, 0x800 }, ++ }; ++ u32 used[2] = {}; ++ unsigned int i; ++ ++ for (i = 0; i < ARRAY_SIZE(objects); i++) { ++ u32 offset = ALIGN(used[objects[i].shared], 64); ++ ++ layout->offset[objects[i].object] = offset; ++ layout->shared[objects[i].object] = objects[i].shared; ++ used[objects[i].shared] = offset + objects[i].size; ++ } ++ return used[0] <= S8003_ROOT_SIZE && used[1] <= S8003_SHARED_SIZE ? 0 : -EINVAL; ++} ++ ++static u64 s8003_object_va(const struct s8003_layout *layout, u32 object) ++{ ++ return (layout->shared[object] ? S8003_SHARED_VA : S8003_ROOT_VA) + layout->offset[object]; ++} ++ ++static u32 s8003_object_addr(const struct s8003_layout *layout, u32 object) ++{ ++ return s8003_meta_addr(s8003_object_va(layout, object)); ++} ++ ++static u8 *s8003_object(struct pvr_apple_s8003 *s, u32 object) ++{ ++ return (s->layout.shared[object] ? s->shared.map : s->root.map) + s->layout.offset[object]; ++} ++ ++/* ++ * Where the image's sections go in the unpacked image, as Apple's kext lays ++ * them out. The local and core memory ranges are not zeroed here: the ++ * bootloader does that. ++ */ ++static int s8003_ldr_mem(void *data, u32 addr, u32 size, bool zero, void **write_addr) ++{ ++ static const struct { ++ u32 base, size, offset; ++ } sections[] = { ++ { 0x10000000, 0x18000, 0x04000 }, ++ { 0x20000000, 0x04000, 0x1c000 }, ++ { 0xc0000000, 0x04000, 0x00000 }, ++ { 0xe0001000, 0x04000, 0x20000 }, ++ { 0x80000080, S8003_IMAGE_COREMEM_SIZE, S8003_IMAGE_COREMEM }, ++ }; ++ struct pvr_apple_s8003 *s = data; ++ unsigned int i; ++ ++ if (zero && addr >= 0x80000000 && addr < 0x88000000) { ++ *write_addr = NULL; ++ return 0; ++ } ++ for (i = 0; i < ARRAY_SIZE(sections); i++) { ++ if (addr >= sections[i].base && size <= sections[i].size && ++ addr - sections[i].base <= sections[i].size - size) { ++ *write_addr = s->image + sections[i].offset + addr - sections[i].base; ++ return 0; ++ } ++ } ++ return -EINVAL; ++} ++ ++static int s8003_ldr_config(void *data, u32 reg, u32 value) ++{ ++ struct pvr_apple_s8003 *s = data; ++ ++ if (s->conf_count == S8003_LDR_CONF_MAX) ++ return -EINVAL; ++ s->conf[s->conf_count][0] = reg; ++ s->conf[s->conf_count][1] = value; ++ s->conf_count++; ++ return 0; ++} ++ ++/* The one thread the image starts: its number, stack pointer and entry */ ++static int s8003_ldr_start_threads(void *data, const u32 *words, u32 count) ++{ ++ struct pvr_apple_s8003 *s = data; ++ ++ if (count < 3 || s->thread_pc) ++ return -EINVAL; ++ s->thread_sp = words[1]; ++ s->thread_pc = words[2]; ++ return 0; ++} ++ ++static const struct pvr_meta_ldr_target s8003_ldr_target = { ++ .mem = s8003_ldr_mem, ++ .config = s8003_ldr_config, ++ .start_threads = s8003_ldr_start_threads, ++}; ++ ++/* ++ * The bootloader's configuration: the segment MMU's view of the image (code, ++ * data, the bootloader itself, one more data range), the image's own register ++ * writes, and where the bootloader finds the addresses of INIT and CONTROL. ++ */ ++static void s8003_image_configure(struct pvr_apple_s8003 *s) ++{ ++ static const struct { ++ u32 base, size, offset; ++ } segments[] = { ++ { 0x10000000, 0x18000, 0x04000 }, ++ { 0x20000000, 0x04000, 0x1c000 }, ++ { 0x40000000, 0x04000, 0x00000 }, ++ { 0x60001000, 0x04000, 0x20000 }, ++ }; ++ u8 *conf = s->image + S8003_IMAGE_CONF; ++ unsigned int i; ++ ++ for (i = 0; i < ARRAY_SIZE(segments); i++) { ++ u64 out = S8003_IMAGE_VA + segments[i].offset; ++ ++ /* The bootloader runs in segment 2: only its cache policy is set */ ++ if (i == 2) { ++ out |= ROGUE_FW_SEGMMU_OUTADDR_TOP_VIVT_SLC_UNCACHED(0); ++ put32(conf, META_CR_MMCU_SEGMENT_N_OUTA1(i)); ++ put32(conf + 4, upper_32_bits(out)); ++ conf += 8; ++ continue; ++ } ++ out |= ROGUE_FW_SEGMMU_OUTADDR_TOP_VIVT_SLC_CACHED(0); ++ put32(conf, META_CR_MMCU_SEGMENT_N_BASE(i)); ++ put32(conf + 4, segments[i].base | ROGUE_FW_SEGMMU_ALLTHRS_WRITEABLE); ++ put32(conf + 8, META_CR_MMCU_SEGMENT_N_LIMIT(i)); ++ put32(conf + 12, segments[i].size - 1); ++ put32(conf + 16, META_CR_MMCU_SEGMENT_N_OUTA0(i)); ++ put32(conf + 20, lower_32_bits(out)); ++ put32(conf + 24, META_CR_MMCU_SEGMENT_N_OUTA1(i)); ++ put32(conf + 28, upper_32_bits(out)); ++ conf += 32; ++ } ++ for (i = 0; i < s->conf_count; i++, conf += 8) { ++ put32(conf, s->conf[i][0]); ++ put32(conf + 4, s->conf[i][1]); ++ } ++ put64(conf, 0); ++ put32(conf + 8, s8003_meta_addr(S8003_IMAGE_VA + S8003_IMAGE_HANDOVER)); ++ put32(s->image + S8003_IMAGE_HANDOVER, s8003_object_addr(&s->layout, S8003_INIT)); ++ put32(s->image + S8003_IMAGE_HANDOVER + 4, s8003_object_addr(&s->layout, S8003_CONTROL)); ++} ++ ++/* ++ * The GPU's performance states for the firmware's governor: its clock table ++ * (MHz per Fender power state, iOS's on the iPad 5) and limits that pin the ++ * governor to state @q, the state the core is powered up in. Pinned, it never ++ * changes the state and never powers the core off while idle: the host does ++ * that (runtime PM). ++ */ ++static void s8003_init_perf(struct pvr_apple_s8003 *s, u32 q) ++{ ++ static const u32 mhz[] = { 0, 340, 474, 550, 723 }; ++ u8 *platform = s8003_object(s, S8003_PLATFORM); ++ u8 *control = s8003_object(s, S8003_CONTROL); ++ u8 *policy = s8003_object(s, S8003_POLICY); ++ u32 top = 100 * (ARRAY_SIZE(mhz) - 1), pin = 100 * q; ++ unsigned int i; ++ ++ put32(platform + 0x710, ARRAY_SIZE(mhz) - 1); ++ put32(platform + 0x714, ARRAY_SIZE(mhz)); ++ for (i = 0; i < ARRAY_SIZE(mhz); i++) ++ put32(platform + 0x718 + 4 * i, mhz[i]); ++ put32(control + 0x1028, 3348); /* power envelope, mW */ ++ put32(control + 0x102c, pin); /* floor */ ++ put32(control + 0x1030, top); ++ put32(control + 0x1038, top); /* overrides of the governor's limits */ ++ put32(control + 0x1040, pin); ++ put32(policy + 0x688, top); ++ put32(policy + 0x68c, top); ++ put32(policy + 0x6f0, pin); ++ put32(policy + 0x6f4, pin); ++ /* ++ * Above state 1 the governor's record starts as it would leave it for ++ * this target: the firmware's first sample then finds the core in the ++ * state it wants. ++ */ ++ if (q > 1) { ++ put64(policy + 0x28, (u64)q << 32 | q); ++ put32(policy + 0x30, 0); ++ put32(policy + 8, 0); ++ } ++} ++ ++/* ++ * The root and shared memory as the firmware finds it when it starts: where ++ * its objects are, the registers it may use, the interrupt it raises, its ++ * thread, and empty rings. What Apple's kext fills from the platform's power ++ * policy beyond s8003_init_perf() stays zero. ++ */ ++static void s8003_init_build(struct pvr_apple *apple, u32 q) ++{ ++ static const u8 rings[] = { ++ S8003_WORK_CTL, S8003_WORK_RING, S8003_WORK_CTL + 4, S8003_WORK_RING + 4, ++ S8003_WORK_CTL + 8, S8003_WORK_RING + 8, S8003_DC_CTL, S8003_DC_RING, ++ S8003_EVENT_CTL, S8003_EVENTS, S8003_LOG_CTL, S8003_LOGS, ++ S8003_STATS_CTL, S8003_STATS, S8003_TRACE_CTL, S8003_TRACE, ++ }; ++ static const u8 objects[] = { ++ S8003_CHANNEL, S8003_CHANNEL + 1, S8003_CHANNEL + 2, S8003_POLICY, 0x4b, ++ }; ++ static const u8 tail[] = { 0x4e, 0x4c, S8003_PLATFORM, 0x85, 0x4f }; ++ struct pvr_apple_s8003 *s = apple->chip_data; ++ const struct s8003_layout *layout = &s->layout; ++ const struct { ++ u64 phys; ++ u32 addr; ++ u32 size; ++ } pio[] = { ++ { s->fender_phys, s8003_meta_addr(S8003_PIO_VA + S8003_PIO_FENDER), 0x5000 }, ++ { s->aic_phys, s8003_meta_addr(S8003_PIO_VA + S8003_PIO_AIC), 0x4000 }, ++ { s->aic_phys + AIC_SW_SET, ++ s8003_meta_addr(S8003_PIO_VA + S8003_PIO_AIC + AIC_SW_SET), 0x4000 }, ++ { s->rgx_phys, 0, 0x20000 }, ++ }; ++ u8 *init, *platform, *policy, *control; ++ u64 now = arch_timer_read_counter(); ++ unsigned int i; ++ ++ memset(s->root.map, 0, S8003_ROOT_SIZE); ++ memset(s->shared.map, 0, S8003_SHARED_SIZE); ++ init = s8003_object(s, S8003_INIT); ++ platform = s8003_object(s, S8003_PLATFORM); ++ policy = s8003_object(s, S8003_POLICY); ++ control = s8003_object(s, S8003_CONTROL); ++ ++ for (i = 0; i < ARRAY_SIZE(rings); i++) ++ put32(init + 4 * i, s8003_object_addr(layout, rings[i])); ++ for (i = 0; i < ARRAY_SIZE(objects); i++) ++ put32(init + 0x64 + 4 * i, s8003_object_addr(layout, objects[i])); ++ for (i = 0; i < ARRAY_SIZE(tail); i++) ++ put32(init + 0x44c + 4 * i, s8003_object_addr(layout, tail[i])); ++ for (i = 0; i < 3; i++) { ++ u8 *channel = s8003_object(s, S8003_CHANNEL + i); ++ u32 addr = s8003_object_addr(layout, S8003_CHANNEL + i); ++ ++ put32(channel + 4, addr); ++ put32(channel + 12, addr + 8); ++ } ++ put32(s8003_object(s, S8003_CHANNEL + 1) + 0x778, 0xffffffff); ++ /* Core memory's contents: uncached address, GPU address, size */ ++ put32(init + 0x480, ++ (lower_32_bits(S8003_IMAGE_VA + S8003_IMAGE_COREMEM) + 0x40000000) | 0x40000000); ++ put64(init + 0x484, S8003_IMAGE_VA + S8003_IMAGE_COREMEM); ++ put32(init + 0x48c, S8003_IMAGE_COREMEM_SIZE); ++ ++ put64(platform, 0x600000000ull); ++ put64(platform + 8, 0xffc00000); ++ put64(platform + 0x10, 0x300000000ull); ++ put64(platform + 0x18, 0x300000000ull); ++ for (i = 0; i < ARRAY_SIZE(pio); i++) { ++ u8 *entry = platform + 0x630 + 16 * i; ++ ++ put64(entry, pio[i].phys); ++ put32(entry + 8, pio[i].addr); ++ put32(entry + 12, pio[i].size); ++ } ++ put32(platform + 0x6c0, 1); ++ put32(platform + 0x6c8, 1); ++ put64(platform + 0x6d8, 0x100000000ull); ++ put32(platform + 0x6e0, 1); ++ put32(platform + 0x6e4, 1); ++ /* From the core's revision (15): 1, 5, and its 6 clusters */ ++ put32(platform + 0x704, 1); ++ put32(platform + 0x708, 5); ++ put32(platform + 0x70c, 6); ++ /* The interrupt to the host: its word and bit among the AIC's software sets */ ++ put32(platform + 0x758, 4 * (s->irq / 32)); ++ put32(platform + 0x75c, BIT(s->irq % 32)); ++ put32(platform + 0x760, AIC_TIME); ++ put32(platform + 0x764, s->thread_pc); ++ put32(platform + 0x768, s->thread_sp); ++ put64(platform + 0x774, S8003_IMAGE_VA); ++ put64(platform + 0x77c, 0x300000); ++ /* +0x784: what the firmware adds to the AIC's time; Linux counts from it */ ++ ++ put64(policy + 8, 0x400000000ull); ++ put64(policy + 0x10, 0x3f800000); ++ put64(policy + 0x28, 0x100000001ull); ++ put32(policy + 0x38, 100); ++ put64(policy + 0x8f8, now); /* the governor's last sample */ ++ ++ put32(control + 0x24, 0x78); ++ put32(control + 0x30, 1); ++ put32(control + 0x1044, s8003_object_addr(layout, 0x84)); ++ put32(control + 0x1048, s8003_object_addr(layout, S8003_MMU_ROOT)); ++ put32(control + 0x104c, 1); ++ put64(s8003_object(s, S8003_MMU_ROOT), ++ pvr_vm_get_page_table_root_addr(apple->pvr_dev->kernel_vm_ctx)); ++ ++ s8003_init_perf(s, q); ++} ++ ++static void s8003_mem_fini(struct pvr_apple *apple, struct s8003_mem *mem) ++{ ++ if (!mem->obj) ++ return; ++ if (mem->mapped) ++ pvr_vm_unmap_obj(apple->pvr_dev->kernel_vm_ctx, mem->obj, mem->device_addr, ++ mem->size); ++ if (mem->map) ++ pvr_gem_object_vunmap(mem->obj); ++ pvr_gem_object_put(mem->obj); ++ memset(mem, 0, sizeof(*mem)); ++} ++ ++static int s8003_mem_init(struct pvr_apple *apple, struct s8003_mem *mem, u64 device_addr, ++ size_t size, u64 flags) ++{ ++ int err; ++ ++ mem->obj = pvr_gem_object_create(apple->pvr_dev, size, flags); ++ if (IS_ERR(mem->obj)) { ++ err = PTR_ERR(mem->obj); ++ mem->obj = NULL; ++ return err; ++ } ++ mem->size = size; ++ mem->map = pvr_gem_object_vmap(mem->obj); ++ if (IS_ERR(mem->map)) { ++ err = PTR_ERR(mem->map); ++ mem->map = NULL; ++ goto err_fini; ++ } ++ err = pvr_vm_map(apple->pvr_dev->kernel_vm_ctx, mem->obj, 0, device_addr, size); ++ if (err) ++ goto err_fini; ++ mem->device_addr = device_addr; ++ mem->mapped = true; ++ return 0; ++ ++err_fini: ++ s8003_mem_fini(apple, mem); ++ return err; ++} ++ ++static void pvr_apple_s8003_fini(struct pvr_apple *apple) ++{ ++ struct pvr_vm_context *vm_ctx = apple->pvr_dev->kernel_vm_ctx; ++ struct pvr_apple_s8003 *s = apple->chip_data; ++ ++ if (s->pio_mapped) { ++ pvr_vm_unmap_io(vm_ctx, S8003_PIO_VA + S8003_PIO_FENDER, S8003_PIO_SIZE); ++ pvr_vm_unmap_io(vm_ctx, S8003_PIO_VA + S8003_PIO_AIC, S8003_PIO_SIZE); ++ s->pio_mapped = false; ++ } ++ s8003_mem_fini(apple, &s->zero); ++ s8003_mem_fini(apple, &s->shared); ++ s8003_mem_fini(apple, &s->root); ++ s8003_mem_fini(apple, &s->fw); ++ if (s->spare) { ++ pvr_gem_object_put(s->spare); ++ s->spare = NULL; ++ } ++} ++ ++/* ++ * The firmware's memory and registers in the kernel VM. Everything is for the ++ * firmware only (PM/firmware protect): the image, the root and shared memory ++ * cache coherent as Apple's kext maps them, the registers past the cache. ++ * One plain zeroed page lies at address 0, which nothing else maps there: an ++ * access the firmware's context makes at 0 while it starts lands in it ++ * instead of faulting (iOS can map a dummy page for every unmapped address). ++ */ ++static int s8003_map(struct pvr_apple *apple) ++{ ++ struct pvr_vm_context *vm_ctx = apple->pvr_dev->kernel_vm_ctx; ++ struct pvr_apple_s8003 *s = apple->chip_data; ++ int err; ++ ++ err = s8003_mem_init(apple, &s->fw, S8003_IMAGE_VA, S8003_IMAGE_SIZE, ++ DRM_PVR_BO_PM_FW_PROTECT); ++ if (!err) ++ err = s8003_mem_init(apple, &s->root, S8003_ROOT_VA, S8003_ROOT_SIZE, ++ DRM_PVR_BO_PM_FW_PROTECT | PVR_BO_DEVICE_CACHE_COHERENT); ++ if (!err) ++ err = s8003_mem_init(apple, &s->shared, S8003_SHARED_VA, S8003_SHARED_SIZE, ++ DRM_PVR_BO_PM_FW_PROTECT | PVR_BO_DEVICE_CACHE_COHERENT); ++ if (!err) ++ err = s8003_mem_init(apple, &s->zero, 0, PVR_DEVICE_PAGE_SIZE, ++ PVR_BO_FW_FLAGS_DEVICE_CACHED); ++ if (!err) { ++ s->spare = pvr_gem_object_create(apple->pvr_dev, PVR_DEVICE_PAGE_SIZE, ++ PVR_BO_FW_FLAGS_DEVICE_CACHED); ++ if (IS_ERR(s->spare)) { ++ err = PTR_ERR(s->spare); ++ s->spare = NULL; ++ } ++ } ++ if (!err) ++ err = pvr_vm_map_io(vm_ctx, s->fender_phys, S8003_PIO_VA + S8003_PIO_FENDER, ++ S8003_PIO_SIZE); ++ if (!err) { ++ err = pvr_vm_map_io(vm_ctx, s->aic_phys, S8003_PIO_VA + S8003_PIO_AIC, ++ S8003_PIO_SIZE); ++ if (err) ++ pvr_vm_unmap_io(vm_ctx, S8003_PIO_VA + S8003_PIO_FENDER, S8003_PIO_SIZE); ++ else ++ s->pio_mapped = true; ++ } ++ if (err) ++ pvr_apple_s8003_fini(apple); ++ return err; ++} ++ ++static void fender_write(struct pvr_apple *apple, u32 reg, u32 value) ++{ ++ writel(value, apple->fender + reg); ++ readl(apple->fender + reg); /* each write lands before the next */ ++} ++ ++static int fender_wait(struct pvr_apple *apple, u32 reg, u32 mask, u32 value) ++{ ++ u32 val; ++ int err; ++ ++ err = readl_poll_timeout(apple->fender + reg, val, (val & mask) == value, 10, 100000); ++ if (err) ++ dev_err(apple_dev(apple), "Fender %#x is %#x, not %#x of %#x\n", reg, val, value, ++ mask); ++ return err; ++} ++ ++/* Take the core's reset request back and power it off (Fender state 0) */ ++static int s8003_core_off(struct pvr_apple *apple) ++{ ++ int err; ++ ++ fender_write(apple, FENDER_CORE, 0); ++ err = fender_wait(apple, FENDER_POWER_STATE, FENDER_POWER_REQUEST, 0); ++ if (!err) ++ err = pvr_apple_request_power_state(apple, 0); ++ return err; ++} ++ ++struct s8003_stream { ++ struct pvr_apple *apple; ++ u32 offset; ++ int err; ++}; ++ ++static void stream_word(struct s8003_stream *stream, u32 word) ++{ ++ void __iomem *reg = stream->apple->fender + FENDER_STREAM + stream->offset; ++ ++ if (stream->err) ++ return; ++ if (stream->offset >= FENDER_STREAM_SIZE) { ++ stream->err = -ENOSPC; ++ return; ++ } ++ writel(word, reg); ++ if (readl(reg) != word) ++ stream->err = -EIO; ++ stream->offset += 4; ++} ++ ++/* ++ * One 32-bit write. iOS's stream has 0xff above the register's offset in the ++ * last (or only) half of a register, and 0 in the upper half before it. ++ */ ++static void stream_write32(struct s8003_stream *stream, u32 reg, u32 value, bool last) ++{ ++ stream_word(stream, reg | (last ? 0xff << 16 : 0)); ++ stream_word(stream, value); ++} ++ ++static void stream_write64(struct s8003_stream *stream, u32 reg, u64 value) ++{ ++ stream_write32(stream, reg + 4, upper_32_bits(value), false); ++ stream_write32(stream, reg, lower_32_bits(value), true); ++} ++ ++/* ++ * Boot the META: with the core off and held in reset, store the boot stream, ++ * then let go and power the core up in state @q. Fender replays the stream, ++ * whose last write releases the reset; the bootloader in the image runs from ++ * the kernel VM's page tables. ++ */ ++static int s8003_release(struct pvr_apple *apple, u32 q) ++{ ++ struct pvr_apple_s8003 *s = apple->chip_data; ++ struct s8003_stream stream = { .apple = apple }; ++ dma_addr_t spare; ++ int err; ++ ++ err = pvr_gem_get_dma_addr(s->spare, 0, &spare); ++ if (err) ++ return err; ++ ++ err = s8003_core_off(apple); ++ if (err) ++ return err; ++ fender_write(apple, FENDER_CORE, FENDER_CORE_HOLD); ++ err = fender_wait(apple, FENDER_CORE, FENDER_CORE_HOLD, FENDER_CORE_HOLD); ++ if (!err) ++ err = fender_wait(apple, FENDER_POWER_STATE, FENDER_POWER_STATE_MASK, 0); ++ if (err) ++ return err; ++ ++ fender_write(apple, FENDER_STREAM_STATE, 0); ++ fender_write(apple, FENDER_CTRL, FENDER_CTRL_BOOT); ++ stream_write64(&stream, ROGUE_CR_SOFT_RESET, S8003_SOFT_RESET_ALL); ++ stream_write64(&stream, ROGUE_CR_SOFT_RESET, S8003_SOFT_RESET_CORE); ++ stream_write64(&stream, ROGUE_CR_SOFT_RESET, ROGUE_CR_SOFT_RESET_GARTEN_EN); ++ stream_write32(&stream, ROGUE_CR_META_BOOT, ROGUE_CR_META_BOOT_MODE_EN, true); ++ stream_write32(&stream, ROGUE_CR_MMU_CBASE_MAPPING_CONTEXT, 0, true); ++ stream_write64(&stream, ROGUE_CR_MMU_CBASE_MAPPING, ++ pvr_vm_get_page_table_root_addr(apple->pvr_dev->kernel_vm_ctx) >> 12); ++ stream_write64(&stream, ROGUE_CR_SLC3_CTRL_MISC, ++ ROGUE_CR_SLC3_CTRL_MISC_ADDR_DECODE_MODE_SCRAMBLE_PVR_HASH); ++ stream_write64(&stream, ROGUE_CR_SLC3_SCRAMBLE, S8003_SLC3_SCRAMBLE); ++ stream_write64(&stream, ROGUE_CR_SLC3_SCRAMBLE2, S8003_SLC3_SCRAMBLE); ++ fender_write(apple, FENDER_BOOT_2010, 0xbf); ++ fender_write(apple, FENDER_BOOT_2014, 0x233); ++ stream_write64(&stream, S8003_CR_E160, spare); ++ stream_write64(&stream, ROGUE_CR_GARTEN_SLC, 0); ++ stream_write64(&stream, ROGUE_CR_SOFT_RESET, 0); ++ stream_word(&stream, FENDER_STREAM_END); ++ if (stream.err) { ++ dev_err(apple_dev(apple), "Fender did not keep the boot stream: %d\n", stream.err); ++ return stream.err; ++ } ++ wmb(); /* the image and init data are in memory before the META runs */ ++ ++ fender_write(apple, FENDER_CORE, 0); ++ err = fender_wait(apple, FENDER_POWER_STATE, FENDER_POWER_STATE_MASK, 0); ++ if (err) ++ return err; ++ mutex_lock(&apple->perf_lock); ++ err = pvr_apple_request_power_state(apple, q); ++ mutex_unlock(&apple->perf_lock); ++ return err; ++} ++ ++static u32 ring_read(u8 *ctl, u32 index) ++{ ++ return le32_to_cpu(READ_ONCE(*(__le32 *)(ctl + index))); ++} ++ ++static void ring_consume(u8 *ctl, u32 consumer) ++{ ++ /* what was read from the entries, before the firmware may reuse them */ ++ mb(); ++ WRITE_ONCE(*(__le32 *)(ctl + S8003_RING_CONSUMER), cpu_to_le32(consumer)); ++} ++ ++/* Give a ring's entries back unread: the firmware's logs, statistics and trace */ ++static int s8003_ring_discard(u8 *ctl) ++{ ++ u32 consumer = ring_read(ctl, S8003_RING_CONSUMER); ++ u32 producer = ring_read(ctl, S8003_RING_PRODUCER); ++ ++ if (consumer >= APPLE_RING_ENTRIES || producer >= APPLE_RING_ENTRIES) ++ return -EPROTO; ++ if (consumer != producer) ++ ring_consume(ctl, producer); ++ return 0; ++} ++ ++/* ++ * Read what the firmware posted, in the order Apple's kext does: the log ++ * lanes, the events, statistics, trace. A full ring stalls the firmware. ++ * Events other than "alive" and a job's stamp mean the firmware stopped (a ++ * fault, a recovery request): none is expected before a job ran. ++ */ ++static int s8003_drain(struct pvr_apple *apple, bool *alive) ++{ ++ struct pvr_apple_s8003 *s = apple->chip_data; ++ u8 *ctl = s8003_object(s, S8003_EVENT_CTL); ++ u8 *events = s8003_object(s, S8003_EVENTS); ++ u32 consumer, producer; ++ unsigned int lane; ++ int err = 0; ++ ++ for (lane = 0; lane < 4 && !err; lane++) ++ err = s8003_ring_discard(s8003_object(s, S8003_LOG_CTL) + ++ lane * S8003_RING_CTL_SIZE); ++ if (err) ++ return err; ++ ++ consumer = ring_read(ctl, S8003_RING_CONSUMER); ++ producer = ring_read(ctl, S8003_RING_PRODUCER); ++ if (consumer >= APPLE_RING_ENTRIES || producer >= APPLE_RING_ENTRIES) ++ return -EPROTO; ++ rmb(); /* the index before the entries it covers */ ++ while (consumer != producer) { ++ const u8 *e = events + consumer * S8003_EVENT_SIZE; ++ u32 type = get_unaligned_le32(e); ++ ++ if (type == S8003_EVENT_ALIVE) { ++ *alive = true; ++ } else if (type != S8003_EVENT_STAMP) { ++ dev_err(apple_dev(apple), "firmware event %u: %*ph\n", type, ++ S8003_EVENT_SIZE - 4, e + 4); ++ err = -EIO; ++ } ++ consumer = (consumer + 1) % APPLE_RING_ENTRIES; ++ ring_consume(ctl, consumer); ++ } ++ if (err) ++ return err; ++ ++ err = s8003_ring_discard(s8003_object(s, S8003_STATS_CTL)); ++ if (!err) ++ err = s8003_ring_discard(s8003_object(s, S8003_TRACE_CTL)); ++ return err; ++} ++ ++/* While the firmware runs and until its interrupt is taken */ ++static void s8003_drain_work(struct work_struct *work) ++{ ++ struct pvr_apple_s8003 *s = container_of(to_delayed_work(work), struct pvr_apple_s8003, ++ drain_work); ++ bool alive = false; ++ int err; ++ ++ err = s8003_drain(s->apple, &alive); ++ if (err) { ++ dev_err(apple_dev(s->apple), "GFX firmware's rings are no longer read: %d\n", err); ++ return; ++ } ++ schedule_delayed_work(&s->drain_work, msecs_to_jiffies(S8003_DRAIN_MS)); ++} ++ ++static int s8003_wait_alive(struct pvr_apple *apple) ++{ ++ bool alive = false; ++ int err, drained = 0; ++ ++ err = read_poll_timeout(s8003_drain, drained, drained || alive, 1000, ++ S8003_ALIVE_TIMEOUT_US, false, apple, &alive); ++ if (!err) ++ err = drained; ++ if (err) ++ dev_err(apple_dev(apple), "GFX firmware did not come up: %d (Fender state %#x)\n", ++ err, readl(apple->fender + FENDER_POWER_STATE)); ++ return err; ++} ++ ++/* ++ * The core is powered and no power state request is in flight. The firmware ++ * makes requests of its own, so one in flight is waited for, as Apple's kext ++ * does before it touches the core. ++ */ ++static bool s8003_core_ready(struct pvr_apple *apple) ++{ ++ u32 state; ++ ++ if (readl_poll_timeout(apple->fender + FENDER_POWER_STATE, state, ++ !(state & FENDER_POWER_REQUEST), 10, 5000)) ++ return false; ++ return state & FENDER_POWER_STATE_MASK; ++} ++ ++/* ++ * Make the firmware look at the device-control ring: Fender's wake first, as ++ * Apple's kext does before every doorbell, then the META's scheduler. ++ */ ++static int s8003_dc_doorbell(struct pvr_apple *apple) ++{ ++ writel(FENDER_WAKE_ALL, apple->fender + FENDER_WAKE); ++ if (readl(apple->fender + FENDER_WAKE) & ~FENDER_WAKE_ALL) ++ return -EPROTO; ++ if (!s8003_core_ready(apple)) ++ return -EHOSTDOWN; ++ pvr_cr_write32(apple->pvr_dev, ROGUE_CR_MTS_SCHEDULE, S8003_MTS_DEVICE_CONTROL); ++ return 0; ++} ++ ++static u32 s8003_dc_consumer(struct pvr_apple *apple, int *err) ++{ ++ struct pvr_apple_s8003 *s = apple->chip_data; ++ bool alive = false; ++ ++ *err = s8003_drain(apple, &alive); ++ return ring_read(s8003_object(s, S8003_DC_CTL), S8003_RING_CONSUMER); ++} ++ ++/* ++ * The first command: an empty device-control command. The firmware takes it ++ * from the ring once its scheduler runs, which is when it takes commands at ++ * all. ++ */ ++static int s8003_first_command(struct pvr_apple *apple) ++{ ++ struct pvr_apple_s8003 *s = apple->chip_data; ++ u8 *ctl = s8003_object(s, S8003_DC_CTL); ++ u8 *ring = s8003_object(s, S8003_DC_RING); ++ u32 producer = ring_read(ctl, S8003_RING_PRODUCER); ++ u32 consumer, target; ++ int err, drained = 0; ++ ++ if (producer >= APPLE_RING_ENTRIES || ring_read(ctl, S8003_RING_CONSUMER) != producer) ++ return -EPROTO; ++ target = (producer + 1) % APPLE_RING_ENTRIES; ++ ++ put32(s8003_object(s, S8003_CONTROL) + 0x1020, 0); /* no payload */ ++ memset(ring + producer * S8003_DC_SIZE, 0, S8003_DC_SIZE); ++ put32(ring + producer * S8003_DC_SIZE, S8003_DC_NONE); ++ wmb(); ++ WRITE_ONCE(*(__le32 *)(ctl + S8003_RING_PRODUCER), cpu_to_le32(target)); ++ mb(); /* the index is in memory before the firmware is told */ ++ ++ err = s8003_dc_doorbell(apple); ++ if (!err) { ++ err = read_poll_timeout(s8003_dc_consumer, consumer, drained || consumer == target, ++ 1000, S8003_DC_TIMEOUT_US, false, apple, &drained); ++ if (!err) ++ err = drained; ++ } ++ if (err) ++ dev_err(apple_dev(apple), "GFX firmware did not take its first command: %d\n", ++ err); ++ return err; ++} ++ ++/* ++ * The registers the firmware is told about, as bus addresses: Fender and the ++ * Rogue core from this node, the interrupt controller and the number of the ++ * firmware's interrupt from the node's last interrupt. ++ */ ++static int s8003_parse_dt(struct pvr_apple *apple) ++{ ++ struct device *dev = apple_dev(apple); ++ struct platform_device *pdev = to_platform_device(dev); ++ struct pvr_apple_s8003 *s = apple->chip_data; ++ struct of_phandle_args irq; ++ struct resource *res, aic; ++ int err; ++ ++ res = platform_get_resource_byname(pdev, IORESOURCE_MEM, "fender"); ++ if (!res) ++ return -EINVAL; ++ s->fender_phys = res->start; ++ res = platform_get_resource_byname(pdev, IORESOURCE_MEM, "rgx"); ++ if (!res) ++ return -EINVAL; ++ s->rgx_phys = res->start; ++ ++ err = of_irq_parse_one(dev->of_node, 4, &irq); ++ if (err) ++ return dev_err_probe(dev, err, "no firmware interrupt\n"); ++ if (irq.args_count < 2 || irq.args[0] != AIC_IRQ) ++ err = -EINVAL; ++ else ++ err = of_address_to_resource(irq.np, 0, &aic); ++ of_node_put(irq.np); ++ if (err) ++ return dev_err_probe(dev, err, "firmware interrupt not on an AIC\n"); ++ s->irq = irq.args[1]; ++ s->aic_phys = aic.start; ++ ++ if ((s->fender_phys | s->aic_phys) & ~PVR_DEVICE_PAGE_MASK) ++ return -EINVAL; ++ return 0; ++} ++ ++static void s8003_free_image(void *data) ++{ ++ kvfree(data); ++} ++ ++static int pvr_apple_s8003_init(struct pvr_apple *apple) ++{ ++ struct device *dev = apple_dev(apple); ++ struct pvr_apple_layout *layout = &apple->layout; ++ struct pvr_apple_s8003 *s; ++ unsigned int i; ++ int err; ++ ++ /* The firmware's page tables have 16 KiB pages, as the kernel's must */ ++ if (PVR_DEVICE_PAGE_SIZE != SZ_16K) ++ return dev_err_probe(dev, -ENODEV, "the A9 GPU needs a 16 KiB page kernel\n"); ++ ++ s = devm_kzalloc(dev, sizeof(*s), GFP_KERNEL); ++ if (!s) ++ return -ENOMEM; ++ apple->chip_data = s; ++ s->apple = apple; ++ INIT_DELAYED_WORK(&s->drain_work, s8003_drain_work); ++ ++ err = s8003_parse_dt(apple); ++ if (err) ++ return err; ++ err = s8003_layout_plan(&s->layout); ++ if (err) ++ return err; ++ ++ s->image = kvzalloc(S8003_IMAGE_SIZE, GFP_KERNEL); ++ if (!s->image) ++ return -ENOMEM; ++ err = devm_add_action_or_reset(dev, s8003_free_image, s->image); ++ if (err) ++ return err; ++ err = pvr_apple_fw_load(apple, SZ_1M); ++ if (err) ++ return err; ++ err = pvr_meta_ldr_process(from_pvr_device(apple->pvr_dev), apple->fw_image, ++ apple->fw_size, &s8003_ldr_target, s, NULL); ++ if (!err && !s->thread_pc) ++ err = -EINVAL; ++ if (err) ++ return dev_err_probe(dev, err, "not the A9's GFX firmware image\n"); ++ s8003_image_configure(s); ++ ++ apple->heap = &s->idle; ++ for (i = 0; i < APPLE_ENGINES; i++) { ++ layout->stamp[i] = offsetof(struct s8003_idle, stamp[i]); ++ layout->producer[i] = offsetof(struct s8003_idle, producer[i]); ++ } ++ layout->frag_ring = offsetof(struct s8003_idle, frag_ring); ++ layout->frag_retired = offsetof(struct s8003_idle, frag_retired); ++ layout->mmu_root = offsetof(struct s8003_idle, mmu_root); ++ layout->mmu_gen = offsetof(struct s8003_idle, mmu_gen); ++ layout->event_consumer = offsetof(struct s8003_idle, event_consumer); ++ layout->event_producer = offsetof(struct s8003_idle, event_producer); ++ return 0; ++} ++ ++/* ++ * Wake Fender and power the core in state 1, the META held in reset: the ++ * Rogue registers answer (the core's ID is read next), no firmware runs. ++ */ ++static int pvr_apple_s8003_boot(struct pvr_apple *apple) ++{ ++ int err; ++ ++ writel(FENDER_STREAM_END, apple->fender + FENDER_STREAM); ++ writel(FENDER_CTRL_BOOT, apple->fender + FENDER_CTRL); ++ err = pvr_apple_request_power_state(apple, 1); ++ if (err) ++ return err; ++ usleep_range(10, 20); ++ return 0; ++} ++ ++static int pvr_apple_s8003_start(struct pvr_apple *apple) ++{ ++ struct pvr_apple_s8003 *s = apple->chip_data; ++ u32 q = pvr_apple_start_power_state(apple); ++ int err; ++ ++ if (!s->shared.obj) { ++ err = s8003_map(apple); ++ if (err) ++ return err; ++ } ++ /* The firmware writes into its image while it runs: start from the copy */ ++ memcpy(s->fw.map, s->image, S8003_IMAGE_SIZE); ++ s8003_init_build(apple, q); ++ ++ err = s8003_release(apple, q); ++ if (!err) ++ err = s8003_wait_alive(apple); ++ if (!err) { ++ dev_info(apple_dev(apple), "GFX firmware is up (power state %u)\n", q); ++ err = s8003_first_command(apple); ++ } ++ if (err) { ++ /* It must not run on once the kernel VM, its page tables, may go */ ++ s8003_core_off(apple); ++ return err; ++ } ++ dev_info(apple_dev(apple), "GFX firmware takes commands\n"); ++ schedule_delayed_work(&s->drain_work, msecs_to_jiffies(S8003_DRAIN_MS)); ++ return 0; ++} ++ ++/* ++ * Power the core off through Fender state 0, which stops the META with it; ++ * the firmware has no part in it, so a hung one stops the same way. Not with ++ * the GPU's power domain off already (a probe that failed after boot()): ++ * Fender is not to be touched then, and nothing is left running. ++ */ ++static void pvr_apple_s8003_stop(struct pvr_apple *apple, bool recover) ++{ ++ struct pvr_apple_s8003 *s = apple->chip_data; ++ ++ cancel_delayed_work_sync(&s->drain_work); ++ if (pm_runtime_status_suspended(apple_dev(apple))) ++ return; ++ s8003_core_off(apple); ++} ++ ++static void pvr_apple_s8003_idle(struct pvr_apple *apple) ++{ ++ /* nothing of the firmware's keeps the GPU's power domain on */ ++} ++ ++static void pvr_apple_s8003_restore_image(struct pvr_apple *apple) ++{ ++ /* start() copies the image and builds the init data anew each time */ ++} ++ ++const struct pvr_apple_chip pvr_apple_chip_s8003 = { ++ .fw_name = "apple/gfx-s8003.bin", ++ .perf_state_max = S8003_PERF_STATE_MAX, ++ .init = pvr_apple_s8003_init, ++ .boot = pvr_apple_s8003_boot, ++ .start = pvr_apple_s8003_start, ++ .stop = pvr_apple_s8003_stop, ++ .idle = pvr_apple_s8003_idle, ++ .fini = pvr_apple_s8003_fini, ++ .restore_image = pvr_apple_s8003_restore_image, ++}; +diff --git a/drivers/gpu/drm/imagination/pvr_device.c b/drivers/gpu/drm/imagination/pvr_device.c +index 01bc08f..0b6fd3c 100644 +--- a/drivers/gpu/drm/imagination/pvr_device.c ++++ b/drivers/gpu/drm/imagination/pvr_device.c +@@ -541,6 +541,10 @@ pvr_gpu_support_level(const struct pvr_gpu_id *gpu_id) + case PVR_PACKED_BVNC(13, 10, 6, 2): /* Apple A10 (T8010) */ + return PVR_GPU_SUPPORTED; + ++ /* Apple A9 (S8003): its firmware is started, no job runs on it yet */ ++ case PVR_PACKED_BVNC(9, 15, 6, 2): ++ return PVR_GPU_EXPERIMENTAL; ++ + default: + return PVR_GPU_UNKNOWN; + } +diff --git a/drivers/gpu/drm/imagination/pvr_drv.c b/drivers/gpu/drm/imagination/pvr_drv.c +index acc4979..206ee1f 100644 +--- a/drivers/gpu/drm/imagination/pvr_drv.c ++++ b/drivers/gpu/drm/imagination/pvr_drv.c +@@ -1514,10 +1514,20 @@ static const struct pvr_device_data pvr_device_data_apple = { + .apple_fw = true, + .apple_chip = &pvr_apple_chip_t8010, + }; ++ ++static const struct pvr_device_data pvr_device_data_apple_s8003 = { ++ .pwr_ops = &pvr_power_sequence_ops_apple, ++ .apple_fw = true, ++ .apple_chip = &pvr_apple_chip_s8003, ++}; + #endif + + static const struct of_device_id dt_match[] = { + #if IS_ENABLED(CONFIG_DRM_POWERVR_APPLE) ++ { ++ .compatible = "apple,s8003-gpu", ++ .data = &pvr_device_data_apple_s8003, ++ }, + { + .compatible = "apple,t8010-gpu", + .data = &pvr_device_data_apple, diff --git a/kernel/patches/drivers/gpu/0021-dt-bindings-gpu-apple-s8003-gpu-Say-which-interrupt-.patch b/kernel/patches/drivers/gpu/0021-dt-bindings-gpu-apple-s8003-gpu-Say-which-interrupt-.patch new file mode 100644 index 00000000..6ad1fdd7 --- /dev/null +++ b/kernel/patches/drivers/gpu/0021-dt-bindings-gpu-apple-s8003-gpu-Say-which-interrupt-.patch @@ -0,0 +1,38 @@ +From 0000000000000000000000000000000000000000 Mon Sep 17 00:00:00 2001 +From: Henrik Klee +Date: Sat, 14 Nov 2026 23:28:48 +0800 +Subject: [PATCH] dt-bindings: gpu: apple,s8003-gpu: Say which interrupt is the + firmware's + +The driver gives the firmware the address of the interrupt controller +and the number of the interrupt it raises towards the host: the last of +the node's five. The other four are not assigned yet. + +Assisted-by: LLM +--- + .../devicetree/bindings/gpu/apple,s8003-gpu.yaml | 12 ++++++++++-- + 1 file changed, 10 insertions(+), 2 deletions(-) + +diff --git a/Documentation/devicetree/bindings/gpu/apple,s8003-gpu.yaml b/Documentation/devicetree/bindings/gpu/apple,s8003-gpu.yaml +index 9539108..b29f915 100644 +--- a/Documentation/devicetree/bindings/gpu/apple,s8003-gpu.yaml ++++ b/Documentation/devicetree/bindings/gpu/apple,s8003-gpu.yaml +@@ -30,8 +30,16 @@ properties: + - const: fender + + interrupts: +- minItems: 5 +- maxItems: 5 ++ items: ++ - description: GPU interrupt 0 ++ - description: GPU interrupt 1 ++ - description: GPU interrupt 2 ++ - description: GPU interrupt 3 ++ - description: ++ The firmware's interrupt to the host, a software interrupt of the ++ interrupt controller. The firmware raises it through that ++ controller's registers, so the host tells it their address and ++ the interrupt's number. + + power-domains: + maxItems: 1 diff --git a/kernel/patches/drivers/gpu/0022-drm-imagination-apple-Let-a-chip-submit-and-complete.patch b/kernel/patches/drivers/gpu/0022-drm-imagination-apple-Let-a-chip-submit-and-complete.patch new file mode 100644 index 00000000..89e31a69 --- /dev/null +++ b/kernel/patches/drivers/gpu/0022-drm-imagination-apple-Let-a-chip-submit-and-complete.patch @@ -0,0 +1,583 @@ +From 0000000000000000000000000000000000000000 Mon Sep 17 00:00:00 2001 +From: Henrik Klee +Date: Sat, 14 Nov 2026 23:57:41 +0800 +Subject: [PATCH] drm/imagination: apple: Let a chip submit and complete its + own jobs + +pvr_apple.c writes a job the way the A10's firmware takes it: one command +per engine at a fixed place, the work ring's index naming it, done when +the engine's stamp reaches the job's number. The A9's firmware takes a +work record and a microsequence per job through a pointer ring, attaches +a VM to a pair of its own slots first and can hold two scenes of one VM. + +A chip may now provide that part itself: submit(), engine_done() and +events() for a job and its completion, vm_init(), vm_fini() and +mmu_flush() for what the firmware keeps per VM, a compute stream +definition, and three properties: scenes (how many scenes of one VM the +firmware holds; above one a fragment job is submitted with its geometry +job), own_pm_pages (free lists hold no pages) and gate_off_ms (a reset +also takes the GPU's power domain off, through runtime PM, and the next +job starts the firmware). + +What stays shared: the fences and their polling, the order of scenes +across VMs, the scheduler queues and their timeout, runtime PM, sleep, +the performance state. + +No change for the A10: it provides none of these, and every new branch +is taken only by a chip that does. + +Assisted-by: LLM +--- + drivers/gpu/drm/imagination/pvr_apple.c | 157 +++++++++++++++--- + drivers/gpu/drm/imagination/pvr_apple.h | 3 + + drivers/gpu/drm/imagination/pvr_apple_chip.h | 54 +++++- + drivers/gpu/drm/imagination/pvr_apple_queue.c | 3 + + drivers/gpu/drm/imagination/pvr_free_list.c | 10 +- + drivers/gpu/drm/imagination/pvr_hwrt.c | 3 + + drivers/gpu/drm/imagination/pvr_hwrt.h | 12 ++ + drivers/gpu/drm/imagination/pvr_job.c | 5 +- + 8 files changed, 214 insertions(+), 33 deletions(-) + +diff --git a/drivers/gpu/drm/imagination/pvr_apple.c b/drivers/gpu/drm/imagination/pvr_apple.c +index 0d9c2e5..e147626 100644 +--- a/drivers/gpu/drm/imagination/pvr_apple.c ++++ b/drivers/gpu/drm/imagination/pvr_apple.c +@@ -26,6 +26,7 @@ + #include "pvr_power.h" + #include "pvr_rogue_cr_defs.h" + #include "pvr_rogue_heap_config.h" ++#include "pvr_stream_defs.h" + #include "pvr_vm.h" + + #include +@@ -372,7 +373,6 @@ static int pvr_apple_boot(struct pvr_apple *apple) + + static int pvr_apple_fw_restart(struct pvr_apple *apple); + static void pvr_apple_fw_stop(struct pvr_apple *apple); +-static int pvr_apple_wait_idle(struct pvr_apple *apple, unsigned int timeout_us); + + /* + * Runtime PM. Every job the firmware runs holds a runtime-PM reference from +@@ -411,8 +411,11 @@ static int pvr_apple_power_off(struct pvr_device *pvr_dev) + if (apple->devfreq) + devfreq_suspend_device(apple->devfreq); + mutex_lock(&apple->submit_lock); +- /* Nothing holds a job reference, so this does not wait in practice. */ +- err = pvr_apple_wait_idle(apple, 1000000); ++ /* ++ * Nothing holds a job reference, so this does not wait in practice. ++ * A firmware that a reset stopped has nothing to wait for. ++ */ ++ err = apple->booted ? pvr_apple_wait_idle(apple, 1000000) : 0; + if (!err) { + pvr_apple_fw_stop(apple); + apple->chip->idle(apple); +@@ -571,9 +574,19 @@ static int pvr_apple_wait(struct pvr_apple *apple, u32 seq, unsigned int timeout + + static u32 pvr_apple_engine_done(struct pvr_apple *apple, enum pvr_apple_engine engine) + { ++ if (apple->chip->engine_done) ++ return apple->chip->engine_done(apple, engine); + return heap_read32(apple, apple->layout.stamp[engine]); + } + ++/* For a wait: what the firmware posted meanwhile is read first */ ++static u32 pvr_apple_engine_poll(struct pvr_apple *apple, enum pvr_apple_engine engine) ++{ ++ if (apple->chip->events) ++ apple->chip->events(apple); ++ return pvr_apple_engine_done(apple, engine); ++} ++ + static u32 pvr_apple_engine_last(struct pvr_apple *apple, enum pvr_apple_engine engine) + { + switch (engine) { +@@ -596,12 +609,12 @@ static int pvr_apple_wait_engine(struct pvr_apple *apple, enum pvr_apple_engine + * switch of GPU VM, for every engine; 1 ms steps cost a scene-heavy + * WebGL page several milliseconds per frame + */ +- return read_poll_timeout(pvr_apple_engine_done, done, (s32)(done - seq) >= 0, 20, ++ return read_poll_timeout(pvr_apple_engine_poll, done, (s32)(done - seq) >= 0, 20, + timeout_us, false, apple, engine); + } + + /* Wait until every engine retired its submitted jobs; submit_lock held */ +-static int pvr_apple_wait_idle(struct pvr_apple *apple, unsigned int timeout_us) ++int pvr_apple_wait_idle(struct pvr_apple *apple, unsigned int timeout_us) + { + int engine, err = 0; + +@@ -746,7 +759,10 @@ static void pvr_apple_poll_work(struct work_struct *work) + bool pending = false; + int engine; + +- pvr_apple_ack_events(apple); ++ if (apple->chip->events) ++ apple->chip->events(apple); ++ else ++ pvr_apple_ack_events(apple); + spin_lock_irq(&apple->fence_lock); + for (engine = 0; engine < APPLE_ENGINES; engine++) { + u32 done = pvr_apple_engine_done(apple, engine); +@@ -757,7 +773,7 @@ static void pvr_apple_poll_work(struct work_struct *work) + * do not go through the GPU, such as the display. Its fence also + * waits for the retirement stamp. + */ +- if (engine == APPLE_ENGINE_FRAG) { ++ if (engine == APPLE_ENGINE_FRAG && !apple->chip->engine_done) { + u32 retired = pvr_apple_frag_retired_seq(apple); + + if ((s32)(retired - done) < 0) +@@ -824,7 +840,19 @@ void pvr_apple_report_hang(struct pvr_device *pvr_dev, struct dma_fence *fence) + heap_read32(apple, apple->layout.producer[APPLE_ENGINE_FRAG]), + heap_read32(apple, apple->layout.event_producer), + heap_read32(apple, apple->layout.event_consumer)); +- apple->chip->report_hang(apple); ++ if (apple->chip->report_hang) ++ apple->chip->report_hang(apple); ++} ++ ++/* No job is ordered after the scenes submitted so far any more */ ++static void pvr_apple_forget_scenes(struct pvr_apple *apple) ++{ ++ dma_fence_put(apple->last_frag); ++ apple->last_frag = NULL; ++ dma_fence_put(apple->prev_frag); ++ apple->prev_frag = NULL; ++ dma_fence_put(apple->last_geom); ++ apple->last_geom = NULL; + } + + static unsigned int pvr_apple_cancel_fences(struct pvr_apple *apple, int error) +@@ -963,6 +991,11 @@ void pvr_apple_mmu_flush(struct pvr_device *pvr_dev) + struct pvr_apple *apple = to_apple(pvr_dev); + __le32 *gen = (__le32 *)(apple->heap + apple->layout.mmu_gen); + ++ if (apple->chip->mmu_flush) { ++ apple->chip->mmu_flush(apple); ++ return; ++ } ++ + /* + * May run in atomic context, so no lock: only the host writes the + * generation, and any change makes the firmware invalidate. A bump +@@ -1000,7 +1033,11 @@ void pvr_apple_vm_fini(struct pvr_device *pvr_dev, struct pvr_vm_context *vm_ctx + { + struct pvr_apple *apple = to_apple(pvr_dev); + +- if (vm_ctx == pvr_dev->kernel_vm_ctx || !apple->booted) ++ if (vm_ctx == pvr_dev->kernel_vm_ctx) ++ return; ++ if (apple->chip->vm_fini) ++ apple->chip->vm_fini(apple, vm_ctx); ++ if (!apple->booted) + return; + pvr_apple_switch_to_kernel_vm(apple, vm_ctx); + } +@@ -1125,6 +1162,8 @@ struct dma_fence *pvr_apple_submit_render_job(struct pvr_job *job) + if (job->type == DRM_PVR_JOB_TYPE_FRAGMENT && + job->fw_ccb_cmd_type == ROGUE_FWIF_CCB_CMD_TYPE_FRAG_PR) + return ERR_PTR(-EOPNOTSUPP); /* partial renders: not yet */ ++ if (apple->chip->submit) ++ return pvr_apple_submit_job(job); + + fence = kzalloc(sizeof(*fence), GFP_KERNEL); + hw = kmalloc(apple->chip->render_job_size, GFP_KERNEL); +@@ -1147,6 +1186,26 @@ struct dma_fence *pvr_apple_submit_render_job(struct pvr_job *job) + return pvr_apple_job_fence(apple, fence, engine, seq); + } + ++/** ++ * pvr_apple_submit_job() - Queue a job on a chip that writes its jobs itself ++ * @job: The job, its command decoded with the chip's stream definitions. ++ * ++ * Return: a fence signalled when the job retires, or an ERR_PTR(). ++ */ ++struct dma_fence *pvr_apple_submit_job(struct pvr_job *job) ++{ ++ struct pvr_apple *apple = to_apple(job->pvr_dev); ++ enum pvr_apple_engine engine = job->type == DRM_PVR_JOB_TYPE_GEOMETRY ? APPLE_ENGINE_GEOM : ++ job->type == DRM_PVR_JOB_TYPE_FRAGMENT ? APPLE_ENGINE_FRAG : ++ APPLE_ENGINE_COMPUTE; ++ struct pvr_apple_fence *fence; ++ ++ fence = kzalloc(sizeof(*fence), GFP_KERNEL); ++ if (!fence) ++ return ERR_PTR(-ENOMEM); ++ return pvr_apple_job_fence(apple, fence, engine, apple->chip->submit(apple, job)); ++} ++ + /** + * pvr_apple_job_add_deps() - Order a job against the parameter manager's scene + * @job: Job about to be armed. +@@ -1158,6 +1217,12 @@ struct dma_fence *pvr_apple_submit_render_job(struct pvr_job *job) + * HWRT reservation does not, as a job's fence is added to it only after all + * jobs of a submission collected their dependencies. + * ++ * A chip may hold two scenes of one VM (&pvr_apple_chip.scenes, ++ * &pvr_apple_chip.vm_scenes): a geometry job then waits for the fragment job ++ * before the last one if the last one is its own VM's, and a fragment job ++ * for its geometry job to be submitted rather than finished ++ * (pvr_apple_job_armed()). ++ * + * Return: 0 or a negative error. + */ + int pvr_apple_job_add_deps(struct pvr_job *job) +@@ -1168,8 +1233,14 @@ int pvr_apple_job_add_deps(struct pvr_job *job) + if (job->type != DRM_PVR_JOB_TYPE_GEOMETRY && job->type != DRM_PVR_JOB_TYPE_FRAGMENT) + return 0; + spin_lock_irq(&apple->fence_lock); +- last = dma_fence_get(job->type == DRM_PVR_JOB_TYPE_GEOMETRY ? apple->last_frag : +- apple->last_geom); ++ if (job->type == DRM_PVR_JOB_TYPE_FRAGMENT) ++ last = apple->last_geom; ++ else if (apple->chip->scenes > 1 && apple->last_frag_vm == job->ctx->vm_ctx && ++ apple->chip->vm_scenes(apple, job->ctx->vm_ctx) > 1) ++ last = apple->prev_frag; ++ else ++ last = apple->last_frag; ++ last = dma_fence_get(last); + spin_unlock_irq(&apple->fence_lock); + return last ? drm_sched_job_add_dependency(&job->base, last) : 0; + } +@@ -1182,6 +1253,7 @@ int pvr_apple_job_add_deps(struct pvr_job *job) + void pvr_apple_job_armed(struct pvr_job *job, struct dma_fence *finished) + { + struct pvr_apple *apple = to_apple(job->pvr_dev); ++ bool joint = apple->chip->scenes > 1; + struct dma_fence **last, *old; + + if (job->type == DRM_PVR_JOB_TYPE_GEOMETRY) +@@ -1190,9 +1262,17 @@ void pvr_apple_job_armed(struct pvr_job *job, struct dma_fence *finished) + last = &apple->last_frag; + else + return; ++ /* The firmware orders a fragment job after its geometry job itself */ ++ if (joint && job->type == DRM_PVR_JOB_TYPE_GEOMETRY) ++ finished = &job->base.s_fence->scheduled; + spin_lock_irq(&apple->fence_lock); + old = *last; + *last = dma_fence_get(finished); ++ if (job->type == DRM_PVR_JOB_TYPE_FRAGMENT) { ++ apple->last_frag_vm = job->ctx->vm_ctx; ++ if (joint) ++ swap(old, apple->prev_frag); ++ } + spin_unlock_irq(&apple->fence_lock); + dma_fence_put(old); + } +@@ -1217,6 +1297,8 @@ int pvr_apple_vm_init(struct pvr_device *pvr_dev, struct pvr_vm_context *vm_ctx) + PVR_DEVICE_PAGE_SIZE); + if (!err) + err = pvr_vm_map_reserved(vm_ctx, apple->zero_obj, 0, PVR_DEVICE_PAGE_SIZE); ++ if (!err && apple->chip->vm_init) ++ err = apple->chip->vm_init(apple, vm_ctx); + return err; + } + +@@ -1303,16 +1385,28 @@ int pvr_apple_reset(struct pvr_device *pvr_dev) + mutex_lock(&apple->submit_lock); + dev_warn(dev, "resetting the GPU\n"); + pvr_apple_cancel_fences(apple, -ETIMEDOUT); +- dma_fence_put(apple->last_frag); +- apple->last_frag = NULL; +- dma_fence_put(apple->last_geom); +- apple->last_geom = NULL; ++ pvr_apple_forget_scenes(apple); + spin_lock_irq(&apple->fence_lock); + apple->busy_since = 0; + spin_unlock_irq(&apple->fence_lock); + + pvr_apple_fw_stop(apple); +- err = pvr_apple_fw_restart(apple); ++ /* ++ * A chip whose core keeps state while its power domain is on: take ++ * the domain off through runtime PM and leave it off for a while. The ++ * next job starts the firmware (pvr_apple_power_on()). If another job ++ * holds the GPU powered at this moment, the domain stays on and the ++ * firmware is started here. ++ */ ++ if (apple->chip->gate_off_ms) { ++ mutex_unlock(&apple->submit_lock); ++ if (pm_runtime_suspend(dev) >= 0) { ++ msleep(apple->chip->gate_off_ms); ++ return 0; ++ } ++ mutex_lock(&apple->submit_lock); ++ } ++ err = apple->booted ? 0 : pvr_apple_fw_restart(apple); + mutex_unlock(&apple->submit_lock); + if (err) + dev_err(dev, "GPU reset failed: %d\n", err); +@@ -1364,10 +1458,7 @@ int pvr_apple_suspend(struct pvr_device *pvr_dev) + n = pvr_apple_cancel_fences(apple, -ETIMEDOUT); + if (n) + dev_warn(dev, "%u GPU jobs ended unfinished before sleep\n", n); +- dma_fence_put(apple->last_frag); +- apple->last_frag = NULL; +- dma_fence_put(apple->last_geom); +- apple->last_geom = NULL; ++ pvr_apple_forget_scenes(apple); + spin_lock_irq(&apple->fence_lock); + apple->busy_since = 0; + spin_unlock_irq(&apple->fence_lock); +@@ -1456,10 +1547,7 @@ void pvr_apple_fw_fini(struct pvr_device *pvr_dev) + struct pvr_apple *apple = to_apple(pvr_dev); + + apple->fw_ready = false; +- dma_fence_put(apple->last_frag); +- apple->last_frag = NULL; +- dma_fence_put(apple->last_geom); +- apple->last_geom = NULL; ++ pvr_apple_forget_scenes(apple); + if (apple->booted) + pvr_apple_switch_to_kernel_vm(apple, NULL); + pvr_apple_cancel_fences(apple, -ENODEV); +@@ -1513,13 +1601,28 @@ void pvr_apple_device_info_init(struct pvr_device *pvr_dev) + */ + bool pvr_apple_takes_jobs(struct pvr_device *pvr_dev) + { +- return pvr_dev->device_data->apple_chip->write_compute != NULL; ++ const struct pvr_apple_chip *chip = pvr_dev->device_data->apple_chip; ++ ++ return chip->write_compute || chip->submit; ++} ++ ++/** ++ * pvr_apple_free_list_has_pages() - Whether a free list holds parameter-manager pages ++ * @pvr_dev: Target device. ++ * ++ * Return: %false on a chip that allocates those pages itself, for each VM: a ++ * free list then only says how many a render may take. ++ */ ++bool pvr_apple_free_list_has_pages(struct pvr_device *pvr_dev) ++{ ++ return !pvr_dev->device_data->apple_chip->own_pm_pages; + } + + /** + * pvr_apple_cmd_stream() - How a render job's command stream is decoded + * @pvr_dev: Target device. +- * @type: %DRM_PVR_JOB_TYPE_GEOMETRY or %DRM_PVR_JOB_TYPE_FRAGMENT. ++ * @type: %DRM_PVR_JOB_TYPE_GEOMETRY, %DRM_PVR_JOB_TYPE_FRAGMENT or ++ * %DRM_PVR_JOB_TYPE_COMPUTE. + * + * Return: the chip's stream definition: the Rogue command plus the registers + * its firmware takes beyond it. +@@ -1529,5 +1632,7 @@ const struct pvr_stream_cmd_defs *pvr_apple_cmd_stream(struct pvr_device *pvr_de + { + const struct pvr_apple_chip *chip = pvr_dev->device_data->apple_chip; + ++ if (type == DRM_PVR_JOB_TYPE_COMPUTE) ++ return chip->compute_stream ?: &pvr_cmd_compute_stream; + return type == DRM_PVR_JOB_TYPE_GEOMETRY ? chip->geom_stream : chip->frag_stream; + } +diff --git a/drivers/gpu/drm/imagination/pvr_apple.h b/drivers/gpu/drm/imagination/pvr_apple.h +index a7f697c..f8a2a20 100644 +--- a/drivers/gpu/drm/imagination/pvr_apple.h ++++ b/drivers/gpu/drm/imagination/pvr_apple.h +@@ -95,6 +95,8 @@ int pvr_apple_suspend(struct pvr_device *pvr_dev); + int pvr_apple_resume(struct pvr_device *pvr_dev); + void pvr_apple_device_info_init(struct pvr_device *pvr_dev); + bool pvr_apple_takes_jobs(struct pvr_device *pvr_dev); ++bool pvr_apple_free_list_has_pages(struct pvr_device *pvr_dev); ++struct dma_fence *pvr_apple_submit_job(struct pvr_job *job); + struct dma_fence *pvr_apple_submit_compute_job(struct pvr_device *pvr_dev, + struct pvr_vm_context *vm_ctx, u64 stream, + u64 ctx_state); +@@ -126,6 +128,7 @@ static inline int pvr_apple_suspend(struct pvr_device *pvr_dev) { return 0; } + static inline int pvr_apple_resume(struct pvr_device *pvr_dev) { return 0; } + static inline void pvr_apple_device_info_init(struct pvr_device *pvr_dev) {} + static inline bool pvr_apple_takes_jobs(struct pvr_device *pvr_dev) { return false; } ++static inline bool pvr_apple_free_list_has_pages(struct pvr_device *pvr_dev) { return true; } + static inline int pvr_apple_vm_init(struct pvr_device *pvr_dev, struct pvr_vm_context *vm_ctx) + { + return -ENODEV; +diff --git a/drivers/gpu/drm/imagination/pvr_apple_chip.h b/drivers/gpu/drm/imagination/pvr_apple_chip.h +index 168338a..201d72a 100644 +--- a/drivers/gpu/drm/imagination/pvr_apple_chip.h ++++ b/drivers/gpu/drm/imagination/pvr_apple_chip.h +@@ -116,6 +116,12 @@ struct pvr_apple { + struct dma_fence *last_frag; + /* The last geometry job pushed: the next fragment job consumes its scene */ + struct dma_fence *last_geom; ++ /* ++ * Chips with &pvr_apple_chip.scenes of 2: the fragment job before ++ * @last_frag, and the VM @last_frag runs in. ++ */ ++ struct dma_fence *prev_frag; ++ const struct pvr_vm_context *last_frag_vm; + + /* + * Performance state (Fender power state) and its scaling: perf_lock +@@ -187,9 +193,38 @@ struct pvr_apple { + * @write_frag: Write fragment job @seq and its work-ring entry. + * @report_hang: Log what else the firmware recorded about a job that timed out. + * +- * A chip without @write_compute takes no jobs: its firmware is started and +- * stopped, no context is created on it (pvr_apple_takes_jobs()), and @kick, +- * the job functions and the streams are not used. ++ * The functions from @kick to @write_frag are for a firmware that takes one ++ * command per engine at a time from a fixed place, named by the work ring's ++ * index (the A10's). A chip whose firmware takes jobs differently provides ++ * these instead, and pvr_apple.c keeps the fences, the order of the jobs and ++ * the power management: ++ * ++ * @compute_stream: How a compute job's command stream is decoded, if not as ++ * the Rogue command alone. ++ * @scenes: Scenes (a geometry job and its fragment job) one VM may have in the ++ * firmware at a time; 0 is 1. Above 1 a fragment job is submitted as soon ++ * as its geometry job was, not when that finished: the firmware orders ++ * the two. Scenes of different VMs never overlap. ++ * @own_pm_pages: The chip allocates the parameter manager's pages itself; a ++ * free list has none (pvr_apple_free_list_has_pages()). ++ * @gate_off_ms: How long the GPU's power domain stays off before the firmware ++ * is started again, and that a reset takes the domain off as well. ++ * @submit: Hand @job to the firmware. Takes &pvr_apple.submit_lock itself. ++ * Return: the job's sequence number on its engine, which @engine_done ++ * reaches when it is finished, or a negative error. ++ * @engine_done: The sequence number of the last finished job of @engine. No ++ * lock held; may be called with interrupts off. ++ * @events: Read what the firmware posted. Before @engine_done wherever a job ++ * is waited for; may sleep. ++ * @vm_init: A userspace VM was created: map what the firmware needs in it. ++ * @vm_fini: A VM is about to go, its jobs retired: take it from the firmware. ++ * @vm_scenes: How many scenes of this VM the firmware may hold now, at most ++ * @scenes. Called in atomic context. ++ * @mmu_flush: Page tables of some VM changed. May be called in atomic context. ++ * ++ * A chip with neither @write_compute nor @submit takes no jobs: its firmware ++ * is started and stopped, no context is created on it ++ * (pvr_apple_takes_jobs()). + * + * pvr_apple.c calls @boot, @start, @stop, @restore_image, @kick and the + * write functions with &pvr_apple.submit_lock held, except @boot and @start +@@ -203,6 +238,10 @@ struct pvr_apple_chip { + size_t render_job_size; + const struct pvr_stream_cmd_defs *geom_stream; + const struct pvr_stream_cmd_defs *frag_stream; ++ const struct pvr_stream_cmd_defs *compute_stream; ++ unsigned int scenes; ++ bool own_pm_pages; ++ unsigned int gate_off_ms; + + int (*init)(struct pvr_apple *apple); + int (*boot)(struct pvr_apple *apple); +@@ -218,6 +257,14 @@ struct pvr_apple_chip { + void (*write_geom)(struct pvr_apple *apple, u32 seq, const void *hw); + void (*write_frag)(struct pvr_apple *apple, u32 seq, const void *hw); + void (*report_hang)(struct pvr_apple *apple); ++ ++ int (*submit)(struct pvr_apple *apple, struct pvr_job *job); ++ u32 (*engine_done)(struct pvr_apple *apple, enum pvr_apple_engine engine); ++ void (*events)(struct pvr_apple *apple); ++ int (*vm_init)(struct pvr_apple *apple, struct pvr_vm_context *vm_ctx); ++ void (*vm_fini)(struct pvr_apple *apple, struct pvr_vm_context *vm_ctx); ++ unsigned int (*vm_scenes)(struct pvr_apple *apple, struct pvr_vm_context *vm_ctx); ++ void (*mmu_flush)(struct pvr_apple *apple); + }; + + /* The heap is coherent memory shared with the firmware, not MMIO. */ +@@ -232,6 +279,7 @@ static inline struct device *apple_dev(struct pvr_apple *apple) + } + + int pvr_apple_fw_load(struct pvr_apple *apple, size_t max_size); ++int pvr_apple_wait_idle(struct pvr_apple *apple, unsigned int timeout_us); + int pvr_apple_request_power_state(struct pvr_apple *apple, u32 state); + u32 pvr_apple_start_power_state(struct pvr_apple *apple); + int pvr_apple_fender_power_on(struct pvr_apple *apple); +diff --git a/drivers/gpu/drm/imagination/pvr_apple_queue.c b/drivers/gpu/drm/imagination/pvr_apple_queue.c +index 4100c40..80bc17b 100644 +--- a/drivers/gpu/drm/imagination/pvr_apple_queue.c ++++ b/drivers/gpu/drm/imagination/pvr_apple_queue.c +@@ -10,6 +10,7 @@ + */ + + #include "pvr_apple.h" ++#include "pvr_apple_chip.h" + #include "pvr_context.h" + #include "pvr_device.h" + #include "pvr_job.h" +@@ -27,6 +28,8 @@ static struct dma_fence *pvr_apple_submit(struct pvr_job *job) + { + const struct rogue_fwif_cmd_compute *cmd = job->cmd; + ++ if (job->type == DRM_PVR_JOB_TYPE_COMPUTE && !job->pvr_dev->device_data->apple_chip->write_compute) ++ return pvr_apple_submit_job(job); + if (job->type == DRM_PVR_JOB_TYPE_GEOMETRY || job->type == DRM_PVR_JOB_TYPE_FRAGMENT) + return pvr_apple_submit_render_job(job); + if (job->type != DRM_PVR_JOB_TYPE_COMPUTE) +diff --git a/drivers/gpu/drm/imagination/pvr_free_list.c b/drivers/gpu/drm/imagination/pvr_free_list.c +index 48c4894..4305f9a 100644 +--- a/drivers/gpu/drm/imagination/pvr_free_list.c ++++ b/drivers/gpu/drm/imagination/pvr_free_list.c +@@ -456,9 +456,13 @@ pvr_free_list_create(struct pvr_file *pvr_file, + goto err_free_fw_id; + } + +- err = pvr_free_list_grow(free_list, args->initial_num_pages); +- if (err < 0) +- goto err_fw_struct_cleanup; ++ /* Not on a chip that allocates the parameter manager's pages itself */ ++ if (!pvr_device_is_apple(pvr_file->pvr_dev) || ++ pvr_apple_free_list_has_pages(pvr_file->pvr_dev)) { ++ err = pvr_free_list_grow(free_list, args->initial_num_pages); ++ if (err < 0) ++ goto err_fw_struct_cleanup; ++ } + + return free_list; + +diff --git a/drivers/gpu/drm/imagination/pvr_hwrt.c b/drivers/gpu/drm/imagination/pvr_hwrt.c +index 550f0b7..ec50237 100644 +--- a/drivers/gpu/drm/imagination/pvr_hwrt.c ++++ b/drivers/gpu/drm/imagination/pvr_hwrt.c +@@ -49,6 +49,9 @@ hwrt_init_kernel_structure(struct pvr_file *pvr_file, + + hwrt->pvr_dev = pvr_dev; + hwrt->max_rts = args->layers; ++ hwrt->width = args->width; ++ hwrt->height = args->height; ++ hwrt->samples = args->samples; + + /* Get pointers to the free lists */ + for (int i = 0; i < ARRAY_SIZE(hwrt->free_lists); i++) { +diff --git a/drivers/gpu/drm/imagination/pvr_hwrt.h b/drivers/gpu/drm/imagination/pvr_hwrt.h +index d30103a..2914275 100644 +--- a/drivers/gpu/drm/imagination/pvr_hwrt.h ++++ b/drivers/gpu/drm/imagination/pvr_hwrt.h +@@ -84,6 +84,18 @@ struct pvr_hwrt_dataset { + /** @max_rts: Maximum render targets for this HWRT data set. */ + u16 max_rts; + ++ /** ++ * @width: Width in pixels, as created; with @height and @samples for a ++ * firmware that takes the target's geometry with each job (Apple A9). ++ */ ++ u32 width; ++ ++ /** @height: Height in pixels. */ ++ u32 height; ++ ++ /** @samples: Samples per pixel. */ ++ u32 samples; ++ + /** @pin_lock: Protects @vm_ctx and @pins. */ + struct mutex pin_lock; + +diff --git a/drivers/gpu/drm/imagination/pvr_job.c b/drivers/gpu/drm/imagination/pvr_job.c +index 0336d25..95c5d2e 100644 +--- a/drivers/gpu/drm/imagination/pvr_job.c ++++ b/drivers/gpu/drm/imagination/pvr_job.c +@@ -235,7 +235,10 @@ pvr_compute_job_fw_cmd_init(struct pvr_job *job, + return -EINVAL; + + job->fw_ccb_cmd_type = ROGUE_FWIF_CCB_CMD_TYPE_CDM; +- err = pvr_fw_cmd_init(job->pvr_dev, job, &pvr_cmd_compute_stream, ++ err = pvr_fw_cmd_init(job->pvr_dev, job, ++ pvr_device_is_apple(job->pvr_dev) ? ++ pvr_apple_cmd_stream(job->pvr_dev, DRM_PVR_JOB_TYPE_COMPUTE) : ++ &pvr_cmd_compute_stream, + args->cmd_stream, args->cmd_stream_len); + if (err) + return err; diff --git a/kernel/patches/drivers/gpu/0023-drm-imagination-apple-Define-the-A9-s-command-stream.patch b/kernel/patches/drivers/gpu/0023-drm-imagination-apple-Define-the-A9-s-command-stream.patch new file mode 100644 index 00000000..71941199 --- /dev/null +++ b/kernel/patches/drivers/gpu/0023-drm-imagination-apple-Define-the-A9-s-command-stream.patch @@ -0,0 +1,330 @@ +From 0000000000000000000000000000000000000000 Mon Sep 17 00:00:00 2001 +From: Henrik Klee +Date: Sun, 15 Nov 2026 00:01:35 +0800 +Subject: [PATCH] drm/imagination: apple: Define the A9's command stream + extensions +MIME-Version: 1.0 +Content-Type: text/plain; charset=UTF-8 +Content-Transfer-Encoding: 8bit + +The A9's firmware takes a job as a work record the kernel builds. Most of +a record's words are userspace's: addresses of its buffers and the +firmware's program descriptors. They do not fit the Rogue commands, so +the A9 gets a stream extension per job type next to the A10's: + + PVR_STREAM_EXTHDR_TYPE_GEOM_G5P struct pvr_apple_g5p_geom_regs + PVR_STREAM_EXTHDR_TYPE_FRAG_G5P struct pvr_apple_g5p_frag_regs + PVR_STREAM_EXTHDR_TYPE_COMPUTE_G5P struct pvr_apple_g5p_compute_regs + +They carry what struct drm_g5p_render_pass_v3 and struct +drm_g5p_compute_pass of Berkay Kösebay's apple-g5p driver carry (orchard +#74, include/uapi/drm/apple_g5p_drm.h), less what the Rogue command and +the HWRT dataset already say: the control stream's address, the scissor, +depth-bias and occlusion-query tables, the background object's depth and +stencil values come from the command, the target's size and sample count +from the dataset. + +The A10's extension types are not valid in the A9's stream definitions. + +Assisted-by: LLM +--- + drivers/gpu/drm/imagination/pvr_apple.h | 71 +++++++++ + .../drm/imagination/pvr_rogue_fwif_stream.h | 31 ++++ + drivers/gpu/drm/imagination/pvr_stream_defs.c | 137 ++++++++++++++++++ + drivers/gpu/drm/imagination/pvr_stream_defs.h | 3 + + 4 files changed, 242 insertions(+) + +diff --git a/drivers/gpu/drm/imagination/pvr_apple.h b/drivers/gpu/drm/imagination/pvr_apple.h +index f8a2a20..cdc8e96 100644 +--- a/drivers/gpu/drm/imagination/pvr_apple.h ++++ b/drivers/gpu/drm/imagination/pvr_apple.h +@@ -86,6 +86,77 @@ struct pvr_apple_cmd_frag { + struct pvr_apple_frag_regs apple; + }; + ++/* ++ * Apple A9 (G5P). Its firmware takes a job as a work record: the kernel ++ * builds the record from the Rogue command, the HWRT dataset and these ++ * stream extensions, which carry the record's words that userspace owns. ++ * Addresses are GPU addresses in the job's VM; "PDS words" are the firmware's ++ * program descriptors as the A9 Vulkan driver builds them, code addresses in ++ * them relative to the PDS heap's base. ++ * ++ * Geometry (PVR_STREAM_EXTHDR_TYPE_GEOM_G5P). Of the Rogue command only ++ * regs.vdm_ctrl_stream_base is read. ++ */ ++struct pvr_apple_g5p_geom_regs { ++ u64 scratch_base; /* TA scratch block, 8-byte aligned */ ++ u64 scratch_ptr[6]; /* TA scratch and marker pointers */ ++ u64 ta_program; /* TA state PDS program: byte offset from the PDS heap's base */ ++ u64 sample_positions; /* x | y << 4 per sample, 1/16 pixel; 0: the standard pattern */ ++ u32 context_pds[2][6]; /* context store and load PDS words */ ++ u32 resume_pds[2][4]; /* resume PDS words */ ++ u32 ta_pds[4]; /* TA state PDS words; word 3 is zero */ ++}; ++ ++struct pvr_apple_g5p_cmd_geom { ++ struct rogue_fwif_cmd_geom cmd; ++ struct pvr_apple_g5p_geom_regs g5p; ++}; ++ ++/* ++ * Fragment (PVR_STREAM_EXTHDR_TYPE_FRAG_G5P). Of the Rogue command ++ * regs.isp_scissor_base, isp_dbias_base, isp_oclqry_base, isp_bgobjdepth and ++ * isp_bgobjvals are read. ++ */ ++#define PVR_APPLE_G5P_FRAG_MSAA_RESOLVE BIT(0) /* four samples only */ ++ ++struct pvr_apple_g5p_frag_regs { ++ u64 background_texture; /* PDS_BGRND1_BASE: the background object's texture state */ ++ u64 zls[6]; /* register images 0xf48, 0xf50, 0xf58, 0xf60, 0xf68, 0xfd0 */ ++ u64 sample_positions; /* as the geometry job's */ ++ u32 background[8]; /* background (load or clear) object */ ++ u32 eot[4]; /* end-of-tile object; word 3 is zero */ ++ u32 eot_resume[4]; /* partial-render end of tile; word 3 is zero */ ++ u32 restore[4]; /* partial-render restore words */ ++ u32 pixel_output_width; /* 2, 4 or 8 */ ++ u32 flags; /* PVR_APPLE_G5P_FRAG_* */ ++}; ++ ++struct pvr_apple_g5p_cmd_frag { ++ struct rogue_fwif_cmd_frag cmd; ++ struct pvr_apple_g5p_frag_regs g5p; ++}; ++ ++/* ++ * Compute (PVR_STREAM_EXTHDR_TYPE_COMPUTE_G5P). Of the Rogue command only ++ * regs.cdm_ctrl_stream_base is read. ++ */ ++struct pvr_apple_g5p_compute_regs { ++ u64 state[4]; /* context state buffers */ ++ u64 tail; /* 16-byte aligned */ ++ u64 terminate; /* address of the control stream's terminate word */ ++ u64 pds_store; /* PDS code | data << 32 */ ++ u64 pds_resume; ++ u64 pds_terminate; ++ u32 config[2]; ++ u32 pds_store1; ++ u32 pds_resume1; ++}; ++ ++struct pvr_apple_g5p_cmd_compute { ++ struct rogue_fwif_cmd_compute cmd; ++ struct pvr_apple_g5p_compute_regs g5p; ++}; ++ + #if IS_ENABLED(CONFIG_DRM_POWERVR_APPLE) + int pvr_apple_fw_init(struct pvr_device *pvr_dev); + void pvr_apple_fw_fini(struct pvr_device *pvr_dev); +diff --git a/drivers/gpu/drm/imagination/pvr_rogue_fwif_stream.h b/drivers/gpu/drm/imagination/pvr_rogue_fwif_stream.h +index 9e808c7..0ff08fc 100644 +--- a/drivers/gpu/drm/imagination/pvr_rogue_fwif_stream.h ++++ b/drivers/gpu/drm/imagination/pvr_rogue_fwif_stream.h +@@ -67,6 +67,17 @@ + + #define PVR_STREAM_EXTHDR_GEOM_APPLE_VALID PVR_STREAM_EXTHDR_GEOM_APPLE_REGS + ++/* ++ * Stream extension header - Geometry, Apple A9 (G5P): what its firmware's ++ * geometry work record takes beyond the Rogue command ++ * (struct pvr_apple_g5p_geom_regs). Required on the A9; the A10's is refused. ++ */ ++#define PVR_STREAM_EXTHDR_TYPE_GEOM_G5P 2U ++ ++#define PVR_STREAM_EXTHDR_GEOM_G5P_REGS BIT(0U) ++ ++#define PVR_STREAM_EXTHDR_GEOM_G5P_VALID PVR_STREAM_EXTHDR_GEOM_G5P_REGS ++ + /* + * Stream extension header - Fragment 0 + */ +@@ -87,6 +98,16 @@ + + #define PVR_STREAM_EXTHDR_FRAG_APPLE_VALID PVR_STREAM_EXTHDR_FRAG_APPLE_REGS + ++/* ++ * Stream extension header - Fragment, Apple A9 (G5P) ++ * (struct pvr_apple_g5p_frag_regs). Required on the A9. ++ */ ++#define PVR_STREAM_EXTHDR_TYPE_FRAG_G5P 2U ++ ++#define PVR_STREAM_EXTHDR_FRAG_G5P_REGS BIT(0U) ++ ++#define PVR_STREAM_EXTHDR_FRAG_G5P_VALID PVR_STREAM_EXTHDR_FRAG_G5P_REGS ++ + /* + * Stream extension header - Compute 0 + */ +@@ -96,4 +117,14 @@ + + #define PVR_STREAM_EXTHDR_COMPUTE0_VALID PVR_STREAM_EXTHDR_COMPUTE0_BRN49927 + ++/* ++ * Stream extension header - Compute, Apple A9 (G5P) ++ * (struct pvr_apple_g5p_compute_regs). Required on the A9. ++ */ ++#define PVR_STREAM_EXTHDR_TYPE_COMPUTE_G5P 1U ++ ++#define PVR_STREAM_EXTHDR_COMPUTE_G5P_REGS BIT(0U) ++ ++#define PVR_STREAM_EXTHDR_COMPUTE_G5P_VALID PVR_STREAM_EXTHDR_COMPUTE_G5P_REGS ++ + #endif /* PVR_ROGUE_FWIF_STREAM_H */ +diff --git a/drivers/gpu/drm/imagination/pvr_stream_defs.c b/drivers/gpu/drm/imagination/pvr_stream_defs.c +index 9d90141..9271741 100644 +--- a/drivers/gpu/drm/imagination/pvr_stream_defs.c ++++ b/drivers/gpu/drm/imagination/pvr_stream_defs.c +@@ -447,3 +447,140 @@ const struct pvr_stream_cmd_defs pvr_apple_cmd_frag_stream = { + + .dest_size = sizeof(struct pvr_apple_cmd_frag), + }; ++ ++/* ++ * Apple A9 (G5P): the Rogue commands, each followed by an extension carrying ++ * what the GFX firmware's work record takes beyond it. ++ */ ++static const struct pvr_stream_def pvr_apple_g5p_geom_regs_stream[] = { ++ PVR_STREAM_DEF(pvr_apple_g5p_cmd_geom, g5p.scratch_base, 64), ++ PVR_STREAM_DEF_ARRAY(pvr_apple_g5p_cmd_geom, g5p.scratch_ptr), ++ PVR_STREAM_DEF(pvr_apple_g5p_cmd_geom, g5p.ta_program, 64), ++ PVR_STREAM_DEF(pvr_apple_g5p_cmd_geom, g5p.sample_positions, 64), ++ PVR_STREAM_DEF_ARRAY(pvr_apple_g5p_cmd_geom, g5p.context_pds), ++ PVR_STREAM_DEF_ARRAY(pvr_apple_g5p_cmd_geom, g5p.resume_pds), ++ PVR_STREAM_DEF_ARRAY(pvr_apple_g5p_cmd_geom, g5p.ta_pds), ++}; ++ ++static const struct pvr_stream_ext_def pvr_apple_g5p_geom_ext_streams[] = { ++ { ++ .stream = pvr_apple_g5p_geom_regs_stream, ++ .stream_len = ARRAY_SIZE(pvr_apple_g5p_geom_regs_stream), ++ .header_mask = PVR_STREAM_EXTHDR_GEOM_G5P_REGS, ++ }, ++}; ++ ++static const struct pvr_stream_ext_header pvr_apple_g5p_geom_ext_headers[] = { ++ [PVR_STREAM_EXTHDR_TYPE_GEOM0] = { ++ .ext_streams = cmd_geom_ext_streams_0, ++ .ext_streams_num = ARRAY_SIZE(cmd_geom_ext_streams_0), ++ .valid_mask = PVR_STREAM_EXTHDR_GEOM0_VALID, ++ }, ++ [PVR_STREAM_EXTHDR_TYPE_GEOM_G5P] = { ++ .ext_streams = pvr_apple_g5p_geom_ext_streams, ++ .ext_streams_num = ARRAY_SIZE(pvr_apple_g5p_geom_ext_streams), ++ .valid_mask = PVR_STREAM_EXTHDR_GEOM_G5P_VALID, ++ }, ++}; ++ ++const struct pvr_stream_cmd_defs pvr_apple_g5p_cmd_geom_stream = { ++ .type = PVR_STREAM_TYPE_GEOM, ++ ++ .main_stream = rogue_fwif_cmd_geom_stream, ++ .main_stream_len = ARRAY_SIZE(rogue_fwif_cmd_geom_stream), ++ ++ .ext_nr_headers = ARRAY_SIZE(pvr_apple_g5p_geom_ext_headers), ++ .ext_headers = pvr_apple_g5p_geom_ext_headers, ++ ++ .dest_size = sizeof(struct pvr_apple_g5p_cmd_geom), ++}; ++ ++static const struct pvr_stream_def pvr_apple_g5p_frag_regs_stream[] = { ++ PVR_STREAM_DEF(pvr_apple_g5p_cmd_frag, g5p.background_texture, 64), ++ PVR_STREAM_DEF_ARRAY(pvr_apple_g5p_cmd_frag, g5p.zls), ++ PVR_STREAM_DEF(pvr_apple_g5p_cmd_frag, g5p.sample_positions, 64), ++ PVR_STREAM_DEF_ARRAY(pvr_apple_g5p_cmd_frag, g5p.background), ++ PVR_STREAM_DEF_ARRAY(pvr_apple_g5p_cmd_frag, g5p.eot), ++ PVR_STREAM_DEF_ARRAY(pvr_apple_g5p_cmd_frag, g5p.eot_resume), ++ PVR_STREAM_DEF_ARRAY(pvr_apple_g5p_cmd_frag, g5p.restore), ++ PVR_STREAM_DEF(pvr_apple_g5p_cmd_frag, g5p.pixel_output_width, 32), ++ PVR_STREAM_DEF(pvr_apple_g5p_cmd_frag, g5p.flags, 32), ++}; ++ ++static const struct pvr_stream_ext_def pvr_apple_g5p_frag_ext_streams[] = { ++ { ++ .stream = pvr_apple_g5p_frag_regs_stream, ++ .stream_len = ARRAY_SIZE(pvr_apple_g5p_frag_regs_stream), ++ .header_mask = PVR_STREAM_EXTHDR_FRAG_G5P_REGS, ++ }, ++}; ++ ++static const struct pvr_stream_ext_header pvr_apple_g5p_frag_ext_headers[] = { ++ [PVR_STREAM_EXTHDR_TYPE_FRAG0] = { ++ .ext_streams = cmd_frag_ext_streams_0, ++ .ext_streams_num = ARRAY_SIZE(cmd_frag_ext_streams_0), ++ .valid_mask = PVR_STREAM_EXTHDR_FRAG0_VALID, ++ }, ++ [PVR_STREAM_EXTHDR_TYPE_FRAG_G5P] = { ++ .ext_streams = pvr_apple_g5p_frag_ext_streams, ++ .ext_streams_num = ARRAY_SIZE(pvr_apple_g5p_frag_ext_streams), ++ .valid_mask = PVR_STREAM_EXTHDR_FRAG_G5P_VALID, ++ }, ++}; ++ ++const struct pvr_stream_cmd_defs pvr_apple_g5p_cmd_frag_stream = { ++ .type = PVR_STREAM_TYPE_FRAG, ++ ++ .main_stream = rogue_fwif_cmd_frag_stream, ++ .main_stream_len = ARRAY_SIZE(rogue_fwif_cmd_frag_stream), ++ ++ .ext_nr_headers = ARRAY_SIZE(pvr_apple_g5p_frag_ext_headers), ++ .ext_headers = pvr_apple_g5p_frag_ext_headers, ++ ++ .dest_size = sizeof(struct pvr_apple_g5p_cmd_frag), ++}; ++ ++static const struct pvr_stream_def pvr_apple_g5p_compute_regs_stream[] = { ++ PVR_STREAM_DEF_ARRAY(pvr_apple_g5p_cmd_compute, g5p.state), ++ PVR_STREAM_DEF(pvr_apple_g5p_cmd_compute, g5p.tail, 64), ++ PVR_STREAM_DEF(pvr_apple_g5p_cmd_compute, g5p.terminate, 64), ++ PVR_STREAM_DEF(pvr_apple_g5p_cmd_compute, g5p.pds_store, 64), ++ PVR_STREAM_DEF(pvr_apple_g5p_cmd_compute, g5p.pds_resume, 64), ++ PVR_STREAM_DEF(pvr_apple_g5p_cmd_compute, g5p.pds_terminate, 64), ++ PVR_STREAM_DEF_ARRAY(pvr_apple_g5p_cmd_compute, g5p.config), ++ PVR_STREAM_DEF(pvr_apple_g5p_cmd_compute, g5p.pds_store1, 32), ++ PVR_STREAM_DEF(pvr_apple_g5p_cmd_compute, g5p.pds_resume1, 32), ++}; ++ ++static const struct pvr_stream_ext_def pvr_apple_g5p_compute_ext_streams[] = { ++ { ++ .stream = pvr_apple_g5p_compute_regs_stream, ++ .stream_len = ARRAY_SIZE(pvr_apple_g5p_compute_regs_stream), ++ .header_mask = PVR_STREAM_EXTHDR_COMPUTE_G5P_REGS, ++ }, ++}; ++ ++static const struct pvr_stream_ext_header pvr_apple_g5p_compute_ext_headers[] = { ++ [PVR_STREAM_EXTHDR_TYPE_COMPUTE0] = { ++ .ext_streams = cmd_compute_ext_streams_0, ++ .ext_streams_num = ARRAY_SIZE(cmd_compute_ext_streams_0), ++ .valid_mask = PVR_STREAM_EXTHDR_COMPUTE0_VALID, ++ }, ++ [PVR_STREAM_EXTHDR_TYPE_COMPUTE_G5P] = { ++ .ext_streams = pvr_apple_g5p_compute_ext_streams, ++ .ext_streams_num = ARRAY_SIZE(pvr_apple_g5p_compute_ext_streams), ++ .valid_mask = PVR_STREAM_EXTHDR_COMPUTE_G5P_VALID, ++ }, ++}; ++ ++const struct pvr_stream_cmd_defs pvr_apple_g5p_cmd_compute_stream = { ++ .type = PVR_STREAM_TYPE_COMPUTE, ++ ++ .main_stream = rogue_fwif_cmd_compute_stream, ++ .main_stream_len = ARRAY_SIZE(rogue_fwif_cmd_compute_stream), ++ ++ .ext_nr_headers = ARRAY_SIZE(pvr_apple_g5p_compute_ext_headers), ++ .ext_headers = pvr_apple_g5p_compute_ext_headers, ++ ++ .dest_size = sizeof(struct pvr_apple_g5p_cmd_compute), ++}; +diff --git a/drivers/gpu/drm/imagination/pvr_stream_defs.h b/drivers/gpu/drm/imagination/pvr_stream_defs.h +index 6cf88a3..892b44b 100644 +--- a/drivers/gpu/drm/imagination/pvr_stream_defs.h ++++ b/drivers/gpu/drm/imagination/pvr_stream_defs.h +@@ -14,5 +14,8 @@ extern const struct pvr_stream_cmd_defs pvr_static_render_context_state_stream; + extern const struct pvr_stream_cmd_defs pvr_static_compute_context_state_stream; + extern const struct pvr_stream_cmd_defs pvr_apple_cmd_geom_stream; + extern const struct pvr_stream_cmd_defs pvr_apple_cmd_frag_stream; ++extern const struct pvr_stream_cmd_defs pvr_apple_g5p_cmd_geom_stream; ++extern const struct pvr_stream_cmd_defs pvr_apple_g5p_cmd_frag_stream; ++extern const struct pvr_stream_cmd_defs pvr_apple_g5p_cmd_compute_stream; + + #endif /* PVR_STREAM_DEFS_H */ diff --git a/kernel/patches/drivers/gpu/0024-drm-imagination-apple-Run-jobs-on-the-A9-S8003.patch b/kernel/patches/drivers/gpu/0024-drm-imagination-apple-Run-jobs-on-the-A9-S8003.patch new file mode 100644 index 00000000..cffbba52 --- /dev/null +++ b/kernel/patches/drivers/gpu/0024-drm-imagination-apple-Run-jobs-on-the-A9-S8003.patch @@ -0,0 +1,1739 @@ +From 0000000000000000000000000000000000000000 Mon Sep 17 00:00:00 2001 +From: =?UTF-8?q?Berkay=20K=C3=B6sebay?= +Date: Sun, 15 Nov 2026 00:08:34 +0800 +Subject: [PATCH] drm/imagination: apple: Run jobs on the A9 (S8003) +MIME-Version: 1.0 +Content-Type: text/plain; charset=UTF-8 +Content-Transfer-Encoding: 8bit + +The A9's firmware is now given geometry, fragment and compute jobs, so the +iPad 5's GPU renders through powervr's UAPI. + +How this firmware takes a job differs from the A10's, and the chip does +that part itself (submit(), engine_done() and the other chip functions): + +- Channels. Each engine has a channel in the shared memory: a ring of 16 + pointers with five watermarks, two stamps, a timestamp queue and a + dependency object. A job is a work record the firmware loads the + engine's registers from and a microsequence it runs for it; a render + job has a command ahead of the record (a geometry job hands over the + parameter manager's record, a fragment job waits for its geometry + job's stamp). An entry of the engine's work ring and the META's + scheduler register tell the firmware. A job is done when both stamps + reached its number and all watermarks its end of the pointer ring. +- Work records. The kernel builds them from the Rogue command, the A9 + stream extension and the HWRT dataset: the words Apple's kext sets the + same for every render, userspace's addresses and program words, and + the target's geometry, which it derives from width, height and samples + as the kext does. +- VMs. A VM gets a pair of the firmware's slots with its first job: two + device-control commands make the firmware load its page-table root. + When powervr flushes the MMU, the tag of every slot in use changes, and + the firmware drops the slot's translations when its next job starts. + A VM that goes leaves its slots on a park VM's tables, because the + parameter manager still writes through them for a while. +- Parameter manager. Its state for a scene is a PM set: three pages in + the firmware's GiB, mapped in every VM where the firmware has them and + protected as the firmware's are, a pool of kernel pages, and the PM's + own tables, kernel pages mapped protected in the VM. Free lists hold + no pages on this chip: their size says how large a VM's pool is. +- Two scenes of one VM may be in the firmware, each in its own PM set, + and a fragment job is submitted right after its geometry job: the + kernel side of the pipelining in orchard #119. The doorbell is rung + for every job unless its own stamps show it done, the rule of #125. +- Completion is polled, as on the A10: the interrupt controller's driver + cannot acknowledge the firmware's interrupt. While a job is waited + for, the host reads the firmware's memory once a millisecond, which + the firmware needs to see its own progress. +- A job that does not finish in the scheduler's 5 s ends every job in + flight; the firmware is stopped, the GPU's power domain goes off for + at least 10 ms through runtime PM, and the next job boots the firmware + again. No context is frozen and there is no fault budget. + +Ported from Berkay Kösebay's apple-g5p driver (orchard #74 with #119 and +#125): drivers/gpu/apple-g5p/render-user-layout.h, render-user-geometry.h, +render-user-msaa.h, render-user-bind.h, render-user-mmu.h, +render-user-pmset.h, render-user-wake.h, compute-user-layout.h, and +drivers/gpu/drm/apple-g5p/drm-render-v3.inc, drm-render.inc, +drm-compute-v1.inc. Every layout, offset and value is his. Not taken: his +own UAPI, VM and scheduler, the validation of userspace's addresses, his +job logs, switches and alternate paths. + +Not run on hardware. + +Assisted-by: LLM +--- + drivers/gpu/drm/imagination/pvr_apple_s8003.c | 1473 ++++++++++++++++- + 1 file changed, 1428 insertions(+), 45 deletions(-) + +diff --git a/drivers/gpu/drm/imagination/pvr_apple_s8003.c b/drivers/gpu/drm/imagination/pvr_apple_s8003.c +index 9b84009..468dd71 100644 +--- a/drivers/gpu/drm/imagination/pvr_apple_s8003.c ++++ b/drivers/gpu/drm/imagination/pvr_apple_s8003.c +@@ -23,28 +23,36 @@ + * The META slave port is not used: the firmware boots itself from the memory + * mapped for it. + * +- * Not here yet: the firmware's commands. No job is written, so the chip takes +- * none (pvr_apple_takes_jobs()), and the firmware's interrupt to the host is +- * not taken: its rings are read on a timer while it runs. ++ * Jobs are written as this firmware takes them, which is not the A10's way: ++ * see "Jobs" below. The firmware's interrupt to the host is not taken (the ++ * interrupt controller's driver cannot acknowledge it): its rings are read ++ * while a job is waited for, and on a timer otherwise. + * +- * The boot sequence, the layout of the shared memory and every value in it +- * are from Berkay Kösebay's G5P driver, which runs the GPU on the iPad 5 +- * (J71t); the names of the firmware's objects are his readings of Apple's +- * kext and firmware. ++ * The boot sequence, the layout of the shared memory, the job records and ++ * every value in them are from Berkay Kösebay's G5P driver, which runs the ++ * GPU on the iPad 5 (J71t); the names of the firmware's objects are his ++ * readings of Apple's kext and firmware. + */ + + #include "pvr_apple.h" + #include "pvr_apple_chip.h" ++#include "pvr_context.h" + #include "pvr_device.h" ++#include "pvr_free_list.h" + #include "pvr_fw_meta.h" + #include "pvr_gem.h" ++#include "pvr_hwrt.h" ++#include "pvr_job.h" + #include "pvr_mmu.h" + #include "pvr_rogue_cr_defs.h" + #include "pvr_rogue_meta.h" ++#include "pvr_stream_defs.h" + #include "pvr_vm.h" + + #include + #include ++#include ++#include + #include + #include + #include +@@ -57,6 +65,7 @@ + #include + #include + #include ++#include + + /* + * Fender beyond the power state. The boot stream is a list of Rogue register +@@ -106,6 +115,39 @@ + #define S8003_PIO_AIC 0x8000 + #define S8003_PIO_SIZE 0x8000 /* of each */ + ++/* The engines' channels: pages of the shared memory, past its objects */ ++#define S8003_TA_CHANNEL 0x54000 ++#define S8003_3D_CHANNEL 0x58000 ++#define S8003_CU_CHANNEL 0x5b000 ++ ++/* ++ * The parameter manager's sets, behind the shared memory: three pages each ++ * (control words with the scene's record at S8003_PM_RECORD, block list, free ++ * list). As many as fit below the 3 MiB the firmware is told at ++ * platform+0x77c; his driver had eight there. ++ */ ++#define S8003_PM_SETS 12 ++#define S8003_PM_SET_SIZE (3 * PVR_DEVICE_PAGE_SIZE) ++#define S8003_PM_SETS_VA (S8003_SHARED_VA + S8003_SHARED_SIZE) ++#define S8003_PM_RECORD 0x80 ++#define S8003_PM_BLOCK_SIZE SZ_128K /* eight pages of the pool */ ++#define S8003_PM_BLOCKS_MAX 128u /* what one page of free list holds */ ++/* ++ * The PM's own tables for a set (macrotile array 0x80 bytes, MList 0x10000, ++ * VHEAP 0x600), in the VM that uses the set, below the userspace heaps. ++ */ ++#define S8003_PM_AUX_VA 0x101000000ull ++#define S8003_PM_AUX_STRIDE 0x20000 ++#define S8003_PM_AUX_SIZE (5 * PVR_DEVICE_PAGE_SIZE) ++ ++/* The firmware's slots for VMs: 0 is its own, the first ones Apple's kext keeps */ ++#define S8003_SLOTS 256 ++#define S8003_SLOT_FIRST 7 ++/* A slot's tag: the field whose change drops the slot's translations from the catalogue down */ ++#define S8003_TAG_CATALOGUE 0x00400000 ++#define S8003_TAG_FIRST 0x00400000 ++#define S8003_TAG_PARK 0x00500000 ++ + /* The unpacked image: the bootloader's configuration and what it hands over */ + #define S8003_IMAGE_CONF 0x200 + #define S8003_IMAGE_HANDOVER 0x3ff8 /* META addresses of INIT and CONTROL */ +@@ -137,6 +179,8 @@ enum { + S8003_LOGS = 0x75, + S8003_STATS = 0x76, + S8003_TRACE = 0x77, ++ S8003_TAG_HOST = 0x84, /* per slot: the host's tag, */ ++ S8003_TAG_SEEN = 0x85, /* the one the firmware saw last, */ + S8003_MMU_ROOT = 0x86, /* the address of the page catalogue */ + S8003_OBJECTS = 0x90, + }; +@@ -195,9 +239,50 @@ struct s8003_idle { + __le32 event_producer; + }; + ++/* A PM set a VM took, and the scene in it */ ++struct s8003_pm { ++ int set; /* of the firmware's sets; -1: none */ ++ u32 blocks; /* of the pool */ ++ u32 built; /* the firmware start it was built for */ ++ struct pvr_gem_object *pool; ++ struct pvr_gem_object *aux; /* the PM's tables, mapped in the VM */ ++ const struct pvr_hwrt_data *hwrt; /* the scene's target */ ++ u32 scene; /* the firmware start the scene belongs to; 0: none */ ++ u32 ta_seq; /* its geometry job */ ++ u32 td_seq; /* its fragment job; 0: not submitted */ ++}; ++ ++struct s8003_vm { ++ struct pvr_vm_context *vm_ctx; ++ u32 slot[2]; /* geometry and compute, fragment; 0: none */ ++ u32 attached; /* the firmware start the slots were loaded for */ ++ struct s8003_pm pm[2]; ++}; ++ + struct pvr_apple_s8003 { + struct pvr_apple *apple; + struct delayed_work drain_work; /* reads the rings while the firmware runs */ ++ struct mutex event_lock; /* the rings the firmware posts to */ ++ ktime_t touched; /* s8003_touch() */ ++ bool failed; /* its rings' indices are not ours any more */ ++ bool fault; /* it posted an event that is neither "alive" nor a stamp */ ++ ++ /* ++ * Per VM (struct s8003_vm, by its VM context) and per start of the ++ * firmware: @gen counts the starts, and what the firmware was told ++ * for an earlier one is told again. ++ */ ++ struct xarray vms; ++ u32 gen; ++ spinlock_t tag_lock; /* @tag and the host's tags in the shared memory */ ++ u32 tag[S8003_SLOTS]; /* 0: slot not loaded */ ++ DECLARE_BITMAP(slots, S8003_SLOTS); ++ DECLARE_BITMAP(parked, S8003_SLOTS); /* left on @park_vm's tables */ ++ u32 next_slot; ++ spinlock_t set_lock; /* @sets, @next_set, &s8003_pm.set */ ++ DECLARE_BITMAP(sets, S8003_PM_SETS); ++ u32 next_set; ++ struct pvr_vm_context *park_vm; /* maps what every VM maps, nothing else */ + + /* Bus addresses the firmware is told: its registers' and ours */ + phys_addr_t fender_phys; +@@ -215,6 +300,7 @@ struct pvr_apple_s8003 { + struct s8003_mem fw; /* the image the firmware runs in */ + struct s8003_mem root; + struct s8003_mem shared; ++ struct s8003_mem pmsets; + struct s8003_mem zero; /* at GPU address 0 */ + struct pvr_gem_object *spare; + bool pio_mapped; +@@ -293,7 +379,7 @@ static int s8003_layout_plan(struct s8003_layout *layout) + layout->shared[objects[i].object] = objects[i].shared; + used[objects[i].shared] = offset + objects[i].size; + } +- return used[0] <= S8003_ROOT_SIZE && used[1] <= S8003_SHARED_SIZE ? 0 : -EINVAL; ++ return used[0] <= S8003_ROOT_SIZE && used[1] <= S8003_TA_CHANNEL ? 0 : -EINVAL; + } + + static u64 s8003_object_va(const struct s8003_layout *layout, u32 object) +@@ -529,8 +615,9 @@ static void s8003_init_build(struct pvr_apple *apple, u32 q) + + put64(platform, 0x600000000ull); + put64(platform + 8, 0xffc00000); +- put64(platform + 0x10, 0x300000000ull); +- put64(platform + 0x18, 0x300000000ull); ++ /* What PDS and USC code addresses are relative to: powervr's code heaps */ ++ put64(platform + 0x10, PVR_APPLE_PDSCODEDATA_HEAP_BASE); ++ put64(platform + 0x18, PVR_APPLE_USCCODE_HEAP_BASE); + for (i = 0; i < ARRAY_SIZE(pio); i++) { + u8 *entry = platform + 0x630 + 16 * i; + +@@ -627,7 +714,12 @@ static void pvr_apple_s8003_fini(struct pvr_apple *apple) + pvr_vm_unmap_io(vm_ctx, S8003_PIO_VA + S8003_PIO_AIC, S8003_PIO_SIZE); + s->pio_mapped = false; + } ++ if (s->park_vm) { ++ pvr_vm_context_put(s->park_vm); ++ s->park_vm = NULL; ++ } + s8003_mem_fini(apple, &s->zero); ++ s8003_mem_fini(apple, &s->pmsets); + s8003_mem_fini(apple, &s->shared); + s8003_mem_fini(apple, &s->root); + s8003_mem_fini(apple, &s->fw); +@@ -659,6 +751,10 @@ static int s8003_map(struct pvr_apple *apple) + if (!err) + err = s8003_mem_init(apple, &s->shared, S8003_SHARED_VA, S8003_SHARED_SIZE, + DRM_PVR_BO_PM_FW_PROTECT | PVR_BO_DEVICE_CACHE_COHERENT); ++ if (!err) ++ err = s8003_mem_init(apple, &s->pmsets, S8003_PM_SETS_VA, ++ S8003_PM_SETS * S8003_PM_SET_SIZE, ++ DRM_PVR_BO_PM_FW_PROTECT | PVR_BO_DEVICE_CACHE_COHERENT); + if (!err) + err = s8003_mem_init(apple, &s->zero, 0, PVR_DEVICE_PAGE_SIZE, + PVR_BO_FW_FLAGS_DEVICE_CACHED); +@@ -844,8 +940,10 @@ static int s8003_ring_discard(u8 *ctl) + /* + * Read what the firmware posted, in the order Apple's kext does: the log + * lanes, the events, statistics, trace. A full ring stalls the firmware. +- * Events other than "alive" and a job's stamp mean the firmware stopped (a +- * fault, a recovery request): none is expected before a job ran. ++ * Events other than "alive" and a job's stamp are a fault or a recovery ++ * request: they are logged, and a job the firmware gave up times out. An ++ * error is a ring whose indices are not the host's and the firmware's any ++ * more. + */ + static int s8003_drain(struct pvr_apple *apple, bool *alive) + { +@@ -856,6 +954,7 @@ static int s8003_drain(struct pvr_apple *apple, bool *alive) + unsigned int lane; + int err = 0; + ++ guard(mutex)(&s->event_lock); + for (lane = 0; lane < 4 && !err; lane++) + err = s8003_ring_discard(s8003_object(s, S8003_LOG_CTL) + + lane * S8003_RING_CTL_SIZE); +@@ -874,15 +973,13 @@ static int s8003_drain(struct pvr_apple *apple, bool *alive) + if (type == S8003_EVENT_ALIVE) { + *alive = true; + } else if (type != S8003_EVENT_STAMP) { +- dev_err(apple_dev(apple), "firmware event %u: %*ph\n", type, +- S8003_EVENT_SIZE - 4, e + 4); +- err = -EIO; ++ dev_warn_ratelimited(apple_dev(apple), "firmware event %u: %*ph\n", type, ++ S8003_EVENT_SIZE - 4, e + 4); ++ WRITE_ONCE(s->fault, true); + } + consumer = (consumer + 1) % APPLE_RING_ENTRIES; + ring_consume(ctl, consumer); + } +- if (err) +- return err; + + err = s8003_ring_discard(s8003_object(s, S8003_STATS_CTL)); + if (!err) +@@ -937,67 +1034,1323 @@ static bool s8003_core_ready(struct pvr_apple *apple) + } + + /* +- * Make the firmware look at the device-control ring: Fender's wake first, as +- * Apple's kext does before every doorbell, then the META's scheduler. ++ * Jobs. ++ * ++ * Each engine has one channel, a page of the shared memory: a ring of 16 ++ * pointers with five watermarks (the host's own among them), two stamps, a ++ * timestamp queue and a dependency object. A job is a work record ("W") the ++ * firmware loads the engine's registers from, plus a microsequence the ++ * firmware runs for it: start, wait for the engine, finish, each step naming ++ * the addresses it reads and the stamp it writes. A render job also has a ++ * small command ahead of its record: for a geometry job the parameter ++ * manager's record to start from, for a fragment job the geometry job's stamp ++ * to wait for. The pointer ring names command and record, an entry of the ++ * engine's 256-entry work ring names the channel and how far the pointer ring ++ * is filled, and the doorbell is the META's scheduler register. ++ * ++ * A job is done when both stamps of its channel reached its number and all ++ * five watermarks its end of the pointer ring. ++ * ++ * A VM is known to the firmware as a pair of "slots", one for geometry and ++ * compute jobs, one for fragment jobs: two device-control commands make the ++ * firmware load the VM's page-table root for them. The firmware compares a ++ * tag per slot with the one it saw last whenever a job starts and drops the ++ * slot's translations if the host changed it. ++ * ++ * The parameter manager's state for one scene is a "PM set": three pages in ++ * the firmware's GiB (control words and the scene's record, the list of the ++ * pool's blocks, the free list), a pool of pages and the pages the PM keeps ++ * its own tables in. Every VM maps all sets' three pages where the firmware ++ * has them, protected as the firmware's are, because the PM reaches them ++ * through the job's slot. A VM takes up to two sets, one per scene in the ++ * firmware, with its first render. ++ */ ++ ++/* A channel page */ ++#define S8003_CH_CTL 0x100 /* watermarks, 16 bytes apart */ ++#define S8003_CH_CTL_HOST 0x40 /* the host's: how far the ring is filled */ ++#define S8003_CH_CTL_ENTRIES 0x50 ++#define S8003_CH_RING 0x200 ++#define S8003_CH_ENTRIES 16 ++#define S8003_CH_CMD 0x300 ++#define S8003_CH_W 0x400 ++#define S8003_CH_TA_USEQ 0xa00 ++#define S8003_CH_3D_USEQ 0xd00 ++#define S8003_CH_TQ 0x1800 ++#define S8003_CH_STAMP 0x1840 /* two words */ ++#define S8003_CH_DEP 0x1900 ++#define S8003_CH_SLOT1 0x1c00 /* every second job: command, W, microsequence */ ++#define S8003_CH_TIMES 0x2800 /* a job's two timestamps, 16 bytes per slot */ ++ ++#define S8003_CMD_SIZE 0x40 ++#define S8003_TA_W_SIZE 0x560 ++#define S8003_3D_W_SIZE 0x8c0 ++#define S8003_TA_USEQ_SIZE 0xc0 ++#define S8003_3D_USEQ_SIZE 0x180 ++ ++/* The compute channel: a quarter page, one job at a time */ ++#define S8003_CU_W_SIZE 0x290 ++#define S8003_CU_USEQ 0x800 ++#define S8003_CU_USEQ_SIZE 0xc0 ++#define S8003_CU_TQ 0x900 ++#define S8003_CU_STAMP 0x940 /* two words */ ++#define S8003_CU_DEP 0x980 ++ ++/* Commands ahead of a render job's record */ ++#define S8003_CMD_WAIT 4 /* for a stamp to reach a number */ ++#define S8003_CMD_PM 6 /* the PM record a geometry job starts from */ ++ ++#define S8003_DC_SLOT 3 /* (re)load a slot's page-table root */ ++ ++#define S8003_JOB_TIMEOUT_US 5000000 ++#define S8003_RING_TIMEOUT_US 1000000 ++ ++static u32 get32(const u8 *p) ++{ ++ return get_unaligned_le32(p); ++} ++ ++/* The firmware numbers the engines: geometry, fragment, compute */ ++static u32 s8003_engine(enum pvr_apple_engine engine) ++{ ++ return engine == APPLE_ENGINE_GEOM ? 0 : engine == APPLE_ENGINE_FRAG ? 1 : 2; ++} ++ ++static u32 s8003_channel_offset(enum pvr_apple_engine engine) ++{ ++ static const u32 offset[] = { S8003_TA_CHANNEL, S8003_3D_CHANNEL, S8003_CU_CHANNEL }; ++ ++ return offset[s8003_engine(engine)]; ++} ++ ++static u8 *s8003_channel(struct pvr_apple_s8003 *s, enum pvr_apple_engine engine) ++{ ++ return s->shared.map + s8003_channel_offset(engine); ++} ++ ++static u32 s8003_channel_addr(enum pvr_apple_engine engine) ++{ ++ return s8003_meta_addr(S8003_SHARED_VA + s8003_channel_offset(engine)); ++} ++ ++/* A channel before its first job: where its parts are, an empty ring */ ++static void s8003_channel_init(u8 *p, u32 addr, u32 tq, u32 dep) ++{ ++ u32 off; ++ ++ put32(p, addr + S8003_CH_CTL); ++ put32(p + 4, addr + S8003_CH_RING); ++ put32(p + 8, addr + tq); ++ put32(p + 0xc, 4); ++ put32(p + 0x18, ~0u); ++ put32(p + S8003_CH_CTL + S8003_CH_CTL_ENTRIES, S8003_CH_ENTRIES); ++ put32(p + tq + 4, addr + tq); ++ put64(p + dep + 0x30, 1); ++ put64(p + dep + 0x48, ~0ull); ++ put32(p + dep + 0x50, ~0u); ++ for (off = 0x70; off < 0x270; off += 0x10) ++ put32(p + dep + off + 8, 0xf); ++} ++ ++static void s8003_channels_init(struct pvr_apple_s8003 *s) ++{ ++ s8003_channel_init(s8003_channel(s, APPLE_ENGINE_GEOM), ++ s8003_channel_addr(APPLE_ENGINE_GEOM), S8003_CH_TQ, S8003_CH_DEP); ++ s8003_channel_init(s8003_channel(s, APPLE_ENGINE_FRAG), ++ s8003_channel_addr(APPLE_ENGINE_FRAG), S8003_CH_TQ, S8003_CH_DEP); ++ s8003_channel_init(s8003_channel(s, APPLE_ENGINE_COMPUTE), ++ s8003_channel_addr(APPLE_ENGINE_COMPUTE), S8003_CU_TQ, S8003_CU_DEP); ++} ++ ++/* ++ * While a job runs the firmware does not see its own progress in the root and ++ * shared memory until the host read those cache lines: without the reads a ++ * phase waited for the firmware's next tick, seconds in all (measured on the ++ * iPad 5: 7.9 s from input to frame, 0.27 s with them every 1 to 2 ms). One ++ * load per 64-byte line, the PM sets in use included. ++ */ ++static void s8003_touch(struct pvr_apple_s8003 *s) ++{ ++ unsigned long set; ++ u32 at; ++ ++ for (at = 0; at < S8003_ROOT_SIZE; at += 64) ++ (void)READ_ONCE(*(const u32 *)(s->root.map + at)); ++ for (at = 0; at < S8003_SHARED_SIZE; at += 64) ++ (void)READ_ONCE(*(const u32 *)(s->shared.map + at)); ++ for_each_set_bit(set, s->sets, S8003_PM_SETS) ++ for (at = 0; at < S8003_PM_SET_SIZE; at += 64) ++ (void)READ_ONCE(*(const u32 *)(s->pmsets.map + set * S8003_PM_SET_SIZE + at)); ++} ++ ++/* What the firmware posted since the last look; wherever a job is waited for */ ++static void pvr_apple_s8003_events(struct pvr_apple *apple) ++{ ++ struct pvr_apple_s8003 *s = apple->chip_data; ++ ktime_t now = ktime_get(); ++ bool alive = false; ++ ++ if (!READ_ONCE(apple->booted) || !s->shared.map) ++ return; ++ if (ktime_us_delta(now, READ_ONCE(s->touched)) >= USEC_PER_MSEC) { ++ WRITE_ONCE(s->touched, now); ++ s8003_touch(s); ++ } ++ if (!READ_ONCE(s->failed) && s8003_drain(apple, &alive)) { ++ WRITE_ONCE(s->failed, true); ++ dev_err(apple_dev(apple), "GFX firmware's rings are no longer read\n"); ++ } ++} ++ ++static u32 s8003_submitted(struct pvr_apple *apple, enum pvr_apple_engine engine) ++{ ++ return READ_ONCE(*(engine == APPLE_ENGINE_GEOM ? &apple->ta_seq : ++ engine == APPLE_ENGINE_FRAG ? &apple->frag_seq : &apple->last_seq)); ++} ++ ++/* ++ * The last job of @engine that is done: both stamps at its number or past it ++ * (the firmware only moves a stamp forward), and every watermark between its ++ * end of the pointer ring and that of the last job submitted. With the ++ * firmware off nothing is in flight. ++ */ ++static u32 pvr_apple_s8003_engine_done(struct pvr_apple *apple, enum pvr_apple_engine engine) ++{ ++ struct pvr_apple_s8003 *s = apple->chip_data; ++ u32 latest = s8003_submitted(apple, engine); ++ u32 per_job = engine == APPLE_ENGINE_COMPUTE ? 1 : 2; ++ u32 stamp = engine == APPLE_ENGINE_COMPUTE ? S8003_CU_STAMP : S8003_CH_STAMP; ++ u32 done, end, span, off; ++ const u8 *p; ++ s32 behind; ++ ++ if (!READ_ONCE(apple->booted) || !s->shared.map) ++ return latest; ++ p = s8003_channel(s, engine); ++ behind = min_t(s32, get32(p + stamp) - latest, get32(p + stamp + 4) - latest); ++ done = latest + min(behind, 0); ++ end = (per_job * done) % S8003_CH_ENTRIES; ++ span = (per_job * (latest - done)) % S8003_CH_ENTRIES; ++ for (off = 0; off <= S8003_CH_CTL_HOST; off += 0x10) { ++ u32 mark = get32(p + S8003_CH_CTL + off); ++ ++ if (mark >= S8003_CH_ENTRIES || ++ (mark + S8003_CH_ENTRIES - end) % S8003_CH_ENTRIES > span) ++ return done - 1; ++ } ++ return done; ++} ++ ++static u32 s8003_done_poll(struct pvr_apple *apple, enum pvr_apple_engine engine) ++{ ++ pvr_apple_s8003_events(apple); ++ return pvr_apple_s8003_engine_done(apple, engine); ++} ++ ++static int s8003_wait_done(struct pvr_apple *apple, enum pvr_apple_engine engine, u32 seq) ++{ ++ u32 done; ++ ++ return read_poll_timeout(s8003_done_poll, done, (s32)(done - seq) >= 0, 20, ++ S8003_JOB_TIMEOUT_US, false, apple, engine); ++} ++ ++/* ++ * Wake the firmware's scheduler for @engine, as Apple's kext does after every ++ * ring entry: Fender's wake, then the META's scheduler register. A core whose ++ * power state is changing is not rung: the firmware looks at its rings when ++ * it is done with that. + */ +-static int s8003_dc_doorbell(struct pvr_apple *apple) ++static int s8003_doorbell(struct pvr_apple *apple, u32 dm) + { + writel(FENDER_WAKE_ALL, apple->fender + FENDER_WAKE); + if (readl(apple->fender + FENDER_WAKE) & ~FENDER_WAKE_ALL) + return -EPROTO; + if (!s8003_core_ready(apple)) +- return -EHOSTDOWN; +- pvr_cr_write32(apple->pvr_dev, ROGUE_CR_MTS_SCHEDULE, S8003_MTS_DEVICE_CONTROL); ++ return 0; ++ wmb(); /* what the firmware is rung for is in memory */ ++ pvr_cr_write32(apple->pvr_dev, ROGUE_CR_MTS_SCHEDULE, ++ ROGUE_CR_MTS_SCHEDULE_TASK_COUNTED | dm); + return 0; + } + +-static u32 s8003_dc_consumer(struct pvr_apple *apple, int *err) ++static u32 s8003_ring_consumer(struct pvr_apple *apple, u8 *ctl) + { +- struct pvr_apple_s8003 *s = apple->chip_data; +- bool alive = false; +- +- *err = s8003_drain(apple, &alive); +- return ring_read(s8003_object(s, S8003_DC_CTL), S8003_RING_CONSUMER); ++ pvr_apple_s8003_events(apple); ++ return ring_read(ctl, S8003_RING_CONSUMER); + } + + /* +- * The first command: an empty device-control command. The firmware takes it +- * from the ring once its scheduler runs, which is when it takes commands at +- * all. ++ * Give the firmware @n device-control commands of two words and wait until ++ * it took them. + */ +-static int s8003_first_command(struct pvr_apple *apple) ++static int s8003_device_control(struct pvr_apple *apple, const u32 (*cmd)[2], unsigned int n) + { + struct pvr_apple_s8003 *s = apple->chip_data; + u8 *ctl = s8003_object(s, S8003_DC_CTL); + u8 *ring = s8003_object(s, S8003_DC_RING); + u32 producer = ring_read(ctl, S8003_RING_PRODUCER); + u32 consumer, target; +- int err, drained = 0; ++ unsigned int i; ++ int err; + + if (producer >= APPLE_RING_ENTRIES || ring_read(ctl, S8003_RING_CONSUMER) != producer) + return -EPROTO; +- target = (producer + 1) % APPLE_RING_ENTRIES; ++ for (i = 0; i < n; i++) { ++ u8 *entry = ring + ((producer + i) % APPLE_RING_ENTRIES) * S8003_DC_SIZE; + +- put32(s8003_object(s, S8003_CONTROL) + 0x1020, 0); /* no payload */ +- memset(ring + producer * S8003_DC_SIZE, 0, S8003_DC_SIZE); +- put32(ring + producer * S8003_DC_SIZE, S8003_DC_NONE); ++ memset(entry, 0, S8003_DC_SIZE); ++ put32(entry, cmd[i][0]); ++ put32(entry + 4, cmd[i][1]); ++ } ++ target = (producer + n) % APPLE_RING_ENTRIES; + wmb(); + WRITE_ONCE(*(__le32 *)(ctl + S8003_RING_PRODUCER), cpu_to_le32(target)); + mb(); /* the index is in memory before the firmware is told */ + +- err = s8003_dc_doorbell(apple); +- if (!err) { +- err = read_poll_timeout(s8003_dc_consumer, consumer, drained || consumer == target, +- 1000, S8003_DC_TIMEOUT_US, false, apple, &drained); +- if (!err) +- err = drained; ++ err = s8003_doorbell(apple, ROGUE_CR_MTS_SCHEDULE_DM_DM0); ++ if (err) ++ return err; ++ return read_poll_timeout(s8003_ring_consumer, consumer, consumer == target, 100, ++ S8003_DC_TIMEOUT_US, false, apple, ctl); ++} ++ ++/* Point a pair of slots at a page-table root (0: at none) under @tag */ ++static int s8003_slots_load(struct pvr_apple *apple, const u32 *slot, u64 root, u32 tag) ++{ ++ const u32 cmd[2][2] = { { S8003_DC_SLOT, slot[1] }, { S8003_DC_SLOT, slot[0] } }; ++ struct pvr_apple_s8003 *s = apple->chip_data; ++ unsigned long flags; ++ unsigned int e; ++ int err; ++ ++ spin_lock_irqsave(&s->tag_lock, flags); ++ for (e = 0; e < 2; e++) { ++ put64(s8003_object(s, S8003_MMU_ROOT) + 8 * slot[e], root); ++ put32(s8003_object(s, S8003_TAG_HOST) + 4 * slot[e], tag); ++ s->tag[slot[e]] = tag; ++ } ++ spin_unlock_irqrestore(&s->tag_lock, flags); ++ ++ err = s8003_device_control(apple, cmd, 2); ++ /* The firmware copies the tag it loaded the root under */ ++ for (e = 0; e < 2 && !err; e++) { ++ u32 seen = get32(s8003_object(s, S8003_TAG_SEEN) + 4 * slot[e]); ++ ++ if (seen != tag && seen != READ_ONCE(s->tag[slot[e]])) ++ err = -EIO; + } ++ if (err) ++ dev_err(apple_dev(apple), "GFX firmware did not load slots %u and %u: %d\n", ++ slot[0], slot[1], err); ++ return err; ++} ++ ++/* ++ * The slot pair of a VM for this start of the firmware. Slots are handed out ++ * in turn, so that one a VM gave back is used again as late as possible; if ++ * it was left on the park tables (pvr_apple_s8003_vm_fini()) it is emptied ++ * first, as Apple's kext does when an address space goes. ++ */ ++static int s8003_vm_attach(struct pvr_apple *apple, struct s8003_vm *vm) ++{ ++ struct pvr_apple_s8003 *s = apple->chip_data; ++ unsigned int e; ++ int err = 0; ++ ++ lockdep_assert_held(&apple->submit_lock); ++ if (vm->attached == s->gen) ++ return 0; ++ for (e = 2; e-- > 0 && !vm->slot[e];) { ++ u32 slot = find_next_zero_bit(s->slots, S8003_SLOTS, s->next_slot); ++ ++ if (slot >= S8003_SLOTS) ++ slot = find_next_zero_bit(s->slots, S8003_SLOTS, S8003_SLOT_FIRST); ++ if (slot >= S8003_SLOTS) { ++ if (vm->slot[1]) ++ __clear_bit(vm->slot[1], s->slots); ++ vm->slot[1] = 0; ++ return -EBUSY; ++ } ++ __set_bit(slot, s->slots); ++ s->next_slot = slot + 1 < S8003_SLOTS ? slot + 1 : S8003_SLOT_FIRST; ++ vm->slot[e] = slot; ++ } ++ /* No job runs while a slot changes */ ++ err = pvr_apple_wait_idle(apple, S8003_JOB_TIMEOUT_US); ++ if (!err && (test_bit(vm->slot[0], s->parked) || test_bit(vm->slot[1], s->parked))) { ++ err = s8003_slots_load(apple, vm->slot, 0, 0); ++ __clear_bit(vm->slot[0], s->parked); ++ __clear_bit(vm->slot[1], s->parked); ++ } ++ if (!err) ++ err = s8003_slots_load(apple, vm->slot, ++ pvr_vm_get_page_table_root_addr(vm->vm_ctx), S8003_TAG_FIRST); ++ if (!err) ++ vm->attached = s->gen; ++ return err; ++} ++ ++/* ++ * Page tables changed: move the tag of every slot in use, in the field that ++ * makes the firmware drop the slot's translations down from the catalogue ++ * when its next job starts. Never to 0, the tag of an empty slot. ++ */ ++static void pvr_apple_s8003_mmu_flush(struct pvr_apple *apple) ++{ ++ struct pvr_apple_s8003 *s = apple->chip_data; ++ unsigned long flags; ++ u32 slot; ++ ++ if (!s->shared.map) ++ return; ++ spin_lock_irqsave(&s->tag_lock, flags); ++ for (slot = S8003_SLOT_FIRST; slot < S8003_SLOTS; slot++) { ++ u32 tag = s->tag[slot]; ++ ++ if (!tag) ++ continue; ++ tag += S8003_TAG_CATALOGUE; ++ if (!tag) ++ tag = S8003_TAG_CATALOGUE; ++ s->tag[slot] = tag; ++ put32(s8003_object(s, S8003_TAG_HOST) + 4 * slot, tag); ++ } ++ spin_unlock_irqrestore(&s->tag_lock, flags); ++ wmb(); ++} ++ ++static int pvr_apple_s8003_vm_init(struct pvr_apple *apple, struct pvr_vm_context *vm_ctx) ++{ ++ struct pvr_apple_s8003 *s = apple->chip_data; ++ struct s8003_vm *vm; ++ int err; ++ ++ if (!s->pmsets.obj) ++ return -ENODEV; ++ vm = kzalloc(sizeof(*vm), GFP_KERNEL); ++ if (!vm) ++ return -ENOMEM; ++ vm->vm_ctx = vm_ctx; ++ vm->pm[0].set = -1; ++ vm->pm[1].set = -1; ++ /* The PM reaches its sets through the job's slot, at the firmware's addresses */ ++ err = pvr_vm_map_reserved(vm_ctx, s->pmsets.obj, S8003_PM_SETS_VA, s->pmsets.size); ++ if (!err) ++ err = xa_err(xa_store(&s->vms, (unsigned long)vm_ctx, vm, GFP_KERNEL)); ++ if (err) ++ kfree(vm); ++ return err; ++} ++ ++/* ++ * How many scenes of a VM the firmware may hold: as many as the VM has PM ++ * sets. It takes its first here, and its second while the firmware has more ++ * than half of them left. Called when a job is ordered, in atomic context. ++ */ ++static unsigned int pvr_apple_s8003_vm_scenes(struct pvr_apple *apple, ++ struct pvr_vm_context *vm_ctx) ++{ ++ struct pvr_apple_s8003 *s = apple->chip_data; ++ struct s8003_vm *vm = xa_load(&s->vms, (unsigned long)vm_ctx); ++ unsigned int i, n = 0; ++ unsigned long flags; ++ ++ if (!vm) ++ return 1; ++ spin_lock_irqsave(&s->set_lock, flags); ++ for (i = 0; i < 2; i++) { ++ unsigned long set; ++ ++ if (vm->pm[i].set >= 0) { ++ n++; ++ continue; ++ } ++ if (i && bitmap_weight(s->sets, S8003_PM_SETS) >= S8003_PM_SETS / 2) ++ break; ++ /* In turn: the PM may still write a set a VM gave back */ ++ set = find_next_zero_bit(s->sets, S8003_PM_SETS, s->next_set); ++ if (set >= S8003_PM_SETS) ++ set = find_first_zero_bit(s->sets, S8003_PM_SETS); ++ if (set >= S8003_PM_SETS) ++ break; ++ __set_bit(set, s->sets); ++ s->next_set = (set + 1) % S8003_PM_SETS; ++ vm->pm[i].set = set; ++ n++; ++ } ++ spin_unlock_irqrestore(&s->set_lock, flags); ++ return max(n, 1u); ++} ++ ++static void s8003_pm_put(struct pvr_apple_s8003 *s, struct s8003_pm *pm) ++{ ++ unsigned long flags; ++ ++ spin_lock_irqsave(&s->set_lock, flags); ++ if (pm->set >= 0) ++ __clear_bit(pm->set, s->sets); ++ spin_unlock_irqrestore(&s->set_lock, flags); ++ if (pm->pool) ++ pvr_gem_object_put(pm->pool); ++ if (pm->aux) ++ pvr_gem_object_put(pm->aux); ++} ++ ++/* ++ * A VM goes; its jobs are done. The PM still writes the free list of the VM ++ * whose scene it handled last, or the one before, through that VM's geometry ++ * slot when a later job runs (seen on the iPad 5 as page faults in a closed ++ * context's slot). So the slots are not emptied but put on the park VM's ++ * tables, which map the PM sets and nothing else, and the sets are taken ++ * again as late as possible. ++ */ ++static void pvr_apple_s8003_vm_fini(struct pvr_apple *apple, struct pvr_vm_context *vm_ctx) ++{ ++ struct pvr_apple_s8003 *s = apple->chip_data; ++ struct s8003_vm *vm = xa_erase(&s->vms, (unsigned long)vm_ctx); ++ unsigned int e; ++ ++ if (!vm) ++ return; ++ mutex_lock(&apple->submit_lock); ++ if (apple->booted && vm->attached == s->gen && s->park_vm && vm_ctx != s->park_vm && ++ !pvr_apple_wait_idle(apple, S8003_JOB_TIMEOUT_US) && ++ !s8003_slots_load(apple, vm->slot, pvr_vm_get_page_table_root_addr(s->park_vm), ++ S8003_TAG_PARK)) { ++ __set_bit(vm->slot[0], s->parked); ++ __set_bit(vm->slot[1], s->parked); ++ } ++ for (e = 0; e < 2; e++) { ++ if (vm->slot[e]) ++ __clear_bit(vm->slot[e], s->slots); ++ s8003_pm_put(s, &vm->pm[e]); ++ } ++ mutex_unlock(&apple->submit_lock); ++ kfree(vm); ++} ++ ++static int s8003_obj_zero(struct pvr_gem_object *obj) ++{ ++ void *map = pvr_gem_object_vmap(obj); ++ ++ if (IS_ERR(map)) ++ return PTR_ERR(map); ++ memset(map, 0, pvr_gem_object_size(obj)); ++ pvr_gem_object_vunmap(obj); ++ return 0; ++} ++ ++static u64 s8003_pm_set_va(const struct s8003_pm *pm, u32 page) ++{ ++ return S8003_PM_SETS_VA + pm->set * S8003_PM_SET_SIZE + page * PVR_DEVICE_PAGE_SIZE; ++} ++ ++static u64 s8003_pm_aux_va(const struct s8003_pm *pm) ++{ ++ return S8003_PM_AUX_VA + (u64)pm->set * S8003_PM_AUX_STRIDE; ++} ++ ++/* ++ * A PM set for @vm with a pool of @blocks blocks, built anew for this start ++ * of the firmware: the pool's pages are 16 KiB pages of ours, listed per ++ * block (eight pages each, the blocks linked by their META addresses) and ++ * once more as the free list, four 4 KiB page numbers a page; the scene's ++ * record names the PM's own tables (their GPU addresses in the VM) and is ++ * otherwise empty. The pool is not rewritten between jobs: each geometry ++ * job's command has the firmware rebuild the free list from the block list. ++ */ ++static int s8003_pm_build(struct pvr_apple *apple, struct s8003_vm *vm, struct s8003_pm *pm, ++ u32 blocks) ++{ ++ struct pvr_apple_s8003 *s = apple->chip_data; ++ u8 *control, *list, *free, *record; ++ u32 previous = 0, i, j, k; ++ u64 list_va, aux_va; ++ int err; ++ ++ if (!pm->aux) { ++ struct pvr_gem_object *aux; ++ ++ aux = pvr_gem_object_create(apple->pvr_dev, S8003_PM_AUX_SIZE, ++ DRM_PVR_BO_PM_FW_PROTECT); ++ if (IS_ERR(aux)) ++ return PTR_ERR(aux); ++ err = pvr_vm_map_reserved(vm->vm_ctx, aux, s8003_pm_aux_va(pm), S8003_PM_AUX_SIZE); ++ if (err) { ++ pvr_gem_object_put(aux); ++ return err; ++ } ++ pm->aux = aux; ++ } ++ if (pm->built == s->gen && pm->blocks >= blocks) ++ return 0; ++ if (pm->blocks < blocks) { ++ struct pvr_gem_object *pool; ++ ++ pool = pvr_gem_object_create(apple->pvr_dev, (size_t)blocks * S8003_PM_BLOCK_SIZE, ++ PVR_BO_FW_FLAGS_DEVICE_CACHED); ++ if (IS_ERR(pool)) ++ return PTR_ERR(pool); ++ if (pm->pool) ++ pvr_gem_object_put(pm->pool); ++ pm->pool = pool; ++ pm->blocks = blocks; ++ } ++ pm->built = 0; ++ err = s8003_obj_zero(pm->pool); ++ if (!err) ++ err = s8003_obj_zero(pm->aux); ++ if (err) ++ return err; ++ ++ control = s->pmsets.map + pm->set * S8003_PM_SET_SIZE; ++ list = control + PVR_DEVICE_PAGE_SIZE; ++ free = list + PVR_DEVICE_PAGE_SIZE; ++ list_va = s8003_pm_set_va(pm, 1); ++ aux_va = s8003_pm_aux_va(pm); ++ memset(control, 0, S8003_PM_SET_SIZE); ++ for (j = 0; j < pm->blocks; j++) { ++ u32 at = 0x80 + j * 0x24; ++ ++ put32(list + at, previous); ++ for (k = 0; k < 8; k++) { ++ dma_addr_t page; ++ u32 ppn; ++ ++ err = pvr_gem_get_dma_addr(pm->pool, (j * 8 + k) * PVR_DEVICE_PAGE_SIZE, ++ &page); ++ if (err) ++ return err; ++ ppn = page >> 12; ++ put32(list + at + 4 + k * 4, ppn); ++ for (i = 0; i < 4; i++) ++ put32(free + ((j * 8 + k) * 4 + i) * 4, ppn + i); ++ } ++ previous = s8003_meta_addr(list_va + at); ++ } ++ put32(list + 0x60, 1); ++ put32(control + 0x10, previous); ++ put64(control + 0x14, s8003_pm_set_va(pm, 2)); ++ put32(control + 0x1c, pm->blocks * 32 * 4); ++ put32(control + 0x20, pm->blocks * 32); ++ put32(control + 0x24, pm->blocks * 32 - 1); ++ put32(control + 0x34, PVR_DEVICE_PAGE_SIZE); ++ put32(control + 0x40, s8003_meta_addr(list_va)); ++ put32(control + 0x54, pm->blocks * 32); ++ put32(control + 0x58, pm->blocks * 32); ++ /* The PM's tables: macrotile array, MList, VHEAP */ ++ record = control + S8003_PM_RECORD; ++ put64(record + 0xa8, aux_va); ++ put64(record + 0xa0, aux_va + 0x80); ++ put64(record + 0xb0, aux_va + 0x80 + 0x10000); ++ put32(record + 0xb8, s8003_meta_addr(list_va + 0x40)); ++ wmb(); ++ pm->built = s->gen; ++ return 0; ++} ++ ++/* ++ * A render target's geometry as the firmware takes it with each job: Apple's ++ * kext computes it from the size and the samples (AGXRenderTargetMemoryG5), ++ * in the Rogue parameter format with 32x32 tiles, 4x4 macrotiles, five-byte ++ * region headers per padded tile and a tail pointer cache of ++ * next_pow2(padded tiles)^2 entries of eight bytes. Integer arithmetic for ++ * the kext's three pairs of single-precision scale values. ++ */ ++struct s8003_rt { ++ u32 header; /* bytes of region headers */ ++ u32 region; ++ u64 positions; /* the standard sample positions */ ++ u32 state, padded; ++ u64 scale[3]; ++ u64 ta_state; ++ u64 ta_tiling[2]; ++ u32 ta_tiling_word, tiles; ++ u64 tpc; /* bytes of the tail pointer cache */ ++}; ++ ++#define S8003_RT_MAX_DIM 8192 ++#define S8003_RT_SCALE_A 0x3e89309cu ++#define S8003_RT_SCALE_B 0x3fddb3d9u ++#define S8003_POSITIONS_1X 0x88ull ++#define S8003_POSITIONS_4X 0xeaa26e26ull ++ ++/* The single-precision float of 1 <= n < 2^24 */ ++static u32 s8003_f32(u32 n) ++{ ++ u32 bits = fls(n); ++ ++ return ((126 + bits) << 23) | ((n << (24 - bits)) & 0x7fffff); ++} ++ ++/* A normal positive single divided by 1 <= n < 2^24, rounded to nearest even */ ++static u32 s8003_fdiv(u32 a, u32 n) ++{ ++ u32 shift = fls(n) + 3; ++ u64 num = (u64)((a & 0x7fffff) | 0x800000) << shift; ++ u32 q = div_u64(num, n), sticky = (u64)q * n != num; ++ u32 s = fls(q) - 24, m = q >> s, low = q & ((1u << s) - 1); ++ ++ if (low > 1u << (s - 1) || (low == 1u << (s - 1) && (sticky || (m & 1)))) ++ m++; ++ if (m >> 24) { ++ m >>= 1; ++ s++; ++ } ++ return ((((a >> 23) & 0xff) - shift + s) << 23) | (m & 0x7fffff); ++} ++ ++static int s8003_rt_derive(const struct pvr_hwrt_dataset *hwrt, struct s8003_rt *rt) ++{ ++ u32 width = hwrt->width, height = hwrt->height; ++ u32 tx = (width + 31) / 32, ty = (height + 31) / 32; ++ u32 mx = ALIGN((tx + 3) >> 2, 4), my = ALIGN((ty + 3) >> 2, 4); ++ u32 s = hwrt->samples == 4 ? 2 : 1; ++ u32 px = mx * s * 4, py = my * s * 4, tpc; ++ ++ if (!width || !height || width > S8003_RT_MAX_DIM || height > S8003_RT_MAX_DIM || ++ (hwrt->samples != 1 && hwrt->samples != 4)) ++ return -EINVAL; ++ tpc = roundup_pow_of_two(max(px, py)); ++ *rt = (struct s8003_rt) { ++ .header = px * py * 5, ++ .region = mx * my * s * s * 5, ++ .positions = s == 2 ? S8003_POSITIONS_4X : S8003_POSITIONS_1X, ++ .state = s == 2 ? 2 : 0, ++ .padded = (mx & 0x3ff) << 16 | ((my * s) & 0x3ff), ++ .scale = { ++ s8003_fdiv(S8003_RT_SCALE_A, width) | ++ (u64)s8003_fdiv(S8003_RT_SCALE_A, height) << 32, ++ s8003_fdiv(S8003_RT_SCALE_B, width) | ++ (u64)s8003_fdiv(S8003_RT_SCALE_B, height) << 32, ++ (s8003_f32(width) + (4u << 23)) | ++ (u64)(s8003_f32(height) + (4u << 23)) << 32, ++ }, ++ .ta_state = s == 2 ? 5 : 0, ++ .ta_tiling = { ++ (u64)((ty - 1) << 12 | (tx - 1)) << 32 | ((height - 1) << 16 | (width - 1)), ++ (u64)(my << 18 | my << 10 | 3 * my) << 32 | (mx << 18 | mx << 10 | 3 * mx), ++ }, ++ .ta_tiling_word = mx * my, ++ .tiles = tx * ty, ++ .tpc = (u64)tpc * tpc * 8, ++ }; ++ /* ++ * What userspace sized its buffers for: the dataset's region header ++ * size is the ISP region size in dwords (pvr_drm.h), the region ++ * headers of one of the 16 macrotiles; its tail pointer cache size is ++ * in bytes. ++ */ ++ if (hwrt->common.rgn_header_size < rt->region / 4 || hwrt->common.tpc_size < rt->tpc) ++ return -EINVAL; ++ return 0; ++} ++ ++/* ++ * A geometry job's work record: the words Apple's kext sets the same for ++ * every render, then userspace's (stream extension), then the target's. ++ */ ++static void s8003_fill_ta(u8 *w, const struct pvr_apple_g5p_cmd_geom *cmd, ++ const struct pvr_hwrt_data *hwrt, const struct s8003_rt *rt) ++{ ++ static const u32 fixed[][2] = { ++ { 0x020, 0x1e3ce508 }, { 0x024, 0x1e3ce508 }, { 0x3a4, 0x00000088 }, ++ { 0x3a8, 0x00000002 }, { 0x3c0, 0x00020000 }, { 0x514, 0x08002358 }, ++ { 0x528, 0x00000001 }, ++ }; ++ const struct pvr_apple_g5p_geom_regs *g = &cmd->g5p; ++ u64 tpc = hwrt->data.tail_ptrs_dev_addr; ++ unsigned int i; ++ ++ memset(w, 0, S8003_TA_W_SIZE); ++ for (i = 0; i < ARRAY_SIZE(fixed); i++) ++ put32(w + fixed[i][0], fixed[i][1]); ++ put64(w + 0x48, g->scratch_ptr[4]); ++ put64(w + 0x50, g->scratch_ptr[5]); ++ for (i = 0; i < 6; i++) { ++ put32(w + 0x58 + i * 4, g->context_pds[0][i]); ++ put32(w + 0x70 + i * 4, g->context_pds[1][i]); ++ } ++ put64(w + 0x88, g->ta_program); ++ put64(w + 0x90, g->scratch_base & ~7ull); ++ put64(w + 0x98, cmd->cmd.regs.vdm_ctrl_stream_base & ~3ull); ++ for (i = 0; i < 4; i++) { ++ put32(w + 0xa0 + i * 4, g->resume_pds[0][i]); ++ put32(w + 0xb0 + i * 4, g->resume_pds[1][i]); ++ } ++ for (i = 0; i < 3; i++) ++ put32(w + 0xc0 + i * 4, g->ta_pds[i]); ++ for (i = 0; i < 4; i++) ++ put64(w + 0x468 + i * 8, g->scratch_ptr[i]); ++ put64(w + 0x488, g->scratch_ptr[4]); ++ ++ put64(w + 0x28, hwrt->data.rgn_header_dev_addr); ++ put64(w + 0x30, tpc); ++ put64(w + 0x38, rt->ta_state); ++ put64(w + 0x40, g->sample_positions ?: rt->positions); ++ put64(w + 0x444, tpc); ++ put32(w + 0x44c, rt->tpc); ++ put64(w + 0x3ac, rt->ta_tiling[0]); ++ put64(w + 0x3b4, rt->ta_tiling[1]); ++ put32(w + 0x3bc, rt->ta_tiling_word); ++} ++ ++/* The pixel output's partition words for @width registers and @shift (10, 11 with 4 samples) */ ++static u32 s8003_fill_output(u8 *w, u32 width, u32 shift) ++{ ++ /* Apple's code per width: 2, 4, 8 */ ++ u32 code = width == 4 ? 0x10 : width == 8 ? 0x20 : 0; ++ u32 count = 0x4000 / (width << shift); ++ unsigned int i; ++ ++ put32(w + 0x3c8, count & 0x1f); ++ for (i = 0; i < 6; i++) ++ put64(w + 0x108 + i * 8, code | (count - 1) << 7 | 0x43); ++ return count; ++} ++ ++static int s8003_fill_3d(u8 *w, const struct pvr_apple_g5p_cmd_frag *cmd, ++ const struct pvr_hwrt_data *hwrt, const struct s8003_rt *rt) ++{ ++ static const u32 fixed[][2] = { ++ { 0x108, 0x000003c3 }, { 0x110, 0x000003c3 }, { 0x118, 0x000003c3 }, ++ { 0x120, 0x000003c3 }, { 0x128, 0x000003c3 }, { 0x130, 0x000003c3 }, ++ { 0x3a4, 0x3f800000 }, { 0x3a8, 0x00000300 }, { 0x3b0, 0x0000001f }, ++ { 0x3c0, 0x0023000f }, { 0x3c4, 0x00000408 }, { 0x3c8, 0x00000008 }, ++ { 0x3cc, 0x000000f8 }, { 0x6a8, 0x3f800000 }, { 0x6ac, 0x00000300 }, ++ { 0x6b0, 0x00000002 }, { 0x868, 0x08002358 }, ++ }; ++ /* The record's two places for each ZLS register image */ ++ static const u16 zls[6][2] = { ++ { 0x80, 0x618 }, { 0x88, 0x628 }, { 0x90, 0x630 }, ++ { 0x98, 0x648 }, { 0xa0, 0x650 }, { 0x138, 0x6d0 }, ++ }; ++ const struct rogue_fwif_frag_regs *regs = &cmd->cmd.regs; ++ const struct pvr_apple_g5p_frag_regs *g = &cmd->g5p; ++ bool resolve = g->flags & PVR_APPLE_G5P_FRAG_MSAA_RESOLVE; ++ bool msaa = hwrt->hwrt_dataset->samples == 4; ++ u64 positions = g->sample_positions ?: rt->positions; ++ u32 width = g->pixel_output_width ?: 2; ++ unsigned int i; ++ ++ if ((width != 2 && width != 4 && width != 8) || ++ (g->flags & ~PVR_APPLE_G5P_FRAG_MSAA_RESOLVE) || (resolve && !msaa)) ++ return -EINVAL; ++ ++ memset(w, 0, S8003_3D_W_SIZE); ++ for (i = 0; i < ARRAY_SIZE(fixed); i++) ++ put32(w + fixed[i][0], fixed[i][1]); ++ for (i = 0; i < 8; i++) ++ put32(w + 0x5b8 + i * 4, g->background[i]); ++ put32(w + 0xb0, g->background[0]); ++ put32(w + 0xb4, g->background[1]); ++ for (i = 4; i < 8; i++) ++ put32(w + 0xc0 + (i - 4) * 4, g->background[i]); ++ /* The kext's upload has the halves of the first end-of-tile pair swapped */ ++ put32(w + 0x3b4, g->eot[1]); ++ put32(w + 0x3b8, g->eot[0]); ++ put32(w + 0x3bc, g->eot[2]); ++ for (i = 0; i < 3; i++) { ++ put32(w + 0x680 + i * 4, g->eot[i]); ++ put32(w + 0x694 + i * 4, g->eot_resume[i]); ++ } ++ put32(w + 0x600, g->restore[0]); ++ put32(w + 0x604, g->restore[1]); ++ put32(w + 0x610, g->restore[2]); ++ put32(w + 0x614, g->restore[3]); ++ ++ put64(w + 0x60, regs->isp_scissor_base & ~3ull); ++ put64(w + 0x4a0, regs->isp_scissor_base); ++ put64(w + 0x68, regs->isp_dbias_base & ~3ull); ++ put64(w + 0x490, regs->isp_dbias_base); ++ put64(w + 0x70, regs->isp_oclqry_base); ++ put64(w + 0x78, regs->isp_oclqry_base); ++ put64(w + 0xb8, g->background_texture); ++ put64(w + 0x5f0, g->background_texture); ++ put32(w + 0x3a4, regs->isp_bgobjdepth); ++ put32(w + 0x6a8, regs->isp_bgobjdepth); ++ put32(w + 0x3a8, regs->isp_bgobjvals); ++ put32(w + 0x6ac, regs->isp_bgobjvals); ++ for (i = 0; i < 6; i++) { ++ put64(w + zls[i][0], g->zls[i]); ++ put64(w + zls[i][1], g->zls[i]); ++ } ++ put32(w + 0x3cc, (width == 4 ? 1 : width == 8 ? 2 : 0) | 0xf8); ++ put32(w + 0x6b0, width); ++ s8003_fill_output(w, width, 10); ++ ++ put64(w + 0x20, hwrt->data.rgn_header_dev_addr); ++ put64(w + 0x28, rt->positions); ++ put32(w + 0x30, rt->state); ++ put32(w + 0x34, rt->padded); ++ put32(w + 0x38, rt->region); ++ put64(w + 0x3c, rt->scale[0]); ++ put64(w + 0x44, rt->scale[1]); ++ put64(w + 0x4c, rt->scale[2]); ++ put64(w + 0x58, positions); ++ put32(w + 0xa8, rt->region >> 2); ++ put32(w + 0x3a0, rt->padded); ++ put64(w + 0x3d0, rt->scale[0]); ++ put64(w + 0x3d8, rt->scale[1]); ++ put32(w + 0x3e0, lower_32_bits(rt->scale[2])); ++ put32(w + 0x3e4, lower_32_bits(rt->scale[0])); ++ ++ /* ++ * Four samples: the AA mode, ISP_CTL without the last pass (and with ++ * the resolve), and half the output partitions for the width. ++ */ ++ if (msaa) { ++ u32 count = s8003_fill_output(w, width, 11); ++ u32 mode = min(count, 2u); ++ ++ put32(w + 0x3c0, (0x0023000f & ~(0xfu | BIT(21))) | 0x7 | ++ (resolve ? BIT(17) : 0) | (mode - 1) << 21); ++ put32(w + 0x3c4, mode * 0x204); ++ put64(w + 0xd0, rt->state); ++ put32(w + 0x86c, resolve); ++ } ++ return 0; ++} ++ ++/* Where job @seq of a render channel has its command, record and microsequence */ ++struct s8003_job_slot { ++ u32 cmd, w, useq, times; ++}; ++ ++static struct s8003_job_slot s8003_slot_of(u32 seq, bool frag) ++{ ++ if (seq & 1) ++ return (struct s8003_job_slot) { ++ S8003_CH_CMD, S8003_CH_W, frag ? S8003_CH_3D_USEQ : S8003_CH_TA_USEQ, ++ S8003_CH_TIMES, ++ }; ++ return (struct s8003_job_slot) { ++ S8003_CH_SLOT1, S8003_CH_SLOT1 + S8003_CMD_SIZE, ++ S8003_CH_SLOT1 + S8003_CMD_SIZE + S8003_3D_W_SIZE, S8003_CH_TIMES + 16, ++ }; ++} ++ ++/* What the 3D engine is waited for between a fragment job's start and finish */ ++static const u32 s8003_3d_wait[] = { ++ 0x201, 0x138204, 0x10, 0x8388204, 0, 0x104000a, 0x1f, ++ 0x1f, 0x8388204, 1, 0x104000a, 0x1f, 0x1f, 0xf48008, ++ 0x10000015, 4, 0, 0x14, 0x109f, 0x13000a, 0x4000, ++ 0x4000, 0x138204, 0x4000, 0x8388204, 0, 0x104000a, 8, ++ 8, 0x8388204, 1, 0x104000a, 8, 8, ++}; ++ ++static u8 *useq_put(u8 *out, const u32 *words, unsigned int n) ++{ ++ for (; n--; out += 4) ++ put32(out, *words++); ++ return out; ++} ++ ++/* ++ * The parts of render job @seq that are the kernel's, around the record's ++ * image at the job's slot of the channel page @p: the record's links (slot, ++ * dependency object, PM set and its scene record, microsequence, stamps), the ++ * microsequence, the command and both pointer-ring entries. @pm is the PM ++ * set's META address, @pm_id its number. A geometry job's command hands over ++ * the PM set; a fragment job's waits for the stamp at @wait_stamp to reach ++ * @wait_seq, its geometry job's. ++ */ ++static void s8003_job_write(u8 *p, u32 addr, u32 global, u32 pm, u32 pm_id, u32 slot, bool frag, ++ u32 seq, u32 wait_stamp, u32 wait_seq) ++{ ++ struct s8003_job_slot at = s8003_slot_of(seq, frag); ++ u32 w = addr + at.w, record = pm + S8003_PM_RECORD; ++ u32 stamp = addr + S8003_CH_STAMP, times = addr + at.times; ++ /* In the record: the microsequence's result, and where its timestamps go */ ++ u32 result = w + (frag ? 0x8ac : 0x550), time = w + (frag ? 0x8b4 : 0x558); ++ u32 ring = (2 * (seq - 1)) % S8003_CH_ENTRIES; ++ u8 *work = p + at.w, *cmd = p + at.cmd, *useq = p + at.useq; ++ const u32 time_start[] = { 0x80000018, result, time, time, seq }; ++ const u32 time_end[] = { 0x18, result, time, time + 4, seq }; ++ const u32 wait_ta = 0x801, end = 0x40000017; ++ ++ put32(work + (frag ? 0x8ac : 0x550), 0); ++ put32(work + (frag ? 0x8b0 : 0x554), 0); ++ put32(work + (frag ? 0x8b4 : 0x558), times); ++ put32(work + (frag ? 0x8b8 : 0x55c), times + 8); ++ memset(p + at.times, 0, 16); ++ memset(useq, 0, frag ? S8003_3D_USEQ_SIZE : S8003_TA_USEQ_SIZE); ++ put32(work, frag); ++ put32(work + 4, slot); ++ put32(work + (frag ? 0x14 : 0xc), addr + S8003_CH_DEP); ++ put32(work + (frag ? 0x18 : 0x14), pm); ++ if (!frag) ++ put32(work + 0x10, pm_id); ++ put32(work + (frag ? 0x1c : 0x18), record); ++ put32(work + (frag ? 0xc : 0x454), addr + at.useq); ++ put32(work + (frag ? 0x10 : 0x458), frag ? S8003_3D_USEQ_SIZE : S8003_TA_USEQ_SIZE); ++ put32(work + (frag ? 0x888 : 0x52c), stamp); ++ put32(work + (frag ? 0x88c : 0x530), stamp + 4); ++ put32(work + (frag ? 0x890 : 0x534), seq); ++ put32(work + (frag ? 0x894 : 0x538), frag); ++ put32(work + (frag ? 0x898 : 0x53c), frag); ++ put32(work + (frag ? 0x8a0 : 0x544), seq); ++ ++ if (frag) { ++ const u32 start[] = { ++ 0x25, w + 0x3a0, w + 0x58, record, global + 0x724, w + 0x7f0, w + 0x7f4, ++ w + 0x878, addr, w, slot, 0, pm_id, get32(work + 0x800), 0, 1, w + 0x6d8, ++ get32(work + 0x87c), get32(work + 0x884), 0, seq, ++ }; ++ const u32 finish[] = { ++ 0x26, seq, stamp + 4, seq, record, pm, 1, global + 0x724, w + 0x878, ++ w + 0x7f0, addr, w, slot, w + 0x6d8, 0, get32(work + 0x87c), ++ get32(work + 0x884), (u32)-0x104, ++ }; ++ ++ useq = useq_put(useq, start, ARRAY_SIZE(start)); ++ useq = useq_put(useq, time_start, ARRAY_SIZE(time_start)); ++ useq = useq_put(useq, s8003_3d_wait, ARRAY_SIZE(s8003_3d_wait)); ++ useq = useq_put(useq, time_end, ARRAY_SIZE(time_end)); ++ useq = useq_put(useq, finish, ARRAY_SIZE(finish)); ++ } else { ++ const u32 start[] = { ++ 0x23, w + 0x3a4, w + 0x20, record, global + 0x724, addr, slot, 0, pm_id, ++ 0, 0, 0, w + 0x464, get32(work + 0x51c), get32(work + 0x520), 0, seq, ++ }; ++ const u32 finish[] = { ++ 0x24, record, pm, global + 0x724, addr, slot, 0, w + 0x464, ++ get32(work + 0x51c), get32(work + 0x520), seq, stamp + 4, seq, ++ (u32)-0x70, ++ }; ++ ++ useq = useq_put(useq, start, ARRAY_SIZE(start)); ++ useq = useq_put(useq, time_start, ARRAY_SIZE(time_start)); ++ useq = useq_put(useq, &wait_ta, 1); ++ useq = useq_put(useq, time_end, ARRAY_SIZE(time_end)); ++ useq = useq_put(useq, finish, ARRAY_SIZE(finish)); ++ } ++ useq_put(useq, &end, 1); ++ ++ memset(cmd, 0, S8003_CMD_SIZE); ++ if (frag) { ++ put32(cmd, S8003_CMD_WAIT); ++ put32(cmd + 4, wait_stamp); ++ put32(cmd + 8, wait_seq); ++ put32(cmd + 12, wait_seq & 0x7fffffff); ++ } else { ++ put32(cmd, S8003_CMD_PM); ++ put32(cmd + 4, slot); ++ put32(cmd + 8, pm_id); ++ put32(cmd + 12, pm); ++ } ++ put32(p + S8003_CH_RING + ring * 4, addr + at.cmd); ++ put32(p + S8003_CH_RING + ((ring + 1) % S8003_CH_ENTRIES) * 4, w); ++} ++ ++/* ++ * Tell the firmware that @engine's pointer ring is filled up to @end: an ++ * entry of the engine's work ring, the host's watermark, the work ring's ++ * producer index, the doorbell. The doorbell is rung unless the job's own ++ * stamps show it done: ringing for an entry the firmware took already adds ++ * no work, and a job that was not rung for waited until something else woke ++ * the firmware (a second of silence with two jobs in flight, then a lost ++ * job, on the iPad 5). ++ */ ++static int s8003_publish(struct pvr_apple *apple, enum pvr_apple_engine engine, u32 seq, u32 end) ++{ ++ struct pvr_apple_s8003 *s = apple->chip_data; ++ u32 fw_engine = s8003_engine(engine); ++ u8 *ctl = s8003_object(s, S8003_WORK_CTL + 4 * fw_engine); ++ u8 *ring = s8003_object(s, S8003_WORK_RING + 4 * fw_engine); ++ u32 stamp = engine == APPLE_ENGINE_COMPUTE ? S8003_CU_STAMP : S8003_CH_STAMP; ++ u8 *p = s8003_channel(s, engine), *entry; ++ u32 start = ring_read(ctl, S8003_RING_PRODUCER), consumer; ++ int err; ++ ++ if (start >= APPLE_RING_ENTRIES) ++ return -EPROTO; ++ /* The firmware takes a work-ring entry at once; the ring never holds two */ ++ err = read_poll_timeout(s8003_ring_consumer, consumer, consumer == start, 20, ++ S8003_RING_TIMEOUT_US, false, apple, ctl); ++ if (err) ++ return err; ++ entry = ring + start * S8003_DC_SIZE; ++ memset(entry, 0, S8003_DC_SIZE); ++ put32(entry, fw_engine); ++ put32(entry + 4, s8003_channel_addr(engine)); ++ put32(entry + 8, end); ++ wmb(); ++ put32(p + S8003_CH_CTL + S8003_CH_CTL_HOST, end); ++ wmb(); ++ WRITE_ONCE(*(__le32 *)(ctl + S8003_RING_PRODUCER), ++ cpu_to_le32((start + 1) % APPLE_RING_ENTRIES)); ++ mb(); ++ if ((s32)(get32(p + stamp) - seq) >= 0 && (s32)(get32(p + stamp + 4) - seq) >= 0) ++ return 0; ++ /* Compute has no doorbell of its own in Apple's kext: the geometry engine's wakes the scheduler */ ++ return s8003_doorbell(apple, engine == APPLE_ENGINE_FRAG ? ROGUE_CR_MTS_SCHEDULE_DM_DM1 : ++ ROGUE_CR_MTS_SCHEDULE_DM_DM0); ++} ++ ++/* Whether a PM set holds no scene the firmware still works on */ ++static bool s8003_pm_idle(struct pvr_apple *apple, const struct s8003_pm *pm) ++{ ++ struct pvr_apple_s8003 *s = apple->chip_data; ++ ++ if (pm->scene != s->gen) ++ return true; ++ if (pm->td_seq) ++ return (s32)(pvr_apple_s8003_engine_done(apple, APPLE_ENGINE_FRAG) - pm->td_seq) >= 0; ++ return false; ++} ++ ++static int s8003_submit_geom(struct pvr_apple *apple, struct s8003_vm *vm, struct pvr_job *job) ++{ ++ const struct pvr_hwrt_dataset *hwrt = job->hwrt->hwrt_dataset; ++ struct pvr_apple_s8003 *s = apple->chip_data; ++ u8 *p = s8003_channel(s, APPLE_ENGINE_GEOM); ++ struct s8003_pm *pm = NULL; ++ struct s8003_rt rt; ++ u32 seq, blocks; ++ unsigned int i; ++ int err; ++ ++ err = s8003_rt_derive(hwrt, &rt); ++ if (err) ++ return err; ++ /* The VM's sets were taken when the job was ordered (pvr_apple_s8003_vm_scenes()) */ ++ for (i = 0; i < 2 && !pm; i++) ++ if (vm->pm[i].set >= 0 && s8003_pm_idle(apple, &vm->pm[i])) ++ pm = &vm->pm[i]; ++ /* ++ * Each holds a scene still: one whose fragment job never came (a ++ * geometry job alone) is given up once its geometry job is done. ++ */ ++ for (i = 0; i < 2 && !pm; i++) ++ if (vm->pm[i].set >= 0 && !vm->pm[i].td_seq && ++ !s8003_wait_done(apple, APPLE_ENGINE_GEOM, vm->pm[i].ta_seq)) ++ pm = &vm->pm[i]; ++ if (!pm) ++ return -EBUSY; ++ ++ /* As many 128 KiB blocks as the free list has 4 KiB pages, and enough for the target */ ++ blocks = max(DIV_ROUND_UP(hwrt->free_lists[0]->initial_num_pages, 32), ++ DIV_ROUND_UP(50 + rt.tiles, 32)); ++ err = s8003_pm_build(apple, vm, pm, min(blocks, S8003_PM_BLOCKS_MAX)); ++ /* The record slot of the job before the last one is free again */ ++ if (!err && apple->ta_seq) ++ err = s8003_wait_done(apple, APPLE_ENGINE_GEOM, apple->ta_seq - 1); ++ if (err) ++ return err; ++ ++ seq = apple->ta_seq + 1; ++ s8003_fill_ta(p + s8003_slot_of(seq, false).w, job->cmd, job->hwrt, &rt); ++ s8003_job_write(p, s8003_channel_addr(APPLE_ENGINE_GEOM), ++ s8003_object_addr(&s->layout, S8003_CHANNEL), ++ s8003_meta_addr(s8003_pm_set_va(pm, 0)), pm->set + 1, vm->slot[0], false, ++ seq, 0, 0); ++ pm->hwrt = job->hwrt; ++ pm->scene = s->gen; ++ pm->ta_seq = seq; ++ pm->td_seq = 0; ++ apple->ta_seq = seq; ++ err = s8003_publish(apple, APPLE_ENGINE_GEOM, seq, (2 * seq) % S8003_CH_ENTRIES); ++ return err ?: seq; ++} ++ ++static int s8003_submit_frag(struct pvr_apple *apple, struct s8003_vm *vm, struct pvr_job *job) ++{ ++ struct pvr_apple_s8003 *s = apple->chip_data; ++ u8 *p = s8003_channel(s, APPLE_ENGINE_FRAG); ++ struct s8003_pm *pm = NULL; ++ struct s8003_rt rt; ++ unsigned int i; ++ u32 seq; ++ int err; ++ ++ /* The scene of this job's geometry job, which was submitted before it */ ++ for (i = 0; i < 2; i++) ++ if (vm->pm[i].scene == s->gen && vm->pm[i].hwrt == job->hwrt && !vm->pm[i].td_seq) ++ pm = &vm->pm[i]; ++ if (!pm) { ++ dev_warn_ratelimited(apple_dev(apple), ++ "fragment job without a geometry job: not run\n"); ++ return -ECANCELED; ++ } ++ err = s8003_rt_derive(job->hwrt->hwrt_dataset, &rt); ++ if (!err && apple->frag_seq) ++ err = s8003_wait_done(apple, APPLE_ENGINE_FRAG, apple->frag_seq - 1); ++ if (err) ++ return err; ++ ++ seq = apple->frag_seq + 1; ++ err = s8003_fill_3d(p + s8003_slot_of(seq, true).w, job->cmd, job->hwrt, &rt); ++ if (err) ++ return err; ++ s8003_job_write(p, s8003_channel_addr(APPLE_ENGINE_FRAG), ++ s8003_object_addr(&s->layout, S8003_CHANNEL + 1), ++ s8003_meta_addr(s8003_pm_set_va(pm, 0)), pm->set + 1, vm->slot[1], true, seq, ++ s8003_channel_addr(APPLE_ENGINE_GEOM) + S8003_CH_STAMP + 4, pm->ta_seq); ++ pm->td_seq = seq; ++ apple->frag_seq = seq; ++ err = s8003_publish(apple, APPLE_ENGINE_FRAG, seq, (2 * seq) % S8003_CH_ENTRIES); ++ return err ?: seq; ++} ++ ++/* ++ * A compute job runs on its VM's geometry slot. One record: the job before ++ * it is done first. The microsequence: start, wait for the engine's ++ * "finished" event and its status, finish. ++ */ ++static int s8003_submit_compute(struct pvr_apple *apple, struct s8003_vm *vm, struct pvr_job *job) ++{ ++ const struct pvr_apple_g5p_cmd_compute *cmd = job->cmd; ++ const struct pvr_apple_g5p_compute_regs *g = &cmd->g5p; ++ struct pvr_apple_s8003 *s = apple->chip_data; ++ u32 addr = s8003_channel_addr(APPLE_ENGINE_COMPUTE); ++ u32 global = s8003_object_addr(&s->layout, S8003_CHANNEL + 2); ++ u32 w = addr + S8003_CH_W, slot = vm->slot[0], seq; ++ u8 *p = s8003_channel(s, APPLE_ENGINE_COMPUTE); ++ u8 *work = p + S8003_CH_W; ++ unsigned int i; ++ int err; ++ ++ err = s8003_wait_done(apple, APPLE_ENGINE_COMPUTE, apple->last_seq); ++ if (err) ++ return err; ++ seq = apple->last_seq + 1; ++ { ++ const u32 useq[] = { ++ 0x32, w + 0x10, w + 0x60, global + 0x720, addr, slot, 0, 0, 0, w + 0x1e8, ++ 0, 0, seq, ++ 0x80000018, w + 0x278, w + 0x280, w + 0x280, seq, ++ 0x101, 0x004a000a, 1, 1, ++ 0x18, w + 0x278, w + 0x280, w + 0x284, seq, ++ 0x33, global + 0x720, addr, slot, w + 0x1e8, 0, 0, seq, ++ addr + S8003_CU_STAMP + 4, seq, (u32)-0x6c, ++ 0x40000017, ++ }; ++ ++ BUILD_BUG_ON(sizeof(useq) > S8003_CU_USEQ_SIZE); ++ memset(p + S8003_CU_USEQ, 0, S8003_CU_USEQ_SIZE); ++ useq_put(p + S8003_CU_USEQ, useq, ARRAY_SIZE(useq)); ++ } ++ memset(work, 0, S8003_CU_W_SIZE); ++ put32(work, 3); ++ put32(work + 8, slot); ++ put32(work + 0xc, addr + S8003_CU_DEP); ++ put32(work + 0x10, g->config[0]); ++ put32(work + 0x14, g->config[1]); ++ put64(work + 0x60, g->tail); ++ put64(work + 0x68, g->pds_terminate); ++ put64(work + 0x70, cmd->cmd.regs.cdm_ctrl_stream_base); ++ put32(work + 0x1e0, addr + S8003_CU_USEQ); ++ put32(work + 0x1e4, S8003_CU_USEQ_SIZE); ++ for (i = 0; i < 4; i++) ++ put64(work + 0x1ec + i * 8, g->state[i]); ++ put64(work + 0x20c, g->tail); ++ put64(work + 0x214, g->terminate); ++ put64(work + 0x21c, g->pds_store); ++ put32(work + 0x224, g->pds_store1); ++ put64(work + 0x22c, g->pds_resume); ++ put32(work + 0x234, g->pds_resume1); ++ put64(work + 0x23c, g->terminate); ++ put32(work + 0x244, 0x00a00004); ++ put32(work + 0x258, addr + S8003_CU_STAMP); ++ put32(work + 0x25c, addr + S8003_CU_STAMP + 4); ++ put32(work + 0x260, seq); ++ put32(work + 0x270, seq); ++ put32(p + S8003_CH_RING + ((seq - 1) % S8003_CH_ENTRIES) * 4, w); ++ apple->last_seq = seq; ++ err = s8003_publish(apple, APPLE_ENGINE_COMPUTE, seq, seq % S8003_CH_ENTRIES); ++ return err ?: seq; ++} ++ ++static int pvr_apple_s8003_submit(struct pvr_apple *apple, struct pvr_job *job) ++{ ++ struct pvr_apple_s8003 *s = apple->chip_data; ++ struct s8003_vm *vm; ++ int err; ++ ++ mutex_lock(&apple->submit_lock); ++ vm = xa_load(&s->vms, (unsigned long)job->ctx->vm_ctx); ++ if (!apple->booted || !vm) ++ err = -ENODEV; ++ else ++ err = s8003_vm_attach(apple, vm); ++ if (err) ++ goto out_unlock; ++ if (job->type == DRM_PVR_JOB_TYPE_GEOMETRY) ++ err = s8003_submit_geom(apple, vm, job); ++ else if (job->type == DRM_PVR_JOB_TYPE_FRAGMENT) ++ err = s8003_submit_frag(apple, vm, job); ++ else ++ err = s8003_submit_compute(apple, vm, job); ++out_unlock: ++ mutex_unlock(&apple->submit_lock); ++ return err; ++} ++ ++static void pvr_apple_s8003_report_hang(struct pvr_apple *apple) ++{ ++ static const char *const names[] = { "compute", "geometry", "fragment" }; ++ struct pvr_apple_s8003 *s = apple->chip_data; ++ u8 *dc = s8003_object(s, S8003_DC_CTL); ++ int engine; ++ ++ for (engine = 0; engine < APPLE_ENGINES; engine++) { ++ const u8 *p = s8003_channel(s, engine); ++ const u8 *ctl = s8003_object(s, S8003_WORK_CTL + 4 * s8003_engine(engine)); ++ u32 stamp = engine == APPLE_ENGINE_COMPUTE ? S8003_CU_STAMP : S8003_CH_STAMP; ++ ++ dev_err(apple_dev(apple), ++ "hang: %s: submitted %u, stamps %u %u, watermarks %u %u %u %u %u, work ring %u/%u\n", ++ names[engine], s8003_submitted(apple, engine), get32(p + stamp), ++ get32(p + stamp + 4), get32(p + S8003_CH_CTL), ++ get32(p + S8003_CH_CTL + 0x10), get32(p + S8003_CH_CTL + 0x20), ++ get32(p + S8003_CH_CTL + 0x30), get32(p + S8003_CH_CTL + 0x40), ++ get32(ctl + S8003_RING_CONSUMER), get32(ctl + S8003_RING_PRODUCER)); ++ } ++ dev_err(apple_dev(apple), ++ "hang: firmware %s, device control %u/%u, Fender state %#x\n", ++ s->failed ? "rings unreadable" : s->fault ? "posted a fault" : "posted no fault", ++ get32(dc + S8003_RING_CONSUMER), ++ get32(dc + S8003_RING_PRODUCER), readl(apple->fender + FENDER_POWER_STATE)); ++} ++ ++/* ++ * The first command: an empty device-control command. The firmware takes it ++ * from the ring once its scheduler runs, which is when it takes commands at ++ * all. ++ */ ++static int s8003_first_command(struct pvr_apple *apple) ++{ ++ static const u32 cmd[1][2] = { { S8003_DC_NONE, 0 } }; ++ struct pvr_apple_s8003 *s = apple->chip_data; ++ int err; ++ ++ put32(s8003_object(s, S8003_CONTROL) + 0x1020, 0); /* no payload */ ++ err = s8003_device_control(apple, cmd, 1); + if (err) + dev_err(apple_dev(apple), "GFX firmware did not take its first command: %d\n", + err); + return err; + } + ++/* A start of the firmware: it knows no VM, and its channels are empty */ ++static void s8003_jobs_reset(struct pvr_apple_s8003 *s) ++{ ++ unsigned long flags; ++ ++ spin_lock_irqsave(&s->tag_lock, flags); ++ memset(s->tag, 0, sizeof(s->tag)); ++ spin_unlock_irqrestore(&s->tag_lock, flags); ++ bitmap_zero(s->parked, S8003_SLOTS); ++ if (!++s->gen) ++ s->gen = 1; ++ WRITE_ONCE(s->failed, false); ++ WRITE_ONCE(s->fault, false); ++ memset(s->pmsets.map, 0, s->pmsets.size); ++ s8003_channels_init(s); ++} ++ + /* + * The registers the firmware is told about, as bus addresses: Fender and the + * Rogue core from this node, the interrupt controller and the number of the +@@ -1062,6 +2415,13 @@ static int pvr_apple_s8003_init(struct pvr_apple *apple) + apple->chip_data = s; + s->apple = apple; + INIT_DELAYED_WORK(&s->drain_work, s8003_drain_work); ++ spin_lock_init(&s->tag_lock); ++ spin_lock_init(&s->set_lock); ++ xa_init(&s->vms); ++ s->next_slot = S8003_SLOT_FIRST; ++ err = devm_mutex_init(dev, &s->event_lock); ++ if (err) ++ return err; + + err = s8003_parse_dt(apple); + if (err) +@@ -1132,12 +2492,13 @@ static int pvr_apple_s8003_start(struct pvr_apple *apple) + /* The firmware writes into its image while it runs: start from the copy */ + memcpy(s->fw.map, s->image, S8003_IMAGE_SIZE); + s8003_init_build(apple, q); ++ s8003_jobs_reset(s); + + err = s8003_release(apple, q); + if (!err) + err = s8003_wait_alive(apple); + if (!err) { +- dev_info(apple_dev(apple), "GFX firmware is up (power state %u)\n", q); ++ dev_dbg(apple_dev(apple), "GFX firmware is up (power state %u)\n", q); + err = s8003_first_command(apple); + } + if (err) { +@@ -1145,7 +2506,15 @@ static int pvr_apple_s8003_start(struct pvr_apple *apple) + s8003_core_off(apple); + return err; + } +- dev_info(apple_dev(apple), "GFX firmware takes commands\n"); ++ dev_dbg(apple_dev(apple), "GFX firmware takes commands\n"); ++ /* Once: the tables a VM's slots are left on when the VM goes */ ++ if (!s->park_vm) { ++ s->park_vm = pvr_vm_create_context(apple->pvr_dev, true); ++ if (IS_ERR(s->park_vm)) { ++ dev_warn(apple_dev(apple), "no park VM: %pe\n", s->park_vm); ++ s->park_vm = NULL; ++ } ++ } + schedule_delayed_work(&s->drain_work, msecs_to_jiffies(S8003_DRAIN_MS)); + return 0; + } +@@ -1179,6 +2548,12 @@ static void pvr_apple_s8003_restore_image(struct pvr_apple *apple) + const struct pvr_apple_chip pvr_apple_chip_s8003 = { + .fw_name = "apple/gfx-s8003.bin", + .perf_state_max = S8003_PERF_STATE_MAX, ++ .geom_stream = &pvr_apple_g5p_cmd_geom_stream, ++ .frag_stream = &pvr_apple_g5p_cmd_frag_stream, ++ .compute_stream = &pvr_apple_g5p_cmd_compute_stream, ++ .scenes = 2, ++ .own_pm_pages = true, ++ .gate_off_ms = 10, + .init = pvr_apple_s8003_init, + .boot = pvr_apple_s8003_boot, + .start = pvr_apple_s8003_start, +@@ -1186,4 +2561,12 @@ const struct pvr_apple_chip pvr_apple_chip_s8003 = { + .idle = pvr_apple_s8003_idle, + .fini = pvr_apple_s8003_fini, + .restore_image = pvr_apple_s8003_restore_image, ++ .report_hang = pvr_apple_s8003_report_hang, ++ .submit = pvr_apple_s8003_submit, ++ .engine_done = pvr_apple_s8003_engine_done, ++ .events = pvr_apple_s8003_events, ++ .vm_init = pvr_apple_s8003_vm_init, ++ .vm_fini = pvr_apple_s8003_vm_fini, ++ .vm_scenes = pvr_apple_s8003_vm_scenes, ++ .mmu_flush = pvr_apple_s8003_mmu_flush, + }; diff --git a/kernel/patches/drivers/gpu/0025-drm-imagination-Count-the-Apple-A9-as-supported.patch b/kernel/patches/drivers/gpu/0025-drm-imagination-Count-the-Apple-A9-as-supported.patch new file mode 100644 index 00000000..4cbc75cd --- /dev/null +++ b/kernel/patches/drivers/gpu/0025-drm-imagination-Count-the-Apple-A9-as-supported.patch @@ -0,0 +1,32 @@ +From 0000000000000000000000000000000000000000 Mon Sep 17 00:00:00 2001 +From: Henrik Klee +Date: Sun, 15 Nov 2026 00:12:59 +0800 +Subject: [PATCH] drm/imagination: Count the Apple A9 as supported + +The A9's GPU (BVNC 9.15.6.2) takes jobs now, so the driver binds to it +without powervr.exp_hw_support, as it does to the A10's: the GPU starts +on every boot of a board that enables its node. + +Assisted-by: LLM +--- + drivers/gpu/drm/imagination/pvr_device.c | 5 +---- + 1 file changed, 1 insertion(+), 4 deletions(-) + +diff --git a/drivers/gpu/drm/imagination/pvr_device.c b/drivers/gpu/drm/imagination/pvr_device.c +index 0b6fd3c..2c06a4a 100644 +--- a/drivers/gpu/drm/imagination/pvr_device.c ++++ b/drivers/gpu/drm/imagination/pvr_device.c +@@ -539,12 +539,9 @@ pvr_gpu_support_level(const struct pvr_gpu_id *gpu_id) + case PVR_PACKED_BVNC(36, 52, 104, 182): + case PVR_PACKED_BVNC(36, 53, 104, 796): + case PVR_PACKED_BVNC(13, 10, 6, 2): /* Apple A10 (T8010) */ ++ case PVR_PACKED_BVNC(9, 15, 6, 2): /* Apple A9 (S8003) */ + return PVR_GPU_SUPPORTED; + +- /* Apple A9 (S8003): its firmware is started, no job runs on it yet */ +- case PVR_PACKED_BVNC(9, 15, 6, 2): +- return PVR_GPU_EXPERIMENTAL; +- + default: + return PVR_GPU_UNKNOWN; + }