Skip to content

kernel: The A9 GPU (iPad 5) in powervr: firmware boot and jobs - #138

Merged
henrikklee merged 11 commits into
mainfrom
gpu-a9-boot
Nov 14, 2026
Merged

henrikklee merged 11 commits into
mainfrom
gpu-a9-boot

Conversation

@henrikklee

@henrikklee henrikklee commented Nov 14, 2026 •

Copy link
Copy Markdown
Contributor

The iPad 5's GPU (A9, PowerVR GT7600) in the upstream powervr driver, as a second chip next to the A10: firmware boot (step 4, already here) and now the whole job path, so that it renders through powervr's UAPI. Built for apple-16k (--device j71t, j72t, j71b), not run on hardware. Per Henrik's decision there are no step-by-step hardware tests: this is merged when complete, and Berkay tests main once on his J71t. An iPad 5 nightly that does not start its GPU in between is accepted (installs use v0.3).

What an iPad 5 does with it

  • Every boot starts the GPU, as on the iPad 6: the driver boots the tablet's own GPU firmware (Apple's unmodified image from the firmware recipe), and stops it a second after its last use. The core now counts as supported; no kernel option is needed. Both iPad 5 models: the cellular one's restore image carries byte for byte the same GPU firmware, so its recipe and device tree got it too.
  • Programs can render and run compute jobs through /dev/dri/renderD*, once Mesa speaks the A9's job format (next step, not in this PR). Until then nothing uses the device and the desktop stays on software rendering.
  • A job that hangs ends after 5 s together with the other jobs in flight; the firmware is stopped, the GPU's power domain goes off for at least 10 ms and the next job boots the firmware again. The program whose job hung loses its GPU context; others go on. Nothing is frozen until reboot and there is no fault budget.
  • Sleep stops the firmware and boots it again on wake, by the same path.
  • If the firmware does not boot, the GPU is powered off and the driver does not bind: no GPU device, software rendering as today.

What is unchanged

  • iPad 6 / 7 (A10): jobs, completion and recovery take the same code as before. The shared file got hooks a chip may provide (submit, completion, per-VM setup); the A10 provides none of them, so every new branch is one the A10 does not take. One addition applies to both: a HWRT dataset remembers its width, height and sample count.
  • A9 iPhones (N71m, N66m, N69): GPU node still disabled. Their iOS 15.8.8 kernelcaches carry a GPU firmware of another size than the iPad's; the recipe step refuses it and the driver's image layout has not been checked against it.
  • UAPI: pvr_drm.h is untouched. The A9 gets three stream extension types next to the A10's.

For Berkay: the one test on main

After the merge, with the Mesa port installed. In order, stopping at the first that fails:

  1. Boot. dmesg | grep -iE 'powervr|gfx|fender' shows Initialized powervr ... and no error line; a second later cat /sys/bus/platform/devices/201000000.gpu/power/runtime_status says suspended.
  2. A compute or render test of your choice (your Mesa test tree has them); then the desktop on the GPU.
  3. While it renders: dmesg stays free of hang: and GFX firmware lines.
  4. Sleep and wake with the desktop on the GPU.

What a failure line means:

  • GFX firmware did not come up, ... did not take its first command, Fender ...: the boot (unchanged since the first version of this PR).
  • GFX firmware did not load slots A and B: attaching a program's address space (two device-control commands) was not taken within 3 s.
  • Apple GPU job timed out followed by hang: ... lines: a job did not finish. The lines give, per engine, the last job submitted, both stamps, the five watermarks and the work ring's indices, then whether the firmware posted that it stopped.
  • firmware event N: ...: the firmware posted something other than "alive" or a job's stamp, a fault or a recovery request (its 40 bytes follow); a job it gave up then times out.
  • fragment job without a geometry job: not run: ordering, mine or Mesa's.

The list "Open on hardware" at the end of docs/bring-up/gpu/a9-g5p.md says what I expect could differ from your driver's behaviour, with the fallback where there is one.

Your driver's code: where it is now

Boot:

apple-g5p now
ldr.h LDR parser documented, not ported, because the driver's own walker (pvr_fw_meta.c pvr_meta_ldr_process()) does it; its pins (119112 bytes, 94 records) are dropped, the recipe pins the file
pack.h image packing pvr_apple_s8003.c s8003_ldr_mem(), s8003_ldr_config(), s8003_ldr_start_threads()
boot-layout.h g5p_plan_layout() s8003_layout_plan()
boot-layout.h g5p_make_boot_config() s8003_image_configure()
boot-layout.h g5p_make_init() s8003_init_build(); PIO addresses and the interrupt from the device tree (s8003_parse_dt())
boot.inc private page tables the kernel VM context: s8003_map(), pvr_vm_map(), new pvr_vm_map_io()
perf-envelope.h pinned record + seed s8003_init_perf()
core.inc Fender wake pvr_apple_s8003_boot()
fender.h, boot.inc boot stream and release s8003_core_off(), s8003_release()
boot.inc event drain, boot-r73b.inc ring order s8003_drain(), every 100 ms instead of on the interrupt
ALIVE wait s8003_wait_alive()
lifecycle.inc command 8, kick.inc s8003_first_command(), s8003_device_control(), s8003_doorbell()
g5p_boot_park() pvr_apple_s8003_stop()
#125 "wait out a power-on request that ALIVE overtook" in another shape: the release waits for the state up to 100 ms whether or not the firmware is alive, the doorbell waits for bit 4 (s8003_core_ready())
#125 "ring a phase's doorbell unless its own stamp retired it" s8003_publish()
phase runner, arm/load/sample stages, snapshots documented, not ported, because they are bring-up tooling
r73b rows/caps/watchdog, r73c/r73e, r74e/wake112, r74, r74h, r80+, heartbeat116 documented, not ported, because one path remains and it polls
r74f, r74g, r87b (attach, firmware-owned MMU pass) the firmware-owned MMU pass is s8003_slots_load(); the power repair and the grace periods around it are documented, not ported
r73f/r75/r79, firmware patching documented by name only (their code is not in #74), not ported, because only Apple's image is booted
overclock (gpu-oc.h) documented (PSINFO decode kept), not ported, because it rewrites a PMGR table outside the GPU
self-park documented (incl. where the coefficient block is), not ported, because the host stops the firmware when idle, as on the A10
r33 gate hold, in-boot recovery, dirty start, kexec teardown hang recovery with the gate off for 10 ms: pvr_apple_reset() through runtime PM; the rest documented, not ported
MMIO licence documented, not ported as an accessor, because no GPU register is read while the firmware runs
page at GPU address 0 ("sink") s8003_map(): a zeroed page at address 0 of the kernel VM, plain leaf flags as yours
page-table cache clean before the release not a step of its own: pvr_mmu.c cleans every table it changes through the DMA API
interrupt 174 and its AIC acknowledge documented, not ported yet, because the AIC driver cannot clear a software-set interrupt

Jobs:

apple-g5p powervr backend
render-user-layout.h g5p_ru_channel_init(), compute-user-layout.h g5p_cu_channel_init() s8003_channel_init()
render-user-layout.h g5p_ru_fill_ta(), g5p_ru_fill_ta_v3(), the TA half of g5p_ru_rt_fill() s8003_fill_ta()
g5p_ru_fill_3d(), g5p_ru_fill_3d_v2(), g5p_ru_fill_3d_width(), the 3D half of g5p_ru_rt_fill(), render-user-msaa.h g5p_ru_fill_msaa() s8003_fill_3d(), s8003_fill_output()
render-user-geometry.h g5p_ru_rt_derive_samples() s8003_rt_derive(), from the HWRT dataset's width, height and samples
render-user-layout.h g5p_ru_job_at() (record links, microsequence, command, pointer ring) s8003_job_write()
compute-user-layout.h g5p_cu_job(), g5p_cu_sku() s8003_submit_compute()
drm-render-v3.inc g5p_rv3_publish_ta(), g5p_rv3_publish_3d(), drm-compute-v1.inc publish s8003_publish()
render-user-wake.h g5p_wk_ring(), drm-render-v3.inc g5p_rv3_dc_kick(), kick.inc s8003_doorbell()
render-user-layout.h g5p_ru_retired_pl(), g5p_cu_retired() pvr_apple_s8003_engine_done()
drm-render-v3.inc g5p_rv3_fw_touch() s8003_touch(), at most once a millisecond while a job is waited for
render-user-bind.h, drm-render-v3.inc g5p_rv3_bind(), g5p_rv3_unpark(), g5p_rv3_dc_wait() s8003_slots_load(), s8003_vm_attach(), s8003_device_control()
render-user-mmu.h, drm-render.inc g5p_render_invalidate() pvr_apple_s8003_mmu_flush()
render-user-window.h, drm-render.inc g5p_render_fw_entry() (firmware window) pvr_apple_s8003_vm_init(): the PM sets mapped in every VM at the firmware's addresses
render-user-pmset.h, render-user-layout.h g5p_ru_pm(), drm-render-v3.inc g5p_rv3_pmctx_build() s8003_pm_build(), pvr_apple_s8003_vm_scenes()
render-user-pmset.h park rule, render_user_close_park pvr_apple_s8003_vm_fini() and the park VM
recover.inc, power-r33.inc (gate cycle, 10 ms) pvr_apple_reset() with the chip's gate_off_ms
#119 render_user_pipeline_depth 2, render_user_joint, job slots two scenes per VM (scenes, vm_scenes()), the fragment job submitted with its geometry job, two record slots per channel
#125 "ring a phase's doorbell unless its own stamp ..." s8003_publish()

The page: docs/bring-up/gpu/a9-g5p.md. It also has the job format for Mesa ("The job interface for userspace"). If something you rely on is missing from it, say so here.

Authorship

Kernel patches whose content is ported from apple-g5p carry Berkay as author with the source files named (0020 boot, 0024 jobs); the glue in the shared driver is mine.

Open, by my own judgement

  • Never run, any of it.
  • Where the backend deliberately differs from apple-g5p, each unproven: the parameter manager's own tables are mapped for the PM and firmware only (yours were open to shaders); PDS and USC code have separate bases at powervr's heaps; a geometry job always hands the PM record over; completion is the stamps and watermarks alone; compute rings the geometry doorbell; a VM's two PM sets exist from its first render; 12 PM sets instead of 8; the gate cycle goes through runtime PM and is skipped when another job holds the GPU on.
  • The kernel does not check userspace's addresses in a job (as on the A10). A wrong one faults in that program's own slot and costs it its context; the PM memory itself is out of a program's reach.
  • No partial renders and no free-list growth: a scene larger than its pool stops the firmware and takes the 5 s path.
  • Completion is polled (100 us while jobs run) and the host reads the firmware's memory once a millisecond meanwhile, as your driver found necessary; the CPU cost of that is not measured.
  • R1+0x2010/0x2014, R0+0xe160, the two reset masks and command 8 are written as your driver writes them; what they mean is not known.

Not merging before CI is green.

henrikklee and others added 10 commits November 14, 2026 23:37
…ssor

Three preparations for the A9's GPU, whose firmware (Apple's) runs on the
Rogue core's META processor and lives in the driver's kernel VM:

- The driver's META LDR walker takes a target, so an image other than
  IMG's can be unpacked with it. IMG's firmware loads as before.
- An object can be mapped cache coherent for the device, and registers
  outside the GPU can be mapped into the kernel VM for a firmware.
- A chip of the Apple firmware path may leave out its job functions: its
  firmware is started and stopped, and no context can be created on it,
  so no client takes it for a GPU that renders. A chip may clean up
  what it put into the kernel VM.

Nothing changes on the iPad 6 and 7: the A10 has every job function, sets
none of the new flags and uses none of the new calls. Built for
apple-16k; not run on hardware.

Assisted-by: LLM
The powervr driver takes the iPad 5's GPU (A9, GT7600) next to the
A10's and starts Apple's firmware on it: it unpacks the tablet's own
firmware image, maps it with the firmware's shared memory and registers
into the kernel VM, programs Fender's boot stream, powers the core up
and waits for the firmware's "alive" event; then it gives the firmware
one empty command and waits until the firmware took it. When idle for a
second the firmware is stopped and the GPU powered off, and started
again on the next use.

One boot path: Apple's unmodified image, the firmware's governor pinned
to the power state the core starts in. The register addresses the
firmware is told and its interrupt number come from the device tree.

Nothing renders yet: the firmware's commands are not written, and no
context can be created. The core counts as experimental, so the driver
binds only with powervr.exp_hw_support=1; without it the probe wakes the
GPU's power block, reads the core's ID and powers it off again.

[Orchard: ported from orchard PR #74 (apple-g5p boot.inc, boot-layout.h,
pack.h, fender.h, perf-envelope.h, lifecycle.inc) by Berkay Kösebay, as
a chip of the powervr driver's Apple firmware path. What was left out is
recorded in docs/bring-up/gpu/a9-g5p.md.]

Built for apple-16k; not run on hardware.

Assisted-by: LLM
The J71t's device tree enables the GPU node, so the powervr driver finds
it. Without powervr.exp_hw_support=1 the driver reads the GPU's ID,
reports the core as unsupported and powers it off again; with it, it
starts the GPU's firmware. Nothing renders either way.

Only the J71t: its firmware recipe is the one that produces the GPU's
image. The J72t and the S8003 phones keep the node disabled.

The binding says which of the five interrupts is the firmware's: the
driver tells the firmware its number and the interrupt controller's
address.

Assisted-by: LLM
The powervr backend for the iPad 5's GPU carries one boot path of Berkay
Kösebay's apple-g5p driver. This page keeps the rest of what that driver
knows about booting the GPU: its alternate boot paths and why each
existed, the overclock, the firmware's own idle park, power cycling and
recovery, the two lost-GPU races of #125, the register access rules,
timing values and register meanings, with what was measured on J71t. It
also maps each piece of his boot code to where it lives now.

[Orchard: written from orchard PR #74, #119, #120 and #125 (apple-g5p)
by Berkay Kösebay; the findings and measurements are his.]

Assisted-by: LLM
Berkay's driver maps one zeroed page at address 0 of the firmware's
address space on every boot, so an access there while the firmware
starts does not fault. The powervr backend left it out; with no hardware
run to show it is not needed, it does what his working driver does.

The bring-up page follows: the page is ported, the page tables' cache
clean is what pvr_mmu already does for every table it changes, and a new
section says what the job path has to bring from his compute and render
code.

[Orchard: ported from orchard PR #74 (apple-g5p boot.inc
g5p_boot_sink_init()) by Berkay Kösebay.]

Built for apple-16k; not run on hardware.

Assisted-by: LLM
The shared Apple part of the powervr driver wrote every job the way the
A10's firmware takes it. The A9's firmware takes jobs differently, so a
chip can now provide that part itself: how a job is written and when it
is done, what the firmware keeps per program (per VM), and three
properties (how many scenes of one program the firmware holds, whether
free lists hold pages, whether a reset also takes the GPU's power
domain off).

Fences and their polling, the scheduler queues and their 5 s timeout,
the order of scenes between programs, runtime PM, sleep and the
performance state stay shared.

iPad 6 and 7 (A10): no change in behaviour. The A10 provides none of the
new functions, and each new branch is taken only by a chip that does.
One addition applies to every chip: a HWRT dataset remembers its width,
height and sample count.

Built for apple-16k (--device j71b, j71t); not run on hardware.

Assisted-by: LLM
The A9's firmware takes a job as a work record the kernel builds. The
record's words that are userspace's (buffer addresses, the firmware's
program words) do not fit the Rogue commands, so the A9 gets one stream
extension per job type next to the A10's: geometry, fragment, compute.
They carry what the render and compute passes of Berkay Kösebay's
apple-g5p UAPI carried (orchard #74), less what the Rogue command and
the HWRT dataset already say. powervr's UAPI header is unchanged.

docs/bring-up/gpu/a9-g5p.md, "The job interface for userspace", defines
every field and where it goes; the Mesa port is built from it.

Assisted-by: LLM
The iPad 5's GPU takes geometry, fragment and compute jobs through
powervr's UAPI now, so it can render once Mesa speaks its job format.

What a user gets with it:
- Programs each get their own address space on the GPU; up to two
  frames of one program are in the GPU at a time, a fragment job starts
  as soon as its geometry job is submitted.
- A job that hangs ends after 5 s together with the other jobs in
  flight. The firmware is stopped, the GPU's power domain goes off for
  at least 10 ms, and the next job boots the firmware again. Only the
  program whose job hung loses its GPU context; nothing stays frozen
  until reboot.
- The firmware is stopped a second after the last job and for sleep,
  and booted again on the next use.
- The parameter manager's memory (its pool, its tables, its free list)
  is the kernel's and out of a program's reach.

[Orchard: ported from orchard PR #74 with #119 and #125 by Berkay
Kösebay (apple-g5p render-user-layout.h, render-user-geometry.h,
render-user-msaa.h, render-user-bind.h, render-user-mmu.h,
render-user-pmset.h, render-user-wake.h, compute-user-layout.h,
drm-render-v3.inc, drm-render.inc, drm-compute-v1.inc), as the A9 chip
of the powervr driver's Apple firmware path. The work records, commands
and microsequences this writes are byte for byte what his headers
write for the same inputs (compared on the host for nine target sizes,
one and four samples). What was left out and where the backend differs
is recorded in docs/bring-up/gpu/a9-g5p.md.]

Built for apple-16k; not run on hardware.

Assisted-by: LLM
The bring-up page now says where each part of Berkay Kösebay's job path
lives in the powervr backend, how the firmware's channels, slots and
parameter-manager sets work as the backend uses them, what was left out,
and, for the Mesa port, exactly what userspace hands the kernel with a
free list, a HWRT dataset and each job on the A9.

"Open on hardware" lists where the backend differs from his driver and
has therefore never run in that form.

Assisted-by: LLM
The A9's GPU counts as supported in the powervr driver, as the A10's
does: the driver binds without powervr.exp_hw_support and the GPU's
firmware starts on every boot.

The cellular iPad 5 (J72t) gets the GPU too. Its restore image is the
J71t's (iPadOS 16.7.10), and its kernelcache carries byte for byte the
same GPU firmware: read with orchard.apple.remote_ipsw and extracted
with the recipe step, 119112 bytes, sha256 33ae20cb...5dca. Its firmware
recipe installs the file and its device tree enables the node.

The A9 iPhones (N71m, N66m, N69) stay without it: their iOS 15.8.8
kernelcaches carry a GPU firmware of another size, which the recipe step
refuses and which the driver's image layout was not checked against.

Until Mesa and the desktop session know the A9, nothing renders on its
GPU; an iPad 5 nightly in between exposes a GPU device that userspace
cannot drive yet.

Built for apple-16k (--device j71t, j72t, j71b); not run on hardware.

Assisted-by: LLM
@henrikklee henrikklee changed the title kernel: Boot the A9 GPU firmware from powervr kernel: The A9 GPU (iPad 5) in powervr: firmware boot and jobs Nov 14, 2026
The A9's check of a HWRT dataset read region_header_size as the bytes of
the whole region header buffer. pvr_drm.h defines it, for this parameter
format, as a size in dwords, and Mesa passes the ISP region size there
for every device: the region headers of one of the 16 macrotiles, a 64th
of what the check asked for. Every render on the A9 would have been
refused with EINVAL.

The check follows the UAPI now: at least mx * my * s * s * 5 / 4 dwords.
The buffer's own size (px * py * 5 bytes) is userspace's to get right;
the kernel cannot see it. The bring-up page's interface section says so.

Found by the Mesa port (OrchardLinux/mesa #8).

Built for apple-16k (--device j71t, j71b); not run on hardware.

Assisted-by: LLM
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants