Repository navigation
kernel: The A9 GPU (iPad 5) in powervr: firmware boot and jobs - #138
Merged
Merged
Conversation
…ssor Three preparations for the A9's GPU, whose firmware (Apple's) runs on the Rogue core's META processor and lives in the driver's kernel VM: - The driver's META LDR walker takes a target, so an image other than IMG's can be unpacked with it. IMG's firmware loads as before. - An object can be mapped cache coherent for the device, and registers outside the GPU can be mapped into the kernel VM for a firmware. - A chip of the Apple firmware path may leave out its job functions: its firmware is started and stopped, and no context can be created on it, so no client takes it for a GPU that renders. A chip may clean up what it put into the kernel VM. Nothing changes on the iPad 6 and 7: the A10 has every job function, sets none of the new flags and uses none of the new calls. Built for apple-16k; not run on hardware. Assisted-by: LLM
The powervr driver takes the iPad 5's GPU (A9, GT7600) next to the A10's and starts Apple's firmware on it: it unpacks the tablet's own firmware image, maps it with the firmware's shared memory and registers into the kernel VM, programs Fender's boot stream, powers the core up and waits for the firmware's "alive" event; then it gives the firmware one empty command and waits until the firmware took it. When idle for a second the firmware is stopped and the GPU powered off, and started again on the next use. One boot path: Apple's unmodified image, the firmware's governor pinned to the power state the core starts in. The register addresses the firmware is told and its interrupt number come from the device tree. Nothing renders yet: the firmware's commands are not written, and no context can be created. The core counts as experimental, so the driver binds only with powervr.exp_hw_support=1; without it the probe wakes the GPU's power block, reads the core's ID and powers it off again. [Orchard: ported from orchard PR #74 (apple-g5p boot.inc, boot-layout.h, pack.h, fender.h, perf-envelope.h, lifecycle.inc) by Berkay Kösebay, as a chip of the powervr driver's Apple firmware path. What was left out is recorded in docs/bring-up/gpu/a9-g5p.md.] Built for apple-16k; not run on hardware. Assisted-by: LLM
The J71t's device tree enables the GPU node, so the powervr driver finds it. Without powervr.exp_hw_support=1 the driver reads the GPU's ID, reports the core as unsupported and powers it off again; with it, it starts the GPU's firmware. Nothing renders either way. Only the J71t: its firmware recipe is the one that produces the GPU's image. The J72t and the S8003 phones keep the node disabled. The binding says which of the five interrupts is the firmware's: the driver tells the firmware its number and the interrupt controller's address. Assisted-by: LLM
The powervr backend for the iPad 5's GPU carries one boot path of Berkay Kösebay's apple-g5p driver. This page keeps the rest of what that driver knows about booting the GPU: its alternate boot paths and why each existed, the overclock, the firmware's own idle park, power cycling and recovery, the two lost-GPU races of #125, the register access rules, timing values and register meanings, with what was measured on J71t. It also maps each piece of his boot code to where it lives now. [Orchard: written from orchard PR #74, #119, #120 and #125 (apple-g5p) by Berkay Kösebay; the findings and measurements are his.] Assisted-by: LLM
Berkay's driver maps one zeroed page at address 0 of the firmware's address space on every boot, so an access there while the firmware starts does not fault. The powervr backend left it out; with no hardware run to show it is not needed, it does what his working driver does. The bring-up page follows: the page is ported, the page tables' cache clean is what pvr_mmu already does for every table it changes, and a new section says what the job path has to bring from his compute and render code. [Orchard: ported from orchard PR #74 (apple-g5p boot.inc g5p_boot_sink_init()) by Berkay Kösebay.] Built for apple-16k; not run on hardware. Assisted-by: LLM
The shared Apple part of the powervr driver wrote every job the way the A10's firmware takes it. The A9's firmware takes jobs differently, so a chip can now provide that part itself: how a job is written and when it is done, what the firmware keeps per program (per VM), and three properties (how many scenes of one program the firmware holds, whether free lists hold pages, whether a reset also takes the GPU's power domain off). Fences and their polling, the scheduler queues and their 5 s timeout, the order of scenes between programs, runtime PM, sleep and the performance state stay shared. iPad 6 and 7 (A10): no change in behaviour. The A10 provides none of the new functions, and each new branch is taken only by a chip that does. One addition applies to every chip: a HWRT dataset remembers its width, height and sample count. Built for apple-16k (--device j71b, j71t); not run on hardware. Assisted-by: LLM
The A9's firmware takes a job as a work record the kernel builds. The record's words that are userspace's (buffer addresses, the firmware's program words) do not fit the Rogue commands, so the A9 gets one stream extension per job type next to the A10's: geometry, fragment, compute. They carry what the render and compute passes of Berkay Kösebay's apple-g5p UAPI carried (orchard #74), less what the Rogue command and the HWRT dataset already say. powervr's UAPI header is unchanged. docs/bring-up/gpu/a9-g5p.md, "The job interface for userspace", defines every field and where it goes; the Mesa port is built from it. Assisted-by: LLM
The iPad 5's GPU takes geometry, fragment and compute jobs through powervr's UAPI now, so it can render once Mesa speaks its job format. What a user gets with it: - Programs each get their own address space on the GPU; up to two frames of one program are in the GPU at a time, a fragment job starts as soon as its geometry job is submitted. - A job that hangs ends after 5 s together with the other jobs in flight. The firmware is stopped, the GPU's power domain goes off for at least 10 ms, and the next job boots the firmware again. Only the program whose job hung loses its GPU context; nothing stays frozen until reboot. - The firmware is stopped a second after the last job and for sleep, and booted again on the next use. - The parameter manager's memory (its pool, its tables, its free list) is the kernel's and out of a program's reach. [Orchard: ported from orchard PR #74 with #119 and #125 by Berkay Kösebay (apple-g5p render-user-layout.h, render-user-geometry.h, render-user-msaa.h, render-user-bind.h, render-user-mmu.h, render-user-pmset.h, render-user-wake.h, compute-user-layout.h, drm-render-v3.inc, drm-render.inc, drm-compute-v1.inc), as the A9 chip of the powervr driver's Apple firmware path. The work records, commands and microsequences this writes are byte for byte what his headers write for the same inputs (compared on the host for nine target sizes, one and four samples). What was left out and where the backend differs is recorded in docs/bring-up/gpu/a9-g5p.md.] Built for apple-16k; not run on hardware. Assisted-by: LLM
The bring-up page now says where each part of Berkay Kösebay's job path lives in the powervr backend, how the firmware's channels, slots and parameter-manager sets work as the backend uses them, what was left out, and, for the Mesa port, exactly what userspace hands the kernel with a free list, a HWRT dataset and each job on the A9. "Open on hardware" lists where the backend differs from his driver and has therefore never run in that form. Assisted-by: LLM
The A9's GPU counts as supported in the powervr driver, as the A10's does: the driver binds without powervr.exp_hw_support and the GPU's firmware starts on every boot. The cellular iPad 5 (J72t) gets the GPU too. Its restore image is the J71t's (iPadOS 16.7.10), and its kernelcache carries byte for byte the same GPU firmware: read with orchard.apple.remote_ipsw and extracted with the recipe step, 119112 bytes, sha256 33ae20cb...5dca. Its firmware recipe installs the file and its device tree enables the node. The A9 iPhones (N71m, N66m, N69) stay without it: their iOS 15.8.8 kernelcaches carry a GPU firmware of another size, which the recipe step refuses and which the driver's image layout was not checked against. Until Mesa and the desktop session know the A9, nothing renders on its GPU; an iPad 5 nightly in between exposes a GPU device that userspace cannot drive yet. Built for apple-16k (--device j71t, j72t, j71b); not run on hardware. Assisted-by: LLM
The A9's check of a HWRT dataset read region_header_size as the bytes of the whole region header buffer. pvr_drm.h defines it, for this parameter format, as a size in dwords, and Mesa passes the ISP region size there for every device: the region headers of one of the 16 macrotiles, a 64th of what the check asked for. Every render on the A9 would have been refused with EINVAL. The check follows the UAPI now: at least mx * my * s * s * 5 / 4 dwords. The buffer's own size (px * py * 5 bytes) is userspace's to get right; the kernel cannot see it. The bring-up page's interface section says so. Found by the Mesa port (OrchardLinux/mesa #8). Built for apple-16k (--device j71t, j71b); not run on hardware. Assisted-by: LLM
This was referenced Nov 14, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The iPad 5's GPU (A9, PowerVR GT7600) in the upstream
powervrdriver, as a second chip next to the A10: firmware boot (step 4, already here) and now the whole job path, so that it renders through powervr's UAPI. Built for apple-16k (--device j71t,j72t,j71b), not run on hardware. Per Henrik's decision there are no step-by-step hardware tests: this is merged when complete, and Berkay testsmainonce on his J71t. An iPad 5 nightly that does not start its GPU in between is accepted (installs use v0.3).What an iPad 5 does with it
/dev/dri/renderD*, once Mesa speaks the A9's job format (next step, not in this PR). Until then nothing uses the device and the desktop stays on software rendering.What is unchanged
pvr_drm.his untouched. The A9 gets three stream extension types next to the A10's.For Berkay: the one test on
mainAfter the merge, with the Mesa port installed. In order, stopping at the first that fails:
dmesg | grep -iE 'powervr|gfx|fender'showsInitialized powervr ...and no error line; a second latercat /sys/bus/platform/devices/201000000.gpu/power/runtime_statussayssuspended.dmesgstays free ofhang:andGFX firmwarelines.What a failure line means:
GFX firmware did not come up,... did not take its first command,Fender ...: the boot (unchanged since the first version of this PR).GFX firmware did not load slots A and B: attaching a program's address space (two device-control commands) was not taken within 3 s.Apple GPU job timed outfollowed byhang: ...lines: a job did not finish. The lines give, per engine, the last job submitted, both stamps, the five watermarks and the work ring's indices, then whether the firmware posted that it stopped.firmware event N: ...: the firmware posted something other than "alive" or a job's stamp, a fault or a recovery request (its 40 bytes follow); a job it gave up then times out.fragment job without a geometry job: not run: ordering, mine or Mesa's.The list "Open on hardware" at the end of
docs/bring-up/gpu/a9-g5p.mdsays what I expect could differ from your driver's behaviour, with the fallback where there is one.Your driver's code: where it is now
Boot:
ldr.hLDR parserpvr_fw_meta.cpvr_meta_ldr_process()) does it; its pins (119112 bytes, 94 records) are dropped, the recipe pins the filepack.himage packingpvr_apple_s8003.cs8003_ldr_mem(),s8003_ldr_config(),s8003_ldr_start_threads()boot-layout.hg5p_plan_layout()s8003_layout_plan()boot-layout.hg5p_make_boot_config()s8003_image_configure()boot-layout.hg5p_make_init()s8003_init_build(); PIO addresses and the interrupt from the device tree (s8003_parse_dt())boot.incprivate page tabless8003_map(),pvr_vm_map(), newpvr_vm_map_io()perf-envelope.hpinned record + seeds8003_init_perf()core.incFender wakepvr_apple_s8003_boot()fender.h,boot.incboot stream and releases8003_core_off(),s8003_release()boot.incevent drain,boot-r73b.incring orders8003_drain(), every 100 ms instead of on the interrupts8003_wait_alive()lifecycle.inccommand 8,kick.incs8003_first_command(),s8003_device_control(),s8003_doorbell()g5p_boot_park()pvr_apple_s8003_stop()s8003_core_ready())s8003_publish()arm/load/samplestages, snapshotss8003_slots_load(); the power repair and the grace periods around it are documented, not portedgpu-oc.h)pvr_apple_reset()through runtime PM; the rest documented, not porteds8003_map(): a zeroed page at address 0 of the kernel VM, plain leaf flags as yourspvr_mmu.ccleans every table it changes through the DMA APIJobs:
render-user-layout.hg5p_ru_channel_init(),compute-user-layout.hg5p_cu_channel_init()s8003_channel_init()render-user-layout.hg5p_ru_fill_ta(),g5p_ru_fill_ta_v3(), the TA half ofg5p_ru_rt_fill()s8003_fill_ta()g5p_ru_fill_3d(),g5p_ru_fill_3d_v2(),g5p_ru_fill_3d_width(), the 3D half ofg5p_ru_rt_fill(),render-user-msaa.hg5p_ru_fill_msaa()s8003_fill_3d(),s8003_fill_output()render-user-geometry.hg5p_ru_rt_derive_samples()s8003_rt_derive(), from the HWRT dataset's width, height and samplesrender-user-layout.hg5p_ru_job_at()(record links, microsequence, command, pointer ring)s8003_job_write()compute-user-layout.hg5p_cu_job(),g5p_cu_sku()s8003_submit_compute()drm-render-v3.incg5p_rv3_publish_ta(),g5p_rv3_publish_3d(),drm-compute-v1.incpublishs8003_publish()render-user-wake.hg5p_wk_ring(),drm-render-v3.incg5p_rv3_dc_kick(),kick.incs8003_doorbell()render-user-layout.hg5p_ru_retired_pl(),g5p_cu_retired()pvr_apple_s8003_engine_done()drm-render-v3.incg5p_rv3_fw_touch()s8003_touch(), at most once a millisecond while a job is waited forrender-user-bind.h,drm-render-v3.incg5p_rv3_bind(),g5p_rv3_unpark(),g5p_rv3_dc_wait()s8003_slots_load(),s8003_vm_attach(),s8003_device_control()render-user-mmu.h,drm-render.incg5p_render_invalidate()pvr_apple_s8003_mmu_flush()render-user-window.h,drm-render.incg5p_render_fw_entry()(firmware window)pvr_apple_s8003_vm_init(): the PM sets mapped in every VM at the firmware's addressesrender-user-pmset.h,render-user-layout.hg5p_ru_pm(),drm-render-v3.incg5p_rv3_pmctx_build()s8003_pm_build(),pvr_apple_s8003_vm_scenes()render-user-pmset.hpark rule,render_user_close_parkpvr_apple_s8003_vm_fini()and the park VMrecover.inc,power-r33.inc(gate cycle, 10 ms)pvr_apple_reset()with the chip'sgate_off_msrender_user_pipeline_depth2,render_user_joint, job slotsscenes,vm_scenes()), the fragment job submitted with its geometry job, two record slots per channels8003_publish()The page:
docs/bring-up/gpu/a9-g5p.md. It also has the job format for Mesa ("The job interface for userspace"). If something you rely on is missing from it, say so here.Authorship
Kernel patches whose content is ported from apple-g5p carry Berkay as author with the source files named (
0020boot,0024jobs); the glue in the shared driver is mine.Open, by my own judgement
R1+0x2010/0x2014,R0+0xe160, the two reset masks and command 8 are written as your driver writes them; what they mean is not known.Not merging before CI is green.