Repository navigation
Conversation
|
Thanks, this is a clear fix for both limits. The arena rewind helps every backend. The give-back is opt-in through The code looks good to us. Two things before it leaves draft:
And the Signed-off-by, as you noted. |
|
Thank you for the review and for offering both runs.
niteria will add the Signed-off-by before this leaves draft. |
|
Thanks, the two H16 commits match the H17 shape well. The range goes into We will run the H17 rows on our A18 Pro (Neo) at this head on plain integration, and your H16 table on our M4 Pro, including the two-client case with lockdep on. Results will follow here. |
The transport copies every command into a 256 KiB pool arena. Each EXECUTE sequence gives its space back once the firmware reported the request complete, but LOAD, CREATE, TERMINATE, UNLOAD and the gating commands outside a request stay until the next firmware boot: 1164 bytes per program loaded and closed. About 210 programs into one firmware boot, h16_pool_room() refuses every LOAD with -ENOSPC. On H17 an idle park or a system sleep boots the firmware again and starts a new arena; a client that keeps the engine busy, or a platform that does not park, runs into it. Take the arena position at a LOAD that finds no program alive in this firmware boot. When the last program is torn down cleanly, with nothing in flight and the session not quarantined, the firmware is done with every command published since, so release the arena to that position. A program still loaded keeps its commands and every later one; a new firmware boot drops the position. Assisted-by: LLM Signed-off-by: Bartosz Nitka <niteria@gmail.com>
Program memory never goes back before reset. A closed program's surfaces become spares, which serve only a later mapping of the same size and permission; the table holds 64, and anything beyond it, or a LOAD that failed before or at the firmware, keeps its memory mapped and unused. A client that loads programs of many shapes (a sweep of held-out conv layers, camera frames at several sizes) thus fills the backend's DVA window: on the J700, 337 programs into a boot, every H16_LOAD failed with -ENOSPC until reboot, and the memory behind the window stayed allocated too. Add an optional ops->unmap that unmaps a surface, frees its memory and its DVA range, and use it for memory the firmware no longer uses: - Keep the spares oldest first, and free the oldest beyond 32 entries or 128 MiB after every give-back and LOAD. A spare table that is full frees the surface instead of keeping it. - When a mapping finds no room (-ENOSPC or -ENOMEM), free every spare and try once more. - A LOAD that fails before the firmware saw the program gives its memory back and frees the program. One the firmware refused, at LOAD or CREATE, joins the programs held for the verified power cycle of the recovery, which gives them back. The DART invalidation of an unmap must reach the engine, so nothing is freed while the session is asleep or parked; the spares keep it until the next LOAD, which runs with the engine up. Every caller holds the session lock. Without ops->unmap nothing changes. Assisted-by: LLM Signed-off-by: Bartosz Nitka <niteria@gmail.com>
The H17 backend hands out program and SHAREDMALLOC mappings by bumping a pointer through a 512 MiB DVA window, and never gives anything back: once the window is used up, every program mapping fails with -ENOSPC until reboot. On the J700 that happened after 337 program loads of a held-out layer sweep and some camera-denoise trials, and from then on every H16_LOAD failed, even of a 6 KB program that had run before. Allocate window ranges from a drm_mm instead, and implement the session's ops->unmap for program surfaces: find the mapping, unmap it from the DARTs (iommu_unmap() invalidates their TLBs before it returns), then free its pages and its range. A short unmap keeps the mapping, its memory and its range, as a failed mapping does. SHAREDMALLOC, pool, IPC, heap and preload mappings are never unmapped. A failed iommu_map() no longer leaves a partial mapping at the DVA the next allocation is given, since its range stays allocated with it. Assisted-by: LLM Signed-off-by: Bartosz Nitka <niteria@gmail.com>
ane_h16_alloc_prot() puts a mapping on h->mappings before it allocates anything, so a 16K block, vmap() or sg table that fails leaves the mapping and the blocks it already got on the list. Before RUN the rollback frees them; from RUN on nothing does, and every program mapping that fails with -ENOMEM (a fragmented buddy allocator, a camera-sized surface) keeps its partial memory until reboot. Free the memory of a mapping that fails before iommu_map_sgtable() right away: nothing of it is visible to the engine. A mapping that iommu_map_sgtable() refused is still recorded and kept until reset, as before. The rollback frees mappings through the same helper. Assisted-by: LLM Signed-off-by: Bartosz Nitka <niteria@gmail.com>
The H16 backend hands out program mappings by bumping a pointer through a 1 GiB DVA window and never gives anything back: once the window is used up, every program mapping fails with -ENOSPC until reboot. A client that loads programs of many shapes gets there, as the H17 backend did on the J700 after 337 loads of a held-out layer sweep. Allocate program ranges from a drm_mm instead, and implement the session's ops->unmap: find the program mapping, unmap it from the DARTs (iommu_unmap() invalidates their TLBs before it returns), then free its pages and its range. A short unmap keeps the mapping, its memory and its range, as a failed mapping does. SHAREDMALLOC, preload, heap tail and identity MMIO mappings have no range in the window and are never unmapped. The session calls ops->unmap only with the engine up, never while the session is asleep for system sleep or a recovery power cycle, so the invalidation reaches a powered engine. A new lock guards the mappings list and the window, since SHAREDMALLOC can be mapped while a program is unmapped. Assisted-by: LLM Signed-off-by: Bartosz Nitka <niteria@gmail.com>
24073e1 to
049ae04
Compare
On H17 (
ane_fw_cold.c), program memory is never given back. Everyane_fw_program_map()andane_fw_alloc()takes DVA from one 512 MiB window through a bump pointer (b->next_dva) that nothing ever lowers, and every mapping stays onb->mappingswith its pages until reboot. The session layer (ane_h16_program.c) reuses a closed program's surfaces only for a later mapping of the same size and permission, and drops them once its 64-entry spare table is full. A workload with varied shapes takes fresh DVA for almost every program. On a J700, a sweep of random conv shapes made everyH16_LOADreturn-ENOSPCafter about 340 loads, even for a 6 KB program that had passed earlier. The memory behind the window stayed allocated too. The H16 backend (ane_h16_backend.c, M4 class) has the same pattern with a 1 GiB window.A second, smaller limit: LOAD, CREATE, TERMINATE, UNLOAD and gate commands are never released from the 256 KiB command arena (only EXECUTE batches are). That is 0x48c bytes per program, so about 210 load/close cycles fit in one firmware boot. After that
h16_pool_room()returns-ENOSPCuntil something re-inits the transport: a system sleep, a recovery, or an idle park (#1863).Five commits, one change each:
session_restart) drops the mark.ops->unmap.-ENOSPC/-ENOMEMfrees all spares and retries once.s->held, so a recovery releases it.ops->unmap, behaviour is unchanged. Both backends implement it in the commits below.drm_mm(DRM_MM_INSERT_LOW, 16K alignment) under a newwindow_lock, which also guardsb->mappings.ane_fw_program_unmap()callsiommu_unmap()(the DART TLB invalidation happens inside it), then frees the pages and the range.ane_h16_alloc_prot()put a mapping onh->mappingsbefore it allocated anything. When a 16K block, thevmap()or the sg table failed, the blocks already allocated stayed on the list, and after RUN nothing freed them: every program mapping that failed with-ENOMEMleaked its partial memory until reboot. The memory is now freed at once, and the mapping is recorded only just beforeiommu_map_sgtable(). A mapping thatiommu_map_sgtable()refused is still kept until reset, as before.h->program_next) becomes adrm_mmof offsets fromh->program_start(DRM_MM_INSERT_LOW, 16K alignment). A newwindow_lockguards it andh->mappings, since SHAREDMALLOC can be mapped while a program surface is unmapped.ane_h16_program_unmap()(ops->unmap) unlinks the program mapping, callsiommu_unmap()(the TLB invalidation of the three DARTs happens inside it), and only then frees the pages and the range. On a short unmap it keeps everything.ops->unmapwith the engine off.The arena rewind assumes the firmware keeps no pointer into LOAD/CREATE command buffers once every program is UNLOADed. The EXECUTE arena already relies on the same assumption after completion. The device test below checks it.
Testing
KUnit (arm64 QEMU, KASAN/UBSAN): 259/259 on this branch. The new cases in
ane_h16_program_test.care:pool_rewindunmap_window-ENOSPC, and 2000 cycles pass withops->unmap.unmap_pressure-ENOSPCfrees the spares and retries.unmap_bytesunmap_parkedunmap_failed_mapunmap_refused_loadane_fw_cold.candane_h16_backend.care not built under KUnit; the backends'ops->unmapimplements the contract these cases test with a fake backend. A W=1 build ofdrivers/accel/ane/(the Neo config builds the H16 backend intoane_t6021) gives no warnings in the touched files, at every commit. The fulltools/asahi/run-host-tests.pygives the same result on this branch as on its base.H17
On a J700 (MacBook Neo, T8140), Aura 1.0.0b9, with ANE built in. These rows ran on a combined kernel, not on this exact head: it also carried #1768 (hang recovery), #1863 (idle park, with the close-while-parked fix) and a local non-contiguous allocation change. The third commit was resolved against that change: pages are freed with
dma_unmap_sgtable+vunmapand mapped withiommu_map_sgtable(). On plain integration/aurora it applies as in this PR. Booted withoutcma=.idle_park_ms=0-ENOSPC, no short unmap. The 6 KB program that used to fail still passes at the end, with byte-identical output.Kernel memory outside every
/proc/meminfobucket stayed flat across the run (about 104 MB before, 56–83 MB after 48 large loads). Nothing logged a WARN, an oops or a quarantine apart from the deliberate hang. (Two of my own compiler's conv tilings also hung the engine during the sweep. That was a bug in the tilings, not in this code, and the driver recovered both.)H16
The two H16 commits are compile- and review-checked only. They have not run on hardware: I have no H16 machine running Linux. Rows to run on an M4 (T8132/T6040/T6041), with
dmesgwatched forH16/ane/dartandMemAvailablenoted around each step:H16 stage 5 program window X..X+3fffffff, the same range as before, no WARNH16 program mapping <dva>lines stay at the bottom of the window (the same few DVAs), no-ENOSPC, noprogram unmap short, no WARN-ENOSPC(the old code fails every LOAD after about 1 GiB),MemAvailableback near the baseline at the endstress-ng --vmhog)-ENOMEMmaps flush the spares and retry; once the hog stops, the memory comes back and nothing stays leaked from failed mapsNotes
.hungto the session ops next to the new.unmap, and accel/ane: T6050: pin the firmware code, and admit the 27.2 build #1957/accel/ane: T8142: admit the macOS 27.2 (26B123) firmware build #1899 touchane_fw_cold.caround preload. These are textual overlaps only; arm64: apple: t6041: support J616c (M4 Max) on macOS 27.2 (26B123) firmware #1954 and Hibernation pm_test dry runs on Apple silicon #1814 touchane_h16_backend.cand merge cleanly with this branch. I can rebase onto whichever lands first.--strictsays the newwindow_lockhas no comment of its own. The comment just abovedva_windowcovers it: the lock guards the DVA window and the mappings list.The commits have
Assisted-by: LLMand noSigned-off-byyet. niteria will add the sign-off before this leaves draft.🤖 Generated with Claude Code