Repository navigation
accel/ane: h17: recover a hung engine and retry a failed boot - #1768
RyanTheTide wants to merge 1 commit into
Conversation
On the Neo (T8140, H17 backend) a request that never completes quarantines the session, and the engine then stays unusable until reboot. A boot from the preload that fails, after a sleep or for the first request to a parked engine, quarantines the engine the same way. Typically that boot fails because ANE_SYS's power-on latched in the noirq resume, where the PMGR cannot run its PS reset recovery, and genpd carried on as if the domain were on. Recover from both, as the H16 and T8112 paths already do: - The session's ops->hung() queues a recovery with the shared restart sequence (ane_t6021_restart_run(), unplanned) and the hung_stop and hung_settle_ms parameters. It clears RUN in the wrapper, waits hung_settle_ms, powers the engine off without the firmware in macOS's order, with the CPU and sub-block power states read back off, then boots it again from the preload and replays every open program behind its handle. Nothing in the wrapper can prove the old firmware stopped, so the verified power-off is the proof. No reset is asserted. Requests wait in ane_h16_session_enter() while it runs; system sleep flushes it and blocks new ones. - A failed boot from the preload is retried once with ANE_SYS power cycled through the PMGR (apple_pmgr_ps_cycle()) in process context, where a latched power-on is cleared. The engine is quarantined only if that fails too. - With hung_stop set, a quarantined engine no longer stays down for the rest of the boot: system sleep powers it off without the firmware and the resume boots it cold. Programs closed while it was quarantined are released once the new firmware is up (ane_h16_session_recovered()). - At runtime a power word that will not go off fails the recovery instead of being left for a deep sleep. hung_stop=0 keeps the old behaviour: a hung session stays quarantined until reboot. debugfs ane-h17/<dev>/ gains recoveries, recover_failures and boot_retries. Assisted-by: LLM Signed-off-by: Ryan Murray <ryan@aurorasilicon.org>
RyanTheTide: accel/ane: h17: recover a hung engine and retry a failed boot With aurora-silicon#1863 (idle park) kept: the idle park leaves the engine alone while a hung-request recovery is queued or running. aurora-silicon#1303 is not taken (aurora-silicon#1768 carries its retry).
…merge The merge gave the shared closing brace to aurora-silicon#1768's last function. Assisted-by: Claude Opus 5.5 Signed-off-by: Nick <nekreshen@yandex.ru>
|
Some Neo hardware data for the hung-request path, from a branch that does the runtime part of this independently: https://github.com/niteria/linux/commits/ane/h17-hung-recover (one commit, 28e9a31, on the Neo's integration base with #1863 idle park; no Signed-off-by yet). Results on a J700 (Linux 7.3.0, Aura b9, booted with
Differences from this PR's runtime path (in case any of them is useful here):
For reference, macOS does the same thing in place: its ANE driver restarts the firmware after a request timeout, powering down without waiting for the firmware's POWER_DOWN acknowledgement, then cold-starting and reloading programs. That's from the driver's log strings in the 26.6.2 kernelcache, behaviour only. Happy to run this PR's hardware rows on the same Neo with the same hang reproducer, or to fold anything from the branch in here, whichever you prefer. |
Summary
On the Neo (T8140, H17 backend,
ane_fw_cold.c) a request that never completes quarantines the session and the ANE stays unusable until reboot. A boot from the preload that fails, after a sleep or for the first request to a parked engine, quarantines the engine the same way. The usual cause of that failed boot is an ANE_SYS power-on that latched in the noirq resume, where the PMGR cannot run its PS reset recovery.This brings H17 in line with the H16 (#1706) and T8112 (#1693) paths:
ops->hung()queues a recovery that runs the sharedane_t6021_restart_run()sequence (unplanned) withhung_stopandhung_settle_ms. It clears RUN in the wrapper, waitshung_settle_ms, powers the engine off without the firmware in macOS's order with the CPU and sub-block power states read back off, then boots it from the preload and replays every open program behind its handle. No reset is asserted. Requests wait inane_h16_session_enter()meanwhile; system sleep flushes the recovery and blocks new ones.apple_pmgr_ps_cycle()in process context, where a latched power-on is cleared. Quarantine only if that fails too. This is accel/ane: h17: retry a failed resume after power cycling ANE_SYS #1303's retry, moved to where the boot happens now (the parked engine boots on the next request, not at PM_POST_SUSPEND).hung_stopset, system sleep powers a quarantined engine off without the firmware and the resume boots it cold, so a failed recovery does not last until reboot. Programs closed while quarantined are released once the new firmware is up.hung_stop=0keeps the old behaviour for a hung session. debugfsane-h17/<dev>/gainsrecoveries,recover_failuresandboot_retries.Supersedes #1303.
Test
Neo hardware rows pending (forced hangs, s2r x3 with encoder rows, s2idle x3, failed-boot retry). checkpatch --strict clean.
🤖 Generated with Claude Code