Repository navigation
Shrink ORT arena per window and abort separation on memory warnings - #83
Merged
sanylax0 merged 1 commit intoJul 18, 2026
Merged
Conversation
Device still jetsams during stem separation with prepacking disabled: the transformer's attention intermediates spike ONNX Runtime's CPU arena to its high-water mark, which is never returned between windows and sits on top of live playback for the whole track. Two levers: - Per-run arena shrinkage (memory.enable_memory_arena_shrinkage) frees the spike after every 7.8 s window; re-growing costs some CPU, which an offline cache-once job can afford. - iOS memory warnings now abort the in-flight separation (partial stems are deleted; the track plays without stems and retries on a later ensureStems pass) and drop the cached session — wired from RootView via PreparationQueue.handleMemoryPressure().
sanylax0
approved these changes
Jul 18, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
The device still jetsams during stem separation even with prepacking disabled (#81). The remaining spike is inference itself: HT-Demucs' attention intermediates grow ONNX Runtime's CPU arena to a high-water mark that is never returned between windows — it sits on top of live playback, decode buffers, and the UI for the entire track.
How
Two independent levers:
ORTRunOptionsconfigmemory.enable_memory_arena_shrinkage = cpu:0on every window'srun, so the arena's spike is returned to the OS after each 7.8 s window instead of held for the whole track. Re-growing the arena per window costs some CPU; separation is an offline cache-once job, so slower-but-alive wins (the same trade Fix launch jetsam + remove static onnxruntime embed #59 made choosing the CPU EP).UIApplication.didReceiveMemoryWarningNotification(wired in RootView) now callsPreparationQueue.handleMemoryPressure(), which flags the streaming loop to bail before its next window (the existing error path deletes the partial stems) and drops the cached ORT session. The track keeps playing without stems and re-enters separation on a laterensureStemspass. The flag is cleared at the start of each fresh separation so a stale warning can't kill future attempts.Testing
Not buildable in this environment. On device: play a fresh track and watch the Xcode memory gauge — RSS should now saw-tooth per window instead of ratcheting up, and if a warning still fires, the console will show the "aborted under memory pressure" stem log instead of a silent kill. If it still jetsams after this, the next diagnostic step is the JetsamEvent report (Settings → Privacy & Security → Analytics Data) to see the actual page counts — please share the header if so.
🤖 Generated with Claude Code
https://claude.ai/code/session_013TJoWkqg8bzkGdzxWhWjWP
Generated by Claude Code