From 3fafa17088b85146d845f0d014a5188c8b9f0c3d Mon Sep 17 00:00:00 2001 From: Claude Date: Sat, 18 Jul 2026 17:31:49 +0000 Subject: [PATCH] Disable ORT graph optimization: session load alone jetsammed the app MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The playback heartbeat pinned it: headroom is flat (~3323 MB) through plain playback, the separation gate passed with 3.3 GB free, and the app died between 'ort session load begin' and 'ort session ready' — session creation alone consumed over 3.3 GB. The fp16 model keeps its weights behind per-weight Cast nodes, and load-time constant folding materializes an fp32 copy of every weight while the protobuf and fp16 originals are still resident. Skip graph optimization entirely: slower per window (fine for an offline cache-once job), but the load peak actually fits. --- Packages/ContinuityKit/Sources/Ingest/StemSeparator.swift | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/Packages/ContinuityKit/Sources/Ingest/StemSeparator.swift b/Packages/ContinuityKit/Sources/Ingest/StemSeparator.swift index dd24656..72b2dea 100644 --- a/Packages/ContinuityKit/Sources/Ingest/StemSeparator.swift +++ b/Packages/ContinuityKit/Sources/Ingest/StemSeparator.swift @@ -132,6 +132,14 @@ final class OnnxStemSeparator: StemSeparating { // for an 85M-param transformer that's hundreds of MB of duplicated RSS for a // marginal speedup on an offline cache-once job. Keeping the originals only. try options.addConfigEntry(withKey: "session.disable_prepacking", value: "1") + // NO graph optimization. Measured on device (mem breadcrumbs): session creation + // alone blew through >3.3 GB of headroom and jetsammed the app before "ort session + // ready". Mechanism: this model stores fp16 weights behind per-weight Cast nodes, + // and load-time constant folding materializes an fp32 copy of EVERY weight while + // the protobuf and the fp16 originals are still resident — several full copies of + // an 85M-param model at once. Skipping optimization trades some per-window CPU + // (fine for an offline cache-once job) for a load peak that actually fits. + try options.setGraphOptimizationLevel(ORTGraphOptimizationLevel.none) let session = try ORTSession(env: env, modelPath: modelURL.path, sessionOptions: options) cachedSession = (modelURL.path, session) MemoryFootprint.breadcrumb("ort session ready")