[Electron][CoreML] Validate ORT 1.27 hybrid execution path

[JackCaow]

Summary

We validated the current PP-OCRv6 Small ONNX models with onnxruntime-node@1.27.0 inside a real Electron 37 main process on Apple Silicon.

The useful baseline is hybrid:

• detection: CoreML EP is materially faster
• recognition: CPU EP remains materially faster
• CPU/CoreML outputs remain within a 1e-3 absolute-error gate

This complements #3 and #6. It does not prove that the accelerated detection operations ran on MLGPU rather than MLCPU; compute-plan/device-placement qualification from #3 is still required.

Environment

• Apple M4
• macOS arm64
• Electron 37.10.0
• Electron Node.js 22.21.1
• onnxruntime-node@1.27.0
• current @arcships/light-ocr-model-ppocrv6-small detection and recognition models

CoreML session options:

{
  executionProviders: [{
    name: 'coreml',
    coreMlFlags: 0x10 | 0x20 // CreateMLProgram | CPUAndGPU
  }],
  graphOptimizationLevel: 'all',
  intraOpNumThreads: 4,
  interOpNumThreads: 1
}

CPU sessions used the same options with executionProviders: ['cpu'].

Each result below is the median of three warm calls after an initial call. Inputs were deterministic Float32 tensors so CPU/CoreML output parity could be measured directly.

Results

Stage / input	CPU warm median	CoreML warm median	Faster path
Detection [1, 3, 1280, 1280]	291.65 ms	92.37 ms	CoreML 3.16x
Recognition [8, 3, 48, 320]	58.14 ms	158.17 ms	CPU 2.72x

Provider partition logs:

• detection: CoreML accepted 236 / 242 nodes in 4 partitions
• recognition: CoreML accepted 270 / 287 nodes in 11 partitions

Output parity:

Stage	Max absolute difference	Mean absolute difference	Elements above 0.001
Detection	0.000241995	0.000003750	0
Recognition	0.000004351	0.000000000144	0

Cold behavior matters:

• CoreML detection session creation: about 407 ms
• first CoreML detection run / compilation: about 1.70 s
• subsequent detection runs: about 91-117 ms
• CoreML recognition session creation: about 729 ms

Sessions therefore need to be reused for the engine lifetime.

Electron compatibility result

onnxruntime-node@1.27.0 loaded and executed the CoreML EP successfully in the Electron 37 main process without an Electron-specific rebuild. The npm package is large (about 258 MB installed locally because it carries multiple platform runtimes), so adding a second production ORT beside light-ocr is not a desirable integration.

The validation dependency remains development-only in the Electron application. The production implementation should ship one ORT through light-ocr.

Proposed upstream direction

1. Upgrade the ORT runtime bundled by light-ocr from 1.22 to a currently qualified release such as 1.27.
2. Allow execution-provider selection per stage, rather than applying one provider to the whole OCR pipeline.
3. Use CoreML MLProgram CPUAndGPU for detection as an initial dynamic-model path.
4. Keep recognition on CPU with 4 threads and batch size 8 until the shape-specialized GPU path in [performance][macOS] Qualify true CoreML GPU execution with bounded/static OCR shapes #3 passes its broader quality/CPU/RSS gates.
5. Reuse both sessions for the engine lifetime and expose requested/actual provider placement as proposed in [performance] Add execution profiles and observable accelerator fallback #4.
6. Add Electron main-process smoke coverage to the Node-API compatibility matrix.

Suggested acceptance checks

• Electron main-process createEngine -> recognizeEncoded -> close succeeds on a packaged macOS arm64 app.
• Detection CoreML and CPU results pass a documented text/box/confidence corpus, not only tensor parity.
• Dynamic detection CoreML shows a repeatable warm speedup on supported Apple Silicon.
• Requested and actual CoreML device placement is observable.
• Recognition defaults to CPU unless a qualified CoreML profile is faster for the selected shape.
• ORT, native addon, JS facade, and model packages remain lockstep in ASAR/unpacked packaging.

Would you be open to a PR that first adds ORT 1.27 plus per-stage provider configuration and Electron smoke coverage, while leaving the static/enumerated-shape work tracked separately in #3?

[eric8810]

see our roadmap

[eric8810]

该探索路线由 #11 后的架构决策取代：Apple 加速最终采用可直接控制 shape、精度与 Compute Plan 的 Direct Core ML backend，而不是升级到 ORT 1.27 hybrid。此 issue 中的 Electron/CoreML 探针证据继续保留；Electron 正式支持仍未声明，如后续进入范围应以独立的 runtime/prebuild/ASAR/lifecycle issue 重新验收。