[performance][macOS] Qualify true CoreML GPU execution with bounded/static OCR shapes

[eric8810]

Why

The product goal on Apple Silicon is not only lower OCR latency: OCR must stop occupying the CPU needed by user-facing workloads. Merely registering CoreMLExecutionProvider is not enough because unsupported/dynamic regions can silently execute on MLCPU or fall back to ORT CPU.

Evidence collected on Apple M4 Max

Controlled probes against the current PP-OCRv6 Small ONNX models found:

• Dynamic recognition model: CoreML accepts most nodes, but the compute plan reports about 250 operations on MLCPU and only 2 on MLGPU when forced to CPUAndGPU; CPUAndNeuralEngine reports no ANE operations.
• Dynamic detection model: the tested compute-unit settings report about 204 operations on MLCPU.
• Static recognition shape (example width 1024): the plan reports 278 CoreML operations on MLGPU.
• Static detector shape (768×960): the plan reports 229 CoreML operations on MLGPU.
• Recognition, 100 warm calls at width 1024: CPU/12 threads = 1.330 s wall and 15.87 process-CPU seconds; static CoreML GPU = 1.069 s wall and 0.80 process-CPU seconds. The important result is about 95% less process CPU, not merely ~1.24× lower wall time.
• Static recognition widths 320/1024/2168/3200 were ~1.20–1.50× faster than CPU/12 and used roughly 0.71–0.74 CPU core on average. Random-input argmax matched in the probe; max absolute output difference was <= 3e-5.
• Static detector, 20 warm calls: CoreML GPU = 0.216 s versus CPU/12 = 0.856 s (~3.97×), using roughly 0.63 CPU core.
• Eight separate static recognition sessions (widths 320..3200) took ~5.97 s to initialize/first-run and peaked around 856 MiB RSS. That is useful spike evidence, but not an acceptable final architecture by itself.

These are microbenchmarks, not yet a full-document product result.

Upstream facts

• ONNX Runtime CoreML EP exposes MLComputeUnits, RequireStaticInputShapes, SpecializationStrategy, compute-plan profiling, and model caching. Its documentation explicitly warns that dynamic shapes may hurt performance: https://onnxruntime.ai/docs/execution-providers/CoreML-ExecutionProvider.html
• Core ML supports bounded flexible/enumerated shapes (up to 128 enumerated shapes) and documents a performance advantage for finite enumerated shapes: https://apple.github.io/coremltools/docs-guides/source/flexible-inputs.html
• Paddle's current macOS installation path remains CPU-only; there is no official Paddle GPU build for Apple Silicon: https://www.paddlepaddle.org.cn/documentation/docs/en/install/compile/macos-compile-make_en.html

Goal

Deliver a qualified Apple Silicon execution path in which both detection and recognition demonstrably execute on CoreML GPU and materially free CPU for foreground application work.

Investigate, in order:

1. A static detector shape matching the product's bounded detection policy.
2. Recognition shape specialization using a bounded width set. Widths 320..3200 in steps of 32 are 91 shapes, within Core ML's 128-shape enumerated limit.
3. Whether one converted Core ML MLProgram with enumerated shapes can replace a bank of ORT sessions. If ORT cannot represent this efficiently, evaluate a small direct-CoreML backend boundary.
4. Model compilation/cache behavior and FastPrediction specialization.
5. A minimal bucketing policy only if it passes OCR quality gates; padding/bucketing must not be assumed output-neutral.

Acceptance criteria

• Compute-plan evidence shows the intended detection and recognition operations on MLGPU, not predominantly MLCPU or ORT CPU fallback.
• Full 15-page reference PDF benchmark reports wall time, process CPU seconds, peak RSS, cold initialization, warm inference, and per-stage timing.
• Process CPU consumption drops by at least 80% versus the qualified CPU-fast profile while maintaining useful wall-clock performance.
• Golden OCR corpus covers narrow, wide, CJK/Latin mixed, dense forms, rotated/low-quality text, and the reference PDF.
• Text, confidence, box geometry, and reading order meet an agreed parity threshold; any precision change is explicit and documented.
• Session/model-cache count and peak RSS are bounded; the eight-session/~856 MiB spike is not adopted without a justified budget.
• Requested CoreML GPU mode fails loudly or reports degraded execution if the model falls back to MLCPU/CPU.
• Packaging, minimum macOS version, architecture support, and cold-cache behavior are documented.

Non-goals

• Claiming acceleration from provider registration alone.
• Making ANE a requirement: current static probes still show no ANE execution for these models.
• Making maximum CPU parallelism the interactive default.
• Changing OCR shapes/precision without corpus-level quality validation.

[eric8810]

已由 #11 的 0.3.0 加速实现完成。Direct Core ML 已交付有界 detector、91-function recognition、ANE/GPU 路由、逐函数 Compute Plan、14-fixture 质量验证、正式性能与 CPU-time、缓存/RSS/冷启动及生命周期 Gate。历史 15 页 PDF 因原文件不可复跑而保留为 spike 记录，正式验收已按文档化的锁定 workload 和 corpus 完成。