diff --git a/README.md b/README.md index 3a3e228..18187f7 100644 --- a/README.md +++ b/README.md @@ -10,138 +10,18 @@ English | [简体中文](README.zh-CN.md) ![light-ocr pixel-art banner](docs/assets/light-ocr-banner.png) -**Offline OCR for native and Node.js applications, powered by PP-OCRv6 Small.** +**Fast, offline OCR for Node.js and C++.** -`light-ocr` turns images into ordered text lines, confidence scores, and quadrilateral boxes—inside your own process, without sending an image to a cloud service or running a Python sidecar. The native Core accepts decoded pixels; the Node.js adapter on `main` also accepts in-memory JPEG and PNG bytes. +Recognize text in JPEG, PNG, or raw image data directly on your machine. `light-ocr` returns lines in reading order with confidence scores and quadrilateral coordinates. For Node.js, the npm package includes PP-OCRv6 Small and prebuilt components for macOS, Linux, and Windows. -It is made for products where OCR should feel like a local capability: quick to invoke, private by default, and straightforward to embed into an existing image pipeline. +## Quick start -> **Available on npm:** `@arcships/light-ocr@0.3.0` includes the default PP-OCRv6 Small model, prebuilt native runtimes for all Tier 1 platforms, opt-in tiled detection, direct in-memory JPEG/PNG input, and descriptor-driven hardware acceleration. See [Package support](#package-support). - -> **`0.3.0` acceleration:** macOS arm64 adds Direct Core ML; Linux x64/Vulkan and Windows x64/D3D12 add the official Native WebGPU Plugin EP. The recorded real-device results are **2.30×–2.85×** on Apple M4 Max, **5.70× aggregate P50** on NVIDIA RTX 5060 Ti, and **2.44× aggregate P50** on AMD Radeon 780M. WebGPU ships an FP32 execution profile; Apple uses its separately qualified FP16 route. macOS x64 remains on the CPU provider. - -## Where light-ocr fits - -| Use case | What light-ocr provides | -| --- | --- | -| **Desktop and local-first apps** | Extract text from screenshots, selections, clipboard images, notes, and imported pages without uploading user content. | -| **Private document workflows** | Read text from scanned forms, receipts, labels, and internal documents after your application renders or decodes them. | -| **Image, camera, and media tools** | Add searchable text, copy-text actions, overlays, indexing, or accessibility features to an existing pixel pipeline. | -| **On-premise and edge software** | Run a consistent OCR model in kiosks, terminals, appliances, or controlled networks where a cloud dependency is undesirable. | -| **Native and Node.js services** | Embed OCR directly instead of deploying and supervising a separate Python process or OCR daemon. | - -The current model is best suited to general text detection and recognition in CJK/Latin mixed content. The Node.js adapter can decode in-memory JPEG and PNG inputs; the native core still accepts decoded pixels only. PDF rendering, other image formats, document layout analysis, tables, formulas, and translation remain the host application's responsibility. - -## Why this project exists - -Cloud OCR is convenient, but it introduces uploads, network availability, recurring cost, and a new privacy boundary. Operating-system OCR APIs avoid the network, but their behavior and availability vary by platform. PaddleOCR offers excellent models, while its usual Python deployment is not always a natural fit for desktop software, native products, or a Node.js application. - -`light-ocr` closes that gap with one reusable native core built around official PP-OCRv6 Small models. Applications keep control of scheduling, storage, and user experience; the library focuses on turning images into structured OCR results while preserving a raw-pixel native boundary. - -## Why use light-ocr - -- **Local by default.** Recognition performs no runtime network access and does not start a child process. -- **Ready for real application pipelines.** It accepts `GRAY8`, `RGB8`, `BGR8`, and `RGBA8` pixel buffers; the Node.js adapter can also decode JPEG and PNG bytes already held in memory. -- **Two deliberate large-image modes.** Bounded/960 remains the fast, memory-conscious default. Opt-in tiled detection preserves more detail for small text and dense 2048-pixel documents while processing one detection tile at a time. -- **Native Apple acceleration when requested.** On macOS arm64, `0.3.0` can route FP16 detection and recognition through Core ML without changing the public OCR result contract. -- **Qualified Native WebGPU acceleration.** Version `0.3.0` packages the official WebGPU Plugin EP and its exact Linux/Vulkan or Windows/D3D12 runtime closure, with hash-verified offline staging and 164/164 real-device Gates on both recorded systems. -- **A pinned, reproducible model.** The self-contained PP-OCRv6 Small bundle is integrity-checked and designed to ship with the application instead of downloading on first use. -- **Consistent across supported platforms.** The same model and result contract are used on macOS, Linux, and Windows. -- **Built for asynchronous hosts.** The Node-API adapter keeps inference away from the JavaScript thread, with bounded queues, cancellation, and explicit lifecycle control. -- **Open and inspectable.** The project is Apache-2.0 licensed and tests real model behavior, high-resolution memory use, lifecycle safety, and output parity in CI. - -## Why PP-OCRv6 Small - -![Official PP-OCRv6 model and VLM accuracy comparison](docs/assets/ppocrv6-model-comparison.png) - -The npm package uses **PP-OCRv6 Small**. On PaddleOCR's in-house multi-scenario benchmark it reports **84.1 detection Hmean** and **81.3 weighted recognition accuracy**, while keeping the model practical for local applications. This chart and these accuracy scores come from the [official PP-OCRv6 evaluation](https://github.com/PaddlePaddle/PaddleOCR/blob/211989f046cc1878460f9e65574690c00a127a1a/docs/version3.x/algorithm/PP-OCRv6/PP-OCRv6.en.md); they are upstream quality results, not latency measurements made by this repository. - -## What results look like - -For each detected line, light-ocr returns the recognized text, a confidence score, and its position in the original image: - -```json -{ - "lines": [ - { - "text": "HELLO 123", - "confidence": 0.99, - "box": [ - {"x": 106, "y": 54}, - {"x": 554, "y": 54}, - {"x": 554, "y": 135}, - {"x": 106, "y": 135} - ] - } - ] -} -``` - -Coordinates are quadrilaterals rather than axis-aligned rectangles, so rotated and perspective text can be represented without discarding geometry. - -## Measured speed - -![The 800 by 180 HELLO 123 benchmark input](docs/assets/benchmark-generated-hello-123.png) - -### `0.3.0` acceleration at a glance - -| Provider and recorded device | Public precision | Measured end-to-end speedup | Quality evidence | -| --- | --- | ---: | --- | -| Apple/Core ML — Apple M4 Max | FP16 | **2.30×** on `HELLO 123`; **2.85×** on XFUND | 14 fixtures passed the locked CPU-parity thresholds | -| Native WebGPU/Vulkan — NVIDIA RTX 5060 Ti | FP32 | **5.70× aggregate P50**; 3.47×–9.30× per fixture | 14/14 byte-identical to CPU FP32; 164/164 Gates | -| Native WebGPU/D3D12 — AMD Radeon 780M | FP32 | **2.44× aggregate P50**; 1.28×–2.98× per fixture | 14/14 byte-identical to CPU FP32; 164/164 Gates | - -The WebGPU aggregate is `sum(CPU fixture P50) / sum(WebGPU fixture P50)` across the locked 14-fixture corpus. These are same-machine comparisons on the named devices, not a universal promise for every GPU or driver. - -### CPU baseline - -For the exact `800×180` BGR input above, light-ocr recognized `HELLO 123` with confidence `0.9893`. The native C++ Release benchmark was run on an Apple M4 Max (16-core CPU, 128 GB RAM), macOS 26.5.1, ONNX Runtime CPU with one intra-op and one inter-op thread, using the default bounded/960 strategy and recognition batch size 1. - -| Measurement | Result | -| --- | ---: | -| Warm end-to-end median | **75.678 ms/image** (~13.2 images/s) | -| Warm end-to-end P95 | **79.788 ms/image** | -| Detection + recognition inference median | **74.125 ms/image** | -| Model bundle load, once | 167.906 ms | -| Engine initialization, once | 30.511 ms | - -The test uses 5 warm-up runs followed by 30 measured runs. It is a small, synthetic, single-line fixture; latency varies with hardware, input dimensions, text density, and line count. The benchmark contract and comparison with the pinned Python oracle are recorded in [Implementation status](docs/implementation-status.md#本机最终验证快照). - -### Apple Core ML acceleration - -The `0.3.0` provider gate compared the opt-in FP16 Core ML path with the `cpu_fast` profile on one Apple M4 Max (16-core CPU, 128 GB RAM) running macOS 26.5.1. The CPU profile used up to 12 intra-op threads; each workload used 5 warm-up runs and 3 independent sets of 30 measured runs. The CPU-time reduction describes host process usage, not energy consumption. - -| Locked workload | CPU warm P50 | Apple warm P50 | End-to-end speedup | OCR process CPU-time reduction | -| --- | ---: | ---: | ---: | ---: | -| Synthetic `HELLO 123`, 800×180 | 19.774 ms | **8.599 ms** | **2.300×** | **95.91%** | -| Dense XFUND form, 113 text lines | 943.627 ms | **331.011 ms** | **2.851×** | **97.67%** | - -The accelerated output also passed all 14 locked quality fixtures: 99.6484% character similarity to the CPU oracle, 100% detection recall, 99.5508% mean matched IoU, 0.004349 mean matched confidence difference, and zero critical failures. These are parity measurements against the CPU output, not independent ground-truth accuracy; FP16 output is not byte-for-byte identical. - -The formal warm performance runs peaked at 692.14 MiB RSS and the self-contained Apple model payload added 25.42 MiB. The separate same-engine 100-dense-page lifecycle run peaked at 888.11 MiB and finished 27.47 MiB below its post-warm-up baseline, showing no sustained growth in that run. First use performs offline compilation and loads recognition functions on demand: the fixed `HELLO 123` startup canary took 7.219 s on a compiled-cache miss and 1.275/1.278 s on hits; the 113-line form took 53.846 s on its first full-page miss and 12.677/12.677 s on hits. No provider, compiler, or model is downloaded at runtime. - -Only that single M4 Max runner has real-device performance data. The evidence contract classifies it under the `Apple M4` device family for `deviceValidated`; this does not represent separate measurements of every M4 SKU. M1–M3 and later Apple Silicon can try the same ANE/GPU route and report `deviceValidated: false` without inheriting a speed claim. The `0.3.0` macOS x64 package remains CPU-only after its release smoke test failed Core ML OCR parity. See the [Apple acceleration design and evidence](docs/apple-device-acceleration.md) for methodology, model placement, quality thresholds, cache behavior, and lifecycle results. - -### Native WebGPU acceleration - -The Linux report used an NVIDIA RTX 5060 Ti through Dawn/Vulkan. Across the same 14 fixtures, CPU P50 totaled 5,475.623 ms and WebGPU FP32 P50 totaled 961.042 ms, for a **5.698× aggregate speedup**. Every fixture was faster, ranging from 3.474× to 9.299×. - -The Windows report used an AMD Radeon 780M through Dawn/D3D12. CPU P50 totaled 6,500.853 ms and WebGPU FP32 P50 totaled 2,669.160 ms, for a **2.436× aggregate speedup**. Every fixture was faster, ranging from 1.277× to 2.982×. Its warmup-aware repeated-lifecycle result finished 22.9 MiB below the post-warm-up baseline. - -Both reports passed 164/164 Gates, including Auto selection, native C++, placement profiles, exact FP32 OCR parity, cold start, memory, lifecycle, and strict fail-closed behavior. The current models require a bounded CPU partition for `Concat`, `Gather`, and `Slice`; `cpuPartition: "forbid"` therefore rejects engine creation instead of silently changing placement. WebGPU FP16 is not a public `0.3.0` execution profile and no FP16 speed claim is made. See [Linux device acceleration](docs/linux-device-acceleration.md) and [Windows device acceleration](docs/windows-device-acceleration.md). - -## Get started - -### Node.js - -Node.js 22 and 24 are supported on macOS arm64/x64, Linux x64 glibc, and Windows x64: +Node.js 22 and 24 are supported. ```bash npm install @arcships/light-ocr ``` -The package installs the matching native runtime and the pinned PP-OCRv6 Small model. It does not download a model at first run or compile native code during `postinstall`. Version 0.3.0 supports both `recognizeEncoded()` and raw-pixel `recognize()`. - ```ts import { createEngine } from "@arcships/light-ocr"; import { readFile } from "node:fs/promises"; @@ -151,117 +31,67 @@ const result = await engine.recognizeEncoded( await readFile("image.jpg"), ); -// Raw pixels remain available when the host already owns a decode pipeline. -const rawResult = await engine.recognize({ - data: pixels, - width, - height, - stride, - pixelFormat: "rgba8", -}); +for (const line of result.lines) { + console.log(line.text, line.confidence, line.box); +} -console.log(result.lines); -console.log(rawResult.lines); await engine.close(); ``` -The published package uses a platform runtime descriptor for Auto selection. Explicit Apple and WebGPU remain strict single-provider requests. On macOS arm64, Apple can be requested directly: - -```ts -const engine = await createEngine({ - execution: { - provider: "apple", - precision: "fp16", - cpuPartition: "allow", - sessionFallback: "error", - }, -}); - -console.log(engine.info.execution.sessions.detection.deviceValidated); -``` - -On Linux x64 and Windows x64, the WebGPU profile is: +`createEngine()` automatically chooses the right execution mode for the current platform. If your application already decodes images, [`recognize()`](bindings/node/README.md#使用) also accepts `GRAY8`, `RGB8`, `BGR8`, and `RGBA8` pixel data. -```ts -const engine = await createEngine({ - execution: { - provider: "webgpu", - precision: "fp32", - cpuPartition: "allow", - sessionFallback: "error", - }, -}); -``` +## What you get -`cpuPartition: "allow"` and the strict GPU-only profile apply to the Apple provider on Apple Silicon. The `0.3.0` macOS x64 package exposes CPU only. Explicit providers never fall through to CPU; only Auto may advance through its descriptor-locked creation candidates. Calling `createEngine()` without `execution` now uses Auto. +- **Local processing.** Images and OCR results stay on your machine. +- **One package to install.** The model and matching prebuilt component are included with the npm package. +- **Useful output.** Every line includes recognized text, confidence, and its position in the original image. +- **Hardware acceleration by default.** Auto tries Core ML first on macOS 15+ Apple Silicon, and WebGPU first on the Linux and Windows builds below. +- **Application-friendly execution.** Recognition runs off the JavaScript main thread and supports queues, cancellation, and explicit cleanup. +- **Small text in large images.** An optional `tiled` mode preserves small and dense text in high-resolution images. -See the [Node.js guide](bindings/node/README.md) for the full API, cancellation, queue limits, and lifecycle behavior. +## Platform acceleration -### C++ core +The npm package provides the following four builds. The default `createEngine()` call uses Auto mode: -Requirements: Python 3 for bootstrap tooling, CMake, and a C++17 compiler. Dependencies and model inputs are pinned; the built runtime does not depend on Python. - -```bash -python3 tools/bootstrap_dependencies.py --cache-dir .cache/dependencies -python3 tools/bootstrap_models.py --cache-dir .cache/models - -cmake --preset release \ - -DLIGHT_OCR_DEPENDENCY_CACHE_DIR="$PWD/.cache/dependencies" -cmake --build --preset release --parallel -ctest --preset release -``` - -See [Build and release](docs/build-and-release.md) for platform prerequisites and [C++ API](docs/native-api.md) for integration. - -## Package support - -| Distribution | Status | Platforms | -| --- | --- | --- | -| C++ core source | Available | macOS arm64/x64, Linux x64 glibc, Windows x64 | -| Node-API adapter source | Available | Node.js 22 and 24 | -| [`@arcships/light-ocr`](https://www.npmjs.com/package/@arcships/light-ocr) | `0.3.0` published | Node.js 22/24 on all Tier 1 platforms | -| [`@arcships/light-ocr-model-ppocrv6-small`](https://www.npmjs.com/package/@arcships/light-ocr-model-ppocrv6-small) | `0.3.0` published | Platform-independent required model dependency | -| Platform native npm packages | `0.3.0` published | macOS arm64/x64, Linux x64 glibc, Windows x64 | - -The npm distribution installs one facade, one required model package, and the native package matching the host platform. Package contents, versioning, and release gates are documented in [npm packaging](docs/npm-packaging.md); immutable `0.3.0` hashes and validation evidence are recorded in the [release record](docs/releases/npm-0.3.0.md). +| Platform | Auto mode | +| --- | --- | +| macOS on Apple Silicon | Core ML on macOS 15+, then CPU | +| macOS on Intel | CPU | +| Linux x64 with glibc | WebGPU through Vulkan, then CPU | +| Windows x64 | WebGPU through D3D12, then CPU | -Direct Core ML acceleration on macOS arm64 is published in `0.3.0` using the same six-package installation shape; it adds no provider package or runtime download. macOS x64 remains CPU-only. +Applications that need explicit control can choose `auto`, `cpu`, `apple`, or `webgpu` through the [`execution` option](bindings/node/README.md#使用). -Version `0.3.0` publishes Native WebGPU on Linux x64 and Windows x64. Explicit WebGPU accepts `auto`/`fp32`; Auto also selects FP32. The three required CPU-partition operators are reported and bounded. Both real-device reports passed 164/164 Gates, and their immutable report/artifact hashes are bound into the production lock. +## Measured performance -## Project status +Version 0.3.0 was measured on three real devices: -`light-ocr` is under active development. Version `0.3.0` includes deterministic `tiled-v1`, bounded in-memory JPEG/PNG decoding, descriptor-driven Auto selection, Direct Core ML execution on macOS arm64, and FP32 Native WebGPU execution on Linux x64/Windows x64 without changing the raw-pixel C++ Core boundary. +![light-ocr 0.3.0 same-device speed and OCR process CPU-time reductions](docs/assets/light-ocr-0.3.0-performance-v2.png) -As a pre-1.0 project, public APIs and package layout may still evolve; the project does not currently promise a stable cross-release C++ ABI. +| Device | Acceleration | End-to-end speedup | OCR process CPU time | +| --- | --- | ---: | ---: | +| Apple M4 Max | Core ML | **2.30×** on `HELLO 123`; **2.85×** on a dense form | **95.91%–97.67% less** | +| NVIDIA RTX 5060 Ti on Linux | WebGPU / Vulkan | **5.70×** overall across 14 test images | **69.97% less** | +| AMD Radeon 780M on Windows | WebGPU / D3D12 | **2.44×** overall across 14 test images | **46.33% less** | -The Core CI builds and tests the project on: +These are same-machine comparisons with the CPU path, and results vary by workload and hardware. For the 14-image results, overall speedup is the sum of the per-image CPU median times divided by the sum of the WebGPU median times. The CPU column measures cumulative OCR process CPU time over the same workloads, rather than an instantaneous system-utilization sample; lower CPU time leaves more capacity for the rest of the application while OCR is active. The Apple run passed its locked CPU-parity thresholds; both WebGPU runs were byte-identical to CPU FP32 on all 14 images. See the [0.3.0 release report](docs/releases/npm-0.3.0.md) for complete measurements and methodology. -- macOS arm64 -- macOS x64 -- Linux x64 with glibc -- Windows x64 +## C++ -It also runs sanitizers, fuzz smoke tests, offline-runtime checks, output parity, quality, and memory gates. Performance qualification is a separate, explicitly triggered workflow and is not part of ordinary CI or release preflight. See the [current implementation status](docs/implementation-status.md) for verified results and known gaps. +C++ projects build the static library from source and link the `light_ocr::core` CMake target. The API accepts decoded `GRAY8`, `RGB8`, `BGR8`, or `RGBA8` pixels; start with the [C++ API guide](docs/native-api.md) and [build instructions](docs/build-and-release.md). ## Documentation -- [Changelog](CHANGELOG.md) +- [Node.js API and examples](bindings/node/README.md) - [C++ API](docs/native-api.md) -- [Node.js adapter](bindings/node/README.md) -- [Build and release](docs/build-and-release.md) +- [Apple Silicon acceleration](docs/apple-device-acceleration.md) +- [Linux WebGPU acceleration](docs/linux-device-acceleration.md) +- [Windows WebGPU acceleration](docs/windows-device-acceleration.md) - [Model bundle](docs/model-bundle.md) -- [Accuracy and parity](docs/parity-testing.md) -- [High-resolution memory behavior](docs/memory-optimization.md) -- [Architecture](docs/architecture.md) -- [Implementation status](docs/implementation-status.md) - -## Community - -Issues and pull requests are welcome. If you are considering light-ocr for a product, feel free to [open an issue](https://github.com/arcships/light-ocr/issues) describing the platform, image source, language mix, and expected workload. Real application scenarios help shape package priorities and future model support. - -When reporting a bug, please include the platform, input dimensions and pixel format, the model bundle ID, and a minimal reproduction when possible. Do not attach private source images unless you are comfortable publishing them. +- [Build and release](docs/build-and-release.md) +- [Changelog](CHANGELOG.md) +- [npm 0.3.0 release report](docs/releases/npm-0.3.0.md) -## License +## Community and license -light-ocr is available under the [Apache License 2.0](LICENSE). Third-party dependencies and model notices are included with their corresponding release artifacts. +Issues and pull requests are welcome. `light-ocr` is available under the [Apache License 2.0](LICENSE). diff --git a/README.zh-CN.md b/README.zh-CN.md index 70ccd6a..f50e4f3 100644 --- a/README.zh-CN.md +++ b/README.zh-CN.md @@ -10,138 +10,18 @@ ![light-ocr 像素风宣传图](docs/assets/light-ocr-banner.png) -**为原生应用和 Node.js 应用准备的离线 OCR,由 PP-OCRv6 Small 驱动。** +**面向 Node.js 与 C++ 的快速离线 OCR。** -`light-ocr` 在应用自己的进程内,把图片转换为按阅读顺序排列的文字、置信度和四边形位置。它不需要把图片上传到云端,也不需要额外运行 Python 服务。原生 Core 接受解码后的像素;`main` 上的 Node.js 适配器还可以直接接收内存中的 JPEG 和 PNG bytes。 +直接在本机识别 JPEG、PNG 或像素数据,返回按阅读顺序排列的文字、置信度和四边形坐标。Node.js 用户安装的 npm 包内置 PP-OCRv6 Small 模型,并提供 macOS、Linux 和 Windows 的预编译组件。 -这个项目面向希望把 OCR 做成真正本地能力的产品:随时调用、默认保护隐私,也能自然嵌入现有的图像处理流程。 +## 快速开始 -> **npm 已可用:**`@arcships/light-ocr@0.3.0` 自带默认 PP-OCRv6 Small 模型和全部 Tier 1 平台的预编译原生运行时,并支持可选 tiled 检测、Node.js 内存 JPEG/PNG 直接输入和 descriptor 驱动的硬件加速。详见[包支持](#包支持)。 - -> **`0.3.0` 加速:**macOS arm64 加入 Direct Core ML;Linux x64/Vulkan 与 Windows x64/D3D12 加入官方 Native WebGPU Plugin EP。已记录的真机结果分别为 Apple M4 Max **2.30×–2.85×**、NVIDIA RTX 5060 Ti **聚合 P50 5.70×**、AMD Radeon 780M **聚合 P50 2.44×**。WebGPU 发布 FP32 执行 profile,Apple 使用独立资格验证的 FP16 路径;macOS x64 保持 CPU provider。 - -## 适合哪些场景 - -| 应用场景 | light-ocr 能提供什么 | -| --- | --- | -| **桌面端与本地优先应用** | 从截图、框选区域、剪贴板图片、笔记和导入页面中提取文字,不上传用户内容。 | -| **私有文档流程** | 在应用完成渲染或解码后,识别扫描表单、票据、标签和内部文档中的文字。 | -| **图片、相机与媒体工具** | 在现有像素流程中加入全文搜索、复制文字、画面标注、内容索引或无障碍能力。 | -| **本地部署与边缘软件** | 在自助终端、设备、边缘节点或受控网络中运行一致的 OCR 模型,摆脱云服务依赖。 | -| **原生与 Node.js 服务** | 把 OCR 直接嵌入应用,不再单独部署和维护 Python 进程或 OCR daemon。 | - -当前模型主要面向常规文字检测和 CJK/拉丁字符混排识别。Node.js 适配器可以解码内存中的 JPEG/PNG;原生 Core 仍只接受解码后的像素。PDF 渲染、其他图片格式、文档版面分析、表格、公式和翻译仍由宿主应用负责。 - -## 为什么要做 light-ocr - -云 OCR 使用方便,但也带来了图片上传、网络可用性、持续成本和新的隐私边界。操作系统 OCR API 不依赖网络,但各个平台的能力与行为并不一致。PaddleOCR 提供了优秀的模型,不过常见的 Python 部署方式并不总适合桌面软件、原生产品和 Node.js 应用。 - -`light-ocr` 希望补上这块空白:围绕官方 PP-OCRv6 Small 模型,提供一套可复用的原生核心。应用继续掌控任务调度、数据存储和用户体验;light-ocr 在保留原生 raw-pixel 边界的同时,把图片稳定地转换为结构化 OCR 结果。 - -## light-ocr 的优势 - -- **默认本地运行。**识别过程不会访问网络,也不会启动子进程。 -- **适合真实应用流程。**直接接收 `GRAY8`、`RGB8`、`BGR8` 和 `RGBA8` 像素;Node.js 适配器也能解码已经在内存中的 JPEG 和 PNG。 -- **两种明确的大图策略。**bounded/960 仍是速度和内存优先的默认模式;可选 tiled 检测为小字和密集的 2048 像素文档保留更多细节,并始终逐个处理 detection tile。 -- **按需启用原生 Apple 加速。**在 macOS arm64 上,`0.3.0` 可以用 Core ML 执行 FP16 detection/recognition,同时保持公共 OCR 结果契约不变。 -- **已完成真机资格验证的 Native WebGPU 加速。**`0.3.0` 会打包官方 WebGPU Plugin EP 及其精确的 Linux/Vulkan 或 Windows/D3D12 运行时闭包,支持哈希校验的离线 staging;两台记录设备均通过 164/164 Gate。 -- **模型固定且可复现。**自包含的 PP-OCRv6 Small bundle 会经过完整性验证,目标是随应用一起安装,而不是首次运行时再下载。 -- **跨平台结果一致。**macOS、Linux 和 Windows 使用同一套模型与结果契约。 -- **适合异步宿主。**Node-API 适配器不会占用 JavaScript 主线程,并提供有界队列、取消和明确的生命周期控制。 -- **开放、可检查。**项目采用 Apache-2.0 协议,并在 CI 中验证真实模型行为、大图内存、生命周期安全和输出对齐。 - -## 为什么选择 PP-OCRv6 Small - -![PP-OCRv6 模型与 VLM 官方准确率对比](docs/assets/ppocrv6-model-comparison.png) - -npm package 使用 **PP-OCRv6 Small**。在 PaddleOCR 的内部多场景基准中,它取得了 **84.1 的检测 Hmean** 和 **81.3 的识别加权准确率**,同时保持适合本地应用的模型规模。图表和准确率来自 [PP-OCRv6 官方评测](https://github.com/PaddlePaddle/PaddleOCR/blob/211989f046cc1878460f9e65574690c00a127a1a/docs/version3.x/algorithm/PP-OCRv6/PP-OCRv6.md);这是上游质量结果,不是本项目测得的耗时。 - -## 返回结果是什么样的 - -对于每一行检测到的文字,light-ocr 都会返回识别文本、置信度,以及它在原图中的位置: - -```json -{ - "lines": [ - { - "text": "HELLO 123", - "confidence": 0.99, - "box": [ - {"x": 106, "y": 54}, - {"x": 554, "y": 54}, - {"x": 554, "y": 135}, - {"x": 106, "y": 135} - ] - } - ] -} -``` - -位置使用四边形而不是普通矩形,因此可以保留旋转文字和透视文字的几何信息。 - -## 实测速度 - -![800×180 的 HELLO 123 benchmark 输入图](docs/assets/benchmark-generated-hello-123.png) - -### `0.3.0` 加速总览 - -| Provider 与记录设备 | 公共精度 | 端到端实测加速 | 质量证据 | -| --- | --- | ---: | --- | -| Apple/Core ML — Apple M4 Max | FP16 | `HELLO 123` **2.30×**;XFUND **2.85×** | 14 fixtures 通过锁定的 CPU parity 阈值 | -| Native WebGPU/Vulkan — NVIDIA RTX 5060 Ti | FP32 | **聚合 P50 5.70×**;单 fixture 3.47×–9.30× | 14/14 与 CPU FP32 字节级一致;164/164 Gate | -| Native WebGPU/D3D12 — AMD Radeon 780M | FP32 | **聚合 P50 2.44×**;单 fixture 1.28×–2.98× | 14/14 与 CPU FP32 字节级一致;164/164 Gate | - -WebGPU 聚合值按锁定的 14-fixture corpus 计算:`CPU fixture P50 之和 / WebGPU fixture P50 之和`。这些数字是在表中指定设备上的同机对照,不是对所有 GPU 和驱动的统一承诺。 - -### CPU 基线 - -对于上面这张 `800×180` BGR 图片,light-ocr 识别结果为 `HELLO 123`,置信度 `0.9893`。原生 C++ Release benchmark 运行在 Apple M4 Max(16 核 CPU、128 GB 内存)、macOS 26.5.1、ONNX Runtime CPU 环境,intra-op 与 inter-op thread 均为 1,并使用默认 bounded/960 策略和 recognition batch size 1。 - -| 指标 | 实测结果 | -| --- | ---: | -| 预热后端到端中位数 | **75.678 ms/张**(约 13.2 张/秒) | -| 预热后端到端 P95 | **79.788 ms/张** | -| 检测 + 识别纯推理中位数 | **74.125 ms/张** | -| 模型 bundle 加载,仅一次 | 167.906 ms | -| Engine 初始化,仅一次 | 30.511 ms | - -测试先预热 5 次,再测量 30 次。这是一张小尺寸、合成的单行图片;实际延迟会随硬件、图片尺寸、文字密度和文本行数变化。benchmark 契约及与固定 Python oracle 的对照见[实施状态](docs/implementation-status.md#本机最终验证快照)。 - -### Apple Core ML 加速 - -`0.3.0` provider Gate 在一台 Apple M4 Max(16 核 CPU、128 GB 内存,macOS 26.5.1)上,对比了显式启用的 FP16 Core ML 路径和 `cpu_fast` profile。CPU profile 最多使用 12 个 intra-op threads;每个 workload 先预热 5 次,再进行 3 组独立的 30 次测量。CPU time 降幅表示宿主进程占用变化,不是能耗数据。 - -| 锁定 workload | CPU 预热 P50 | Apple 预热 P50 | 端到端加速 | OCR 进程 CPU time 降幅 | -| --- | ---: | ---: | ---: | ---: | -| 合成 `HELLO 123`,800×180 | 19.774 ms | **8.599 ms** | **2.300×** | **95.91%** | -| XFUND 密集表单,113 行文字 | 943.627 ms | **331.011 ms** | **2.851×** | **97.67%** | - -加速输出同时通过全部 14 个锁定质量 fixtures:相对 CPU oracle 的字符相似度为 99.6484%,detection recall 为 100%,匹配框平均 IoU 为 99.5508%,匹配结果平均置信度差为 0.004349,critical failure 为 0。这些是相对 CPU 输出的 parity 指标,不是独立 ground-truth 准确率;FP16 输出也并非逐字节一致。 - -正式 warm 性能测量的 peak RSS 为 692.14 MiB,自包含 Apple 模型 payload 增加 25.42 MiB。独立的同 engine 100 个密集页生命周期测试 peak RSS 为 888.11 MiB,结束时比预热后基线低 27.47 MiB,该次测试未出现持续增长。首次使用会离线编译,并按需加载 recognition functions:固定 `HELLO 123` 启动 canary 的 compiled-cache miss 为 7.219 s,hit 为 1.275/1.278 s;113 行表单的首次整页 miss 为 53.846 s,hit 为 12.677/12.677 s。运行时不会下载 provider、编译器或模型。 - -真实设备性能数据只来自这一台 M4 Max。证据契约把它归入 `Apple M4` device family 并据此设置 `deviceValidated`,不代表每一种 M4 SKU 都做过独立测量。M1–M3 和后续 Apple Silicon 可以尝试同一 ANE/GPU 路径并报告 `deviceValidated: false`,但不继承性能承诺。`0.3.0` macOS x64 package 的 Core ML OCR 未通过发布 smoke parity,因此保持 CPU-only。完整方法、模型放置、质量阈值、缓存与生命周期结果见 [Apple 加速技术方案](docs/apple-device-acceleration.md)。 - -### Native WebGPU 加速 - -Linux 报告使用 NVIDIA RTX 5060 Ti 与 Dawn/Vulkan。14 个 fixture 的 CPU P50 总和为 5,475.623 ms,WebGPU FP32 P50 总和为 961.042 ms,得到 **5.698× 聚合加速**;每个 fixture 都更快,范围为 3.474×–9.299×。 - -Windows 报告使用 AMD Radeon 780M 与 Dawn/D3D12。CPU P50 总和为 6,500.853 ms,WebGPU FP32 P50 总和为 2,669.160 ms,得到 **2.436× 聚合加速**;每个 fixture 都更快,范围为 1.277×–2.982×。其 warmup-aware repeated lifecycle 结束时比预热后基线低 22.9 MiB。 - -两份报告均通过 164/164 Gate,覆盖 Auto、native C++、真实 placement、FP32 OCR 字节级对齐、cold start、内存、生命周期和 strict fail-closed。当前模型需要 `Concat`、`Gather`、`Slice` 三类有界 CPU partition;设置 `cpuPartition: "forbid"` 会拒绝创建 engine,不会静默改变 placement。WebGPU FP16 不属于 `0.3.0` 公共执行 profile,也不发布 FP16 性能数字。完整证据见 [Linux 加速技术方案](docs/linux-device-acceleration.md)与 [Windows 加速技术方案](docs/windows-device-acceleration.md)。 - -## 开始使用 - -### Node.js - -Node.js 22 和 24 支持 macOS arm64/x64、Linux x64 glibc 与 Windows x64: +支持 Node.js 22 和 24。 ```bash npm install @arcships/light-ocr ``` -安装会自动取得当前平台的原生运行时和固定版本的 PP-OCRv6 Small 模型;首次运行不会再下载模型,`postinstall` 也不会现场编译原生代码。0.3.0 同时支持下面的 `recognizeEncoded()` 和 raw-pixel `recognize()`。 - ```ts import { createEngine } from "@arcships/light-ocr"; import { readFile } from "node:fs/promises"; @@ -151,117 +31,67 @@ const result = await engine.recognizeEncoded( await readFile("image.jpg"), ); -// 如果宿主已有图片解码流程,仍可直接传入 raw pixels。 -const rawResult = await engine.recognize({ - data: pixels, - width, - height, - stride, - pixelFormat: "rgba8", -}); +for (const line of result.lines) { + console.log(line.text, line.confidence, line.box); +} -console.log(result.lines); -console.log(rawResult.lines); await engine.close(); ``` -已发布 package 通过平台 runtime descriptor 执行 Auto 选择;显式 Apple 与 WebGPU 都是严格的单 provider 请求。在 macOS arm64 上可以直接请求 Apple: - -```ts -const engine = await createEngine({ - execution: { - provider: "apple", - precision: "fp16", - cpuPartition: "allow", - sessionFallback: "error", - }, -}); - -console.log(engine.info.execution.sessions.detection.deviceValidated); -``` - -Linux x64 与 Windows x64 的 WebGPU profile 为: +`createEngine()` 会根据当前平台自动选择合适的执行方式。如果应用已经完成图片解码,[`recognize()`](bindings/node/README.md#使用) 也可以直接接收 `GRAY8`、`RGB8`、`BGR8` 和 `RGBA8` 像素数据。 -```ts -const engine = await createEngine({ - execution: { - provider: "webgpu", - precision: "fp32", - cpuPartition: "allow", - sessionFallback: "error", - }, -}); -``` - -`cpuPartition: "allow"` 与 strict GPU-only profile 适用于 Apple Silicon 上的 Apple provider;`0.3.0` macOS x64 package 只暴露 CPU。显式 provider 失败不会转入 CPU,只有 Auto 可以沿 descriptor 锁定的创建候选继续。不传 `execution` 的 `createEngine()` 现在使用 Auto。 - -完整 API、取消、队列限制和生命周期行为见 [Node.js 指南](bindings/node/README.md)。 - -### C++ Core - -构建需要 Python 3(仅用于 bootstrap 工具)、CMake 和支持 C++17 的编译器。依赖与模型输入均已锁定;构建后的运行时不依赖 Python。 - -```bash -python3 tools/bootstrap_dependencies.py --cache-dir .cache/dependencies -python3 tools/bootstrap_models.py --cache-dir .cache/models +## 主要能力 -cmake --preset release \ - -DLIGHT_OCR_DEPENDENCY_CACHE_DIR="$PWD/.cache/dependencies" -cmake --build --preset release --parallel -ctest --preset release -``` - -各平台的准备方式见[构建与发布](docs/build-and-release.md),接入方式见 [C++ API](docs/native-api.md)。 +- **本地处理。**图片和 OCR 结果始终留在本机。 +- **只需安装一个包。**模型和当前平台的预编译组件会随 npm 包一起安装。 +- **直接得到可用结果。**每一行都包含识别文字、置信度和原图位置。 +- **默认使用硬件加速。**Auto 在 macOS 15+ Apple Silicon 上优先使用 Core ML,在下表的 Linux 和 Windows 版本中优先使用 WebGPU。 +- **适合应用内调用。**识别任务在 JavaScript 主线程之外执行,并支持队列、取消和明确释放资源。 +- **识别大图中的小字。**可选的 `tiled` 模式可以保留高分辨率图片中的小字和密集文字。 -## 包支持 +## 平台加速 -| 分发方式 | 当前状态 | 平台 | -| --- | --- | --- | -| C++ Core 源码 | 可用 | macOS arm64/x64、Linux x64 glibc、Windows x64 | -| Node-API 适配器源码 | 可用 | Node.js 22 和 24 | -| [`@arcships/light-ocr`](https://www.npmjs.com/package/@arcships/light-ocr) | 已发布 `0.3.0` | 全部 Tier 1 平台的 Node.js 22/24 | -| [`@arcships/light-ocr-model-ppocrv6-small`](https://www.npmjs.com/package/@arcships/light-ocr-model-ppocrv6-small) | 已发布 `0.3.0` | 与平台无关的必需模型依赖 | -| 各平台 native npm packages | 已发布 `0.3.0` | macOS arm64/x64、Linux x64 glibc、Windows x64 | +npm 包提供以下四个平台版本。默认的 `createEngine()` 使用 Auto 模式: -npm 分发会安装一个统一入口、一个必需的模型包,以及与当前系统匹配的 native 包。包内容、版本策略和发布门槛见 [npm package 设计](docs/npm-packaging.md);`0.3.0` 的不可变哈希和验证证据见[发布记录](docs/releases/npm-0.3.0.md)。 +| 平台 | Auto 模式 | +| --- | --- | +| macOS / Apple Silicon | macOS 15+ 优先使用 Core ML,然后使用 CPU | +| macOS / Intel | CPU | +| Linux x64 glibc | 通过 Vulkan 使用 WebGPU,然后使用 CPU | +| Windows x64 | 通过 D3D12 使用 WebGPU,然后使用 CPU | -macOS arm64 Direct Core ML 加速已经随 `0.3.0` 发布,并复用现有六包安装结构,没有新增 provider package 或运行时下载;macOS x64 保持 CPU-only。 +需要明确控制时,可以通过 [`execution` 选项](bindings/node/README.md#使用)选择 `auto`、`cpu`、`apple` 或 `webgpu`。 -`0.3.0` 已在 Linux x64 与 Windows x64 发布 Native WebGPU。显式 WebGPU 接受 `auto/fp32`,Auto 同样选择 FP32;三个必要 CPU partition 算子会被显式报告并限制范围。两份真机报告均已通过 164/164 Gate,其报告与产物的不可变哈希已绑定进 production lock。 +## 实测性能 -## 项目状态 +0.3.0 在三台真实设备上完成了同机对比: -`light-ocr` 仍在积极开发。`0.3.0` 包含确定性的 `tiled-v1` 大图模式、Node.js 内存 JPEG/PNG 解码、descriptor-driven Auto、macOS arm64 Direct Core ML,以及 Linux x64/Windows x64 FP32 Native WebGPU 执行;C++ Core 的 raw-pixel 边界保持不变。 +![light-ocr 0.3.0 同机速度与 OCR 进程 CPU time 降幅](docs/assets/light-ocr-0.3.0-performance-v2.png) -作为 pre-1.0 项目,公共 API 和 package 布局仍可能调整;项目目前不承诺跨版本稳定的 C++ ABI。 +| 设备 | 加速方式 | 端到端加速 | OCR 进程 CPU time | +| --- | --- | ---: | ---: | +| Apple M4 Max | Core ML | `HELLO 123` **2.30×**;密集表单 **2.85×** | **降低 95.91%–97.67%** | +| Linux / NVIDIA RTX 5060 Ti | WebGPU / Vulkan | 14 张测试图片整体 **5.70×** | **降低 69.97%** | +| Windows / AMD Radeon 780M | WebGPU / D3D12 | 14 张测试图片整体 **2.44×** | **降低 46.33%** | -Core CI 当前覆盖: +以上数字均为表中设备相对本机 CPU 路径的结果,并会随任务和硬件变化。14 张图片的整体加速比,是各图片 CPU 中位耗时之和除以 WebGPU 中位耗时之和。CPU 列统计同一批任务的 OCR 进程累计 CPU time,而不是某一瞬间的系统利用率;更低的累计 CPU time 意味着 OCR 运行期间能给前台应用留出更多 CPU 资源。Apple 测试通过了锁定的 CPU 一致性门槛;两组 WebGPU 测试的 14 张图片均与 CPU FP32 输出逐字节一致。完整测量和测试方法见 [0.3.0 发布报告](docs/releases/npm-0.3.0.md)。 -- macOS arm64 -- macOS x64 -- Linux x64 glibc -- Windows x64 +## C++ -CI 还会执行 sanitizer、fuzz smoke、离线运行、输出对齐、质量和内存门槛。性能资格审查使用独立、显式触发的 workflow,不属于普通 CI 或发布预检。已验证结果和当前缺口见[实施状态](docs/implementation-status.md)。 +C++ 项目从源码构建静态库,并链接 `light_ocr::core` CMake target。API 可以直接识别解码后的 `GRAY8`、`RGB8`、`BGR8` 或 `RGBA8` 像素;接入方式见 [C++ API](docs/native-api.md),各平台准备方式见 [构建说明](docs/build-and-release.md)。 ## 文档 -- [更新日志](CHANGELOG.md) +- [Node.js API 与示例](bindings/node/README.md) - [C++ API](docs/native-api.md) -- [Node.js 适配器](bindings/node/README.md) +- [Apple Silicon 加速](docs/apple-device-acceleration.md) +- [Linux WebGPU 加速](docs/linux-device-acceleration.md) +- [Windows WebGPU 加速](docs/windows-device-acceleration.md) +- [模型包](docs/model-bundle.md) - [构建与发布](docs/build-and-release.md) -- [模型 bundle](docs/model-bundle.md) -- [准确率与输出对齐](docs/parity-testing.md) -- [大图内存表现](docs/memory-optimization.md) -- [架构](docs/architecture.md) -- [实施状态](docs/implementation-status.md) - -## 参与社区 - -欢迎提交 issue 和 pull request。如果你正在评估 light-ocr 是否适合自己的产品,可以[创建 issue](https://github.com/arcships/light-ocr/issues),告诉我们目标平台、图片来源、语言组合和大致负载。真实应用场景会直接影响 package 优先级和后续模型支持。 - -报告问题时,请尽量提供平台、输入尺寸、像素格式、模型 bundle ID 和最小复现。除非你确认可以公开,否则不要上传包含隐私信息的原始图片。 +- [更新日志](CHANGELOG.md) +- [npm 0.3.0 发布报告](docs/releases/npm-0.3.0.md) -## 开源协议 +## 社区与协议 -light-ocr 使用 [Apache License 2.0](LICENSE)。第三方依赖和模型 notice 会随对应的发布制品一起提供。 +欢迎提交 issue 和 pull request。`light-ocr` 使用 [Apache License 2.0](LICENSE)。 diff --git a/docs/assets/light-ocr-0.3.0-performance-v2.png b/docs/assets/light-ocr-0.3.0-performance-v2.png new file mode 100644 index 0000000..a88714f Binary files /dev/null and b/docs/assets/light-ocr-0.3.0-performance-v2.png differ diff --git a/docs/releases/npm-0.3.0.md b/docs/releases/npm-0.3.0.md index 0010d13..c47e18b 100644 --- a/docs/releases/npm-0.3.0.md +++ b/docs/releases/npm-0.3.0.md @@ -27,15 +27,15 @@ 所有数字都是表中设备上的同机 CPU 对照,不外推到其他设备或 driver。 -| Provider 与记录设备 | 已记录的端到端结果 | 质量与 Gate | -| --- | ---: | --- | -| Apple/Core ML,Apple M4 Max | `HELLO 123` 2.300×;XFUND 2.851× | 14 fixtures 通过锁定的 CPU parity 阈值 | -| WebGPU/Vulkan,NVIDIA RTX 5060 Ti | 14-fixture 聚合 P50 5.698×;单项 3.474×–9.299× | 14/14 与 CPU FP32 字节级一致;164/164 Gate | -| WebGPU/D3D12,AMD Radeon 780M | 14-fixture 聚合 P50 2.436×;单项 1.277×–2.982× | 14/14 与 CPU FP32 字节级一致;164/164 Gate | +| Provider 与记录设备 | 已记录的端到端结果 | OCR 进程 CPU time | 质量与 Gate | +| --- | ---: | ---: | --- | +| Apple/Core ML,Apple M4 Max | `HELLO 123` 2.300×;XFUND 2.851× | 降低 95.91% / 97.67% | 14 fixtures 通过锁定的 CPU parity 阈值 | +| WebGPU/Vulkan,NVIDIA RTX 5060 Ti | 14-fixture 聚合 P50 5.698×;单项 3.474×–9.299× | 14 fixtures 整体降低 69.97% | 14/14 与 CPU FP32 字节级一致;164/164 Gate | +| WebGPU/D3D12,AMD Radeon 780M | 14-fixture 聚合 P50 2.436×;单项 1.277×–2.982× | 14 fixtures 整体降低 46.33% | 14/14 与 CPU FP32 字节级一致;164/164 Gate | Apple 结果对比最多 12 intra-op threads 的 `cpu_fast` profile;`HELLO 123` 与 XFUND 的 warm P50 分别从 19.774/943.627 ms 降至 8.599/331.011 ms,宿主 OCR 进程 CPU time 分别降低 95.91%/97.67%。14 个 fixture 的字符相似度为 99.6484%,detection recall 为 100%,平均 matched IoU 为 99.5508%;这是 CPU parity,不是独立 ground-truth accuracy。 -WebGPU 聚合值为锁定 14-fixture corpus 的 `sum(CPU P50) / sum(WebGPU P50)`。两份报告还通过 cold start、native C++、memory、placement、strict rejection 和 repeated-lifecycle Gate;Windows lifecycle 最终比预热后基线低 22.9 MiB。 +WebGPU 聚合值为锁定 14-fixture corpus 的 `sum(CPU P50) / sum(WebGPU P50)`。CPU time 降幅按同一批标准 `cpu`/`allow` case 的 `1 - sum(WebGPU processCpuUs) / sum(CPU processCpuUs)` 计算;[Linux 报告](../../reports/webgpu-qualification/linux-x64/qualification-report.json)为 69.973%,[Windows 报告](../../reports/webgpu-qualification/windows-x64/qualification-report.json)为 46.330%。这里统计的是 OCR 进程完成同一批任务的累计 CPU time,不是瞬时系统 CPU 利用率。两份报告还通过 cold start、native C++、memory、placement、strict rejection 和 repeated-lifecycle Gate;Windows lifecycle 最终比预热后基线低 22.9 MiB。 ## 发布与验证证据