Skip to content

feat: add encoded JPEG/PNG image input - #2

Merged
eric8810 merged 4 commits into
arcships:mainfrom
luojiyin1987:feat/encoded-image-input
Jul 14, 2026
Merged

eric8810 merged 4 commits into
arcships:mainfrom
luojiyin1987:feat/encoded-image-input

Conversation

@luojiyin1987

@luojiyin1987 luojiyin1987 commented Jul 14, 2026

Copy link
Copy Markdown
Contributor

Linked issue

Closes #1

What

Adds recognizeEncoded() to the Node.js adapter so callers can pass in-memory JPEG or PNG bytes directly instead of decoding images into raw pixels first.

The native C++ Core boundary remains unchanged and continues to accept decoded pixel buffers only.

Changes

Encoded image decoder

Adds bindings/node/src/encoded_image.{hpp,cpp} using a pinned stb_image revision with:

  • JPEG and PNG support only
  • file and stdio APIs disabled
  • internal symbols isolated with STB_IMAGE_STATIC
  • request-scoped memory accounting through custom malloc, realloc, and free
  • dimension checks against maxWidth, maxHeight, and maxPixels
  • decoder and overlapping output-buffer checks against maxTemporaryBytes
  • allocation failures mapped to resource_limit_exceeded

Decoded images are emitted as RGB8 and passed into the existing Core raw-pixel API, which performs the existing RGB-to-BGR conversion.

Node.js adapter

Adds:

engine.recognizeEncoded(
  data: Uint8Array,
  options?: RecognizeOptions,
): Promise<OcrResult>

Encoded requests reuse the existing:

  • per-engine FIFO worker
  • bounded admission controls
  • snapshot ownership semantics
  • AbortSignal cancellation behavior
  • Promise completion and error mapping

The JavaScript thread validates and snapshots the encoded byte view but does not parse or decode the image. Format, dimensions, pixel count, and decoder memory limits are validated on the engine worker thread.

Because encoded metadata is validated after admission, a malformed or oversized encoded image may return queue_full when the engine is already at capacity. This is an intentional API contract that avoids synchronous and duplicated image parsing on the JavaScript thread.

Timing

Adds:

timingUs.decode

For raw-pixel input, decode is 0.

For encoded input, decoder duration is reported in microseconds and included in timingUs.total.

Build and release metadata

  • Adds pinned stb dependency metadata and cache support.
  • Fetches and configures stb only when the Node adapter or fuzzers are enabled.
  • Keeps Core-only builds independent of the decoder dependency.
  • Includes the stb license and SBOM entry only in builds that contain the decoder.
  • Adds the encoded image decoder to the fuzzing targets.

Documentation

Updates:

  • root README files
  • Node.js adapter README
  • Node-API design documentation
  • build and dependency documentation
  • npm packaging and release documentation
  • architectural decisions describing the unchanged raw-pixel Core boundary

Supported formats

Supported:

  • JPEG
  • PNG

Not supported:

  • WebP
  • GIF
  • PDF
  • automatic EXIF orientation handling
  • filesystem paths or decoder file I/O

Safety and resource limits

The decoder:

  • rejects empty, detached, non-Uint8Array, and SharedArrayBuffer-backed inputs
  • rejects unsupported or malformed image data
  • checks dimensions before full decoding
  • restricts stb internal allocations with a request-level allocator
  • accounts for the simultaneous lifetime of the stb output and copied RGB buffer
  • maps decoder allocation and system allocation failures to resource_limit_exceeded
  • supports concurrent decoding across independent engine instances without a global decoder mutex

The encoded-byte snapshot remains subject to maxPendingInputBytes. Decoder memory and Core inference memory are independently constrained by maxTemporaryBytes.

Tests

Adds coverage for:

  • JPEG decoding
  • PNG decoding
  • input snapshot semantics
  • malformed and unsupported inputs
  • empty input
  • invalid typed arrays
  • detached and shared backing stores
  • oversized dimensions
  • decoder memory-budget enforcement
  • encoded input admission and queue_full priority
  • concurrent decoding across independent engines
  • raw-input timingUs.decode === 0
  • encoded decoder fuzzing

Test plan

node --test bindings/node/test/adapter.test.cjs
ctest --test-dir build-node -R light_ocr_node_tests --output-on-failure

Also validate:

  • Core-only build with Node and fuzzers disabled
  • Node addon builds on Linux, macOS, and Windows
  • encoded image fuzz target
  • ASan/UBSan
  • concurrent decoder execution under TSan

@luojiyin1987
luojiyin1987 force-pushed the feat/encoded-image-input branch from a6dde57 to 183bf43 Compare July 14, 2026 10:41
@luojiyin1987
luojiyin1987 marked this pull request as draft July 14, 2026 10:58
@luojiyin1987
luojiyin1987 marked this pull request as ready for review July 14, 2026 11:04
@eric8810

Copy link
Copy Markdown
Contributor

Looks good overall! Could we add one non-blank JPEG/PNG test comparing recognizeEncoded() with raw recognize()? This would catch channel-order or decoding regressions that the blank fixtures may miss. Once CI is green, this looks ready to merge.

@luojiyin1987

Copy link
Copy Markdown
Contributor Author

add non-blank JPEG/PNG test , @eric8810

@eric8810
eric8810 merged commit d33c65d into arcships:main Jul 14, 2026
6 checks passed
@luojiyin1987
luojiyin1987 deleted the feat/encoded-image-input branch July 14, 2026 16:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

feat: support encoded JPEG/PNG image input

2 participants