Skip to content

feat(pdf): make PaddleOCR-VL a getPDFReader provider - #146

Closed
chadtalsma-svg wants to merge 3 commits into
happyvertical:mainfrom
chadtalsma-svg:claude/paddleocr-reader-145
Closed

chadtalsma-svg wants to merge 3 commits into
happyvertical:mainfrom
chadtalsma-svg:claude/paddleocr-reader-145

Conversation

@chadtalsma-svg

Copy link
Copy Markdown
Contributor

Makes PaddleOCR-VL a full getPDFReader backend, interchangeable with unpdf and kreuzberg, so code written against the reader interface can switch to it by name. For example, @happyvertical/smrt-ingestion builds its reader in a child process from { provider }.

getPDFReader({ provider: 'paddleocr' }) (or HAVE_PDF_PROVIDER=paddleocr) returns a reader that splits the work as follows:

Operation Served by
extractText PaddleOCR-VL /layout-parsing, returning Markdown with tables as HTML. pages selects from the parsed pages.
performOCR(images) PaddleOCR-VL, one image per request (fileType: 1). WebP and raw pixels are converted to PNG first.
getInfo, extractMetadata, extractImages, renderPages Local unpdf. The service doesn't return any of these.

Rules:

  • Explicit only. auto never selects it, and it runs on Node only. A missing or invalid service URL fails at getPDFReader(), not on first use.
  • No fallback to another engine. It never retries and never falls back to local OCR. Any ocrProvider other than auto is refused, including tesseract.
  • Page count is checked. If the service returns a different page count from the local one, it throws EPADDLERESPONSE rather than passing the gap off as blank pages. A PDF unpdf can't open (0 local pages) skips the check.
  • Local-only modes. preferEmbeddedText (option, or HAVE_PDF_PADDLEOCR_PREFER_EMBEDDED_TEXT) reads PDFs that already have a text layer locally. skipOCRFallback and enableOCR: false read only the local text layer and send nothing.
  • Limits. paddleocr.maxFileSize falls back to the reader's maxFileSize. The reader's timeout is deliberately not inherited, because a limit sized for local parsing would cut off a cold start.
  • Client. PaddleOCRClient gains parseImage() for PNG and JPEG. parse() behaves as before.
  • Bundle. src/index.ts exports only the options type, and the provider loads lazily through the factory.

This fits the rule "Keep OCR behavior behind @happyvertical/ocr": the PR adds no OCR provider logic. It routes the reader's OCR step to the document-parsing service this package already integrates (#142), and the local OCR providers are untouched.

Validation: pnpm lint (5 warnings, all already on main), docs:api:check, typecheck and build all pass. pnpm test passes 215 tests with 2 skipped; the 18 new contract tests run against a local HTTP stub. Live GPU parsing is checked downstream in teamworksfab/teamworks-os.

Independent review: Claude Opus 5.5, standard risk, three passes.

  • Pass 1 found three should-fix issues and two nits, all fixed in 3e3055e: tesseract was accepted, WebP couldn't be OCR'd, the reader timeout was inherited, a page-count mismatch passed silently, and enableOCR: false was ignored.
  • Pass 2 found that the page-count check rejected PDFs unpdf can't open, plus two WebP nits, all fixed in 7d9b5cf.
  • Pass 3 was CLEAN.

Closes #145

🤖 Generated with Claude Code

chadtalsma-svg and others added 3 commits October 16, 2026 07:03
provider: 'paddleocr' returns a complete PDFReader: extractText and
performOCR go to the PaddleOCR-VL /layout-parsing service; getInfo,
metadata, embedded images and page renders run locally through unpdf.
Explicit only (auto never selects it), no retry and no local OCR
fallback. PaddleOCRClient gains parseImage() for PNG and JPEG.

Closes happyvertical#145

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Refuse every named OCR engine (tesseract included), convert WebP to PNG
before OCR, stop inheriting the reader timeout, throw when the service
returns a different page count, and honour enableOCR: false.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…rtical#145)

A damaged or encrypted PDF probes as 0 pages locally but may still parse
remotely. Also match WebP by canonical format and report an undecodable
WebP as EINVAL.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@willgriffin

Copy link
Copy Markdown
Contributor

Reviewed and integrated with original authorship into #148. Fixed the per-image response contract: zero or multiple returned pages now fail instead of being accepted as OCR output. Also replaced the broken pnpm pin under #147. All 218 tests and packed-consumer smoke checks pass; independent review is CLEAN. #148 is running normal same-repository CI and will supersede this PR after merge and release.

@willgriffin

Copy link
Copy Markdown
Contributor

Released implementation is integrated through #148, which preserved your original commits and passed PR and merge-queue CI. The on-main Publish workflow is now running; closing this superseded source PR.

@chadtalsma-svg
chadtalsma-svg deleted the claude/paddleocr-reader-145 branch October 28, 2026 16:52
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Make PaddleOCR-VL a getPDFReader provider (provider: 'paddleocr')

2 participants