Skip to content

fix: pro OCR ROI calibration and 24-word mnemonic pipeline fixes#32

Merged
0xchloe996 merged 2 commits into
mainfrom
fix/pro-ocr-roi-and-mnemonic-pipeline
Jul 17, 2026
Merged

fix: pro OCR ROI calibration and 24-word mnemonic pipeline fixes#32
0xchloe996 merged 2 commits into
mainfrom
fix/pro-ocr-roi-and-mnemonic-pipeline

Conversation

@wabicai

@wabicai wabicai commented Jul 16, 2026

Copy link
Copy Markdown
Member

Summary

Device-calibrated OCR scenes for pro plus a chain of recognition/merge fixes for the 24-word create-wallet flow. Root causes were found by calibrating against saved captures and full-pipeline reproduction; final state: 12/18/24-word flows recognize all words correctly and the merged 24-word phrase validates against the BIP39 checksum.

OCR scene calibration (pro)

  • Per-layout ROIs measured from real captures: mnemonic 12 (225,690 665x580), 18 (225,580 665x830), 24 part1 (225,695 665x715, stops above the 继续 button), 24 part2 (225,520 665x890, full list viewport), verify-number (160,530 520x170)
  • Scene selection is word-count and part-aware (allowPartial picks part1/part2); pro2 behavior unchanged
  • Calibrated scenes use base + horizontal-only expansion candidates (vertical expansion re-included noise the calibration deliberately excluded and blurred >1280px crops lexicon-snapped to wrong words, e.g. gym→game)

Recognition pipeline (paddleocr_en_infer.py)

  • Blue-channel input for mnemonic layouts: screen text is cyan, warm-light glare lives in R/G — recovers words washed out by reflections (plain grayscale does not)
  • Per-column row recognition: whole-row crops made the rec model glue the left word to the right index ("gym13. stove") which broke renderer index parsing
  • Cross-row column-anchor filtering drops glare det boxes; viewport-clipped scroll rows are dropped (their half-glyphs parsed as phantom indexed words)
  • scriptVersion in the server ready message, logged by the main process to make stale-server debugging obvious

Renderer fixes (CameraPanel)

  • Wordlist decoupled from requireBip39: 24-word part1 (requireBip39: false) was treated as a SLIP39 flow and every valid BIP39 word got snapped onto the SLIP39 wordlist (skull→skunk, young→round…) — the root cause of the recurring wrong words at indices 1/2/13/14
  • Isolated phantom-index guard before part merge: clipped scroll rows parsed as e.g. "1. ens" and overwrote the correct stored word 1
  • Letter→digit seam re-splitting before index parsing; removed the dead 12-word column-row fallback; diagnostic logs for capture consensus and final merged validation

Sequences

  • 500ms delayBefore PIN entry (both first entry and re-confirm); ControlPanel executor now honors delayBefore (previously only the MCP executor did)

Assets

  • PP-OCRv6 small det/rec onnx models (runtime default) and BIP39/SLIP39 wordlists

Test plan

  • Reproduced every historical failing capture through the full pipeline (12w glare, 18w row-9 clipping, 24w part1/part2 across four separate runs): all words correct
  • 24-word merged phrase validates with @scure/bip39 + english wordlist
  • MCP flow consistency: all sequence IDs referenced by hardware-js-sdk automation (/health sequenceIds drift check) still present
  • tsc --noEmit clean; python py_compile clean

🤖 Generated with Claude Code

@socket-security

socket-security Bot commented Jul 16, 2026

Copy link
Copy Markdown

Review the following changes in direct dependencies. Learn more about Socket for GitHub.

Diff Package Supply Chain
Security
Vulnerability Quality Maintenance License
Addedpypi/​onnxruntime@​1.27.074100100100100

View full report

0xchloe996 and others added 2 commits July 17, 2026 12:38
Device-calibrated OCR scenes for pro (mnemonic 12/18/24 part1/part2 and
verify-number ROIs measured from real captures), plus a chain of recognition
and merge fixes for the 24-word create-wallet flow:

- deviceScenes: per-layout pro ROIs; scene selection is word-count and
  part-aware (allowPartial picks part1/part2)
- CameraPanel: wordlist choice decoupled from requireBip39 (20/33-word
  shares are SLIP39, everything else BIP39) — part1 words were being
  snapped onto the SLIP39 wordlist (skull->skunk)
- CameraPanel: drop isolated phantom indices before part merge (viewport-
  clipped scroll rows parsed as e.g. "1. ens" and overwrote correct words)
- CameraPanel: calibrated scenes use base + horizontal-only expansion
  candidates; removed dead 12-word column-row fallback
- paddleocr_en_infer: blue-channel input for mnemonic layouts (suppresses
  warm-light glare), per-column row recognition, cross-row column-anchor
  glare filtering, clipped-row dropping, script version marker
- sequences: 500ms delayBefore PIN entry; ControlPanel executor now honors
  delayBefore (previously MCP executor only)
- ship PP-OCRv6 small onnx models and BIP39/SLIP39 wordlists

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Optional higher-accuracy models selectable via QA_AUTO_HW_OCR_MODEL_DIR /
QA_AUTO_HW_OCR_DET_MODEL_DIR; the runtime default remains the small models.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@wabicai
wabicai force-pushed the fix/pro-ocr-roi-and-mnemonic-pipeline branch from 406bd3f to c1c157c Compare July 17, 2026 04:39
@0xchloe996
0xchloe996 merged commit 9bbe421 into main Jul 17, 2026
5 checks passed
@0xchloe996
0xchloe996 deleted the fix/pro-ocr-roi-and-mnemonic-pipeline branch July 17, 2026 04:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants