This document explains the OCR cascade — what it computes, why those specific design choices, how to tune it, where it breaks, and what the realistic accuracy ceiling looks like.
The companion document engineering_notes.md
covers practical pitfalls you only see once you implement it.
You have a glyph — either a vector outline from a font file, or a raster rendered from one — and you want to know which CJK character it draws. You also have a known reference font (Noto Sans SC or similar) and the assumption that your unknown glyph is shape-equivalent to some glyph in that reference font, just labelled with a different Unicode codepoint.
This is not OCR in the usual sense. It is nearest neighbour over a fixed reference set. Tens of thousands of references, one query, one answer. Everything below is structured around making that lookup fast and accurate.
Both the query and each reference glyph have to be converted to a fixed representation before they can be compared.
The reference impl renders each glyph to a grayscale bitmap, then crops
to the inked bounding box. The bounding box is critical: glyphs differ
wildly in extent (一 is a horizontal line; 麗 fills the em square).
Comparing them in their original frame would pick up nothing but
positional noise. Comparing them in their own bounding boxes normalises
shape and ignores position — exactly the goal.
Pseudo-code:
bitmap = render(char, font, large_canvas)
bbox = tight_crop_to_nonzero(bitmap)
working_bitmap = bitmap[bbox]
Two alternative front-ends exist if you skip bitmap rendering:
- Vector outline → scanline fill. Walk the font's path commands (M/L/Q/C/Z), flatten Béziers into line segments, do scanline fill at the grid resolutions you care about. Faster, but you have to handle curve flattening, fill rules, and CID-keyed fonts (see engineering notes). The original JS impl this repo grew out of did exactly this.
- OS-level text APIs. Render through Skia / Core Text / DirectWrite. Highest visual fidelity, most platform-dependent. Overkill here.
For correctness, all three produce equivalent answers up to a constant threshold; raster with sufficient super-sampling is the simplest, and the one used here.
The occupancy grid is a binary downsample of the bitmap. Divide the
bounding box into a GS × GS grid (12×12 by default), and for each
cell, set 1 if the cell's average ink value exceeds a threshold (0.5),
else 0.
Why binary:
- Hamming distance is a few-cycle CPU operation per word (
popcounton XOR). ForN = 30 000references andGS = 12, you're doing30 000 × 144 bits = ~4.3 Mbits = ~135 KBof XOR per query. On a modern CPU this is sub-millisecond. - Threshold smoothing destroys exactly the noise that a continuous feature wouldn't be able to filter at this resolution.
Why 12:
- Below 8, distinct characters collide too often (
日vs曰vs白). - Above 16, you're paying linearly more compute for diminishing discrimination — and density grid is going to do the fine work anyway.
- 12 hits a sweet spot: ~97% top-1 alone, which is enough to keep the correct answer in any reasonable top-K.
After computing distances, the cascade keeps the top-K survivors (default K = 40). The choice of K is a tradeoff:
- K too small → the correct answer can be filtered out by occupancy noise (rendering artefacts in stage 0 that randomly flip a cell).
- K too large → stage 2's cost grows linearly. At K = 40 the cost is small enough that you can afford to be safe.
Empirically, the correct answer is in the stage-1 top-K for ≥ 99.97% of queries at K = 40, against a 30 000-entry reference set.
For the K survivors, the cascade now runs a more discriminating
comparison. The density grid is GS × GS with GS = 16 and continuous float
values in [0, 1] representing the fraction of ink within each cell.
The reference impl computes this by area-averaging the high-resolution
bitmap (PIL's Image.Resampling.BOX); equivalently, you can compute it
by 3×3 sub-sampling within each cell during outline-based rendering.
Distance is L2 between flattened density grids. The candidate with the smallest L2 wins.
Why this works as a re-ranker but not as a primary filter:
- L2 over
16 × 16 = 256floats is~30 000 × 256 = 7.7Mmultiply-add per query if you applied it to the full reference set. Manageable, but ~50× slower than the binary cascade for marginal accuracy gain on the easy cases. - The density signal is sensitive to small shape differences that binary occupancy throws away — corner roundness, stroke width modulation, exact relative proportions of components. This is precisely what distinguishes the visually-similar pairs that survived stage 1.
Reasonable tuning knobs:
GS_fine: 12 is too coarse for fine discrimination, 16 is good, 20–24 marginally better for very confusable pairs at proportional compute cost.- Sub-sampling resolution (when computing from outlines): 3×3 per cell is the standard choice; 5×5 is overkill, 1×1 collapses to the occupancy grid.
A small number of CJK character pairs are visually almost identical at the resolutions stage 2 uses. The cascade can return the wrong member of the pair, particularly when the obfuscated source's rendering differs subtly from the reference (slightly different stroke modulation, hinting, or anti-aliasing).
The fix is a region-targeted feature: for each known confusable pair, examine a specific region of the glyph that distinguishes them.
Example: 日 vs 曰.
- The two are nearly identical at 12×12 occupancy.
曰has a slightly emptier interior (one horizontal stroke vs two).- The discriminating feature: density inside the central 50% region.
The rule encodes:
{
"primary": "日", // what the cascade returns
"alternative": "曰", // what may be preferred instead
"region": "centre", // where to measure
"threshold": 0.42, // ink density in that region
"prefer_alternative_when_below": true
}
See ../obfuscated_cjk_ocr/confusables.py
for the implementation and
../examples/confusables_example.json
for the schema. The example file ships a few illustrative pairs; in
practice new rules come from running the benchmark, looking at the top
confusions, and inspecting each pair manually.
This stage gets you from ~99.7% to ~99.97% on a 5 000-character random sample. The rules themselves are font-family-specific — what works on Noto Sans SC won't necessarily transfer to Source Han Serif because the discriminating features live at different positions.
For one query, against a reference set of 30 000 characters:
| Stage | What it does | Operations | Wall time (rough) |
|---|---|---|---|
| 0 | Render + tight crop | One render | ~2 ms |
| 1 | Hamming, 144 bits × 30k refs | ~4.3M bit ops | <1 ms |
| 2 | L2, 256 floats × 40 refs | 10K float ops | <1 ms |
| 3 | Region density on ~5% of queries | tiny | <1 ms |
End-to-end: 2–4 ms per query on a modern laptop. The hot loop is fully vectorisable with NumPy or SIMD intrinsics if you need to go faster.
Building the reference table is one-time and proportional:
30 000 × (render + grids) ≈ 30 000 × 2 ms = ~60 s.
On synthetic data (query font == reference font) the cascade hits ~99.99% top-1, with errors concentrated entirely in confusables.
On near-synthetic data (query and reference are different weights of the same family, e.g. Noto Sans SC Regular vs Medium), accuracy drops to ~99.5% — the stroke-width difference shifts density values uniformly upward.
On unrelated families (Noto Sans SC vs PingFang SC), accuracy is around
~95–98% depending on how similar the shape designs are. This is the
regime where the right question is "do I actually have the right
reference font?", not "how do I tune the cascade?". The
find_closest_font routine
addresses exactly that question.
Hard failures (the cascade is fundamentally wrong, not just close):
- Glyphs that the reference font doesn't contain at all. Catch this
by checking
min_distance > thresholdand falling back to a secondary recogniser (or a human-in-the-loop step). - Heavily stylised display fonts where the reference's shapes are unrelated. No amount of cascade tuning fixes this.
- Glyphs with hand-drawn or animated rendering (e.g. SVG fonts with random stroke variation). The technique assumes deterministic rendering.
The cascade structure is not specific to CJK obfuscated fonts. The
same coarse binary filter → fine continuous re-rank pattern works
for any nearest-neighbour problem over shape descriptors:
- Latin / Cyrillic / Arabic glyph identification, especially for scripts with many visually-similar letters.
- Logo / icon classification against a known reference set.
- Map symbol recognition (cartographic symbology has the same "small known vocabulary" structure).
- Sign language fingerspelling against a reference handshape library.
The features (occupancy + density) are deliberately script-agnostic — they describe where ink lives in a bounding box, nothing more. For finer scripts (Latin sans-serif) you may want a slightly higher fine resolution; for very intricate scripts (Tibetan, Devanagari combinations) you may want directional gradient features in addition. The cascade shape stays the same.