Return Unknown for unassigned codepoints instead of raising - #72
Open
nagieeb0 wants to merge 1 commit into
Open
Conversation
Scripts.txt only lists assigned codepoints, and script_from_codepoint/1 generates one clause per range with no fallback. Any codepoint outside the bundled data raises FunctionClauseError, so a caller that merely passed along a recent character crashes rather than degrading. U+20C1 SAUDI RIYAL SIGN (Unicode 16.0) is a common way to hit this: it takes down OpenType.Layout.detect_script/1 for any string containing a Saudi Riyal amount. Adds the UCD's documented @missing default (Unknown) as a final clause. That value is already handled everywhere it lands — script_to_tag/1 maps it to "zzzz", and detect_script/1 filters it out with Common and Inherited — so nothing downstream needs to change. Also refreshes Scripts.txt from Unicode 12.0 (2019) to 17.0, which gives codepoints added since then their real script rather than Unknown. U+20C1 becomes Common, as it should be.
nagieeb0
added a commit
to nagieeb0/qalam
that referenced
this pull request
Jul 30, 2026
A brand face rarely covers everything, and what it misses is usually a currency sign or a second script sitting mid-sentence. Before this, such a codepoint rendered as an empty .notdef box with no way to fix it short of editing the font. Qalam.line/4 now takes a list of fonts. Runs are split again wherever the covering face changes, so no run spans two faces, and each carries the index of the face that should draw it. Coverage is a cmap lookup, not a shaping pass. Measurement follows the same split, so widths still match what lands on the page. A codepoint no face covers stays on the primary and renders as a visible box — silently dropping text is the worse failure. Also documents the unicode_data override needed until jbowtie/unicode_data#72 lands. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Scripts.txtonly lists assigned codepoints, andscript_from_codepoint/1generates one clause per range with no fallback. Any codepoint outside the bundled data raisesFunctionClauseError, so a caller that merely passed along a recent character crashes rather than degrading.U+20C1 SAUDI RIYAL SIGN (Unicode 16.0) is a common way to hit this — it takes down
OpenType.Layout.detect_script/1for any string containing a Saudi Riyal amount:Changes
A final clause returning
"Unknown"— the UCD's documented@missingdefault for Script. That value is already handled everywhere it lands:script_to_tag/1maps it to"zzzz", anddetect_script/1filters it out alongsideCommonandInherited. Nothing downstream needs to change.Scripts.txtrefreshed from Unicode 12.0 (2019) to 17.0, so codepoints added since then get their real script rather thanUnknown. U+20C1 becomesCommon.Verification
Existing suite passes unchanged, plus two new cases covering U+20C1 and the unassigned fallback.
Note:
mix testneededexcoverallsdropped locally to run at all — the pinned version fails to compile on OTP 28. That is out of scope here, but happy to send a separate PR modernising the dev deps if useful.🤖 Generated with Claude Code