Skip to content

chore(i18n): gate unreferenced catalog keys in the extraction check and retire 1,644 of them - #22056

Merged
OrcaWin merged 30 commits into
stablyai:mainfrom
smwbev:chore/orphaned-catalog-key-detector
Oct 28, 2026
Merged

OrcaWin merged 30 commits into
stablyai:mainfrom
smwbev:chore/orphaned-catalog-key-detector

Conversation

@smwbev

@smwbev smwbev commented Sep 21, 2026 •

Copy link
Copy Markdown
Contributor

Summary

This PR combines #22056 (the detector) and #19801 (the prune and the gate). It supersedes #19801. Thanks to @smwbev for the design, the plural-base rule and the investigation behind both PRs.

  • The detector is now part of the extraction check. verify-localization-extraction.mjs already computes the set of English keys that extraction misses. It now narrows that set to keys that no source names, using its own extraction pass. There is no second i18next run, no separate script and no new package.json entry. A key stays live when any of these is true:

    • extraction references it;
    • a key-shaped string literal under src/ or mobile/ names it (tests excluded). This covers search-keyword maps, title-copy maps, labelKey tables, the plugin chrome allowlist and translateMain;
    • it is a plural variant (_zero/_one/_two/_few/_many/_other) and its base key is live;
    • it sits under a prefix that a t/translate* call builds at runtime, such as translate(`…AutomationRunsDashboard.status.${key}`).

    The source walk reuses collectSourceFiles from verify-localization-catalog.mjs.

  • The prune was recomputed on current main, not replayed. 1,644 keys come out of all 7 catalogs. en-runtime-required.json was regenerated: 676 entries, all removals are pruned keys, nothing was added. 285 stale entries come out of the override tables: 260 in locale-ko-key-overrides.json, 18 in locale-key-overrides.mjs, 5 in locale-cross-locale-key-overrides.mjs and 2 in locale-ja-key-overrides.mjs.

  • The check now blocks. After the prune it reports 0 keys on main, so verify:localization-extraction fails on any key no source names. I tested this by planting a stale key and a _one/_other pair: the check reported all three and exited 1. The fallback-drift failure and the existing checks are unchanged.

  • Tests: I removed the 27-case standalone suite. Two cases in verify-localization-extraction.test.mjs cover the real edge cases: plural variants and keys held in data structures, and dynamic prefixes in translate calls (a non-translate template is ignored).

Verification of the prune

  • No pruned key appears as a quoted literal anywhere in src/ or mobile/, tests included.
  • I scanned every template head and concatenation head in the source for dynamic composition. The only heads that match pruned keys are RPC method names (browser., jira.), not i18n keys.
  • I scanned template tails as well. Every match was a git config name or a file name.
  • I hand-checked 30 sampled non-hash keys against their components. For example, settings.browser.sshWorkspaceRouting.title is now rowTitle, and AiVaultSessionDetails.session is now sessionRole.

Testing

  • pnpm verify:localization-extraction, verify:localization-catalogs, verify:localization-catalog, verify:localization-coverage and verify:localization-runtime-catalog all pass.
  • pnpm test src/renderer/src/i18n plus all config/scripts/*locali* and *locale* tests plus every test that loads catalogs: 89 files, 1,161 tests, all passing.
  • pnpm check:changed passed all 21 checks. pnpm tc:web passes.

Note: mobile/ is in the literal scan but not in the extraction check's change scope. Mobile does not read the desktop catalog today, so a key named only from mobile would be unusual.

A key can leave the code without leaving the catalog: a feature is reverted, a
setting is rewritten in place, a component is deleted. Nothing fails, so the
entries stay. Every locale carries them, translators spend effort on strings no
screen renders, and coverage numbers understate how much is actually translated.

`verify:localization-extraction` already reports an unreferenced set, but it
cannot be deleted as-is: i18next-cli does not see a key held in a data structure
(`{ key: '…' }`) or passed as a variable, and 18 files do exactly that. So this
requires two independent conditions — extraction misses the key AND the exact
string appears in no source literal under src/ or mobile/.

Two properties matter and both were found the hard way:

- Plural variants are judged by their base key. i18next appends the CLDR
  category at runtime, so `…_one` never appears in a source file even though the
  base does. Without this the scan eats every plural in the catalog.
- The scan is iterative. Removing a base key can strand the variants that
  survived on its account, so it must be re-run until it reports none.

Registered as `verify:localization-orphans` but deliberately left out of the
lint chain: it reports 1,401 keys on current main, so gating now would fail CI.
The prune and the gate come together in a follow-up.
@coderabbitai

coderabbitai Bot commented Sep 21, 2026 •

Copy link
Copy Markdown
Contributor

Review in Change Stack →

Note

Reviews paused

It looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the reviews.auto_review.auto_pause_after_reviewed_commits setting.

Use the following commands to manage reviews:

  • @coderabbitai resume to resume automatic reviews.
  • @coderabbitai review to trigger a single review.

Use the checkboxes below for quick actions:

  • ▶️ Resume reviews
  • 🔍 Trigger review

No actionable comments were generated in the recent review.

ℹ️ Recent review info
⚙️ Run configuration
  • Configuration used: Repository UI
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: 0a770b89-af2a-4fb2-a757-67f76f397c51
📥 Commits

Reviewing files that changed from the base of the PR and between 3fd7ccd and c04d84a.

📒 Files selected for processing: 2
  • config/scripts/verify-localization-orphans.mjs
  • config/scripts/verify-localization-orphans.test.mjs
🚧 Files skipped from review as they are similar to previous changes: 1
  • config/scripts/verify-localization-orphans.test.mjs

Included review availability: This review used your included allowance. Your plan provides up to 10 included reviews per hour; 8 remain after this review.

Walkthrough

The change adds a script that compares English catalog keys with keys found by i18next extraction and qualifying literals in runtime source files. It handles plural-category keys through their base keys, reports orphaned keys, and sets a failing exit code in check mode or when extraction fails. Tests cover literal collection, source discovery, and reference matching. A package script runs the verifier in check mode.

Priority: Low

Merge Risk: Low · previous review c04d8

The verifier can overlook a catalog key if it appears only in a source comment. No affected key was demonstrated in the current scan, and the check is not yet a CI gate, so this remains a bounded tooling-accuracy risk rather than an application runtime issue.

Pre-merge checks: Passed 4
Check name Status Explanation
Linked Issues check Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check Passed Check skipped because no linked issues were found for this pull request.
Title check Passed The title names the main change: gating unreferenced i18n catalog keys in the localization extraction check and pruning 1,644 of them. It is specific and scoped to the changeset, and it uses the conve…
Description check Passed The description follows the template's main sections: Summary, Verification, and Testing. It explains what changed and why it supersedes #19801, and it gives concrete evidence for the prune and the bl…
Full details: Title check

Explanation

The title names the main change: gating unreferenced i18n catalog keys in the localization extraction check and pruning 1,644 of them. It is specific and scoped to the changeset, and it uses the conventional chore(i18n) prefix. The raw summary describes only a new orphan-detector script, so the title covers more than that script, but the PR description confirms the prune and the blocking gate are real parts of this PR.

Full details: Description check

Explanation

The description follows the template's main sections: Summary, Verification, and Testing. It explains what changed and why it supersedes #19801, and it gives concrete evidence for the prune and the blocking gate. Several template sections are absent or left unfilled: ELI5, Why (beyond the supersession note), Linked Issue (the 'Fixes #' line is blank), Visual Proof (no N/A statement), the Testing checkbox list, AI Disclosure, the Agent skill upstream boundary checkbox, and the Checklist. The Linked Issue section is required for outside contributors, and the author is not identified as a maintainer. The description is mostly complete on substance but misses several template sections.

Resolution

Consider completing these template items: add an ELI5 summary; state the before and after user-visible behavior under What Changed; fill in the Linked Issue (Fixes #) or confirm the author is a maintainer; add an N/A reason under Visual Proof; tick the Testing checkboxes; complete the Agent skill upstream boundary checkbox; and complete the Checklist items.

  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2


ℹ️ Review info
⚙️ Run configuration

Configuration used: Repository UI

Review profile: CHILL

Plan: Advanced

Run ID: 69ff5c3e-99b6-4ebe-aaef-4fcd95091020

📥 Commits

Reviewing files that changed from the base of the PR and between 5e25c7a and 3fd8aed.

📒 Files selected for processing (2)
  • config/scripts/find-orphaned-catalog-keys.mjs
  • package.json

Included review availability: Your plan provides up to 10 included reviews per hour; 8 remain after this review.

@pullfrog pullfrog Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ℹ️ Minor suggestions only — I ran the detector and could not find a correctness problem; two cleanup items inline.

Reviewed changes

  • New orphan detector — config/scripts/find-orphaned-catalog-keys.mjs reports keys that i18next extraction misses and that appear as no quoted literal under src/mobile; --check exits 1 with remediation text.
  • npm script — adds verify:localization-orphans, deliberately left out of lint/pr.yml until the follow-up prune lands.

I verified the script end to end: it reports 14,459 en.json keys / 12,887 extraction-referenced / 1,401 orphans (matching the PR body), exits 1 under --check, and leaves no working-tree side effects. A repo-wide exact-token scan of the 1,401 orphans found no runtime consumers outside src/mobile; the only non-source hits are localization tooling (see the second inline note). The plural-base rule and the ${NS}.status… template-literal case both behave as described.

ℹ️ Nitpicks

  • The two non-obvious invariants (plural base-key rule, literal scan) have no regression test, and the author's stated safety argument (the follow-up deletes the list and keeps suites green) validates the current list, not the logic. Factoring isReferenced/the literal scan into exported pure functions and unit-testing the plural and data-structure cases would guard future edits; optional given the follow-up plan.

Pullfrog  | Fix all ➔ | Fix 👍s ➔ | View workflow run | Using DeepSeek Flash (free via Pullfrog for OSS) | 𝕏

Comment thread package.json Resolved
Both rules decide whether a key is live, and both are easy to break without
noticing: the plural rule keeps `…count_one` when only `…count` appears in
source, and the literal scan is the sole evidence for keys extraction cannot
see — held in a data structure or passed as a variable.

Extracting them as pure functions changes no behaviour: the detector still
reports 14,459 keys, 12,887 referenced, 1,401 orphans, and --check still
exits 1.
@smwbev

smwbev commented Sep 26, 2026

Copy link
Copy Markdown
Contributor Author

Took the nitpick — both invariants now have tests.

isKeyReferenced and collectLiteralKeys are exported pure functions, covered in config/scripts/find-orphaned-catalog-keys.test.mjs (21 cases). The plural rule is tested per CLDR category and in both failing directions: a variant whose base key only the catalog carries stays live, one whose base is itself unreachable does not, and a key ending in a non-CLDR word gets no base-key fallback. The literal scan is tested on the data-structure case, all three quoting forms, and the non-key literals the dot requirement is there to reject.

Behaviour is unchanged — the detector still reports 14,459 keys / 12,887 referenced / 1,401 orphans, and --check still exits 1.

@pullfrog pullfrog Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ No new issues found.

Reviewed changes

  • Exported the two pure helpers — collectLiteralKeys and isKeyReferenced are lifted out of collectLiteralStrings/findOrphanedKeys and exported so tests can drive them directly; the regex, plural rule, and call sites are otherwise unchanged.
  • Added find-orphaned-catalog-keys.test.mjs — 21 vitest cases covering the plural-base rule across all six CLDR categories, the literal shape filter and its three quoting forms, cross-call accumulation into a shared set, and the no-base fallback for non-CLDR suffixes. The suite is included by config/vitest.config.ts (config/scripts/**/*.test.mjs), and it is failable: dropping english.has(base) from isKeyReferenced turns the English-catalog case red, and removing the base-key fallback turns the plural cases red.

Pullfrog  | Fix it ➔ | View workflow run | Using DeepSeek Flash (free via Pullfrog for OSS) | 𝕏

main gained the anti-slop rule no-shape-in-symbol-names after this branch
was cut, and KEY_SHAPED trips it once the branch is merged with main.
Renamed to CATALOG_KEY_PATTERN; no behaviour change.
@smwbev

smwbev commented Oct 10, 2026

Copy link
Copy Markdown
Contributor Author

Pushed one mechanical commit, 5ae966d347: the constant KEY_SHAPED is now CATALOG_KEY_PATTERN. main gained the anti-slop rule no-shape-in-symbol-names after this branch was cut, and the old name trips it as soon as the branch meets main — I hit that while refreshing #19801, which is stacked on this one. No behaviour change; the 21 unit tests pass unchanged.

smwbev added a commit to smwbev/orca that referenced this pull request Oct 10, 2026
Brings in the exported helpers, the unit tests and the constant rename the
detector gained in its own PR (stablyai#22056), so the script here matches the one
this branch is stacked on.
…ifiers

CODEOWNERS routes /config/scripts/*localization*.mjs to the localization owner, and find-orphaned-catalog-keys.mjs matched neither that pattern nor *locale*, so a change to it asked nobody for review. The file now carries the name of the npm script that runs it, verify:localization-orphans.
…base key

isKeyReferenced kept a plural variant alive whenever en.json still carried its base key, even when no source reached that base key. The base key was reported and the variants were not, so a prune needed a second run to find them. Only a base key that extraction or a source literal references keeps its variants now; the --check message drops the re-run advice and names the override tables, which sit outside the scan roots.
The scan captured only literals of seven characters or more and then applied CATALOG_KEY_PATTERN, which has no such floor. A live key shorter than that, held in a data structure, would have been reported as an orphan. en.json has no key that short today, so the result is unchanged; the capture now leaves the shape decision to the one pattern.
@smwbev
smwbev requested a review from brennanb2025 as a code owner October 14, 2026 00:59

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1


ℹ️ Review info
⚙️ Run configuration
  • Configuration used: Repository UI
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: fa775867-4647-490f-8bdb-5bb6511dd850
📥 Commits

Reviewing files that changed from the base of the PR and between 5ae966d and 3fd7ccd.

📒 Files selected for processing: 3
  • config/scripts/verify-localization-orphans.mjs
  • config/scripts/verify-localization-orphans.test.mjs
  • package.json

Included review availability: This review used your included allowance. Your plan provides up to 10 included reviews per hour; 9 remain after this review.

collectSourceFiles swallowed every readdir error, so a directory it could not read contributed no literals and a key named only there would have been reported as safe to delete. Only a missing mobile/ root is skipped now, since a checkout without the mobile app is still a complete desktop scan; any other error stops the scan.
smwbev and others added 10 commits October 21, 2026 06:08
…nd retire 1,644 unreferenced keys

verify-localization-extraction.mjs now filters its own unreferenced set by
source literals (src/ and mobile/), plural base keys and dynamic prefixes
composed inside translate calls, and fails when any key is named nowhere.
That replaces the standalone verify-localization-orphans.mjs script and its
second extraction pass.

The prune is recomputed on current main: 1,644 keys leave all seven locale
catalogs, 285 stale entries leave the key-override tables, and
en-runtime-required.json is regenerated.

Supersedes stablyai#19801.

Co-authored-by: Evgenii <413011+smwbev@users.noreply.github.com>
@OrcaWin OrcaWin changed the title chore(i18n): add a detector for catalog keys no source references chore(i18n): gate unreferenced catalog keys in the extraction check and retire 1,644 of them Oct 28, 2026
OrcaWin and others added 2 commits October 27, 2026 20:07
…ynamic prefix

mobile/ edits do not trigger the extraction check, so literals there must not
decide its verdict; a dotless runtime-built prefix would keep unrelated keys alive.

Co-authored-by: Evgenii <413011+smwbev@users.noreply.github.com>
@OrcaWin
OrcaWin merged commit 76d4b01 into stablyai:main Oct 28, 2026
34 checks passed
smwbev added a commit to smwbev/orca-russian that referenced this pull request Oct 29, 2026
Orca v1.4.244 ships Russian built in (stablyai/orca#34717), so this is
the pack's last release. Both READMEs now say so and point to
Settings → Appearance → Language → Русский.

v1.4.244 is cut from main after #35365, so 908 strings from the
twenty-four drafts that waited for it are released, with 142 plural
forms: the rest of the source control dropdown, WSL projects moving to
their host, Linear and Jira errors, SSH, workspace and host status, the
Google cookie cleanup, editor conflicts, and more. 36 strings follow
rewritten English. 89 strings and 54 plural forms leave the pack because
v1.4.244 removed them, among them the forms that #34693 made
unreachable.

stablyai/orca#22056 removed the catalog keys no code read, so
tools/unread-catalog-keys.json is empty. Plural forms render since
v1.4.243 (#12505); the READMEs, the glossary and the tools now say so.

Pack 16,476 plus 331 plural forms, 18,958 entries; coverage of the
strings Orca reads that a pack may translate is 99.9%.
audichuang added a commit to audichuang/orca that referenced this pull request Oct 29, 2026
…te keys

Upstream stablyai#22056 gates catalog keys no source references; these two have had
no caller since the Git Graph branch list moved to Local/remote sections.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
audichuang added a commit to audichuang/orca that referenced this pull request Oct 30, 2026
…te keys

Upstream stablyai#22056 gates catalog keys no source references; these two have had
no caller since the Git Graph branch list moved to Local/remote sections.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants