diff --git a/MODULE.bazel b/MODULE.bazel index e69fd0014..cb7220548 100644 --- a/MODULE.bazel +++ b/MODULE.bazel @@ -2,7 +2,7 @@ module( name = "opencc", - version = "1.4.1", + version = "1.4.2", ) bazel_dep(name = "aspect_bazel_lib", version = "2.22.5") diff --git a/NEWS.md b/NEWS.md index 2b20bdd64..3e44fe4d2 100644 --- a/NEWS.md +++ b/NEWS.md @@ -1,14 +1,17 @@ # Change History of OpenCC -## Unreleased +## Version 1.4.2 -* **發佈重點**：轉換熱路徑大幅加速（純文字語料的整體轉換時間最多降至原本的 1/7），修復大端序平台載入 legacy `.ocd` 字典得到空字典的問題，並新增兩個實驗性 API（詞級候選查詢與一對多歧義標註，含 CLI `--ambiguities`）。詞庫方面修正了 `s2twp` 的數處貪婪匹配錯誤與 `tw2t`／`tw2s` 對簡體「么」的破壞性轉換，並補充臺灣／香港地區詞。C++ ABI 與 1.4.1 相同（SOVERSION 1.4），實驗性 API 皆位於未安裝的私有標頭或巨集開關之後，下游程式無需重新連結。 +2026年8月22日 + +* **發佈重點**：轉換熱路徑大幅加速（純文字語料的整體轉換時間最多降至原本的 1/7），修復大端序平台載入 legacy `.ocd` 字典得到空字典的問題，並新增兩個實驗性 API（詞級候選查詢與一對多歧義標註，含 CLI `--ambiguities`）。詞庫方面修正了 `s2twp` 的數處貪婪匹配錯誤與 `tw2t`／`tw2s` 對簡體「么」的破壞性轉換，補充臺灣／香港地區詞，並修正古籍語境下 `s2t` 的一對多字用字。C++ ABI 與 1.4.1 相同（SOVERSION 1.4），實驗性 API 皆位於未安裝的私有標頭或巨集開關之後，下游程式無需重新連結。 * **詞庫更新**： * 修復 `tw2t`／`tw2s`／`tw2sp` 會把已是簡體的「什么」轉成「什幺」的問題：生成的 `TWVariantsRev` 含 `么 -> 幺`（臺灣標準 `幺 -> 么` 的反向），在 `TWVariantsRevPhrases.txt` 補上「什么／怎么／这么／那么／多么／要么／好么／甚么」等例外映射至「麼」形，同時保留「老么 → 老幺」等真正的 yao1 詞（[#1469](https://github.com/BYVoid/OpenCC/issues/1469), [#1473](https://github.com/BYVoid/OpenCC/pull/1473)）。 * 修復 `s2twp` 因較短詞條先命中而拆錯詞的問題：補上「互联网络」「快闪存储器」「老挝人民民主共和国」的完整詞條，並將「老撾人民民主共和國」對應到臺灣官方譯名「寮人民民主共和國」（[#1467](https://github.com/BYVoid/OpenCC/issues/1467), [#1468](https://github.com/BYVoid/OpenCC/pull/1468)）。 * 「梁／樑」：新增臺灣地區用字映射與人名、地名保護詞條（[#1407](https://github.com/BYVoid/OpenCC/pull/1407)）；並將 `TWVariantsRevPhrases` 與 `STPhrases` 對齊，補上 41 條梁（棟梁義）的反向詞組映射，使 `tw2t` 能從上下文還原「橋樑」而不誤傷姓氏、地名（[#1424](https://github.com/BYVoid/OpenCC/pull/1424)）。 * 新增奧斯卡最佳導演與最佳影片製片人的臺灣／香港譯名詞條（取自中文維基百科 `NoteTA` 地區詞表並以 Wikidata 交叉核對，僅收錄真正存在音譯差異者，如 卡梅隆／卡麥隆／金馬倫）（[#1466](https://github.com/BYVoid/OpenCC/pull/1466)）。 * 新增地區詞「內存條 → 記憶體模組」（`s2twp`／`s2hkp`，含反向詞條）（[#1475](https://github.com/BYVoid/OpenCC/pull/1475)）、「数字人文 → 數位人文」（`s2twp`／`tw2sp`）（[#1465](https://github.com/BYVoid/OpenCC/pull/1465)）。 + * `s2t`：修正古籍語境下一對多字的用字。`征`／`咸`／`斗`／`里`／`范`／`岳`／`冲`／`郁`／`谷` 在 `STPhrases` 沒有對應詞條時會退回 `STCharacters`，而那裏的候選次序是按現代用法排的，碰上文言文和書名就常常選錯。此次按義項補入詞條：征伐義（`征南`、`从征实录`）、年號與副詞義（`咸淳`、`少长咸集`）、星宿義（`斗姆`、`东斗星君`）、地名與音譯（`甫里`、`摩里支`）、姓氏義（`范文正`、`岳武穆`）、道教與梵語音譯（`道冲`、`郁迦罗越`）；`谷神` 改為保留「谷」（《老子》「谷神不死」），`谷神星` 則因鍵更長，仍映射為「穀神星」。這批問題是在清理殆知閣文獻時發現的，該批書名中 47 條誤轉修正了 46 條，現代語義的轉換一律不受影響（[#1479](https://github.com/BYVoid/OpenCC/pull/1479)）。 * `s2t`：「小丑」不再預設轉換為「小醜」（[#1423](https://github.com/BYVoid/OpenCC/pull/1423)）；「雇」改為 `STCharacters` 中「僱」以外的非預設候選（[#906](https://github.com/BYVoid/OpenCC/pull/906)）；新增「这周／那周／若干周／几周／周线 → 週」等指示與量化用法，並補上「周围／周边／周遭」的分詞保護詞條（[#1428](https://github.com/BYVoid/OpenCC/pull/1428)）；新增顯式詞條「复盘 → 復盤」，使其不再只依賴 `复` 的候選順序（[#1427](https://github.com/BYVoid/OpenCC/pull/1427)）。 * `t2s`：修正過度轉換「乾斷食 → 干断食」「乾紅 → 干红」（[#621](https://github.com/BYVoid/OpenCC/issues/621), [#1429](https://github.com/BYVoid/OpenCC/pull/1429)）。 * 修正「鑒」相關詞條與若干提稱語，如「郝铭鉴 → 郝銘鑒」（[#1421](https://github.com/BYVoid/OpenCC/pull/1421)）。 @@ -23,12 +26,12 @@ * npm 套件的源碼編譯 fallback 從 node-gyp 改為 Bazel：原生 addon 優先透過 `@opencc/opencc--` 預編譯 scoped 套件安裝（涵蓋 macOS x64/arm64、Linux x64/arm64、Windows x64，均以 Bazel 構建，字典隨包預編譯）；沒有預編譯二進位套件的平台，`npm install` 會以 Bazel 完整地從源碼構建——編譯 `//node:opencc` 並重新生成字典（`bazel` / `bazelisk` 取自 PATH，或經 `npx` 自動下載 bazelisk；字典生成腳本的 Python 由 Bazel hermetic 工具鏈提供）。`binding.gyp` 與 `.gypi` 構建定義刪除，`node-gyp`、`node-gyp-build`、`node-addon-api`、`prebuildify` 依賴與 `prebuild` npm script 一併移除；tarball 不再攜帶 `deps/`（C++ 三方依賴改由 Bazel Central Registry 提供），改為攜帶 Bazel workspace、`src/` C++ 源碼與 `data/` 字典源檔（[#1426](https://github.com/BYVoid/OpenCC/pull/1426), [#1430](https://github.com/BYVoid/OpenCC/pull/1430)）。 * CI 測試任務、Makefile target 與源碼 checkout 的開發構建一併改用 Bazel，與 npm 發佈所用的構建路徑一致；Windows CI 任務移至 `windows-2025-vs2026`（[#1426](https://github.com/BYVoid/OpenCC/pull/1426)）。 * **打包 / 構建系統**： - * `packaging/debian/` 同步至 Debian 的 `1.4.1+ds1-7`。Debian 已採納上游先前提出的兩項提案：jieba 插件打包（拆為 `opencc-jieba` 與 arch:all 的 `opencc-jieba-data` 兩個套件，內嵌 cppjieba 的理由記於 `debian/README.source`）與 `*.ocd2` 移入 `libopencc-data`；Debian 端亦重新啟用了 autopkgtest。`packaging/debian/SPLIT_PROPOSAL.md` 隨提案落地移除。 + * `packaging/debian/` 同步至 Debian 的 `1.4.1+ds1-8`。Debian 已採納上游先前提出的 jieba 插件打包提案（拆為 `opencc-jieba` 與 arch:all 的 `opencc-jieba-data` 兩個套件，內嵌 cppjieba 的理由記於 `debian/README.source`）；Debian 端亦重新啟用了 autopkgtest。`packaging/debian/SPLIT_PROPOSAL.md` 隨提案落地移除。另外 `1.4.1+ds1-8` 修復了 `.ocd2` 被放進 `Architecture: all` 套件的問題：`.ocd2` 是依主機位元組序寫出的 marisa-trie 映像，所有架構都會拿到 arch:all buildd 上構建的小端序副本，opencc 在 s390x 上因而以 `size > avail_` 失敗（Debian bug [#1145010](https://bugs.debian.org/1145010)，由重新啟用的 s390x autopkgtest 揪出）。`*.ocd2` 已移回 arch:any 的 `libopencc1.4`、`jieba_merged.ocd2` 移回 arch:any 的 `opencc-jieba`；`libopencc-data` 與 `opencc-jieba-data` 現在只含 JSON 配置與純文字 jieba 詞典來源。 * Debian 套件構建改在 `debian:sid` 容器中進行（同步來的打包以 unstable 為目標，Ubuntu LTS runner 無法滿足），並在 `dpkg-buildpackage` 前套用 `debian/patches` quilt series（自動跳過已在樹中的補丁）；arm64 改用 `ubuntu-24.04-arm` runner 原生構建，取代先前的交叉構建（[#1420](https://github.com/BYVoid/OpenCC/pull/1420)）。 * 新增 `packaging/freebsd/`：FreeBSD ports tree `chinese/opencc` 條目的參考副本，供上游審閱與準備提交給 port maintainer 的改動。 * 新增精簡 C++ 源碼包發佈流程（`release-source.yml`）：以 `OPENCC_SOURCE_PACKAGE_PROFILE=opencc` 產生僅含 C++ 核心的 CPack 壓縮包（不含 Node.js、Python、文檔、packaging 與 plugins），並以 CMake 與 Bazel 雙路徑冒煙測試後上傳至 draft release；為此保留 Bazel workspace 檔案與頂層 `patches/`，僅排除可能含 API key 的 `.bazelrc.user`。 - * 修正 `cmake/GitVersion.cmake` 的 fallback 版本號：1.4.1 發佈時仍停留在 1.4.0，導致從釋出的 tarball（無 git 資訊）構建時版本回報為 1.4.0。 - * Windows 發佈簽章改用 SignPath 的 release-signing policy，並新增對 win32-x64 npm 二進位（`opencc.node`、`opencc-jieba.dll`）的 Authenticode 簽章（[#1458](https://github.com/BYVoid/OpenCC/pull/1458), [#1459](https://github.com/BYVoid/OpenCC/pull/1459)）。 + * 修正 `cmake/GitVersion.cmake` 的 fallback 版本號：1.4.1 發佈時仍停留在 1.4.0，導致從釋出的 tarball（無 git 資訊）構建時版本回報為 1.4.0；本次發佈同步將其更新至 1.4.2，並讓 `scripts/compute-version.sh` 的 fallback 與之一致（先前停留在 1.2.1）。因此 Debian 打包的 `0010-fix-git-version-fallback.patch` 對 1.4.2 已無作用而移除，`0012-fix-legacy-dict-detection-on-big-endian.patch` 亦已包含於本次發佈。 + * Windows 發佈自本版起啟用程式碼簽章憑證：可攜式 CLI 壓縮包中的執行檔，以及 win32-x64 npm 二進位（`opencc.node`、`opencc-jieba.dll`）皆經 Authenticode 簽章，簽章服務由 [SignPath.io](https://about.signpath.io/) 免費提供、憑證由 [SignPath Foundation](https://signpath.org/) 免費提供，發佈流程改用其 release-signing policy；相關聲明與簽章／隱私政策見 README 的「Code Signing 程式碼簽章」一節（[#1458](https://github.com/BYVoid/OpenCC/pull/1458), [#1459](https://github.com/BYVoid/OpenCC/pull/1459)）。 * **測試**： * `PrefixMatchTest` 改用測試專屬的字典檔名，避免與 `MarisaDictTest` 在共享工作目錄下爭用同一個 `dict.ocd2`，該競爭會使並行 ctest（如 `dh_auto_test` 的 `ctest -j8`）偶發失敗。 diff --git a/README.md b/README.md index 81694f88d..1a6991e0d 100644 --- a/README.md +++ b/README.md @@ -64,16 +64,18 @@ For details, see [Design Principles](./DESIGN_PRINCIPLES.md) and [Regional Phras ### Prebuilt binaries 預編譯二進位檔 -OpenCC 1.4.1 修復了 `opencc` npm 套件在無預編譯二進制平台上源碼編譯安裝失敗的 -問題，並包含一批詞庫修正；C++ ABI 與 1.4.0 相同（SOVERSION 1.4），自 1.4.0 -升級的下游 C++ 程式無需重新連結。 +OpenCC 1.4.2 大幅加速了轉換熱路徑（純文字語料的整體轉換時間最多降至原本的 +1/7），修復了大端序平台載入 legacy `.ocd` 字典得到空字典的問題，並包含一批詞庫 +修正；C++ ABI 與 1.4.1 相同（SOVERSION 1.4），自 1.4.0／1.4.1 升級的下游 C++ +程式無需重新連結。 -* Windows (x86_64): [OpenCC-1.4.1](https://github.com/BYVoid/OpenCC/releases/download/ver.1.4.1/OpenCC-1.4.1-windows-x64-portable.zip) ([SHA-256](https://github.com/BYVoid/OpenCC/releases/download/ver.1.4.1/OpenCC-1.4.1-windows-x64-portable.zip.sha256)) +* Windows (x86_64): [OpenCC-1.4.2](https://github.com/BYVoid/OpenCC/releases/download/ver.1.4.2/OpenCC-1.4.2-windows-x64-portable.zip) ([SHA-256](https://github.com/BYVoid/OpenCC/releases/download/ver.1.4.2/OpenCC-1.4.2-windows-x64-portable.zip.sha256)) * This Windows release is available from WinGet. For details, see [doc/windows-winget-release.md](doc/windows-winget-release.md). + * The executables are Authenticode-signed; see [Code Signing 程式碼簽章](#code-signing-程式碼簽章). * Requires Microsoft Visual C++ Redistributable for Visual Studio 2015-2026. Download the latest version from [Microsoft](https://learn.microsoft.com/en-us/cpp/windows/latest-supported-vc-redist?view=msvc-170#latest-supported-redistributable-version). * Debian/Ubuntu: - * [opencc-1.4.1-1-deb-amd64.zip](https://github.com/BYVoid/OpenCC/releases/download/ver.1.4.1/opencc-1.4.1-1-deb-amd64.zip) - * [opencc-1.4.1-1-deb-arm64.zip](https://github.com/BYVoid/OpenCC/releases/download/ver.1.4.1/opencc-1.4.1-1-deb-arm64.zip) + * [opencc-1.4.2-1-deb-amd64.zip](https://github.com/BYVoid/OpenCC/releases/download/ver.1.4.2/opencc-1.4.2-1-deb-amd64.zip) + * [opencc-1.4.2-1-deb-arm64.zip](https://github.com/BYVoid/OpenCC/releases/download/ver.1.4.2/opencc-1.4.2-1-deb-arm64.zip) * Each zip bundles the `opencc`, `opencc-jieba`, and `libopencc*` deb packages for one architecture, with a `SHA256SUMS` file. ## Usage 使用 @@ -513,6 +515,33 @@ Apache License 2.0 - Optional dependency used by the experimental `opencc-jieba` plugin. - 試驗性 `opencc-jieba` 插件使用的可選依賴。 +## Code Signing 程式碼簽章 + +自 1.4.2 起，Windows 發佈的二進位檔（可攜式 CLI 壓縮包中的執行檔，以及 npm +win32-x64 套件中的 `opencc.node` 與 `opencc-jieba.dll`）皆經過 Authenticode +簽章。 + +Since 1.4.2, the Windows binaries — the executables in the portable CLI zip +and the `opencc.node` / `opencc-jieba.dll` shipped in the win32-x64 npm +packages — are Authenticode-signed. + +This program uses free code signing provided by +[SignPath.io](https://about.signpath.io/), and a free code signing +certificate by the [SignPath Foundation](https://signpath.org/). + +本項目使用 [SignPath.io](https://about.signpath.io/) 免費提供的程式碼簽章服務， +憑證由 [SignPath Foundation](https://signpath.org/) 免費提供。 + +* **Code signing policy 簽章政策**：只有合併進 `master` 的提交會被簽章，簽章請求 + 由 [維護者](#contributors-貢獻者)在發佈流程中透過 GitHub Actions 發起 + （見 [`release-winget.yml`](.github/workflows/release-winget.yml) 與 + [`release-npm-binaries.yml`](.github/workflows/release-npm-binaries.yml)）。 +* **Privacy policy 隱私政策**：OpenCC 為離線的文字轉換程式庫與命令列工具，除非 + 使用者、安裝者或操作者明確要求，否則不會將任何資訊傳輸至其他網路系統。 + This program will not transfer any information to other networked systems + unless specifically requested by the user or the person installing or + operating it. + ## Change History 版本歷史 * [NEWS](https://github.com/BYVoid/OpenCC/blob/master/NEWS.md) diff --git a/cmake/GitVersion.cmake b/cmake/GitVersion.cmake index 8ac25b65b..d4e2fd021 100644 --- a/cmake/GitVersion.cmake +++ b/cmake/GitVersion.cmake @@ -14,7 +14,7 @@ set(_OPENCC_FALLBACK_MAJOR 1) set(_OPENCC_FALLBACK_MINOR 4) -set(_OPENCC_FALLBACK_REVISION 1) +set(_OPENCC_FALLBACK_REVISION 2) find_package(Git QUIET) diff --git a/node/BUILD.bazel b/node/BUILD.bazel index ade6c346d..296cfa533 100644 --- a/node/BUILD.bazel +++ b/node/BUILD.bazel @@ -46,7 +46,7 @@ node_addon( "//conditions:default": ["-std=c++20"], }), defines = [ - 'OPENCC_VERSION=\\"1.4.1\\"', + 'OPENCC_VERSION=\\"1.4.2\\"', "NAPI_DISABLE_CPP_EXCEPTIONS", "Opencc_BUILT_AS_STATIC", ], diff --git a/package-lock.json b/package-lock.json index 455206d86..083bc6ba0 100644 --- a/package-lock.json +++ b/package-lock.json @@ -1,12 +1,12 @@ { "name": "opencc", - "version": "1.4.1", + "version": "1.4.2", "lockfileVersion": 3, "requires": true, "packages": { "": { "name": "opencc", - "version": "1.4.1", + "version": "1.4.2", "hasInstallScript": true, "license": "Apache-2.0", "bin": { @@ -19,11 +19,11 @@ "node": ">=20.17" }, "optionalDependencies": { - "@opencc/opencc-darwin-arm64": "1.4.1", - "@opencc/opencc-darwin-x64": "1.4.1", - "@opencc/opencc-linux-arm64": "1.4.1", - "@opencc/opencc-linux-x64": "1.4.1", - "@opencc/opencc-win32-x64": "1.4.1" + "@opencc/opencc-darwin-arm64": "1.4.2", + "@opencc/opencc-darwin-x64": "1.4.2", + "@opencc/opencc-linux-arm64": "1.4.2", + "@opencc/opencc-linux-x64": "1.4.2", + "@opencc/opencc-win32-x64": "1.4.2" } }, "node_modules/@apidevtools/json-schema-ref-parser": { @@ -51,71 +51,6 @@ "dev": true, "license": "MIT" }, - "node_modules/@opencc/opencc-darwin-arm64": { - "version": "1.4.1", - "resolved": "https://registry.npmjs.org/@opencc/opencc-darwin-arm64/-/opencc-darwin-arm64-1.4.1.tgz", - "integrity": "sha512-aZjxFCzczT4/qLcsy8UvNTX7wIvb4sT40HoVeqgf4O/NaGb0GmMGH1UJA5o9HYf8riCPpSn/6IcYcfl8JwPzKQ==", - "cpu": [ - "arm64" - ], - "license": "Apache-2.0", - "optional": true, - "os": [ - "darwin" - ] - }, - "node_modules/@opencc/opencc-darwin-x64": { - "version": "1.4.1", - "resolved": "https://registry.npmjs.org/@opencc/opencc-darwin-x64/-/opencc-darwin-x64-1.4.1.tgz", - "integrity": "sha512-+ee6P8rUOR8MLocZcNu6pH/tRQ8fToNVaKpSxE4gPAkDYEYesfMQ+HV4chgV/ZYTPjJK0X4uDB09UGF98G74Sg==", - "cpu": [ - "x64" - ], - "license": "Apache-2.0", - "optional": true, - "os": [ - "darwin" - ] - }, - "node_modules/@opencc/opencc-linux-arm64": { - "version": "1.4.1", - "resolved": "https://registry.npmjs.org/@opencc/opencc-linux-arm64/-/opencc-linux-arm64-1.4.1.tgz", - "integrity": "sha512-bWXabiyb8Q+AiQXSit/nqQiujyzFOdUo2skUZj1v9FBjV8J93AT653L7T7ffjCSIYp5eFm8CovYRoh3v6rrl1A==", - "cpu": [ - "arm64" - ], - "license": "Apache-2.0", - "optional": true, - "os": [ - "linux" - ] - }, - "node_modules/@opencc/opencc-linux-x64": { - "version": "1.4.1", - "resolved": "https://registry.npmjs.org/@opencc/opencc-linux-x64/-/opencc-linux-x64-1.4.1.tgz", - "integrity": "sha512-jH8wn2r84e0Upn5ot50m7TtTXL9T/CJ+XePk/excub81l79vP9WRA0oj//XssNpNlccmwsTX5EdK+D+akpD5vQ==", - "cpu": [ - "x64" - ], - "license": "Apache-2.0", - "optional": true, - "os": [ - "linux" - ] - }, - "node_modules/@opencc/opencc-win32-x64": { - "version": "1.4.1", - "resolved": "https://registry.npmjs.org/@opencc/opencc-win32-x64/-/opencc-win32-x64-1.4.1.tgz", - "integrity": "sha512-YwD+YErlg6XQNvMVAJJ0le2Qn/fBx/mJykUMOQejZoyvXPnnbpIsTNZIF92BRJ0KJP5Ov/lShtM+IMVg3xXf+w==", - "cpu": [ - "x64" - ], - "license": "Apache-2.0", - "optional": true, - "os": [ - "win32" - ] - }, "node_modules/@types/json-schema": { "version": "7.0.15", "resolved": "https://registry.npmjs.org/@types/json-schema/-/json-schema-7.0.15.tgz", diff --git a/package.json b/package.json index fea206195..3eac98682 100644 --- a/package.json +++ b/package.json @@ -1,6 +1,6 @@ { "name": "opencc", - "version": "1.4.1", + "version": "1.4.2", "description": "Conversion between Traditional and Simplified Chinese", "author": "Carbo Kuo ", "license": "Apache-2.0", @@ -74,10 +74,10 @@ "json-schema-to-typescript": "^15.0.4" }, "optionalDependencies": { - "@opencc/opencc-darwin-arm64": "1.4.1", - "@opencc/opencc-darwin-x64": "1.4.1", - "@opencc/opencc-linux-arm64": "1.4.1", - "@opencc/opencc-linux-x64": "1.4.1", - "@opencc/opencc-win32-x64": "1.4.1" + "@opencc/opencc-darwin-arm64": "1.4.2", + "@opencc/opencc-darwin-x64": "1.4.2", + "@opencc/opencc-linux-arm64": "1.4.2", + "@opencc/opencc-linux-x64": "1.4.2", + "@opencc/opencc-win32-x64": "1.4.2" } } diff --git a/packaging/debian/README.md b/packaging/debian/README.md index 331560917..7a9eed1d7 100644 --- a/packaging/debian/README.md +++ b/packaging/debian/README.md @@ -10,16 +10,23 @@ authoritative packaging is whatever Debian actually ships. To keep proposals reviewable as deltas, `debian/` here is periodically synced to the packaging Debian currently ships: -- Upstream: BYVoid/OpenCC `ver.1.4.1` -- Debian source package: `opencc 1.4.1+ds1-7` (unstable) +- Upstream: BYVoid/OpenCC `ver.1.4.2` +- Debian source package: `opencc 1.4.1+ds1-8` (unstable) +- Synced from packaging-repository commit `e4d7c5e8` (2026-08-21, + "Override arch-dep-package-has-big-usr-share for opencc-jieba") - Debian packaging repository: https://salsa.debian.org/debian/opencc - Patch reference index: https://udd.debian.org/patches.cgi?src=opencc -As of this sync there are **no outstanding upstream deltas**: `debian/` -is a verbatim copy of the Debian packaging repository. The two proposals -previously carried here have been adopted, with implementation changes: +As of this sync the only upstream deltas are release bookkeeping for +1.4.2: a `1.4.2-1` `UNRELEASED` changelog entry (`release-deb.yml` stamps +the built artifacts with `dpkg-parsechangelog`, which would otherwise +still report `1.4.1+ds1-8`) and the removal of the now-inapplicable +`0010-fix-git-version-fallback.patch`. Otherwise `debian/` is a verbatim +copy of the Debian packaging repository. The two proposals previously +carried here have been adopted, with implementation changes — and one of +them has since been partly reverted: - **Jieba plugin packaging** (Debian bug [#1141451](https://bugs.debian.org/1141451)): adopted in @@ -30,9 +37,20 @@ previously carried here have been adopted, with implementation changes: The bundled `cppjieba` question is documented in `debian/README.source` rather than treated as a blocker. - **`*.ocd2` dictionaries in `libopencc-data`**: adopted in - `1.4.1+ds1-4`. `debian/libopencc-data.install` now lists the non-Jieba - configurations explicitly instead of globbing `usr/share/opencc/*.json`, - which would otherwise also capture the `*_jieba.json` files. + `1.4.1+ds1-4`, then **reverted in `1.4.1+ds1-8`** — this upstream + proposal was wrong. `.ocd2` files are marisa-trie images written in + host byte order, so shipping them from an `Architecture: all` package + gave every architecture the little-endian copies built on the arch:all + buildd, and opencc failed at runtime on s390x with `size > avail_` + (Debian bug [#1145010](https://bugs.debian.org/1145010), caught by the + s390x autopkgtest re-enabled in `-7`). `usr/share/opencc/*.ocd2` is + back in the arch:any `libopencc1.4` and + `jieba_dict/jieba_merged.ocd2` back in the arch:any `opencc-jieba`, + with matching `Breaks`/`Replaces` against the `-8~` versions of the + data packages. What survives of the proposal is that + `debian/libopencc-data.install` lists the non-Jieba configurations + explicitly instead of globbing `usr/share/opencc/*.json`, which would + otherwise also capture the `*_jieba.json` files. ## 2. Patch set @@ -42,27 +60,30 @@ Current quilt patch set (see `debian/patches/series`): - 0004-Use-system-googletest.patch - 0005-Disable-build-in-setup.py.patch - 0009-setup.py-Handle-python-binding-instead-of-cmake.patch -- 0010-fix-git-version-fallback.patch - 0011-drop-python-cli-entry-point.patch - 0012-fix-legacy-dict-detection-on-big-endian.patch Patch numbering reflects historical Debian quilt ordering and is not -contiguous. 0012 is `Applied-Upstream` (commit `8aeb5e69`) and can be -dropped once Debian moves to a release containing it. +contiguous. 0010 was dropped for the 1.4.2 release: upstream now sets +`_OPENCC_FALLBACK_REVISION` to the release revision in +`cmake/GitVersion.cmake`, so the patch no longer applies. 0012 is +`Applied-Upstream` (commit `8aeb5e69`) and is contained in 1.4.2; it can +be dropped once Debian moves to that release. ## 3. Binary packages (in this tree) - `opencc` — command line tools -- `libopencc1.4` — runtime shared library (tracks `OPENCC_ABI_VERSION`; - renamed from `libopencc1.3`, with a regenerated `.symbols` file) +- `libopencc1.4` — runtime shared library and the `*.ocd2` dictionaries + (tracks `OPENCC_ABI_VERSION`; renamed from `libopencc1.3`, with a + regenerated `.symbols` file) - `libopencc-dev` — headers, pkg-config, CMake package config -- `libopencc-data` — JSON configs and `*.ocd2` dictionaries - (arch-independent) +- `libopencc-data` — JSON configurations only (arch-independent) - `libopencc-doc` — Doxygen HTML API docs - `python3-opencc` — Python bindings built via pybuild/pyproject -- `opencc-jieba` — Jieba segmentation plugin (`libopencc-jieba.so`) -- `opencc-jieba-data` — merged Jieba dictionary and `*_jieba.json` - configurations (arch-independent) +- `opencc-jieba` — Jieba segmentation plugin (`libopencc-jieba.so`) and + the merged `jieba_merged.ocd2` dictionary +- `opencc-jieba-data` — plain-text `jieba_dict/*.utf8` sources and the + `*_jieba.json` configurations (arch-independent) ## 4. Notes on the current packaging @@ -75,6 +96,15 @@ in `DIR_SHARE_OPENCC`; it is deliberately *not* `Multi-Arch: same`, since those generated CMake files embed the architecture triplet and so are not byte-identical across architectures. +`libopencc1.4` must not be `Multi-Arch: same` either, even though the +multiarch hinter suggests it: since `1.4.1+ds1-8` it ships the `*.ocd2` +dictionaries, whose marisa-trie images are written in host byte order +and therefore differ between architectures of opposite endianness. The +same constraint means `*.ocd2` must never be shipped from an +`Architecture: all` package — see `debian/README.source` and section 1. +`opencc-jieba` carries an `arch-dep-package-has-big-usr-share` lintian +override for the resulting `jieba_merged.ocd2`. + Several historical Debian patches correspond to functionality now covered by upstream CMake options (e.g. `USE_SYSTEM_*`); `debian/rules` builds with those options against Debian-packaged libraries. The exception is @@ -92,7 +122,8 @@ Autopkgtest was re-enabled in `1.4.1+ds1-7`: `debian/tests/control` now runs `cli-smoke` (installed CLI checks), `integration` (the forked `CommandLineConvertTest` built against `libopencc-dev` and run over `test/testcases/testcases.json`), and `jieba-plugin` (plugin loading and -`*_jieba.json` conversions). The forked test reads configurations from +`*_jieba.json` conversions). It immediately earned its keep: the s390x +run is what surfaced the arch:all `*.ocd2` breakage fixed in `-8`. The forked test reads configurations from `/usr/share/opencc` and passes `--include-tofu-risk-dictionaries`, so changes to `test/testcases/testcases.json` or to the CLI's config resolution can break it — see `debian/tests/README.md`. @@ -101,5 +132,5 @@ resolution can break it — see `debian/tests/README.md`. Copy the `debian/` directory from the salsa repository checkout over `packaging/debian/debian/`, re-apply any upstream proposal deltas that -Debian has not adopted (currently none), and update the baseline section -above. +Debian has not adopted (currently only the release bookkeeping described +in section 1), and update the baseline section above. diff --git a/packaging/debian/debian/README.source b/packaging/debian/debian/README.source index 2d9404625..f446faa57 100644 --- a/packaging/debian/debian/README.source +++ b/packaging/debian/debian/README.source @@ -35,4 +35,33 @@ Until then the embedded copy is built and shipped. It is covered by the "Files: plugins/jieba/deps/cppjieba/*" stanza in debian/copyright. Note that a security update to cppjieba would require rebuilding opencc. - -- Boyuan Yang , Wed, 12 Aug 2026 11:04:33 -0400 + +The .ocd2 dictionaries are architecture-dependent +================================================= + +The generated .ocd2 dictionaries (/usr/share/opencc/*.ocd2 and +/usr/share/opencc/jieba_dict/jieba_merged.ocd2) are marisa-trie images +that opencc mmaps and uses in place. The marisa-trie on-disk format is +written in host byte order and is not portable: loading a little-endian +image on a big-endian host fails with + + Invalid OpenCC Marisa dictionary: + lib/marisa/grimoire/io/mapper.cc:99: std::runtime_error: size > avail_ + +They must therefore never be shipped in an Architecture: all package. +1.4.1+ds1-4 moved *.ocd2 into libopencc-data and 1.4.1+ds1-5 put +jieba_merged.ocd2 into opencc-jieba-data; both are arch:all, so every +architecture got the dictionaries built on the (little-endian) buildd +that produced the arch:all binaries, and opencc was unusable on s390x +(#1145010 / the s390x autopkgtest failure). 1.4.1+ds1-8 moved them back +into the arch:any libopencc1.4 and opencc-jieba. + +For the same reason libopencc1.4 must not be marked Multi-Arch: same, +even though the multiarch hinter on tracker.debian.org suggests it: the +/usr/share/opencc/*.ocd2 files differ between architectures of different +byte order, so two such instances are not co-installable. Only the +.json configurations and the plain-text jieba dictionary sources are +architecture-independent, and those stay in libopencc-data and +opencc-jieba-data. + + -- Boyuan Yang , Fri, 21 Aug 2026 15:20:00 -0400 diff --git a/packaging/debian/debian/changelog b/packaging/debian/debian/changelog index f6d9d6e18..922bd4459 100644 --- a/packaging/debian/debian/changelog +++ b/packaging/debian/debian/changelog @@ -1,3 +1,48 @@ +opencc (1.4.2-1) UNRELEASED; urgency=medium + + * Update packaging for the upstream 1.4.2 release: + - Drop 0010-fix-git-version-fallback.patch: upstream 1.4.2 sets + _OPENCC_FALLBACK_REVISION to 2 in cmake/GitVersion.cmake, so builds + from the release tarball (which carries no .git) already report the + correct version and the patch no longer applies. + - 0012-fix-legacy-dict-detection-on-big-endian.patch is contained in + the 1.4.2 release and can be dropped as well. + - 0003, 0004, 0005, 0009 and 0011 apply unchanged against 1.4.2. + + -- Frank Lin Sat, 22 Aug 2026 00:00:00 +0000 + +opencc (1.4.1+ds1-8) unstable; urgency=medium + + * Do not ship the generated .ocd2 dictionaries in Architecture: all + packages. They are marisa-trie images written in host byte order and + are unusable on hosts of the opposite endianness, so every big-endian + architecture received the little-endian copies built on the arch:all + buildd. On s390x this made opencc fail at runtime with "Invalid OpenCC + Marisa dictionary: [...] size > avail_", which is what the s390x + autopkgtest has been catching since the tests were re-enabled in -7. + - debian/libopencc1.4.install, debian/libopencc-data.install: Move + usr/share/opencc/*.ocd2 back from libopencc-data (arch:all) to + libopencc1.4 (arch:any), reverting the move made in 1.4.1+ds1-4. + This also fixes the undeclared file conflict between libopencc-data + and libopencc1.4/libopencc1.1, as libopencc-data now only ships the + architecture-independent *.json configurations. (Closes: #1145010) + - debian/opencc-jieba.install, debian/opencc-jieba-data.install: + Likewise move usr/share/opencc/jieba_dict/jieba_merged.ocd2 from + opencc-jieba-data (arch:all) to opencc-jieba (arch:any). The + plain-text jieba_dict/*.utf8 sources stay in opencc-jieba-data. + - debian/control: Add the matching Breaks/Replaces against + libopencc-data (<< 1.4.1+ds1-8~) on libopencc1.4 and against + opencc-jieba-data (<< 1.4.1+ds1-8~) on opencc-jieba. + - debian/control: Adjust the binary package descriptions to say which + package now holds the dictionaries. + - debian/README.source: Document that the .ocd2 files are byte-order + dependent, must never be shipped arch:all, and that libopencc1.4 + therefore cannot be marked Multi-Arch: same. + - debian/opencc-jieba.lintian-overrides: New file, overriding + arch-dep-package-has-big-usr-share for the same reason. + + -- Boyuan Yang Fri, 21 Aug 2026 14:17:22 -0400 + opencc (1.4.1+ds1-7) unstable; urgency=medium * Team upload. diff --git a/packaging/debian/debian/control b/packaging/debian/debian/control index 81ed1610f..6c4331fa5 100644 --- a/packaging/debian/debian/control +++ b/packaging/debian/debian/control @@ -43,10 +43,13 @@ Breaks: libopencc2-data (<< 1.1.1~), Replaces: libopencc2-data (<< 1.1.1~), -Description: ${source:Synopsis} - data files +Description: ${source:Synopsis} - configuration files ${source:Extended-Description} . - This package includes the data files. + This package includes the architecture-independent conversion + configuration files (/usr/share/opencc/*.json). The binary conversion + dictionaries (*.ocd2) they refer to are shipped in libopencc1.4, as + their marisa-trie format is byte-order dependent. Package: libopencc-dev Architecture: any @@ -75,17 +78,20 @@ Architecture: any Depends: libopencc-data (= ${source:Version}), Breaks: + libopencc-data (<< 1.4.1+ds1-8~), libopencc1.1, libopencc1.2, libopencc1.3, Replaces: + libopencc-data (<< 1.4.1+ds1-8~), libopencc1.1, libopencc1.2, libopencc1.3, Description: ${source:Synopsis} - runtime ${source:Extended-Description} . - This package includes the shared library. + This package includes the shared library and the binary conversion + dictionaries (/usr/share/opencc/*.ocd2). Package: opencc Architecture: any @@ -107,6 +113,10 @@ Section: utils Depends: opencc (= ${binary:Version}), opencc-jieba-data (= ${source:Version}), +Breaks: + opencc-jieba-data (<< 1.4.1+ds1-8~), +Replaces: + opencc-jieba-data (<< 1.4.1+ds1-8~), Enhances: opencc, Description: ${source:Synopsis} - Jieba segmentation plugin @@ -114,7 +124,8 @@ Description: ${source:Synopsis} - Jieba segmentation plugin . This package provides the optional Jieba-based word segmentation plugin (libopencc-jieba.so) that enables word-segmentation-aware regional phrase - conversion. + conversion, together with the merged binary Jieba dictionary + (/usr/share/opencc/jieba_dict/jieba_merged.ocd2). Package: opencc-jieba-data Architecture: all @@ -123,8 +134,11 @@ Section: utils Description: ${source:Synopsis} - Jieba plugin data files ${source:Extended-Description} . - This package provides the merged Jieba dictionary and the *_jieba.json - configurations used by the Jieba-based word segmentation plugin. + This package provides the architecture-independent Jieba dictionary + sources and the *_jieba.json configurations used by the Jieba-based word + segmentation plugin. The merged binary dictionary (jieba_merged.ocd2) is + shipped in opencc-jieba, as its marisa-trie format is byte-order + dependent. Package: python3-opencc Architecture: any diff --git a/packaging/debian/debian/libopencc-data.install b/packaging/debian/debian/libopencc-data.install index a36461469..73fd7b13b 100644 --- a/packaging/debian/debian/libopencc-data.install +++ b/packaging/debian/debian/libopencc-data.install @@ -1,4 +1,3 @@ -usr/share/opencc/*.ocd2 usr/share/opencc/hk2s.json usr/share/opencc/hk2sp.json usr/share/opencc/hk2t.json diff --git a/packaging/debian/debian/libopencc1.4.install b/packaging/debian/debian/libopencc1.4.install index 3ddde5841..28d741f23 100644 --- a/packaging/debian/debian/libopencc1.4.install +++ b/packaging/debian/debian/libopencc1.4.install @@ -1 +1,2 @@ usr/lib/*/lib*.so.* +usr/share/opencc/*.ocd2 diff --git a/packaging/debian/debian/opencc-jieba-data.install b/packaging/debian/debian/opencc-jieba-data.install index 0da4fd4b7..e026250d4 100644 --- a/packaging/debian/debian/opencc-jieba-data.install +++ b/packaging/debian/debian/opencc-jieba-data.install @@ -1,2 +1,2 @@ -usr/share/opencc/jieba_dict/ +usr/share/opencc/jieba_dict/*.utf8 usr/share/opencc/*_jieba.json diff --git a/packaging/debian/debian/opencc-jieba.install b/packaging/debian/debian/opencc-jieba.install index c3b536d94..ac46e9d2d 100644 --- a/packaging/debian/debian/opencc-jieba.install +++ b/packaging/debian/debian/opencc-jieba.install @@ -1 +1,2 @@ usr/lib/*/opencc/plugins/ +usr/share/opencc/jieba_dict/jieba_merged.ocd2 diff --git a/packaging/debian/debian/opencc-jieba.lintian-overrides b/packaging/debian/debian/opencc-jieba.lintian-overrides new file mode 100644 index 000000000..6bbe308d8 --- /dev/null +++ b/packaging/debian/debian/opencc-jieba.lintian-overrides @@ -0,0 +1,4 @@ +# jieba_merged.ocd2 is a marisa-trie image written in host byte order and is +# therefore architecture-dependent; it cannot be moved to the arch:all +# opencc-jieba-data package. See debian/README.source. +opencc-jieba: arch-dep-package-has-big-usr-share diff --git a/packaging/debian/debian/patches/0010-fix-git-version-fallback.patch b/packaging/debian/debian/patches/0010-fix-git-version-fallback.patch deleted file mode 100644 index 88d274bbc..000000000 --- a/packaging/debian/debian/patches/0010-fix-git-version-fallback.patch +++ /dev/null @@ -1,26 +0,0 @@ -From: xiao sheng wen -Date: Wed, 15 Jul 2026 04:34:09 +0000 -Subject: Fix stale fallback version when building from a tarball without .git - -Upstream derives OPENCC_VERSION from `git describe` in -cmake/GitVersion.cmake, falling back to hardcoded -_OPENCC_FALLBACK_MAJOR/MINOR/REVISION when no .git directory is -present (as is always the case for Debian source tarballs). The -fallback was left at 1.4.0 in the 1.4.1 release, so builds from the -released tarball report version 1.4.0 instead of 1.4.1. - -Forwarded: not-needed -Last-Update: 2026-07-15 -Index: opencc/cmake/GitVersion.cmake -=================================================================== ---- opencc.orig/cmake/GitVersion.cmake -+++ opencc/cmake/GitVersion.cmake -@@ -14,7 +14,7 @@ - - set(_OPENCC_FALLBACK_MAJOR 1) - set(_OPENCC_FALLBACK_MINOR 4) --set(_OPENCC_FALLBACK_REVISION 0) -+set(_OPENCC_FALLBACK_REVISION 1) - - find_package(Git QUIET) - diff --git a/packaging/debian/debian/patches/series b/packaging/debian/debian/patches/series index 44a74cd2e..7ad02bbcc 100644 --- a/packaging/debian/debian/patches/series +++ b/packaging/debian/debian/patches/series @@ -2,6 +2,5 @@ 0004-Use-system-googletest.patch 0005-Disable-build-in-setup.py.patch 0009-setup.py-Handle-python-binding-instead-of-cmake.patch -0010-fix-git-version-fallback.patch 0011-drop-python-cli-entry-point.patch 0012-fix-legacy-dict-detection-on-big-endian.patch diff --git a/plugins/jieba/node/package.json b/plugins/jieba/node/package.json index 94d6f44d2..b628a5001 100644 --- a/plugins/jieba/node/package.json +++ b/plugins/jieba/node/package.json @@ -1,6 +1,6 @@ { "name": "opencc-jieba", - "version": "1.4.1", + "version": "1.4.2", "description": "Jieba segmentation plugin for OpenCC", "author": "Carbo Kuo and OpenCC contributors", "license": "Apache-2.0", @@ -14,11 +14,11 @@ "opencc": ">=1.4.0" }, "optionalDependencies": { - "@opencc/opencc-jieba-darwin-arm64": "1.4.1", - "@opencc/opencc-jieba-darwin-x64": "1.4.1", - "@opencc/opencc-jieba-linux-arm64": "1.4.1", - "@opencc/opencc-jieba-linux-x64": "1.4.1", - "@opencc/opencc-jieba-win32-x64": "1.4.1" + "@opencc/opencc-jieba-darwin-arm64": "1.4.2", + "@opencc/opencc-jieba-darwin-x64": "1.4.2", + "@opencc/opencc-jieba-linux-arm64": "1.4.2", + "@opencc/opencc-jieba-linux-x64": "1.4.2", + "@opencc/opencc-jieba-win32-x64": "1.4.2" }, "files": [ "index.js", diff --git a/scripts/build-cli-windows-zig.sh b/scripts/build-cli-windows-zig.sh index d976c74a8..2a0ca9e83 100755 --- a/scripts/build-cli-windows-zig.sh +++ b/scripts/build-cli-windows-zig.sh @@ -48,7 +48,7 @@ if [[ -z "$version" ]]; then MODULE.bazel | head -n 1 )" fi -version="${version:-1.4.1}" +version="${version:-1.4.2}" export ZIG_LOCAL_CACHE_DIR="${ZIG_LOCAL_CACHE_DIR:-$ROOT_DIR/.zig-local-cache}" export ZIG_GLOBAL_CACHE_DIR="${ZIG_GLOBAL_CACHE_DIR:-$ROOT_DIR/.zig-global-cache}" diff --git a/scripts/build-node-windows-zig.sh b/scripts/build-node-windows-zig.sh index fd69113f7..2b7da959d 100755 --- a/scripts/build-node-windows-zig.sh +++ b/scripts/build-node-windows-zig.sh @@ -69,7 +69,7 @@ set +e -w \ -Wno-nullability-completeness \ -DNDEBUG \ - '-DOPENCC_VERSION="1.4.1"' \ + '-DOPENCC_VERSION="1.4.2"' \ -DNAPI_DISABLE_CPP_EXCEPTIONS \ -DOpencc_BUILT_AS_STATIC \ -I. \ diff --git a/scripts/compute-version.sh b/scripts/compute-version.sh index ed5b8e3fc..61db3365c 100755 --- a/scripts/compute-version.sh +++ b/scripts/compute-version.sh @@ -16,7 +16,7 @@ set -euo pipefail # Fallback version — must be kept in sync with cmake/GitVersion.cmake -FALLBACK_VERSION="1.2.1" +FALLBACK_VERSION="1.4.2" # Detect dirty working tree DIRTY="" diff --git a/setup.py b/setup.py index d5b35581e..f9829bec4 100644 --- a/setup.py +++ b/setup.py @@ -15,7 +15,7 @@ _cmake_file = os.path.join(_this_dir, 'CMakeLists.txt') _author_file = os.path.join(_this_dir, 'AUTHORS') _readme_file = os.path.join(_this_dir, 'README.md') -_fallback_version = '1.4.1' +_fallback_version = '1.4.2' def _get_version_from_git(): diff --git a/src/README.md b/src/README.md index bf5f4e717..f2ac65ec9 100644 --- a/src/README.md +++ b/src/README.md @@ -102,6 +102,48 @@ behavior because phrase priority and multi-stage conversion order matter. - `DictConverter.hpp`, `DictConverter.cpp` - Converts dictionary files between supported formats. +### Binary Format Portability + +Neither binary dictionary format is portable across platforms, but they +fail differently, and the constraints differ per format. + +| | Byte order | Word size (32/64-bit) | +| --- | --- | --- | +| `.ocd` | dependent | **independent** (since 1.4.1) | +| `.ocd2` | dependent | dependent | + +- **`.ocd` is word-size independent.** The Darts double array uses + `id_type = unsigned int` (4 bytes on every supported platform), and + `DartsDict::SerializeToFile` writes the array length as an explicit + `uint32_t`. `BinaryDict` writes all of its integer fields as fixed-width + `uint32_t` as well. Files written by pre-1.4.1 64-bit builds used native + 8-byte `size_t` fields; `BinaryDict::DetectFieldWidth()` still recognizes + that legacy layout, and `LooksLikeLegacy64Field()` performs the probe + through a native `uint64_t` read so the detection is correct on + big-endian hosts too. +- **`.ocd2` is not.** The `SerializedValues` payload OpenCC writes uses + fixed-width `uint32_t` fields, but the marisa-trie image that precedes it + does not: `MARISA_WORD_SIZE` is derived from `UINTPTR_MAX`, and the + serialized `BitVector::Unit` / `FlatVector::Unit` are `uint64_t` on + 64-bit builds and `uint32_t` on 32-bit builds. A 64-bit-built `.ocd2` + loaded by a 32-bit build is not reliably rejected either — marisa's + `Vector::map` only checks `total_size % sizeof(T)`, and a multiple of + 8 is always a multiple of 4 — so the mismatch can be read as silently + wrong data rather than an error. + +Forcing `MARISA_WORD_SIZE` to 32 would not make `.ocd2` portable, since +byte order would still differ; it would not apply to `USE_SYSTEM_MARISA` +builds (which distributions use), where the format would then depend on +which marisa the build linked against; and it would narrow the rank/select +word width in `MarisaDict`'s lookup hot path. Making `.ocd2` genuinely +portable means normalizing the trie image on write and converting back on +load, which gives up the in-place mmap that makes the format fast. + +The practical consequence is that generated `.ocd2` files must be built +for the architecture that consumes them, and must never be shipped from an +architecture-independent package — see `packaging/debian/README.md` for +how this played out in Debian. + ## Public APIs ### C++ API