From 3393d7c24acf78055af6fa3b6fb65948b3c11a3e Mon Sep 17 00:00:00 2001 From: Hamhire Hu Date: Wed, 9 Sep 2026 20:56:38 +0800 Subject: [PATCH 1/3] chore(dev): bump to 0.13.0-dev Marks dev as the development state after shipping 0.12.0, per the -dev rule. Co-Authored-By: Claude Opus 5 (1M context) --- apps/desktop/package.json | 2 +- package-lock.json | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/apps/desktop/package.json b/apps/desktop/package.json index 74370c9c..6f92eed8 100644 --- a/apps/desktop/package.json +++ b/apps/desktop/package.json @@ -1,6 +1,6 @@ { "name": "@meebox/desktop", - "version": "0.12.0", + "version": "0.13.0-dev", "private": true, "description": "meebox Electron desktop app", "author": { diff --git a/package-lock.json b/package-lock.json index e7f52868..fcf709ce 100644 --- a/package-lock.json +++ b/package-lock.json @@ -32,7 +32,7 @@ }, "apps/desktop": { "name": "@meebox/desktop", - "version": "0.12.0", + "version": "0.13.0-dev", "dependencies": { "@iconify-json/material-icon-theme": "^1.2.66", "@iconify/react": "^5.2.1", From 8c65e3255b9d4c9c4ab6bbb066e13b4c2e7b3e3c Mon Sep 17 00:00:00 2001 From: Hamhire Hu Date: Thu, 10 Sep 2026 09:54:44 +0800 Subject: [PATCH 2/3] fix(pragent): bake the tiktoken encodings so a read-only install works Every /describe and /review on an installed Windows build died with PermissionError under C:\Program Files. The cause is a bad file in the package, not a permissions setting. litellm points TIKTOKEN_CACHE_DIR at its own tokenizers/ directory and ships encoding files there to be offline-capable, but the files it ships fail the expected_hash pinned by the tiktoken it depends on -- all three of them. tiktoken therefore treats the cache as invalid on every run and re-downloads into site-packages, which is writable while building and read-only once installed. It is not model specific: every path that counts tokens goes through it. scripts/tiktoken-cache.py prime repairs the bundled bytes at assemble time, so the cache is a hit and nothing is written at runtime. verify asserts it, and runs after slimming so a slim rule that deletes the cache is caught too. The assertion compares hashes rather than just loading the encoding, because loading proves nothing here: a stale file loads fine on a machine that can re-download it. That is exactly how 0.12.0 shipped -- assembled clean, smoke tested clean, then failed on the first run of every install. Expected hashes are read from the installed tiktoken rather than copied here, so they cannot drift from it on the next upgrade. Documents the general rule in AGENTS.md and the runtime design doc: the install directory is read-only, a dev machine never shows this because vendor/ is writable, so a dependency that writes under its own package must be pre-populated at assemble time and asserted. Co-Authored-By: Claude Opus 5 (1M context) --- AGENTS.md | 1 + .../scripts/assemble-pragent-runtime.mjs | 26 ++++- apps/desktop/scripts/tiktoken-cache.py | 106 ++++++++++++++++++ docs/arch/02-agent/05-pragent-runtime.md | 11 ++ 4 files changed, 142 insertions(+), 2 deletions(-) create mode 100644 apps/desktop/scripts/tiktoken-cache.py diff --git a/AGENTS.md b/AGENTS.md index f8a739e6..7e838fdc 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -132,6 +132,7 @@ GUI text goes through **react-i18next** (keys are neutral identifiers; `zh-CN` / Missing step 2, Node treats it as an external package loaded per `main: src/index.ts`, hits `export … from './x.js'` while the file is `.ts` → crashes at runtime. - **pr-agent runtime / shim**: the embedded CPython + pinned pr-agent is installed to `vendor/pragent` (gitignored) by `assemble-pragent-runtime.mjs`; the non-invasive patches to pr-agent are in `scripts/pragent-shim/`. The mechanism and hard rules (lazy-import split · version guard · run `prepare:pragent` to sync after changes · debug with `MEEBOX_SHIM_DEBUG=1`) are in [02-agent/05-pragent-runtime](docs/arch/02-agent/05-pragent-runtime.md). On a weak network add `PIP_DEFAULT_TIMEOUT=120` for pip timeouts. +- **The installed runtime directory is READ-ONLY — nothing may write into `vendor/pragent` at runtime**: once installed, the app lives under `C:\Program Files` / `/Applications`, where a write raises `PermissionError` and usually kills the whole command. The trap is that **a dev machine never shows this**: `vendor/` is writable there, so a library that repairs itself on first run looks permanently fine locally while every installed copy fails. Third-party packages do this more often than expected — two cases so far, both fixed by **baking the file at assemble time** rather than letting it be written at runtime: pr-agent's `.secrets.toml`, and tiktoken's encoding cache (litellm ships encoding files that fail the hash its own tiktoken pins, so tiktoken re-downloads them into site-packages on every run — see `scripts/tiktoken-cache.py`). When adding or upgrading a Python dependency, ask whether it writes anywhere under its own package at runtime; if so, pre-populate it in `assemble-pragent-runtime.mjs` **and add a smoke-test assertion**, because "it worked in CI" proves nothing here — the build machine can always write. - **Binary assets go through Git LFS** (`*.png/.ico/.icns` etc.): without git-lfs installed locally you get pointer files, and electron-builder's icon conversion crashes → `brew install git-lfs && git lfs pull`. - **dev won't start**: if `npm run dev` reports `electron does not provide an export named …`, the environment has `ELECTRON_RUN_AS_NODE=1` (injected by the VSCode extension host) → `unset ELECTRON_RUN_AS_NODE` and rerun. - **grep silent on certain files**: e.g. `repo-mirror-manager.ts` is judged `data` by `file` (contains non-UTF-8 bytes), so plain grep stays silent → use `grep -a`. diff --git a/apps/desktop/scripts/assemble-pragent-runtime.mjs b/apps/desktop/scripts/assemble-pragent-runtime.mjs index ade01ef2..5709e267 100644 --- a/apps/desktop/scripts/assemble-pragent-runtime.mjs +++ b/apps/desktop/scripts/assemble-pragent-runtime.mjs @@ -39,6 +39,7 @@ const SHIM_DIR = join(__dirname, 'pragent-shim'); const SHIM_PKG_NAME = 'meebox_pragent_shim'; const SHIM_LOADER = join(SHIM_DIR, 'sitecustomize.py'); const SHIM_RUNTIME = join(SHIM_DIR, SHIM_PKG_NAME, 'runtime.py'); // where _EXPECTED_PRAGENT_VERSION lives +const TIKTOKEN_SCRIPT = join(__dirname, 'tiktoken-cache.py'); // prime/verify the bundled tiktoken encodings const UA = 'meebox-runtime-assembler'; const FORCE = process.argv.includes('--force') || process.env.MEEBOX_PRAGENT_FORCE === '1'; @@ -266,6 +267,19 @@ async function slimRuntime(root) { * the shim patch chain is in place, and the stdlib C extensions / pure-py modules pr-agent actually depends on are all present. Any failure calls fail() * to turn the build red, so **over-trimming is blocked directly in CI and never ships**. */ +/** + * Bake the tiktoken encodings into vendor (prime), and assert they will be used without a write (verify). + * + * The rationale and the failure it prevents are documented in scripts/tiktoken-cache.py — kept there rather than as an + * inline snippet because the check has to read tiktoken's own pinned hashes, which is real code, not a one-liner. + */ +function runTiktokenCache(pythonExe, mode) { + const r = spawnSync(pythonExe, [TIKTOKEN_SCRIPT, mode], { encoding: 'utf8' }); + if (r.status !== 0) + fail(`tiktoken cache ${mode} failed: ${r.stdout || ''}${r.stderr || r.error?.message || ''}`); + return r.stdout.trim(); +} + function smokeTest(pythonExe) { // (0) Split rule: importing meebox_pragent_shim alone must not pull pr_agent into sys.modules (no eager // import pr_agent at the top level, else it slows every python startup). Verified in a fresh interpreter. @@ -294,7 +308,11 @@ function smokeTest(pythonExe) { ].join('\n'); const out = pythonStdout(pythonExe, code); if (!out.includes('MEEBOX_SMOKE_OK')) fail(`smoke test failed (output: ${out.slice(0, 300)})`); - log('smoke test OK: pr_agent importable + shim patch active + key stdlib + litellm completion path intact'); + // Token encodings must be a cache **hit**, not merely loadable. A stale file still loads on this machine, because + // this machine can re-download it — which is precisely how the 0.12.0 build passed here and then died in a read-only + // install. Runs after slimming, so it also catches a slim rule that deletes the cache. + runTiktokenCache(pythonExe, 'verify'); + log('smoke test OK: pr_agent importable + shim patch active + key stdlib + litellm completion path + tiktoken cache usable read-only'); } async function main() { @@ -330,6 +348,7 @@ async function main() { const sp = await syncShim(pythonExe); await ensureSecretsPlaceholders(sp); // Slimming is idempotent: skip if already deleted, so the fast path runs it too — an already-assembled old vendor slims after one prepare:pragent. + runTiktokenCache(pythonExe, 'prime'); await slimRuntime(VENDOR_DIR); smokeTest(pythonExe); log(`ready, skipping rebuild (${versionKey}); re-synced shim + slimmed + smoke tested → ${sp}. Use --force to force a full rebuild.`); @@ -391,7 +410,10 @@ async function main() { await ensureSecretsPlaceholders(sitePackages); log('wrote empty pr_agent/settings(_prod)/.secrets.toml placeholder'); - // 7. Slim (B) + smoke test (CI safety net) + write VERSION + // 7. Prime tiktoken (must precede slimming, so the smoke test also catches a slim rule that deletes the cache) + runTiktokenCache(pythonExe, 'prime'); + + // 8. Slim (B) + smoke test (CI safety net) + write VERSION await slimRuntime(VENDOR_DIR); smokeTest(pythonExe); await writeFile( diff --git a/apps/desktop/scripts/tiktoken-cache.py b/apps/desktop/scripts/tiktoken-cache.py new file mode 100644 index 00000000..d7837c4d --- /dev/null +++ b/apps/desktop/scripts/tiktoken-cache.py @@ -0,0 +1,106 @@ +"""Bake the tiktoken encodings into the embedded runtime, and verify they will be used without a write. + +Run by assemble-pragent-runtime.mjs with a mode argument: + + prime fetch/repair the bundled encoding files (needs network, runs on a machine that can still write) + verify assert the bundled files match what tiktoken pins (pure local check, no network) + +Why this exists +--------------- +litellm points ``TIKTOKEN_CACHE_DIR`` at its own ``litellm_core_utils/tokenizers/`` and ships encoding files there, so +that tiktoken works offline. But the files it ships are **out of sync with the tiktoken it depends on**: each one fails +the ``expected_hash`` pinned in ``tiktoken_ext/openai_public.py``. tiktoken therefore treats the cache as invalid on +every run — it deletes the file and re-downloads it into that same directory, inside site-packages. + +Harmless on a build machine, fatal once installed: under ``C:\\Program Files`` (or any read-only install root) the write +raises PermissionError, which propagates out of TokenHandler and kills the whole command. It is not model specific — +every path that counts tokens goes through this. + +Priming makes the shipped bytes match what tiktoken expects, so the cache is a **hit** and nothing is written at all. +`verify` is the part that matters for not shipping this again: a stale file still *loads* on the build machine, because +that machine can re-download it — so "it worked in CI" proves nothing. Comparing against the pinned hash tests the thing +that actually decides fetch-or-reuse at runtime. + +Only the two encodings modern models resolve to are handled (``o200k_base``, ``cl100k_base``). ``p50k``/``r50k`` belong +to GPT-3-era models that are no longer reachable; priming them would cost download time and size for a path nothing +takes. If that ever changes, `verify` fails loudly rather than letting a broken runtime ship. +""" + +import hashlib +import os +import pathlib +import re +import sys + +# The encodings any model we support resolves to: pr-agent uses encoding_for_model() when the model name contains +# "gpt" (→ o200k_base or cl100k_base depending on the model) and o200k_base for everything else, including CLI +# providers, whose model field is a command name. +REQUIRED = ("o200k_base", "cl100k_base") + + +def _cache_dir() -> pathlib.Path: + import litellm # noqa: F401 — importing it is what sets TIKTOKEN_CACHE_DIR + + cache = os.environ.get("TIKTOKEN_CACHE_DIR", "") + if not cache: + raise SystemExit("litellm did not set TIKTOKEN_CACHE_DIR; the cache location is unknown") + return pathlib.Path(cache) + + +def _pins() -> dict: + """URL → expected sha256, read from the tiktoken build actually installed (never hardcoded here: a hash copied + into this file would drift from tiktoken on its next upgrade, reintroducing the bug it exists to prevent).""" + import tiktoken_ext.openai_public as pub + + src = pathlib.Path(pub.__file__).read_text(encoding="utf-8") + pattern = r'"(https://openaipublic[^"]+\.tiktoken)",\s*\n\s*expected_hash="([0-9a-f]{64})"' + return dict(re.findall(pattern, src)) + + +def _url_for(pins: dict, name: str) -> str: + for url in pins: + if url.endswith(f"/{name}.tiktoken"): + return url + raise SystemExit(f"tiktoken does not pin an encoding named {name}") + + +def prime() -> None: + import tiktoken + + _cache_dir() # side effect: sets TIKTOKEN_CACHE_DIR + for name in REQUIRED: + # get_encoding validates the cached file against expected_hash and re-fetches on mismatch — exactly the repair + # wanted here, while the directory is still writable. + tiktoken.get_encoding(name) + # Interrupted downloads leave `..tmp` siblings behind; they are dead weight in the shipped runtime. + cache = _cache_dir() + for stray in cache.glob("*.tmp"): + stray.unlink(missing_ok=True) + print("MEEBOX_TIKTOKEN_PRIMED") + + +def verify() -> None: + cache = _cache_dir() + pins = _pins() + for name in REQUIRED: + url = _url_for(pins, name) + path = cache / hashlib.sha1(url.encode()).hexdigest() + if not path.exists(): + raise SystemExit(f"tiktoken encoding {name} is missing from the bundled cache ({path})") + actual = hashlib.sha256(path.read_bytes()).hexdigest() + if actual != pins[url]: + raise SystemExit( + f"bundled tiktoken {name} does not match the hash tiktoken pins, so it would be re-downloaded at " + f"runtime — which fails in a read-only install dir. expected {pins[url]}, got {actual}" + ) + print("MEEBOX_TIKTOKEN_VERIFIED") + + +if __name__ == "__main__": + mode = sys.argv[1] if len(sys.argv) > 1 else "" + if mode == "prime": + prime() + elif mode == "verify": + verify() + else: + raise SystemExit(f"usage: tiktoken-cache.py prime|verify (got {mode!r})") diff --git a/docs/arch/02-agent/05-pragent-runtime.md b/docs/arch/02-agent/05-pragent-runtime.md index 21b067fa..e6664351 100644 --- a/docs/arch/02-agent/05-pragent-runtime.md +++ b/docs/arch/02-agent/05-pragent-runtime.md @@ -70,6 +70,17 @@ Current patches (pinned pr-agent **0.45.0**): **Upstream defaults that must stay pinned** (in `buildPragentEnv`): 0.45.0 turned on three features that append to the output this app *parses* — `pr_reviewer.persistent_finding_state` (a "resolved findings" section carrying upstream's own cross-run state, which the app already owns via drafts / finding closures / re-review verdicts) and two coverage footers — plus `pr_description.pr_diagram_direction='adaptive'`, which turns a longer diagram top-down. **Re-check this list on every upgrade**, and note the two distinct ways a default can be wrong here: one *adds output* (landing as a bogus finding), the other *changes presentation* for a surface upstream does not know about — pr-agent assumes a full-width page, while this diagram is read in a narrow chat column. Neither shows up as an error. +### Read-only install directory + +Once installed the runtime sits under `C:\Program Files` / `/Applications` and **cannot be written to**; a write raises `PermissionError` that propagates out and kills the command. Assembly therefore has to bake in anything a run would otherwise create: + +- `.secrets.toml` — pr-agent warns on every startup when it is missing, so an empty placeholder is written at assemble time. +- **tiktoken encodings** — litellm points `TIKTOKEN_CACHE_DIR` at its own `litellm_core_utils/tokenizers/` and ships encoding files there to be offline-capable, but the files it ships **fail the `expected_hash` pinned by the tiktoken it depends on** (all of them, as of litellm 1.99.0 / tiktoken 0.12.0). tiktoken then deletes and re-downloads them into site-packages on every run. `scripts/tiktoken-cache.py prime` repairs the bundled bytes at assemble time so the cache is a **hit** and nothing is written; `verify` asserts it. + +**Why the smoke test has to compare hashes rather than just load the encoding**: a stale file still loads on the build machine — that machine can re-download it. So "CI passed" says nothing about the installed copy. This is exactly how 0.12.0 shipped: the runtime assembled and smoke-tested cleanly, and every Windows install then failed on the first `/describe`. The check compares the bundled bytes against the hash tiktoken pins, which is the thing that actually decides fetch-or-reuse at runtime, and it runs **after** slimming so a slim rule that deletes the cache is caught too. + +The general rule is in [AGENTS.md](../../../AGENTS.md) under engineering maintenance pitfalls: when adding or upgrading a Python dependency, check whether it writes under its own package at runtime, and if so pre-populate it here plus assert it. + ### Real token usage Inline-wrap pr-agent's `_get_completion`, take `prompt/completion/total_tokens` from the returned `response.usage`, From 3b3fa543d25cb1ae223a72a0d73af15fb06e44b4 Mon Sep 17 00:00:00 2001 From: Hamhire Hu Date: Thu, 10 Sep 2026 10:11:16 +0800 Subject: [PATCH 3/3] chore(release): 0.12.1 Patch release for the read-only install failure (#232): version, changelog section in both locales with their compare links, lockfile synced. Co-Authored-By: Claude Opus 5 (1M context) --- CHANGELOG.md | 7 +++++++ CHANGELOG.zh-CN.md | 7 +++++++ apps/desktop/package.json | 2 +- package-lock.json | 2 +- 4 files changed, 16 insertions(+), 2 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 13338a97..72082c77 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -5,6 +5,12 @@ All notable changes to this project are recorded here. The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/), and the versioning follows [Semantic Versioning](https://semver.org/). +## [0.12.1] - 2026-09-10 + +### 🔧 Fixed + +- Reviews no longer fail on Windows when the app is installed to its default location. A file bundled with the review engine was out of date, which made it try to re-download itself into the installation directory on every run — where it has no write permission. The correct file now ships with the app, so nothing is downloaded or written at run time (which also means it works offline). + ## [0.12.0] - 2026-09-09 ### ✨ Added @@ -541,6 +547,7 @@ and the versioning follows [Semantic Versioning](https://semver.org/). License: [Apache-2.0](LICENSE). The package bundles third-party components (pr-agent, Electron, etc.), each distributed under its own license, see [NOTICE](NOTICE). [Unreleased]: https://github.com/huhamhire/code-meeseeks/compare/v0.11.1...HEAD +[0.12.1]: https://github.com/huhamhire/code-meeseeks/compare/v0.12.0...v0.12.1 [0.12.0]: https://github.com/huhamhire/code-meeseeks/compare/v0.11.2...v0.12.0 [0.11.2]: https://github.com/huhamhire/code-meeseeks/compare/v0.11.1...v0.11.2 [0.11.1]: https://github.com/huhamhire/code-meeseeks/compare/v0.11.0...v0.11.1 diff --git a/CHANGELOG.zh-CN.md b/CHANGELOG.zh-CN.md index 63fdb30c..fd63159b 100644 --- a/CHANGELOG.zh-CN.md +++ b/CHANGELOG.zh-CN.md @@ -5,6 +5,12 @@ 本项目所有重要变更记录于此。格式参考 [Keep a Changelog](https://keepachangelog.com/zh-CN/1.1.0/)， 版本号遵循 [语义化版本](https://semver.org/lang/zh-CN/)。 +## [0.12.1] - 2026-09-10 + +### 🔧 修复 + +- Windows 上将应用安装到默认位置后，评审不再失败。评审引擎捆绑的一个文件已过期，导致每次运行都尝试把它重新下载到安装目录——而那里没有写入权限。现在应用直接携带正确的文件，运行时不再下载或写入任何内容（因此离线也可用）。 + ## [0.12.0] - 2026-09-09 ### ✨ 新增 @@ -541,6 +547,7 @@ 许可证：[Apache-2.0](LICENSE)。打包内含第三方组件（pr-agent、Electron 等），各按其许可证分发，见 [NOTICE](NOTICE)。 [Unreleased]: https://github.com/huhamhire/code-meeseeks/compare/v0.11.1...HEAD +[0.12.1]: https://github.com/huhamhire/code-meeseeks/compare/v0.12.0...v0.12.1 [0.12.0]: https://github.com/huhamhire/code-meeseeks/compare/v0.11.2...v0.12.0 [0.11.2]: https://github.com/huhamhire/code-meeseeks/compare/v0.11.1...v0.11.2 [0.11.1]: https://github.com/huhamhire/code-meeseeks/compare/v0.11.0...v0.11.1 diff --git a/apps/desktop/package.json b/apps/desktop/package.json index 6f92eed8..89ba433a 100644 --- a/apps/desktop/package.json +++ b/apps/desktop/package.json @@ -1,6 +1,6 @@ { "name": "@meebox/desktop", - "version": "0.13.0-dev", + "version": "0.12.1", "private": true, "description": "meebox Electron desktop app", "author": { diff --git a/package-lock.json b/package-lock.json index fcf709ce..f5c35939 100644 --- a/package-lock.json +++ b/package-lock.json @@ -32,7 +32,7 @@ }, "apps/desktop": { "name": "@meebox/desktop", - "version": "0.13.0-dev", + "version": "0.12.1", "dependencies": { "@iconify-json/material-icon-theme": "^1.2.66", "@iconify/react": "^5.2.1",