diff --git a/AGENTS.md b/AGENTS.md index f8a739e6..7e838fdc 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -132,6 +132,7 @@ GUI text goes through **react-i18next** (keys are neutral identifiers; `zh-CN` / Missing step 2, Node treats it as an external package loaded per `main: src/index.ts`, hits `export … from './x.js'` while the file is `.ts` → crashes at runtime. - **pr-agent runtime / shim**: the embedded CPython + pinned pr-agent is installed to `vendor/pragent` (gitignored) by `assemble-pragent-runtime.mjs`; the non-invasive patches to pr-agent are in `scripts/pragent-shim/`. The mechanism and hard rules (lazy-import split · version guard · run `prepare:pragent` to sync after changes · debug with `MEEBOX_SHIM_DEBUG=1`) are in [02-agent/05-pragent-runtime](docs/arch/02-agent/05-pragent-runtime.md). On a weak network add `PIP_DEFAULT_TIMEOUT=120` for pip timeouts. +- **The installed runtime directory is READ-ONLY — nothing may write into `vendor/pragent` at runtime**: once installed, the app lives under `C:\Program Files` / `/Applications`, where a write raises `PermissionError` and usually kills the whole command. The trap is that **a dev machine never shows this**: `vendor/` is writable there, so a library that repairs itself on first run looks permanently fine locally while every installed copy fails. Third-party packages do this more often than expected — two cases so far, both fixed by **baking the file at assemble time** rather than letting it be written at runtime: pr-agent's `.secrets.toml`, and tiktoken's encoding cache (litellm ships encoding files that fail the hash its own tiktoken pins, so tiktoken re-downloads them into site-packages on every run — see `scripts/tiktoken-cache.py`). When adding or upgrading a Python dependency, ask whether it writes anywhere under its own package at runtime; if so, pre-populate it in `assemble-pragent-runtime.mjs` **and add a smoke-test assertion**, because "it worked in CI" proves nothing here — the build machine can always write. - **Binary assets go through Git LFS** (`*.png/.ico/.icns` etc.): without git-lfs installed locally you get pointer files, and electron-builder's icon conversion crashes → `brew install git-lfs && git lfs pull`. - **dev won't start**: if `npm run dev` reports `electron does not provide an export named …`, the environment has `ELECTRON_RUN_AS_NODE=1` (injected by the VSCode extension host) → `unset ELECTRON_RUN_AS_NODE` and rerun. - **grep silent on certain files**: e.g. `repo-mirror-manager.ts` is judged `data` by `file` (contains non-UTF-8 bytes), so plain grep stays silent → use `grep -a`. diff --git a/CHANGELOG.md b/CHANGELOG.md index 13338a97..72082c77 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -5,6 +5,12 @@ All notable changes to this project are recorded here. The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/), and the versioning follows [Semantic Versioning](https://semver.org/). +## [0.12.1] - 2026-09-10 + +### 🔧 Fixed + +- Reviews no longer fail on Windows when the app is installed to its default location. A file bundled with the review engine was out of date, which made it try to re-download itself into the installation directory on every run — where it has no write permission. The correct file now ships with the app, so nothing is downloaded or written at run time (which also means it works offline). + ## [0.12.0] - 2026-09-09 ### ✨ Added @@ -541,6 +547,7 @@ and the versioning follows [Semantic Versioning](https://semver.org/). License: [Apache-2.0](LICENSE). The package bundles third-party components (pr-agent, Electron, etc.), each distributed under its own license, see [NOTICE](NOTICE). [Unreleased]: https://github.com/huhamhire/code-meeseeks/compare/v0.11.1...HEAD +[0.12.1]: https://github.com/huhamhire/code-meeseeks/compare/v0.12.0...v0.12.1 [0.12.0]: https://github.com/huhamhire/code-meeseeks/compare/v0.11.2...v0.12.0 [0.11.2]: https://github.com/huhamhire/code-meeseeks/compare/v0.11.1...v0.11.2 [0.11.1]: https://github.com/huhamhire/code-meeseeks/compare/v0.11.0...v0.11.1 diff --git a/CHANGELOG.zh-CN.md b/CHANGELOG.zh-CN.md index 63fdb30c..fd63159b 100644 --- a/CHANGELOG.zh-CN.md +++ b/CHANGELOG.zh-CN.md @@ -5,6 +5,12 @@ 本项目所有重要变更记录于此。格式参考 [Keep a Changelog](https://keepachangelog.com/zh-CN/1.1.0/)， 版本号遵循 [语义化版本](https://semver.org/lang/zh-CN/)。 +## [0.12.1] - 2026-09-10 + +### 🔧 修复 + +- Windows 上将应用安装到默认位置后，评审不再失败。评审引擎捆绑的一个文件已过期，导致每次运行都尝试把它重新下载到安装目录——而那里没有写入权限。现在应用直接携带正确的文件，运行时不再下载或写入任何内容（因此离线也可用）。 + ## [0.12.0] - 2026-09-09 ### ✨ 新增 @@ -541,6 +547,7 @@ 许可证：[Apache-2.0](LICENSE)。打包内含第三方组件（pr-agent、Electron 等），各按其许可证分发，见 [NOTICE](NOTICE)。 [Unreleased]: https://github.com/huhamhire/code-meeseeks/compare/v0.11.1...HEAD +[0.12.1]: https://github.com/huhamhire/code-meeseeks/compare/v0.12.0...v0.12.1 [0.12.0]: https://github.com/huhamhire/code-meeseeks/compare/v0.11.2...v0.12.0 [0.11.2]: https://github.com/huhamhire/code-meeseeks/compare/v0.11.1...v0.11.2 [0.11.1]: https://github.com/huhamhire/code-meeseeks/compare/v0.11.0...v0.11.1 diff --git a/apps/desktop/package.json b/apps/desktop/package.json index 6f92eed8..89ba433a 100644 --- a/apps/desktop/package.json +++ b/apps/desktop/package.json @@ -1,6 +1,6 @@ { "name": "@meebox/desktop", - "version": "0.13.0-dev", + "version": "0.12.1", "private": true, "description": "meebox Electron desktop app", "author": { diff --git a/apps/desktop/scripts/assemble-pragent-runtime.mjs b/apps/desktop/scripts/assemble-pragent-runtime.mjs index ade01ef2..5709e267 100644 --- a/apps/desktop/scripts/assemble-pragent-runtime.mjs +++ b/apps/desktop/scripts/assemble-pragent-runtime.mjs @@ -39,6 +39,7 @@ const SHIM_DIR = join(__dirname, 'pragent-shim'); const SHIM_PKG_NAME = 'meebox_pragent_shim'; const SHIM_LOADER = join(SHIM_DIR, 'sitecustomize.py'); const SHIM_RUNTIME = join(SHIM_DIR, SHIM_PKG_NAME, 'runtime.py'); // where _EXPECTED_PRAGENT_VERSION lives +const TIKTOKEN_SCRIPT = join(__dirname, 'tiktoken-cache.py'); // prime/verify the bundled tiktoken encodings const UA = 'meebox-runtime-assembler'; const FORCE = process.argv.includes('--force') || process.env.MEEBOX_PRAGENT_FORCE === '1'; @@ -266,6 +267,19 @@ async function slimRuntime(root) { * the shim patch chain is in place, and the stdlib C extensions / pure-py modules pr-agent actually depends on are all present. Any failure calls fail() * to turn the build red, so **over-trimming is blocked directly in CI and never ships**. */ +/** + * Bake the tiktoken encodings into vendor (prime), and assert they will be used without a write (verify). + * + * The rationale and the failure it prevents are documented in scripts/tiktoken-cache.py — kept there rather than as an + * inline snippet because the check has to read tiktoken's own pinned hashes, which is real code, not a one-liner. + */ +function runTiktokenCache(pythonExe, mode) { + const r = spawnSync(pythonExe, [TIKTOKEN_SCRIPT, mode], { encoding: 'utf8' }); + if (r.status !== 0) + fail(`tiktoken cache ${mode} failed: ${r.stdout || ''}${r.stderr || r.error?.message || ''}`); + return r.stdout.trim(); +} + function smokeTest(pythonExe) { // (0) Split rule: importing meebox_pragent_shim alone must not pull pr_agent into sys.modules (no eager // import pr_agent at the top level, else it slows every python startup). Verified in a fresh interpreter. @@ -294,7 +308,11 @@ function smokeTest(pythonExe) { ].join('\n'); const out = pythonStdout(pythonExe, code); if (!out.includes('MEEBOX_SMOKE_OK')) fail(`smoke test failed (output: ${out.slice(0, 300)})`); - log('smoke test OK: pr_agent importable + shim patch active + key stdlib + litellm completion path intact'); + // Token encodings must be a cache **hit**, not merely loadable. A stale file still loads on this machine, because + // this machine can re-download it — which is precisely how the 0.12.0 build passed here and then died in a read-only + // install. Runs after slimming, so it also catches a slim rule that deletes the cache. + runTiktokenCache(pythonExe, 'verify'); + log('smoke test OK: pr_agent importable + shim patch active + key stdlib + litellm completion path + tiktoken cache usable read-only'); } async function main() { @@ -330,6 +348,7 @@ async function main() { const sp = await syncShim(pythonExe); await ensureSecretsPlaceholders(sp); // Slimming is idempotent: skip if already deleted, so the fast path runs it too — an already-assembled old vendor slims after one prepare:pragent. + runTiktokenCache(pythonExe, 'prime'); await slimRuntime(VENDOR_DIR); smokeTest(pythonExe); log(`ready, skipping rebuild (${versionKey}); re-synced shim + slimmed + smoke tested → ${sp}. Use --force to force a full rebuild.`); @@ -391,7 +410,10 @@ async function main() { await ensureSecretsPlaceholders(sitePackages); log('wrote empty pr_agent/settings(_prod)/.secrets.toml placeholder'); - // 7. Slim (B) + smoke test (CI safety net) + write VERSION + // 7. Prime tiktoken (must precede slimming, so the smoke test also catches a slim rule that deletes the cache) + runTiktokenCache(pythonExe, 'prime'); + + // 8. Slim (B) + smoke test (CI safety net) + write VERSION await slimRuntime(VENDOR_DIR); smokeTest(pythonExe); await writeFile( diff --git a/apps/desktop/scripts/tiktoken-cache.py b/apps/desktop/scripts/tiktoken-cache.py new file mode 100644 index 00000000..d7837c4d --- /dev/null +++ b/apps/desktop/scripts/tiktoken-cache.py @@ -0,0 +1,106 @@ +"""Bake the tiktoken encodings into the embedded runtime, and verify they will be used without a write. + +Run by assemble-pragent-runtime.mjs with a mode argument: + + prime fetch/repair the bundled encoding files (needs network, runs on a machine that can still write) + verify assert the bundled files match what tiktoken pins (pure local check, no network) + +Why this exists +--------------- +litellm points ``TIKTOKEN_CACHE_DIR`` at its own ``litellm_core_utils/tokenizers/`` and ships encoding files there, so +that tiktoken works offline. But the files it ships are **out of sync with the tiktoken it depends on**: each one fails +the ``expected_hash`` pinned in ``tiktoken_ext/openai_public.py``. tiktoken therefore treats the cache as invalid on +every run — it deletes the file and re-downloads it into that same directory, inside site-packages. + +Harmless on a build machine, fatal once installed: under ``C:\\Program Files`` (or any read-only install root) the write +raises PermissionError, which propagates out of TokenHandler and kills the whole command. It is not model specific — +every path that counts tokens goes through this. + +Priming makes the shipped bytes match what tiktoken expects, so the cache is a **hit** and nothing is written at all. +`verify` is the part that matters for not shipping this again: a stale file still *loads* on the build machine, because +that machine can re-download it — so "it worked in CI" proves nothing. Comparing against the pinned hash tests the thing +that actually decides fetch-or-reuse at runtime. + +Only the two encodings modern models resolve to are handled (``o200k_base``, ``cl100k_base``). ``p50k``/``r50k`` belong +to GPT-3-era models that are no longer reachable; priming them would cost download time and size for a path nothing +takes. If that ever changes, `verify` fails loudly rather than letting a broken runtime ship. +""" + +import hashlib +import os +import pathlib +import re +import sys + +# The encodings any model we support resolves to: pr-agent uses encoding_for_model() when the model name contains +# "gpt" (→ o200k_base or cl100k_base depending on the model) and o200k_base for everything else, including CLI +# providers, whose model field is a command name. +REQUIRED = ("o200k_base", "cl100k_base") + + +def _cache_dir() -> pathlib.Path: + import litellm # noqa: F401 — importing it is what sets TIKTOKEN_CACHE_DIR + + cache = os.environ.get("TIKTOKEN_CACHE_DIR", "") + if not cache: + raise SystemExit("litellm did not set TIKTOKEN_CACHE_DIR; the cache location is unknown") + return pathlib.Path(cache) + + +def _pins() -> dict: + """URL → expected sha256, read from the tiktoken build actually installed (never hardcoded here: a hash copied + into this file would drift from tiktoken on its next upgrade, reintroducing the bug it exists to prevent).""" + import tiktoken_ext.openai_public as pub + + src = pathlib.Path(pub.__file__).read_text(encoding="utf-8") + pattern = r'"(https://openaipublic[^"]+\.tiktoken)",\s*\n\s*expected_hash="([0-9a-f]{64})"' + return dict(re.findall(pattern, src)) + + +def _url_for(pins: dict, name: str) -> str: + for url in pins: + if url.endswith(f"/{name}.tiktoken"): + return url + raise SystemExit(f"tiktoken does not pin an encoding named {name}") + + +def prime() -> None: + import tiktoken + + _cache_dir() # side effect: sets TIKTOKEN_CACHE_DIR + for name in REQUIRED: + # get_encoding validates the cached file against expected_hash and re-fetches on mismatch — exactly the repair + # wanted here, while the directory is still writable. + tiktoken.get_encoding(name) + # Interrupted downloads leave `..tmp` siblings behind; they are dead weight in the shipped runtime. + cache = _cache_dir() + for stray in cache.glob("*.tmp"): + stray.unlink(missing_ok=True) + print("MEEBOX_TIKTOKEN_PRIMED") + + +def verify() -> None: + cache = _cache_dir() + pins = _pins() + for name in REQUIRED: + url = _url_for(pins, name) + path = cache / hashlib.sha1(url.encode()).hexdigest() + if not path.exists(): + raise SystemExit(f"tiktoken encoding {name} is missing from the bundled cache ({path})") + actual = hashlib.sha256(path.read_bytes()).hexdigest() + if actual != pins[url]: + raise SystemExit( + f"bundled tiktoken {name} does not match the hash tiktoken pins, so it would be re-downloaded at " + f"runtime — which fails in a read-only install dir. expected {pins[url]}, got {actual}" + ) + print("MEEBOX_TIKTOKEN_VERIFIED") + + +if __name__ == "__main__": + mode = sys.argv[1] if len(sys.argv) > 1 else "" + if mode == "prime": + prime() + elif mode == "verify": + verify() + else: + raise SystemExit(f"usage: tiktoken-cache.py prime|verify (got {mode!r})") diff --git a/docs/arch/02-agent/05-pragent-runtime.md b/docs/arch/02-agent/05-pragent-runtime.md index 21b067fa..e6664351 100644 --- a/docs/arch/02-agent/05-pragent-runtime.md +++ b/docs/arch/02-agent/05-pragent-runtime.md @@ -70,6 +70,17 @@ Current patches (pinned pr-agent **0.45.0**): **Upstream defaults that must stay pinned** (in `buildPragentEnv`): 0.45.0 turned on three features that append to the output this app *parses* — `pr_reviewer.persistent_finding_state` (a "resolved findings" section carrying upstream's own cross-run state, which the app already owns via drafts / finding closures / re-review verdicts) and two coverage footers — plus `pr_description.pr_diagram_direction='adaptive'`, which turns a longer diagram top-down. **Re-check this list on every upgrade**, and note the two distinct ways a default can be wrong here: one *adds output* (landing as a bogus finding), the other *changes presentation* for a surface upstream does not know about — pr-agent assumes a full-width page, while this diagram is read in a narrow chat column. Neither shows up as an error. +### Read-only install directory + +Once installed the runtime sits under `C:\Program Files` / `/Applications` and **cannot be written to**; a write raises `PermissionError` that propagates out and kills the command. Assembly therefore has to bake in anything a run would otherwise create: + +- `.secrets.toml` — pr-agent warns on every startup when it is missing, so an empty placeholder is written at assemble time. +- **tiktoken encodings** — litellm points `TIKTOKEN_CACHE_DIR` at its own `litellm_core_utils/tokenizers/` and ships encoding files there to be offline-capable, but the files it ships **fail the `expected_hash` pinned by the tiktoken it depends on** (all of them, as of litellm 1.99.0 / tiktoken 0.12.0). tiktoken then deletes and re-downloads them into site-packages on every run. `scripts/tiktoken-cache.py prime` repairs the bundled bytes at assemble time so the cache is a **hit** and nothing is written; `verify` asserts it. + +**Why the smoke test has to compare hashes rather than just load the encoding**: a stale file still loads on the build machine — that machine can re-download it. So "CI passed" says nothing about the installed copy. This is exactly how 0.12.0 shipped: the runtime assembled and smoke-tested cleanly, and every Windows install then failed on the first `/describe`. The check compares the bundled bytes against the hash tiktoken pins, which is the thing that actually decides fetch-or-reuse at runtime, and it runs **after** slimming so a slim rule that deletes the cache is caught too. + +The general rule is in [AGENTS.md](../../../AGENTS.md) under engineering maintenance pitfalls: when adding or upgrading a Python dependency, check whether it writes under its own package at runtime, and if so pre-populate it here plus assert it. + ### Real token usage Inline-wrap pr-agent's `_get_completion`, take `prompt/completion/total_tokens` from the returned `response.usage`, diff --git a/package-lock.json b/package-lock.json index fcf709ce..f5c35939 100644 --- a/package-lock.json +++ b/package-lock.json @@ -32,7 +32,7 @@ }, "apps/desktop": { "name": "@meebox/desktop", - "version": "0.13.0-dev", + "version": "0.12.1", "dependencies": { "@iconify-json/material-icon-theme": "^1.2.66", "@iconify/react": "^5.2.1",