Conversation
Issue #421 asked for Python embedded in a single EXE so updating would not mean copying ~20k loose files over an existing install. A onefile EXE is not viable for this stack (onefile modes re-extract the whole multi-GB payload on every launch, and torch/onnxruntime fight frozen-import hooks), so this addresses the root cause instead: ship less, and stop making users hand-copy a full zip for a release that only changed app code. Leaner package (make-portable.ps1): - Stripping is now unconditional. The -StripVenv opt-in gate was a silent regression risk: nothing stopped a future workflow edit from shipping the unstripped venv with no error. - Also strips stdlib base/Lib/test and per-package test/tests dirs. - Deliberately does NOT strip .dist-info/RECORD. pip needs it to replace a package, and install_cuda_torch pip-installs into this venv on every NVIDIA machine at first run; removing it yields "Failed to uninstall ... missing RECORD file". - Adds a post-strip import check so an over-aggressive strip fails the build rather than a release. Updater (main.rs, catalog.js): - New commands installed_runtime_id, download_app_update, apply_app_update. - Opt-in: the check on launch is unchanged, but download and apply are each an explicit click. It never auto-applies and never interrupts a running job. - Replaces StemDeck.exe and backend/ only. python/ is never touched, because an NVIDIA install rewrites it with CUDA torch at first run and torchDeviceSettled skips ensure_torch_device once the device is cuda, so swapping the directory would silently drop that machine to CPU with no recovery. - The runtime id (uv.lock + interpreter major.minor) is a compatibility gate, not a download trigger: if a release changed the Python dependency set the updater stands down and points at the full download. Only 19 of the last 200 commits touch uv.lock, so the fast path covers most releases. - apply_app_update stages and validates everything before any destructive rename, stops the backend synchronously first (the existing stop_backend returns before the process dies, which would have made every update fail on Windows), and retries renames past transient AV/indexer handles. - Known gap, documented in code: the two exe renames are not atomic. A hard crash in that window leaves StemDeck.exe.old needing a manual rename. Closing it needs a bootstrap launcher that is never itself replaced. CI publishes -app.zip, its .sha256 and -runtime-version.json alongside the unchanged full zips. Fresh installs are unaffected. i18n: the 5 new strings are translated into all 7 language tables, not just English. t() falls back to English silently, so an English-only key looks correct in testing and ships untranslated to six locales. Verified: Windows and Linux (WSL) both compile clean with no new clippy warnings, 39 Rust tests pass on both, JS suites pass, ruff clean. Two new unit tests pin the JSON contract between the PowerShell writer and the Rust reader. Not yet verified: no end-to-end run against a real release.
…nd (#421) Built both packages on a real Windows box and drove the whole flow. Four bugs that only surfaced by running it, none of which static checks could see. 1. Stale version after updating. app_version() read installed dist metadata, which lives in python/ -- the directory the updater deliberately never replaces. A self-updated install kept reporting the old version and would re-offer an update it had already applied, forever. It now prefers the app layer's static/version.json, which moves with backend/. Gitignored, so Docker and source checkouts still fall through to the hatch-vcs metadata. Proven: after a real update, python/ dist-info says 0.13.0 while /api/health reports 0.13.1. 2. The page CSP blocked the whole feature. The UI is served over http by the Python backend, so its connect-src applies: api.github.com is allowed, github.com and objects.githubusercontent.com are not, and that is where release assets live. Fetching the checksum and runtime id from JS was refused, so the pill would simply never appear. Those two reads moved into Rust (check_app_update), whose HTTP client is not bound by the page CSP, so the policy from #171 stays exactly as tight as it was. 3. plugin:event|listen refused by the Tauri ACL. App-defined commands are not ACL-gated but plugin commands are, and the capability does not cover the remote http origin the UI is served from. The progress bar is now indeterminate instead of granting a remote origin event permissions to put a percentage on a 5 MB download. 4. The post-strip import check re-bloated the package. Running Python regenerated 1,912 files / 39 MB of __pycache__ that the strip had just removed, cancelling nearly all of it: the net saving was 180 files. Swept once after the last interpreter run, and backend/ no longer ships a developer's local __pycache__ either. Also: the *.old sweep now runs on every launch rather than only on a version change. apply_app_update relaunches then exits, so on the first launch of the new build Windows still holds StemDeck.exe.old open, the delete fails silently, and gated on a change that already happened it would never retry. Observed for real: 15.7 MB stranded. Verified swept on the next launch. UI: "Update now" is an accent pill BESIDE Download, not a replacement, so the zip stays one click away and is the escape hatch if an update fails. Measured against the published v0.13.0 package: 18,143 -> 16,056 files (-2,087, -11.5%) and 883 -> 850 MB. The real win for #421 is the update path itself: 5 MB and 123 files instead of 284 MB and 16,056. Verified on this machine: a real 6-stem Demucs separation through the stripped package; the full notify -> Update now -> download -> restart -> relaunch cycle, after which user data (job, 7 stems, 130 MB of models), portable.txt, cpu-only and python/ were all untouched; and the safety gate correctly declining, with no download attempted, when the release's runtime id differs. Not covered: the NVIDIA package was not built, though the risk that motivated the gate is structurally gone now that python/ is never swapped.
Verified end to end on real Windows + RTX 3080Built both packages locally and drove the whole update flow against a local stand-in for the Releases API. Testing found four bugs that static checks could not see, all fixed in What the run proved
The four bugs
Measured, not estimatedAgainst the published v0.13.0 package: 18,143 -> 16,056 files (-11.5%), 883 -> 850 MB. The real answer to #421 is the update path: 5 MB / 123 files instead of 284 MB / 16,056. UI"Update now" is an accent pill beside Download, not a replacement, so the zip stays one click away as the escape hatch. Separate pre-existing bug found (not fixed here)A second StemDeck instance silently adopts the first one's backend. Still not coveredThe NVIDIA package was not built. The risk that motivated the gate is structurally gone now that |
![github-code-quality[bot]](https://avatars.githubusercontent.com/u/9919?s=60&v=4){kind=small}
Both findings from the automated review were fair. Narrow the bare `except Exception: pass` in app_version() to (OSError, ValueError, AttributeError). That is bandit B110, which this repo's own security conventions call out. The three cover every real failure here -- absent or unreadable file, invalid JSON or bad encoding, and valid JSON that is not an object so has no .get -- while letting an actual bug in the function surface instead of silently degrading the reported version. Bandit now reports no issues for the file. Use one import style in test_health_api.py so app.main is no longer imported both as `import app.main as main` and `from app.main import app` in the same module. Also added a "[]" case: JSON that parses but is not an object, which is the AttributeError branch the narrowed except now names explicitly.
|
Both automated findings were fair, fixed in Empty except (
A genuine bug in the function now surfaces instead of silently degrading the reported version. Mixed import styles ( |
Linux ships the same shape as Windows -- executable, backend/ and python/ side by side -- so the updater generalises rather than needing a second design. The platform-specific parts are now three small seams: the archive format, the executable name, and one new gate. Rust: - widen the updater's cfg gates from `windows` to `any(windows, linux)`, and replace extract_zip_archive with extract_update_archive, which uses zip on Windows and the existing extract_tar_archive on Linux - APP_EXE_NAME so the swap and the leftover sweep stop hardcoding StemDeck.exe - stop_backend_and_wait now sends SIGTERM and waits before escalating on unix, matching what stop_backend already does on window close - new app_root_is_writable gate: packaging/linux/install.sh offers a global install into /opt/stemdeck, which is root-owned while the app runs as the user. check_app_update declines up front rather than failing part way through a swap. Windows portable installs are user-writable by construction, but the probe is cheap and honest on both. tar rather than zip on Linux is deliberate: it preserves the executable bit. A zip would land StemDeck without +x and the relaunch after an update would fail with a permission error. Packaging (scripts/linux/make-portable.sh): - write python/runtime-version.json using the same uv.lock + interpreter major.minor formula as the Windows script, so the compatibility gate behaves identically on both - bring the strip to parity: stdlib test/, per-package test/tests, a post-strip import check, and a final __pycache__ sweep after the last interpreter run - PUBLISH_UPDATER_ASSETS=1 emits the slim app-layer tarball, its checksum and the runtime marker; wired into the CPU build in linux-release.yml since StemDeck and backend/ are identical between both variants Frontend: updaterAssetNames() maps the platform to its asset names, and the wiring is gated on that rather than on os === "windows". macOS is deliberately still excluded, and the comments now say why rather than just that it is: backend_dir() resolves the backend inside the downloaded runtime pack rather than the .app, so its app layer is a different thing and the existing runtime-pack updater already covers most of it. Verified: both platforms compile clean with no new clippy warnings, 42 tests on Windows and 43 on Linux (the extra one is the read-only-root gate, which is meaningless on Windows). The app-layer archive was round-tripped on Linux to confirm it contains exactly StemDeck + backend/, that python/ does not leak into it, that the executable bit survives, and that replacing a running binary works. Not yet run end to end against a real Linux release.
Linux support added (
|
| Windows | Linux (WSL) | |
|---|---|---|
| clippy | clean, no new warnings | clean, no new warnings |
| tests | 42 pass | 43 pass |
The extra Linux test is the read-only-root gate, which is meaningless on Windows and correctly skipped there.
The app-layer archive was round-tripped on Linux to confirm it contains exactly StemDeck + backend/, that python/ does not leak into it, that the executable bit survives extraction, and that replacing a running binary works.
Not covered
A full make-portable.sh run end to end against a real Linux release. The archive mechanics and both compilers are verified; the complete packaging path is not. That is the same state Windows was in before I built it.
Worth noting in context of the audit: no Rust compiler pass ran in CI for this commit either. windows-check.yml and macos-check.yml are workflow_dispatch only, so all of the above rests on local verification. A paths: filter on desktop/src-tauri/** would close that.
macOS
Still excluded, and the comments now explain why rather than just stating it: backend_dir() resolves the backend inside the downloaded runtime pack rather than the .app, so its app layer is a different thing entirely and the existing runtime-pack updater already covers much of it.
Two gaps that would each have undermined the update flow on release day. The update check polled /releases/latest, which GitHub defines as the most recent NON-PRERELEASE, non-draft release. Ship a version with the pre-release box ticked and it becomes invisible: no notification, no update button, on any platform, with nothing in the logs to explain it. StemDeck has always published even its alphas as normal releases (v0.8.0-alpha.17 has prerelease=false), which is the only reason this has not bitten yet -- it was a trap waiting on someone ticking a box. Now polls the releases list and takes the newest non-draft, so it is correct either way. Drafts stay excluded: they are already invisible unauthenticated, and a maintainer should not be offered a release whose assets do not exist yet. windows-check.yml and macos-check.yml now also run on pull requests that touch desktop/src-tauri/**, not workflow_dispatch only. This PR added roughly 600 lines of mostly cfg-gated Rust across two commits and every CI check passed without compiling a single line of it; the comment at the top of windows-check.yml notes that exact gap already shipped a broken Windows build in v0.11.1's first release attempt. Scoped by path so the self-hosted runners see no extra load from the majority of PRs, which never go near src-tauri. This also gets the macOS branch compiled for the first time. Local verification covered Windows and Linux, so the cfg(not(any(windows, linux))) arm of the three updater commands has never been near a compiler.
The update-check stub returned a single release object, which was right for /releases/latest. The app now polls the releases list so a pre-release is still seen, so the fixture has to return an array or checkForUpdate bails and the release card never appears. Caught by frontend-e2e on the previous commit, which is the suite doing exactly its job: the only assertion that covers this path is report-failure.spec.mjs:98, and it went red immediately.
All 12 checks green, including Rust for the first time
Both The macOS run matters beyond the gate itself: local verification only covered Windows and Linux, so the Two bugs this roundThe pre-release trap. The update check polled A regression I introduced, caught by the suite. That API change broke Worth noting given the audit's finding on the test suite: this is the second time in this PR that an existing test caught something real. The suite is worth keeping honest. |
French is a complete table, not a partial one: 435 keys, the same set German
and Portuguese carry (English's 443 minus the ten Polish-only .few/.many forms
and the bare upload.skippedFiles, plus singular forms for the three
playlist.skip.* families). French takes the one/other buckets, so plural()
needs no change.
Verified with the checks from .claude/rules/i18n.md: the drift check reports
clean, and separately there are zero {placeholder} mismatches and zero HTML tag
mismatches against English. The 27 strings identical to English are genuinely
identical in French (Piano, Solo, Transport, Position, LUFS, Standard, Port,
the brand names, CUDA (NVIDIA), MPS (Apple Silicon)).
Separately: the runtime id was being computed from the raw bytes of uv.lock, so
a Windows checkout with core.autocrlf=true hashed CRLF and Linux hashed LF, and
the same lockfile produced two different ids -- caught by building the Linux
package and seeing py3.12-d74d6ef80c5e9d1f where Windows had produced
py3.12-dbda45e38e1044cf. Each platform stayed self-consistent so the gate still
worked, but the id would shift spuriously if a runner's autocrlf ever changed,
silently declining app-only updates that were in fact compatible. Both scripts
now hash the content with newlines normalised; PowerShell, bash and a reference
Python implementation all agree on d74d6ef80c5e9d1f.
Per .claude/rules/unraid-template-version.md this is an explicit decision each time, not a default. Confirmed for this release. The 0.14.0 GHCR image is published by docker-publish.yml when the release is created, so the tag exists shortly after this lands.
Merging11 of 12 checks green, including windows-check on the head commit. The one red is macos-check, and it is a stalled self-hosted runner rather than a defect:
What shippedWindows and Linux both get a click-initiated update. An accent "Update now" pill sits beside Download, which stays as the escape hatch. The check is automatic; download and apply are each an explicit click, and it never applies silently or interrupts a running job. Verified on real hardware, not asserted:
Nine real bugs found by building and running rather than reading: stale version after update; CSP silently blocking the whole feature; the Tauri ACL refusing Measured: 18,143 → 16,056 files, 883 → 850 MB. The update path itself is 5 MB / 123 files instead of 284 MB / 16,056. Zero new test failures: identical 14 on French added as a complete table (435 keys), drift clean across all 8 languages with zero placeholder or HTML-tag mismatches. |
1 participant
Closes #421 (as far as code goes; see Not yet verified before merging).
Integration branch for the next release. I will likely push more features here before this merges.
Why
#421 asked for Python embedded in a single EXE so updating would not mean copying ~20k loose files over an existing install. A onefile EXE is not viable for this stack: onefile modes re-extract the whole multi-GB payload on every launch, and torch/onnxruntime fight frozen-import hooks. So this fixes the root cause instead. Ship less, and stop making users hand-copy a full zip for a release that only changed app code.
Leaner package
scripts/windows/make-portable.ps1:-StripVenvopt-in gate was a silent regression risk: nothing stopped a future workflow edit from shipping the unstripped venv with no error.base/Lib/testand per-packagetest/testsdirs..dist-info/RECORD. pip needs it to replace a package, andinstall_cuda_torchpip-installs into this venv on every NVIDIA machine at first run. Removing it producesFailed to uninstall ... missing RECORD file, which I hit for real while working on this.Updater
New Tauri commands
installed_runtime_id,download_app_update,apply_app_update, built on the existing runtime-pack download/verify primitives rather thantauri-plugin-updater(that plugin expects to replace an installed NSIS/MSI bundle; this project ships a hand-assembled portable folder, andbundle.targetsis["app"]).Opt-in. The launch-time check is unchanged. Download and apply are each an explicit click. It never auto-applies and never interrupts a running job.
It replaces
StemDeck.exeandbackend/only.python/is never touched. An NVIDIA install rewrites that directory with CUDA torch at first run, andtorchDeviceSettled(desktop/ui/setup.js:246) skipsensure_torch_deviceonce the device is alreadycuda, so swapping the directory would silently drop that machine back to CPU with no recovery path. I had this wrong in the first draft and caught it on review.The runtime id is a compatibility gate, not a download trigger. It is derived from
uv.lockplus the interpreter major/minor, so it is stable across releases that do not change dependencies and identical for the CPU and NVIDIA variants. If a release changed the Python dependency set, the updater stands down and points at the full download. Only 19 of the last 200 commits touchuv.lock, so the fast path covers roughly 90% of releases and the rest fall back to today's behavior.apply_app_updatestages and validates everything before any destructive rename, stops the backend synchronously first (the existingstop_backendhands the kill to a thread and returns before the process dies, which would have made every update fail on Windows), and retries renames past transient AV/indexer handles.Known gap, documented in code: the two exe renames are back to back but not atomic. A hard crash in that window leaves
StemDeck.exe.oldneeding a manual rename. Closing it properly needs a bootstrap launcher that is never itself replaced, which is out of scope here.CI
Publishes
StemDeck-Windows-x64-app.zip, its.sha256, and-runtime-version.jsonalongside the unchanged full zips. Fresh installs are completely unaffected.i18n
The 5 new strings are translated into all 7 language tables, not just English.
t()falls back to English silently, so an English-only key looks correct in testing and ships untranslated text to six locales.Verified
JS syntax and all four JS suites pass, PowerShell parses,
ruff checkandruff format --checkclean. Two new unit tests pin the JSON contract between the PowerShell writer and the Rust reader.Not yet verified
Nothing here has run end to end against a real release. Before merging:
data/survives,portable.txt/cpu-onlystill gate correctly, and an NVIDIA install still reportsdemucs_device: cudaafter updating.uv.lockdiffers.*.old/_update_app.tmpsweep recovers.Note that
/releases/latestexcludes pre-releases, so testing needs either a full release or a locally served fake release manifest.Also outstanding per the repo's own rule: decide which image version the Unraid template should pin before this lands on
main.