🐛 package cache can become unstable across concurrent utoo installs #3256

What happened?

We observed install failures when multiple utoo/musejs install processes share the same npm cache directory. The production symptom is that cache entries under /opt/npm-cache/<name>/<version>/package/package.json can be left as 0-byte files, and later installs fail while cloning packages from cache into node_modules.

Example zero-byte cache files found on one affected machine:

/opt/npm-cache/@babel/plugin-transform-react-jsx-source/7.29.7/package/package.json
/opt/npm-cache/cookie-es/3.1.1/package/package.json
/opt/npm-cache/d3-time-format/4.1.0/package/package.json
/opt/npm-cache/date-fns/4.4.0/package/package.json
/opt/npm-cache/recharts-scale/0.4.5/package/package.json
/opt/npm-cache/tapable/2.3.3/package/package.json

The install failure log has this shape:

WARN hardlink failed for /opt/npm-cache/.../package/... -> .../node_modules/...: No such file or directory (os error 2); falling back to copy
ERROR Failed to install packages: clone_dir /opt/npm-cache/.../package -> .../node_modules/...: No such file or directory (os error 2)

This points to readers treating a cache slot as resolved while another process is still mutating, retrying, cleaning, or being killed while writing the same slot.

Reproduction notes

A draft PR with a release-based GitHub Actions stress workflow was created here:

• PR: test(pm): reproduce downloader cache race on linux #3255
• Run: https://github.com/utooland/utoo/actions/runs/29839975417

That workflow downloads the released utoo-v1.1.4 binary and starts many concurrent utoo install processes with a shared cache. It did not reproduce final 0-byte package/package.json on GitHub-hosted Ubuntu after 15 attempts, but it did reproduce the same error family:

WARN hardlink failed ... No such file or directory
ERROR Failed to install packages: clone_dir ... No such file or directory
ERROR Failed to install packages: Cannot find valid source directory in ".../utoo-npm-cache/..."

The uploaded cache summaries also show multiple temporary cache directories such as date-fns/.tmpXXXX/package/package.json while final resolved slots exist, which suggests concurrent cache slot lifecycle races are still visible through the release binary.

Expected result

A cache slot should only become visible/resolved to readers after the package content is completely and durably written. Concurrent installs sharing the same cache should not allow readers to observe partially written, stale, cleaned, or still-mutating cache directories.

Current hypothesis

downloader currently has in-process de-duplication, but no cross-process cache-slot lock. With multiple install processes sharing /opt/npm-cache, this kind of interleaving is possible:

1. Process A and process B both decide /opt/npm-cache/<name>/<version> is not resolved.
2. One process writes _resolved or otherwise exposes the final slot while another process is still writing/retrying/cleaning the same package version.
3. A reader starts clone_dir from the cache slot.
4. Files or directories disappear/change during hardlink/copy, causing ENOENT.
5. If a process is killed after opening/truncating a file but before writing content, the final cache can retain a 0-byte file.

Possible fixes

• Add a cross-process file lock per cache slot, e.g. lock by <cache>/<name>/<version>.
• Write package contents into a same-filesystem temp directory first, then atomically rename/promote to the final slot.
• Create _resolved only after promotion and after validating key files like package/package.json exist and are non-empty.
• When a reader sees _resolved, optionally validate the cache slot before cloning.
• Avoid stale-cache cleanup on a final slot that another process may be cloning.

Related code

• crates/pm/src/util/downloader.rs: per-process DOWNLOAD_LOCKS; unpack/write path; _resolved marker.
• crates/pm/src/util/cloner.rs: clone from cache to node_modules; hardlink/copy fallback can hit ENOENT when source/target changes concurrently.
• crates/pm/src/service/install.rs: install concurrency and cache path use.