Skip to content

engine: default CARGO_TARGET_DIR/TMPDIR off the data volume - #15

Open
Morgandri1 wants to merge 1 commit into
mainfrom
sdk/scratch-dirs-off-data-volume
Open

engine: default CARGO_TARGET_DIR/TMPDIR off the data volume#15
Morgandri1 wants to merge 1 commit into
mainfrom
sdk/scratch-dirs-off-data-volume

Conversation

@Morgandri1

Copy link
Copy Markdown
Owner

Summary

PM ruling: this is both the ENOSPC root cause (a cargo build --workspace on the small persistent /data volume fills it — hit live by me and by API independently this session) and the biggest measured contributor to the reported 7-8GB memory number (page cache from that build I/O counts against the project's cgroup).

Every spawned child now gets CARGO_TARGET_DIR and TMPDIR under /tmp/wheel-scratch/<project_id>/, created 0700 and checked the same way cargo_home() already is (refuses a symlink, refuses a mode that won't stick). Unlike CARGO_HOME, which deliberately stays on the project's data dir (a small, worth-persisting registry cache), this is disposable build output that runs to gigabytes and belongs off the persistent volume.

Refactored the create/verify/tighten dance out of cargo_home() into a shared private_dir() helper so the same security property (private to this project's uid, belt-and-braces against a symlinked shared directory) isn't re-derived per scratch dir.

Test plan

  • cargo build --workspace — clean
  • cargo test -p wheel-engine — 278 passed, including 3 new tests: the_scratch_dirs_are_off_the_data_volume_and_private, two_projects_get_different_scratch_dirs, the_child_actually_receives_the_off_volume_scratch_dirs
  • cargo clippy -p wheel-engine --all-targets -- -D warnings — clean
  • cargo fmt -p wheel-engine -- --check — clean
  • Mutation-checked: reverted the CARGO_TARGET_DIR env line, confirmed the_child_actually_receives_the_off_volume_scratch_dirs fails; restored

@vercel

vercel Bot commented Sep 6, 2026

Copy link
Copy Markdown

Deployment failed for project wheel with the following error:

Resource is limited - try again in 24 hours (more than 100, code: "api-deployments-free-per-day").

Learn More: https://vercel.com/morgandri1s-projects?upgradeToPro=build-rate-limit

… pattern as CARGO_HOME

PM ruling: this is both the ENOSPC root cause (a cargo build --workspace
on the small persistent /data volume fills it) and the biggest measured
contributor to the reported 7-8GB memory number (page cache from that
build I/O counts against the project's cgroup) -- and it was being
independently rediscovered and self-fixed by whichever agent hit the
ENOSPC first, rather than fixed once here.

Every spawned child now gets CARGO_TARGET_DIR and TMPDIR under
/tmp/wheel-scratch/<project_id>/, created 0700 and checked the same way
cargo_home() already is (refuses a symlink, refuses a mode that will not
stick) -- unlike CARGO_HOME, which stays on the project's data dir on
purpose (it's a small, worth-persisting registry cache; this is
disposable build output that runs to gigabytes).

Refactored the create/verify/tighten dance out of cargo_home() into a
shared private_dir() so the same security property (private to this
project's uid, belt-and-braces against a symlinked shared directory)
does not have to be re-derived per scratch dir.
@Morgandri1
Morgandri1 force-pushed the sdk/scratch-dirs-off-data-volume branch from 966930c to 503bb4f Compare September 6, 2026 17:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant