Skip to content

Commit 17fe597

Browse files
authored
perf(fspy): use sparse Windows shared memory (#524)
## Motivation Avoid reserving disk space for the full Windows mapping while keeping record writes memory-mapped. ## Changes - Maps a sparse, delete-on-close temporary file through a named Windows section. - Uses a size-independent section name and opens another view using only the identifier. - Maps the complete section and derives its length from the mapped view. - Fails creation when the temporary volume does not support sparse files. - Keeps existing views valid after the owner closes its file and view. - Replaces linker inputs previously supplied by `shared_memory` with direct `windows-sys` imports. - Covers cross-process access, cleanup, sparse allocation, and changed environments.
1 parent 14807a2 commit 17fe597

11 files changed

Lines changed: 448 additions & 25 deletions

File tree

CHANGELOG.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,6 @@
11
# Changelog
22

3+
- **Improved** Windows file-access tracking now uses sparse temporary backing files where supported, avoiding upfront allocation of the full backing file on disk ([#524](https://github.com/voidzero-dev/vite-task/pull/524)).
34
- **Fixed** Automatic file-access tracking on Linux now works in containers and Kubernetes runners with limited `/dev/shm` space ([#353](https://github.com/voidzero-dev/vite-task/issues/353), [#523](https://github.com/voidzero-dev/vite-task/pull/523)).
45
- **Fixed** Windows automatic input tracking now records executable image reads when process creation performs the lookup inside `NtCreateUserProcess` ([#518](https://github.com/voidzero-dev/vite-task/pull/518)).
56
- **Fixed** Failures while waiting for a started task process to exit no longer incorrectly say the process failed to spawn ([#515](https://github.com/voidzero-dev/vite-task/pull/515)).

Cargo.lock

Lines changed: 2 additions & 0 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

Cargo.toml

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -168,6 +168,7 @@ wax = "0.7.0"
168168
which = "8.0.0"
169169
widestring = "1.2.0"
170170
winapi = "0.3.9"
171+
windows-sys = "0.61"
171172
winsafe = { version = "0.0.27", features = ["kernel"] }
172173
xxhash-rust = { version = "0.8.15", features = ["const_xxh3"] }
173174
ntest = "0.9.5"

crates/fspy_preload_windows/Cargo.toml

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -23,6 +23,7 @@ winapi = { workspace = true, features = [
2323
"memoryapi",
2424
"std",
2525
] }
26+
windows-sys = { workspace = true, features = ["Win32_Foundation", "Win32_UI_Shell"] }
2627
winsafe = { workspace = true }
2728

2829
[target.'cfg(target_os = "windows")'.dev-dependencies]

crates/fspy_preload_windows/src/windows/winapi_utils.rs

Lines changed: 6 additions & 13 deletions
Original file line numberDiff line numberDiff line change
@@ -6,8 +6,8 @@ use widestring::{U16CStr, U16Str};
66
use winapi::{
77
ctypes::c_long,
88
shared::{
9-
minwindef::{BOOL, FALSE, HLOCAL, MAX_PATH, ULONG},
10-
ntdef::{HANDLE, HRESULT, PCWSTR, PWSTR, UNICODE_STRING},
9+
minwindef::{BOOL, FALSE, MAX_PATH},
10+
ntdef::{HANDLE, PWSTR, UNICODE_STRING},
1111
winerror::{NO_ERROR, S_OK},
1212
},
1313
um::{
@@ -18,6 +18,10 @@ use winapi::{
1818
},
1919
},
2020
};
21+
use windows_sys::Win32::{
22+
Foundation::LocalFree,
23+
UI::Shell::{PATHCCH_ALLOW_LONG_PATHS, PathAllocCombine},
24+
};
2125
use winsafe::{GetLastError, co};
2226

2327
pub fn ck(b: BOOL) -> winsafe::SysResult<()> {
@@ -99,16 +103,6 @@ pub const fn access_mask_to_mode(desired_access: ACCESS_MASK) -> AccessMode {
99103
}
100104
}
101105

102-
unsafe extern "system" {
103-
fn LocalFree(hmem: HLOCAL) -> HLOCAL;
104-
fn PathAllocCombine(
105-
pszpathin: PCWSTR,
106-
pszmore: PCWSTR,
107-
dwflags: ULONG,
108-
ppszpathout: *mut PWSTR,
109-
) -> HRESULT;
110-
}
111-
112106
pub struct HeapPath(PWSTR);
113107
impl HeapPath {
114108
#[must_use]
@@ -125,7 +119,6 @@ impl Drop for HeapPath {
125119
}
126120

127121
pub fn combine_paths(path1: &U16CStr, path2: &U16CStr) -> winsafe::SysResult<HeapPath> {
128-
const PATHCCH_ALLOW_LONG_PATHS: ULONG = 0x0000_0001;
129122
let mut out = std::ptr::null_mut();
130123
// SAFETY: FFI call to PathAllocCombine with valid null-terminated wide string pointers
131124
let hr = unsafe {

crates/fspy_shm/Cargo.toml

Lines changed: 12 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -7,7 +7,7 @@ license.workspace = true
77
publish = false
88
rust-version.workspace = true
99

10-
[target.'cfg(not(target_os = "linux"))'.dependencies]
10+
[target.'cfg(not(any(target_os = "linux", target_os = "windows")))'.dependencies]
1111
shared_memory = { workspace = true, features = ["logging"] }
1212

1313
[target.'cfg(target_os = "linux")'.dependencies]
@@ -19,6 +19,17 @@ tokio-util = { workspace = true }
1919
tracing = { workspace = true }
2020
uuid = { workspace = true, features = ["v4"] }
2121

22+
[target.'cfg(target_os = "windows")'.dependencies]
23+
uuid = { workspace = true, features = ["v4"] }
24+
windows-sys = { workspace = true, features = [
25+
"Win32_Foundation",
26+
"Win32_Security",
27+
"Win32_Storage_FileSystem",
28+
"Win32_System_IO",
29+
"Win32_System_Ioctl",
30+
"Win32_System_Memory",
31+
] }
32+
2233
[dev-dependencies]
2334
ctor = { workspace = true }
2435
subprocess_test = { workspace = true }

crates/fspy_shm/README.md

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -34,5 +34,6 @@ The channel hides that difference with its lock file. [`ChannelConf::sender`](..
3434
Each platform keeps its implementation rationale beside its source:
3535

3636
- [Linux: `memfd` with a descriptor broker](src/linux/README.md)
37+
- [Windows: sparse file-backed named mapping](src/windows/README.md)
3738

38-
At this point in the stack, Windows and macOS still delegate mapping creation and opening to the [`shared_memory`](https://crates.io/crates/shared_memory) crate.
39+
At this point in the stack, macOS still delegates mapping creation and opening to the [`shared_memory`](https://crates.io/crates/shared_memory) crate.

crates/fspy_shm/src/lib.rs

Lines changed: 10 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -1,18 +1,22 @@
11
#![doc = include_str!("../README.md")]
22

3-
#[cfg(not(target_os = "linux"))]
3+
#[cfg(not(any(target_os = "linux", target_os = "windows")))]
44
use std::io;
55

66
#[cfg(target_os = "linux")]
77
mod linux;
8+
#[cfg(target_os = "windows")]
9+
mod windows;
810

911
#[cfg(target_os = "linux")]
1012
pub use linux::{Shm, create, open};
11-
#[cfg(not(target_os = "linux"))]
13+
#[cfg(not(any(target_os = "linux", target_os = "windows")))]
1214
use shared_memory::{Shmem, ShmemConf};
15+
#[cfg(target_os = "windows")]
16+
pub use windows::{Shm, create, open};
1317

1418
/// An owned shared-memory mapping.
15-
#[cfg(not(target_os = "linux"))]
19+
#[cfg(not(any(target_os = "linux", target_os = "windows")))]
1620
pub struct Shm {
1721
inner: Shmem,
1822
}
@@ -26,11 +30,9 @@ pub struct Shm {
2630
/// # Errors
2731
///
2832
/// Returns an error if the platform cannot create or map the region.
29-
#[cfg(not(target_os = "linux"))]
33+
#[cfg(not(any(target_os = "linux", target_os = "windows")))]
3034
pub fn create(size: usize) -> io::Result<Shm> {
3135
let conf = ShmemConf::new().size(size);
32-
#[cfg(target_os = "windows")]
33-
let conf = conf.allow_raw(true);
3436

3537
let inner = conf.create().map_err(io::Error::other)?;
3638
Ok(Shm { inner })
@@ -45,17 +47,15 @@ pub fn create(size: usize) -> io::Result<Shm> {
4547
/// # Errors
4648
///
4749
/// Returns an error if the mapping does not exist or cannot be mapped.
48-
#[cfg(not(target_os = "linux"))]
50+
#[cfg(not(any(target_os = "linux", target_os = "windows")))]
4951
pub fn open(id: &str) -> io::Result<Shm> {
5052
let conf = ShmemConf::new().os_id(id);
51-
#[cfg(target_os = "windows")]
52-
let conf = conf.allow_raw(true);
5353

5454
let inner = conf.open().map_err(io::Error::other)?;
5555
Ok(Shm { inner })
5656
}
5757

58-
#[cfg(not(target_os = "linux"))]
58+
#[cfg(not(any(target_os = "linux", target_os = "windows")))]
5959
#[expect(clippy::len_without_is_empty, reason = "shared-memory mappings are always non-empty")]
6060
impl Shm {
6161
/// Returns this mapping's opaque platform identifier.
Lines changed: 23 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,23 @@
1+
# Windows backend
2+
3+
The Windows backend maps a sparse temporary file and gives the mapping a Windows section name. Another process opens the same section using only that name. Creating the mapping does not reserve its full size in system commit or disk space.
4+
5+
## Options considered
6+
7+
| Option | Decision |
8+
| ------------------------------------------------------------ | ----------------------------------------------------------------------------------------------------------------------------- |
9+
| Paging-file-backed section | Rejected because the section's full size is charged against system commit. |
10+
| Reserved section with incremental `VirtualAlloc(MEM_COMMIT)` | Rejected because writers would need to coordinate when committing more pages. |
11+
| Sparse temporary file with a named section | Selected. Disk blocks are allocated only for written ranges, and another process can open the mapping using the section name. |
12+
13+
`FILE_ATTRIBUTE_TEMPORARY` tells Windows to keep file data in memory when possible, but Windows may still write dirty pages to disk under memory pressure. Creation fails if the temporary volume does not support sparse files.
14+
15+
## Why not `shared_memory`
16+
17+
`shared_memory` creates and extends a regular file before fspy can mark it sparse. It therefore cannot ensure that untouched ranges use no disk blocks.
18+
19+
`shared_memory` also uses the file path to identify the mapping. Fspy uses the section name, so another process does not need the creator's temporary-file path.
20+
21+
## Lifetime semantics
22+
23+
Dropping the owner unmaps its view and closes the delete-on-close backing file. Existing views keep the section alive. The section name may remain openable during that period. `ChannelConf::sender` checks the receiver's lock file first to prevent new senders after the receiver has shut down.

0 commit comments

Comments
 (0)