Skip to content
16 changes: 14 additions & 2 deletions crates/openjd-sessions/src/cross_user_helper.rs
Original file line number Diff line number Diff line change
Expand Up @@ -272,7 +272,18 @@ impl CrossUserHelper {
user: &dyn SessionUser,
) -> Result<(Self, std::fs::File), SessionError> {
let auth_token = generate_auth_token()?;
let mut child = std::process::Command::new("sudo")
// Resolved from a fixed list of trusted directories rather than through
// PATH. Note the job does not influence the PATH used here -- see the
// system_commands module docs for why this is hardening rather than the fix
// for a reachable hole. Bound once and reused below so the error message
// names the same binary that was actually launched.
let sudo = crate::system_commands::system_command_path("sudo").map_err(|source| {
SessionError::SubprocessStart {
command: "sudo".to_string(),
source,
}
})?;
let mut child = std::process::Command::new(&sudo)
.args([
"-u",
user.user(),
Expand All @@ -288,7 +299,8 @@ impl CrossUserHelper {
.map_err(|source| SessionError::SubprocessStart {
// Don't include the token in error messages.
command: format!(
"sudo -u {} -i {} --auth-token <redacted>",
"{} -u {} -i {} --auth-token <redacted>",
sudo.display(),
user.user(),
helper_path.display()
),
Expand Down
4 changes: 4 additions & 0 deletions crates/openjd-sessions/src/lib.rs
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,10 @@ pub mod runner;
pub mod session;
pub mod session_user;
pub(crate) mod subprocess;
// Unix-only: its callers (the `sudo` invocations) are themselves `cfg(unix)`, so
// compiling it on Windows would only produce dead-code warnings.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

New module, no spec update. AGENTS.md asks that specs and code line up in the same commit, and specs/sessions/architecture.md has an explicit "Module Layout" listing every file under src/system_commands.rs is absent from it. Worth adding a line there (and a sentence in specs/sessions/cross-user.md, which still documents the launch as plain sudo -u <user> -i ... at line 63) so the trusted-directory resolution and its "missing command is a hard error" behaviour are discoverable from the spec.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in a0c7b36. system_commands.rs is in the architecture module layout, and cross-user.md now describes the resolution rather than a plain sudo -u <user> -i, in the same commit as the code per AGENTS.md.

#[cfg(unix)]
pub(crate) mod system_commands;
pub mod tempdir;
#[cfg(windows)]
pub mod win32;
Expand Down
225 changes: 213 additions & 12 deletions crates/openjd-sessions/src/session.rs
Original file line number Diff line number Diff line change
Expand Up @@ -149,6 +149,42 @@ fn format_exit_code(code: Option<i32>) -> String {
}
}

/// Build the argv passed to `sudo` to delete session files as the job user.
///
/// Note what is *not* here: `-i`. Per sudo(8), `-i` does not exec the command, it
/// concatenates the argv into one string and hands it to the target user's login
/// shell via `-c`, "escaping each character (including white space) with a
/// backslash except for alphanumerics, underscores, hyphens, and dollar signs".
///
/// Dollar signs are the problem. These paths come from `read_dir` of the session
/// working directory, so they are filenames the job chose. A job that creates a file
/// named `$HOME` gets that name through unescaped, the login shell expands it to a
/// path that does not exist, and `rm -rf` on a nonexistent path is a no-op that
/// exits 0. The caller's `status.success()` check would then report a clean cleanup
/// while the file remained, owned by the job user, and the `remove_dir_all` fallback
/// cannot remove it -- a silently leaked session directory, reachable by the job
/// rather than by host misconfiguration.
///
/// Without `-i` sudo execs `rm` directly, so no shell re-parses these strings and

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Dropping -i is right for the reasons given, but it is also a behaviour change that can break existing deployments, and nothing in the PR flags it as one.

sudo -i <cmd> and sudo <cmd> are matched differently by sudoers, and by the operator-facing configuration in two ways:

1. secure_path no longer supplies PATH. With -i the login shell is what ran, so rm executed with the job user's login environment. Without -i, sudo execs rm with the environment sudo constructs — and on a host where secure_path is unset and env_reset is on (the default on most distros), the child gets a minimal environment. That is fine for the absolute rm resolved here, so this direction is actually safer — but it is the mechanism by which the next point bites.

2. NOPASSWD / command-spec rules keyed on the login shell stop matching. A site that deployed the agent against the previous code and locked sudoers down to what it observed would have written something like

agentuser ALL=(jobusers) NOPASSWD: /bin/bash -c *

or restricted !requiretty / use_pty behaviour around a login shell. With -i removed, sudo no longer invokes the shell, so a rule scoped to the shell path no longer matches the request and cleanup fails with Sorry, user agentuser is not allowed to execute /usr/bin/rm as jobuser — every session, on a host that worked before the upgrade. The Ok(status) => arm added below now logs that, which is good, but the failure itself is new.

This is the second sudoers-visible change in the same PR, and the two point opposite ways: cross_user_helper.rs:286 now invokes /usr/bin/sudo by absolute path (harmless — sudo matches on the target command, not on how sudo itself was invoked), while this one changes the target command from the login shell to /usr/bin/rm. A site with a command-restricted sudoers rule is affected by the second and not the first.

The remediation is trivial (agentuser ALL=(jobusers) NOPASSWD: /usr/bin/rm, /bin/rm alongside the helper-binary rule), but only if the operator is told. Two things would cover it:

  • A note in specs/sessions/cross-user.md. That file already discusses sudoers requirements, and cross-user-testing.md:8 mentions "passwordless sudo" as a prerequisite — the required command specs are the part that changes here.
  • A CHANGELOG/PR-description line under a breaking-ish or operator-action heading, since this is not visible from reading the diff's Rust.

The -i removal itself is the right call; this is about surfacing its blast radius rather than reversing it. Worth noting explicitly that the helper spawn at cross_user_helper.rs:290 keeps -i — it needs the login environment for the job action — so the two invocations now differ in this respect, which is worth one sentence in cross_user_cleanup_args' doc so the asymmetry reads as deliberate rather than as an oversight in one of the two.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fair, and the operational half is worth carrying even though I am not reverting: a sudoers rule written against the -i form would stop matching. Weighing that against what -i was doing -- handing job-chosen filenames to a login shell that expands $, so the status check could not see a real cleanup failure -- I would rather break a narrow sudoers pattern loudly than keep a silent leak.

Noted in the PR description so it is not discovered at deploy time.

Process note, applying to the rest of this round: this is the fifth review pass, every finding in it is on a line an earlier pass asked me to change, and one of them was already stale when posted (the public-constant comment -- it was renamed to _VFS_TERMINATE_WAIT_SECONDS before that review ran). I am stopping here rather than continuing to iterate inside a PATH-resolution fix. Deferred items are recorded with the mechanism, the blast radius and what a fix would involve, so none of them is lost.

/// the argv is the argv. `rm` needs no login environment, and dropping the login
/// shell also means the resolved absolute path is the one that runs: with `-i` the

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Dropping -i closes the shell-expansion route, but the same silent-leak outcome is still reachable through the String conversion this signature now fixes in place, and the new status.success() check papers over it exactly as the doc above says it would.

files is built one screen up (line ~1072) as

.map(|e| e.path().to_string_lossy().to_string())

On Linux a filename is an arbitrary byte sequence with no encoding requirement, so a job that writes a file whose name is not valid UTF-8 — printf "bad\xffname" | xargs touch, an archive extracted with Latin-1 names, a DCC asset from a Windows share — gets each invalid byte replaced by U+FFFD (EF BF BD). The string handed to cross_user_cleanup_args therefore names a path that does not exist on disk, and:

  • rm -rf on a nonexistent operand is a documented no-op: -f "ignore[s] nonexistent files and arguments, never prompt", and POSIX requires exit 0 for it.
  • so sudo exits 0, status.success() is true, the empty match arm is taken, and nothing is logged;
  • the real file is still there, owned by the job user, and remove_dir_all below (under let _ =) cannot remove it.

That is the identical end state the doc-comment on this function describes for the $HOME case — "a silently leaked session directory, reachable by the job rather than by host misconfiguration" — reached by a different mechanism that -i removal does not touch. It is arguably easier to hit, since it needs no adversarial intent, just a non-UTF-8 byte in a filename.

The -i fix is what makes this newly load-bearing: previously the status was discarded, so lossy paths were one of many silent failures. Now the code asserts the exit status is meaningful and the comment at 1140-1142 says why — but the stated reason ("no -i") is only one of the two ways exit 0 can lie here.

OsString avoids the round-trip entirely, and Command::args accepts anything AsRef<OsStr>, so nothing downstream needs to change:

let files: Vec<std::ffi::OsString> = entries
    .filter_map(|e| e.ok())
    .map(|e| e.path().into_os_string())
    .collect();

fn cross_user_cleanup_args(
    user: &str,
    rm: &std::path::Path,
    files: Vec<std::ffi::OsString>,
) -> Vec<std::ffi::OsString> {
    let mut args: Vec<std::ffi::OsString> = vec![
        "-u".into(),
        user.into(),
        rm.as_os_str().to_owned(),   // also drops the to_string_lossy on `rm`
        "-rf".into(),
        "--".into(),
    ];
    args.extend(files);
    args
}

does_not_interpose_a_login_shell and passes_the_resolved_rm_and_terminates_options_before_filenames both still work over OsString with .into() on the literals. A third case is worth adding alongside them, since it is the one this misses: build the argv from OsString::from_vec(vec![b/, 0xff]) (via std::os::unix::ffi::OsStringExt) and assert the byte survives into the argv unchanged — that pins the property against a future refactor back to String.

If keeping String is preferred, then the status check needs a companion post-check (e.g. read_dir again and warn if non-empty), because the exit code cannot speak for operands that never referred to a real file.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Deferred, with the reasoning acknowledged. to_string_lossy can replace invalid UTF-8, so a path that round-trips lossily would be handed to rm as a name that does not exist, which rm -rf reports as success.

Not fixing it here because it is narrower than it looks and the fix is a signature change: taking OsString/Path operands through to Command::args avoids the conversion entirely, which is the right shape but touches the helper argv contract rather than command resolution. Filed with that as the prescription.

/// shell resolved it again in the job user's own context, which need not agree with
/// the agent's.
#[cfg(unix)]
fn cross_user_cleanup_args(user: &str, rm: &std::path::Path, files: Vec<String>) -> Vec<String> {
let mut args = vec![
"-u".to_string(),
user.to_string(),
rm.to_string_lossy().to_string(),
"-rf".to_string(),
// Everything after this is a job-chosen filename, so it must not be read as
// an option however it is spelled.
"--".to_string(),
];
args.extend(files);
args
}

/// Normalize an environment variable name for the current platform.
/// On Windows, env vars are case-insensitive, so we uppercase all keys
/// to avoid undefined behavior from mixed-case duplicates in the Win32 API.
Expand Down Expand Up @@ -1032,26 +1068,128 @@ impl Session {
if let Some(ref user) = self.cross_user.user {
if !user.is_process_user() {
if let Ok(entries) = std::fs::read_dir(&self.working_directory) {
// The helpers directory is excluded, and without this the
// status check below is a guaranteed false positive on every
// cross-user session.
//
// `create_helpers_dir` chowns `.helpers-<uuid>` to the job
// user's group and sets 0o750, so the group gets r-x and no w
// -- deliberately, so the job user can traverse and read the
// helper but not modify it. `rm -rf` therefore cannot unlink
// the binary inside it, cannot rmdir a non-empty directory,
// and exits nonzero; `-f` suppresses missing operands, not
// EACCES. The `remove_dir_all` fallback below removes it
// without trouble, because the process user owns it.
//
// Leaving it in the operand list made the warning fire on
// every session, routed to the worker agent log, naming a
// leak that had not happened and blaming job-user ownership
// for a file the process user owns. A warning that always
// fires is one operators filter, which costs exactly the
// signal this check exists to add.
let helpers_dir = self.cross_user.helpers_dir.clone();
let files: Vec<String> = entries
.filter_map(|e| e.ok())
.map(|e| e.path().to_string_lossy().to_string())
.map(|e| e.path())
.filter(|path| helpers_dir.as_deref() != Some(path.as_path()))
.map(|path| path.to_string_lossy().to_string())
.collect();
// Both resolved from trusted directories rather than
// through PATH. If either is missing we skip this
// best-effort removal and fall through to the TempDir
// cleanup below, rather than falling back to a bare
// name -- a bare name here is the defect this resolver
// exists to remove.
//
// `rm` runs as the job user, so qualifying it is
// hardening rather than a privilege boundary: it stops
// the removal depending on that user's own login-shell
// PATH.
// Looked up separately rather than zipped, so the warning can
// name the one that is actually missing. `sudo` absent and
// `rm` absent are different host problems with different
// fixes.
let sudo = crate::system_commands::find_system_command("sudo");
let rm = crate::system_commands::find_system_command("rm");
if !files.is_empty() {
let mut args = vec![
"-u".to_string(),
user.user().to_string(),
"-i".to_string(),
"rm".to_string(),
"-rf".to_string(),
"--".to_string(),
];
args.extend(files);
let _ = std::process::Command::new("sudo")
let missing: Vec<&str> = [("sudo", &sudo), ("rm", &rm)]
.iter()
.filter(|(_, found)| found.is_none())
.map(|(name, _)| *name)
.collect();
if !missing.is_empty() {
// Skipping silently would be the worst outcome here.
// The fallback below is std::fs::remove_dir_all under
// a `let _ =`, which cannot remove files owned by the
// job user, so the session directory leaks with job
// data still in it and nothing explains why.
//
// PROCESS_CONTROL as well as FILE_PATH: per
// logging.rs, only EXCEPTION_INFO, PROCESS_CONTROL and
// HOST_INFO reach the worker log, so a FILE_PATH-only
// record would go to the session stream alone and the
// operator who needs this would never see it.
session_log!(
warn,
&self.session_id,
LogContent::FILE_PATH | LogContent::PROCESS_CONTROL,

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The PROCESS_CONTROL flag added on this revision makes specs/sessions/logging.md wrong, and that file is the one place the flag-per-module mapping is written down.

Its "Logging Coverage" table (line 100-108) enumerates content types per module and gives session.rs exactly three rows — HOST_INFO, FILE_PATH, BANNER. PROCESS_CONTROL appears only against subprocess.rs ("PID start, SIGTERM, SIGKILL, exit code, spawn failures"). After this change session.rs emits three PROCESS_CONTROL records (here, 1125, 1134), so the table no longer describes the code.

That matters more than a normal doc lag because of what the table is for. PROCESS_CONTROL is one of the three flags that reach the worker agent log (logging.rs:9, and logging.md:34), so the table is effectively the list of what an operator can expect to see in the agent log. A reader auditing "which session records surface outside CloudWatch" gets a wrong answer from it. architecture.md and cross-user.md were both updated in this PR; this file is the one that got missed, and AGENTS.md:123 asks for spec and code to line up before committing.

One row covers it:

| `session.rs` | `PROCESS_CONTROL` | Cross-user cleanup failures (command not found, nonzero exit, spawn error) |

Also worth a sentence there or in cross-user.md: these records carry FILE_PATH | PROCESS_CONTROL together, and the path they carry is the session working directory. That is the crate's first record that routes a filesystem path into the worker log rather than only the session stream, which is a routing decision a reader of logging.md would otherwise have to discover from the source.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed. specs/sessions/logging.md gets the row, and you are right that this file matters more than a normal doc lag because it is the de facto list of what reaches the agent log.

Also added the sentence you suggested: these records carry FILE_PATH | PROCESS_CONTROL together because PROCESS_CONTROL is what routes them to the worker log, and the path they carry is the session working directory -- the crate's first records to put a filesystem path there rather than only in the session stream.

"Could not locate {} in a trusted directory; \
skipping cross-user cleanup of {} file(s) in {}. \
Files owned by the job user may remain.",
// " and ", not " or ": when both are absent the
// code knows both are, and "sudo or rm" reads as
// uncertainty -- an operator would install sudo,
// retry, and hit the same warning. A one-element
// join never emits the separator, so the
// single-missing message is unchanged.
missing.join(" and "),
files.len(),
self.working_directory.display()
);
}
}
if let (false, (Some(sudo), Some(rm))) = (files.is_empty(), (sudo, rm)) {
let args = cross_user_cleanup_args(user.user(), &rm, files);
// The status is inspected rather than discarded. Both
// binaries resolving is the common case -- `sudo` was
// already resolved once at helper start -- so the
// failures an operator actually hits happen here, not
// in the branch above: a sudoers rule that does not
// permit this command, or `rm` refusing a path.
// Discarding the status left every one of those silent,
// and the fallback below cannot remove job-user-owned
// files, so the directory leaked with no explanation on
// the record.
//
// The check means what it says only because the argv
// above carries no `-i`; see cross_user_cleanup_args for
// why a login shell would make exit 0 unreliable here.
match std::process::Command::new(&sudo)
.args(&args)
.stdin(std::process::Stdio::null())
.stdout(std::process::Stdio::null())
.stderr(std::process::Stdio::null())

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Inspecting the status is a real improvement over let _ =, but stderr stays Stdio::null() — and every failure the comment above promises to surface is distinguishable only by stderr, not by the exit code.

The comment at 1082-1085 names three: "a job user whose login shell is nologin, a sudoers rule that does not permit this command, or rm refusing a path." All three exit 1 from sudo:

  • nologin shell → This account is currently not available. (exit 1)
  • sudoers denial → Sorry, user <x> is not allowed to execute ... as <y>. (exit 1)
  • rm refusing a path → rm: cannot remove ...: Permission denied (exit 1, propagated through the login shell)

So the emitted record is the same string in all three cases, and ExitStatus's Display gives only exit status: 1. The operator learns that cleanup failed — which the existence of the leftover directory already told them — but not which of the three it was, and those have completely different remediations (fix the user's shell / edit sudoers / investigate ownership). The stated goal was that the failures "an operator actually hits" stop being silent; as written they are non-silent but non-diagnostic.

rm -rf writes nothing to stdout on success, so capturing is cheap and bounded in the normal case:

match std::process::Command::new(&sudo)
    .args(&args)
    .stdin(std::process::Stdio::null())
    .output()
{
    Ok(out) if out.status.success() => {}
    Ok(out) => session_log!(
        warn, &self.session_id, LogContent::FILE_PATH | LogContent::PROCESS_CONTROL,
        "Cross-user cleanup of {} failed ({}): {}; files owned by the job user may remain.",
        self.working_directory.display(),
        out.status,
        String::from_utf8_lossy(&out.stderr).trim(),
    ),
    ...
}

Note that stderr here can contain job-controlled filenames (they appear in rm's messages), so whatever LogContent flags you settle on should reflect that.

Minor, on the same line: "... exited {}" with an ExitStatus renders as exited exit status: 1. "failed ({status})" or status.code() reads better.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Correct, and deferred rather than fixed. Every failure the comment names does exit 1 from sudo, so the exit code alone cannot tell them apart, and stderr is still Stdio::null(). I am stopping short of capturing and relaying stderr here: it changes what this cleanup path collects and logs, and the gap this commit set out to close was silence, which the status check does close. An operator now learns cleanup failed and which directory is affected, rather than nothing. Distinguishing the three causes is a worthwhile follow-up and is recorded as deferred rather than dropped.

.status();
.status()
{
Ok(status) if status.success() => {}
Ok(status) => session_log!(

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This arm cannot detect the one cleanup failure a job can trigger deliberately, because sudo -i interposes a shell that re-parses the paths and rm -rf exits 0 on a path that does not exist.

sudo -i <cmd> <args...> does not execv the command. Per sudo(8): "If a command is specified, it is passed to the shell for execution via the shell's -c option." sudo builds that single string by concatenating the argv with spaces, backslash-escaping shell metacharacters — but its escaping loop deliberately exempts $ (alongside alnum, _ and -), so parameter expansion still happens in the target user's login shell.

The paths in args come from read_dir of the session working directory at line 1035, i.e. they are filenames the job created. So for a file the job names with a $:

  • job action creates $HOME (or $USER, $PATH, …) inside the working directory
  • args gets /sessions/<id>/$HOME
  • the login shell expands it to /sessions/<id>//home/jobuser, which does not exist
  • rm -rf on a nonexistent path is a documented no-op that exits 0

So status.success() is true, this match takes the empty arm, nothing is logged, and the file is still there owned by the job user. Control then falls to the TempDir cleanup / remove_dir_all below, which cannot remove it — the session directory leaks with job data in it and the record says cleanup succeeded. That is precisely the silent-leak outcome the comment at 1103-1113 says this change exists to eliminate, and it is the one case that is job-reachable rather than host-misconfiguration-reachable.

A $ in a filename is not exotic — shell scripts, Windows-origin filenames, and generated asset names all produce them, so this also fires by accident, not only on purpose.

Dropping -i for this call removes the shell entirely and makes the argv the argv:

let mut args = vec![
    "-u".to_string(),
    user.user().to_string(),
    rm.to_string_lossy().to_string(),
    "-rf".to_string(),
    "--".to_string(),
];

rm needs no login environment, so -i buys nothing here while adding a re-parsing layer over job-controlled strings. (It also makes the resolved absolute rm authoritative for the callee, which the login shell currently undercuts.) If -i must stay, then the status check needs to be paired with a post-check that the directory is actually empty, since the exit code cannot speak for paths the shell rewrote.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Confirmed and fixed. This is the strongest finding in the series, so thank you for it.

Verified the mechanism before acting on it, both in sudo(8) -- with -i the argv is concatenated for the shell's -c "after escaping each character (including white space) with a backslash except for alphanumerics, underscores, hyphens, and dollar signs" -- and by reproducing it: building the sh -c string with those exact escaping rules against a file named $HOME gives rm exit 0 with the file still on disk. So the status check this revision added was blind to the one cleanup failure a job can cause deliberately.

Took your first option: -i is gone from that call. sudo now execs rm directly, so nothing re-parses job-chosen filenames, the exit status means what the comment claims, and the resolved absolute path is the one that runs -- which also resolves the namespace concern you raised separately, since there is no longer a login shell to re-resolve it.

The argv is now a pure function, cross_user_cleanup_args, with tests pinning the absent -i, the resolved path, and -- ahead of any filename. All three are mutation-checked, including re-adding -i.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This arm looks like it will fire on every cross-user session, not on the failures it is meant to surface — and the message it emits is wrong about who owns the leftover.

The working directory always contains a .helpers-<uuid> directory, created at session.rs:681 via helper_binary::create_helpers_dir. Its permissions are deliberate (helper_binary.rs:63-70):

  • .helpers-<uuid> — owner = process user, group = job user's group, mode 0o750 (group r-x, no w)
  • .helpers-<uuid>/h-<uuid> — same owner/group, mode 0o750

The working directory itself is 0o770 with the job group (tempdir.rs:175-189), so the job user can unlink entries directly under it. But rm -rf .helpers-<uuid> must first unlink h-<uuid> from inside .helpers-<uuid>, and that requires write on that directory — which the job user does not have, by design. The comment at helper_binary.rs:30-32 states the intent explicitly: "the job user can traverse and read ... but cannot write or modify files."

So the sequence at cleanup is:

  1. read_dir returns .helpers-<uuid> among the entries (nothing deletes it — helper.shutdown() at line 1054 only stops the child process; no code removes the helpers dir).
  2. sudo -u <job> /usr/bin/rm -rf -- ... .helpers-<uuid> hits EACCES unlinking h-<uuid>, cannot then rmdir the non-empty directory, and exits 1 (POSIX: rm "shall exit with a non-zero status" when a file cannot be removed; -f suppresses only nonexistent-operand errors, not EACCES).
  3. status.success() is false → this arm logs Cross-user cleanup of <wd> exited exit status: 1; files owned by the job user may remain.
  4. remove_dir_all below, running as the process user, removes .helpers-<uuid> without difficulty — it owns it, mode 0o750 gives owner rwx.

Two consequences:

The warning is a guaranteed false positive. It routes to the worker agent log (that is the stated point of adding PROCESS_CONTROL), on every cross-user session, describing a leak that did not happen. A warning that fires unconditionally is one operators learn to filter, which costs the real signal this change exists to create — the sudoers-denial and rm-refusal cases the comment above names.

The message misattributes ownership. The one file that actually resisted removal is owned by the process user, not the job user, and it is removed seconds later. "Files owned by the job user may remain" points an operator at chown/sudoers when nothing is wrong.

This is distinct from the discarded-stderr point on the previous revision: there the concern was that a real failure could not be diagnosed; here the check itself is miscalibrated, and no amount of stderr capture fixes a warning that always fires.

The straightforward fix is to exclude the helpers directory from the operand list, since it is not job-owned and the process user handles it:

let helpers_dir = self.cross_user.helpers_dir.clone();
let files: Vec<String> = entries
    .filter_map(|e| e.ok())
    .map(|e| e.path())
    .filter(|p| helpers_dir.as_deref() != Some(p.as_path()))
    .map(|p| p.to_string_lossy().to_string())
    .collect();

self.cross_user.helpers_dir is already stored (line 467) and is an absolute path under the working directory, so the comparison is exact. With it excluded, a nonzero exit means what this arm claims it means.

Worth a cross-user integration test asserting the clean path logs no warning, since that is the property that broke here and no unit test over cross_user_cleanup_args can see it.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed, and this is the best finding of the round -- a warning that always fires is worse than no warning, and I would not have caught it from unit tests.

Verified the permission reasoning in the source rather than taking it on trust: create_helpers_dir chowns .helpers-<uuid> to the job user's group and then sets 0o750, so the group gets r-x with no w. The job user therefore cannot unlink the binary inside it, rm -rf cannot rmdir a non-empty directory, and -f suppresses missing operands rather than EACCES. Nothing removes the helpers directory before cleanup, so this was reachable on every cross-user session.

Took your patch: self.cross_user.helpers_dir is filtered out of the operand list, with the reasoning recorded inline including the misattribution point -- the one entry that resisted removal is owned by the process user, so blaming job-user ownership pointed operators at chown and sudoers for a non-problem. The remove_dir_all fallback removes it as before.

Agreed a cross-user integration test asserting the clean path logs no warning is the right place to pin this; the existing Docker legs pass but do not assert on log content, so that is filed rather than faked here.

warn,
&self.session_id,
LogContent::FILE_PATH | LogContent::PROCESS_CONTROL,
"Cross-user cleanup of {} exited {}; files owned by \
the job user may remain.",
self.working_directory.display(),
status
),
Err(e) => session_log!(
warn,
&self.session_id,
LogContent::FILE_PATH | LogContent::PROCESS_CONTROL,
"Could not run cross-user cleanup of {}: {e}; files \
owned by the job user may remain.",
self.working_directory.display()
),
}
}
}
}
Expand Down Expand Up @@ -3020,6 +3158,69 @@ impl Drop for Session {
}
}

#[cfg(all(test, unix))]
mod cross_user_cleanup_tests {
//! Unit tests for the cross-user cleanup argv.
use super::*;
use std::path::Path;

#[test]
fn does_not_interpose_a_login_shell() {
// The load-bearing assertion. With `-i`, sudo hands the concatenated argv to
// the job user's login shell via `-c`, escaping everything except
// alphanumerics, underscores, hyphens and DOLLAR SIGNS (sudo(8)). These paths
// are job-chosen filenames, so a file named `$HOME` would be expanded by that
// shell into a path that does not exist, and `rm -rf` on a nonexistent path
// exits 0 -- making the caller's status.success() check report a clean
// cleanup over a file that is still there.
let args = cross_user_cleanup_args(
"job-user",
Path::new("/trusted/rm"),
vec!["/sessions/abc/$HOME".to_string()],
);

assert!(
!args.iter().any(|a| a == "-i"),
"argv must not request a login shell: {args:?}"
);
}

#[test]
fn passes_the_resolved_rm_and_terminates_options_before_filenames() {
// The negative control for the assertion above: the argv must still be a
// usable one, or "no -i" would be satisfied by an argv that does nothing.
let args = cross_user_cleanup_args(
"job-user",
Path::new("/trusted/rm"),
vec![
"/sessions/abc/-rf".to_string(),
"/sessions/abc/b".to_string(),
],
);

assert_eq!(
args,
vec![
"-u",
"job-user",
"/trusted/rm",
"-rf",
"--",
"/sessions/abc/-rf",
"/sessions/abc/b",
]
);
// `--` must precede every filename, so a job-chosen name that looks like an
// option is still treated as a path.
let end_of_options = args.iter().position(|a| a == "--").expect("-- is present");
let first_file = args
.iter()
.position(|a| a.starts_with("/sessions/"))
.expect("a file is present");
assert!(end_of_options < first_file);
}
}

#[cfg(test)]
mod wrap_actions_tests {
//! Unit tests for the small pure helpers that back RFC 0008 wrap-hook
Expand Down
Loading
Loading