Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/copilot-instructions.md
Original file line number Diff line number Diff line change
Expand Up @@ -185,7 +185,7 @@ The Rust workspace (`src/`) implements multiple sandboxing backends behind the `
| WSLc | `wxc-exec.exe` | Windows | `backends/wslc/common/src/` β€” feature-gated behind `wslc`, experimental, uses the WSLc SDK (`wslcsdk.dll`, loaded at runtime) to run Linux containers in a WSL2 VM. Supports both one-shot (`WSLContainerRunner`, via `ScriptRunner` + streaming `SandboxBackend`) and state-aware (`state_aware.rs` `WslcStateAwareRunner`, via `StatefulSandboxBackend`) modes. Because the WSLc SDK has **no cross-process re-attach**, state-aware keeps the session (VM) + container warm across separate `wxc-exec` phase processes behind a persistent per-user daemon (`wxc-wslc-daemon.exe`, `backends/wslc/daemon/`) that owns the live `WslcSession`/`WslcContainer` handles; phase processes are thin named-pipe clients (`daemon_client.rs`). The daemon runs all SDK calls on one apartment-affine worker thread (so exec is currently serialized across sandboxes β€” see `docs/wsl/wslc-state-aware.md`). Honors `readwritePaths`/`readonlyPaths` at provision (β†’ container volumes) + `network.defaultPolicy` (`Block`β†’`None`, `Allow`β†’`Bridged`; networking is all-or-nothing β€” no per-host filtering, since the container lacks `CAP_NET_ADMIN`); rejects `deniedPaths` nested under a mount and rejects proxy/host-filtering at provision. exec honors `network.proxy` **url-form only** (injected as `HTTP_PROXY`/`HTTPS_PROXY`); start/stop/deprovision reject all policy. ID prefix `wslc` (`wslc:<32-hex>`). Idle-timeout is env-overridable via `MXC_WSLC_DAEMON_IDLE_TIMEOUT_SECS`/`MXC_WSLC_DAEMON_IDLE_POLL_SECS`. See `docs/wsl/wslc-state-aware.md`. |
| LXC | `lxc-exec` | Linux | `core/lxc/src/main.rs` + `backends/lxc/common/` |
| Seatbelt | `mxc-exec-mac` | macOS | `core/mxc_darwin/src/main.rs` + `backends/seatbelt/common/` β€” uses macOS App Sandbox (Seatbelt) profiles for process containment. Requires schema `0.7.0-alpha`+. Supports `network.proxy` via the same cooperative env-var model as Bubblewrap (injects `HTTP_PROXY`/`HTTPS_PROXY` into the sandbox, reusing `wxc_common::unix_proxy_coordinator`; `builtinTestServer` spawns the shared `unix-test-proxy`). See `docs/macos-support/seatbelt-backend.md`. |
| Bubblewrap | `lxc-exec` | Linux | `backends/bubblewrap/common/src/bwrap_runner.rs` β€” unprivileged sandboxing via Linux user namespaces and `bwrap`. Experimental β€” requires `--experimental`. Uses shared filesystem/network policy fields; per-host network filtering via `NetworkIptablesManager` from `backends/lxc/common`. For schema 0.8+, proxy mode uses a private network namespace with rootless `slirp4netns` routing and fails if `slirp4netns` is unavailable; schema 0.6/0.7 and absent-version requests retain the legacy shared-network proxy behavior. See `docs/bwrap-support/bubblewrap-backend.md`. |
| Bubblewrap | `lxc-exec` | Linux | `backends/bubblewrap/common/src/bwrap_runner.rs` β€” unprivileged sandboxing via Linux user namespaces and `bwrap`. Experimental β€” requires `--experimental`. Uses shared filesystem/network policy fields; per-host network filtering via `NetworkIptablesManager` from `backends/lxc/common`. For schema 0.8+, proxy mode uses a private network namespace with rootless `slirp4netns` routing and a default-DROP egress chain that permits only loopback and the translated proxy endpoint (installed via `nsenter` + `iptables`/`ip6tables` from a supervisor that holds the namespaces); it requires `slirp4netns`, util-linux `unshare`/`nsenter` and `iptables`/`ip6tables` on PATH and fails validation if any is unavailable. `iptables`/`ip6tables` must also resolve to the `nf_tables` backend, unless `/run/xtables.lock` is writable by the calling user: the legacy backend opens that lock unconditionally, and the rules are installed by an unprivileged same-uid supervisor that cannot open a root-owned one β€” `validate` refuses such a host rather than letting the supervisor die at the first rule. The host proxy endpoint is rewritten to slirp's gateway `10.0.2.2`, so `127.0.0.1`/`0.0.0.0`/`::` are translated while `::1` is rejected (an IPv6-loopback listener cannot accept the gateway's IPv4 connection). Schema 0.6/0.7 and absent-version requests retain the legacy shared-network proxy behavior. See `docs/bwrap-support/bubblewrap-backend.md`. |

### Config flow

Expand Down
138 changes: 95 additions & 43 deletions docs/bwrap-support/bubblewrap-backend.md
Original file line number Diff line number Diff line change
Expand Up @@ -27,22 +27,39 @@ requiring root privileges or a container runtime.
newer** is required. Platform detection probes `bwrap --version` and reports
the backend as unavailable β€” with the detected version β€” when the host is
below that floor.
- **Schema 0.8 proxy mode only:** `slirp4netns` installed and on PATH. It is
not required when `network.proxy` is omitted or when a 0.6/0.7 policy uses
- **Schema 0.8 proxy mode only:** `slirp4netns` installed and on PATH, plus
`nsenter`, `iptables`, and `ip6tables` for the proxy-only egress rules. None
are required when `network.proxy` is omitted or when a 0.6/0.7 policy uses
the legacy proxy behavior.
```bash
# Debian/Ubuntu
sudo apt install slirp4netns
sudo apt install slirp4netns util-linux iptables

# Fedora/RHEL
sudo dnf install slirp4netns
sudo dnf install slirp4netns util-linux iptables

# Alpine
apk add slirp4netns
apk add slirp4netns util-linux iptables
```
Proxy mode fails explicitly if `slirp4netns` is unavailable; it never falls
back to sharing the host network namespace. The host must also provide the
util-linux `unshare` command with `--map-current-user` and `--keep-caps`.
`iptables` must resolve to the **`nf_tables` backend** (the default on
Debian 10+, Ubuntu 20.10+, RHEL 8+, and Alpine). The legacy backend opens
`/run/xtables.lock` before touching any table, and the rules are installed by
an unprivileged supervisor that keeps the caller's uid β€” so on a stock host,
where `/run` is root-owned, it cannot take that lock. `validate` refuses such
a host with a message naming the backend, rather than letting the supervisor
die at the first rule. If a host is pinned to legacy, switch it:
```bash
sudo update-alternatives --set iptables /usr/sbin/iptables-nft
sudo update-alternatives --set ip6tables /usr/sbin/ip6tables-nft
```
(A legacy backend is still accepted where the lock *is* writable β€” for
example when running as root β€” since it works there.)
Proxy mode fails explicitly if any of these is unavailable; it never falls
back to sharing the host network namespace or to running without egress
rules. The host must also provide the util-linux `unshare` command with
`--map-current-user` and `--keep-caps`. No root is needed: `iptables` runs
against the sandbox's own network namespace, where the supervisor holds
`CAP_NET_ADMIN`.
- User namespaces must be enabled:
```bash
# Check: should print "1"
Expand Down Expand Up @@ -276,6 +293,19 @@ request fails if its private namespace cannot be configured.

### How it works

0. Before anything is launched, `validate` probes the host tools this mode
depends on β€” `slirp4netns`, `unshare` (checked for `--map-current-user` and
`--keep-caps`), `nsenter`, `iptables`, and `ip6tables` β€” so a host that is
missing one fails immediately with a message naming it, rather than partway
through supervisor startup. For `iptables`/`ip6tables` presence is not
enough: the probe also reads the backend from the version banner and refuses
a legacy backend whose `/run/xtables.lock` this user cannot open, because
the unprivileged supervisor would otherwise die at the first rule. Each
probe is bounded by a short timeout: a
wedged binary is reported as hung and named, instead of stalling every
proxy-mode execution on the host indefinitely. A successful probe is cached
for the life of the process; failures are not, so installing the missing
tool takes effect without a restart.
1. When `network.proxy` is set, the runner launches an unprivileged HTTP
proxy on loopback (`127.0.0.1:N`). For tests, the bundled
`unix-test-proxy` binary is used (`builtinTestServer: true`,
Expand All @@ -285,8 +315,19 @@ request fails if its private namespace cannot be configured.
with `--unshare-net`, and keeps the workload behind a startup barrier.
3. The supervisor attaches `slirp4netns` to Bubblewrap's private network
namespace. Host-loopback proxy endpoints are presented to the sandbox
through slirp's `10.0.2.2` host gateway. The workload starts only after
slirp reports that `tap0` is configured.
through slirp's `10.0.2.2` host gateway. Once slirp is up, the supervisor
programs a default-DROP `MXC_EGRESS` chain into that namespace via
`nsenter`, permitting only loopback and the proxy endpoint (IPv6 gets a
DROP-only chain). The workload is released only after every rule is
installed, so it can never run with egress open. A failure to program any
rule aborts the supervisor rather than starting an unenforced sandbox.

Bubblewrap joins the supervisor's user namespace (`--userns`) rather than
creating its own, so the sandbox lives in the namespace that owns the
rule-bearing network namespace. This relies on Bubblewrap dropping
capabilities in the sandboxed process β€” the runner passes no `--cap-add` β€”
which is what prevents the workload from holding the `CAP_NET_ADMIN` needed
to flush the chain.
Comment on lines +318 to +330

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thought: nice! I do something similar for Windows in AppInfoSvc. That said, I'd be interested in when we clean up things. In Windows we clean up the Windows Filtering Platform rules I add once all processes in the Job object has gone away.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cleanup here is structural rather than bookkeeping: the rules live inside the sandbox's own network namespace, not the host's tables. When the last process in that namespace exits, the kernel tears the namespace down and every rule goes with it β€” so there is nothing to reconcile, and no way for a rule to outlive the sandbox or affect anything else on the host. Drop for ProxyNetworkNamespace still explicitly reaps the supervisor, slirp4netns and the temp state dir.

The closer analogue to your WFP case is the legacy firewall mode, which does write host-global rules and needs explicit removal β€” one of the reasons 0.8 moved the rules inside the namespace.

4. The command builder sets `HTTP_PROXY`, `HTTPS_PROXY`, `ALL_PROXY`,
`FTP_PROXY`, and their lowercase variants inside the sandbox via
`bwrap --setenv` (caller-supplied values for these keys, including
Expand All @@ -295,13 +336,14 @@ request fails if its private namespace cannot be configured.
bypass the configured proxy policy.
5. Cooperative tools (curl, wget, Python `requests`, Node `https`, etc.)
honor the env vars and traffic flows through the proxy, which applies
the `allowedHosts` / `blockedHosts` lists.
the `allowedHosts` / `blockedHosts` lists. Non-cooperating clients are not
merely unrouted β€” their traffic is dropped by the egress chain.

### Example: builtin test proxy with allowlist

```json
{
"version": "0.6.0-alpha",
"version": "0.8.0-alpha",
"platform": "linux",
"containment": "bubblewrap",
"process": {
Expand All @@ -319,7 +361,7 @@ request fails if its private namespace cannot be configured.

```json
{
"version": "0.6.0-alpha",
"version": "0.8.0-alpha",
"containment": "bubblewrap",
"process": { "commandLine": "curl -fsSL https://example.com" },
"network": {
Expand All @@ -328,6 +370,11 @@ request fails if its private namespace cannot be configured.
}
```

> Both examples declare `0.8.0-alpha` deliberately: the private-namespace and
> egress-enforcement behavior described above is selected by the schema version,
> so the same config on `0.6`/`0.7` runs the legacy shared-host-network proxy
> path instead.

### Caveats

- **Host loopback moves to the gateway address (breaking change in 0.8+
Expand All @@ -340,21 +387,14 @@ request fails if its private namespace cannot be configured.
namespace**, so `127.0.0.1` did reach the host; that is the behavior
changing here.

Host-local services remain reachable, but at slirp's gateway address
The configured proxy remains reachable, but at slirp's gateway address
`10.0.2.2` instead β€” which is exactly how the runner rewrites a
`localhost` proxy endpoint so the sandbox can still find it.

**This reachability is not limited to the configured proxy.** slirp runs
without `--disable-host-loopback`, so the workload can open a connection
to *any* service bound to host loopback via `10.0.2.2:<port>` β€” a local
database, a metadata endpoint, an unrelated daemon. That is a deliberate
exception to the private-network boundary and a more sensitive one than
generic outbound internet egress, because host-loopback services often
assume that only host-local callers can reach them. `--disable-host-loopback`
would close the path, but it would also break the proxy rewrite above, so
the gateway stays reachable until a single-port forwarding mechanism
replaces it. Restricting egress to the configured proxy endpoint is the
job of the proxy-only enforcement work that builds on this change.
`localhost` proxy endpoint so the sandbox can still find it. Other
host-local services do **not** come along: slirp itself runs without
`--disable-host-loopback`, so the gateway can in principle carry traffic to
any host-loopback port, but the egress chain admits only the single
`10.0.2.2:<proxy-port>` destination and drops the rest. The host-loopback
surface is therefore the proxy endpoint alone.
- **The supervisor's user namespace is visible to the sandbox**: in proxy
mode `bwrap` joins the supervisor's user namespace via `--userns` rather
than creating its own, and the namespace descriptor stays open in the
Expand All @@ -365,21 +405,33 @@ request fails if its private namespace cannot be configured.
`CapBnd`/`CapEff`/`CapPrm` all zero. The end-to-end test suite asserts
those are zero, because that assumption is what makes the exposed
descriptor inert.
- **Cooperative model**: the runner enforces by injecting
`HTTP_PROXY` / `HTTPS_PROXY` into the sandbox environment, so only
well-behaved clients that honor those vars are routed through the
proxy. Tools that bypass them (raw sockets, custom HTTP clients,
statically-linked binaries that ignore the env) can still use slirp's direct
egress and are **not yet enforced**.
This applies to **both** the builtin test proxy and external (BYO)
proxy modes β€” the limitation is in the env-var injection mechanism,
not in the proxy itself; a BYO proxy can do whatever it likes for
cooperating clients. For strict whole-network isolation, omit
`network.proxy` so the runner can apply `--unshare-net` instead.
- **Cooperative routing, enforced egress (schema 0.8+)**: the runner injects
`HTTP_PROXY` / `HTTPS_PROXY` so cooperating clients route through the proxy,
and additionally programs a default-DROP egress chain inside the sandbox's
private network namespace. Clients that ignore the env vars (raw sockets,
custom HTTP clients) can no longer reach the network directly: only loopback
and the proxy endpoint are permitted. DNS is deliberately **not** opened β€”
the proxy resolves on the workload's behalf β€” so the proxy endpoint must be
an IPv4 literal. IPv6 egress is denied outright.

Consequently the only accepted host-local proxy endpoints are `localhost`,
`127.0.0.0/8` and the wildcards `0.0.0.0` / `::`, each rewritten to
`10.0.2.2`. `::1` is **rejected at validation time**: a proxy bound only to
the IPv6 loopback cannot accept the IPv4 connection slirp's gateway
produces, so translating it would hand the sandbox an address nothing
answers on. Bind such a proxy to `127.0.0.1` or to a dual-stack wildcard
instead.
On schema **0.6/0.7** the legacy behavior applies: the sandbox shares the
host network namespace, no egress rules are installed, and only the
cooperative env-var routing is in effect β€” a client that ignores
`HTTP_PROXY`/`HTTPS_PROXY` reaches the network directly. For strict
whole-network isolation on those versions, omit `network.proxy` so the
runner can apply `--unshare-net` instead.
- **Mutually exclusive with iptables enforcement**: setting
`network.proxy` together with `network.enforcementMode` of `"firewall"`
or `"both"` is rejected at config-parse time because iptables-based
enforcement requires root and would defeat the proxy's privilege story.
or `"both"` is rejected at config-parse time. (The rejection message cites a
root requirement that proxy mode has since disproved; see the firewall
section.)
- **External proxy delegates policy**: when `network.proxy` uses
`localhost: <port>` or `url: <url>` (not `builtinTestServer`), the
external proxy is responsible for any host filtering. The runner does
Expand Down Expand Up @@ -417,10 +469,10 @@ resolution.
|--------|-----|------------|
| Privileges | Root required | Unprivileged (user namespaces) |
| Rootfs | Downloads distro rootfs | Bind-mounts host filesystem |
| Startup | Create β†’ Start β†’ Attach | Single `bwrap` exec (proxy mode adds a namespace supervisor) |
| Startup | Create β†’ Start β†’ Attach | Single `bwrap` exec; proxy mode adds a user/network-namespace supervisor, a `slirp4netns` instance and an egress rule set |
| Network isolation | iptables + veth | `--unshare-net`, private netns + slirp4netns, or iptables |
| Dependencies | `lxc-*` tools, templates | `bwrap`; proxy mode also needs `slirp4netns` and util-linux `unshare` |
| Lifecycle | Create/destroy containers | Process dies on exit |
| Dependencies | `lxc-*` tools, templates | `bwrap`; proxy mode also needs `slirp4netns`, util-linux `unshare` and `nsenter`, plus `iptables` and `ip6tables` on the `nf_tables` backend |
| Lifecycle | Create/destroy containers | Process dies on exit; proxy mode's supervisor is reaped with it |

**When to use Bubblewrap:**
- Quick sandboxing without root access
Expand Down
18 changes: 12 additions & 6 deletions src/backends/bubblewrap/common/src/bwrap_command.rs
Original file line number Diff line number Diff line change
Expand Up @@ -98,8 +98,8 @@ pub(crate) enum ResolvedNetworkMode {
LegacyProxy,
/// The host network namespace with per-destination iptables filtering.
FirewallFiltered,
/// Cooperative proxy routing inside a slirp-backed private namespace.
/// Later work adds proxy-only egress enforcement.
/// Cooperative proxy routing inside a slirp-backed private namespace, with
/// egress closed to everything but the proxy.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

issue: The existing config warning now contradicts this schema 0.8+ guarantee.

config_parser.rs:1314-1329 still warns every Bubblewrap proxy + defaultPolicy='block' request that raw-socket clients bypass the proxy and reach the host network. Its condition does not check the schema version, while schema 0.8+ resolves this case to ProxyOnly and installs the default-DROP chain described here.

Could we scope that warning to the legacy proxy path (schema 0.6/0.7), or otherwise suppress it for schema 0.8+ ProxyOnly requests?

@SohamDas2021 Soham Das (SohamDas2021) Aug 20, 2026 •

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch, and agreed the warning is wrong under 0.8. Tracked in #970 (Priority3) rather than fixed here.

ProxyOnly,
}

Expand Down Expand Up @@ -152,9 +152,13 @@ impl ResolvedNetworkMode {
matches!(self, Self::ProxyOnly)
}

/// Whether current behavior installs iptables filtering rules.
/// Whether the mode uses the host-side firewall manager.
///
/// False for `ProxyOnly`, which is *not* iptables-free: it programs rules
/// directly into the sandbox's own network namespace from the supervisor
/// (see `proxy_network`) rather than going through the host manager.
#[cfg(any(target_os = "linux", test))]
pub(crate) fn requires_iptables(self) -> bool {
pub(crate) fn requires_host_firewall_manager(self) -> bool {
matches!(self, Self::FirewallFiltered)
}
}
Expand Down Expand Up @@ -566,8 +570,10 @@ mod tests {
assert!(ResolvedNetworkMode::ProxyOnly.uses_external_userns());
assert!(!ResolvedNetworkMode::Isolated.uses_external_userns());
assert!(!ResolvedNetworkMode::LegacyProxy.uses_private_netns());
assert!(ResolvedNetworkMode::FirewallFiltered.requires_iptables());
assert!(!ResolvedNetworkMode::Shared.requires_iptables());
assert!(ResolvedNetworkMode::FirewallFiltered.requires_host_firewall_manager());
assert!(!ResolvedNetworkMode::Shared.requires_host_firewall_manager());
// ProxyOnly enforces via in-netns rules, not the host manager.
assert!(!ResolvedNetworkMode::ProxyOnly.requires_host_firewall_manager());
}

#[test]
Expand Down
Loading
Loading