[LXC/Bubblewrap] Network policy: egress enforcement, hook scoping, and proxy pinning #896

Status audit — 2026-09-17

Verified against origin/main @ be51dc4cb.

18 items — 5 fixed, 3 moot, 1 retracted, 2 postponed, 7 outstanding. Of the seven outstanding, four are carried by ADO 64113943, one by ADO 63911310, and two are untracked.

Moot means the code the report describes was deleted rather than corrected, so nothing remains to fix. It is not evidence that the class of defect cannot recur — where the guarantee still matters it has been re-checked against the replacement.

What remains to close this issue

Code. #1010 is the only verified-live defect with no tracker. Bubblewrap already solves the same problem correctly, by subtracting exclusions from the parent CIDR rather than emitting carve-outs ahead of it (bubblewrap/common/src/network_rules.rs:434, subtract_blocks). Porting that also needs a rule-count budget, which LXC has no equivalent of, and it multiplies the per-rule process spawns recorded in ADO 64111906.

Decision. The :: proxy divergence should be settled before any shared extraction, or the extraction freezes one backend's answer by default.

Coverage. #1012 needs a regression test. Reverting that fix leaves all 309 LXC tests passing, so a refactor could reintroduce the fail-open silently.

Related, not folded here. #975 describes configured_net_interfaces() and VETH_PIN_SCRIPT, neither of which exists on main. It is moot by the same deletion as the #755 LXC half and #878, yet it is open, labelled a GA blocker, and has no ADO item.

Egress-side network policy -- where the FORWARD hook is installed, what it matches, when it fails open, and how the proxy host is pinned. Mostly LXC, but #755 covers Bubblewrap too and #766 is shared across backends.

This consolidates 12 separately filed issues so the backlog carries one entry per area instead of one per finding. Every original report is reproduced below in full, unedited. The originals are closed and point here.

[LXC/Bubblewrap] Firewall mode fails open when no veth interface is set (no FORWARD hook)

Originally #755.

Relevant area(s)

Linux (LXC and Bubblewrap backends)

Brief description of your issue

In firewall enforcement mode (network.enforcementMode: "firewall" / "both"), NetworkIptablesManager creates the per-container iptables/ip6tables chain(s) with a default policy, but when no veth interface is set it skips the FORWARD hook and still returns success. The chain is therefore never in the packet path, so no traffic is filtered even though the caller believes the network policy was applied.

• The Bubblewrap backend never calls set_veth_interface (src/backends/bubblewrap/common/src/bwrap_runner.rs:205-206), so every Bubblewrap firewall-mode run installs an orphan chain and enforces nothing — IPv4 and IPv6 alike.
• The LXC backend discovers the veth (src/backends/lxc/common/src/lxc_runner.rs:208-221) but only sets it if let Some(...), so it hits the same gap whenever discover_veth_interface fails.

This is a silent fail-open on a security control. It is pre-existing on main (not introduced by any single PR); it was identified during review of #724, which corrects the LXC FORWARD direction (-o → -i) and adds LXC hook assertions but does not close this gap and does not cover Bubblewrap.

Relevant code: src/backends/lxc/common/src/network_iptables.rs:811-818 (the else branch logs "No veth interface set for container. Cannot scope iptables rules. Skipping FORWARD hook." and returns Ok(())), surfacing as apply_firewall_rules returning Ok(true) with rules_applied = true at network_iptables.rs:670-673.

Steps to reproduce

1. Run a Bubblewrap container (or an LXC container where veth discovery fails) with:
   • network.enforcementMode: "firewall"
   • network.defaultPolicy: "block"
   • a non-empty allowedHosts / blockedHosts list.
2. Observe the log line: Warning: No veth interface set for container. Cannot scope iptables rules. Skipping FORWARD hook.
3. From inside the container, connect to a destination that the policy should block (e.g. any host not in allowedHosts under defaultPolicy: "block").

Expected behavior

When network enforcement is requested but cannot be safely scoped to the container (no veth interface), setup should fail closed — return an error so the container does not run with an unenforced network policy. For Bubblewrap specifically, the backend should either implement process/cgroup-scoped enforcement (e.g. OUTPUT with cgroup/UID matching) or explicitly reject firewall mode rather than reporting success.

Actual behavior

apply_firewall_rules returns Ok(true) and sets rules_applied = true. The chain exists but is never hooked into FORWARD, so all container egress (IPv4 and IPv6) flows unfiltered while the run reports success. For Bubblewrap this occurs on every firewall-mode run because set_veth_interface is never called; for LXC it occurs whenever veth discovery fails.

[LXC] Container runs unfiltered between start and firewall hook installation

Originally #764.

Split out of review feedback on #632.

Problem

The LXC runner starts the container before it installs any firewall hook, so there is a window in which the container has a live interface and no policy on it.

In src/backends/lxc/common/src/lxc_runner.rs on the #632 branch:

• container.start() at line 330
• optional wait_for_network, up to 10 seconds, at line 358
• veth discovery at lines 361-376
• apply_firewall_rules at line 408

Anything already inside the container -- init services, or processes retained across runs by destroyOnExit: false -- can send arbitrary egress for the whole span between line 330 and line 408. Under the deny-all-except-proxy posture that window is a hole in the guarantee the mode is named for.

#632 moved proxy pinning to before container.start(), which removed host-side DNS work from the window, but it did not close the window itself.

Why it is not fixed in #632

Closing it needs a second, earlier chain: a veth-scoped quarantine carrying only the DHCP carve-outs, installed as soon as the interface exists, then atomically replaced by the completed policy. That is a new enforcement stage with its own rollback, ownership, and teardown semantics rather than a reordering of the existing one, so it does not belong in a PR that is already addressing a different set of gaps.

Suggested shape

1. Discover the veth as early as the interface exists rather than after the address wait.
2. Install a quarantine chain that drops everything except DHCPv4 and DHCPv6.
3. Let wait_for_network proceed under quarantine.
4. Replace the quarantine with the completed policy, and make the swap ordering such that no instant leaves the interface unhooked.

Not verified

None of this has been exercised against live iptables or a live LXC host. The window is established by reading the call ordering in lxc_runner.rs, not by observing a packet escape.

[Network] Extract shared CIDR and address-family handling used by multiple backends

Originally #766.

Split out of review feedback on #724.

Problem

Address and CIDR handling for network policy is implemented per backend rather than once. The LXC path in network_iptables.rs parses CIDR prefixes, classifies IPv4 versus IPv6, and buckets resolved addresses by family; equivalent logic exists elsewhere for other backends. Duplicated parsing of security-relevant input is how two backends end up disagreeing about what a policy means, and a fix applied to one silently leaves the other wrong.

Suggested shape

Extract a single module owning:

1. CIDR parsing and validation, including prefix-length bounds per family.
2. IPv4 versus IPv6 classification, including IPv4-mapped IPv6 forms.
3. Bucketing a resolver result into per-family destination lists.

Then have each backend depend on it rather than reimplementing it.

Why it was not done in #724

#724 is a behavior fix for IPv6 destinations and CIDR ranges in firewall mode. Extracting a shared module touches every backend that does address handling and would turn a scoped fix into a cross-backend refactor, which is a worse change to review and a worse change to revert.

Not verified

This describes duplication observed by reading the backends, not a divergence observed at runtime. Whether the implementations currently disagree on any specific input has not been established -- that is worth checking as the first step of the extraction, since any disagreement found is a live bug rather than just duplication.

LXC network policy fails on RHEL 10: deprecated iptables -m state match unsupported

Originally #777.

Relevant area(s)

Linux

Brief description of your issue

The LXC backend builds its firewall rules with the deprecated -m state iptables
match. On RHEL 10 (kernel 6.12, iptables 1.8.11 nf_tables backend) the kernel does
not expose state revision 0, so applying any network policy fails outright.

-m state (xt_state) has been superseded by -m conntrack --ctstate since ~2011
and was folded into xt_conntrack. Ubuntu and Debian still ship the compatibility
shim, which is why this only surfaces on RHEL.

The failure is fail-closed — lxc_runner destroys the container and returns a
backend_error rather than running unprotected — so there is no silent policy
bypass. However, it means MXC cannot enforce LXC network policy at all on RHEL 10;
any run with a network section fails.

Source: src/backends/lxc/common/src/network_iptables.rs (the ESTABLISHED,RELATED
rule in apply_firewall_rules).

The same deprecated syntax also appears in
src/backends/wslc/common/src/policy_mapping.rs, which will hit this whenever its
distro drops the shim.

Steps to reproduce

1. On a RHEL 10 x64 host, install LXC (lxc, lxc-templates, dnsmasq, iptables) and build/obtain lxc-exec.
2. Run a config with a network policy using firewall enforcement, e.g. tests/configs/lxc_network_test.json:

   ./lxc-exec tests/configs/lxc_network_test.json
   

3. Observe the run fails before the script executes.

Expected behavior

The firewall chain is created and the network policy is enforced, matching the
behavior on Ubuntu and Debian. The container runs and host filtering is applied.

Actual behavior

Rule application fails and the run aborts:

Network policy error: iptables -A MXC-CLI-LXC-Network-Test -m state --state ESTABLISHED,RELATED -j ACCEPT failed: Warning: Extension state revision 0 not supported, missing kernel module?
iptables v1.8.11 (nf_tables):  RULE_APPEND failed (No such file or directory): rule in chain MXC-CLI-LXC-Network-Test

{"error":{"code":"backend_error","message":"Network policy error: ..."}}

Observed in CI on the 1es-mxc-e2e-rhel-10-x64 pool (image rhel-10-x64,
version 10.2.2026080415, kernel 6.12.0-211.40.1.el10_2.x86_64). The other seven
LXC tests pass on the same host, so LXC itself is functional — only the firewall
rule syntax fails.

Suggested fix: emit -m conntrack --ctstate ESTABLISHED,RELATED instead, which is
the modern form supported across all currently supported distributions.

[LXC] Firewall-enforced start accepts a container whose only interface is lxc.net.N for N != 0

Originally #851. Fixed in #849; closes on merge.

Found by Copilot review on #849: #849 (comment)

Problem

LxcStateAwareBackend::start refuses a container it cannot enforce a firewall
policy on, but the refusal has a gap. It rejects lxc.include, rejects more
than one interface, and rejects zero interfaces — then accepts any remaining
case, which includes a container whose single interface is at a nonzero index:

if net.indices.len() > 1 { /* refuse */ }
if net.indices.is_empty() { /* refuse */ }
// net.indices == [1] falls through and is accepted

src/backends/lxc/common/src/state_aware.rs:311

Everything downstream assumes index 0. The code pins
lxc.net.0.veth.pair and hooks that deterministic veth, so a container
configured with only lxc.net.1 gets a chain installed against an interface
it does not use. Start reports success and the caller believes a default-deny
policy is in force while traffic routes around it.

Why this matters

This is a fail-open hole in the exact invariant the LXC state-aware work
exists to establish: a container must never run believing it is filtered when
no rule filters it. Every other branch in this function fails closed for that
reason.

Suggested fix

Require the sole index to be 0 before installing the firewall, and refuse
with the same policy_validation shape the neighboring branches use. Add a
parser/spec test for the lxc.net.1 case alongside the existing
single-interface tests.

Notes

Deferred out of #849 to keep that PR reviewable — #633 was closed after
accumulating 87 review comments, and this is a behavior change to fail-closed
start validation that deserves its own review rather than a late amendment.

LXC state-aware firewall mode does not verify that lxc.net.0 is a veth

Originally #858. Fixed in #849; closes on merge.

Found while reviewing #849. Distinct from #851, which covers the nonzero-index case.

What happens

configured_net_interfaces() returns only interface indices and an include flag -- it never reads the interface type:

• src/backends/lxc/common/src/lxc_bindings.rs:133-138 -- pub struct NetInterfaceConfig { pub indices: Vec<u32>, pub has_include: bool }
• src/backends/lxc/common/src/lxc_bindings.rs:154-185 -- parse_net_interface_config matches lxc.net.<N>. keys and lxc.include only, never a .type value.

So the firewall path cannot check the type, and pins net.0 regardless:

• state_aware.rs:301 refuses has_include
• state_aware.rs:311 refuses more than one interface
• state_aware.rs:330 refuses zero interfaces
• state_aware.rs:346 set_config_item("lxc.net.0.veth.pair", &veth)

An adopted container with a single lxc.net.0.type = macvlan or phys at index 0 passes every check, then gets a FORWARD hook pinned to a veth name that will never exist, while traffic uses the non-veth interface. MXC reports the firewall as enforced.

Why it is not #851

#851 is indices == [1] -- a nonzero index. A single macvlan at index 0 gives indices == [0], which never triggers #851. Same underlying assumption that net.0 is a veth, but a different trigger and a different fix.

Suggested direction

Capture the interface type in NetInterfaceConfig, and fail closed in apply_network_policy unless index 0 is a veth.

Note: the code facts above are verified. That a macvlan/phys interface then routes around the FORWARD hook follows from veth.pair applying only to type=veth; that part is reasoned from the LXC config schema, not executed.

[LXC] Scope the FORWARD return path by container address so a DROP-policy host still works

Originally #869.

Filed from review feedback on #798 (thread on src/backends/lxc/common/src/network_iptables.rs:669).

What the reviewer found

These return rules do not match replies on the default bridged LXC topology, as the comment
itself notes: FORWARD sees -o lxcbr0 before a bridge port is selected. Consequently, on
hosts whose FORWARD policy is DROP (notably hosts running Docker), allowed destinations and an
off-host proxy time out. The workflow masks this by changing FORWARD to ACCEPT. The
implementation needs a return rule scoped by the container address (or equivalent) so the proxy
exception works under a normal DROP host policy.

This is accurate, and #798 documents it

Both return-rule forms were measured inert on the default bridged topology:

• The interface form matches -o <veth>, but a reply routed toward the bridge has an output
  device of lxcbr0, not the veth.
• The physdev form matches --physdev-out <veth>, which requires that a bridge port has already
  been selected. On a reply routed in from outside, no port is selected while FORWARD runs.

build_forward_return_physdev_rule_args says so in its doc comment, and
.github/workflows/lxc-e2e.yml says so where it sets the host policy, naming the address-scoped
return rule as the fix and calling it deferred.

Impact

On a host whose FORWARD policy is DROP — which includes any host running Docker — replies to the
container are dropped by the host policy. The deny half of the posture still holds, but the one
permitted exception does not: the proxy times out, so the container reaches nothing at all.

The E2E suite does not catch this because the workflow sets FORWARD to ACCEPT first, and it does
that for a real reason: under a DROP policy a container with no MXC hook at all is equally
unreachable, so every deny case would pass vacuously and the suite would report success against a
firewall that filters nothing.

The fix

Scope the return direction by the container's address rather than by its port — -d <container-ip>
with --state ESTABLISHED,RELATED — so the rule matches while FORWARD still sees -o lxcbr0.

That needs the container address plumbed through to the manager, which does not have it today, and
a live bridged measurement to confirm the match. Adding a third rule form without that measurement
would repeat the mistake the two current forms already made.

Why it was not done in #798

It needs a new input to the manager and a real bridged host to verify against, and #798 is already
31 files. The limitation is documented where it bites rather than left for someone to rediscover.

[LXC] deny-all-except-proxy does not cover the INPUT path to the bridge gateway

Originally #870.

Filed from review feedback on #798 (thread on tests/scripts/run_lxc_network_proxy_test.sh:211).

What the reviewer found

This leaves a direct egress path open in the posture advertised as "deny all except the proxy."
Traffic to any host-local listener on the bridge gateway bypasses FORWARD, not only dnsmasq;
the reachable resolver can also be used as a DNS exfiltration channel. The proxy policy needs a
per-container INPUT hook (with only the configured proxy endpoint/port allowed) before this
test can treat gateway DNS as informational rather than an isolation failure.

This is accurate

The chain #798 installs is hooked into FORWARD only. A packet from the container to the bridge
gateway's own address is delivered locally and traverses INPUT, so nothing in the chain ever sees
it. run_lxc_network_proxy_test.sh reports GATEWAY_DNS_REACHED as a note rather than a failure
for exactly this reason.

The reviewer's point is broader than that note admits. The gap is not that gateway DNS is still
answered. It is that every host-local listener bound on the bridge gateway is reachable from a
container whose policy says it may reach the proxy and nothing else, and that a reachable resolver
is an exfiltration channel in its own right — a name lookup carries attacker-chosen bytes out even
when no other egress exists.

The fix

Hook INPUT per container the way FORWARD is already hooked, scoped with --physdev-in and -i
on the container veth, allowing only the configured proxy endpoint and port and dropping the rest.

Once that lands, run_lxc_network_proxy_test.sh should assert GATEWAY_DNS_BLOCKED instead of
reporting the verdict, and the note explaining why it only reports should be deleted with it.

Why it was not done in #798

It is a second enforcement surface with its own ownership, rollback, and teardown, and its own
failure modes — an INPUT chain left behind is a host-wide artifact rather than a container-scoped
one. #798 hooks one chain into one place and proves that hook works; adding a second hook belongs
with its own tests.

[LXC] Rewrite the /etc/hosts proxy pin from Rust with an open-once, no-follow handle

Originally #875.

What

Rewrite the container's /etc/hosts proxy pin from Rust with an open-once,
no-follow file handle instead of the current grep + printf shell command
that lxc_runner sends through attach_run.

Why

The shell command is in LxcScriptRunner::build_hosts_pin_command and
build_hosts_unpin_command, both of which share
LxcScriptRunner::hosts_read_prologue
(src/backends/lxc/common/src/lxc_runner.rs). It reads the file into a shell
variable, then truncates the file and writes the variable back. Three known
gaps follow from that shape, and the doc comment on hosts_read_prologue
already records the last two as unclosed:

1. The whole file is buffered in shell state. A workload that ran earlier
   in a reused container can enlarge /etc/hosts, and the command then does
   unbounded command-substitution work and holds the entire result in memory
   before the rewrite. Raised in review on PR [LXC] Enforce the deny-all-except-proxy network policy (model 2) #798.

2. A successful read still loses NUL bytes, because a shell variable
   cannot hold one. A hosts file containing a NUL is rewritten truncated at
   that byte and still exits 0, so nothing observes the loss.

3. A symlink swapped in between the -h test and the > redirect is still
   followed. The -h test covers the unraced shape only. A dangling
   symlink is the worst case: it fails -e as well, and a redirect onto a
   dangling link creates the link's target, which on a writable host bind
   mount lands outside the container.

All three come from the same constraint -- the command has to work on BusyBox,
so it is restricted to grep and printf, and neither can open a file once
and rewrite it in place.

Proposed fix

Do the rewrite from Rust against /proc/<init_pid>/root/etc/hosts, which is
the container's own mount namespace as seen from the host, so no helper binary
has to exist inside the container. Open with O_NOFOLLOW, stream the filter
rather than buffering the file, and write through the same descriptor.

Two things to get right, and neither is free:

• O_NOFOLLOW only protects the final component. A hostile /etc inside the
  container still redirects the open, so this wants openat2 with
  RESOLVE_NO_SYMLINKS, or a component-by-component openat walk, rather
  than a single open on the joined path.
• The current command runs inside the container's namespaces via attach_run.
  Reaching the file through /proc/<pid>/root from the host instead changes
  which credentials the write happens under, so the permission model needs to
  be re-checked rather than assumed equivalent.

Not urgent

The pin is only written when a proxy is configured, /etc/hosts is normally
a few hundred bytes, and gaps 2 and 3 both need a workload that is already
manipulating its own /etc/hosts. This is correctness and robustness debt,
not a live break.

Raised by Gudge (@MGudgin) in review on #798:
#798 (comment)

LXC firewall mode silently drops unresolvable blockedHosts entries and still reports the policy enforced

src/backends/lxc/common/src/network_iptables.rs skips a blockedHosts entry that does not resolve, and the start still reports the policy enforced.

LXC veth discovery reads a failed probe as no interface, so firewall setup can succeed with no FORWARD hook

src/backends/lxc/common/src/network_iptables.rs reads a failed veth probe as no interface, so the FORWARD hook is never installed.

[LXC] An allowedHosts hostname is pinned to the IPs the host resolved, so a container resolving it differently is blocked

resolve_host at src/backends/lxc/common/src/network_iptables.rs:835 expands each
allowedHosts name to the addresses the host sees and programs those as literal
iptables destinations. The container resolves the name again when it connects. Divergent
answers mean an allowed destination is dropped while the policy reports as enforced.

Proven by three CI attempts on one unchanged commit (3 failed suites, then 1, then 0), with
api.github.com timing out on 140.82.114.6 and then 172.182.252.137 -- GitHub and
Azure ranges respectively. Distinct from #877, which covers names that do not resolve at all.