Environment
- Host has dual NICs: eth0 (internal network), eth1 (external network)
- eth0 gateway
10.20.11.1, reachable to 10.20.11.x (same subnet) and 10.69.85.x (other internal subnets)
- eth1 gateway is an external gateway, does not route internal addresses
- CubeSandbox is deployed on eth1
allow_internet_access = false, allow_out = ["10.20.11.74/32", "10.69.85.182/32"]
Problem
On the same host:
| Target |
Host curl |
Sandbox curl |
Result |
10.20.11.74:32312 (same L2 subnet) |
Reachable |
Reachable |
✅ |
10.69.85.182:80 (cross-subnet internal IP) |
Reachable (HTTP 403) |
Timeout, no SYN-ACK |
❌ |
Sandbox curl -v --connect-timeout 5 http://10.69.85.182:80 output:
* Trying 10.69.85.182:80...
* Connection timed out after 5003 milliseconds
Host routing confirms 10.69.85.182 is reachable via eth0:
$ ip route get 10.69.85.182
10.69.85.182 via 10.20.11.1 dev eth0
Root Cause
This shares the same root cause as #495: the from_cube BPF bpf_redirect bypasses the kernel routing table.
Host network topology:
┌─────────────────────────────────────┐
│ Host 10.20.11.35 │
│ │
│ eth0 (internal) eth1 (external) ← CubeSandbox bound here
│ gateway 10.20.11.1 gateway ??? │
│ reachable: reachable: │
│ 10.20.11.x internet │
│ 10.69.85.x │
└─────────────────────────────────────┘
Sandbox → from_cube BPF → SNAT → bpf_redirect(eth1_ifindex, 0) → eth1
│
10.69.85.182 is NOT reachable via eth1's gateway
→ SYN goes out the wrong NIC, never reaches target
Why 10.20.11.74 works: It shares the same L2 broadcast domain as eth1 (both are 10.20.11.x). eth1 can ARP directly, no gateway needed.
Why 10.69.85.182 fails: Cross-subnet, requires gateway 10.20.11.1 which is only reachable via eth0. bpf_redirect forces all traffic to eth1, whose external gateway does not know how to route 10.69.85.x. The packet is dropped.
Expected Behavior
The from_cube BPF should support per-destination NIC selection (policy routing), or at minimum allow snat_iplist to carry per-NIC SNAT IP/ifindex pairs and select the correct egress interface based on the destination address.
Affected Components
- from_cube BPF (TC ingress on TAP)
- snat_iplist map
- CubeNet / network-agent
Relationship to #495
#495 describes "allow internet traffic to egress via a non-default NIC (GRE tunnel)". This issue describes the reverse: "allow internal traffic to egress via the correct NIC when CubeSandbox is bound to an external-facing NIC". Both share the same root cause — bpf_redirect lacks multi-NIC policy routing capability.
Environment
10.20.11.1, reachable to10.20.11.x(same subnet) and10.69.85.x(other internal subnets)allow_internet_access = false,allow_out = ["10.20.11.74/32", "10.69.85.182/32"]Problem
On the same host:
10.20.11.74:32312(same L2 subnet)10.69.85.182:80(cross-subnet internal IP)Sandbox
curl -v --connect-timeout 5 http://10.69.85.182:80output:Host routing confirms
10.69.85.182is reachable via eth0:Root Cause
This shares the same root cause as #495: the
from_cubeBPFbpf_redirectbypasses the kernel routing table.Why
10.20.11.74works: It shares the same L2 broadcast domain as eth1 (both are10.20.11.x). eth1 can ARP directly, no gateway needed.Why
10.69.85.182fails: Cross-subnet, requires gateway10.20.11.1which is only reachable via eth0.bpf_redirectforces all traffic to eth1, whose external gateway does not know how to route10.69.85.x. The packet is dropped.Expected Behavior
The
from_cubeBPF should support per-destination NIC selection (policy routing), or at minimum allowsnat_iplistto carry per-NIC SNAT IP/ifindex pairs and select the correct egress interface based on the destination address.Affected Components
Relationship to #495
#495 describes "allow internet traffic to egress via a non-default NIC (GRE tunnel)". This issue describes the reverse: "allow internal traffic to egress via the correct NIC when CubeSandbox is bound to an external-facing NIC". Both share the same root cause —
bpf_redirectlacks multi-NIC policy routing capability.