Feature gap: DataPlane interface contract is BPF-shaped; userspace Manager embeds eBPF Manager

[psaab]

Gap

The dataplane.DataPlane interface (pkg/dataplane/dataplane.go:104-300) is BPF-shaped: ~80 methods are direct BPF map writers (SetZone, SetPolicyRule, SetSNATRule, SetDNATEntry, SetNAT64Config, SetScreenConfig, SetMirrorConfig, SetPolicerConfig, SetFlowConfig, etc., plus the matching ClearXxx / DeleteStaleXxx / ReadXxxCounter methods). The userspace Manager (pkg/dataplane/userspace/manager.go:57-123) embeds dataplane.DataPlane — its interface satisfaction comes from the embedded eBPF dataplane.Manager. Every config write call from the daemon today still flows through bpf_map_update_elem even when running on userspace-dp; the userspace XDP shim doesn't consult those maps. After Phase 4 (BPF source removal) those maps don't exist, and every embedded method becomes either undefined or a compile failure.

This isn't a forwarding feature gap — it's the Phase-3-blocking plumbing concern that all Phase 0–2 work must surface to.

eBPF implementation (source of truth)

• pkg/dataplane/dataplane.go:104-300 — full DataPlane interface, 80+ methods, every one BPF-shaped
• pkg/dataplane/dataplane.go:13 — var _ DataPlane = (*Manager)(nil) — eBPF Manager satisfies it directly
• pkg/dataplane/userspace/manager.go:57-58:

  type Manager struct {
      dataplane.DataPlane     // <- this embed
      inner *dataplane.Manager

• pkg/dataplane/userspace/manager.go:148-156 — New() returns &Manager{ DataPlane: inner, inner: inner, ... } (the embed IS the eBPF manager)
• pkg/dataplane/userspace/snapshot.go:458-472 — userspaceMapPins() references BPF pin paths the userspace helper opens (Ctrl, Bindings, Heartbeat, XSK redirect map, conntrack v4/v6, DNAT tables, Trace)

Userspace-dp gap

pkg/dataplane/userspace/manager.go adds only ~5 methods of its own (Mode, EventStream, Status, SetForwardingArmed, InjectPacket, DrainSessionDeltas, etc.) — the other ~75 methods come from the embedded eBPF type. There's NO standalone DataPlane implementation for userspace.

Recommended fix

Phase 3 plan needs to address this before Phase 4 can land. Two reasonable approaches:

1. Split the interface: define ConfigSink (high-level: ApplyConfig(*config.Config) error) and ControlPlane (read-only: Status, Counters, Sessions). Move BPF-specific methods onto *ebpf.Manager directly, not the abstract interface. Userspace Manager implements only the new high-level interface.

2. Stub out BPF writers in userspace: keep the DataPlane interface name but replace every Set*/Clear*/Delete* method on userspace Manager with a no-op (or no-op-with-snapshot-mutation). Daemon code that builds BPF entries continues to compile but is dead code; remove at Phase 3.

(1) is cleaner; (2) is faster to land. Either way, this issue is the central plumbing problem Phase 3 must resolve before Phase 4 can rm -rf bpf/ without orphaning ~80 interface methods.

The userspace XDP shim itself (userspace-xdp/src/lib.rs) is unaffected — it stays. The XSK redirect map, userspace_bindings/ctrl/heartbeat/trace pins, and conntrack pins it uses are the boundary the user-space helper crosses regardless. Only the legacy xdp_main / xdp_screen / xdp_zone / xdp_policy etc. and their associated BPF maps are retired.

Blocker for #1373

This is the Phase 3 blocker. Without splitting or stubbing this interface, pkg/dataplane/*.go cannot shrink to userspace-only and Phase 4 cannot proceed.

---

Refined contract (added 2026-05-17 after triple-review of #1383)

See docs/pr/1381-dataplane-interface-split/plan.md for the full implementation contract refined through 4 rounds of Claude+Codex+Gemini Pro 3 review. New since the original issue body:

• Import-cycle avoidance contract: SessionDeltaSource interface lives in pkg/dataplane/runtime (NOT pkg/dataplane/userspace). DTOs (SessionDelta, SessionDeltaSnapshot) live in the same neutral runtime package. Public interface must not reference any pkg/dataplane/userspace type. Backed by an import canary test.
• ApplyResult widened with FilterIDs map[string]uint32, FilterSpans map[string]FilterCounterSpan (FilterID, RuleStart, RuleCount), NATCounterIDs map[string]uint32 — required by current server_show_firewall.go:41, cli_show_nat.go:352, cli_show_security.go:1398 callers.
• GC migration side-effect ownership explicitly assigned: session-change telemetry (GlobalCtrSessionsNew/Closed) → Telemetry.GlobalCounter; per-IP session-limit map publish → backend-private; persistent-NAT preservation → SessionStore.Delete* atomic or BeforeDelete hook; DNAT/NAT64 reverse cleanup → backend session/NAT store owns it; v4/v6 stats counting → backend-neutral.
• Cluster stale-reconcile path at pkg/cluster/sync.go:556,571,599 must use same SessionStore.DeleteWithCompanionsV4/V6 / ReconcileClusterBulk semantics as GC. Phase 1 acceptance gate requires tests that fail if cluster sync keeps a local DeleteDNATEntry* cleanup copy.
• Architectural-mismatch rationale added — explicitly distinguishes this plan from killed patterns Extract PacketContext: explicit per-packet ownership state machine #961 (PacketContext), Refactor: Pipeline / Chain of Responsibility Pattern for Vector Packet Processing #946 Phase 2 (batched pipeline), Refactor: SessionTable is a 'Memory Allocator' — Missing Multi-Index Data-Oriented Design #964 Step 3 (cache-line packing) so reviewers don't reject by reflex.
• Compatibility matrix (REST/gRPC/CLI/cluster sync/metrics/config-only mode) + Risks section (stale handles, lifetime ordering, atomic apply visibility, replay windows, backend escape hatches, failure attribution) + Phase 1 acceptance gate with explicit go/no-go criteria.

[psaab]

Implementation contract

This is the architectural precondition for #1374, #1375, #1376, #1377, #1378, #1379, #1380. Land first.

Acceptance criteria

1. pkg/dataplane/userspace/manager.go::Manager does NOT embed dataplane.DataPlane. The struct's interface satisfaction is explicit, not inherited from the eBPF Manager.
2. The dataplane.DataPlane interface is either:
   • Split into ConfigSink (high-level: ApplyConfig(*config.Config) error) + ControlPlane (read-only: Status, Counters, Sessions), with BPF-specific Set*/Clear*/Delete* methods relocated onto *ebpf.Manager (eBPF-only); OR
   • Stubbed — userspace Manager provides no-op implementations for the ~75 BPF-shaped writers, with the daemon code path for those writers marked deprecated for Phase 4 removal.
3. Method count on the abstract interface drops from ~80 to ≤15 (split) or stays at ~80 with all userspace stubs returning early (stub).
4. go build ./... clean; go test ./... passes (including embedded compile-test that userspace Manager does not pull in cilium/ebpf symbols).
5. pkg/daemon/ callers that use the now-eBPF-only methods are routed through the appropriate eBPF/userspace branch in ApplyConfig.

Test plan

• go build ./... — no unresolved references
• go test ./pkg/dataplane/... ./pkg/dataplane/userspace/... ./pkg/daemon/... — green
• New test: TestUserspaceManagerDoesNotEmbedEbpfDataPlane asserts via reflect that the userspace Manager does not have an embedded *ebpf.Manager field exposing the BPF methods
• Compile-time canary: a deliberate call to Set*BpfMap from userspace path should fail to compile (split) or hit a panic/error path (stub)
• Full smoke matrix on loss:xpf-userspace-fw0/1 — no regression in current functionality

Dependencies

None. This is the precondition.

Size estimate

L (1-2 weeks). ~80 method audit + signature reorganization. Daemon-side caller migration. New test coverage. Worth the effort because every other #1373 phase needs this to be done.

[psaab]

Plan-review round 1 — PLAN-NEEDS-MAJOR

Reviewer	Verdict
Codex	PLAN-NEEDS-MAJOR (4 substantive findings)
Gemini Pro 3	PLAN-KILL (scope-overread — see below)

Codex findings (verified)

1. Interface is 130 methods, not 80. Current pkg/dataplane/dataplane.go:104 exposes ~130 methods spanning loader lifecycle, BPF program attachment, config writes, runtime session state, counters, stale-map cleanup, HA/fabric control, event sources, and raw Map(name). The split needs to be more nuanced than ConfigSink/ControlPlane.
2. Stub-mode is a trap. Userspace Manager still implements the full interface; daemon-side compilers (pkg/dataplane/compiler.go:239-240, compiler_filter.go:47) unconditionally drive writers like port mirroring and policers. Silent no-ops mask missing features. Stub only acceptable as short-lived scaffolding with failing follow-up canaries.
3. Read/write split is leaky. UpdatePolicyScheduleState reads then writes (pkg/dataplane/maps.go:1495); BumpFIBGeneration is lookup + update; DeleteStale* methods iterate then delete. Split by domain / ownership, not verb shape.
4. Restart behavior under-specified. Userspace snapshots include BPF pin paths (pkg/dataplane/userspace/snapshot.go:458); Rust helper still opens BPF session/DNAT maps (coordinator/mod.rs:486). Plan must name which state stays pinned-BPF (none, post-Phase-4), which moves to userspace, and how stale cleanup + restart replay continues.

Plus Codex's reflect-canary critique: current test should reject BOTH dataplane.DataPlane embedding AND named inner *dataplane.Manager. Use go/types/AST for robustness.

Size revision: XL (2-4 weeks), not L.

Gemini PLAN-KILL — overread

Gemini's KILL rationale: "stubbing BPF writers starves the eBPF fallback pipeline for ModeUserspaceCompat traffic (GRE/ESP); breaks immediately."

Cross-check: this is true today for the userspace-with-eBPF-fallback hybrid mode. It is irrelevant for #1373's Phase 4 endpoint — after eBPF removal, there IS no fallback to starve. Gemini reasoned about the transition period as if it were the steady state. Documented "Gemini widens scope" pattern; #1373 phases 0-3 are explicitly designed to handle the transition before Phase 4 kills the fallback. Treating as MAJOR convergent with Codex's stub-trap finding, not as KILL.

Recommended plan v2

1. Drop "stub" path entirely. Commit to split-by-domain.
2. Define the exact new interfaces with concrete methods. Suggested domains: ConfigSink (high-level ApplyConfig), Lifecycle (Start/Stop/Reload), Telemetry (Status/Counters/Events), SessionControl (Sessions/Conntrack), HA (chassis cluster control). 130-method split goes from a single bag to ~5 narrow interfaces.
3. Resolve UpdatePolicyScheduleState explicitly. Either (a) make it part of ConfigSink.ApplyDelta, or (b) carve out a PolicyControl interface — but pick one in v2.
4. Document Phase 3 state-ownership decision per BPF pin. Userspace XDP shim keeps XSK redirect / heartbeat / ctrl / trace pins; everything else gets removed.
5. Size estimate: XL (2-4 weeks). Honest framing.
6. Strengthen the compile-time canary via go/types AST analysis, not reflection.

Plan v2 needed before implementation. PR #1373 Phase 3 stays blocked until this lands.

Codex task: `task-mp8remxh-tibcym`. Gemini Pro 3 task: `task-mp8reyvc-e1fcgx`.

[psaab]

Update after #1399: keep open. Landed: RuntimeDataPlane/domain interface scaffold, ApplyResult widening, SessionStore/reconcile scaffolding, import canary. Remaining: migrate daemon/API/gRPC/CLI/GC callers, remove legacy BPF-shaped DataPlane from the abstract surface, and remove userspace Manager embedded eBPF Manager dependency.