Skip to content

Userspace three-color policer core#1395

Merged
psaab merged 7 commits into
masterfrom
codex/1375-userspace-three-color-core
May 17, 2026
Merged

Userspace three-color policer core#1395
psaab merged 7 commits into
masterfrom
codex/1375-userspace-three-color-core

Conversation

@psaab
Copy link
Copy Markdown
Owner

@psaab psaab commented May 17, 2026

Summary

Tests

  • cargo test --manifest-path userspace-dp/Cargo.toml policer
  • git diff --check

Remaining gaps

  • Go userspace snapshot still carries only single-rate two-color policer fields
  • userspace capability gate still rejects three-color policers until snapshot/runtime forwarding are wired
  • forwarding path and flow-cache hits do not yet consume the three-color meter
  • per-color counters/status/CLI/Prometheus export remain follow-on wiring

Copilot AI review requested due to automatic review settings May 17, 2026 01:53
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR introduces a userspace three-color policer core (RFC 2697 srTCM and RFC 2698 trTCM) as the math/state foundation for issue #1375, while also migrating the legacy two-color token bucket off f64 to scaled u128 integer math. The change is intentionally scoped to the policer module: snapshot plumbing, forwarding-path wiring, flow-cache integration, per-color counters/status, and removal of the Go capability gate are explicitly deferred and called out in both the PR description and the updated README.

Changes:

  • Replace f64-based token math in legacy PolicerState with u128 tokens scaled by TOKEN_SCALE=1e9 and a rate_bits_per_sec: u64 refill rate.
  • Add ThreeColorPolicerState implementing srTCM and trTCM with color-aware/color-blind modes, ColorTreatment (DSCP rewrite / drop) per color, and validation errors.
  • Add focused unit tests covering threshold transitions, srTCM C→E overflow, trTCM independent CIR/PIR refill, color-aware non-promotion, color-blind ignore, u64-boundary inputs, and DSCP rewrite plumbing.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated no comments.

File Description
userspace-dp/src/filter/README.md Documents the integer token-math switch and the new three-color meter core, plus an explicit list of still-gated follow-on work.
userspace-dp/src/filter/policer.rs Reworks PolicerState to scaled-u128 tokens, adds ThreeColorPolicerState, PacketColor, ColorTreatment, ThreeColorTreatments, ThreeColorDecision, PolicerConfigError, and unit tests.

@psaab
Copy link
Copy Markdown
Owner Author

psaab commented May 17, 2026

Claude round-1 review on ccbda62b

Verdict: MERGE-READY (foundation only — three-color core for #1375)

Meter core only. Capability gate remains closed. The README explicitly lists what's still gated.

What's in scope

498+/18- in userspace-dp/src/filter/policer.rs + README. Adds:

  • srTCM (RFC 2697): single committed + excess buckets, refill at CIR
  • trTCM (RFC 2698): independent committed + peak buckets at CIR/PIR
  • Color-blind + color-aware classification
  • DSCP rewrite + drop decisions in meter result

Token math (the #1375 risk)

policer.rs:1-20:

const TOKEN_SCALE: u128 = 1_000_000_000;

fn refill_scaled(rate_bytes_per_sec: u64, elapsed_ns: u64) -> u128 {
    u128::from(rate_bytes_per_sec) * u128::from(elapsed_ns)
}

Multiplication stays in u128. Max rate × max elapsed: 2^64 × 2^64 = 2^128 — fits, no overflow. ✓ matches the contract requirement.

Color-aware preservation (the #1375 critical safety)

Per README:

"Color-aware classification never promotes incoming yellow or red packets."

This is THE critical invariant — if yellow gets promoted to green, the security control becomes a bandwidth grant. Codex/Gemini will verify the actual branch in policer.rs follows this.

Still gated (per README)

  • Go snapshot publishes only two-color PolicerSnapshot fields; three-color snapshot fields and validation NOT yet wired to Rust
  • Filter terms carry policer name; hot path must move to stable policer IDs before three-color enablement
  • Flow-cache hits don't execute policer decisions yet
  • Per-color counters, red drops, DSCP rewrites, Rust/Go status, CLI, Prometheus — all follow-on wiring

Capability gate stays in place. ✓

Hostile checks

  • PIR<CIR / PBS<CBS rejection at config compile? Need to verify in Go-side compile path (out-of-scope for this PR — checking the Rust core's behavior when given invalid config).
  • Wraparound: monotonic clock — saturating arithmetic if now < last_refill_ns?
  • Atomicity: is policer state shared across workers? Sharded? false-sharing safe?
  • Counter attribution: survive snapshot rebuild by stable identity?

Recommendation

Block on: nothing structural for the foundation slice.

Strongly consider: confirm the color-aware branch never promotes (Codex/Gemini will trace).

Defer to follow-up PRs: Go-side three-color snapshot publish, stable policer IDs for hot path, flow-cache policer execution, per-color counters + Prometheus, capability gate removal.

Awaiting Codex (task-mp955bjw-cz8j4y) and Gemini Pro 3 (task-mp9565kp-5zyyy1).

@psaab
Copy link
Copy Markdown
Owner Author

psaab commented May 17, 2026

Claude round-1 self-correction on ccbda62b

Revising verdict from MERGE-READY → MERGE-NEEDS-MAJOR to match Codex. Same pattern as #1394 and #1396 — I labeled this "foundation only" without verifying integration with the actual capability gate / snapshot path.

Codex MAJOR — #1375 not wired

"Userspace still rejects configs with Firewall.ThreeColorPolicers at manager.go:767, the userspace snapshot schema still has only two-color PolicerSnapshot fields at protocol.go:331, and Rust protocol mirrors only those fields at protocol.rs:548."

The core is implemented; the Go capability gate still rejects; the Go snapshot doesn't carry three-color fields; the Rust protocol doesn't deserialize three-color either. So the new core is dormant code with no caller path. The PR's README admits this, but admission ≠ contract progress.

Codex MAJOR — Compile-time invariants not enforced at Go boundary

"PIR < CIR and PBS < CBS are rejected only by the new Rust constructor at policer.rs:254/257. The Go config compiler parses three-color policers at compiler_firewall.go:64-137 with no equivalent validation."

Operator config with invalid PIR < CIR would commit successfully on Go side, then fail at Rust constructor at the dataplane level. Should fail at commit check time, not dataplane time. Per #1375 contract: "validated at config compile, rejected before reaching dataplane."

Codex MAJOR — No per-color counters / stable identity

"There are no green/yellow/red/drop counters in the snapshot, Rust status, or forwarding state."

Per #1375 contract: "green/yellow/red/drop counters survive snapshot rebuilds by stable identity." This PR doesn't add counters at all. Existing Arc<FilterTermCounter> is filter-term-keyed, not policer-keyed, so it can't satisfy stable per-policer attribution.

Codex MAJOR — No atomicity story for production

"ThreeColorPolicerState is a plain mutable struct at policer.rs:161, not sharded, not cache-line padded, not atomic, and not ID-indexed."

The README admits this is required before enablement, but as merged the struct is single-owner. Multi-worker access requires sharding/CAS. The hot path can't use this state until that's resolved.

Codex finding — Wraparound underflow-safe but not wrap-recovering

"The core uses if now_ns <= self.last_refill_ns { return; }, not saturating_sub. That prevents subtraction underflow, but after a true u64 monotonic wrap the policer stops refilling until time exceeds the old pre-wrap value."

u64 nanosecond wrap is at ~584 years from epoch start. Probably acceptable in practice but not a clean monotonic-clock contract.

What's correctly verified (passing)

  • u128 refill multiplication: Codex confirms refill math stays in u128. ✓
  • Color-aware preservation: Codex confirms yellow/red are never promoted to green in the new core. ✓
  • Test coverage of math core: srTCM/trTCM thresholds, color-aware no-promotion, u128 boundary, DSCP decision all covered.

Claude miss pattern

Same as #1394 and #1396: I labeled "foundation core only" without checking whether the Go capability gate / snapshot / counter / atomicity machinery existed to actually USE the core. Codex consistently catches this.

Recommendation (revised)

Block on:

  1. Wire Go-side three-color PolicerSnapshot + Rust protocol deserialization.
  2. Add PIR < CIR / PBS < CBS validation at Go commit check, not just Rust constructor.
  3. Define stable policer ID + sharded atomic state plan (or land it in this PR if scope can absorb it).
  4. Add per-color/drop counters in the snapshot schema.

Strongly consider: Saturating-arithmetic wrap-recovery for monotonic clock.

Defer: capability gate removal, forwarding-path flow-cache execution, Prometheus export (acknowledged in README, follow-up PRs).

Codex task: task-mp955bjw-cz8j4y. Gemini Pro 3 task: task-mp9565kp-5zyyy1. Not merging — author's decision.

@psaab
Copy link
Copy Markdown
Owner Author

psaab commented May 17, 2026

Round-1 quad-review consolidated synthesis on ccbda62b

Reviewer Verdict
Claude MERGE-READY → MERGE-NEEDS-MAJOR (self-corrected)
Codex MERGE-NEEDS-MAJOR
Gemini Pro 3 MERGE-READY
Copilot (no review yet on this HEAD)

Reviewer disagreement: scope philosophy

Codex and Gemini disagree on the same code. Both verified the math core is correct (u128 multiplication, color-aware preservation, srTCM/trTCM bucket math, DSCP rewrite, runtime PIR<CIR validation). Both verified test coverage of the math core. The disagreement is what counts as merge-ready:

Codex MAJORs (real gaps if scope = full #1375)

  1. Not wired Go-sidemanager.go:767 still rejects three-color configs; protocol.go:331 snapshot has only two-color fields; userspace-dp/src/protocol.rs:548 mirrors only those. New core is dormant code with no caller.
  2. Compile-time validation missingpkg/config/compiler_firewall.go:64-137 parses three-color configs but doesn't validate PIR<CIR / PBS<CBS at commit. Invalid config commits, then fails at dataplane construction. Per Feature gap: three-color policers (RFC 2697/2698) implemented in eBPF, missing from userspace-dp #1375 contract: validate at compile.
  3. No per-color/drop counters — required by Feature gap: three-color policers (RFC 2697/2698) implemented in eBPF, missing from userspace-dp #1375 contract for "stable identity across snapshot rebuilds." Existing FilterTermCounter is filter-term-keyed, not policer-keyed.
  4. No atomicity storyThreeColorPolicerState is plain &mut self. Multi-worker access requires sharding/CAS. README admits this is required before enablement.

What both reviewers verified correct

  • u128 refill math (no overflow at any reasonable rate × elapsed)
  • Color-aware never promotes incoming yellow/red (lines 284-301 of policer.rs)
  • trTCM independent CIR/PIR bucket refill
  • DSCP rewrite per color
  • Runtime PIR<CIR rejection at constructor (just not at config compile)

Claude miss pattern

Same as #1394 and #1396: labeled "foundation only" without verifying production caller path. Codex consistently flags this; Gemini accepts the README disclaimers.

My revised read (matching Codex): the scope question matters. If this PR claims to advance #1375, the Go-side wiring + commit validation are the load-bearing pieces and should be in scope. If this PR is explicitly the math core + atomicity-deferred, the README should say "Phase 1 of 4" not just "still gated."

Recommendation

Block on:

  1. Add PIR<CIR / PBS<CBS validation at Go commit check (pkg/config/compiler_firewall.go), not just Rust constructor.
  2. Decide on a counter shape (per-color, per-policer-ID) and include it in the foundation slice OR explicitly say "counters in PR #N."

Strongly consider:

  • Sharded atomic state plan documented (even if not implemented).
  • Saturating-arithmetic wrap-recovery instead of if now_ns <= last_refill_ns return.

Defer to follow-up PRs (acknowledged by README):

  • Go three-color snapshot wire format
  • Hot-path stable policer IDs
  • Flow-cache policer execution
  • Capability gate removal

Codex task: task-mp955bjw-cz8j4y. Gemini Pro 3 task: task-mp9565kp-5zyyy1. Not merging — author's decision.

@psaab
Copy link
Copy Markdown
Owner Author

psaab commented May 17, 2026

Round-2 update: schema + validation wired, runtime still fail-closed

Pushed 61ccc1b7.

What changed:

  • Added three_color_policers to the Go ConfigSnapshot wire schema and Rust DTO.
  • Publish deterministic single-rate/two-rate three-color policer snapshots from buildSnapshot.
  • Added commit-time structural validation for missing/invalid CIR/CBS/PBS/PIR and PIR/PBS ordering.
  • Kept userspace forwarding fail-closed when any firewall three-color-policer exists.
  • Updated userspace-dp/src/filter/README.md so the module docs match the new invariant: schema/validation are wired, but runtime application/counters/status/Prometheus remain gated until stable ID-indexed state and hot-path application exist.

Validation run:

  • go test ./pkg/config
  • go test ./pkg/dataplane/userspace
  • cargo test --manifest-path userspace-dp/Cargo.toml config_snapshot_three_color_policers_roundtrip -- --nocapture
  • cargo test --manifest-path userspace-dp/Cargo.toml policer -- --nocapture
  • git diff --check

The Rust runs pass with the repository's existing warning set.

@psaab
Copy link
Copy Markdown
Owner Author

psaab commented May 17, 2026

Claude round-2 review on 61ccc1b7

Verdict: MERGE-READY-pending (commit-time validation + Go-side schema wired; counter/atomicity still deferred but framed correctly)

Round-2: 9 files +371/-59. PR comment claims "wired three-color policer schema/strict validation while keeping userspace forwarding fail-closed until runtime/counters land."

Round-1 MAJORs status (pending Codex verification):

1. Commit-time validation — PR claims fixed. pkg/config/compiler.go (+39) + parser_ast_test.go (+68). Codex will verify PIR<CIR / PBS<CBS rejected at commit time, not just Rust constructor.

2. Go-side schema — PR claims fixed. pkg/dataplane/userspace/protocol.go (+70/-15) + Rust protocol.rs (+21). Should now carry three-color snapshot fields.

3. Userspace forwarding fail-closed — verified. PR comment is explicit: "keeping userspace forwarding fail-closed until runtime/counters land." This is the right scope discipline — admit at Go schema level, reject at userspace runtime.

4. Per-color counters — STILL DEFERRED. PR comment explicitly notes "runtime/counters land" as follow-up. Codex r1 wanted counters in this slice; author argues separate PR. Reasonable boundary if the README is clear.

5. Atomicity — STILL DEFERRED. Same framing.

Awaiting Codex (task-mp96mmn0-4rvzhn).

@psaab
Copy link
Copy Markdown
Owner Author

psaab commented May 17, 2026

Round-2 review consolidated on 61ccc1b7

Reviewer Verdict
Claude MERGE-READY-pending → MERGE-NEEDS-MINOR (aligning with Codex)
Codex MERGE-NEEDS-MINOR
Copilot (no fresh review on this HEAD)

Round-1 MAJORs status — all closed cleanly

Round-1 MAJOR Status
Not wired Go-side three_color_policers field added; manager.go:767 capability gate stays closed for runtime
Commit-time PIR<CIR / PBS<CBS validation compiler.go rejects before dataplane construction
Per-color/drop counters ⊝ Acceptable deferral while forwarding stays gated
Atomicity story ⊝ Acceptable deferral while forwarding stays gated

The "deferred while gated" framing works because the userspace capability gate explicitly rejects three-color configs from reaching the dataplane. Author can wire schema + validation now, defer counters + atomicity until runtime ungating.

Codex r2 MINORs

1. Ambiguous configs not commit-rejected

"'Strict validation' still accepts ambiguous three-color policers with both single-rate and two-rate; compiler_firewall.go processes both and lets two-rate win. It also does not reject both color-aware and color-blind. Not runtime-dangerous while userspace stays gated, but it should be commit-rejected before ungating."

A config with conflicting mode flags should fail commit check, not silently pick one. Fix is small: reject in compiler.go when both flags are set.

2. End-to-end commit test gap

"PIR/CIR and PBS/CBS rejection is tested via CompileConfig, not configstore.CommitCheck/Commit. The commit path does call compileTree -> CompileConfig, so real commits should fail, but the PR claim would be better pinned with one configstore test."

CompileConfig is called by the commit path, so the rejection should work in practice. One configstore-level test would lock the contract.

Recommendation

Block on: nothing structural — both MINORs are small.

Strongly consider in this PR:

  • Reject single-rate AND two-rate set simultaneously at commit
  • Reject color-aware AND color-blind set simultaneously at commit
  • Add one configstore.CommitCheck/Commit test exercising PIR<CIR rejection end-to-end

Defer to follow-up PRs (acknowledged): per-color counters, sharded atomic state, hot-path stable policer IDs, capability gate removal.

Codex task: task-mp96mmn0-4rvzhn. Not merging — author's decision.

Copilot AI review requested due to automatic review settings May 17, 2026 03:19
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 13 out of 13 changed files in this pull request and generated no new comments.

@psaab
Copy link
Copy Markdown
Owner Author

psaab commented May 17, 2026

Round-3 fix update

Pushed d36b0e7d to close the r2 MINORs.

What changed:

  • The compiler now retains explicit three-color policer mode/color markers for validation.
  • Commit validation rejects ambiguous single-rate + two-rate configs.
  • Commit validation rejects ambiguous color-blind + color-aware configs.
  • Added parser-level and configstore-level regression tests.
  • Updated userspace-dp/src/filter/README.md so the module docs match the validation behavior.

Validation:

  • go test ./pkg/config ./pkg/configstore
  • git diff --check

@psaab
Copy link
Copy Markdown
Owner Author

psaab commented May 17, 2026

Claude round-3 review on d36b0e7d

Verdict: MERGE-READY (both round-2 MINORs addressed)

Round-3 diff: 6 files, +81/-7. Small targeted fix matching the two MINOR findings.

Round-2 MINOR 1 (ambiguous configs) — CLOSED ✓

pkg/config/compiler_firewall.go +10 + parser_ast_test.go +24 should reject:

  • Both single-rate AND two-rate set
  • Both color-aware AND color-blind set

at commit-check time.

Round-2 MINOR 2 (configstore E2E test) — CLOSED ✓

pkg/configstore/store_test.go +25 adds the configstore-level rejection test.

Recommendation

Block on: nothing.
Defer: counters, atomicity, capability gate removal (still acknowledged in README as follow-up).

Awaiting Codex (task-mp99djst-s2bdia).

@psaab
Copy link
Copy Markdown
Owner Author

psaab commented May 17, 2026

Claude round-3 self-correction on d36b0e7d

Revising verdict from MERGE-READY → MERGE-NEEDS-MAJOR to match Codex.

I verified the set-command path was fixed and posted MERGE-READY without checking the alternative input paths. Codex traced load merge / load override and found the bypass.

Codex MAJOR — Duplicate hierarchical blocks bypass the check

"NewParser appends duplicate sibling nodes, LoadOverride installs that raw tree, and compileFirewall later overwrites same-name ThreeColorPolicers in the map. A config with two three-color-policer bad blocks, one single-rate and one two-rate, will compile whichever block is last instead of rejecting the combined ambiguity."

Worked failure:

three-color-policer bad {
    single-rate { ... }
}
three-color-policer bad {
    two-rate { ... }
}

NewParser appends both blocks as sibling nodes. compileFirewall walks them: first iteration sees single-rate, sets SingleRateConfigured=true. Second iteration overwrites the same-name entry in the ThreeColorPolicers map. The final compiled policer has only TwoRateConfigured set (the last write wins). The new ambiguity check at compiler.go never sees both flags on the same compiled entity.

Same pattern for color-aware + color-blind if hidden in duplicate single-rate blocks — FindChild("single-rate") only inspects the first.

Cited paths: parser.go:76, store.go:539, compiler_firewall.go:74,137.

Round-2 MINOR 2 (configstore E2E) — partially fixed

Codex notes: configstore/store_test.go adds CommitCheck coverage for ambiguous color mode, but NOT for PIR<CIR rejection. PIR<CIR is still only tested via CompileConfig in parser_ast_test.go.

Claude miss pattern

Set-command path is one of several input paths to the compiler. I verified the obvious one and missed the load-merge/load-override flow. Same pattern as the prior "didn't trace the production path" misses in #1394/#1396.

Recommendation (revised)

Block on:

  1. Reject duplicate three-color-policer <name> blocks at parse OR compile, OR accumulate flags across duplicates and reject after the merge (current "last-write wins" silently bypasses).
  2. Also reject duplicate single-rate / two-rate siblings within a single policer block.

Strongly consider: add configstore.Commit E2E test for PIR<CIR rejection (not just CompileConfig).

Codex task: task-mp99djst-s2bdia. Not merging — author's decision.

@psaab
Copy link
Copy Markdown
Owner Author

psaab commented May 17, 2026

Round-3 review consolidated on d36b0e7d

Reviewer Verdict
Claude MERGE-READY → MERGE-NEEDS-MAJOR (self-corrected)
Codex MERGE-NEEDS-MAJOR

Codex MAJOR — Hierarchical duplicate-block bypass

set-command path correctly rejects ambiguous configs (verified by Codex). But load merge / load override with duplicate hierarchical blocks bypasses the check.

Worked failure:

three-color-policer bad {
    single-rate { ... }
}
three-color-policer bad {
    two-rate { ... }
}

parser.go:76 appends both blocks as siblings. compileFirewall at compiler_firewall.go:74 walks them and overwrites same-name ThreeColorPolicers map entries — last write wins. The compiled policer has only TwoRateConfigured=true; the new ambiguity check at compiler.go never sees both flags on one compiled entity.

Same pattern hides duplicate single-rate blocks where one has color-blind and another has color-aware because FindChild("single-rate") only inspects the first.

Round-2 MINOR 2 (configstore E2E) — partial

"configstore/store_test.go only adds CommitCheck coverage for ambiguous color mode. PIR<CIR remains only covered through CompileConfig in parser_ast_test.go."

Recommendation

Block on:

  1. Reject duplicate three-color-policer <name> blocks at parse OR compile, OR accumulate flags across duplicates before the ambiguity check.
  2. Reject duplicate single-rate / two-rate siblings within a single policer block.

Strongly consider: add configstore.Commit E2E test for PIR<CIR rejection.

Codex task: task-mp99djst-s2bdia. Not merging — author's decision.

@psaab
Copy link
Copy Markdown
Owner Author

psaab commented May 17, 2026

Round-4 blocker fixes pushed: 591f4fba

Addressed the r3 MERGE-NEEDS-MAJOR finding:

  • Duplicate hierarchical firewall three-color-policer <name> blocks are now compiled into one logical policer before strict validation. This preserves Junos-style split-block/load-file behavior while preventing last-write-wins from hiding single-rate+two-rate or color-aware+color-blind ambiguity.
  • Added configstore coverage for load override with split duplicate blocks that previously collapsed in the compiler map.
  • Added configstore coverage for peak-information-rate < committed-information-rate; the rejection is now pinned through CommitCheck, not just CompileConfig.
  • Updated userspace-dp/src/filter/README.md with the duplicate-block merge invariant.

Validation:

go test ./pkg/config ./pkg/configstore
git diff --check

@psaab
Copy link
Copy Markdown
Owner Author

psaab commented May 17, 2026

Claude round-4 review on 591f4fba

Verdict: MERGE-READY-pending (small targeted fix for duplicate-block bypass)

Round-4 diff: 3 files, +73/-6:

  • pkg/config/compiler_firewall.go +12/-6 — duplicate-block rejection
  • pkg/configstore/store_test.go +61 — E2E coverage
  • userspace-dp/src/filter/README.md +6

Need Codex verification that:
A. Duplicate three-color-policer <name> blocks now rejected at compile (or flags accumulated then ambiguity-checked)
B. Duplicate single-rate/two-rate siblings within a policer rejected
C. configstore.Commit E2E test covers all of: PIR<CIR + ambiguous mode + ambiguous color via hierarchical config (not just set-command)

Awaiting Codex (task-mp9a4wiv-lhmsql).

@psaab
Copy link
Copy Markdown
Owner Author

psaab commented May 17, 2026

Claude round-4 self-correction on 591f4fba

Revising verdict from MERGE-READY-pending → MERGE-NEEDS-MAJOR to match Codex.

I posted "MERGE-READY-pending" without verifying the duplicate-block handling at the nested level. Codex traced the same pattern: top-level fixed, sub-block level still bypassed.

Codex MAJOR — Duplicate same-mode siblings still bypassed

compiler_firewall.go:78 still uses FindChild("single-rate") and FindChild("two-rate") — picks first sibling only.

Worked failure (Codex):

firewall {
  three-color-policer bad {
    single-rate {
      color-blind;
      committed-information-rate 10m;
      committed-burst-size 100k;
      excess-burst-size 200k;
    }
    single-rate {
      color-aware;
    }
  }
}

Second single-rate block is silently dropped. ColorAwareConfigured never flips. Strict ambiguity validation at compiler.go:243-247 never sees the conflict. Same pattern works to hide a duplicate two-rate with PIR < CIR.

What's actually fixed (round-4)

  • ✓ Top-level duplicate three-color-policer <name> blocks now accumulated (not last-write-wins). Codex traced reused map object at compiler_firewall.go:69-76.
  • single-rate + two-rate set on different siblings rejected (different-mode case)
  • ⊝ Duplicate same-mode siblings still bypassed

Codex test coverage gap

"E2E coverage is incomplete. It covers hierarchical LoadOverride ambiguous mode at store_test.go:114-148, set-syntax ambiguous color at 89-112, set-syntax PIR<CIR at 150-172. It does NOT cover hierarchical ambiguous color or hierarchical nested duplicate same-mode blocks."

Recommendation

Block on:

  1. Replace FindChild("single-rate") and FindChild("two-rate") with FindChildren + accumulate/reject duplicates.
  2. Add configstore.Commit test for hierarchical duplicate same-mode blocks (the Codex worked example).
  3. Add configstore.Commit test for hierarchical ambiguous color (different single-rate blocks each with one color flag).

Claude miss pattern

Same as round-2: I verified the obvious path (top-level duplicates) and didn't grep FindChild callers in the compile path. This is the 3rd self-correction on #1395 alone.

Codex task: task-mp9a4wiv-lhmsql. Not merging — author's decision.

@psaab
Copy link
Copy Markdown
Owner Author

psaab commented May 17, 2026

@copilot review

Round-4 commit 591f4fb. Codex flagged 1 MAJOR: duplicate same-mode siblings still bypassed via FindChild taking first only. Worked failure: two single-rate blocks where second's color-aware silently overrides first's color-blind. Re-review against this specific failure mode at compiler_firewall.go:78.

Copilot AI review requested due to automatic review settings May 17, 2026 04:42
@psaab psaab review requested due to automatic review settings May 17, 2026 04:42
Copilot AI review requested due to automatic review settings May 17, 2026 04:44
@psaab psaab review requested due to automatic review settings May 17, 2026 04:44
Copilot AI review requested due to automatic review settings May 17, 2026 04:44
@psaab psaab review requested due to automatic review settings May 17, 2026 04:44
Copy link
Copy Markdown
Contributor

Copilot AI commented May 17, 2026

@copilot review

Round-4 commit 591f4fb. Codex flagged 1 MAJOR: duplicate same-mode siblings still bypassed via FindChild taking first only. Worked failure: two single-rate blocks where second's colo...

Fixed on 3f79822 (plus core fix in ac55aee): compileFirewall now merges all sibling single-rate/two-rate blocks using FindChildren(...), so the second same-mode block is no longer dropped. Added hierarchical regressions in pkg/config/parser_ast_test.go and pkg/configstore/store_test.go that now fail on split color-blind + color-aware across duplicate single-rate siblings.

@psaab
Copy link
Copy Markdown
Owner Author

psaab commented May 17, 2026

Round-4 quad-review consolidated synthesis on 591f4fba

Reviewer Verdict
Claude MERGE-NEEDS-MAJOR (self-corrected from READY-pending after Codex MAJOR)
Codex MERGE-NEEDS-MAJOR
Gemini Pro 3 MERGE-NEEDS-MAJOR
Copilot Re-review requested; still on stale commit d36b0e7d

Codex + Gemini converge on the same single MAJOR. Block.

MAJOR — Duplicate same-mode siblings silently bypassed (Codex + Gemini)

pkg/config/compiler_firewall.go:78:

if sr := tcpInst.node.FindChild("single-rate"); sr != nil {

FindChild only returns the FIRST matching child. Duplicate same-mode siblings in one three-color-policer block are silently dropped.

Worked failure (Codex + Gemini both reproduced):

firewall {
  three-color-policer bad {
    single-rate { color-blind; committed-information-rate 10m; committed-burst-size 100k; excess-burst-size 200k; }
    single-rate { color-aware; }   // <-- SILENTLY DROPPED
  }
}

Second single-rate is silently ignored. ColorAwareConfigured never flips to true. Strict ambiguity validation at pkg/config/compiler.go:243-247 (pol.ColorBlindConfigured && pol.ColorAwareConfigured) is statically unreachable for sibling conflicts. Policer compiles with hidden color-aware parameters.

Round-3 wins (both reviewers confirm)

✓ Top-level duplicate three-color-policer <name> blocks now accumulated at compiler_firewall.go:69-76 (verified by Gemini quote):

tcp := fw.ThreeColorPolicers[tcpInst.name]
if tcp == nil { tcp = &ThreeColorPolicerConfig{ ... }; fw.ThreeColorPolicers[tcpInst.name] = tcp }

✓ Cross-mode siblings (single-rate + two-rate in same parent) are rejected.

Test coverage gap (both reviewers)

pkg/configstore/store_test.go covers:

  • LoadOverride ambiguous mode across blocks (114-148) ✓
  • Set-syntax ambiguous color (89-112) ✓
  • Set-syntax PIR<CIR (150-172) ✓

Missing:

  • ⊝ Hierarchical duplicate same-mode blocks (the worked failure above)
  • ⊝ Hierarchical ambiguous color (single-rate{color-blind} + single-rate{color-aware})

Gemini follow-up finding (defer)

then block inside three-color-policer likely uses the same FindChild pattern → sibling then blocks similarly dropped. Recommends a sweep changing semantic-weight FindChild lookups to FindChildren loops. Defer to follow-up issue.

Block on

  1. Replace FindChild("single-rate") and FindChild("two-rate") at compiler_firewall.go:78+ with FindChildren iteration; either accumulate fields (consistent with top-level behavior) or reject duplicates explicitly.
  2. Add configstore.Commit test for hierarchical duplicate same-mode blocks (the Codex worked example).
  3. Add configstore.Commit test for hierarchical ambiguous color.

Round-4 claim trace

  • Codex task-mp9a4wiv-lhmsql — MAJOR verified at file:line
  • Gemini task-mp9adl0r-xsi6ut — independently reproduced with full input/output trace
  • Copilot d36b0e7d — stale; re-review @copilot requested

Not merging — author's decision.

Copy link
Copy Markdown
Contributor

Copilot AI commented May 17, 2026

Round-4 quad-review consolidated synthesis on 591f4fba

| Reviewer | Verdict |...

Addressed on ac55aee (with cleanup in 361f431): compiler_firewall.go now uses FindChildren("single-rate") / FindChildren("two-rate") and merges all same-mode siblings into one logical policer before strict validation. Added hierarchical regressions in pkg/config/parser_ast_test.go and pkg/configstore/store_test.go to pin the split-sibling color-blind + color-aware failure path. Re-ran go test ./pkg/config ./pkg/configstore on current HEAD (3f79822).

Copilot finished work on behalf of psaab May 17, 2026 04:48
@psaab
Copy link
Copy Markdown
Owner Author

psaab commented May 17, 2026

Round-5 verification on 3f79822a

I read the round-4 feedback and verified the current Copilot SWE-agent fix against the blocker.

compileFirewall now uses FindChildren(\"single-rate\") and FindChildren(\"two-rate\"), so repeated same-mode sibling blocks are merged into one logical policer before strict validation. The hierarchical duplicate same-mode regression is covered in both pkg/config/parser_ast_test.go and pkg/configstore/store_test.go.

Validation:

go test ./pkg/config ./pkg/configstore
git diff --check

I did not push changes to this PR; current HEAD already closes the round-4 blocker.

@psaab
Copy link
Copy Markdown
Owner Author

psaab commented May 17, 2026

Claude round-5 review on 3f79822a

Verdict: MERGE-READY pending Codex+Gemini r5

Round-4 MAJOR addressed cleanly. Verified against current commit:

Verifications

FindChildren — FIXED at both call sites:

pkg/config/compiler_firewall.go:78:

singleRates := tcpInst.node.FindChildren("single-rate")
if len(singleRates) > 0 {
    tcp.SingleRateConfigured = true
    tcp.TwoRate = false
}
for _, sr := range singleRates {
    ...
}

pkg/config/compiler_firewall.go:106 mirrors the same pattern for two-rate.

The optimization hoists the Configured-flag write outside the loop so it's set once per parent regardless of sibling count.

Cross-mode ambiguity still detected. If single-rate AND two-rate siblings both appear:

  • singleRates loop sets SingleRateConfigured=true, TwoRate=false
  • twoRates loop sets TwoRateConfigured=true, TwoRate=true
  • pol.SingleRateConfigured && pol.TwoRateConfigured triggers ambiguity validation at compiler.go

Color ambiguity within a single mode — now detected. Two single-rate blocks where one has color-blind and the other has color-aware both accumulate into the same tcp struct → both ColorBlindConfigured and ColorAwareConfigured set → strict validation fires.

Tests verified

  • pkg/config/parser_ast_test.go: TestThreeColorPolicerStrictValidation_HierarchicalDuplicateSameModeSiblings — directly compiles the Codex worked-failure input via NewParser + CompileConfig, asserts the "cannot configure both color-blind and color-aware" error.
  • pkg/configstore/store_test.go: TestCommitCheck_RejectsAmbiguousThreeColorPolicerAcrossLoadOverrideSameModeSiblings — same input via LoadOverride + CommitCheck, asserts same error.

Both tests exercise the production paths (compile + commit), not isolated helpers.

Defer to follow-up (Gemini r4 noted)

then block inside three-color-policer at compiler_firewall.go still uses FindChild. Sibling then blocks would still be silently dropped. Not in this PR's scope — file as follow-up issue for a sweep across all semantic-weight FindChild uses.

Recommendation

Block on: nothing. Round-4 MAJOR closed with both tests + verification.
Defer: sibling then block silent-drop (own issue).

Awaiting Codex (task-mp9bdpjj-xtxma1) and Gemini Pro 3 (task-mp9besv9-h0vof3). Not merging — author's decision.

@psaab
Copy link
Copy Markdown
Owner Author

psaab commented May 17, 2026

@copilot review

Round-5 commit 3f79822 closes round-4 MAJOR: FindChild → FindChildren for both single-rate and two-rate in compiler_firewall.go. New tests in parser_ast_test.go and store_test.go cover hierarchical duplicate same-mode siblings.

Copy link
Copy Markdown
Contributor

Copilot AI commented May 17, 2026

@copilot review

Round-5 commit 3f79822 closes round-4 MAJOR: FindChild → FindChildren for both single-rate and two-rate in compiler_firewall.go. New tests in parser_ast_test.go and store_test.go c...

Confirmed on 3f79822 (core fix in ac55aee, cleanup in 361f431): compiler_firewall.go uses FindChildren("single-rate") and FindChildren("two-rate"), so same-mode sibling blocks are merged before strict ambiguity checks. Regression coverage is present in pkg/config/parser_ast_test.go and pkg/configstore/store_test.go, and go test ./pkg/config ./pkg/configstore passes on current HEAD.

Copilot finished work on behalf of psaab May 17, 2026 05:12
@psaab
Copy link
Copy Markdown
Owner Author

psaab commented May 17, 2026

Round-5 quad-review consolidated synthesis on 3f79822a

Reviewer Verdict
Claude MERGE-READY
Codex MERGE-READY
Gemini Pro 3 MERGE-READY
Copilot Stale at d36b0e7d; no r5 review yet — re-trigger if needed

Triple converge MERGE-READY (Codex + Gemini + Claude).

Round-4 MAJOR — CLOSED

FindChild → FindChildren at pkg/config/compiler_firewall.go:78 and :109. Cross-mode ambiguity validation still fires correctly (Codex traced SingleRateConfigured && TwoRateConfigured through validateThreeColorPolicersStrict). Color ambiguity within same mode (e.g., two single-rate blocks with color-blind + color-aware) now accumulates into both flags → strict validator catches.

Tests verified by all reviewers

  • TestThreeColorPolicerStrictValidation_HierarchicalDuplicateSameModeSiblings parses + compiles Codex worked-failure input
  • TestCommitCheck_RejectsAmbiguousThreeColorPolicerAcrossLoadOverrideSameModeSiblings exercises full LoadOverride → CommitCheck → CompileConfig lifecycle

Both production paths covered.

Defer to follow-up (Gemini + Codex agreed)

Systemic FindChild pattern still vulnerable to same silent-drop bug:

  • three-color-policer then block at compiler_firewall.go:144
  • Legacy policer if-exceeding and then blocks
  • Filter term from and then blocks

Codex notes Gemini correctly identifies these as "pre-existing compiler weaknesses" outside this PR's targeted scope. Recommended follow-up issue: "Sweep semantically-weighted FindChildFindChildren across compiler_firewall.go".

Recommendation

Block on: nothing.
Defer: systemic FindChild sweep (separate issue).

Round-5 claim trace

  • Codex task-mp9bdpjj-xtxma1 — MERGE-READY, both call sites verified
  • Gemini Pro 3 task-mp9besv9-h0vof3 — MERGE-READY, full ambiguity validation traced + systemic FindChild pattern noted as defer
  • Copilot — Stale at d36b0e7d; no inline findings on 3f79822a

Not merging — author's decision.

@psaab psaab merged commit b9c364c into master May 17, 2026
2 checks passed
@psaab
Copy link
Copy Markdown
Owner Author

psaab commented May 17, 2026

@copilot review

#1395 stable at 3f79822. Prior Copilot review (d36b0e7) was stale. Codex+Gemini+Claude all returned MERGE-READY at 3f79822:

  • FindChildren replaces FindChild for both single-rate and two-rate
  • New hierarchical duplicate same-mode tests in parser_ast_test.go and store_test.go
  • Cross-mode + same-mode ambiguity validation now correctly catches color conflicts

Please review the current SHA.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants