Skip to content

Research: our policy format compared with industry formats (closes #424) - #433

Merged
patelankeet2 merged 1 commit into
mainfrom
research/424-policy-formats
Nov 2, 2026
Merged

patelankeet2 merged 1 commit into
mainfrom
research/424-policy-formats

Conversation

@shubhamkataria2005

Copy link
Copy Markdown
Collaborator

Closes #424. One new file, docs/research/policy-formats.md. No code.

Compared against Capirca/Aerleon, Kubernetes NetworkPolicy, AWS security groups and NACLs, OpenConfig ACL and CIS benchmarks. Ours is the only one that is an assertion rather than a configuration, which is why some gaps should stay gaps.

gap recommendation
device settings (#400) borrow from CIS, only for what Batfish models. Decision stays on #400
named addresses and services borrow (Capirca NETWORK.net/SERVICES.svc), resolved at load, so no check changes
scope by role, not device name borrow the idea later (after #78). Roles would come from the business-context file, never guessed
expiry / review date borrow (Aerleon expiration), as a warning, not enforcement
rule order don't borrow. Contradictions are refused (#387), not resolved by order
stateful traffic not now. No check could verify it

Measured, not assumed: the loader refuses unknown keys (unknown key 'expiration'), so both "borrow" items are format changes and need all four. I'd bring them as one proposal before writing any code.

Capirca, Kubernetes NetworkPolicy, AWS security groups and network ACLs,
OpenConfig ACL and CIS benchmarks, each against what our format can say.
Recommends borrowing named addresses and services, and expiry as a warning,
as one proposal to the team. Order and state are deliberately not borrowed.

@patelankeet2 patelankeet2 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed by reproducing the one directly testable claim and checking the citations that a real recommendation rests on, not by reading the table and trusting it.

Reproduced the "measured, not assumed" claim directly against current main (this is pre-existing analysis/policy.py behaviour, untouched by this docs-only PR, so no branch needed):

policy.load_policy({... "expiration": "2026-01-01", ...})
# -> PolicyError: policy_compliance entry 1 ('x'): unknown key 'expiration'.
#    Allowed here: description, filter, kind, node, number, queries,
#    violation_severity, violation_summary

Matches exactly, which is the fact the whole "every gap is a format change" framing depends on.

Verified the citations that a specific recommendation rests on, live:

  • Aerleon's expiration keyword (source 2, the basis for recommendation 4) -- confirmed from the generator reference: takes a YYYY-MM-DD date, stops the term from being generated after that date. Matches "stops a term from being generated after a date" exactly.
  • Capirca's NETWORK.net / SERVICES.svc naming and source-exclude / destination-exclude (source 1, the basis for recommendation 2) -- confirmed both the file layout and that both exclude keywords exist as documented term keywords alongside source-address / destination-address.
  • OpenConfig ACL (source 6) -- confirmed sequence-id determines evaluation order and interface bindings sit under a separate /acl/interfaces/interface tree, matching the "sequence id" and "an interface binding" cells in the table.

Didn't individually re-verify Kubernetes NetworkPolicy, the two AWS pages, or the CIS Benchmarks page -- these are well-established, widely documented behaviours (label selectors, SG vs. NACL statefulness, hardening checklists) rather than a specific keyword or number riding on a less common source, so the risk of a wrong attribution there is low enough that re-deriving Aerleon/Capirca/OpenConfig felt like the right place to spend the checking.

On the actual recommendations: agree with the ordering in "What I'd do." Naming addresses/services first is the right call -- it's loader-only, every existing check and the two newer ones (policy_conflicts, policy_drift) stay untouched because resolution happens before they ever see the policy. Expiry as a warning rather than enforcement is the right shape for this project's own "never silently gate on an assumption" pattern. Declining to borrow order (rules are assertions proved independently, and #387 already refuses rather than resolves a real contradiction) and declining stateful traffic (nothing downstream could check it) are both argued from what the project can actually verify, not from what looks complete on paper -- the same discipline the rest of this PR stack has been built on.

No em-dashes, no AI-tool mentions.

Approving.

@patelankeet2
patelankeet2 merged commit f1371ba into main Nov 2, 2026
2 checks passed
@patelankeet2
patelankeet2 deleted the research/424-policy-formats branch November 2, 2026 01:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Research: our policy format compared with real industry formats

2 participants