You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
P2: One real compliance benchmark pack -- CIS Cisco IOS #220
Measured: the repository contains zero compliance-benchmark content. A grep for CIS, STIG, PCI-DSS and NIST returns 91 apparent hits, of which 80 are the letters "cis" inside Cisco and 15 are "nist" inside administrator. With word boundaries: none.
Nipper, Tufin, AlgoSec and FireMon all ship regulatory packs. It is most of what a buyer in this category is actually paying for.
CIS Cisco IOS is public and finite. Expressing it in our own policy format has a second benefit: it is the largest real test the format will ever get, and it will find whatever the format cannot say.
After adding the Phase 1 items, the board looks like this for you:
#218 Multi-device snapshots size M
#220 CIS Cisco IOS benchmark pack size M <- this one
#229 Uncovered devices (in review)
#238 Attack-path chaining size L <- the largest Phase 1 item
That is more than anyone else is carrying, and #238 is explicitly "the largest of the four".
What I would move, and why it is not just load-balancing
#220 to me, because it pairs with #235 (Coverage & Certainty Report) into one coherent story rather than two adjacent tasks:
here is what we checked, here is what we did not, and here is which CIS controls that covers.
A coverage report that cannot say which standard it covers is half a feature, and a CIS pack nobody reports against is the other half.
There is also a clean seam: I own the policy format (analysis/policy.py), so expressing CIS as policy rules is writing in the format I designed. Your check consumes it unchanged. Neither of us reaches into the other's file.
Why you might say no, and that is fine
policy_compliance is yours, CIS is policy content, and splitting content from the check that runs it is a real cost. If you would rather keep it, say so and I will take something else off the list instead — the point is the total, not this specific item.
Say the word either way. I have not touched the assignment.
Your pairing argument is the right one. "Here is what we checked against a recognised standard, and here is exactly what we did not" is one story, and splitting it across two people makes the seam between the benchmark and the coverage report somebody's coordination problem rather than nobody's.
The argument for keeping it, and why it loses
This issue says the pack is "the largest real test the format will ever get, and it will find whatever the format cannot say." That format is mine — docs/policy-rules.md, POLICY_RULES, the two rule kinds, the query mapping. The instinct that its owner should be the one to stress it is not wrong.
But that is an argument about who fixes what it finds, not about who writes it. Those separate cleanly:
Writing CIS Cisco IOS into the format is content work. Reading a public benchmark and expressing each control as a rule needs the format documented, not authored.
Whatever it cannot express is a format limitation, and that is mine.
So: you write it, and the moment you hit a control the format cannot say — a rule about a config setting rather than a flow, which I expect is where it breaks first — that comes back to me as its own issue rather than getting worked around in the pack. I would rather find out the format is too narrow than have someone quietly bend a benchmark to fit it.
On load, plainly
I am at 3 sized points against your 2, and #238 is an L I have not started. When #181 lands, three of my issues unblock at once — #196, #195's example, and the honest half of #87 — because all three wait on active_policy() existing on main. That is a real surge rather than a hypothetical one, and it arrives the day you merge.
So the load table understates my week rather than overstating it, and handing #220 over is the difference between me doing three unblocked things properly and four things at once.
One thing I would ask you to keep from the issue as filed
The measurement in it — 91 apparent hits for CIS/STIG/PCI/NIST, of which 80 are the letters "cis" inside Cisco and 15 are "nist" inside administrator, and none with word boundaries. That number is the reason the issue exists, and it is the kind of thing that gets dropped when an issue changes hands. Reassigning, not closing and reopening, so it stays attached.
Assignment is yours to change — I have not touched it, same as you did not.
Reassigning to @ARSH871-bot, as agreed on this issue — flagging rather than silently moving it, since the board should show why.
You proposed taking it because the CIS pack pairs with #235 (Coverage Report) into one story. I agreed, with one thing carved back: whatever the pack cannot express is a format limitation and comes back to me as its own issue, rather than getting worked around inside the pack.
That agreement is here. The assignment never moved, so the board has been showing it as mine for four days.
Keep the measurement in the issue body when you pick it up — 91 apparent hits for CIS/STIG/PCI/NIST, of which 80 are the letters "cis" inside Cisco and 15 are "nist" inside administrator, and none with word boundaries. That number is the reason the issue exists and it is the kind of thing that gets dropped when an issue changes hands.
One thing that has changed since we agreed, and it makes your half easier: #268 ships docs/examples/policy.example.json, a validated example with both rule kinds and a multi-arm rule, tested through load_policy_file() on every run. That is the starting shape for expressing CIS controls, and the test file (tests/test_policy_example_is_valid.py) is the pattern for keeping a benchmark pack from rotting the same way.
Taking this back, as #398 records. Sized it by measuring before planning on 19 October, and the measurement changes the size.
Of twelve standard IOS hardening lines, Batfish models three. Logging, NTP and SNMP servers appear in nodeProperties. Password encryption, the HTTP server, SSH version, the login banner and the VTY settings parse cleanly — PASSED, zero warnings — and are modelled nowhere. Detail and the full method are in #400, raised as its own issue as agreed here in August.
That splits this into three pieces of very different size:
piece
size
CIS controls that are flow rules — expressible in the format today
S
CIS controls that are Batfish-visible settings — needs a new rule kind
M, and it is mine (format)
CIS controls Batfish cannot see — needs a decision first
So L is right for the whole, but nothing past the first row should start until #400 is decided. I would propose at planning that this sprint takes the first row plus the honest "cannot assess" card for the third — so the pack never claims coverage it does not have — and the text-inspection question is decided by all four of us.
The measurement in this issue's body (80 of 91 "CIS" hits being inside "Cisco") still stands and is still the reason the issue exists.
Activity
ARSH871-bot commentedon Aug 26, 2026
@shubhamkataria2005 — a proposal, not a change. Your issue, your call.
After adding the Phase 1 items, the board looks like this for you:
That is more than anyone else is carrying, and #238 is explicitly "the largest of the four".
What I would move, and why it is not just load-balancing
#220 to me, because it pairs with #235 (Coverage & Certainty Report) into one coherent story rather than two adjacent tasks:
A coverage report that cannot say which standard it covers is half a feature, and a CIS pack nobody reports against is the other half.
There is also a clean seam: I own the policy format (
analysis/policy.py), so expressing CIS as policy rules is writing in the format I designed. Your check consumes it unchanged. Neither of us reaches into the other's file.Why you might say no, and that is fine
policy_complianceis yours, CIS is policy content, and splitting content from the check that runs it is a real cost. If you would rather keep it, say so and I will take something else off the list instead — the point is the total, not this specific item.Say the word either way. I have not touched the assignment.
shubhamkataria2005 commentedon Aug 27, 2026
Take it — with one thing carved back to me.
Your pairing argument is the right one. "Here is what we checked against a recognised standard, and here is exactly what we did not" is one story, and splitting it across two people makes the seam between the benchmark and the coverage report somebody's coordination problem rather than nobody's.
The argument for keeping it, and why it loses
This issue says the pack is "the largest real test the format will ever get, and it will find whatever the format cannot say." That format is mine —
docs/policy-rules.md,POLICY_RULES, the two rule kinds, the query mapping. The instinct that its owner should be the one to stress it is not wrong.But that is an argument about who fixes what it finds, not about who writes it. Those separate cleanly:
So: you write it, and the moment you hit a control the format cannot say — a rule about a config setting rather than a flow, which I expect is where it breaks first — that comes back to me as its own issue rather than getting worked around in the pack. I would rather find out the format is too narrow than have someone quietly bend a benchmark to fit it.
On load, plainly
I am at 3 sized points against your 2, and #238 is an L I have not started. When #181 lands, three of my issues unblock at once — #196, #195's example, and the honest half of #87 — because all three wait on
active_policy()existing onmain. That is a real surge rather than a hypothetical one, and it arrives the day you merge.So the load table understates my week rather than overstating it, and handing #220 over is the difference between me doing three unblocked things properly and four things at once.
One thing I would ask you to keep from the issue as filed
The measurement in it — 91 apparent hits for CIS/STIG/PCI/NIST, of which 80 are the letters "cis" inside Cisco and 15 are "nist" inside administrator, and none with word boundaries. That number is the reason the issue exists, and it is the kind of thing that gets dropped when an issue changes hands. Reassigning, not closing and reopening, so it stays attached.
Assignment is yours to change — I have not touched it, same as you did not.
shubhamkataria2005 commentedon Aug 30, 2026
Reassigning to @ARSH871-bot, as agreed on this issue — flagging rather than silently moving it, since the board should show why.
You proposed taking it because the CIS pack pairs with #235 (Coverage Report) into one story. I agreed, with one thing carved back: whatever the pack cannot express is a format limitation and comes back to me as its own issue, rather than getting worked around inside the pack.
That agreement is here. The assignment never moved, so the board has been showing it as mine for four days.
Keep the measurement in the issue body when you pick it up — 91 apparent hits for CIS/STIG/PCI/NIST, of which 80 are the letters "cis" inside Cisco and 15 are "nist" inside administrator, and none with word boundaries. That number is the reason the issue exists and it is the kind of thing that gets dropped when an issue changes hands.
One thing that has changed since we agreed, and it makes your half easier: #268 ships
docs/examples/policy.example.json, a validated example with both rule kinds and a multi-arm rule, tested throughload_policy_file()on every run. That is the starting shape for expressing CIS controls, and the test file (tests/test_policy_example_is_valid.py) is the pattern for keeping a benchmark pack from rotting the same way.shubhamkataria2005 commentedon Oct 13, 2026
Taking this back, as #398 records. Sized it by measuring before planning on 19 October, and the measurement changes the size.
Of twelve standard IOS hardening lines, Batfish models three. Logging, NTP and SNMP servers appear in
nodeProperties. Password encryption, the HTTP server, SSH version, the login banner and the VTY settings parse cleanly —PASSED, zero warnings — and are modelled nowhere. Detail and the full method are in #400, raised as its own issue as agreed here in August.That splits this into three pieces of very different size:
needs: all fourSo L is right for the whole, but nothing past the first row should start until #400 is decided. I would propose at planning that this sprint takes the first row plus the honest "cannot assess" card for the third — so the pack never claims coverage it does not have — and the text-inspection question is decided by all four of us.
The measurement in this issue's body (80 of 91 "CIS" hits being inside "Cisco") still stands and is still the reason the issue exists.