Skip to content

Refuse input files Netwise used to crash on or read wrongly - #446

Merged
ARSH871-bot merged 1 commit into
mainfrom
fix/stability-input-files
Nov 9, 2026
Merged

ARSH871-bot merged 1 commit into
mainfrom
fix/stability-input-files

Conversation

@ARSH871-bot

Copy link
Copy Markdown
Owner

Input files that Netwise either crashed on (HTTP 500) or, worse, read wrongly while carrying on as if nothing was wrong. All four were found in a stability audit (21–30 Oct) and re-measured on current main (db59b00). This is the second of the audit PRs; #442 was the first.

What changes

Policy and business-context files (analysis/policy.py, analysis/business_context.py):

Input Before, on main After
A key written twice, e.g. {"policy_compliance": [<rule>], "policy_compliance": []} json.loads keeps the last one, so the rule is silently dropped, and the upload returns HTTP 200 Refused, naming the key
"tier": "critical" then "tier": "standard" in one context entry Silently becomes standard Refused, naming the key
Not UTF-8 (latin-1, or the UTF-16 PowerShell 5.1's > writes) HTTP 500 400: "is not UTF-8 text (byte N…)". No second encoding is guessed
Nesting deeper than Python's recursion limit HTTP 500 400: "nested too deeply"
An integer of more than 4,300 digits HTTP 500 400: "holds a value that cannot be read"

The PF Sense converter (analysis/pfsense_convert.py). Each case below was converted on main and on this branch, then the result was put to real Batfish (testFilters on the LAN ACL):

Case main This branch pfSense
A namespaced interface tag (<x:opt1> with a crafted xmlns) Injects permit ip any any at the top of acl_in. A firewall whose only rule is a block permits everything Refused at the bad tag, with the same allowlist hostnames already get —
A <disabled/> block rule Converted as live, so everything is denied Left out pfSense does not load it (filter.inc:5543), so traffic passes
A pass rule with <not/> on its destination Converted without the not: it permits exactly the traffic the rule excludes Rule skipped and named in skipped Inverts the match (filter.inc:4249)
An IPv6-only (inet6) block rule Applied to IPv4, so everything is denied Rule skipped and named IPv6 only (filter.inc:4333)
An ICMP rule limited to certain ICMP types Widened to all ICMP Rule skipped and named Only those types (filter.inc:4522)

One trade is stated honestly. A skipped pass rule (the <not/> case) makes the emitted ACL stricter than the firewall, as #302 and #306 already describe. It is never silent: the rule is named in skipped, which reaches the upload response and the downloaded report.

Evidence (measured on db59b00)

Each fix's tests, run on main without the fix:

policy / context loaders (P6)       12 failed, 130 passed
namespaced interface tag (P2)        1 failed,  84 passed
ignored pfSense fields (P3)          4 failed,  84 passed
endpoint 400s (folded in, see below)  6 failed,  57 passed
with every fix: the same five test files, 277 passed

Full suite on this branch: 2222 passed, 54 skipped, 0 failed with Batfish down (as on CI). Live against real Batfish: 2263 passed, 13 skipped, 0 failed. Lint: CI's exact command (ruff check --select E4,E7,E9,F analysis/ ai/ web/ tools/ tests/, ruff 0.14.14) reports all checks passed.

Notes for review

  • Two audit fixes became one. The audit produced two fixes for the loaders. This PR keeps the more complete one (it also covers duplicate keys and huge integers), and folds in the other's endpoint tests. Those reach /api/business-context, which the loader tests alone did not, and they add the UTF-16 case.
  • Docs move with the code. The list of what the converter refuses is updated in all three places that state it: CLAUDE.md §7, CLAUDE.md's status table, and the README. REFUSALS in the converter is still the source of truth.
  • Not here: rule order. P1 ("every interface rule is quick") waits on pfSense converter reverses every interface rule list: a missing <quick/> does not mean last-match-wins #441, and the README's "models last-match-wins" sentence is left for that PR.
  • No F-1 change, and nothing is guessed. Every new case is a refusal, an exact model (disabled rules), or a named skip.

@patelankeet2: the converter is yours. @shubhamkataria2005: policy.py. @SamikaPerera: business context and the upload endpoints.

Found in a stability audit (21-30 Oct), re-measured on main db59b00.

Policy and business-context files:
- a key written twice is refused; json.loads used to keep the last one,
  so a policy rule (or a "critical" tier) was silently dropped and the
  upload still returned 200
- non-UTF-8 text, nesting past the recursion limit, and integers too
  long to parse are 400s naming the file, not 500s; no second encoding
  is guessed

PF Sense converter:
- an interface tag that is not a plain name is refused; a namespaced
  tag carried free text into the ACL name and injected "permit ip any
  any" ahead of the real rules
- a <disabled/> rule is left out, as pfSense itself does
  (filter.inc:5543); it used to be converted as live
- rules with <not/> (filter.inc:4249), IPv6-only rules (4333) and ICMP
  rules limited to certain types (4522) are skipped and named, not
  converted as something they are not

The list of what the converter refuses is updated where it is stated:
CLAUDE.md section 7, CLAUDE.md's status table, and the README.
@ARSH871-bot ARSH871-bot added bug Something isn't working layer: analysis Layer 1 — Batfish orchestration, checks, the pipeline layer: web Layer 3 — FastAPI backend and dashboard safety Touches F-4, grounding, or the refuse-rather-than-guess boundary. Review with care labels Nov 3, 2026
@ARSH871-bot ARSH871-bot added bug Something isn't working layer: analysis Layer 1 — Batfish orchestration, checks, the pipeline labels Nov 3, 2026
@ARSH871-bot ARSH871-bot added the layer: web Layer 3 — FastAPI backend and dashboard label Nov 3, 2026
@ARSH871-bot ARSH871-bot added the safety Touches F-4, grounding, or the refuse-rather-than-guess boundary. Review with care label Nov 3, 2026

@patelankeet2 patelankeet2 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Re-tested properly in an isolated worktree at the branch's real head (0aff2c2), since this touches analysis/pfsense_convert.py and that's my module.

Spent most of my time on the namespaced-interface-tag fix, since that's the most serious of the five. Reproduced the claimed vulnerability directly against main's actual pre-fix code (not just read the diff): a <x:opt1> interface tag with a crafted xmlns:x="&#10;ip access-list extended acl_in&#10; permit ip any any" does get reported by ElementTree as {<namespace>}opt1, confirmed live.

Worth noting for the record, since it changes how serious this is: on main, that exact payload gets refused today, but only by accident. _parse_interfaces() defaults a missing <descr> to role.upper(), and that default happens to pass through the existing control-character check -- so it's caught only because my first test had no <descr> on the malicious interface. Added one (<descr>Second LAN</descr>, completely innocuous) and reran against main's real code: it converts cleanly and the output literally contains a second ip access-list extended acl_in block with permit ip any any injected into the middle of the real one. So this was a live, trivially bypassable vulnerability, not a theoretical one, and the existing control-character check was never actually protecting the role field on purpose. This PR's fix (_ROLE_PATTERN.fullmatch(role), checked unconditionally, independent of whether <descr> is given) closes it properly. Reran the same explicit-descr payload against this branch: refused, correctly, for the right reason.

Mutation-tested all four converter guards together (role pattern, <not/> refusal, ipv6/icmp-type refusal, disabled-rule filtering): disabling all four at once breaks exactly five of their direct tests plus one meta-test (test_every_raise_site_actually_uses_a_refusals_entry, which caught the role mutation too, as a bonus), 6 failed / 83 passed. Restored, back to 89 passed.

Checked the policy.py/business_context.py duplicate-key and encoding fixes too. One thing worth confirming rather than assuming: PolicyError and BusinessContextError both subclass ValueError, so the new except PolicyError: raise / except BusinessContextError: raise guards placed before the generic except ValueError handler are load-bearing, not decorative -- without that ordering, every normal validation failure (D5's unknown-key message, for example) would get caught by the new generic handler and re-wrapped into a confusing "holds a value that cannot be read" message. Tested live: an unknown-section policy still comes back with its original D5 message, unwrapped, confirming the ordering actually works.

Confirmed the doc-sync claim too, since "update it in all three places, only update two" is this project's own recurring failure pattern: CLAUDE.md's §7 bullet list, CLAUDE.md's status table row, and README.md's bullet list are all three genuinely updated with the new refusal categories, consistently worded.

ruff clean on all five changed Python files.

Approving.

@shubhamkataria2005 shubhamkataria2005 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approving 0aff2c2 for analysis/policy.py, the part you asked me to look at. @patelankeet2 has covered the converter.

Merged onto current main (d424e05): 2282 passed, 13 skipped, ruff clean on CI's selection.

The duplicate-key fix matters more to the policy tools than the description says. Every one of my command-line tools reads policies through load_policy_file() (contradictions, the two-policy check, drift, the coverage map and change impact), so they all get it for free. I measured what it was costing them. With an old policy written {"policy_compliance": [<rule>], "policy_compliance": []}:

main         python -m analysis.policy_drift old.json new.json
             Not weaker: every rule in the old policy is still in force, at least as wide.
this branch  old.json: not loaded -- the key 'policy_compliance' appears twice in one object.
             JSON keeps only the last, so the first would be silently ignored

On main the drift check compared against an empty policy and gave false reassurance. That's exactly the "read wrongly while carrying on" failure this PR is about, reaching a feature it doesn't mention. policy_conflicts is refused the same way here.

Also checked:

  • The upload goes through load_policy_file() (web/main.py:791), so the endpoint gets the same refusals. The other json.loads in web/main.py (:833) reads Netwise's own skip record, not user input, so leaving it alone is right.
  • except PolicyError: raise comes before except ValueError, which matters because PolicyError is a ValueError. A rule-level refusal keeps its own message and isn't re-labelled as "a value that cannot be read".

One small note, not a blocker. Because load_policy() now runs inside the same try, any other ValueError a future validation bug raises would reach the user as "holds a value that cannot be read". It's still a refusal (the safe direction), just a misleading one. Narrowing the ValueError catch to json.loads alone would keep bugs looking like bugs.

@ARSH871-bot
ARSH871-bot merged commit 94871cb into main Nov 9, 2026
4 checks passed
@ARSH871-bot
ARSH871-bot deleted the fix/stability-input-files branch November 9, 2026 02:37
@ARSH871-bot

Copy link
Copy Markdown
Owner Author

You're right, and I'll fix it in a small follow-up of my own, since it's my code from #446.

The catch is needed, but only around json.loads. That's where the real case comes from: a number over 4,300 digits makes json.loads raise a plain ValueError ("Exceeds the limit (4300 digits)"), not a JSONDecodeError. That's why the catch is there. Checked on main: such a file is refused with "holds a value that cannot be read", which is correct.

Wrapping load_policy() in the same try is the mistake you describe. A bug in a validator would reach the user as "a value that cannot be read". So the fix:

  • json.loads keeps its own try, with JSONDecodeError and the ValueError for the digit limit.
  • load_policy() moves outside it, keeping only the RecursionError guard it needs. Any other exception then shows up as a bug, not as a refusal.
  • The same change in analysis/business_context.py, which has the same shape.
  • A test that monkeypatches a validator to raise a plain ValueError and asserts it is not turned into a PolicyError. It should fail on today's main.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working layer: analysis Layer 1 — Batfish orchestration, checks, the pipeline layer: web Layer 3 — FastAPI backend and dashboard safety Touches F-4, grounding, or the refuse-rather-than-guess boundary. Review with care

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants