Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 5 additions & 3 deletions CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -117,8 +117,10 @@ Batfish runs in Docker container `batfish` (image `batfish/allinone`), exposing
- The client's real firewall is **PF Sense**, which exports XML that Batfish
cannot read at all. `analysis/pfsense_convert.py` translates it into Cisco
IOS text, so a converted config re-enters the same `analyse()` as everything
else. Interfaces and filter rules are covered; NAT, aliases, DHCP, VPN, IPv6
and combined `tcp/udp` rules are not, and each raises rather than guessing.
else. Interfaces and filter rules are covered; NAT, aliases, DHCP, VPN, IPv6,
combined `tcp/udp`, negated (`<not/>`) addresses and ICMP-type rules are
not, and each raises rather than guessing. A disabled rule is left out,
exactly as pfSense itself leaves it out.

**Rule order: PF Sense and Cisco disagree, and the converter now MODELS the
difference where it can and refuses where it cannot.** PF Sense evaluates
Expand Down Expand Up @@ -722,7 +724,7 @@ project's recurring failure family arriving through process rather than code.
| **AI explanation on screen** | Ankeet + Samika | Done (#56, closing #31). `/api/findings` attaches an `explanation` to every `status="found"` finding, **not** an F-1 field, added downstream of validation so the contract is untouched. Since #109/#143, also attaches `explanation_source` (`"model"` or `"fallback"`), same reasoning, so the dashboard can eventually stop labelling deterministic fallback text as AI-authored, which it still does today (#109 stays open until the byline itself changes) |
| **`risk` scoring** | Samika | Done (#60) — `POST_PROCESSORS`, ruleset in `docs/severity-rules.md`. Re-rates severity and sorts worst-first; the two limits are enforced in `pipeline.run_post_processors()`, not trusted |
| Dashboard + secure upload | Samika | Done — real findings on screen since #39 |
| PF Sense conversion | Ankeet | Done — `analysis/pfsense_convert.py`, and **hardened**: refuses config injection and path traversal via free-text fields (#53), an unbound ACL / empty rule set / unvalidated addressing (#54), and ambiguous rule order (#58, closing #47). See §7 |
| PF Sense conversion | Ankeet | Done — `analysis/pfsense_convert.py`, and **hardened**: refuses config injection and path traversal via free-text fields (#53), an unbound ACL / empty rule set / unvalidated addressing (#54), and ambiguous rule order (#58, closing #47). Also refuses an interface tag that is not a plain name (a namespaced tag injected config lines), and, per rule, `<not/>`, IPv6-only and ICMP-type rules; disabled rules are left out, as pfSense does. See §7 |
| **`change_impact`** | Shubham | Done (#140, merged 17 August) — `analysis/change_impact.py`, `analyse_change(before, after)`. Shape C: **not** in `CHECKS`, and both the module docstring and the registry comment say so. Pairs `compareFilters` (which ACL *lines* moved) with `differentialReachability` (which *traffic* changed fate), because a changed line that moves no traffic is noise and moved traffic with no changed line is the case a filter diff alone misses. Direction-aware per `docs/policy-rules.md`: opening `high`, tightening `medium`, both reported |
| **A user policy reaching a check** | Arsh | Done (#181, merged 28 August) — `analysis/policy.py` loads and validates it; `policy_compliance` reads it via `rules_in_use()`, keyed on `active_policy() is None` rather than on emptiness. Measured below: policy-driven detections on a stranger's network go 3 → 8 |
| **Draft a policy from your own config** | Arsh | Done (#326) — `analysis/draft_policy.py`, `/api/policy/draft`, a button under the policy picker. One `policy_compliance` rule per line of each applied filter, citing the file and line Batfish records for it. Each rule is checked with the same `searchFilters` question the check will ask, so a line an earlier line partly decides is listed as not drafted, with Batfish's example packet, rather than copied. `kind` and `violation_severity` are left as `<FILL IN ...>`, and `analysis/policy.py` refuses the file until they are decided and its `draft` block is deleted: accepted as it stands, a draft passes on `rtr-us5-insecure` |
Expand Down
6 changes: 4 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -124,8 +124,10 @@ permitted. Those remain ours.
its own ACL, checked independently — and it **models** PF Sense's
last-match-wins order rather than refusing it wherever it can do so exactly
(#104). It **refuses rather than guesses** on NAT, aliases, VPN, IPv6,
interfaces with no static address, interfaces named by a rule but never
declared, and rule orders it cannot model exactly. A real client export is
negated (`<not/>`) addresses, ICMP-type rules, interface tags that are not
plain names, interfaces with no static address, interfaces named by a rule
but never declared, and rule orders it cannot model exactly. A disabled
rule is left out, as pfSense itself does. A real client export is
still refused, on the last two of those.
`analysis/pfsense_convert.REFUSALS` is the single source of truth for
exactly what is refused and why (#155).
Expand Down
40 changes: 37 additions & 3 deletions analysis/business_context.py
Original file line number Diff line number Diff line change
Expand Up @@ -334,17 +334,51 @@ def load_business_context_file(path: Any) -> BusinessContext:
if not file_path.exists():
raise BusinessContextError(f"no business context file at {file_path}")

text = file_path.read_text(encoding="utf-8").strip()
try:
text = file_path.read_text(encoding="utf-8").strip()
except UnicodeDecodeError as error:
raise BusinessContextError(
f"{file_path} is not UTF-8 text: byte {error.start} cannot be "
f"decoded. Save the file as UTF-8 and try again"
) from error
if not text:
# An empty FILE is an empty CONTEXT, which is valid.
return BusinessContext()

# Same three escapes, and the same fix, as policy.load_policy_file().
try:
parsed = json.loads(text)
parsed = json.loads(text, object_pairs_hook=_refuse_duplicate_keys)
return load_business_context(parsed)
except json.JSONDecodeError as error:
raise BusinessContextError(
f"{file_path} is not valid JSON: line {error.lineno}, "
f"column {error.colno}: {error.msg}"
) from error
except BusinessContextError:
raise
except RecursionError as error:
raise BusinessContextError(
f"{file_path} is nested too deeply to be a business context"
) from error
except ValueError as error:
raise BusinessContextError(
f"{file_path} holds a value that cannot be read: {error}"
) from error


def _refuse_duplicate_keys(pairs: List[Tuple[str, Any]]) -> Dict[str, Any]:
"""Build one JSON object, refusing a key that appears twice in it.

return load_business_context(parsed)
Mirrors policy._refuse_duplicate_keys(). json.loads keeps the LAST of two
equal keys: `{"device": "rtr-us5", "tier": "critical", "tier":
"standard"}` loaded as standard, the user's "critical" dropped silently.
"""
built: Dict[str, Any] = {}
for key, value in pairs:
if key in built:
raise BusinessContextError(
f"the key {key!r} appears twice in one entry. JSON keeps only "
f"the last, so the first would be silently ignored"
)
built[key] = value
return built
63 changes: 59 additions & 4 deletions analysis/pfsense_convert.py
Original file line number Diff line number Diff line change
Expand Up @@ -178,6 +178,10 @@
"injection or path traversal"
),
"no_interfaces_section": "the export has no <interfaces> section at all",
"invalid_interface_role": (
"an interface's tag under <interfaces> is not a plain name like "
"'lan' or 'opt1', and it is written into the generated config"
),
"invalid_interface_address": (
"an interface's <ipaddr>/<subnet> do not form a valid IPv4 network"
),
Expand Down Expand Up @@ -205,6 +209,18 @@
"like 'tcp/udp' need two Cisco ACL lines and are not handled"
),
"missing_source_or_destination": "a rule has no <source> or <destination>",
"negated_endpoint": (
"a <source> or <destination> carries <not/>, meaning everything "
"EXCEPT that address -- one Cisco ACL line cannot express 'except'"
),
"ipv6_rule": (
"an IPv6-only rule (<ipprotocol>inet6) -- this module models IPv4 "
"only, so the rule's IPv6 traffic is not analysed"
),
"icmp_type": (
"an ICMP rule limited to particular ICMP types -- this module does "
"not translate pfSense's ICMP type names"
),
"invalid_port": (
"a rule's <port> is not a single numeric port -- named aliases and "
"ranges are out of scope"
Expand Down Expand Up @@ -357,6 +373,10 @@ def _sanitised_hostname(text: str) -> str:
return text


# pfSense's own interface keys: wan, lan, opt1, ... -- see _parse_interfaces().
_ROLE_PATTERN = re.compile(r"[A-Za-z0-9_-]+")


def _parse_interfaces(root: ET.Element) -> Dict[str, Dict[str, str]]:
"""Return {role: {"if": ..., "ipaddr": ..., "subnet": ..., "descr": ...}}
for every interface PF Sense defines, in document order.
Expand All @@ -373,6 +393,16 @@ def _parse_interfaces(root: ET.Element) -> Dict[str, Dict[str, str]]:
interfaces: Dict[str, Dict[str, str]] = {}
for iface_el in interfaces_el:
role = iface_el.tag
# The role becomes part of an ACL name in the emitted config (see
# _acl_name()), so it gets the same allowlist treatment as
# <hostname>. A tag name looks safe, but ElementTree reports a
# namespaced tag as "{namespace}name", and the namespace is free
# text: `xmlns:x="&#10;ip access-list extended acl_in&#10; permit ip
# any any"` on <x:opt1> put that permit line at the top of acl_in.
if not _ROLE_PATTERN.fullmatch(role):
raise PfSenseConversionError(
f"{REFUSALS['invalid_interface_role']}: {role!r}"
)
ipaddr = _text(iface_el, "ipaddr")
subnet = _text(iface_el, "subnet")
if not ipaddr or not subnet:
Expand Down Expand Up @@ -516,10 +546,9 @@ def _acl_name(role: str, *, rule_bearing_roles: "set[str]") -> str:
the same convention _assign_cisco_interface_names() already uses for
interface names, rather than a second naming scheme.

`role` is an XML tag name (see _parse_interfaces()), not free text pulled
from an element's content the way <descr> or <hostname> are, so it is
already constrained by XML's own tag-name grammar and does not need the
separate validation those fields do.
`role` is an XML tag name, validated against _ROLE_PATTERN in
_parse_interfaces(). This docstring used to say a tag name was already
safe by XML's own grammar; a namespaced tag is not -- see there.
"""
if role in rule_bearing_roles and len(rule_bearing_roles) == 1:
return ACL_NAME
Expand Down Expand Up @@ -561,6 +590,14 @@ def _resolve_endpoint(el: ET.Element, interfaces: Dict[str, Dict[str, str]]) ->
if el.find("any") is not None:
return "any"

# <not/> inverts the match: "everything EXCEPT this" (filter.inc:4249).
# It used to be ignored, so the emitted line matched exactly the traffic
# the rule exempts and none of the traffic it covers. One Cisco line
# cannot say "except", so the rule is refused. After the <any/> check on
# purpose: pfSense ignores <not/> beside <any/> too (filter.inc:4103).
if el.find("not") is not None:
raise PfSenseConversionError(REFUSALS["negated_endpoint"])

network = _text(el, "network")
if network is not None:
if network not in interfaces:
Expand Down Expand Up @@ -755,6 +792,18 @@ def _rule_to_acl_line(rule_el: ET.Element, interfaces: Dict[str, Dict[str, str]]
f"{pf_protocol!r}"
)

# Two fields that make pfSense match LESS than the fields above say, and
# that used to be ignored (filter.inc at 480a1c0626). Ignored, a block
# rule blocked more in the model than on the firewall: a stricter ACL,
# the direction that turns a real violation into a clean result (#302).
if _text(rule_el, "ipprotocol") == "inet6": # filter.inc:4333
raise PfSenseConversionError(REFUSALS["ipv6_rule"])
icmp_type = _text(rule_el, "icmptype")
if pf_protocol == "icmp" and icmp_type not in (None, "any"): # filter.inc:4522
raise PfSenseConversionError(
f"{REFUSALS['icmp_type']}: <icmptype> is {icmp_type!r}"
)

source_el = rule_el.find("source")
destination_el = rule_el.find("destination")
if source_el is None or destination_el is None:
Expand Down Expand Up @@ -848,6 +897,12 @@ def convert(xml_path: Union[str, Path]) -> ConversionResult:
filter_el = root.find("filter")
rule_els = filter_el.findall("rule") if filter_el is not None else []

# A disabled rule is not loaded by pfSense at all (filter.inc:5543), so
# leaving it out is the exact model, not a gap -- nothing is skipped.
# It used to be converted as live: a disabled block denied traffic in
# the model that the real firewall lets through.
rule_els = [r for r in rule_els if r.find("disabled") is None]

# WHY AN EMPTY RULE SET IS REFUSED, NOT CONVERTED AS "NO ACL"
# Found in a senior-level adversarial QA pass. PF Sense fails CLOSED
# with no rules configured on an interface -- it blocks everything.
Expand Down
43 changes: 39 additions & 4 deletions analysis/policy.py
Original file line number Diff line number Diff line change
Expand Up @@ -699,20 +699,55 @@ def load_policy_file(path: Any) -> Policy:
if not file_path.exists():
raise PolicyError(f"no policy file at {file_path}")

text = file_path.read_text(encoding="utf-8").strip()
try:
text = file_path.read_text(encoding="utf-8").strip()
except UnicodeDecodeError as error:
raise PolicyError(
f"{file_path} is not UTF-8 text: byte {error.start} cannot be "
f"decoded. Save the policy as UTF-8 and try again"
) from error
if not text:
# An empty FILE is an empty POLICY, which is valid and loud (D4).
return Policy({section: [] for section in POLICY_SECTIONS})

# Three more ways a file used to escape as a raw exception -- an HTTP 500
# on upload rather than a refusal naming the file: an integer of more
# than 4,300 digits (ValueError), and nesting deeper than Python's
# recursion limit, hit by json.loads or by load_policy() walking it.
try:
parsed = json.loads(text)
parsed = json.loads(text, object_pairs_hook=_refuse_duplicate_keys)
return load_policy(parsed)
except json.JSONDecodeError as error:
raise PolicyError(
f"{file_path} is not valid JSON: line {error.lineno}, "
f"column {error.colno}: {error.msg}"
) from error

return load_policy(parsed)
except PolicyError:
raise
except RecursionError as error:
raise PolicyError(f"{file_path} is nested too deeply to be a policy") from error
except ValueError as error:
raise PolicyError(f"{file_path} holds a value that cannot be read: {error}") from error


def _refuse_duplicate_keys(pairs: List[Tuple[str, Any]]) -> Dict[str, Any]:
"""Build one JSON object, refusing a key that appears twice in it.

json.loads keeps the LAST of two equal keys and drops the first without a
word. Measured: `{"policy_compliance": [<a rule>], "policy_compliance":
[]}` loaded as an EMPTY policy, and the upload said so with HTTP 200 --
the user's rule gone. An unknown key is refused (D5) because a key nobody
reads is a rule nobody enforces; a key written twice is the same failure.
"""
built: Dict[str, Any] = {}
for key, value in pairs:
if key in built:
raise PolicyError(
f"the key {key!r} appears twice in one object. JSON keeps only "
f"the last, so the first would be silently ignored"
)
built[key] = value
return built


# ------------------------------------------------------------------------------
Expand Down
23 changes: 23 additions & 0 deletions tests/test_business_context.py
Original file line number Diff line number Diff line change
Expand Up @@ -383,6 +383,29 @@ def test_malformed_json_names_the_line_and_column(tmp_path):
assert "line" in message and "column" in message


@pytest.mark.parametrize(
"contents, said",
[
# Loaded as "standard" before the fix: the user's "critical" dropped.
pytest.param(b'[{"device": "rtr-us5", "tier": "critical", "tier": "standard"}]',
"appears twice", id="duplicate-key"),
pytest.param(b'[{"device": "r\xe9", "tier": "critical"}]', "not UTF-8", id="not-utf8"),
pytest.param(b'[{"device": "r", "tier": 1' + b"0" * 5000 + b"}]", "cannot be read",
id="5000-digit-int"),
pytest.param(b"[" * 100000 + b"]" * 100000, "nested too deeply", id="deep-json"),
],
)
def test_a_file_json_cannot_hold_faithfully_is_refused_not_crashed_or_trimmed(
tmp_path, contents, said
):
"""Mirrors the policy loader's test: a raw exception (HTTP 500 on
upload), or a silently dropped value, before the fix."""
path = tmp_path / "business-context.json"
path.write_bytes(contents)
with pytest.raises(BusinessContextError, match=said):
load_business_context_file(path)


def test_a_real_file_round_trips(tmp_path):
path = tmp_path / "business-context.json"
path.write_text(
Expand Down
24 changes: 24 additions & 0 deletions tests/test_business_context_wiring.py
Original file line number Diff line number Diff line change
Expand Up @@ -227,6 +227,30 @@ def test_an_invalid_context_is_rejected_with_the_loaders_own_message(body, expec
assert expected in response.json()["detail"]


@pytest.mark.parametrize(
"raw, expected",
[
# PowerShell 5.1's `>` and Notepad's "Unicode" both save UTF-16.
(json.dumps([{"device": "rtr-us5", "tier": "critical"}]).encode("utf-16"),
"not UTF-8"),
(json.dumps([{"device": "rtr-café", "tier": "critical"}],
ensure_ascii=False).encode("latin-1"),
"not UTF-8"),
(b"[" * 200_000 + b"]" * 200_000, "nested too deeply"),
],
ids=["utf-16", "latin-1", "deep-nesting"],
)
def test_a_file_the_loader_cannot_decode_is_a_400_not_a_500(raw, expected):
"""Measured on origin/main: all three were HTTP 500 Internal Server
Error, the same escape as the policy loader's (see
test_web_policy_upload.py). Refused, never guessed at."""
response = _post_context(raw)

assert response.status_code == 400
assert expected in response.json()["detail"]
assert not main.BUSINESS_CONTEXT_PATH.exists()


def test_a_rejected_context_is_never_staged():
"""The safety property the validate-then-stage order exists for."""
_post_context([{"device": "rtr-us5", "tier": "critical"}])
Expand Down
Loading
Loading