diff --git a/CLAUDE.md b/CLAUDE.md index 6e89dad..89eb1b1 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -117,8 +117,10 @@ Batfish runs in Docker container `batfish` (image `batfish/allinone`), exposing - The client's real firewall is **PF Sense**, which exports XML that Batfish cannot read at all. `analysis/pfsense_convert.py` translates it into Cisco IOS text, so a converted config re-enters the same `analyse()` as everything - else. Interfaces and filter rules are covered; NAT, aliases, DHCP, VPN, IPv6 - and combined `tcp/udp` rules are not, and each raises rather than guessing. + else. Interfaces and filter rules are covered; NAT, aliases, DHCP, VPN, IPv6, + combined `tcp/udp`, negated (``) addresses and ICMP-type rules are + not, and each raises rather than guessing. A disabled rule is left out, + exactly as pfSense itself leaves it out. **Rule order: PF Sense and Cisco disagree, and the converter now MODELS the difference where it can and refuses where it cannot.** PF Sense evaluates @@ -722,7 +724,7 @@ project's recurring failure family arriving through process rather than code. | **AI explanation on screen** | Ankeet + Samika | Done (#56, closing #31). `/api/findings` attaches an `explanation` to every `status="found"` finding, **not** an F-1 field, added downstream of validation so the contract is untouched. Since #109/#143, also attaches `explanation_source` (`"model"` or `"fallback"`), same reasoning, so the dashboard can eventually stop labelling deterministic fallback text as AI-authored, which it still does today (#109 stays open until the byline itself changes) | | **`risk` scoring** | Samika | Done (#60) — `POST_PROCESSORS`, ruleset in `docs/severity-rules.md`. Re-rates severity and sorts worst-first; the two limits are enforced in `pipeline.run_post_processors()`, not trusted | | Dashboard + secure upload | Samika | Done — real findings on screen since #39 | -| PF Sense conversion | Ankeet | Done — `analysis/pfsense_convert.py`, and **hardened**: refuses config injection and path traversal via free-text fields (#53), an unbound ACL / empty rule set / unvalidated addressing (#54), and ambiguous rule order (#58, closing #47). See §7 | +| PF Sense conversion | Ankeet | Done — `analysis/pfsense_convert.py`, and **hardened**: refuses config injection and path traversal via free-text fields (#53), an unbound ACL / empty rule set / unvalidated addressing (#54), and ambiguous rule order (#58, closing #47). Also refuses an interface tag that is not a plain name (a namespaced tag injected config lines), and, per rule, ``, IPv6-only and ICMP-type rules; disabled rules are left out, as pfSense does. See §7 | | **`change_impact`** | Shubham | Done (#140, merged 17 August) — `analysis/change_impact.py`, `analyse_change(before, after)`. Shape C: **not** in `CHECKS`, and both the module docstring and the registry comment say so. Pairs `compareFilters` (which ACL *lines* moved) with `differentialReachability` (which *traffic* changed fate), because a changed line that moves no traffic is noise and moved traffic with no changed line is the case a filter diff alone misses. Direction-aware per `docs/policy-rules.md`: opening `high`, tightening `medium`, both reported | | **A user policy reaching a check** | Arsh | Done (#181, merged 28 August) — `analysis/policy.py` loads and validates it; `policy_compliance` reads it via `rules_in_use()`, keyed on `active_policy() is None` rather than on emptiness. Measured below: policy-driven detections on a stranger's network go 3 → 8 | | **Draft a policy from your own config** | Arsh | Done (#326) — `analysis/draft_policy.py`, `/api/policy/draft`, a button under the policy picker. One `policy_compliance` rule per line of each applied filter, citing the file and line Batfish records for it. Each rule is checked with the same `searchFilters` question the check will ask, so a line an earlier line partly decides is listed as not drafted, with Batfish's example packet, rather than copied. `kind` and `violation_severity` are left as ``, and `analysis/policy.py` refuses the file until they are decided and its `draft` block is deleted: accepted as it stands, a draft passes on `rtr-us5-insecure` | diff --git a/README.md b/README.md index 1878e9b..20d2a76 100644 --- a/README.md +++ b/README.md @@ -124,8 +124,10 @@ permitted. Those remain ours. its own ACL, checked independently — and it **models** PF Sense's last-match-wins order rather than refusing it wherever it can do so exactly (#104). It **refuses rather than guesses** on NAT, aliases, VPN, IPv6, - interfaces with no static address, interfaces named by a rule but never - declared, and rule orders it cannot model exactly. A real client export is + negated (``) addresses, ICMP-type rules, interface tags that are not + plain names, interfaces with no static address, interfaces named by a rule + but never declared, and rule orders it cannot model exactly. A disabled + rule is left out, as pfSense itself does. A real client export is still refused, on the last two of those. `analysis/pfsense_convert.REFUSALS` is the single source of truth for exactly what is refused and why (#155). diff --git a/analysis/business_context.py b/analysis/business_context.py index eca110e..30fb48a 100644 --- a/analysis/business_context.py +++ b/analysis/business_context.py @@ -334,17 +334,51 @@ def load_business_context_file(path: Any) -> BusinessContext: if not file_path.exists(): raise BusinessContextError(f"no business context file at {file_path}") - text = file_path.read_text(encoding="utf-8").strip() + try: + text = file_path.read_text(encoding="utf-8").strip() + except UnicodeDecodeError as error: + raise BusinessContextError( + f"{file_path} is not UTF-8 text: byte {error.start} cannot be " + f"decoded. Save the file as UTF-8 and try again" + ) from error if not text: # An empty FILE is an empty CONTEXT, which is valid. return BusinessContext() + # Same three escapes, and the same fix, as policy.load_policy_file(). try: - parsed = json.loads(text) + parsed = json.loads(text, object_pairs_hook=_refuse_duplicate_keys) + return load_business_context(parsed) except json.JSONDecodeError as error: raise BusinessContextError( f"{file_path} is not valid JSON: line {error.lineno}, " f"column {error.colno}: {error.msg}" ) from error + except BusinessContextError: + raise + except RecursionError as error: + raise BusinessContextError( + f"{file_path} is nested too deeply to be a business context" + ) from error + except ValueError as error: + raise BusinessContextError( + f"{file_path} holds a value that cannot be read: {error}" + ) from error + + +def _refuse_duplicate_keys(pairs: List[Tuple[str, Any]]) -> Dict[str, Any]: + """Build one JSON object, refusing a key that appears twice in it. - return load_business_context(parsed) + Mirrors policy._refuse_duplicate_keys(). json.loads keeps the LAST of two + equal keys: `{"device": "rtr-us5", "tier": "critical", "tier": + "standard"}` loaded as standard, the user's "critical" dropped silently. + """ + built: Dict[str, Any] = {} + for key, value in pairs: + if key in built: + raise BusinessContextError( + f"the key {key!r} appears twice in one entry. JSON keeps only " + f"the last, so the first would be silently ignored" + ) + built[key] = value + return built diff --git a/analysis/pfsense_convert.py b/analysis/pfsense_convert.py index b80dd2a..d4bc636 100644 --- a/analysis/pfsense_convert.py +++ b/analysis/pfsense_convert.py @@ -178,6 +178,10 @@ "injection or path traversal" ), "no_interfaces_section": "the export has no section at all", + "invalid_interface_role": ( + "an interface's tag under is not a plain name like " + "'lan' or 'opt1', and it is written into the generated config" + ), "invalid_interface_address": ( "an interface's / do not form a valid IPv4 network" ), @@ -205,6 +209,18 @@ "like 'tcp/udp' need two Cisco ACL lines and are not handled" ), "missing_source_or_destination": "a rule has no or ", + "negated_endpoint": ( + "a or carries , meaning everything " + "EXCEPT that address -- one Cisco ACL line cannot express 'except'" + ), + "ipv6_rule": ( + "an IPv6-only rule (inet6) -- this module models IPv4 " + "only, so the rule's IPv6 traffic is not analysed" + ), + "icmp_type": ( + "an ICMP rule limited to particular ICMP types -- this module does " + "not translate pfSense's ICMP type names" + ), "invalid_port": ( "a rule's is not a single numeric port -- named aliases and " "ranges are out of scope" @@ -357,6 +373,10 @@ def _sanitised_hostname(text: str) -> str: return text +# pfSense's own interface keys: wan, lan, opt1, ... -- see _parse_interfaces(). +_ROLE_PATTERN = re.compile(r"[A-Za-z0-9_-]+") + + def _parse_interfaces(root: ET.Element) -> Dict[str, Dict[str, str]]: """Return {role: {"if": ..., "ipaddr": ..., "subnet": ..., "descr": ...}} for every interface PF Sense defines, in document order. @@ -373,6 +393,16 @@ def _parse_interfaces(root: ET.Element) -> Dict[str, Dict[str, str]]: interfaces: Dict[str, Dict[str, str]] = {} for iface_el in interfaces_el: role = iface_el.tag + # The role becomes part of an ACL name in the emitted config (see + # _acl_name()), so it gets the same allowlist treatment as + # . A tag name looks safe, but ElementTree reports a + # namespaced tag as "{namespace}name", and the namespace is free + # text: `xmlns:x=" ip access-list extended acl_in permit ip + # any any"` on put that permit line at the top of acl_in. + if not _ROLE_PATTERN.fullmatch(role): + raise PfSenseConversionError( + f"{REFUSALS['invalid_interface_role']}: {role!r}" + ) ipaddr = _text(iface_el, "ipaddr") subnet = _text(iface_el, "subnet") if not ipaddr or not subnet: @@ -516,10 +546,9 @@ def _acl_name(role: str, *, rule_bearing_roles: "set[str]") -> str: the same convention _assign_cisco_interface_names() already uses for interface names, rather than a second naming scheme. - `role` is an XML tag name (see _parse_interfaces()), not free text pulled - from an element's content the way or are, so it is - already constrained by XML's own tag-name grammar and does not need the - separate validation those fields do. + `role` is an XML tag name, validated against _ROLE_PATTERN in + _parse_interfaces(). This docstring used to say a tag name was already + safe by XML's own grammar; a namespaced tag is not -- see there. """ if role in rule_bearing_roles and len(rule_bearing_roles) == 1: return ACL_NAME @@ -561,6 +590,14 @@ def _resolve_endpoint(el: ET.Element, interfaces: Dict[str, Dict[str, str]]) -> if el.find("any") is not None: return "any" + # inverts the match: "everything EXCEPT this" (filter.inc:4249). + # It used to be ignored, so the emitted line matched exactly the traffic + # the rule exempts and none of the traffic it covers. One Cisco line + # cannot say "except", so the rule is refused. After the check on + # purpose: pfSense ignores beside too (filter.inc:4103). + if el.find("not") is not None: + raise PfSenseConversionError(REFUSALS["negated_endpoint"]) + network = _text(el, "network") if network is not None: if network not in interfaces: @@ -755,6 +792,18 @@ def _rule_to_acl_line(rule_el: ET.Element, interfaces: Dict[str, Dict[str, str]] f"{pf_protocol!r}" ) + # Two fields that make pfSense match LESS than the fields above say, and + # that used to be ignored (filter.inc at 480a1c0626). Ignored, a block + # rule blocked more in the model than on the firewall: a stricter ACL, + # the direction that turns a real violation into a clean result (#302). + if _text(rule_el, "ipprotocol") == "inet6": # filter.inc:4333 + raise PfSenseConversionError(REFUSALS["ipv6_rule"]) + icmp_type = _text(rule_el, "icmptype") + if pf_protocol == "icmp" and icmp_type not in (None, "any"): # filter.inc:4522 + raise PfSenseConversionError( + f"{REFUSALS['icmp_type']}: is {icmp_type!r}" + ) + source_el = rule_el.find("source") destination_el = rule_el.find("destination") if source_el is None or destination_el is None: @@ -848,6 +897,12 @@ def convert(xml_path: Union[str, Path]) -> ConversionResult: filter_el = root.find("filter") rule_els = filter_el.findall("rule") if filter_el is not None else [] + # A disabled rule is not loaded by pfSense at all (filter.inc:5543), so + # leaving it out is the exact model, not a gap -- nothing is skipped. + # It used to be converted as live: a disabled block denied traffic in + # the model that the real firewall lets through. + rule_els = [r for r in rule_els if r.find("disabled") is None] + # WHY AN EMPTY RULE SET IS REFUSED, NOT CONVERTED AS "NO ACL" # Found in a senior-level adversarial QA pass. PF Sense fails CLOSED # with no rules configured on an interface -- it blocks everything. diff --git a/analysis/policy.py b/analysis/policy.py index 50d15bd..6209d4c 100644 --- a/analysis/policy.py +++ b/analysis/policy.py @@ -699,20 +699,55 @@ def load_policy_file(path: Any) -> Policy: if not file_path.exists(): raise PolicyError(f"no policy file at {file_path}") - text = file_path.read_text(encoding="utf-8").strip() + try: + text = file_path.read_text(encoding="utf-8").strip() + except UnicodeDecodeError as error: + raise PolicyError( + f"{file_path} is not UTF-8 text: byte {error.start} cannot be " + f"decoded. Save the policy as UTF-8 and try again" + ) from error if not text: # An empty FILE is an empty POLICY, which is valid and loud (D4). return Policy({section: [] for section in POLICY_SECTIONS}) + # Three more ways a file used to escape as a raw exception -- an HTTP 500 + # on upload rather than a refusal naming the file: an integer of more + # than 4,300 digits (ValueError), and nesting deeper than Python's + # recursion limit, hit by json.loads or by load_policy() walking it. try: - parsed = json.loads(text) + parsed = json.loads(text, object_pairs_hook=_refuse_duplicate_keys) + return load_policy(parsed) except json.JSONDecodeError as error: raise PolicyError( f"{file_path} is not valid JSON: line {error.lineno}, " f"column {error.colno}: {error.msg}" ) from error - - return load_policy(parsed) + except PolicyError: + raise + except RecursionError as error: + raise PolicyError(f"{file_path} is nested too deeply to be a policy") from error + except ValueError as error: + raise PolicyError(f"{file_path} holds a value that cannot be read: {error}") from error + + +def _refuse_duplicate_keys(pairs: List[Tuple[str, Any]]) -> Dict[str, Any]: + """Build one JSON object, refusing a key that appears twice in it. + + json.loads keeps the LAST of two equal keys and drops the first without a + word. Measured: `{"policy_compliance": [], "policy_compliance": + []}` loaded as an EMPTY policy, and the upload said so with HTTP 200 -- + the user's rule gone. An unknown key is refused (D5) because a key nobody + reads is a rule nobody enforces; a key written twice is the same failure. + """ + built: Dict[str, Any] = {} + for key, value in pairs: + if key in built: + raise PolicyError( + f"the key {key!r} appears twice in one object. JSON keeps only " + f"the last, so the first would be silently ignored" + ) + built[key] = value + return built # ------------------------------------------------------------------------------ diff --git a/tests/test_business_context.py b/tests/test_business_context.py index 6d9429e..3635fd8 100644 --- a/tests/test_business_context.py +++ b/tests/test_business_context.py @@ -383,6 +383,29 @@ def test_malformed_json_names_the_line_and_column(tmp_path): assert "line" in message and "column" in message +@pytest.mark.parametrize( + "contents, said", + [ + # Loaded as "standard" before the fix: the user's "critical" dropped. + pytest.param(b'[{"device": "rtr-us5", "tier": "critical", "tier": "standard"}]', + "appears twice", id="duplicate-key"), + pytest.param(b'[{"device": "r\xe9", "tier": "critical"}]', "not UTF-8", id="not-utf8"), + pytest.param(b'[{"device": "r", "tier": 1' + b"0" * 5000 + b"}]", "cannot be read", + id="5000-digit-int"), + pytest.param(b"[" * 100000 + b"]" * 100000, "nested too deeply", id="deep-json"), + ], +) +def test_a_file_json_cannot_hold_faithfully_is_refused_not_crashed_or_trimmed( + tmp_path, contents, said +): + """Mirrors the policy loader's test: a raw exception (HTTP 500 on + upload), or a silently dropped value, before the fix.""" + path = tmp_path / "business-context.json" + path.write_bytes(contents) + with pytest.raises(BusinessContextError, match=said): + load_business_context_file(path) + + def test_a_real_file_round_trips(tmp_path): path = tmp_path / "business-context.json" path.write_text( diff --git a/tests/test_business_context_wiring.py b/tests/test_business_context_wiring.py index 35ddbb9..1e550ff 100644 --- a/tests/test_business_context_wiring.py +++ b/tests/test_business_context_wiring.py @@ -227,6 +227,30 @@ def test_an_invalid_context_is_rejected_with_the_loaders_own_message(body, expec assert expected in response.json()["detail"] +@pytest.mark.parametrize( + "raw, expected", + [ + # PowerShell 5.1's `>` and Notepad's "Unicode" both save UTF-16. + (json.dumps([{"device": "rtr-us5", "tier": "critical"}]).encode("utf-16"), + "not UTF-8"), + (json.dumps([{"device": "rtr-café", "tier": "critical"}], + ensure_ascii=False).encode("latin-1"), + "not UTF-8"), + (b"[" * 200_000 + b"]" * 200_000, "nested too deeply"), + ], + ids=["utf-16", "latin-1", "deep-nesting"], +) +def test_a_file_the_loader_cannot_decode_is_a_400_not_a_500(raw, expected): + """Measured on origin/main: all three were HTTP 500 Internal Server + Error, the same escape as the policy loader's (see + test_web_policy_upload.py). Refused, never guessed at.""" + response = _post_context(raw) + + assert response.status_code == 400 + assert expected in response.json()["detail"] + assert not main.BUSINESS_CONTEXT_PATH.exists() + + def test_a_rejected_context_is_never_staged(): """The safety property the validate-then-stage order exists for.""" _post_context([{"device": "rtr-us5", "tier": "critical"}]) diff --git a/tests/test_pfsense_convert.py b/tests/test_pfsense_convert.py index cfa978a..099d295 100644 --- a/tests/test_pfsense_convert.py +++ b/tests/test_pfsense_convert.py @@ -987,6 +987,74 @@ def test_a_whitespace_only_interface_gets_the_right_refusal_message(): ) +# --- Fields that change what a rule matches (pfSense filter.inc at 480a1c0626) --- +# Each used to be ignored, so the rule converted as if it matched more traffic +# than it does. Every rule below is , so the order model is not in play. + + +def _acl_lines(text: str) -> list: + return [ln.strip() for ln in text.splitlines() if ln.strip().startswith(("permit", "deny"))] + + +def test_a_disabled_rule_is_left_out_as_pfsense_does(): + """Measured before the fix: a disabled catch-all block came out as the + first line of acl_in, so Batfish denied LAN->8.8.8.8:80, which the real + firewall permits (it does not load a disabled rule, filter.inc:5543).""" + result = _convert_string_full(_minimal_xml(rules_xml=""" + blocklan + + passlan + lan + """)) + assert _acl_lines(result.text) == ["permit ip 10.0.0.0 0.0.0.255 any"] + assert result.skipped == [], "an exact model of a disabled rule is not a gap" + + +def test_a_negated_destination_is_refused_not_inverted(): + """ means "every destination EXCEPT 10.20.0.5". Before the fix it + emitted `permit tcp ... host 10.20.0.5`: Batfish permitted exactly the + host the rule exempts and denied the traffic it allows.""" + result = _convert_string_full(_minimal_xml(rules_xml=""" + passlantcp + lan +
10.20.0.5
+ blocklan + + """)) + assert _acl_lines(result.text) == ["deny ip any any"] + assert len(result.skipped) == 1 + assert REFUSALS["negated_endpoint"] in result.skipped[0] + + +def test_an_ipv6_only_rule_is_not_applied_to_ipv4_traffic(): + """An inet6 block on the LAN subnet came out as an IPv4 `deny ip + 10.0.0.0 0.0.0.255 any` ahead of the IPv4 pass, so every IPv4 flow from + LAN was denied in the model and permitted by the firewall.""" + result = _convert_string_full(_minimal_xml(rules_xml=""" + blocklaninet6 + lan + passlaninet + lan + """)) + assert _acl_lines(result.text) == ["permit ip 10.0.0.0 0.0.0.255 any"] + assert len(result.skipped) == 1 + assert REFUSALS["ipv6_rule"] in result.skipped[0] + + +def test_an_icmp_type_limit_is_refused_not_widened_to_all_icmp(): + result = _convert_string_full(_minimal_xml(rules_xml=""" + blocklanicmp + echoreq + + passlanicmp + any + + """)) + assert _acl_lines(result.text) == ["permit icmp any any"] + assert len(result.skipped) == 1 + assert REFUSALS["icmp_type"] in result.skipped[0] + + # --- Rule order is preserved ----------------------------------------------------- @@ -1164,6 +1232,29 @@ def test_if_with_an_embedded_newline_also_raises(): _convert_string(xml_text) +def test_a_namespaced_interface_tag_cannot_inject_config_lines(): + """The interface ROLE is a tag name, and ElementTree reports a namespaced + tag as "{namespace}name" -- free text, newlines included. Before the fix + this emitted `permit ip any any` as the first line of acl_in, and + Batfish's testFilters on the LAN ACL answered PERMIT for every flow, + against a rule set whose only rule is a deny.""" + xml_text = _minimal_xml( + interfaces_xml=""" + em1LAN10.0.0.124 + + em2OPT110.30.0.124 + + """, + rules_xml=""" + blocklan + + """, + ) + with pytest.raises(PfSenseConversionError) as excinfo: + _convert_string(xml_text) + assert REFUSALS["invalid_interface_role"] in str(excinfo.value) + + def test_a_hostname_with_ordinary_punctuation_still_converts(): """The fix must not become a blanket ban -- real PF Sense hostnames can contain dots and hyphens (e.g. "fw-01.branch.example"), and that must diff --git a/tests/test_policy_loader.py b/tests/test_policy_loader.py index 8ffa184..985189e 100644 --- a/tests/test_policy_loader.py +++ b/tests/test_policy_loader.py @@ -210,6 +210,38 @@ def test_an_empty_file_is_an_empty_policy_not_a_parse_error(tmp_path): assert load_policy_file(path).is_empty +_A_RULE = (b'{"description": "DNS allowed", "node": "rtr-us5", "filter": "acl_in", ' + b'"kind": "requirement", "queries": [{"dstIps": "218.8.104.58"}], ' + b'"violation_severity": "high", "violation_summary": "DNS is blocked"}') + + +@pytest.mark.parametrize( + "contents, said", + [ + # The rule is silently dropped by json.loads: the policy loaded EMPTY. + pytest.param(b'{"policy_compliance": [' + _A_RULE + b'], "policy_compliance": []}', + "appears twice", id="duplicate-key"), + pytest.param(b'{"device": "r\xe9", "policy_compliance": []}', "not UTF-8", id="not-utf8"), + pytest.param(b'{"policy_compliance": [{"number": 1' + b"0" * 5000 + b"}]}", + "cannot be read", id="5000-digit-int"), + pytest.param(b"[" * 100000 + b"]" * 100000, "nested too deeply", id="deep-json"), + # json.loads manages this one; the validation walking it did not. + pytest.param(b'{"policy_compliance": [' + b"[" * 2000 + b"]" * 2000 + b"]}", + "nested too deeply", id="deep-validation"), + ], +) +def test_a_file_json_cannot_hold_faithfully_is_refused_not_crashed_or_trimmed( + tmp_path, contents, said +): + """Each of these escaped as a raw UnicodeDecodeError, ValueError or + RecursionError (HTTP 500 on upload), or -- the duplicate key -- loaded + with the user's rule gone and HTTP 200.""" + path = tmp_path / "policy.json" + path.write_bytes(contents) + with pytest.raises(PolicyError, match=said): + load_policy_file(path) + + # --------------------------------------------------------------------------- # D2 -- per entry, with an optional top-level default # --------------------------------------------------------------------------- diff --git a/tests/test_web_policy_upload.py b/tests/test_web_policy_upload.py index c4d8a94..77e36c7 100644 --- a/tests/test_web_policy_upload.py +++ b/tests/test_web_policy_upload.py @@ -330,6 +330,12 @@ def test_an_empty_upload_is_rejected_rather_than_read_as_asserting_nothing(): {"policy_compliance": [{"filter": "acl_in"}]}, "missing the required description/node", ), + # The next three were an HTTP 500 (raw exception) or, the last, a 200 + # with the rule silently dropped, until the loader refused them. + pytest.param(b'{"device": "r\xe9"}', "not UTF-8", id="not-utf8"), + pytest.param(b"[" * 100000 + b"]" * 100000, "nested too deeply", id="deep"), + pytest.param(b'{"routing": [{"description": "x"}], "routing": []}', + "duplicate key", id="duplicate-key"), ], ) def test_a_rejected_policy_is_never_staged(bad_body, why): @@ -351,6 +357,29 @@ def test_a_rejected_policy_is_never_staged(bad_body, why): ) +@pytest.mark.parametrize( + "raw, expected", + [ + # PowerShell 5.1's `>` and Notepad's "Unicode" both save UTF-16. + (json.dumps(VALID_POLICY).encode("utf-16"), "not UTF-8"), + (json.dumps(VALID_POLICY).replace("rtr-us5", "rtr-café").encode("latin-1"), + "not UTF-8"), + (b"[" * 200_000 + b"]" * 200_000, "nested too deeply"), + ], + ids=["utf-16", "latin-1", "deep-nesting"], +) +def test_a_file_the_loader_cannot_decode_is_a_400_not_a_500(raw, expected): + """Measured on origin/main: all three were HTTP 500 Internal Server + Error, because `UnicodeDecodeError` and `RecursionError` escaped the + loader instead of becoming a `PolicyError` the endpoint already turns + into a message. Refused, never guessed at: no other encoding is tried.""" + response = _post_policy(raw) + + assert response.status_code == 400 + assert expected in response.json()["detail"] + assert not main.POLICY_PATH.exists() + + def test_a_rejection_leaves_an_earlier_good_policy_alone(): """A bad upload must not destroy the policy already staged.