Repository navigation
A computed fact for routing-assertion findings (#395) - #399
Conversation
ai/explain.py had three computed-fact functions (dead rules, policy compliance, access_control policy statements) for one class of mistake: a model attributing which of two opposite states is which. Routing assertions had no equivalent, and this one found a different failure, not the one it was built looking for. Live-tested a real routing violation six times with no computed fact: the reachability direction itself (reachable vs not) was never inverted, but 3 of 6 runs inverted the flow's source and destination -- 'the HQ network cannot be reached FROM the branch network' instead of the real 'HQ cannot reach branch'. Both IPs were genuine in every run, so the existing invented-fact guard would not have caught it -- the addresses were just attributed to the wrong role in the relationship. _compute_routing_outcome() states the flow's direction explicitly (source before destination, 'that direction, not the reverse') as well as its real disposition, rather than only the disposition the way the other three functions state only their two-state fact. Re-ran the same live finding six times with the fix in place: 0 of 6 inverted the direction. Tests use the real routing._evaluate() to build the evidence string, not a hand-typed literal, same discipline test_the_safeguard_can_read_ what_policy_compliance_actually_writes already established (#145/#169) -- a future rewording of _evaluate() fails these tests loudly instead of leaving them green while testing a string that no longer exists. python -m pytest tests/test_explain_safety_net.py tests/test_routing_classification.py -q 96 passed.
Found by mutation-testing my own previous commit: removing _compute_routing_outcome() from _build_prompt()'s chain broke nothing in the direct unit tests, because they call the function in isolation rather than through the chain it is supposed to be wired into. Each of the other three computed-fact functions already has a dedicated test_prompt_includes_the_computed_outcome_for_a_X_finding test that exercises the real wiring -- routing was missing its equivalent. python -m pytest tests/test_explain_safety_net.py tests/test_routing_classification.py -q 97 passed (was 96).
shubhamkataria2005
left a comment
There was a problem hiding this comment.
Reviewed 94bc75c, merged onto current main (b69d591). The direction fix is real and I reproduced it live, but I'm requesting one change: a partly failing multi-path flow gets a computed fact that is false for the paths that succeed.
Live, 6 generations each, real RT-001 from routing-missing-route:
main 3 of 6 reverse the flow (runs 4-6):
"The HQ network cannot be reached from the branch network."
this PR 0 of 6. Every run: "The HQ network cannot reach the branch network."
That's exactly your 3 of 6. Two smaller things improved as well: on main, runs 1-3 also invented roles ("the internal server", "the external network"), and none of the PR's runs did. One PR run said the flow "is currently allowed by the routing configuration, does not actually reach its destination". That's muddled, but it doesn't reverse the direction, so I'm not counting it.
Other checks:
- Merged onto
main: 1907 passed, 13 skipped, ruff clean. - Mutations: unwired from
_build_prompt(),src/dstswapped, the two dispositions swapped,expecteddropped from the sentence. 4 of 4 killed. The wiring test from your second commit is the one that catches the first. - Merges cleanly with #401 (both edit
ai/explain.py).
Change requested: the multi-path suffix
routing._evaluate() adds (n of m paths failed) / (n of m paths succeeded) when there's more than one trace. The regex ignores that suffix, so a flow where one of two paths works gets a flat claim, labelled as already verified. Built with the real _evaluate(), two traces (one ACCEPTED, one NO_ROUTE):
DETAIL Expected REACHABLE for a flow from 10.10.10.5 to 10.20.20.5, but traceroute
ended in NO_ROUTE. Path: rtr-hq (1 of 2 paths failed)
COMPUTED the flow FROM 10.10.10.5 TO 10.20.20.5 ... currently does NOT reach 10.20.20.5;
the trace ended in NO_ROUTE. That is the real, observed outcome.
Half of that traffic does arrive. The other three computed-fact functions return None rather than pick a side when their evidence is mixed (_compute_dead_rule_outcome() when blocking lines disagree), and ai/query.py reports this exact case as "Mixed result: n of m traced paths ... The paths disagree". The UNREACHABLE mirror ("DOES reach", 1 of 2 succeeded) is true as far as it goes, but it drops the "1 of 2".
Two fixes, either is fine:
- return
Nonewhen the suffix is present andn < m, or - state it: "n of m traced paths FROM src TO dst do not reach dst".
Plus one test that builds the detail through _real_routing_detail() with two traces, so a future change to the suffix fails loudly.
None of our fixtures have equal-cost paths today, so this can't show up in a demo yet. But this function's job is to say only what it's sure of, and as written it's surest in exactly the case it hasn't handled.
routing._evaluate() appends "(n of m paths failed/succeeded)" when more than one path was traced. The regex ignored that suffix, so a flow where only some paths disagree got the same flat "does NOT reach"/"DOES reach" statement as a unanimous result -- an overclaim on exactly the evidence this function exists to get right. When n < m (a real split), the computed fact now states the fraction explicitly, the same way ai/query.py's traceroute answer already states a mixed result rather than guessing a single yes/no, instead of returning None the way the other three computed-fact functions do on ambiguous evidence -- the fraction itself is the verified fact, and withholding it would leave the model to describe the split in its own words. When n == m every path agrees, so the flat statement is unchanged and still exactly accurate. Live-tested against the real model, 6 of 6 runs correctly conveyed the split rather than flattening it. Addresses the change requested on PR #399 (review by Shubham).
|
Fixed, taking the "state it" option rather than returning None. Extended Went with stating the fraction rather than returning Reproduced your exact repro (two traces, one ACCEPTED, one NO_ROUTE) and added three tests built against the real Mutation-tested the new branch: forced Live-tested against the real model on your exact scenario, 6 of 6 runs: All six correctly conveyed the split; none flattened it into a single-direction claim, none inverted source/destination. Pushed as |
shubhamkataria2005
left a comment
There was a problem hiding this comment.
Approving d2d12ae. I re-tested the multi-path fix myself on current main (d4535ef, which now has #298), rather than taking the reply on trust.
The fix: with (n of m paths ...) and n < m, the computed fact now states the split. With n == m it keeps the flat statement, which is still exactly true. Built through the real routing._evaluate():
1 of 2 failed "1 of 2 traced paths for the flow FROM 10.10.10.5 TO 10.20.20.5 -- that direction,
not the reverse -- do NOT reach 10.20.20.5 (one ended in NO_ROUTE); the rest do."
Mutations, 5 of 5 killed: mixed always true, mixed never, < changed to <= (so a unanimous 2-of-2 gets hedged), the suffix group not captured, and the mixed REACHABLE sentence flipped to "DO reach". The unanimous test is what catches <=. Good call adding it.
Live, netwise-warden: 6 runs on the 1-of-2 case, plus 3 runs on a 2-of-3 case your tests don't cover (ACCEPTED, NO_ROUTE, NULL_ROUTED). 9 of 9 convey the split, none flatten it, and none reverse the direction. Two wording slips by the model, not the code: one 2-of-3 run says "the remaining paths are reachable" (there is one), and another says "if you were to take any of these paths, you would not be able to reach" it. Neither contradicts the fact; I'm noting them only.
Merged onto d4535ef: 2013 passed, 13 skipped, ruff clean on CI's selection.
Follow-up worth an issue, not this PR: every run, including the clean ones, opens with "The HQ network cannot reach the branch network." That's the finding's own summary, from the assertion's violation_summary in routing.py. It's fixed text, so it stays flat even when half the paths work, and the model leads with it before giving the split. The computed fact can't fix that, because the card's title says it before any explanation. One option: routing.py appends "(on 1 of 2 paths)" to the summary when the result is mixed.
|
Approved on the current head and everything's verified. Please merge when you have a moment. |
|
Merging. My approval is of |
#385's per-finding prompt hash correctly flagged AC-002 as stale the moment main (with #385 and #399) merged onto this branch -- the prompt _build_prompt() builds for it moved when the previous commit fixed #384. Re-captured one live generation under the new prompt and rated it by hand (Ankeet): accuracy 5, usefulness 4, up from 2/2. The old rating's defect is gone -- the dead line's own PERMIT action is now named correctly, and the real blocked outcome is stated without inverting which rule is which. Updated docs/evaluation.md Part 4 with a dated note recording the re-capture rather than silently rewriting the AC-002 row, and updated the two tests that pin the published numbers (mean accuracy 4.25 -> 5.0, mean usefulness 3.25 -> 3.75; the breakdown's low-scorer demonstration now points at PC-005's usefulness, the new minimum, since AC-002 no longer has an outlier to show). Full suite: 2245 passed, 13 skipped, 0 failed.
Part of #395 (US-62), not closing it.
What #395 actually covers
ai/explain.pyhad three computed-fact functions for one class of mistake: a model attributing which of two opposite states is which (_compute_dead_rule_outcome,_compute_policy_outcome,_compute_expected_actual_outcome). Before writing anything, I inventoried everystatus="found"evidence shape acrossaccess_control.py,policy_compliance.py,routing.pyandchange_impact.pyto see which ones have no equivalent guard yet -- six shapes, not one.Alternatives considered for scope
Went with a version of 3, informed by 2. Three of the six shapes (
access_control's GUARANTEES proof, undefined-reference, and unused-structure findings) are flat, one-sided facts with nothing to attribute to the wrong side -- adding a computed-fact function there would be guarding against a risk that doesn't exist, which isn't what these functions are for. That leaves three real candidates: the routing-assertion shape (this PR), and twochange_impact.pyshapes (compareFilters, differentialReachability) as follow-ups. Landing them one at a time, each independently live-tested and reviewed, matches how the first three functions were added and keeps each PR small enough to actually verify rather than trust.What this PR does
_compute_routing_outcome()forrouting.py's assertion findings ("Expected REACHABLE/UNREACHABLE for a flow from X to Y, but traceroute...").A different failure than the one I was looking for
Live-tested a real routing violation (
routing-missing-route, RT-001) six times with no computed fact, expecting to find the same "which state is which" inversion the other three guards catch. That never happened -- 0 of 6 runs inverted REACHABLE vs UNREACHABLE.What happened instead: 3 of 6 runs inverted the flow's source and destination. "The HQ network cannot reach the branch network" (correct) became "The HQ network cannot be REACHED FROM the branch network" -- which describes the opposite flow, branch-to-HQ instead of HQ-to-branch. Both IP addresses were genuine in every run, so
_names_an_invented_fact()would not have caught it: the addresses were just attributed to the wrong role in the relationship, not invented.So the computed fact states the direction explicitly (
"the flow FROM {src} TO {dst} -- that direction, not the reverse") as well as the real disposition, rather than only the disposition the way the other three functions state only their two-state fact.Re-verified after the fix
Same finding, six more live runs, fix in place:
0 of 6 inverted the direction.
A gap I found in my own first commit
Mutation-tested by removing
_compute_routing_outcome()from_build_prompt()'s chain -- and my own unit tests for the function (tested in isolation) didn't catch it, because none of them went through the actual wiring. Each of the other three shapes has a dedicatedtest_prompt_includes_the_computed_outcome_for_a_X_findingtest that exercises the real chain; routing was missing its equivalent. Added it in a second commit, re-ran the same mutation, caught correctly the second time. Recording this because a mutation survivor from your own code is easy to wave past, and this one would have shipped a function that looked tested but wasn't wired to anything.Tests against the real producer, not a hardcoded literal
_real_routing_detail()calls the genuinerouting._evaluate()to build the evidence string, the same discipline_real_detail()already established forpolicy_compliance(#145/#169) -- a future rewording of_evaluate()fails these tests loudly instead of leaving them green while testing a string that no longer exists.Verified
No em-dashes in either changed file.
What this deliberately does not do
Does not cover the two
change_impact.pyshapes (compareFilters, differentialReachability) -- planned as follow-up PRs against the same issue, see "Alternatives considered" above. Does not touch_compute_dead_rule_outcome,_compute_policy_outcome, or_compute_expected_actual_outcome. Does not add a computed-fact function for the three flat-fact shapes, with the reasoning for each documented in the issue discussion rather than silently skipped.Part of #395. Does not close it.