Skip to content

Three stability fixes: ZIP download, CSV formulas, huge history id - #476

Open
ARSH871-bot wants to merge 5 commits into
mainfrom
fix/stability-csv-history-zip
Open

ARSH871-bot wants to merge 5 commits into
mainfrom
fix/stability-csv-history-zip

Conversation

@ARSH871-bot

Copy link
Copy Markdown
Owner

Three small fixes from the October stability audit, as three commits that can be reviewed separately. Each defect was reproduced on main at ee769b4, and the branch is rebased onto b6c104d (#467) first.

1. The suite runs from a downloaded ZIP (25e9786). GitHub's "Download ZIP" has no .git folder, so the git-dependent tests in test_no_configs_in_git.py can only give false alarms there.

copy of main with no .git      before: 20 failed, 2263 passed, 54 skipped
this branch, same copy         after:   0 failed, 2294 passed, 77 skipped

The git checks skip with a reason outside a checkout and still fail on CI, so constraint 1's guard cannot silently stop running there. test_suite_runs_from_a_download.py runs the module somewhere with no .git, plus a positive control in a real clone.

2. A CSV export no longer lets config text run as a formula (2d7b20f). A hostname, ACL name or policy phrase reaches the CSV unchanged, so a hostname of =HYPERLINK(...) is evaluated when the export is opened in a spreadsheet. The HTML report already escapes the same text. Cells starting with =, +, -, @, tab or CR now get a leading apostrophe; text that starts harmlessly is untouched. On main, the 24 new cases (6 triggers x summary, detail, source, device) all fail.

3. Deleting a saved scan with a huge id is a 404 (f1a68ed). SQLite ids are 64-bit; a wider number in the path raised OverflowError, a 500. On main, 3 new cases fail (2**63, 99999999999999999999, -(2**63)-1). delete() is the only history method that takes an id.

Checked on this branch

  • CI-style (Batfish and Ollama refused), on the rebased branch: 2321 passed, 54 skipped (main b6c104d is 2287). Live, with Batfish and Ollama running, measured before the rebase onto Guard only json.loads with the bare ValueError catch in the policy and business-context loaders #467: 2358 passed, 13 skipped.
  • Ruff is clean on analysis, web and tests. The 8 findings it reports elsewhere are in docs/document-builds and are on main too.
  • Mutation: 13 mutants of the two code fixes, all caught: each of the six trigger characters dropped, no apostrophe, every cell prefixed, cells not passed through the function, and four history bounds (guard removed, upper only, lower only, off by one).

Not in this PR: nothing touches web/main.py, ai/ or the pfSense converter, so it does not overlap #401, #405, #406 or #463.

Drafted with AI assistance (Claude), in line with the academic integrity policy.

GitHub's "Download ZIP" has no .git folder. From exactly that download,
`pytest tests/` gave 20 failed, all in test_no_configs_in_git.py and all
false alarms: those tests ask git what it ignores, and with no repository
git cannot answer. Two more passed there having checked nothing, because
`ls-files` lists no files and `check-ignore` says "not ignored" for every
path outside a repository.

The four git-dependent tests now take a `git_checkout` fixture. They run
where git's top level is this repository, skip with the reason anywhere
else (a ZIP, or a ZIP unzipped inside some other repository), and fail on
CI, where a missing checkout would mean constraint 1's guard silently
stopped running.

tests/test_suite_runs_from_a_download.py copies the module somewhere with
no .git and runs it as a user would, plus a positive control that the
checks still run in a real clone. Four mutants of the fixture (dropped
from a test, no CI branch, any repository above counts, checkout never
recognised) are each caught by the test written for it.

Measured:
  from GitHub's ZIP of dc93ccb   20 failed  ->  0 failed (1832 passed, 72 skipped)
  this branch, Batfish live      1891 passed, 13 skipped
  this branch, Batfish down      1855 passed, 49 skipped
A hostname, ACL name or policy phrase reaches the CSV unchanged, so a config
whose hostname is =HYPERLINK(...) became a cell a spreadsheet evaluates on
opening (OWASP "CSV Injection"). The HTML report already escapes the same text.

Cells starting with = + - @ tab or CR now get a leading apostrophe, the
spreadsheet convention for "this is text". Text that starts harmlessly is
left exactly as it was.

24 parametrised cases (6 triggers x summary, detail, source, device) fail on
main and pass here.
FastAPI accepts any size of integer in the path, and SQLite raised
OverflowError binding one wider than its 64-bit ids, which reached the user as
an Internal Server Error. An id outside the 64-bit range names no scan, so
delete() now says so (False, which the endpoint already turns into a 404).

3 cases (2**63, 99999999999999999999, -(2**63)-1) fail on main and pass here.

@shubhamkataria2005 shubhamkataria2005 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Requesting changes on f1a68ed for one thing. The CSV fix covers render_csv but not the other CSV this module writes, render_comparison_csv, which web/main.py:2764 serves for "Compare with an earlier scan". The other two fixes check out.

The gap, on this head:

f = make_finding(..., device='=HYPERLINK("http://evil","x")', summary="=1+1",
                 detail="@SUM(A1:A2)", source="+cmd", ...)
report.render_comparison_csv([f], [], 0)
introduced,AC-001,access_control,found,high,"=HYPERLINK(""http://evil"",""x"")",=1+1,@SUM(A1:A2),+cmd

render_csv gives '=HYPERLINK(…), '@SUM(…) and '+cmd for the same finding, so the same hostname is defused in one export and live in the other. The fix is the one you already wrote: writer.writerow(_csv_cell(cell) for cell in [...]) at line 501, plus the same parametrised test pointed at render_comparison_csv. The change_type and summary rows hold only our own words, so they can stay as they are.

Checked and fine:

  • Merged onto current main (b6c104d): 2362 passed, 13 skipped, ruff clean on CI's selection.
  • The ZIP case, reproduced. A git archive copy with no .git: test_no_configs_in_git.py gives 2 passed, 22 skipped. The same copy with CI=true gives 22 errors, so constraint 1's guard can't quietly stop running on CI. Checking that the top level is this repository, not any repository above it, is the right detail.
  • render_csv: every trigger cell is prefixed, and benign text is untouched.
  • Huge ids through the real API: DELETE /api/history/99999999999999999999, 2**63 and -(2**63)-1 are all 404, not 500.

render_comparison_csv, behind "Compare with an earlier scan", has its own
writerow, and the earlier commit only reached render_csv. The same hostname
was defused in one export and live in the other. Every row of every bucket
(introduced, resolved, newly_blind, newly_sighted) now goes through the same
function. 96 new cases (4 buckets x 4 fields x 6 triggers) fail without it.

Found by @shubhamkataria2005 in review.
@ARSH871-bot

Copy link
Copy Markdown
Owner Author

Fixed on 2a7d39c. Thanks @shubhamkataria2005: render_comparison_csv had its own writerow and I only reached render_csv. Every row of every bucket (introduced, resolved, newly_blind, newly_sighted) now goes through _csv_cell, in ffedae2. 96 new cases (4 buckets x 4 fields x 6 triggers) fail without it and pass with it. CI-style on the merged tree: 2583 passed, 63 skipped, ruff clean. Live (Batfish and Ollama running) on the same content before merging main in: 2633 passed, 13 skipped. Re-requested your review.

@shubhamkataria2005 shubhamkataria2005 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approving 2a7d39c, which replaces my request for changes.

  • My reproduction now comes out defused: introduced,AC-001,…,"'=HYPERLINK(…)",'=1+1,'@SUM(A1:A2),'+cmd.
  • Removing _csv_cell from the comparison rows fails 96, so all four buckets × four fields × six triggers are pinned.
  • No third CSV to miss. Every csv.writer in analysis/, ai/, web/ and tools/ is one of the two in report.py (:216 and :483), and both now go through _csv_cell.
  • Merged onto current main (cceffec): 2633 passed, 13 skipped, ruff clean on CI's selection.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants