Repository navigation
Three stability fixes: ZIP download, CSV formulas, huge history id - #476
ARSH871-bot wants to merge 5 commits into
Conversation
GitHub's "Download ZIP" has no .git folder. From exactly that download, `pytest tests/` gave 20 failed, all in test_no_configs_in_git.py and all false alarms: those tests ask git what it ignores, and with no repository git cannot answer. Two more passed there having checked nothing, because `ls-files` lists no files and `check-ignore` says "not ignored" for every path outside a repository. The four git-dependent tests now take a `git_checkout` fixture. They run where git's top level is this repository, skip with the reason anywhere else (a ZIP, or a ZIP unzipped inside some other repository), and fail on CI, where a missing checkout would mean constraint 1's guard silently stopped running. tests/test_suite_runs_from_a_download.py copies the module somewhere with no .git and runs it as a user would, plus a positive control that the checks still run in a real clone. Four mutants of the fixture (dropped from a test, no CI branch, any repository above counts, checkout never recognised) are each caught by the test written for it. Measured: from GitHub's ZIP of dc93ccb 20 failed -> 0 failed (1832 passed, 72 skipped) this branch, Batfish live 1891 passed, 13 skipped this branch, Batfish down 1855 passed, 49 skipped
A hostname, ACL name or policy phrase reaches the CSV unchanged, so a config whose hostname is =HYPERLINK(...) became a cell a spreadsheet evaluates on opening (OWASP "CSV Injection"). The HTML report already escapes the same text. Cells starting with = + - @ tab or CR now get a leading apostrophe, the spreadsheet convention for "this is text". Text that starts harmlessly is left exactly as it was. 24 parametrised cases (6 triggers x summary, detail, source, device) fail on main and pass here.
FastAPI accepts any size of integer in the path, and SQLite raised OverflowError binding one wider than its 64-bit ids, which reached the user as an Internal Server Error. An id outside the 64-bit range names no scan, so delete() now says so (False, which the endpoint already turns into a 404). 3 cases (2**63, 99999999999999999999, -(2**63)-1) fail on main and pass here.
shubhamkataria2005
left a comment
There was a problem hiding this comment.
Requesting changes on f1a68ed for one thing. The CSV fix covers render_csv but not the other CSV this module writes, render_comparison_csv, which web/main.py:2764 serves for "Compare with an earlier scan". The other two fixes check out.
The gap, on this head:
f = make_finding(..., device='=HYPERLINK("http://evil","x")', summary="=1+1",
detail="@SUM(A1:A2)", source="+cmd", ...)
report.render_comparison_csv([f], [], 0)introduced,AC-001,access_control,found,high,"=HYPERLINK(""http://evil"",""x"")",=1+1,@SUM(A1:A2),+cmd
render_csv gives '=HYPERLINK(…), '@SUM(…) and '+cmd for the same finding, so the same hostname is defused in one export and live in the other. The fix is the one you already wrote: writer.writerow(_csv_cell(cell) for cell in [...]) at line 501, plus the same parametrised test pointed at render_comparison_csv. The change_type and summary rows hold only our own words, so they can stay as they are.
Checked and fine:
- Merged onto current
main(b6c104d): 2362 passed, 13 skipped, ruff clean on CI's selection. - The ZIP case, reproduced. A
git archivecopy with no.git:test_no_configs_in_git.pygives 2 passed, 22 skipped. The same copy withCI=truegives 22 errors, so constraint 1's guard can't quietly stop running on CI. Checking that the top level is this repository, not any repository above it, is the right detail. render_csv: every trigger cell is prefixed, and benign text is untouched.- Huge ids through the real API:
DELETE /api/history/99999999999999999999,2**63and-(2**63)-1are all 404, not 500.
render_comparison_csv, behind "Compare with an earlier scan", has its own writerow, and the earlier commit only reached render_csv. The same hostname was defused in one export and live in the other. Every row of every bucket (introduced, resolved, newly_blind, newly_sighted) now goes through the same function. 96 new cases (4 buckets x 4 fields x 6 triggers) fail without it. Found by @shubhamkataria2005 in review.
|
Fixed on |
shubhamkataria2005
left a comment
There was a problem hiding this comment.
Approving 2a7d39c, which replaces my request for changes.
- My reproduction now comes out defused:
introduced,AC-001,…,"'=HYPERLINK(…)",'=1+1,'@SUM(A1:A2),'+cmd. - Removing
_csv_cellfrom the comparison rows fails 96, so all four buckets × four fields × six triggers are pinned. - No third CSV to miss. Every
csv.writerinanalysis/,ai/,web/andtools/is one of the two inreport.py(:216and:483), and both now go through_csv_cell. - Merged onto current
main(cceffec): 2633 passed, 13 skipped, ruff clean on CI's selection.
Three small fixes from the October stability audit, as three commits that can be reviewed separately. Each defect was reproduced on
mainatee769b4, and the branch is rebased ontob6c104d(#467) first.1. The suite runs from a downloaded ZIP (
25e9786). GitHub's "Download ZIP" has no.gitfolder, so the git-dependent tests intest_no_configs_in_git.pycan only give false alarms there.The git checks skip with a reason outside a checkout and still fail on CI, so constraint 1's guard cannot silently stop running there.
test_suite_runs_from_a_download.pyruns the module somewhere with no.git, plus a positive control in a real clone.2. A CSV export no longer lets config text run as a formula (
2d7b20f). A hostname, ACL name or policy phrase reaches the CSV unchanged, so a hostname of=HYPERLINK(...)is evaluated when the export is opened in a spreadsheet. The HTML report already escapes the same text. Cells starting with=,+,-,@, tab or CR now get a leading apostrophe; text that starts harmlessly is untouched. Onmain, the 24 new cases (6 triggers x summary, detail, source, device) all fail.3. Deleting a saved scan with a huge id is a 404 (
f1a68ed). SQLite ids are 64-bit; a wider number in the path raisedOverflowError, a 500. Onmain, 3 new cases fail (2**63,99999999999999999999,-(2**63)-1).delete()is the only history method that takes an id.Checked on this branch
mainb6c104dis 2287). Live, with Batfish and Ollama running, measured before the rebase onto Guard only json.loads with the bare ValueError catch in the policy and business-context loaders #467: 2358 passed, 13 skipped.analysis,webandtests. The 8 findings it reports elsewhere are indocs/document-buildsand are onmaintoo.Not in this PR: nothing touches
web/main.py,ai/or the pfSense converter, so it does not overlap #401, #405, #406 or #463.Drafted with AI assistance (Claude), in line with the academic integrity policy.