ci: run tests/security/ on pull requests - #533
Open
craig-dt wants to merge 1 commit into
Open
Conversation
tests/security/ holds 84 tests across 4 files, including the deny-by-default assertions added after the 2026-05 disclosure, and no pull-request job ran any of them. ci-cd.yml runs tests/unit/ and tests/integration/ only -- both path-scoped -- so the directory was visible solely to nightly.yml (pytest tests/, 02:00 UTC), i.e. after a regression had already merged. Add it as its own step in test-unit-backend rather than widening the unit path, so a security failure is legible instead of buried in ~1000 unit tests. DEV_MODE and JWT_SECRET_KEY are set explicitly because backend/middleware/auth.py defaults DEV_MODE to false, and with no .env in CI an unset JWT_SECRET_KEY makes _load_jwt_secret() raise. test_route_auth_coverage.py does not set it itself -- it currently only works because test_unauth_endpoints.py calls os.environ.setdefault at collection time. CI must not depend on that ordering. --no-cov keeps the step from overwriting the coverage.xml that the Codecov step uploads. Test content is unchanged. Note that test_route_auth_coverage.py is itself vacuous under FastAPI 0.137.1 and will pass without asserting anything until Vigil-SOC#532 lands; the other 83 assertions become real immediately. Closes Vigil-SOC#531 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Signed-off-by: Craig <craig@deeptempo.ai>
This was referenced Aug 3, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Adds a
Run security testsstep to the existingtest-unit-backendjob sotests/security/runs on every pull request.Why
tests/security/holds 84 tests across 4 files — including the deny-by-default assertions added after the 2026-05 disclosure — and no pull-request job ran any of them.ci-cd.ymlrunstests/unit/andtests/integration/only, both path-scoped, so the whole directory was visible solely tonightly.yml(pytest tests/, 02:00 UTC) — after a regression had already merged.test_unauth_endpoints.pytest_path_traversal.py../traversaltest_ssrf.pytest_route_auth_coverage.py/api/*route requires auth or is onPUBLIC_API_PATHSDecisions worth reviewing
Its own step, not a widened path. Running
pytest tests/security/separately rather than extendingpytest tests/unit/keeps a security failure legible in the job log instead of buried among ~1000 unit tests.DEV_MODEandJWT_SECRET_KEYare explicit, not defensive.backend/middleware/auth.py:24defaultsDEV_MODEtofalse, and with no.envin CI an unsetJWT_SECRET_KEYmakes_load_jwt_secret()raise.test_route_auth_coverage.pynever sets it — that file currently only works becausetest_unauth_endpoints.pycallsos.environ.setdefault("JWT_SECRET_KEY", ...)at collection time while the coverage test importsbackend.mainlazily inside its test body. CI must not depend on that accidental cross-file ordering.--no-cov. Stops the step from overwriting thecoverage.xmlthat the Codecov step immediately below uploads.Scope
Test content is unchanged; this only makes existing tests run. Nothing else in CI changes.
Note that
test_route_auth_coverage.pyis itself vacuous under FastAPI 0.137.1 — it examines 1 route out of 359 and passes unconditionally — so it will pass without asserting anything until #532 lands. The other 83 assertions become real immediately, which is why this is worth landing on its own.For the record, the auth boundary is currently intact: 359 effective
/api/*routes, 351 requiring auth, 7 un-authed and all 7 onPUBLIC_API_PATHS, 0 unprotected. This restores regression protection; it is not a live-vulnerability fix.Testing
Also verified: the workflow YAML parses, the new step lands between
Run unit testsandUpload coverage to Codecov, and--no-covleavescoverage.xmluntouched.Clean-environment behaviour (no
.env) is proven by this PR's own CI run — that could not be verified locally without moving the developer's.env.One incidental change: the blank line above the new step previously held trailing whitespace and is now genuinely blank.
Closes #531
🤖 Generated with Claude Code