Answer 404 for paths that match no page - #67
Merged
Conversation
use_pages=True registers a catch-all /<path:path> that serves the Dash index for ANY url and leaves the "page not found" screen to the client-side router, so the HTTP status was 200 no matter what was asked for. One hour of this deployment's access log is a few hundred webshell probes -- /shell.php, /wp-is.php, /404.php, /mail.php -- each answered 200 with a full 9.8 KB app shell. Three costs, in increasing order: bandwidth; a visit log that cannot separate a page view from a probe; and a host that reads to a scanner as a live PHP target worth coming back to. The guard is registered BEFORE record_visit, which is the part that matters operationally. Flask runs before_request handlers in registration order and stops at the first that returns a response, so that position keeps a probe out of the visitor DB and out of the third-party geolocation lookup record_visit performs on every logged request -- a lookup that is rate-limited at 45/min and was being spent on scanner traffic. Membership and the body live in src/routing.py, pure and importable without a Dash app or a store, following the split main.py already makes between check_price_store and price_store_verdict. The page registry and route list are read per request rather than captured at import, so a page added later is served without touching the guard. Verified against the live app: all 16 real paths (including the trailing-slash spelling and nested /citpy/view) still 200, all 9 probe paths now 404, and the 404 body is 707 bytes against the shell's 9851. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
use_pages=Trueregisters a catch-all/<path:path>that serves the Dash index for any url and leaves the "page not found" screen to the client-side router. The HTTP status was therefore 200 no matter what was asked for.One hour of this deployment's access log is a few hundred webshell probes —
/shell.php,/wp-is.php,/404.php,/mail.php,/xxx.php— each answered200with a full 9.8 KB app shell.Three costs, in increasing order of importance:
Ordering is the operational part
The guard is registered before
record_visit. Flask runsbefore_requesthandlers in registration order and stops at the first that returns a response, so that position is what keeps a probe out of the visitor DB and out of the third-party geolocation lookuprecord_visitperforms on every logged request — a lookup that is rate-limited at 45/min and was being spent on scanner traffic.Structure
Membership and the 404 body live in
src/routing.py, pure and importable without a Dash app or a store, following the splitmain.pyalready makes betweencheck_price_store(I/O) andprice_store_verdict(policy). The page registry and route list are read per request rather than captured at import, so a page added later is served without touching the guard.Verification
Against the live app via the Flask test client:
200, including the trailing-slash spelling (/exposure/) and the nested/citpy/view, plus Dash internals (/_dash-layout,/_dash-update-component) and both favicon paths.404, including/exposure/../etcand/exposure/anything(a page prefix must not admit everything under it).Tests: 44 new (
tests/test_routing.py), pinning membership in both directions — admit too much and the 200-to-everything problem returns; admit too little and a real page 404s, which is a worse outage than the one being fixed. Full suite 536 passed,ruff check src testsclean.🤖 Generated with Claude Code