Repository navigation
Proxy meter: stop counting chrome:// and other local loads in Selenium and Puppeteer - #26
Merged
Merged
Conversation
…m and Puppeteer Chrome's new-tab page loads chrome:// scripts (about 2.8 MB) that CDP reports as finished loads; they never cross a proxy but were added to proxy_bytes. Only http(s) requests and responses are counted now, and the budget block is handed to the page's own event loop. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CKsd7q25N19WzUZjrAdzx8
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
The proxy meter added in the previous PR counted Chrome's own
chrome://loads in Selenium and Puppeteer. The new-tab page's scripts come withresponseReceivedandloadingFinishedevents, and none of them crosses a proxy. Measured on farfetch-scraper's Selenium CLI through a stub proxy: the meter counted 2,779,683 bytes while the stub saw about 6 KB. That spends a large share of the default 10 MB--proxy-budget-mbon traffic that never happened.Resulting behaviour
http:///https://requests and responses are counted (proxy_meter.feed_cdp).data:,blob:,chrome://and other local schemes count nothing.call_soon_threadsafe. Where this repo already had that, nothing changes.proxy_meter.pyandtests/test_proxy_meter.pyare now identical across the repos that carry the meter.Evidence
chrome://new-tab-page/x.jsrequest, response and finish count 0 bytes. Against the previousproxy_meter.pyit fails, and with this one it passes.test_pyppeteer_block_from_a_worker_threadalso fails against the previousproxy_meter.py.tests/test_proxy_meter.pypasses in the Selenium and pyppeteer venvs, with the real-browser tests run, not skipped.Not changed
Engines, sidecar fields, exit codes and the canary are unchanged. There is no release.
🤖 Generated with Claude Code
https://claude.ai/code/session_01CKsd7q25N19WzUZjrAdzx8