Repository navigation
Count only network traffic in the pyppeteer and Selenium proxy meter - #12
Merged
Merged
Conversation
Chrome's own chrome:// new-tab loads, data:/blob: URLs, cache hits and its error page were counted as proxy traffic. Only http(s) requests and responses that came over the network are counted now, and the pyppeteer budget block reaches the page's loop when the budget runs out on another thread. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CKsd7q25N19WzUZjrAdzx8
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
In pyppeteer and Selenium, the proxy meter added in #11 counted every CDP
loadingFinished. That included loads that never cross a proxy:chrome://new-tab scripts;data:andblob:URLs;responseReceived.A sibling repo's Selenium CLI, run through a stub proxy, showed the meter reading 2,779,683 bytes while the proxy carried about 6 KB. That spends the default 10 MB
--proxy-budget-mbon traffic that never happened.Separately, the pyppeteer block sent once the budget is spent used
asyncio.ensure_futurefrom whatever thread crossed the cap. A thread with no event loop raised there, the error was swallowed, and the block was never sent.Resulting behaviour
CdpCounter(proxy_meter.py) counts onlyhttp:///https://requests.responseReceivedthat came over the network. A response markedrequestServedFromCache,fromDiskCache,fromPrefetchCacheorfromServiceWorkercounts nothing, and neither does its request. Chrome's error page costs only the request that was attempted.responseReceivedandrequestServedFromCacheare now subscribed (pyppeteer) and read from the performance log (Selenium).call_soon_threadsafe.canary.ymlare unchanged.Evidence
chrome://new-tab-page/x.jsrequest, response and finish (2,779,683 B), plus adata:and ablob:load, count 0;requestServedFromCache,fromDiskCache,fromPrefetchCacheandfromServiceWorkercounts 0;Network.setBlockedURLs ["*"]on the page's loop.🤖 Generated with Claude Code
https://claude.ai/code/session_01CKsd7q25N19WzUZjrAdzx8