[Bug]: web fetch challenge detection false-positives on CSP headers (news.ycombinator.com fails with FETCH_BLOCKED) #264

What happens

isChallengeResponse in src/fetch/classify.ts:3 builds its evidence string from all response headers plus the body, then matches challengeMarkers (cloudflare|cf-chl|datadome|perimeterx|px-captcha|akamai|captcha|just a moment|verify you are human).

Hacker News serves a perfectly good 200 whose content-security-policy header mentions https://cdnjs.cloudflare.com/ and https://www.google.com/recaptcha/. Both markers hit, so a healthy page is classified as a challenge.

plain 200 challenge=true jsShell=false
content-security-policy: default-src 'self'; script-src 'self' 'unsafe-inline' https://www.google.com/recaptcha/ … https://cdnjs.cloudflare.com/; …

Consequence: the plain tier already had the full 34 KB of usable HTML, but the impit ladder runs anyway (chrome, then firefox), both fetches return the same 200 that is again "a challenge", and the command ends in:

$ webcmd web fetch --url https://news.ycombinator.com
CliError: The site blocked non-browser fetches.
  hint: Use webcmd web fetch-browser for this URL.

The page was never blocked.

Why it matters

Any site whose CSP allow-list names a CDN or reCAPTCHA — a very large share of the web — pays two wasted upstream fetches and then fails with a misleading FETCH_BLOCKED telling the agent to escalate to a browser it does not need.

Suggested fix

Narrow the evidence. Header matching should look at the headers that actually signal a challenge (server, cf-mitigated, cf-chl-*, x-datadome, set-cookie names like __cf_bm/datadome) rather than every header value, and CSP/report-to/link should be excluded outright — they are allow-lists of third parties, not evidence about this response. Body matching can stay as-is.

Found while fixing #249; the two are independent (that one is about the retry ladder blowing the --timeout budget, this one is about the ladder running at all).