Fail fast on 4xx navigation status (fixes testaro-tool hang on blocked pages)

[wittjeff]

What

launch() fails fast on a 4xx navigation status instead of retrying it and switching browser types.

Why (diagnosis for #48)

Investigating pages that hang testing (from #48 and our own corpus), I isolated the stall to a single tool. On https://www.salesforce.com/, running each tool separately:

tool	result
axe / alfa / qualWeb / ibm / aslint / htmlcs / ed11y	complete in 14–42 s
testaro	hangs (killed at 110 s; would run to job timeout)

The site returns 403 to the automated browser. In launch(), a failed navigation retries, and when retries are exhausted for the current browser it switches browser type (chromium → webkit → firefox) and resets the retry counter. For a persistent 403 that's ~11 launch+navigate attempts, most on the slow-to-launch WebKit/Firefox — and because the testaro tool relaunches per rule, it compounds into a multi-minute (up to job-timeout) stall on one target. No other tool does per-rule relaunch, so only testaro hangs.

A 4xx is the server refusing — retrying and browser-switching can't change it. This change skips the retry loop for 4xx, except 408 (Request Timeout) and 429 (Rate Limited), which are legitimately transient (429 already has dedicated handling below). 5xx / timeouts / other transient failures are unchanged.

Verified

With the change, salesforce.com for the testaro tool returns in ~1 s (definitive block) instead of hanging. Not a full close of #48 — the 403 itself is a WAF-block question — but it removes the "prevent jobs from being completed" hang, which is one of #48's stated symptoms.

[jrpool]

Thanks for addressing this problem, @wittjeff.

Reproducing the issue on https://www.salesforce.com, I get each launch attempt returning 403 errors on all retries with chromium and then succeeding after 1 retry with webkit. This is true for all launches after the catalog launch, including the 9 launches of testaro. The reported elapsed times per rule engine are:

    "toolTimes": {
      "testaro": 67,
      "alfa": 28,
      "axe": 10,
      "aslint": 9,
      "ed11y": 8,
      "ibm": 8,
      "nuVnu": 8,
      "qualWeb": 8,
      "htmlcs": 7,
      "nuVal": 7
    },

There is no timeout, the job completes, and the report is written with rule violations from all rule engines. Treating a 4xx as fatal would seem to prevent this useful result.

If webkit is the job-specified browser type, I get a healthy result with no retries at all. If firefox, a healthy result with 2 total retries in the job.

What about making Testaro learn from experience that repeated 4xx failures on one browser type should stop that browser type from being tried again?

[jrpool]

Version 76.2.0 now alters the value of the job property browserID if the current browser exhausts its retries and any unused browser IDs are still left. On Salesforce with chromium as the initial value the retries get exhausted on the first or second launch and then webkit becomes the job browser type and there are no more timeouts.