Skip to content

ESD-1700: Harden the WASM fetch smoke check against transient network failures - #550

Open
Phil-Browne wants to merge 3 commits into
mainfrom
esd-1700-wasm-smoke-failure-path
Open

ESD-1700: Harden the WASM fetch smoke check against transient network failures#550
Phil-Browne wants to merge 3 commits into
mainfrom
esd-1700-wasm-smoke-failure-path

Conversation

@Phil-Browne

@Phil-Browne Phil-Browne commented Jul 27, 2026

Copy link
Copy Markdown
Contributor

The Browser Fetch Smoke job makes one live staging API call with no retry, and misreports it when it fails. On PR #531 a 10.5s connect timeout came out as expected a non-empty JSON array, which describes neither the failure nor anything that PR changed. Seven sibling PRs passed the identical check in the same 20-second window, and it passed on rerun with no code change.

Three things combined to produce that message: the script checked result.error but the CLI puts errors in a JSON envelope inside result.output; the script ran the command exactly once; and the WASM transport read only the JS error's message, which under Node is a bare fetch failed with the real reason (UND_ERR_CONNECT_TIMEOUT, ECONNREFUSED) sitting on cause.

  • Inspect the error envelope before asserting the array shape, so that assertion only fires on genuinely wrong-shaped success output.
  • Retry the whole attempt once after 5s, logging the retry. It lives in the script, not the workflow, so make wasm-smoke behaves the same locally and in CI.
  • Append the rejection's cause to the transport's error, preferring its message and falling back to its code. TestRoundTrip_FetchRejectionSurfacesCause covers five shapes, all taken from real Node 22 failures.
  • Gate both catch handlers on Type() == js.TypeObject. A non-object rejection reason used to panic on Value.Get, and a panic on a js.FuncOf goroutine kills the whole WASM process rather than just the request.

No retry in the transport itself. That's browser product code and retry policy there is a separate call; this flake belongs to the CI harness.

Verified: make wasm-smoke exits 0 (621 records). With no credentials and WASM_SMOKE_COMMAND='ports list --output json' it exits 1 reporting the login error, after one logged retry. A scratch copy of the script with a one-shot injected fault exits 0 after the retry. Both new panic guards were checked by reverting them and watching the tests panic with syscall/js: call of Value.Get on string.

What I could not verify: the retry window is a judgment call. Two attempts 5s apart covers a single undici connect timeout, but not a longer staging outage, and it doubles worst-case job time to roughly 125s against a 10-minute job timeout. If this job keeps flaking, a staging health gate beats more retries.

Worth a follow-up, not fixed here: the same panic class is still open on the success path, which this PR does not touch. thenFunc reads response.Get("status") and extractHeaders reads headers.Get("forEach") with no type guard, so a fetch that resolves with a non-object (a patched global or a service worker) still takes the process down. Reproduced both. Left alone because it's pre-existing and on the success path, so it wants its own ticket and its own tests.

… failures

The Browser Fetch Smoke job makes one live API call with no retry, and when
that call fails it reports the wrong reason.

- scripts/wasm-smoke.mjs: inspect the CLI JSON error envelope before the
  array-shape assertion, and retry the whole attempt once after 5s.
- internal/wasm/wasmhttp: append a rejected fetch cause to the error message,
  so Node bare "fetch failed" carries ETIMEDOUT / ENOTFOUND / ECONNREFUSED.
@codecov

codecov Bot commented Jul 27, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 79.48%. Comparing base (6737d3b) to head (27c49e8).

Additional details and impacted files

Impacted file tree graph

@@            Coverage Diff             @@
##             main     #550      +/-   ##
==========================================
- Coverage   79.48%   79.48%   -0.01%     
==========================================
  Files         193      193              
  Lines       18859    18859              
==========================================
- Hits        14991    14990       -1     
  Misses       2820     2820              
- Partials     1048     1049       +1     

see 1 file with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Hardens the WASM “fetch transport” smoke check and improves WASM fetch error reporting so transient network failures are retried and, when they do fail, surface actionable root-cause details (e.g., ETIMEDOUT, ENOTFOUND) instead of misleading JSON-shape assertions.

Changes:

  • Update wasm-smoke.mjs to detect the CLI JSON error envelope before asserting array shape, and to retry once after a delay with explicit logging.
  • Enhance WASM fetch transport error strings to append rejection cause details when present.
  • Add WASM transport tests covering multiple cause shapes to ensure the detailed error message is preserved.

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 1 comment.

File Description
scripts/wasm-smoke.mjs Adds error-envelope inspection and a single retry to reduce CI flakes and improve failure diagnosis.
internal/wasm/wasmhttp/transport.go Appends fetch rejection cause details to transport errors for better troubleshooting.
internal/wasm/wasmhttp/transport_test.go Adds table-driven tests to validate surfaced cause details for rejected fetches.

Comment thread internal/wasm/wasmhttp/transport.go Outdated
Reading a property off the rejection value panics when it is not an object,
and a panic on a js.FuncOf goroutine takes the whole WASM process down rather
than just the one request.

- Gate both fetch error handlers on Type() == js.TypeObject instead of ruling
  out undefined and null one at a time.
- Cover null, undefined, string and number rejections with a regression test.
- Rework the cause cases to the shapes real Node 22 fetch failures produce,
  including the dual-stack AggregateError whose message is empty and whose
  code carries the reason.
…uard

Round 2 review found the "prefers the cause message" case used an
ETIMEDOUT shape Node never produces, under a comment claiming the shapes
were reproduced. Node 22 raises undici's own ConnectTimeoutError there
(UND_ERR_CONNECT_TIMEOUT), so use that.

Also tighten textCatch's message check to match the sibling fetch
handler, and cover that path: it was changed with no test behind it.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 3 out of 3 changed files in this pull request and generated no new comments.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 3 out of 3 changed files in this pull request and generated no new comments.

Comments suppressed due to low confidence (1)

internal/wasm/wasmhttp/transport.go:217

  • name.String() can panic if the rejected value has a non-string name property (syscall/js panics when calling String() on non-TypeString). Since this code is already hardening against unusual rejection shapes, the AbortError check should also gate on name.Type() == js.TypeString before calling String().
		if len(catchArgs) > 0 && catchArgs[0].Type() == js.TypeObject {
			if name := catchArgs[0].Get("name"); !name.IsUndefined() && name.String() == "AbortError" {
				isAbort = true
			}

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants