Skip to content

test(configuration): pin save-time url and image refusal rules that survived mutation - #2377

Merged
freeqaz merged 3 commits into
mainfrom
mutant-sweep-1014
Oct 14, 2026
Merged

freeqaz merged 3 commits into
mainfrom
mutant-sweep-1014

Conversation

@freeqaz

@freeqaz freeqaz commented Oct 14, 2026 •

Copy link
Copy Markdown
Collaborator

Problem

The save-time refusals that landed on 10-13 (#2351, #2353, #2359, #2371, #2362, #2363, #2372) were proven live, but nobody had checked that their tests would catch a regression. A mutation sweep of 193 mutants over those rules found 57 that no test killed. Examples:

  • dropping the Codex socksUrl, Discord voice TTS, voice-call TTS or Talk realtime proxy checks;
  • dropping ftp or ws(s) as special schemes;
  • making the URL work budget per field instead of per document;
  • not budgeting GIT_CONFIG_PARAMETERS proxies (a 60 KB value then takes 12 s);
  • splitting 'key=value' at the last =;
  • dropping the sha384 digest or the localhost registry rule.

Change

This change adds tests only, in two files. Each new row fails on its mutant and passes on main.

  • tests/integration/configuration-controller.test.mjs
    • URL userinfo: the four missing proxy fields; scheme-less proxies with a backslash or double quote in the user name; http:, wss: and ftp: with no slashes; a Preset variable scheme with no slashes; a URL that starts inside a reference's fallback. Two accepts: an escaped $${NAME}, and a Windows path in a file: fallback.
    • git config proxies: a password that ends in =; '\'' quoting; a reference as the key; a user-only proxy without a scheme; a Preset key http.https://{{ vars.url }}/.proxy; an accepted escaped key.
    • Work budget: seven proxies that each fit the budget but not together; a Preset URL with 2000 variables; a scheme-less HTTPS_PROXY of 415 fallbacks, charged for both readings; a GIT_CONFIG_PARAMETERS proxy of fallbacks, which took 12 s unbudgeted (each refusal under 500 ms); an accepted URL of escaped fallbacks, which cost nothing.
    • baseUrl with references: tabs, leading spaces and scheme case; backslashes as slashes; one row for each rendering (all set, all fallbacks, the first fallback alone).
    • MCP headers: an accepted Authorization-V2: Bearer ${NAME}.
  • tests/fixtures/image-reference-cases.mjs (used by preflight, Compute, the chart and both image tools)
    • SHA-384 digest; an empty registry port; a 255-character path after localhost; a 256-character index.docker.io path with library/; a double hyphen after a registry port. The chart's remote-name regex can read a plain dotted host as a path component, which hides its first-component rule.

Verification

  • Kill table: before, the suites killed 136 of 193 mutants; after, 183.
    • Every one of the 47 newly killed mutants is killed by a new row.
    • The owning suites were configuration-controller, kubernetes-configuration, presets, profile-preflight-chart-parity (with helm template), bootstrap-volume, production-image-upgrade, kubernetes-compute, configuration-startup, logging-packaging and production-kubernetes-packaging.
  • 10 survivors, not pinned:
    • Equivalent (4):
      • the namesGitProxy length guard, which is dead code (a key that starts with a section and ends in .proxy is always long enough);
      • the slot index of a name with a variable, which never matches the slot pattern;
      • > vs >= at the budget;
      • the uncharged linear scheme scan.
    • Changes verdicts only where a variable supplies the scheme, password or @ (1): the covered-candidate skip. A differential fuzz found 1 to 3 differences per 90k cases, in the obfuscation class that fix(presets): refuse url userinfo in every env map and any env value with a scheme #2353 left on purpose.
    • Over-refusal only (4):
      • backslashes read as slashes after a non-special scheme;
      • the authority's end delimiter, which a quote turns into port text;
      • the NO_PROXY exemption, which only matters for an @ in NO_PROXY;
      • lowercase git_config_* names, which git ignores.
    • For the over-refusal mutants, the differential fuzz found no credential that a mutant misses and main catches.
  • Pinned runs: 20/20 of each touched suite, pinned with taskset -c 20,21: configuration-controller (3 s; rerun 20/20 after merging main), profile-preflight-chart-parity (31 s, same as main), bootstrap-volume and production-image-upgrade.
  • Review: an independent review checked the five image verdicts against distribution/reference v0.6.0 and go-digest. It confirmed with curl 8.22 that the backslash, quote and = proxy rows are userinfo to a real client. Its nits are applied.
  • Merged main: after merging fix(presets): say the // is missing when a baseUrl scheme lacks it #2375 (the missing-// message from finding 1285), two new rows expect that message: HTTPS:${MODEL_HOST}/v1 and https:${MODEL_SLASHES:-//}…. All 18 baseUrl mutants were rerun and are killed.
  • Lint: eslint and prettier --check are clean on both files. Prettier was first shown to fail on a badly formatted file.

Risks

This is test-only. The budget rows sit between the thresholds on main and on the mutant:

  • HTTPS_PROXY: main refuses from 385 fallbacks and the mutant from 445; the row uses 415.
  • Seven proxies: each costs 641,451, six cost 3.85M and seven 4.49M against the 4,194,304 budget (about 7% margin).
  • Preset variables: main refuses from 142 fallbacks and the mutant from 537; the row uses 300.

So retuning the budget constant would need these rows updated, as the existing budget rows would.

…urvived mutation

A mutation sweep of the 10-13 save-time refusals (#2351, #2353, #2359, #2371,
#2362, #2363, #2372) found 57 of 193 mutants that no test killed. New rows kill
49 of them; the other 8 are equivalent or change only refusals of values no
client reads as a credential.

- URL userinfo: Codex socksUrl, Discord voice, voice-call and Talk realtime
  provider proxies; scheme-less proxies with a backslash or double quote in the
  user; http, wss and ftp without slashes; a variable special scheme; a URL in
  a reference's fallback; escaped references and Windows paths in fallbacks.
- git config: lowercase variable names, first-= split, quoted quotes, a
  reference key, a user-only proxy, and a URL subsection around a variable.
- Work budget: shared across URL fields, Preset variables charged, scheme-less
  proxies charged for both readings, git -c proxies budgeted (12 s unbudgeted).
- baseUrl slash rule with references: tabs, leading spaces, scheme case,
  backslashes, and each of the three renderings.
- Image references: SHA-384, empty registry port, localhost and index.docker.io
  path lengths, and a double hyphen after a registry port.
Drop the lowercase git_config_* rows (git reads only the uppercase names, so
they pinned an over-refusal), give the escaped-fallback accept its own
subtest, move the 12 s note to the git config row, and reword the fixture's
registry-port comment. The MCP Authorization-V2 accept row (first commit) was
missing from its message.
@freeqaz
freeqaz requested a review from a team as a code owner October 14, 2026 01:58
@clawsweeper

clawsweeper Bot commented Oct 14, 2026 •

Copy link
Copy Markdown
Contributor

🦞👀
ClawSweeper picked this up.

Pull request received. I will update this pull request when review starts.

ClawSweeper review complete

ClawSweeper finished reviewing this revision. The review result is being finalized.

View the workflow run.

# Conflicts:
#	tests/integration/configuration-controller.test.mjs
@clawsweeper

clawsweeper Bot commented Oct 14, 2026

Copy link
Copy Markdown
Contributor

ClawSweeper status: review started.

I am starting a fresh review of this pull request: test(configuration): pin save-time url and image refusal rules that survived mutation This is item 1/1 in the current shard. Shard 0/1.

This temporary status tracks the active review worker. The completed review will appear in the durable ClawSweeper review comment.

Crustacean status: shell secured, claws on keyboard, evidence pebbles being sorted.

@freeqaz
freeqaz merged commit c28bd23 into main Oct 14, 2026
43 of 45 checks passed
@freeqaz
freeqaz deleted the mutant-sweep-1014 branch October 14, 2026 02:05

This branch had an error being deployed

1 failed deployment
integration-qa-pr — 1567e772 Deployed Oct 14, 2026 by freeqaz via QA advisory (compose) #767
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant