From 03cd07ab6ed4a62e6d127fa505db681915473880 Mon Sep 17 00:00:00 2001 From: CodeWhale Bot Date: Tue, 15 Sep 2026 22:54:04 -0700 Subject: [PATCH] fix(ci): bound telemetry_kill_switch_dispatch so it stops flaking on Ubuntu MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit `codewhale-cli::telemetry_kill_switch_dispatch::missing_preference_defaults_on_without_inventing_acceptance` has failed twice on Ubuntu CI at `telemetry_kill_switch_dispatch.rs:180` ("default-on writes dry run") on changes that touch neither telemetry nor the CLI: PR #6104 (`c810bc0458`, 2026-09-12) and PR #6267 (`75e032b61d`, 2026-09-16). Both times macOS and Windows passed. It is not an env race. CI runs cargo-nextest, which gives every test its own process, and the test never mutates process-global state — each case spawns the real binary with `env_clear()` and a fresh `TempDir` HOME/CODEWHALE_HOME (`telemetry_kill_switch_dispatch.rs:231-276`). The panic is a *missing file*: `$CODEWHALE_HOME/telemetry/dryrun.jsonl` was never written. The mechanism is a wall-clock deadline. `features list` resolves to `Surface::Cli`, whose exit path waits `CLI_PERSIST_TIMEOUT` — 250 ms (`crates/telemetry/src/lib.rs:75`) — for a detached writer thread that must re-run `decision::re_decide` against disk and then fsync an append before the process exits. The code deliberately fails open, so a missed deadline silently produces no receipt. All three existing test-group overrides filter `binary(integration)`, and this is a *different* binary, so none of them ever matched it: the five cases ran at full parallelism beside 15,774 tests. The CI log shows the runner was saturated at that moment — neighbouring subprocess-spawning tests took 3.5-3.6 s for work that normally finishes well under a second, while this one failed in 0.747 s. This adds the missing override, putting the binary in the existing `telemetry-contract` group (max-threads = 1) for the same reason that group exists: these tests spawn the real binary and cannot absorb scheduler latency. Verified on this machine (macOS aarch64): cargo nextest show-config test-groups -p codewhale-cli --all-features \ --locked -E 'binary(telemetry_kill_switch_dispatch)' group: telemetry-contract (max threads = 1) * override for default profile with filter 'binary(telemetry_kill_switch_dispatch)': codewhale-cli::telemetry_kill_switch_dispatch: (all 5 tests) group: spawns-binaries (max threads = 3) (no matches) sh scripts/with-hermetic-test-home.sh cargo nextest run -p codewhale-cli \ --all-features --locked -E 'binary(telemetry_kill_switch_dispatch)' Summary [0.621s] 5 tests run: 5 passed, 0 skipped Not fixed here: the 250 ms budget itself. Bounding the group removes the load that makes the deadline reachable, but a slow enough host can still miss it. The product-side options — raise CLI_PERSIST_TIMEOUT, hoist the `re_decide` disk read out of the deadline window, or have the CLI path join the writer for the local non-network case — are a behavioural decision, not a test fix, and are left for the issue. Signed-off-by: CodeWhale Bot Co-Authored-By: Claude Opus 5 (1M context) --- .config/nextest.toml | 14 ++++++++++++++ 1 file changed, 14 insertions(+) diff --git a/.config/nextest.toml b/.config/nextest.toml index 174b6319a8..070069fff8 100644 --- a/.config/nextest.toml +++ b/.config/nextest.toml @@ -42,6 +42,20 @@ exec-persistent-service = { max-threads = 1 } filter = 'binary(integration) & test(/^telemetry_contract::/)' test-group = 'telemetry-contract' +# Same hazard, different binary. `telemetry_kill_switch_dispatch` is a +# codewhale-cli integration binary, not `integration`, so none of the filters +# below ever matched it and it ran at full parallelism beside ~15.7k tests. +# Each of its cases spawns the real binary, which gives a detached writer +# thread only CLI_PERSIST_TIMEOUT (250 ms, crates/telemetry/src/lib.rs) to +# re-read config from disk and fsync its batch before the process exits. Under +# Ubuntu CI load that deadline is missed and the dry-run receipt never lands — +# observed at telemetry_kill_switch_dispatch.rs:180 on PR #6104 (2026-09-12) +# and again on PR #6267 (2026-09-16), both Ubuntu-only, both on changes that +# touch neither telemetry nor the CLI. +[[profile.default.overrides]] +filter = 'binary(telemetry_kill_switch_dispatch)' +test-group = 'telemetry-contract' + [[profile.default.overrides]] filter = 'binary(integration) & test(/^exec_persistent_service::/)' test-group = 'exec-persistent-service'