Skip to content

tests: nextest fails without a UTF-8 locale, and is flaky on macOS #136

Description

@Stiven-Gjekaj

cargo nextest run --workspace is one of the four commands CONTRIBUTING.md requires before a pull request. It does not pass here. One run reported 25 failures of 1634 tests, 15 in mandible-tui and 10 in mandible-extract.

The two causes are independent. Split this issue if you want one each.

Environment: macOS 26.6.2, arm64, 10 logical cores, rustc 1.97.1, cargo-nextest 0.9.143, at 0.7.0 (4c607d2). The other three gates pass. cargo fmt --all -- --check is clean, cargo clippy --workspace --all-targets -- -D warnings is clean, and cargo run -p xtask -- corpus reports 149 fixtures, 132 ok, 17 xfail as expected, 0 failed.

Cause 1: 15 tests read the ambient locale

mandible-tui/src/app.rs:452 builds the App with glyphs: crate::glyphs::from_env(). mandible-tui/src/glyphs.rs:112 returns ASCII unless LC_ALL, LC_CTYPE or LANG names UTF-8. A non-login shell exports none of the three.

The tests then assert a rounded border against an ASCII one.

thread 'borders_survive_help_overlay' panicked at mandible-tui/tests/border_integrity.rs:136:9:
assertion `left == right` failed: corner at (0,0) of rect Rect { x: 0, y: 0, width: 80, height: 3 } expected '╭', got "+"
  left: "+"
 right: "╭"

The failure is deterministic and the count is exact.

$ env -u LANG -u LC_ALL -u LC_CTYPE cargo nextest run -p mandible-tui --no-fail-fast
Summary [0.557s] 294 tests run: 279 passed, 15 failed, 0 skipped

$ LANG=en_US.UTF-8 cargo nextest run -p mandible-tui --no-fail-fast
Summary [0.593s] 294 tests run: 294 passed, 0 skipped

The 15 are one test in render::detail_pane::tests, eight in border_integrity, three in detail_sections, and three in unverified_notice_wrap.

This is not a macOS defect. The same 15 fail on any host whose shell exports no UTF-8 locale.

The file already holds the fix pattern. border_integrity.rs:622 sets app.glyphs = ASCII before it asserts ASCII borders, and border_integrity.rs:690 sets app.glyphs = UNICODE. The 15 set neither, so they read the environment instead of stating what they test.

I can send this fix if you want it.

Cause 2: the shim-spawning tests are flaky under parallelism

These fail nondeterministically. Four workspace runs with the locale set gave 2, 5, 10 and 8 failures. The set differs each run. Every member comes from the tests that spawn a real shim: exec::spawn::tests, exec_policy, completion_script_gate, native::tests::detect_sends_the_literal_dunder_complete_word_in_argv, and framework::tests::identify_help_text_signature_falls_back_when_artifact_scan_finds_nothing.

Every failure lands at the test's own 2 second timeout. The assertion reads as empty output.

thread 'exec::spawn::tests::stdin_is_null_child_sees_immediate_eof' panicked at mandible-extract/src/exec/spawn.rs:773:9:
assertion `left == right` failed
  left: ""
 right: "EOF"

Serially the crate is clean, three runs of three.

$ LANG=en_US.UTF-8 cargo nextest run -p mandible-extract --no-fail-fast --test-threads 1
Summary [18.245s] 621 tests run: 621 passed, 0 skipped
Summary [18.523s] 621 tests run: 621 passed, 0 skipped
Summary [18.932s] 621 tests run: 621 passed, 0 skipped

The cause is a macOS cost on the first execution of a newly written file. Measured on one script from one process:

per exec
first exec 225.9 ms
next 20 execs of the same file 3.6 ms

The scan also serializes across processes. Twelve first-execs of twelve new scripts took 4.799s one at a time and 3.433s in parallel on 10 logical cores.

Each of these tests writes a fresh shim through write_shim, runs it once, and allows Duration::from_secs(2). One execution spends about 225 ms of that budget before the script starts. Under the default thread count the scans queue and some tests pass 2 seconds.

I am not proposing a fix here. A larger timeout and a nextest profile that caps the thread count both work, and the choice is yours.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions