[rjb32]

Run the unit tests under valgrind memcheck on weekdays at 9:00, 12:00 and 17:00 Paris time.

[rjb32]

Note

AI-generated summary.

Summary

Goal. Run every unit test under valgrind memcheck on weekdays at 9:00, 12:00 and 17:00 Paris time, so uninitialised reads and reads of freed memory surface within hours. The ASan and UBSan nightly does not report an uninitialised read: #1461 fixed 3 that only memcheck found.

Approach. One new workflow, ci_valgrind.yml, shaped like ci_sanitizers.yml: the same runner label, dependency cache and cleanup. It builds with RELDEBUG=1, so the test binaries keep their symbols, then runs ctest -T memcheck. A test fails on any memcheck error not covered by test/valgrind_suppressions.supp.

Where to look.

• .github/workflows/ci_valgrind.yml:5 — the schedule, with the timezone key GitHub added in March 2026
• .github/workflows/ci_valgrind.yml:34 RELDEBUG — set for the whole job, because cmake reads it from the environment
• .github/workflows/ci_valgrind.yml:72 — the ctest call: the 2 excluded tests and the valgrind options

on weekdays 9:00, 12:00, 17:00 Europe/Paris
  install valgrind if the runner image lacks it
  build dependencies (cached), configure with RELDEBUG=1, ninja
  ctest -T memcheck
    -E test_query_parser_mutant      1267 s under memcheck
    -E test_query_ir_fuzz_hang       its own 120 s timeout stops it
    --error-exitcode=99              a memcheck error fails the test
    --max-threads=2048               JobSystemTest starts 1024 workers
  on failure: print each MemoryChecker log that has errors, upload all of them

Evidence

• Before: no job runs valgrind. The first run of all 1252 tests under memcheck on a dev machine found 3 tests with uninitialised bytes, fixed in IO: fix the uninitialised bytes memcheck reports in the unit tests #1461.
  After: the same ctest call, run by hand on the IO: fix the uninitialised bytes memcheck reports in the unit tests #1461 branch at -j8:

  1275 tests, 0 memcheck errors, 32 min 16 s wall, 4.0 h of test time
  largest single test process   12.9 GB RSS
  

  This workflow has not run yet. GitHub only dispatches a workflow that is on the default branch, so its first run comes after merge. The YAML parses.

Merge Danger

Door: two-way

Blast Radius: runners

Each run holds one ubuntu-24.04-turing runner for about an hour at -j8: the same tests took 32 min at -j8 on a dev machine, and a RELDEBUG rebuild took 21 min there. The job sets -j8 itself and ignores HOST_BUILD_JOBS, so ubuntu-runner-aws1 runs it at 8 too, not 16. That happens three times on each weekday, during working hours, so PR jobs on that label have one runner fewer while it runs. The RELDEBUG tree takes 49 GB of disk, against 3.4 GB for a release tree, and the cleanup step deletes it after the job. With 8 tests at once, memory is the limit to watch: one test alone reached 12.9 GB under memcheck, and the dev-machine run stayed inside a 32 GB cap. A dependency cache miss also builds LLVM at -j8.