Skip to content

test(server): data race between startBackgroundIndexBuilds and PacketStore.Load under -race (TestHashMigrate_LogsMaxWriteLockHold_215) #301

Description

@dborup

Problem

cd cmd/server && go test . -race -count=1 intermittently fails in TestHashMigrate_LogsMaxWriteLockHold_215 with a data race between startBackgroundIndexBuilds (index_ready_1008.go) and PacketStore.Load (store.go). The test reaches it through the hash-migrate test harness.

It was seen during the local review of #294, on master 4f1de049 plus #294, which touches neither file. CI's Go Build & Test was green on the same head, so the race is intermittent.

This is not #267: that was the wall-clock write-hold budget of the backfill test.

Task

  1. Reproduce: go test -race -count=50 -run TestHashMigrate_LogsMaxWriteLockHold_215 . in cmd/server, under parallel CPU load if needed. Capture the full race report: both goroutine stacks and the shared variable.
  2. Find out whether the race is in production code, for example background index builds reading store state that Load is still writing without the lock, or only in the test harness, for example the harness starting index builds before Load returns.
  3. Fix the cause:
    • Production race: fix it with the correct lock or ordering, with no expensive work held under the lock. Add a regression test that is red under -race before the fix.
    • Harness only: fix the harness and explain why production cannot hit the same path.
  4. Show 100 consecutive -race runs of the test green after the fix.

Acceptance

  • The cause is named with the race report as evidence.
  • go test -race -count=100 -run TestHashMigrate_LogsMaxWriteLockHold_215 passes.
  • The full cmd/server go test -race ./... is green.
  • No new map[string]interface{}, and cmd/server stays read-only.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't workingtype:bugSomething broken

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions