Skip to content

test(bench): make the coarse-clock tie case deterministic - #7514

Closed
llb555 wants to merge 1 commit into
bytedance:mainfrom
llb555:test/prewarm-clock-tie-deterministic
Closed

llb555 wants to merge 1 commit into
bytedance:mainfrom
llb555:test/prewarm-clock-tie-deterministic

Conversation

@llb555

@llb555 llb555 commented Oct 25, 2026

Copy link
Copy Markdown
Contributor

test_bench_sandbox_prewarm.py::test_prewarm_reused_survives_a_coarse_clock_tie fails at random on busy machines, including in CI — it just took down the Unit Tests shard 4 run for #7505, and it reproduced 2 of 3 local runs on a tree that did not touch it.

The cause is the setup, not the assertion. The test quantizes time.monotonic() to 15 ms and then spins until a fresh bucket starts, relying on the whole overlap scenario — several awaits, a scheduled task, and two events — finishing inside that remaining tick. When the machine is slow, the tool request lands in the next bucket and the precondition assert fires before the actual regression is even exercised.

This replaces the race with a frozen clock:

    # Freeze the quantized clock instead of racing a real bucket: the scenario
    # between the acquire and the tool request takes several awaits, so waiting
    # for a fresh tick and hoping both land inside it made this test fail
    # whenever the machine was busy. A constant reading still reproduces #7044
    # exactly (acquire and tool request share one timestamp) and cannot flake.
    frozen_tick = math.floor(real_monotonic() / coarse_s) * coarse_s

    def coarse_monotonic():
        return frozen_tick

The test still pins #7044: with the original strict timestamp comparison restored in bench_prewarm.py, it fails (assert False is True on prewarm_reused), because equal timestamps never satisfy <.

Verified: pytest tests/test_bench_sandbox_prewarm.py → 19 passed, five consecutive runs (previously 1 of 3 passed on the same machine); ruff check and ruff format --check clean.

@github-actions github-actions Bot added risk:medium Medium risk: regular code changes size/XS PR changes < 20 lines labels Oct 25, 2026
@WillemJiang

Copy link
Copy Markdown
Collaborator

The issue has been fixed by #7513

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

risk:medium Medium risk: regular code changes size/XS PR changes < 20 lines

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants