Skip to content

Flaky unit test: TestReaderReconnector.test_reconnect_on_repeatable_error (timeout on 3.12) #850

Description

@vgvoleg

Summary

ydb/_topic_reader/topic_reader_asyncio_test.py::TestReaderReconnector::test_reconnect_on_repeatable_error fails intermittently in CI with a timeout. Recurring flake — has been seen on several unrelated PRs.

Failure

ydb._topic_common.test_helpers.WaitConditionError: Bad condition in test
ydb/_topic_common/test_helpers.py:67: WaitConditionError

The test drives ReaderReconnector through one repeatable error (Overloaded) and then a healthy stream, and waits via:

await wait_for_fast(reconnector.wait_message())   # topic_reader_asyncio_test.py:1561

wait_for_fastwait_condition budgets 1s wall / 1000 loop iterations (test_helpers.py:46-67). When reconnector.wait_message() does not resolve within that budget the helper raises WaitConditionError. So this is a timing-sensitive timeout, not an assertion about reconnect behavior.

A teardown side-effect also shows up after the failure (likely secondary, from the aborted coroutine):

PytestUnraisableExceptionWarning: Exception ignored in: <coroutine object Queue.get ...>
RuntimeError: Event loop is closed

Environment / occurrence

Hypothesis / directions to investigate

  • The reconnect path (error stream → recreate → first message) occasionally needs more than the 1s / 1000-iteration budget under CI load, especially on 3.12's event-loop scheduling. Bumping the budget would only mask it.
  • Worth checking whether ReaderReconnector does extra await asyncio.sleep(0) hops on the reconnect path on 3.12, or whether a backoff/retry delay leaks into the loop and eats the budget.
  • Confirm the Event loop is closed teardown warning is purely secondary (GC of the parked wait_forever/Queue.get coroutine) and not contributing to the hang.

Workaround for now

Re-run the job; failure is intermittent and not a regression from the triggering PR.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions