Flaky CI Failure: test_message_streaming_when_removed_then_added — amal receives 3 messages, expected 2
Workflow: Test
Failed run: https://github.com/xmtp/libxmtp/actions/runs/29768888526
Commit: 3e26e0f
Failed jobs: test-workspace / Test (Rust Workspace)
Summary
The xmtpv3::mls::tests::streaming::test_message_streaming_when_removed_then_added test failed all 4 retries with the same assertion error. Amal's stream callback received 3 messages when only 2 were expected (hello1 and hello2 sends). This caused the test run to finish with 2444/2445 passed, 1 failed. A second test (xmtp_mls::client::tests::should_reconnect) was flaky but passed on retry 2/4.
Error Details
thread 'mls::tests::streaming::test_message_streaming_when_removed_then_added' (63532) panicked at bindings/mobile/src/mls/tests/streaming.rs:293:5:
assertion `left == right` failed
left: 3
right: 2
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
Nextest summary:
Summary [ 38.637s] 2445 tests run: 2444 passed (1 flaky), 1 failed, 17 skipped
FLAKY 2/4 [ 1.001s] xmtp_mls client::tests::should_reconnect
TRY 4 FAIL [ 0.147s] xmtpv3 mls::tests::streaming::test_message_streaming_when_removed_then_added
Failed on all 4 tries with left: 3, right: 2 every time.
Analysis
The failing assertion is at bindings/mobile/src/mls/tests/streaming.rs:293:
assert_eq!(amal_stream_callback.message_count(), 2);
The test sends two messages (hello1, hello2) and expects both stream callbacks to report count=2. Amal's callback consistently shows 3 instead of 2. The test already attempts to guard against this by syncing both clients to drain the add-member transcript message before starting streams (lines 252–261), but a third message still leaks through after the stream is open. Possible causes: a system/protocol message (e.g. group membership update) is arriving after the sync but before or alongside the user messages, or the wait_for_delivery mechanism is counting a message the test doesn't account for. This is a test/code failure, not infrastructure.
Reported by Flaky Failure Watcher
Flaky CI Failure: test_message_streaming_when_removed_then_added — amal receives 3 messages, expected 2
Workflow: Test
Failed run: https://github.com/xmtp/libxmtp/actions/runs/29768888526
Commit: 3e26e0f
Failed jobs: test-workspace / Test (Rust Workspace)
Summary
The
xmtpv3::mls::tests::streaming::test_message_streaming_when_removed_then_addedtest failed all 4 retries with the same assertion error. Amal's stream callback received 3 messages when only 2 were expected (hello1 and hello2 sends). This caused the test run to finish with 2444/2445 passed, 1 failed. A second test (xmtp_mls::client::tests::should_reconnect) was flaky but passed on retry 2/4.Error Details
Nextest summary:
Failed on all 4 tries with
left: 3, right: 2every time.Analysis
The failing assertion is at
bindings/mobile/src/mls/tests/streaming.rs:293:The test sends two messages (
hello1,hello2) and expects both stream callbacks to report count=2. Amal's callback consistently shows 3 instead of 2. The test already attempts to guard against this by syncing both clients to drain the add-member transcript message before starting streams (lines 252–261), but a third message still leaks through after the stream is open. Possible causes: a system/protocol message (e.g. group membership update) is arriving after the sync but before or alongside the user messages, or thewait_for_deliverymechanism is counting a message the test doesn't account for. This is a test/code failure, not infrastructure.Reported by Flaky Failure Watcher