Add opt-in session-scoped FIX socket write timeout - #22
Draft
0xAmaan wants to merge 3 commits into
Draft
Conversation
A socket write to a peer that has stopped reading blocks forever: writeLoop called connection.Write with no deadline, and on error it logged and kept looping. When both sides of a session answer each other's resend requests simultaneously (two-way sequence gap after an unclean disconnect), each side's session goroutine blocks pushing its replay into messageOut while its peer does the same — mutual TCP backpressure with no timeout anywhere. This is the deadlock class of quickfixgo/quickfix#169; the responding side was left unbounded, and it froze a production-path session for 29.5 hours on 2026-07-07/08 (peak6-labs/kalshi incident). writeLoop now takes the net.Conn (both call sites already pass one), arms a 30s write deadline before every write, and treats any write error as fatal to the connection: 1. Close the conn — readLoop's blocked Read fails and it closes msgIn. 2. Drain messageOut — a session goroutine blocked in a sendBytes channel send (e.g. mid-resendMessages) unblocks, returns to its event loop, sees the closed msgIn, and runs onDisconnect, which closes messageOut and ends the drain. The close-before-drain order is load-bearing: the drain terminates only via that disconnect path. Callers need no changes — the initiator wrapper and acceptor handleConnection already Close on their own paths (second close is logged and harmless), so a wedged session now becomes a clean disconnect + standard reconnect within seconds. Note: TestSessionFactorySuite/TestConfigureSocketConnectAddress fails on the unmodified base commit (d520d75) as well — pre-existing, untouched. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
0xAmaan
changed the title
0xAmaan
marked this pull request as draft
July 22, 2026 21:04
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Add a session-scoped, opt-in
SocketWriteTimeoutsetting to prevent a FIX connection from remaining silently blocked forever when its peer stops reading.This addresses the deadlock class seen after the emulator and fix-router reconnected with two-way sequence gaps: both session goroutines could synchronously produce replay traffic, stop consuming inbound messages, fill the TCP buffers, and leave both write loops blocked indefinitely. Because disconnect/reconnect handling runs on the blocked session goroutine, the connection could not self-heal.
Behavior and scope
SocketWriteTimeoutto a positive Go duration such as30son selected sessions.messageOut. Closing unblocks the read loop, lets the session process its disconnect path, closesmessageOut, and terminates the drain. This also unwedges a session producer blocked mid-replay.The dependent Kalshi change is peak6-labs/kalshi#1793. It opts in only the emulator acceptor's order and drop-copy sessions. Normal Kalshi venue sessions do not receive this setting.
Validation
net.Pipeblocked-write test verifies the deadline terminates the write and unblocks teardown.-race;go vetpasses.The single red CI job is a baseline repository failure:
TestSessionFactorySuite/TestConfigureSocketConnectAddressandTestConfigureSocketConnectAddressMultifail with the same assertions on unmodifiedmain(run 25184216827). No new timeout or teardown test fails.