Skip to content

guides/Cluster.md: client_connection_check_interval does not release a dead node's locks #84

Closed

Description

@vheathen

Follow-up to #42.

Severity: high when it happens: until the server drops the dead node's session, no other node can take that node's subscriptions, so if the node is down their events wait; the Linux defaults (7200 s before the first probe, then 9 probes 75 s apart) work out to about 2 h 11 min, and I saw the locks still held after 13 min. Likelihood: medium: a node's host dies or the network partitions without FIN or RST.

The last paragraph of "How long a node takes to learn its lock session ended" in guides/Cluster.md says: "PostgreSQL only releases the locks of a session once it notices the client is gone, which can take longer still unless tcp_keepalives_idle or client_connection_check_interval is configured on the server." In my runs only tcp_keepalives_idle, tcp_keepalives_interval and tcp_keepalives_count set together released the locks quickly (15 to 25 s); tcp_keepalives_idle alone took 769 s, and client_connection_check_interval had not released them after 13 min.

How I measured it

I tried both on PostgreSQL 16.14 on Linux, with clients holding pg_advisory_lock cut off without FIN or RST. With tcp_keepalives_idle=10, tcp_keepalives_interval=5 and tcp_keepalives_count=3, the locks were released 15 to 25 s after the cut (3 of 3). With client_connection_check_interval=1s they were still held at 70 s in all three runs and at 13 min in the long one, both for an idle session and for one running pg_sleep. tcp_keepalives_idle=10 alone released them after 769 s (one run), and with defaults they were still held at 13 min.

Activity

  1. yordis commented on Oct 28, 2026

    @yordis
    Member

    Fixed in #87: guides/Cluster.md now says to set tcp_keepalives_idle, tcp_keepalives_interval and tcp_keepalives_count together, and explains why tcp_keepalives_idle alone and client_connection_check_interval do not release the locks.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions