Skip to content

Owners can still hear about a lost lock session about 5 s late when lock calls are queued #83

Closed

Description

@vheathen

Follow-up to #42.

I re-ran this on a2ce825. With a busy lock server and a silent lock connection, owners now hear after about 16 s instead of 35 s (3 of 3), and the idle case stays at about 17 s, which matches what guides/Cluster.md says about idle_interval and ping_timeout.

Severity: low: for about 5 s the owner gets no lock_released notice even though its lock is gone. Likelihood: low: it needs reconnects that hang during a failover, calls queued together (such as claim retries of subscriptions started at the same time), and those calls reaching the dead session before the pool's ping does.

One ordering still delays the notice by about 5 s: the server ends the lock session (a terminated backend, a restart or a failover) while two or more lock calls are queued and new connections hang. The first call fails and returns before the connection process sends session_lost, so handle_pending_session_loss/1 finds nothing when the second call comes up, and that call blocks in checkout until the pool drops it.

How I measured it

After terminating the lock backend and blackholing new connections, I sent four lock calls back to back without suspending any process. Owners got lock_released 4787 to 5796 ms after the calls in 18 of 18 recorded iterations, right after the second call's checkout ended; with the database accepting connections, 4 iterations took 6 to 13 ms. With the connection process suspended to pin that order, the lock server sat in DBConnection.Holder.checkout_call with {AdvisoryLocks, :session_lost, conn} unhandled in its mailbox for about 5.5 s (3 of 3), and with a verification timer firing ahead of the notice, owners heard after about 4.7 s.

Activity

  1. yordis commented on Oct 28, 2026

    @yordis
    Member

    Fixed in #87: the lock server now waits for the lost-session notice before running the next queued call, so owners hear about it without waiting on the pool.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions