Reconnect on live (tier1) drift via session linear-handoff block

[YaroShkvorets]

Problem

--live-drift-reconnect gated the drift check on the sink library's DeltaLivenessChecker isLive flag, which only latches once a block lands within --live-block-time-delta of wall-clock. An endpoint that is perpetually lagging — or stalls before ever reaching the live edge — never arms the trigger, so the sink drifts indefinitely without reconnecting (observed: live:true, drift well past the configured 15m, no reconnect).

Note the live:true in the stream-stats log comes from runningFromTier1, which is a different signal than the isLive flag that gated the check — hence the confusion.

Fix

Gate the reconnect on whether we're in the live (tier1) linear-streaming segment, detected via the per-session LinearHandoffBlock (from the Response_Session protobuf), newly exposed by substreams-sink-go v0.5.11:

• a processed block >= LinearHandoffBlock is live tier1 output → arm the drift check;
• blocks below it are replayed from tier2 backprocessing → don't reconnect.

This fires precisely when streaming live and behind (triggering a reconnect → fresh tier2 parallel backfill to close the gap), and avoids reconnect loops during the initial backfill or a reconnect's own catch-up, because the Session message refreshes the handoff (≈ new head) before any data flows — so replayed blocks fall below it. Using the session protobuf rather than the ProgressMessageLastContiguousBlock Prometheus gauge also avoids stale-across-reconnect state.

Behavior change

The flag default is flipped from 1h to empty/disabled (opt-in). An unsupplied --live-drift-reconnect now performs no automatic reconnect — same as before the feature. Enable with e.g. --live-drift-reconnect=15m.

Dependency

Requires substreams-sink-go v0.5.11 (replace bumped, go.sum updated). That release adds Sinker.LinearHandoffBlock().

🤖 Generated with Claude Code

[Copilot]

Pull request overview

This PR fixes cases where --live-drift-reconnect never triggers on perpetually-lagging endpoints by arming the drift check based on whether the stream is in the live (tier1) linear streaming segment, using the session’s LinearHandoffBlock signal exposed by substreams-sink-go v0.5.11. It also makes the reconnect behavior opt-in by defaulting the flag to disabled.

Changes:

• Arm --live-drift-reconnect when block >= LinearHandoffBlock (tier1 live segment) instead of relying on the sink’s DeltaLivenessChecker isLive latch.
• Flip --live-drift-reconnect default from 1h to disabled (empty string), making the feature opt-in.
• Bump github.com/pinax-network/substreams-sink-go replacement to v0.5.11 and update module sums accordingly.

Reviewed changes

Copilot reviewed 3 out of 4 changed files in this pull request and generated 1 comment.

File	Description
sinker/sinker.go	Switches drift-reconnect arming to tier1 detection via LinearHandoffBlock and adds helper runningFromTier1.
cmd/substreams-sink-sql/run.go	Changes --live-drift-reconnect default to disabled and updates flag help text.
go.mod	Updates the replace for substreams-sink-go to v0.5.11 (and adjusts dependency classification).
go.sum	Updates checksums for the substreams-sink-go version bump.

Comments suppressed due to low confidence (1)

sinker/sinker.go:138

• The reconnect error message still says "live mode" but the trigger condition is now specifically "streaming live (tier1)" via LinearHandoffBlock. Updating the wording would make logs/errors less confusing, especially since this PR explicitly distinguishes between different "live" signals.

	if s.liveDriftReconnectDuration > 0 && s.runningFromTier1(data.Clock.Number) {
		blockTime := data.Clock.GetTimestamp().AsTime()
		drift := time.Since(blockTime)
		if drift > s.liveDriftReconnectDuration {
			return fmt.Errorf("live mode drift exceeded threshold: block timestamp is %s behind current time (threshold: %s), triggering reconnect for backfilling", drift, s.liveDriftReconnectDuration)
		}

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.