Skip to content

fix(signing): refuse rounds an earlier run can have signed after a restart - #1098

Merged
Shitikyan merged 4 commits into
stabilisationfrom
fix/signing-restore-floor
Oct 30, 2026
Merged

Shitikyan merged 4 commits into
stabilisationfrom
fix/signing-restore-floor

Conversation

@Shitikyan

@Shitikyan Shitikyan commented Oct 30, 2026 •

Copy link
Copy Markdown
Contributor

Problem

After a restart, a validator could sign a second block in a round an earlier run of the same key had already signed. This could happen when:

  • its signing state files had been restored from an older backup;
  • .prevote.json was lost beside a surviving .signed.json;
  • the commit lock file was older than the signing record.

Change

  • Restart signing floor. Once the network clock is calibrated, the node refuses every round up to the network time plus the 5 s lead. If the signing record, commit lock or prevote record holds a later round, the floor is raised to it.
    • Signing is held from the start of main() until the floor is set, because the RPC server can start a round before calibration.
  • Lock versus record. When the signing record outranks the commit lock, the gates judge by the record (recordOutranksLock).
  • Boot clock. The floor is derived once, so boot now needs two NTP readings in a row that agree within 30 s.
    • Implausible answers (an unsynchronised server answers 1900) are rejected.
    • With no trusted clock the node does not start.
  • Runbook. docs/runbooks/validator-key.md explains the floor, its limits and the clock requirement.

Evidence

  • scripts/test-consensus.sh: 69 files, 0 failed.
  • New tests cover a restored older backup, a lost prevote record, the hold before calibration, a NaN clock reading and boot calibration (1900 answers, disagreeing readings, no server).
  • Devnet: race-strict, race and control scenarios pass.

Refs kynesyslabs/mainnet-launch#2

Summary by CodeRabbit

  • Reliability

    • After restart, signing remains paused until the network clock is calibrated. The node then refuses to sign rounds at or below a safety threshold based on the calibrated time and recorded signing state.
    • Clock calibration now requires two consecutive readings within 30 seconds. Invalid or unsynchronized readings are rejected, and startup stops if no trusted readings are available.
    • Signing records now take precedence over outdated or conflicting lock files when determining whether a round can proceed.
  • Documentation

    • Added guidance on restart signing safeguards and scenarios they may not cover.

…ter a restart

The signing gates judged only the state files on disk. An older but
consistent copy of the signing record, commit lock and prevote record, or
a prevote record lost while the signing record survived, let a restarted
node sign the same height again in a round it had already signed in.

At boot, before the clock is calibrated, every signature is held. Once
it is, the node sets an in-memory floor at network time plus the maximum
round lead, raised to the highest round found in the state files, and
the block signature, prevote and precommit gates refuse any round at or
below it. The runbook documents the floor and what it does not cover.
…k file

A commit lock at (H, X, round p) beside a signing record at (H, Y, q > p),
or for another block in the lock's round, is an older copy of the lock:
every precommit is signed after its lock is written. Candidate selection
still used that lock, so the node could prevote X or a block with a
proof of lock from a round below q, and the precommit path judged the
lock rule against it.

lockAt no longer reports a lock the record outranks, so candidate
selection and getLock take the record-bound path, and the precommit
path judges by the record and rewrites the lock. The comment on the
write order of lock and record now matches the code.
…e restart floor

The restart signing floor is derived once from the boot clock. A single
unsynchronised NTP answer (1900) or a stray reading could reopen rounds an
earlier run signed, or block signing until the next restart. Boot now needs
two readings that agree within 30 s, rejects implausible times, and stops
when no trusted clock is found.

@claude claude Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Claude Code Review

This repository is configured for manual code reviews. Comment @claude review for a one-time review, or @claude review always to subscribe this PR to a review on every future push.

Tip: disable this comment in your organization's Code Review settings.

@qodo-code-review

Copy link
Copy Markdown
Contributor

ⓘ Qodo reviews are paused because the subscription is no longer active. Ask your workspace admin to reactivate the subscription to resume reviews. Manage billing

@coderabbitai

coderabbitai Bot commented Oct 30, 2026 •

Copy link
Copy Markdown
Contributor

Review in Change Stack →

Caution

Review failed

The pull request is closed.

No actionable comments were generated in the recent review.

Recent review info
Run configuration
  • Source mode: Auto
  • Repository: Unavailable
  • Branch: Unavailable
  • File: No configuration file from this repository applied
  • Commit: Unavailable
  • Layers: Organization UI
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: 931dd4c9-88fd-48ca-b4f3-0f9ea171384f
Commits

Reviewing files that changed from the base of the PR and between aba1089 and 3a8f9a5.

Files selected for processing: 15
  • docs/runbooks/validator-key.md
  • scripts/test-consensus.sh
  • src/index.ts
  • src/libs/consensus/v2/commitLock/candidate.test.ts
  • src/libs/consensus/v2/commitLock/candidate.ts
  • src/libs/consensus/v2/commitLock/lockStore.test.ts
  • src/libs/consensus/v2/commitLock/lockStore.ts
  • src/libs/consensus/v2/commitLock/precommit.test.ts
  • src/libs/consensus/v2/commitLock/precommit.ts
  • src/libs/consensus/v2/commitLock/prevoteRecord.ts
  • src/libs/consensus/v2/commitLock/restartFloor.test.ts
  • src/libs/consensus/v2/commitLock/restartFloor.ts
  • src/libs/identity/signingRecord.ts
  • src/libs/utils/calibrateTime.boot.test.ts
  • src/libs/utils/calibrateTime.ts

Included review availability: This review used your included allowance. Your plan provides up to 10 included reviews per hour; 8 remain after this review.

Walkthrough

The node now requires trusted boot-time clock readings and establishes a restart signing floor from calibrated network time and persisted signing state. Consensus signing and prevote paths enforce the floor. Lock recovery also checks whether the signing record outranks the stored commit lock.

Changes

Restart signing protection

Layer / File(s) Summary
Trusted boot clock calibration
src/libs/utils/calibrateTime.ts, src/libs/utils/calibrateTime.boot.test.ts, src/index.ts, scripts/test-consensus.sh, docs/runbooks/validator-key.md
Boot calibration requires agreeing NTP readings and rejects non-finite or implausibly old timestamps. Startup uses the boot calibration function, and the consensus test script includes its tests.
Restart floor calculation and enforcement
src/libs/identity/signingRecord.ts, src/libs/consensus/v2/commitLock/restartFloor.ts, src/libs/consensus/v2/commitLock/restartFloor.test.ts, src/libs/consensus/v2/commitLock/prevoteRecord.ts, src/index.ts, docs/runbooks/validator-key.md
Startup holds signing until the floor is set from calibrated time and the highest available recorded round. Signing and prevote paths reject rounds at or below the floor. The runbook documents the floor, clock requirements, and stated limits.
Signing-record-aware lock recovery
src/libs/consensus/v2/commitLock/lockStore.ts, src/libs/consensus/v2/commitLock/precommit.ts, src/libs/consensus/v2/commitLock/candidate.ts, src/libs/consensus/v2/commitLock/*test.ts
lockAt disregards a lock when a matching signing record outranks it. Precommit restoration and candidate tests cover older or conflicting lock state.

Estimated code review effort: 3 (Moderate) | ~25 minutes

Change: Bug fix

Sequence Diagram(s)

sequenceDiagram
  participant main
  participant getBootTimestampCorrection
  participant NTPServers
  participant armRestartSigningFloor
  participant SigningStateFiles
  participant signingRecord.evaluate
  participant recordPrevoteRound
  main->>main: holdSigningUntilRestartFloor
  main->>getBootTimestampCorrection: request trusted clock correction
  getBootTimestampCorrection->>NTPServers: request paired readings
  NTPServers-->>getBootTimestampCorrection: return readings
  getBootTimestampCorrection-->>main: return correction
  main->>armRestartSigningFloor: pass identity file and network timestamp
  armRestartSigningFloor->>SigningStateFiles: read recorded rounds
  SigningStateFiles-->>armRestartSigningFloor: return available state
  armRestartSigningFloor->>main: set and return restart floor
  main->>signingRecord.evaluate: evaluate signing request
  signingRecord.evaluate-->>main: refuse rounds at or below floor
  main->>recordPrevoteRound: record prevote request
  recordPrevoteRound-->>main: refuse rounds at or below floor
Loading

Possibly related PRs

  • kynesyslabs/node#1095: Updates lost commit-lock handling in the same consensus signing path and uses signing records during recovery.
  • kynesyslabs/node#1097: Changes the boot error path used when trusted NTP readings cannot be obtained.

Suggested reviewers: hackobi

Merge Risk: Moderate · up to c60d2

Two consistent but incorrect future clock readings can prevent the validator from signing for an extended period; reject implausibly future NTP times before merging.

Pre-merge checks: Passed 4 · Failed 1

Failed Failed checks: 1 warning

Check name Status Explanation Resolution
Docstring Coverage Warning Docstring coverage is 68.75% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff with cyclomatic complexity of at least 2. Documenta… Write docstrings for the functions missing them to satisfy the coverage threshold.
Passed Passed checks: 4 passed
Check name Status Explanation
Description Check Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check Passed The title clearly and concisely describes the main change: refusing rounds that may have been signed before a restart.
Linked Issues check Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check Passed Check skipped because no linked issues were found for this pull request.
Full details: Docstring Coverage

Explanation

Docstring coverage is 68.75% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff with cyclomatic complexity of at least 2. Documentation-only edits and deletions within surviving functions are included; unchanged diff context and fully deleted functions are excluded. Analyzed 16 functions across 14 files. (1 skipped: 1 unsupported.)

Resolution

Write docstrings for the functions missing them to satisfy the coverage threshold.

Finishing Touches
📝 Generate docstrings
  • Commit to this branch
  • Create a new PR
🧪 Generate unit tests (beta)
  • Commit to this branch
  • Create a new PR
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

A rabbit checks the clock at dawn
Then marks the rounds that must be gone
Old locks yield when records say
The newer signed round leads the way
Safe startup hops into the day

Comment @coderabbitai help to get the list of available commands.

@greptile-apps

greptile-apps Bot commented Oct 30, 2026 •

Copy link
Copy Markdown

RetriggerView in GreptileConfidence Score: 5/5

[High impact] The PR appears safe to merge; no new actionable issues were found.

Summary

The PR prevents a restarted validator from signing rounds that an earlier run may have signed.

  • Restarted validators wait before signing rounds again.
  • Precommit choices follow a signed record when a lock copy is older.
  • Node startup waits for two believable clock readings that agree.

The previous review commit is an ancestor of the head, but their merge bases with the target differ. This review therefore covered the full PR diff. No previous Greptile threads were supplied. No new actionable issues were found. Tests were not run during this review.

Diagram

%%{init: {'theme': 'neutral'}}%%
flowchart TD
    A[Start main] --> B[Hold signing]
    B --> C[Read network time twice]
    C --> D{Readings agree?}
    D -->|No, after retries| E[Stop startup]
    D -->|Yes| F[Find highest surviving recorded round]
    F --> G[Set floor to later of recorded round or clock plus 5 seconds]
    G --> H[Release signing hold]
    H --> I{Requested round above floor?}
    I -->|No| J[Refuse signature]
    I -->|Yes| K[Apply existing signing and lock checks]
Loading

Reviews (2) · Last reviewed commit: "Merge remote-tracking branch 'origin/sta..." · Reviewed by Greptile

…restore-floor

# Conflicts:
#	scripts/test-consensus.sh
@Shitikyan
Shitikyan merged commit e30ee46 into stabilisation Oct 30, 2026
11 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant