Context
Consumers of raftly (e.g., kv-fabric) implement their own snapshot managers that persist state machine snapshots to disk. After a snapshot is taken at log index N, the Raft log entries up to N are no longer needed for recovery — the snapshot is the authoritative state. Without a way to tell raftly to discard those entries, the log grows forever and recovery time scales with total log length rather than entries-since-last-snapshot.
Requested API
// TruncateLog discards all log entries with index <= upToIndex.
// The caller guarantees that a snapshot covering upToIndex has been
// durably persisted before calling this. After truncation, the node
// will not be able to serve AppendEntries for entries <= upToIndex
// to lagging followers — those followers must receive a snapshot instead.
TruncateLog(upToIndex uint64) error
And a complementary method for restart recovery:
// SetAppliedIndex tells the Raft node that the state machine has already
// applied all entries up to and including appliedIndex (e.g., from a
// loaded snapshot). Raft will not re-deliver those entries via CommittedEntries.
SetAppliedIndex(appliedIndex uint64)
Why both are needed
On a fresh start after a crash:
- The consumer loads its snapshot (state machine at version N).
- It calls
SetAppliedIndex(N) so raftly knows to start delivering entries from N+1, not from 0.
- Without
SetAppliedIndex, raftly re-delivers entries 1..N through CommittedEntries, which the state machine applies on top of the already-restored snapshot — corrupting state.
TruncateLog handles steady-state log growth. SetAppliedIndex handles crash recovery. Both are required for log compaction to be correct end-to-end.
Consumer
ani03sha/kv-fabric tracks this in kv-fabric#7. The SnapshotManager.TakeSnapshot() and LoadLatestSnapshot() methods are already written with hook points for these calls once the API exists.
Context
Consumers of raftly (e.g., kv-fabric) implement their own snapshot managers that persist state machine snapshots to disk. After a snapshot is taken at log index N, the Raft log entries up to N are no longer needed for recovery — the snapshot is the authoritative state. Without a way to tell raftly to discard those entries, the log grows forever and recovery time scales with total log length rather than entries-since-last-snapshot.
Requested API
And a complementary method for restart recovery:
Why both are needed
On a fresh start after a crash:
SetAppliedIndex(N)so raftly knows to start delivering entries from N+1, not from 0.SetAppliedIndex, raftly re-delivers entries 1..N throughCommittedEntries, which the state machine applies on top of the already-restored snapshot — corrupting state.TruncateLoghandles steady-state log growth.SetAppliedIndexhandles crash recovery. Both are required for log compaction to be correct end-to-end.Consumer
ani03sha/kv-fabric tracks this in kv-fabric#7. The
SnapshotManager.TakeSnapshot()andLoadLatestSnapshot()methods are already written with hook points for these calls once the API exists.