Open log service loses redo data in multi-node-group deployments (record key collision + global checkpoint-ts fold) #496

Two independent defects in the open log service (log_service/, OPEN_LOG_SERVICE=ON) cause silent committed-data loss on recovery whenever more than one cc node group writes to the same log group. Default single-node EloqKV (one NG) is unaffected; multi-NG clusters sharing a log group are exposed.

Defect 1 — data-log records of different NGs collide on the same key

Log records are keyed by (commit_ts, txn) only — 16 bytes, no node-group id:

log_service/src/log_state_rocksdb_impl.cpp:56-67:

for (const auto &[tx_number, timestamp, log_message] : batch_logs) {
    std::array<char, 16> key{};
    Serialize(key, timestamp, tx_number);          // ts(8) + txn(8)
    batch.Put(rocksdb::Slice(key.data(), key.size()), log_message);
}

A single distributed transaction produces one node_txn_logs blob per participating NG, all carrying the same txn and commit_ts:

log_service/src/open_log_task.cpp:109-115:

for (auto it = data_log.node_txn_logs().begin(); it != data_log.node_txn_logs().end(); ++it)
    batch_logs.emplace_back(txn, timestamp, it->second);   // same (txn,ts) key for every NG

Within one WriteBatch, later puts overwrite earlier ones → only one NG's redo blob survives. On recovery the other NGs' committed writes are gone. (The non-batched AddLogItem path uses the same key, so this is a key-design issue, not just a batching bug — distinct from #494.)

Fix: include the cc-node-group id in the record key.

Defect 2 — checkpoint ts is folded across all NGs into one global value

HandleUpdateCkptTs ignores cc_node_group_id:

log_service/src/open_log_task.cpp:283-289:

void OpenLogTaskWorker::HandleUpdateCkptTs(const UpdateCheckpointTsRequest &req, ...) {
    log_state_->UpdateCkptTs(req.ckpt_timestamp());   // cc_node_group_id dropped

LogState keeps a single cc_ng_info_.last_ckpt_ts_ (log_state.h:578-601) shared by all NGs, advanced to the max. The replay floor and SST purge use it, so an NG that is behind on checkpointing has its committed-but-unflushed records skipped on cold replay and physically purged; RecoverTx can then answer NotCommitted for a committed tx (→ rollback of committed data / orphan-lock mis-resolution).

Fix: track and purge checkpoint ts per cc-node-group.

Found during a code audit (docs work, PR #493). Verified against source at the cited lines.

🤖 Found with Claude Code