A highly performant, distributed, partitioned, and persistent key-value store built in modern C++ (C++20).
-
Custom Framed Protocol: Uses a length-prefixed TCP protocol with JSON payloads (
nlohmann/json). -
Concurrent Storage Engine: Thread-safe in-memory
std::unordered_mapwith 16-shard fine-grained locking. -
Write-Ahead Log (WAL): Ensures strict crash consistency with tunable durability (
ASYNC,SYNC). Features magic headers, CRC checksums, and startup replay logic. -
LRU Cache: A fast-path
$O(1)$ memory cache for accelerating reads (GET). -
Global Metrics: Lock-free (
std::atomic) counters tracking request volumes, cache hits/misses, and storage ops. - Raft Consensus: Core Raft algorithm implementation handling elections, randomized timeouts, heartbeats, log replication, and majority-quorum commits.
- Dynamic Sharding: A Consistent Hash Ring with virtual nodes that automatically maps keys to partitions.
- Failure Detection: Seamless integration between Raft heartbeats and the hash ring. Unresponsive nodes are evicted dynamically, auto-healing the cluster.
- Docker & Docker Compose
run_dev.ps1helper for local compiling
We provide a helper script to build the project inside a fully POSIX-compliant Ubuntu 24.04 environment.
.\run_dev.ps1 "cmake -S . -B build -DCMAKE_BUILD_TYPE=Release && cmake --build build -j4"We provide a docker-compose.yml to spin up the distributed topology instantly.
docker-compose up --build -dUse the built-in CLI to interact with the cluster.
# Set a value on Node 1 (if the key hashes to another node, Node 1 will respond with REDIRECT)
.\run_dev.ps1 "./build/src/client/kv-cli -h 127.0.0.1 -p 7001 set mykey hello_world"
# Get a value
.\run_dev.ps1 "./build/src/client/kv-cli -h 127.0.0.1 -p 7001 get mykey"
# View Cache and Node Metrics
.\run_dev.ps1 "./build/src/client/kv-cli -h 127.0.0.1 -p 7001 stats"Run the GoogleTest suite:
.\run_dev.ps1 "cmake -DBUILD_TESTING=ON -S . -B build && cmake --build build -j4 && cd build && ctest"Run the benchmark (Python required on host):
# Requires python with numpy installed
pip install numpy
python benchmark.py --host 127.0.0.1 --port 7001 --threads 10 --requests 1000