Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 5 additions & 4 deletions app/lavik.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -163,7 +163,7 @@ int main(int argc, char** argv) {
->check(CLI::PositiveNumber);
app.add_option("-t,--threads,--shards", options.shard_count_,
"Data shard count (default: selected CPUs, reserving one with "
"--meta-exclusive-cpu)")
"--meta-exclusive-cpu in Meta-managed mode)")
->default_str("auto")
->check(CLI::PositiveNumber);
app.add_option("--maxclients", options.max_clients_,
Expand All @@ -173,9 +173,10 @@ int main(int argc, char** argv) {
app.add_flag("--pin-workers,!--no-pin-workers", options.pin_workers_,
"Pin workers cyclically to the selected or inherited CPUs")
->capture_default_str();
app.add_flag("--meta-exclusive-cpu,!--no-meta-exclusive-cpu",
options.meta_exclusive_cpu_,
"Reserve the last selected CPU for the Meta worker")
app.add_flag(
"--meta-exclusive-cpu,!--no-meta-exclusive-cpu",
options.meta_exclusive_cpu_,
"Reserve the last selected CPU for Meta (ignored in standalone mode)")
->capture_default_str();
app.add_option("--cpus", options.cpu_ids_,
"Logical CPU IDs, cycled over workers")
Expand Down
8 changes: 4 additions & 4 deletions docs/architecture/01-overview.md
Original file line number Diff line number Diff line change
Expand Up @@ -150,14 +150,14 @@ in the [deployment guide](../operations/cluster-deployment.md).
command/storage bindings, metrics shards, transaction runtime, and Bycorf
service graph. Meta-managed mode starts the outbound Meta control client
fenced; no topology or positive authority is restored locally.
3. The runtime has N data workers and one final control worker, where N is
the configured shard count. Redis listeners and connections, metrics,
3. The runtime has N data workers, where N is the configured shard count.
Meta-managed mode adds one final control worker; standalone mode does not. Redis listeners and connections, metrics,
replication data flows, memory accounting and storage occupy only the data
prefix. The control worker owns Meta sessions and NodeControl transitions;
it remains idle when Meta is not configured. All workers use the same
it is absent when Meta is not configured. All workers use the same
selected network and storage backends. CPU placement cycles over an explicit
list or the inherited affinity mask, allowing multiple workers on one CPU.
Optional exclusive Meta placement
In Meta-managed mode, optional exclusive Meta placement
reserves its final logical CPU and keeps all data workers on the others;
automatic sizing subtracts that reserved CPU from N.
4. On every data worker, `RedisService::Run` binds the memory and transaction shards
Expand Down
17 changes: 11 additions & 6 deletions docs/operations/quick-start-tuning.md
Original file line number Diff line number Diff line change
Expand Up @@ -49,23 +49,28 @@ taskset -c "$LAVIK_CPUSET" ./build/lavik \

`LAVIK_CPUSET` and `LAVIK_THREADS` select the CPU set and data shard count.
`--shards N` is the preferred spelling; `--threads N` and `-t N` remain aliases.
Lavik starts N data workers plus one final control worker, including in
standalone mode. The data layout and per-shard budgets continue to use N.
Lavik starts N data workers. Meta-managed mode adds one final control worker;
standalone mode has no extra worker. The data layout and per-shard budgets use N.

With pinning enabled, workers cycle over the inherited affinity mask. Use
`--cpus 2,4,6,8` (or `cpus 2,4,6,8` in a config file) to select a different
ordered list inside that mask. Four shards then map to CPUs 2,4,6,8 and the
control worker maps to CPU 2. Supplying N+1 entries specifies the complete
control worker, when present, maps to CPU 2. Supplying N entries in standalone
mode or N+1 in Meta-managed mode specifies the complete
mapping; repeated CPU IDs are supported. `--no-pin-workers` disables pinning
and cannot be combined with an explicit CPU list. Startup logs show the
resolved worker count, shard count, control worker ID and CPU mapping.
`--meta-exclusive-cpu` (config: `meta-exclusive-cpu yes`) defaults to false.
It reserves the final selected logical CPU for the Meta worker and cycles data
In standalone mode it is ignored, including with `--no-pin-workers` when no
explicit CPU list is set. In Meta-managed mode, it reserves the final selected
logical CPU for the Meta worker and cycles data
workers over the remaining CPUs. Without an explicit shard count, ten selected
CPUs produce ten data workers plus Meta by default, or nine data workers plus
CPUs produce ten data workers in standalone mode. In Meta-managed mode, they
produce ten data workers plus Meta by default, or nine data workers plus
Meta with this option. An explicit `--shards N` keeps N data workers, even when
N exceeds the available data CPUs; none are pinned to the reserved Meta CPU.
The option requires pinning and at least two selected CPUs; the reserved CPU
In Meta-managed mode the option requires pinning and at least two selected
CPUs; the reserved CPU
must not also appear earlier in an explicit list. This isolates Lavik workers,
not other processes, IRQs, or SMT siblings. Keep an explicit shard count when
restarting existing storage: changing automatic sizing can change its layout.
Expand Down
7 changes: 4 additions & 3 deletions include/lavik/config.h
Original file line number Diff line number Diff line change
Expand Up @@ -70,11 +70,12 @@ absl::Status RewriteRedisConfigFile(const std::string& path,
absl::Status ValidateServerOptions(const ServerOptions& options);

// Resolve an automatic (zero) shard count after config and CLI overrides.
// Exclusive Meta placement reserves one selected CPU, including when unpinned
// placement would otherwise have been requested (that combination is rejected).
// With Meta seeds configured, exclusive Meta placement reserves one selected
// CPU and requires pinning. Standalone sizing uses every selected CPU.
absl::Status ResolveAutomaticShardCount(ServerOptions* options);

// Resolve cyclic shard/control placement against the inherited CPU affinity.
// Resolve cyclic data worker placement against the inherited CPU affinity.
// Append a control worker only when Meta seeds are configured.
// Empty result means unpinned; explicit CPU IDs must be permitted by the OS.
absl::StatusOr<std::vector<unsigned>> ResolveWorkerCpuIds(
const ServerOptions& options);
Expand Down
6 changes: 4 additions & 2 deletions include/lavik/server.h
Original file line number Diff line number Diff line change
Expand Up @@ -64,11 +64,13 @@ struct ServerOptions {
std::string masteruser_ = "default";
std::string masterauth_;
// Data shards occupy workers [0, shard_count_). Meta uses the additional
// worker shard_count_; it owns no data shard or storage buffers.
// worker shard_count_ only when Meta seeds are configured; it owns no data
// shard or storage buffers. Standalone starts exactly shard_count_ workers.
unsigned shard_count_ = 1;
std::uint64_t max_clients_ = kDefaultMaxClients;
bool pin_workers_ = true;
// Reserve the final selected CPU for Meta; data workers cycle over the rest.
// In Meta-managed mode, reserve the final selected CPU for Meta; data
// workers cycle over the rest. Ignored in standalone mode.
bool meta_exclusive_cpu_ = false;
std::vector<unsigned> cpu_ids_;
int idle_timeout_ms_ = -1;
Expand Down
26 changes: 17 additions & 9 deletions src/config.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -641,7 +641,8 @@ absl::Status ValidateServerOptions(const ServerOptions& options) {
options.shard_count_ > std::numeric_limits<bycorf::WorkerId>::max() - 1) {
return absl::InvalidArgumentError("shards exceeds runtime worker capacity");
}
if (options.meta_exclusive_cpu_ && !options.pin_workers_) {
if (!options.meta_seeds_.empty() && options.meta_exclusive_cpu_ &&
!options.pin_workers_) {
return absl::InvalidArgumentError(
"meta-exclusive-cpu requires pin-workers");
}
Expand Down Expand Up @@ -841,7 +842,7 @@ absl::StatusOr<std::vector<unsigned>> SelectedWorkerCpus(
absl::StrCat("CPU ", cpu, " is outside inherited affinity"));
}
}
if (options.meta_exclusive_cpu_) {
if (!options.meta_seeds_.empty() && options.meta_exclusive_cpu_) {
if (!options.pin_workers_)
return absl::InvalidArgumentError(
"meta-exclusive-cpu requires pin-workers");
Expand All @@ -864,27 +865,34 @@ absl::Status ResolveAutomaticShardCount(ServerOptions* options) {
auto cpus = SelectedWorkerCpus(*options);
if (!cpus.ok()) return cpus.status();
options->shard_count_ = static_cast<unsigned>(cpus->size()) -
static_cast<unsigned>(options->meta_exclusive_cpu_);
static_cast<unsigned>(!options->meta_seeds_.empty() &&
options->meta_exclusive_cpu_);
return absl::OkStatus();
}

absl::StatusOr<std::vector<unsigned>> ResolveWorkerCpuIds(
const ServerOptions& options) {
if (!options.pin_workers_ && !options.meta_exclusive_cpu_)
if (!options.pin_workers_ &&
(options.meta_seeds_.empty() || !options.meta_exclusive_cpu_))
return std::vector<unsigned>{};
auto selected = SelectedWorkerCpus(options);
if (!selected.ok()) return selected.status();
const auto& cpus = *selected;
const auto data_cpu_count = cpus.size() - options.meta_exclusive_cpu_;
// Seed configuration is authoritative here: automatic sizing runs before
// RunServer derives meta_managed_. Standalone never reserves a control CPU.
const bool has_control_worker = !options.meta_seeds_.empty();
const bool exclusive = has_control_worker && options.meta_exclusive_cpu_;
const auto data_cpu_count = cpus.size() - exclusive;
std::vector<unsigned> result;
const unsigned total = options.shard_count_ + 1;
const unsigned total = options.shard_count_ + has_control_worker;
result.reserve(total);
for (unsigned worker = 0; worker < options.shard_count_; ++worker) {
result.push_back(cpus[worker % data_cpu_count]);
}
result.push_back(options.meta_exclusive_cpu_
? cpus.back()
: cpus[options.shard_count_ % cpus.size()]);
if (has_control_worker) {
result.push_back(exclusive ? cpus.back()
: cpus[options.shard_count_ % cpus.size()]);
}
return result;
}

Expand Down
13 changes: 8 additions & 5 deletions src/redis/server.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -2407,13 +2407,16 @@ int RunServer(ServerOptions options) {

bycorf::ServerOptions runtime_options;
runtime_options.bind_addresses_ = options.bind_addresses_;
runtime_options.thread_count_ = options.shard_count_ + 1;
// A service-free worker still spins in DPDK poll mode. Only Meta-managed
// nodes need the additional control worker.
runtime_options.thread_count_ = options.shard_count_ + options.meta_managed_;
runtime_options.pin_workers_ = options.pin_workers_;
runtime_options.cpu_ids_ = std::move(*cpu_ids);
spdlog::info("runtime workers={} data_shards={} control_worker={} cpus={}",
runtime_options.thread_count_, options.shard_count_,
options.shard_count_,
absl::StrJoin(runtime_options.cpu_ids_, ","));
spdlog::info(
"runtime workers={} data_shards={} control_worker={} cpus={}",
runtime_options.thread_count_, options.shard_count_,
options.meta_managed_ ? std::to_string(options.shard_count_) : "none",
absl::StrJoin(runtime_options.cpu_ids_, ","));
runtime_options.idle_timeout_ms_ = options.idle_timeout_ms_;
runtime_options.recv_buffer_count_ = options.recv_buffer_count_;
runtime_options.busy_poll_us_ = options.busy_poll_us_;
Expand Down
7 changes: 7 additions & 0 deletions tests/config_test.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -448,6 +448,13 @@ TEST(RedisConfigTest, AutomaticShardsAndExclusiveMetaCpu) {
EXPECT_EQ(options.shard_count_, cpus.size());
auto shared = lavik::ResolveWorkerCpuIds(options);
ASSERT_TRUE(shared.ok()) << shared.status();
EXPECT_EQ(*shared, cpus);

// CLI automatic sizing precedes RunServer setting meta_managed_.
options.meta_seeds_ = {"127.0.0.1:17001"};
EXPECT_FALSE(options.meta_managed_);
shared = lavik::ResolveWorkerCpuIds(options);
ASSERT_TRUE(shared.ok()) << shared.status();
EXPECT_EQ(shared->size(), cpus.size() + 1);
EXPECT_EQ(shared->back(), cpus.front());

Expand Down
7 changes: 4 additions & 3 deletions tests/runtime_backends_smoke.py
Original file line number Diff line number Diff line change
Expand Up @@ -103,7 +103,7 @@ def run(binary, network, data, directory, iteration, populate):
while time.monotonic() < ready:
assert p.poll() is None, path.read_text()[-5000:]
if network == "dpdk" and not tap_ready:
if path.read_text().count("bycorf0: Ethernet address:") < 3:
if path.read_text().count("bycorf0: Ethernet address:") < 2:
time.sleep(0.05)
continue
sp.run(
Expand Down Expand Up @@ -140,7 +140,8 @@ def run(binary, network, data, directory, iteration, populate):
except FileNotFoundError:
pass # unrelated short-lived descriptors

assert sum("io_uring" in f for f in rings) == 3, rings
# Standalone creates only the two requested data workers.
assert sum("io_uring" in f for f in rings) == 2, rings
if network == "kernel":
assert "EAL:" not in path.read_text()
values = {
Expand All @@ -164,7 +165,7 @@ def run(binary, network, data, directory, iteration, populate):
print(
network,
iteration,
"PASS: data, recovery, 2 shards + control / 3 rings, clean shutdown",
"PASS: data, recovery, 2 standalone shards / 2 rings, clean shutdown",
flush=True,
)
finally:
Expand Down
20 changes: 20 additions & 0 deletions tests/worker_placement_test.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -33,6 +33,7 @@ TEST(WorkerPlacementTest, CpuPolicyIncludesControlWorkerAndCycles) {
ASSERT_LT(cpu, CPU_SETSIZE);
lavik::ServerOptions options;
options.shard_count_ = 3;
options.meta_seeds_ = {"127.0.0.1:17001"};
options.cpu_ids_ = {cpu};
auto mapped = lavik::ResolveWorkerCpuIds(options);
ASSERT_TRUE(mapped.ok()) << mapped.status();
Expand All @@ -46,6 +47,25 @@ TEST(WorkerPlacementTest, CpuPolicyIncludesControlWorkerAndCycles) {
EXPECT_TRUE(lavik::ResolveWorkerCpuIds(options)->empty());
}

TEST(WorkerPlacementTest, StandaloneHasOnlyDataWorkers) {
const unsigned cpu = FirstAllowedCpu();
ASSERT_LT(cpu, CPU_SETSIZE);
lavik::ServerOptions options;
options.shard_count_ = 3;
options.cpu_ids_ = {cpu};
options.meta_exclusive_cpu_ = true;
auto mapped = lavik::ResolveWorkerCpuIds(options);
ASSERT_TRUE(mapped.ok()) << mapped.status();
EXPECT_EQ(*mapped, (std::vector<unsigned>{cpu, cpu, cpu}));
options.shard_count_ = 0;
ASSERT_TRUE(lavik::ResolveAutomaticShardCount(&options).ok());
EXPECT_EQ(options.shard_count_, 1u);
options.cpu_ids_.clear();
options.pin_workers_ = false;
ASSERT_TRUE(lavik::ResolveWorkerCpuIds(options).ok());
EXPECT_TRUE(lavik::ResolveWorkerCpuIds(options)->empty());
}

TEST(WorkerPlacementTest, ShardAndCpuConfigurationIsExplicit) {
lavik::ServerOptions options;
ASSERT_TRUE(lavik::ApplyRedisConfigDirective({"shards", "2"}, &options).ok());
Expand Down