Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 0 additions & 2 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -10,8 +10,6 @@
.clangd
.cursor

AGENTS.md

# Superpowers scratch (plans/specs), not part of the repo
docs/superpowers/

Expand Down
36 changes: 36 additions & 0 deletions AGENTS.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,36 @@
# AGENTS.md

Agent guidance for the **EloqKV** repo (Redis/Valkey-compatible distributed DB;
transaction/storage engine lives in the `data_substrate` submodule).

**Read [CLAUDE.md](CLAUDE.md) first** — it is the source of truth for
architecture, the build/test commands, and code style. This file only restates
the operational essentials plus a few facts that are otherwise hard to discover.

## Build & run (short form; see CLAUDE.md for full flags)

```bash
mkdir -p bld && cd bld
cmake .. -DWITH_DATA_STORE=ELOQDSS_ELOQSTORE -DWITH_LOG_STATE=ROCKSDB \
-DCMAKE_INSTALL_PREFIX=../install
cmake --build . --parallel "$(nproc)" && cmake --install .
cd .. # install/ and eloqkv.ini are at the repo root
./install/bin/eloqkv --config=eloqkv.ini # single node, port 6379, foreground
Comment thread
coderabbitai[bot] marked this conversation as resolved.
Resolved
```

## Where are the logs?

Server logs are **glog** files that default to **`install/logs/`** — i.e.
`<install_prefix>/logs`, resolved relative to the binary, **not** the current
working directory. You'll find:

- `eloqdb.log.INFO` / `eloqdb.log.WARNING` / `eloqdb.log.ERROR`
- `host_manager.log.*`

The `*.INFO` / `*.WARNING` entries are symlinks pointing at the current
timestamped file (`eloqdb.log.INFO.YYYYMMDD-HHMMSS.<pid>`).

Discoverability note: nothing states this default except the startup banner
(`Running logs will be written to the following path: …/install/logs`). `--help`
only documents `--log_dir` as the *override*, never the default. To send logs
elsewhere, pass `--log_dir=<dir>` (or `--logtostderr` to log to stderr instead).
4 changes: 4 additions & 0 deletions CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -29,6 +29,10 @@ cmake --install . --prefix ../install

# Run a single-node server (port 6379, foreground)
./install/bin/eloqkv --config=eloqkv.ini
# Logs: glog files default to install/logs/ (i.e. <install_prefix>/logs, relative
# to the binary, NOT the cwd) — eloqdb.log.{INFO,WARNING,ERROR} and
# host_manager.log.*, with *.INFO/*.WARNING symlinks to the current file. The
# startup banner is the only place that prints this; override with --log_dir=<dir>.

# Run a single TCL test file (e.g. tests/unit/eloq/hash.tcl)
# against the running server; loop over tests/unit/eloq/*.tcl for the full suite
Expand Down
5 changes: 5 additions & 0 deletions docs/02-command-processing.md
Original file line number Diff line number Diff line change
Expand Up @@ -309,6 +309,11 @@ the special MOVED/READONLY translations of §6.
`ExecuteMultiObjTxRequest` (4486-4496, 4528-4536). `INFO` is a `DirectCommand`
(`include/redis_command.h:811`) assembled from these counters plus service fields captured at
Init (OS info, exe path, memory; `src/redis_service.cpp:587-630`).
- `INFO` `# Stats` also reports `active_extern_txms` — a live gauge of external transactions
(txms handed to the redis layer that have not committed/aborted), summed from the
per-TxProcessor `ext_active_tx_cnt_` counters via `TxService::ExternActiveTxCount()`.
Populated on every INFO regardless of `--enable_redis_stats`; exposed for txm-leak
regression testing (#528, the #506 leak class).
- Prometheus-style metrics (eloq_metrics): per-command duration histograms and totals plus
read/write aggregates, registered with the engine at Init (289-313) and collected in
`DispatchCommand`/`MultiExec` on sampled rounds (`CheckAndUpdateRedisCmdRound`, 6405). A
Expand Down
9 changes: 9 additions & 0 deletions eloqkv.ini
Original file line number Diff line number Diff line change
@@ -1,5 +1,14 @@
# EloqKV Configuration File
# For more information, visit: https://www.eloqdata.com/eloqkv/configuration
#
# Server logs: the runtime glog files default to <install_prefix>/logs (i.e.
# install/logs/), resolved relative to the eloqkv binary -- NOT this config file
# or the current working directory. Files: eloqdb.log.{INFO,WARNING,ERROR} and
# host_manager.log.* (the *.INFO/*.WARNING names are symlinks to the current
# timestamped file). This is a CLI flag, not an ini key: override the directory
# with --log_dir=<dir> (or --logtostderr to log to stderr instead).
# NOTE: this is distinct from the "log service"/WAL (redo log) *data* below,
# which lives under eloq_data_path.

[local]
# Whether to listen on all interfaces. If bind_all is enabled, EloqKV will listen on 0.0.0.0,
Expand Down
3 changes: 3 additions & 0 deletions include/redis_command.h
Original file line number Diff line number Diff line change
Expand Up @@ -905,6 +905,9 @@ struct InfoCommand : public DirectCommand
int64_t multi_cmd_count_{0};
int64_t cmds_per_sec_{0};
double cmd_latency_{0};
// Live external txm gauge read from the tx service, not a RedisStats
// statistic — populated regardless of enable_redis_stats.
uint64_t active_extern_txms_{0};
};

struct MemoryStatsCommand : public DirectCommand
Expand Down
3 changes: 3 additions & 0 deletions include/redis_service.h
Original file line number Diff line number Diff line change
Expand Up @@ -519,6 +519,9 @@ class RedisServiceImpl : public brpc::RedisService
{
return tx_service_;
}
// Live external transactions across all tx processors (INFO
// active_extern_txms). tx_service_ is null until Start(); reports 0 then.
size_t ActiveExternTxCount() const;
const char *GetVersion() const
{
return version_;
Expand Down
5 changes: 5 additions & 0 deletions src/redis_command.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -1786,6 +1786,7 @@ void InfoCommand::Execute(RedisServiceImpl *redis_impl,
executable_path_ = redis_impl->GetExecutablePath();
total_system_memory_kb_ = redis_impl->GetTotalSystemMemoryKB();
max_connection_count_ = redis_impl->MaxConnectionCount();
active_extern_txms_ = redis_impl->ActiveExternTxCount();

if (redis_impl->IsEnableRedisStats())
{
Expand Down Expand Up @@ -2614,6 +2615,10 @@ void InfoCommand::OutputResult(OutputHandler *reply) const
result += "\r\ntotal_commands_processed:" +
std::to_string(cmd_read_count_ + cmd_write_count_ +
multi_cmd_count_);
// Live external-transaction gauge (see issue #528): transactions
// created for client commands that have not yet committed/aborted.
result +=
"\r\nactive_extern_txms:" + std::to_string(active_extern_txms_);
// result +=
// "\r\nread_commands_processed:" +
// std::to_string(cmd_read_count_);
Expand Down
5 changes: 5 additions & 0 deletions src/redis_service.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -2039,6 +2039,11 @@ void RedisServiceImpl::AddHandlers()
AddCommandHandler("slowlog", slowlog_hd.get());
}

size_t RedisServiceImpl::ActiveExternTxCount() const
{
return tx_service_ != nullptr ? tx_service_->ExternActiveTxCount() : 0;
}

TransactionExecution *RedisServiceImpl::NewTxm(IsolationLevel iso_level,
CcProtocol protocol)
{
Expand Down
67 changes: 67 additions & 0 deletions tests/unit/eloq/multi.tcl
Original file line number Diff line number Diff line change
Expand Up @@ -309,6 +309,73 @@ start_server {tags {"multi"}} {
assert_equal "*-1" [exec_raw]
} {}

# INFO # Stats gauge of live external transactions (#528). The suite's
# s/status helpers only fetch the clients section, so read stats directly.
# Configs with data store + WAL keep a nonzero, slightly jittery background
# baseline, so the assertions are baseline-relative and use controlled
# signals large enough to dominate that noise. `txms_floor` samples the
# low-water mark over a short window (background daemons only add txms, so
# the minimum approximates the quiescent baseline).
proc active_txms {} {
getInfoProperty [r info stats] active_extern_txms
}
proc txms_floor {} {
set m [active_txms]
for {set i 0} {$i < 5} {incr i} {
after 20
set v [active_txms]
if {$v < $m} { set m $v }
}
return $m
}

test {active_extern_txms gauge is live and empty EXEC does not leak txms (#528)} {
# Positive control: hold N concurrent BEGIN sessions (each a live extern
# txm). The gauge must rise by ~N and fall back after rollback — a
# dead/constant gauge cannot pass. N is large enough that a few
# background completions can't mask the rise.
set N 10
set base [txms_floor]
set conns {}
try {
for {set i 0} {$i < $N} {incr i} {
set c [redis_client]
$c begin
lappend conns $c
}
wait_for_condition 50 100 {
[active_txms] >= $base + $N - 2
} else {
fail "gauge did not rise by ~$N held BEGIN txms over base $base: [active_txms]"
}
} finally {
foreach c $conns { catch {$c rollback}; $c close }
Comment on lines +341 to +352

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🩺 Stability & Availability | 🟡 Minor | ⚡ Quick win

Register each client before issuing BEGIN.

If BEGIN fails, that client is absent from conns and remains open. Also catch close failures so one broken client does not prevent cleanup of the rest.

Proposed cleanup fix
             for {set i 0} {$i < $N} {incr i} {
                 set c [redis_client]
-                $c begin
                 lappend conns $c
+                $c begin
             }
@@
-            foreach c $conns { catch {$c rollback}; $c close }
+            foreach c $conns {
+                catch {$c rollback}
+                catch {$c close}
+            }
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
for {set i 0} {$i < $N} {incr i} {
set c [redis_client]
$c begin
lappend conns $c
}
wait_for_condition 50 100 {
[active_txms] >= $base + $N - 2
} else {
fail "gauge did not rise by ~$N held BEGIN txms over base $base: [active_txms]"
}
} finally {
foreach c $conns { catch {$c rollback}; $c close }
for {set i 0} {$i < $N} {incr i} {
set c [redis_client]
lappend conns $c
$c begin
}
wait_for_condition 50 100 {
[active_txms] >= $base + $N - 2
} else {
fail "gauge did not rise by ~$N held BEGIN txms over base $base: [active_txms]"
}
} finally {
foreach c $conns {
catch {$c rollback}
catch {$c close}
}
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@tests/unit/eloq/multi.tcl` around lines 341 - 352, The connection cleanup in
the transaction setup loop must cover clients even when BEGIN fails. Append each
client to conns immediately after redis_client and before issuing BEGIN, and
make the finally-loop cleanup independently tolerate both rollback and close
failures so one client cannot prevent the remaining clients from closing.

}
wait_for_condition 50 100 {
[txms_floor] <= $base + 2
} else {
fail "gauge did not return to base $base after rolling back $N BEGINs: [txms_floor]"
}
# Regression for the #506 leak class: a large batch of empty WATCH/EXEC
# cycles must not accumulate outstanding txms. A one-per-cycle leak adds
# +$cycles, which dwarfs the whole baseline even if it fully drained; a
# healthy gauge settles back to the floor.
set cycles 100
set base [txms_floor]
r set x 30
for {set j 0} {$j < $cycles} {incr j} {
r watch x
r multi
assert_equal "*0" [exec_raw]
}
wait_for_condition 50 100 {
[txms_floor] <= $base + 5
} else {
fail "empty WATCH/EXEC cycles leaked extern txms above base $base: [txms_floor]"
}
assert {[string is integer -strict [active_txms]]}
} {}

# TODO: transaction conflict with flushdb/flushall. issue tracked. (tx1:watch-multi/exec; tx2:flushdb)
# test {FLUSHALL is able to touch the watched keys} {
# r set x 30
Expand Down
Loading