Skip to content

optimize and fix info command, add compact command - #476

Merged
thweetkomputer merged 12 commits into
eloqdata:mainfrom
thweetkomputer:optimize-info
Jun 2, 2026
Merged

thweetkomputer merged 12 commits into
eloqdata:mainfrom
thweetkomputer:optimize-info

Conversation

@thweetkomputer

@thweetkomputer thweetkomputer commented May 22, 2026 •

Copy link
Copy Markdown
Collaborator

Summary by CodeRabbit

  • New Features

    • INFO now shows OS info, executable path, total system memory, process RSS, and store disk key count.
    • New compact command (available in cloud-enabled builds) to trigger store compaction and report success.
  • Improvements

    • Service collects system metadata at startup for faster, more accurate INFO responses.
  • Chores

    • Removed legacy Test Build CI workflow.

Review Change Stack

@coderabbitai

coderabbitai Bot commented May 22, 2026 •

Copy link
Copy Markdown
Contributor

Note

Reviews paused

It looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the reviews.auto_review.auto_pause_after_reviewed_commits setting.

Use the following commands to manage reviews:

  • @coderabbitai resume to resume automatic reviews.
  • @coderabbitai review to trigger a single review.

Use the checkboxes below for quick actions:

  • ▶️ Resume reviews
  • 🔍 Trigger review

Walkthrough

Caches OS/executable/memory at RedisService startup, exposes getters, refactors INFO to use cached values plus /proc VmRSS parsing, conditionally adds a COMPACT command and handler guarded by ELOQKV_WITH_DSS_ROCKSDB_CLOUD, and updates CI and a submodule pointer.

Changes

System Info Caching for INFO Command

Layer / File(s) Summary
Service getters and InfoCommand fields
include/redis_service.h, include/redis_command.h
Added RedisServiceImpl getters/backing fields and InfoCommand stored fields for OS, executable path, and total system memory (KB).
Service startup initialization
src/redis_service.cpp
Added ExecCommand helper and Init-time logic to populate os_info_, executable_path_, total_system_memory_kb_, and event_dispatcher_num_.
INFO command refactor
src/redis_command.cpp
InfoCommand now reads cached service fields, added GetProcessVmRSSKb(pid) to parse /proc//status for VmRSS, and INFO output emits Server/Memory/Keyspace using cached values and RSS parsing.

COMPACT command (DSS RocksDB Cloud)

Layer / File(s) Summary
Command macro, types and parser
include/redis_command.h, src/redis_command.cpp
Introduce ELOQKV_WITH_DSS_ROCKSDB_CLOUD conditional, add COMPACT type, CompactCommand struct, and ParseCompactCommand.
Handler, wiring and registration
include/redis_handler.h, src/redis_handler.cpp, src/redis_service.cpp, src/redis_command.cpp
Declare/implement CompactCommandHandler::Run, register "compact" handler in service, route COMPACT via ParseMultiCommand, implement CompactCommand execution calling DataStoreHandler::CompactStore(), and report store:disk_keys in INFO when enabled.

CI & Submodule

Layer / File(s) Summary
Build workflow private deps
.github/workflows/build.yml
Derive a release branch name from tag_name and checkout that branch for private dependency repos before updating submodules.
Submodule pointer update
data_substrate
Bump data_substrate submodule pointer to a new commit hash.

Sequence Diagram (COMPACT flow)

sequenceDiagram
  participant Client
  participant RedisHandler as CompactCommandHandler
  participant RedisService
  participant DataStore as DataStoreHandler
  Client->>RedisHandler: "compact" request (args)
  RedisHandler->>RedisService: ParseCompactCommand + ExecuteCommand(CompactCommand)
  RedisService->>DataStore: CompactStore()
  DataStore-->>RedisService: success/failure
  RedisService-->>RedisHandler: command result (success/failure)
  RedisHandler-->>Client: REDIS reply (OK / error)
Loading

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~45 minutes

Possibly related PRs

  • eloqdata/eloqkv#453: Also updates the data_substrate submodule commit pointer.
  • eloqdata/eloqkv#309: Changes around store_hd_ initialization that interact with COMPACT/store handler usage.
  • eloqdata/eloqkv#477: Overlapping CI/workflow updates related to private dependency checkout behavior.

Suggested reviewers

  • liunyl
  • githubzilla
  • MrGuin

Poem

🐰 I hopped through init at break of dawn,
gathered uname and /proc upon.
INFO now whispers with cached grace,
RSS counted, paths in place.
Compact hums to tidy cloud-bound space.

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title accurately captures the main changes: optimizing/fixing the INFO command and adding a new COMPACT command. It covers the primary objectives evident in the changeset.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 3

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@src/redis_command.cpp`:
- Around line 2102-2116: GetProcessVmRSSKb currently reads tokens in triplets
which breaks when non-numeric lines appear before VmRSS; change it to read
/proc/<pid>/status line-by-line (use std::getline on the opened file), check for
a line that starts with "VmRSS:", then parse that line with a std::istringstream
(or find the first number) to extract and return the numeric KB value; update
GetProcessVmRSSKb to return 0 on any parse/open failure and ensure you reference
the function name GetProcessVmRSSKb when making the change.

In `@src/redis_service.cpp`:
- Around line 558-580: Replace the shell-dependent ExecCommand calls used to set
os_info_ and total_system_memory_kb_ with direct system calls/parsing: call
uname(2) (or the uname() libc wrapper) to build os_info_ instead of
ExecCommand("uname -srm"), keep and tighten the readlink("/proc/self/exe", ...)
logic used to populate executable_path_ (ensure proper error handling and no
cached failure string), and obtain total_system_memory_kb_ via
sysconf(_SC_PHYS_PAGES)*page_size or by parsing /proc/meminfo's "MemTotal:" line
as a fallback rather than ExecCommand("free | grep Mem | awk '{print $2}'");
also avoid caching shell error output or zero by catching/handling errors and
leaving fields empty or using sensible defaults when the system calls fail.
- Around line 152-170: The code currently dereferences rst.rbegin() without
ensuring rst is non-empty; add guards to avoid undefined behavior by checking
rst.empty() before any use of *rst.rbegin() — update the loop condition that
appends a newline (replace the check "if (rst.size() > 0 && *rst.rbegin() !=
'\n')" with a test that first ensures !rst.empty() before reading the last char)
and also guard the final if that trims the trailing newline (check rst.empty()
and return rst immediately if empty, otherwise inspect the last character). Use
the existing variables (rst, line) and keep the same pclose(fp) and return logic
otherwise.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: e7231402-f26c-4d3f-94f8-9d9e8115ba54

📥 Commits

Reviewing files that changed from the base of the PR and between 5f725fe and f15c834.

📒 Files selected for processing (5)
  • data_substrate
  • include/redis_command.h
  • include/redis_service.h
  • src/redis_command.cpp
  • src/redis_service.cpp

Comment thread src/redis_command.cpp
Comment thread src/redis_service.cpp Outdated
Comment thread src/redis_service.cpp
thweetkomputer and others added 4 commits May 25, 2026 09:59
…oqdata#477)

* feat: checkout private deps at version-matched branch

Derive branch name (rel_x_x_x_eloqkv) from tag_name for
eloq_log_service and raft_host_manager clones in CI.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

* remove unused test-build.yml

* clean

---------

Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 3

🧹 Nitpick comments (1)
.github/workflows/build.yml (1)

68-70: 💤 Low value

Optimize: clone, checkout, then update submodules once.

Cloning with --recurse-submodules initializes submodules for the default branch, but line 69 immediately checks out a different branch, then line 70 updates submodules again. This causes redundant submodule initialization.

♻️ More efficient approach
-          git clone --recurse-submodules git@github.com:eloqdata/eloq_log_service.git logservice_src
+          git clone git@github.com:eloqdata/eloq_log_service.git logservice_src
           git -C logservice_src checkout $REL_BRANCH
           git -C logservice_src submodule update --init --recursive
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In @.github/workflows/build.yml around lines 68 - 70, The current workflow
redundantly initializes submodules by using git clone --recurse-submodules
before checking out $REL_BRANCH and then running git -C logservice_src submodule
update --recursive again; change the sequence to plain git clone (remove
--recurse-submodules), then git -C logservice_src checkout $REL_BRANCH, and
finally run git -C logservice_src submodule update --init --recursive so
submodules are initialized/updated only after the branch checkout (referencing
the existing commands git clone, git -C ... checkout $REL_BRANCH, and git -C ...
submodule update --recursive).
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In @.github/workflows/build.yml:
- Line 66: The REL_BRANCH assignment uses unvalidated interpolation of
inputs.tag_name, which risks shell injection; sanitize and quote the tag before
building REL_BRANCH: validate inputs.tag_name by stripping or replacing any
characters except a safe whitelist (e.g., [A-Za-z0-9._-]) and assign that
sanitized value to a temporary variable, then construct REL_BRANCH using the
sanitized variable and wrap it in double quotes to prevent
word-splitting/expansion; reference the REL_BRANCH assignment and the
inputs.tag_name expansion when making this change.
- Around line 69-70: Before running git -C logservice_src checkout $REL_BRANCH
and git -C raft_host_manager_src checkout $REL_BRANCH, validate that
origin/$REL_BRANCH exists; replicate the logic from scripts/git-checkout.sh by
running git ls-remote --heads origin "$REL_BRANCH" | grep -q "$REL_BRANCH" and
if found use git checkout -b "$REL_BRANCH" "origin/$REL_BRANCH", otherwise print
a clear error like "Expected release branch $REL_BRANCH not found on origin" and
exit 1; apply this change for both the logservice_src and raft_host_manager_src
checkout steps so the workflow fails fast when the remote release branch is
missing.

In `@include/redis_command.h`:
- Around line 279-281: The enum RedisCommandType must not insert COMPACT
mid-list because RedisCommandType is serialized as a raw uint8_t in Serialize()
implementations; move the COMPACT enumerator to the end of the RedisCommandType
enum (after FAILOVER and all existing values) or alternatively give every enum
member an explicit stable numeric value so IDs don't change across builds;
update any related comments to reflect the new ordering and ensure
Serialize()/Deserialize() continue to use the stable numeric IDs.

---

Nitpick comments:
In @.github/workflows/build.yml:
- Around line 68-70: The current workflow redundantly initializes submodules by
using git clone --recurse-submodules before checking out $REL_BRANCH and then
running git -C logservice_src submodule update --recursive again; change the
sequence to plain git clone (remove --recurse-submodules), then git -C
logservice_src checkout $REL_BRANCH, and finally run git -C logservice_src
submodule update --init --recursive so submodules are initialized/updated only
after the branch checkout (referencing the existing commands git clone, git -C
... checkout $REL_BRANCH, and git -C ... submodule update --recursive).
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: 78f5ab39-9339-498d-add7-d92b753cf6a9

📥 Commits

Reviewing files that changed from the base of the PR and between e734557 and 5b58d4c.

📒 Files selected for processing (8)
  • .github/workflows/build.yml
  • .github/workflows/test-build.yml
  • include/redis_command.h
  • include/redis_handler.h
  • include/redis_service.h
  • src/redis_command.cpp
  • src/redis_handler.cpp
  • src/redis_service.cpp
💤 Files with no reviewable changes (1)
  • .github/workflows/test-build.yml
👮 Files not reviewed due to content moderation or server errors (1)
  • src/redis_command.cpp

Comment thread .github/workflows/build.yml
Comment thread .github/workflows/build.yml
Comment thread include/redis_command.h Outdated
@thweetkomputer
thweetkomputer requested a review from liangjchen June 1, 2026 07:55
@thweetkomputer
thweetkomputer merged commit 55eec4d into eloqdata:main Jun 2, 2026
5 checks passed
@thweetkomputer
thweetkomputer deleted the optimize-info branch June 2, 2026 02:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants