Skip to content

Latest commit

 

History

History
228 lines (168 loc) · 9.04 KB

File metadata and controls

228 lines (168 loc) · 9.04 KB

Journey 14 — Performance Tuning

lean-ctx is fast by default, but on a huge monorepo, a constrained CI runner, or a low-RAM laptop you may want to bound how much disk/RAM it uses or find what's slow. This journey covers the memory profile, the index/cache caps, and the slow-command log — with the exact knobs and their defaults.

Source files:

  • rust/src/core/config/mod.rsmemory_profile, bm25_max_cache_mb, graph_index_max_files
  • rust/src/cli/config_cmd.rsconfig show (effective limits)
  • rust/src/core/bm25_index.rs, graph_index.rs — index caps
  • rust/src/shell/exec.rsslow_command_threshold_ms

0. See your effective limits first

Before tuning, look at what's actually in effect. lean-ctx config show resolves config + env + defaults into one view and tags the source of each value:

╭─── Simplified (high-level) ───────────────────────────────╮
│ compression_level   = Max         ← config
│ max_disk_mb         =          0  ← default
│ max_ram_percent     =          5  ← default
│ max_staleness_days  =          0  ← default
│ memory_profile      = Performance  ← default
╰────────────────────────────────────────────────────────────╯

╭─── Derived effective limits ────────────────────────────────╮
│ archive_max_disk_mb    =    500 MB
│ bm25_max_cache_mb      =    512 MB
│ archive_max_age_hours  =     48 h
│ graph_index_max_files  =      0
╰────────────────────────────────────────────────────────────╯

← config vs ← default tells you whether a value is yours or the built-in. 0 means "unbounded / use the derived default" (see each knob below).


1. The memory profile — one dial for the footprint

memory_profile sets the overall disk/RAM posture; the derived limits (archive size, BM25 cache, staleness) follow from it unless you override them individually.

memory_profile = "balanced"     # balanced | performance | conservative
# or per-process, no config edit:
LEAN_CTX_MEMORY_PROFILE=conservative lean-ctx serve --daemon

Reach for conservative on small CI runners or low-RAM machines; performance trades disk for speed on a workstation.


2. Bounding the search index (BM25)

The BM25 full-text index is the biggest disk consumer on large repos.

bm25_max_cache_mb = 512          # cap the BM25 cache (derived from profile if unset)
extra_ignore_patterns = ["vendor/**", "*.min.js"]   # never index these
LEAN_CTX_BM25_MAX_CACHE_MB=256 lean-ctx index build

When the cap is hit, lean-ctx tells you exactly how to react (raise the cap or add ignore patterns). If an index is oversized or corrupt, reclaim it with lean-ctx cache prune — the next read rebuilds a clean one.


3. Bounding the code graph

graph_index_max_files = 0        # 0 = unlimited; set a cap on giant monorepos

On a very large tree, capping graph_index_max_files keeps graph builds fast and bounded; when the limit is reached, lean-ctx prints [graph_index: reached configured limit of N files. Set graph_index_max_files = 0 for unlimited.] so the truncation is never silent.

To skip indexing entirely (e.g. an ephemeral CI job that only needs reads):

LEAN_CTX_NO_INDEX=1 lean-ctx <cmd>          # or LEAN_CTX_DISABLE_SEARCH_INDEX=1

4. Disk / RAM / staleness budgets

These cross-cutting budgets apply across caches and indexes; 0 means "use the profile-derived default":

Knob Env override Meaning
max_disk_mb LEAN_CTX_MAX_DISK_MB total on-disk budget across caches/indexes
max_ram_percent LEAN_CTX_MAX_RAM_PERCENT RAM ceiling as % of system memory (default 5)
max_staleness_days LEAN_CTX_MAX_STALENESS_DAYS auto-prune entries older than N days

config show warns if max_disk_mb is set lower than archive.max_disk_mb + bm25_max_cache_mb, so your sub-budgets can't quietly exceed the global cap.


5. Finding what's slow — slow-log

lean-ctx records commands that exceed slow_command_threshold_ms (default 5000) so you can see where wall-clock time goes:

slow_command_threshold_ms = 5000
lean-ctx slow-log list      # show recorded slow commands
lean-ctx slow-log clear     # reset the log

Pair this with lean-ctx gain --deep (cost + heatmap) and lean-ctx ghost (uncompressed-command waste) from Journey 11 to turn "it feels slow" into a concrete list.


6. Keeping caches lean

lean-ctx cache stats              # size + hit rate
lean-ctx cache prune              # drop oversized/quarantined/orphaned indexes
lean-ctx cache reset --project    # wipe just this project's cache

A healthy cache has a high hit rate (each hit is a ~13-token re-read). cache prune is the safe periodic maintenance command; it never touches valid, in-budget entries.


7. Workload fit — where lean-ctx nets out (and where it doesn't)

lean-ctx saves tokens two ways: cold-read compression (a single read sent smaller) and cached re-reads (an unchanged file re-read collapses to a ~13-token back-reference). It also adds a fixed per-turn prefix — the MCP tool schemas, the server instructions, and the rules block. Whether you net out ahead depends on the harness and the provider:

Factor Nets ahead Can cost tokens
Context lifetime one long-lived context (agent loop, interactive session) — re-reads land in the same window, so the ~13-token stub is usable phase-isolated harness (a fresh process/context per phase) — the back-reference can't resolve in a cold context, so there is no re-read dividend
Provider pricing prompt-cache-priced (the injected prefix rides the provider cache and is billed once) non-caching / request-metered — the prefix is re-sent and re-billed every turn

On a phase-isolated and non-caching workload the cached-re-read lever has no surface and the injected prefix is pure re-billed overhead. That is an architecture–surface fit, not a failure mode — but you should tune for it.

The meter's denominator (read this before quoting gain)

lean-ctx gain measures compression on lean-ctx-touched traffic (the reads and shell output it actually processed) — its denominator is that traffic, not your full provider bill. It does not subtract the per-turn prefix lean-ctx injects. So on a non-caching rail the dashboard can read net-positive while the billed input moved net-negative. To keep this honest, gain now prints a Methodology line and gain --json carries injected_overhead_tokens_per_turn:

net bill impact ≈ tokens_saved − injected_overhead_tokens_per_turn × turns

Reaching tool output the ctx_* tools can't wrap

The ctx_* tools only compress their own results — they cannot wrap the output of another MCP server's tools (e.g. a host's store/artifact tools). The proxy can: it sits at the provider API and compresses every tool_result in the request body regardless of which tool produced it.

lean-ctx proxy enable        # redirect the provider base URL through lean-ctx

So if the heaviest token sink arrives via another server's MCP tools, route the provider through the proxy rather than relying on the tool layer. (On a non-caching provider the proxy shrinks what is sent; it cannot un-bill a prefix the client re-sends each turn.)

Recommended config for a phase-isolated / non-caching harness

rules_injection = "off"       # host supplies its own steering — write no rules file (#361)
LEAN_CTX_MINIMAL=1 lean-ctx serve --daemon     # trim the tool surface to the core set

Make every cold read carry its weight by defaulting to a compressing read mode via a persona (default_read_mode = "map" or "signatures"), since there are no warm re-reads to harvest. The cold-read modes are the right lever here.


Tuning checklist

Constraint Knob
Low-RAM / small CI runner memory_profile = "conservative"
Index eats too much disk bm25_max_cache_mb + extra_ignore_patterns
Giant monorepo, slow graph graph_index_max_files = <N>
No index needed at all LEAN_CTX_NO_INDEX=1
Hard disk/RAM ceiling LEAN_CTX_MAX_DISK_MB / LEAN_CTX_MAX_RAM_PERCENT
"What's slow?" slow-log list + gain --deep
Reclaim space now cache prune
Host supplies its own steering rules_injection = "off"
Phase-isolated / non-caching harness LEAN_CTX_MINIMAL=1 + persona default_read_mode = "map" + proxy enable
Reach another server's tool output lean-ctx proxy enable