Skip to content

README.md

Latest commit

 

History

History
672 lines (467 loc) · 15.9 KB

README.md

File metadata and controls

672 lines (467 loc) · 15.9 KB

Remi

CI Release License

Unified coding-agent session memory for Pi, Factory Droid, OpenCode, Claude Code, Amp, and Codex, plus separate local-document indexing/search in the same SQLite database.

Tired of hunting through multiple agent transcript formats and folders? Remi gives you one searchable memory layer.

Remi ingests local agent transcripts into one SQLite database, keeps sync state with checkpoints, supports ranked session search, lets you index local documentation roots for separate docs search, and provides safe archive/restore workflows.

Current support status: Linux-first (default source discovery paths are validated on Linux).

Table of contents

What Remi does

  • Incremental sync from multiple coding-agent sources.
  • Deterministic IDs (blake3) for idempotent upserts.
  • Checkpointed ingestion using a composite cursor (timestamp + source_id) to avoid missing same-timestamp records.
  • Structured content normalization including tool-call/tool-result payloads into searchable text.
  • Separate local docs indexing for user-selected directories, stored in the same SQLite DB as sessions.
  • Lexical search using SQLite FTS5 + BM25.
  • Ranking fusion with recency via Reciprocal Rank Fusion (RRF).
  • Substring fallback when lexical matches are empty.
  • Two-layer search UX: ranked session list first, then export selected session to HTML/Markdown (or emit JSON).
  • Dedicated docs search UX: query indexed docs directly without mixing them into session search results.
  • Archive planning/execution/restore with dry-run defaults and verification before optional deletion.

Demo

Full CLI demo (55s):

Remi CLI demo

Quickstart

remi init && remi sync --agent all && remi search query "panic"
remi docs index --root ~/docs && remi docs search "retry budget"

If remi is not on your path yet, run from source:

cargo run -p cli -- init
cargo run -p cli -- sync --agent all
cargo run -p cli -- search query "panic"
cargo run -p cli -- docs index --root ~/docs
cargo run -p cli -- docs search "retry budget"

Install / build

Install from source

cargo install --path crates/cli

Install from git (optional)

cargo install --git https://github.com/lsj5031/Remi --bin remi

Build workspace

cargo build --workspace

Run from source

cargo run -p cli -- --help

Data locations

Default paths on Linux (via dirs crate):

  • Database: ~/.local/share/remi/remi.db
  • Search exports (HTML/Markdown default output): ~/.local/share/remi/exports/
  • Archive bundles: ~/.local/share/remi/archive/<run_id>/

The same database stores both synced sessions and indexed local docs. Docs roots are user-selected via remi docs index --root <PATH> rather than auto-discovered.

macOS and Windows builds are available in releases, but default agent source discovery paths are currently Linux-oriented.

Supported agent sources

Remi currently discovers and ingests from:

Agent Paths scanned
Pi ~/.pi/agent/sessions/**/*.jsonl, ~/.pi/sessions/**/*.jsonl
Factory Droid ~/.factory/sessions/**/*.jsonl, ~/.local/share/factory-droid/sessions/**/*.jsonl
OpenCode ~/.local/share/opencode/opencode.db (preferred) or legacy ~/.local/share/opencode/storage/message/**/*.json (+ part text from ~/.local/share/opencode/storage/part/<message_id>/*.json; session metadata from ~/.local/share/opencode/storage/session/**/*.json)
Claude Code ~/.claude/transcripts/**/*.jsonl, ~/.claude/projects/**/*.jsonl, ~/.local/share/claude-code/**/*.jsonl
Amp ~/.local/share/amp/threads/**/*.json
Codex ~/.codex/sessions/**/*.jsonl

CLI reference

Top-level commands:

remi init
remi sync --agent <pi|droid|opencode|claude|amp|codex|all>
remi docs <index|search>
remi sessions <list|show>
remi search query <QUERY> [options]
remi archive <plan|run|restore>
remi doctor

If built with --features semantic, Remi also supports:

  • remi embed --rebuild
  • Global flags: remi --ort-dylib-path <PATH> ... and remi --auto-ort ...

remi init

Initializes/open database schema (schema is also initialized automatically by other commands).

remi init

remi sync

Sync a specific adapter or all adapters:

remi sync --agent pi
remi sync --agent droid
remi sync --agent opencode
remi sync --agent claude
remi sync --agent amp
remi sync --agent codex
remi sync --agent all

Behavior:

  • Discovers source files.
  • Scans only records after the last checkpoint.
  • Normalizes to canonical sessions/messages/provenance.
  • Upserts into SQLite + refreshes FTS rows for touched sessions.
  • Updates checkpoint cursor.

remi docs

Usage:

remi docs index --root <PATH>
remi docs search [OPTIONS] <QUERY>

docs index indexes one local directory root at a time. You can rerun it for the same root to refresh incrementally, or run it again with another root to add more docs roots to the same DB.

Supported document file types:

  • .md
  • .markdown
  • .txt
  • .rst

Indexing behavior:

  • canonicalizes the requested root path before storing it
  • skips hidden files/directories
  • skips symlinks
  • skips unreadable or non-UTF-8 files
  • updates changed files in place
  • treats rename/delete as reconciliation on the next successful re-index

Example:

remi docs index --root ~/notes/project-docs

Typical output:

root=/home/alice/notes/project-docs
indexed=42
updated=3
skipped=5
deleted=1
errors=0

docs search searches only indexed docs; it does not mix docs into remi search query session results.

Options:

  • --raw-fts
  • --limit <N> (default: 20)

Examples:

remi docs search "retry budget"
remi docs search "fts5 NEAR tokenizer" --raw-fts --limit 10

Output includes the doc title, relative path, and a content snippet for each hit.

remi sessions

List sessions:

remi sessions list

Show one session’s messages:

remi sessions show <session_id>

Example:

remi sessions show 0d5f0e...c9a

remi search query

Usage:

remi search query [OPTIONS] <QUERY>

Options:

  • --format <html|markdown|json> (default: html)
  • --no-interactive
  • --select <auto|index> (default: auto)
  • --index <N> (required when --select index in non-interactive mode)
  • --agent <STRING>
  • --title <STRING>
  • --id <STRING>
  • --contains <STRING>
  • --raw-fts
  • --html-safety <strict|relaxed|trusted> (default: relaxed)
  • --output-dir <PATH>

Interactive mode (default)

remi search query "retry logic"

Flow:

  1. Remi ranks matching sessions.
  2. You can type an optional fuzzy filter.
  3. You choose an index.
  4. Remi exports selected session (HTML by default) and prints the file path.

Interactive fuzzy filter supports field prefixes:

agent:claude title:auth contains:oauth refresh token

Supported fuzzy fields:

  • agent:
  • title:
  • id:
  • contains:

Non-interactive examples

Auto-select top result, export Markdown:

remi search query "panic on startup" \
  --no-interactive \
  --select auto \
  --format markdown

Select exact ranked session index:

remi search query "sql migration" \
  --no-interactive \
  --select index \
  --index 2 \
  --format html \
  --output-dir ./exports

Emit JSON instead of writing HTML/Markdown:

remi search query "cache invalidation" \
  --no-interactive \
  --select auto \
  --format json

Filter sessions before selection:

remi search query "build failure" \
  --no-interactive \
  --agent droid \
  --title release \
  --contains linker

remi archive

1) Create an archive plan

remi archive plan --older-than 30d --keep-latest 5
  • --older-than uses human duration parsing (examples: 30d, 12h, 90m)
  • --keep-latest is applied per agent
  • Output format includes: plan <run_id>

2) Dry-run an archive run (default behavior)

remi archive run --plan <run_id>

Equivalent explicit dry-run:

remi archive run --plan <run_id> --dry-run

3) Execute archive

remi archive run --plan <run_id> --execute

This writes:

  • ~/.local/share/remi/archive/<run_id>/sessions.json
  • ~/.local/share/remi/archive/<run_id>/manifest.json

Remi verifies the bundle checksum after writing before allowing deletion.

4) Execute and delete source sessions from DB

remi archive run --plan <run_id> --execute --delete-source

Flag precedence:

  • Writes/deletes only occur when --execute is set.
  • If both --execute and --dry-run are passed, --dry-run wins and execution is suppressed.

5) Restore from a bundle

remi archive restore --bundle ~/.local/share/remi/archive/<run_id>/sessions.json

remi doctor

Run integrity checks and basic stats:

remi doctor

Current output includes:

  • SQLite PRAGMA integrity_check result
  • total session count

Semantic search (optional feature)

Semantic support is feature-gated at compile time.

Build with semantic feature

cargo build -p cli --features semantic

When built with semantic, additional CLI surface is enabled:

  • Global flags on remi:
    • --ort-dylib-path <PATH>
    • --auto-ort
  • remi embed --rebuild
  • remi search query ... --semantic <auto|on|off>

Semantic config

~/.config/remi/config.toml:

[semantic]
enabled = true
model_path = "/path/to/bge-small-en-v1.5"
pooling = "cls" # or "mean"
query_prefix = "Represent this sentence for searching relevant passages: "

Model directory must contain:

  • model.onnx
  • tokenizer.json

Auto-detected model locations

If model_path is not set, Remi checks:

  • <binary_dir>/models/bge-small-en-v1.5
  • <binary_dir>/model
  • ~/.cache/remi/bge-small-en-v1.5

Semantic command examples

Rebuild embeddings:

remi embed --rebuild

Search with semantic mode:

remi search query "memoization strategy" --semantic auto
remi search query "memoization strategy" --semantic on
remi search query "memoization strategy" --semantic off

Set ONNX Runtime path explicitly:

remi --ort-dylib-path /opt/onnx/libonnxruntime.so search query "vector index" --semantic on

Auto-detect ONNX Runtime shared library:

remi --auto-ort search query "vector index" --semantic auto

End-to-end workflow examples

Workflow A: First-time ingest + browse

remi init
remi sync --agent all
remi sessions list

Then inspect a session:

remi sessions show <session_id>

Workflow B: Investigate an issue across all agents

remi search query "panic: index out of bounds" --format markdown

Open the emitted .md export path and review the full conversation.

Workflow C: CI/script-friendly JSON output

remi search query "release tagging" --no-interactive --format json --select auto

Workflow D: Safe archival lifecycle

# plan
remi archive plan --older-than 60d --keep-latest 10

# inspect impact
remi archive run --plan <run_id>

# execute archive without deletion
remi archive run --plan <run_id> --execute

# optional deletion after validation
remi archive run --plan <run_id> --execute --delete-source

Restore when needed:

remi archive restore --bundle ~/.local/share/remi/archive/<run_id>/sessions.json

Workflow E: Index and search a docs corpus

remi docs index --root ~/docs/remi-notes
remi docs search "retry budget"

If you rename or remove files under the indexed root, rerun:

remi docs index --root ~/docs/remi-notes

and Remi will reconcile deleted/renamed paths before the next docs search.

Helper scripts (examples)

This repo includes helper scripts in scripts/:

  • scripts/remi-diary.sh — generate a daily Markdown summary from remi.db, with optional Telegram send.
  • scripts/markie-export.sh — render Markdown to SVG/PNG using local markie CLI (MarkieCli).

Example diary usage:

# Generate today's diary markdown
scripts/remi-diary.sh --date today

# Generate today's diary with structured progress events
DIARY_PROGRESS_JSONL=/tmp/remi-diary.progress.jsonl scripts/remi-diary.sh --date today --sync

# Generate and send yesterday's diary image to Telegram
scripts/remi-diary.sh --date yesterday --send

Example Markdown → SVG rendering:

scripts/markie-export.sh \
  --input ~/diary/remi/$(date +%F).md \
  --output /tmp/remi-diary.svg \
  --format svg

remi-diary.sh prints a safety warning by default because it may send transcript data to external services:

  • your configured summary command/provider (default: codex exec ...) for summary generation
  • Telegram (when --send is enabled)

If you intentionally want to suppress the warning:

DIARY_SKIP_EXTERNAL_WARNING=1 scripts/remi-diary.sh --date today

If you want machine-readable progress for timers or other automation, set DIARY_PROGRESS_JSONL or pass --progress-jsonl <path>; the script will append one JSON object per phase transition, including per-agent sync events.

Use DIARY_* environment overrides (shown in scripts/remi-diary.sh --help) to adapt paths/commands for your environment.

Architecture at a glance

Workspace crates:

  • core-model: canonical types + adapter trait + deterministic IDs
  • store-sqlite: SQLite schema, session/doc upserts, FTS index maintenance, archive planning helpers
  • ingest: sync orchestration with progress phases
  • search: session ranking plus separate docs lexical/substring search helpers
  • archive: plan/run/restore archive workflows
  • adapter-common (at crates/adapters/common): shared file/JSON parsing + cursor logic
  • adapters/{pi,droid,opencode,claude,amp,codex}: per-agent ingestion adapters
  • embeddings (optional): ONNX + tokenizer embedding generation
  • cli: remi command-line interface for session sync/search, docs index/search, archive, and doctor flows

Release artifacts

GitHub release workflow publishes:

  • remi-linux-x64-simple.tar.gz (binary only)
  • remi-linux-x64-bundled.tar.gz (binary + ONNX Runtime + BGE model files)

As of v0.1.0 (April 1, 2026), published assets are Linux x64. The workflow in this repo is set up to also produce macOS/Windows simple artifacts on future tagged releases.

Homebrew (optional)

Homebrew tap setup (recommended layout):

  1. Create a tap repo, e.g. lsj5031/homebrew-remi.
  2. Start from this repo’s Formula/remi.rb and update URL/version/SHA per tagged release.
  3. Update SHA256 per release.

End-user install:

brew tap lsj5031/remi
brew install remi

License

Dual-licensed under:

  • MIT
  • Apache-2.0