Skip to content

Collector reference: harness store locations and formats for ~50 agents, from tokscale's client registry #214

Description

@willwashburn

Summary

#177 item 4 asks for one collector issue per harness in burn's backlog (Gemini CLI, Copilot CLI, VS Code Copilot Chat, Qwen, Kiro, Kimi, Hermes, OpenClaw, Every Code, Droid, Pi, Amp, Roo/Kilo, Mux, Crush, OpenHands, iFlow, Cortex, Octofriend). tokscale (MIT, commit d8fd670, 2026-09-18) ships parsers for most of these and its registry (crates/tokscale-core/src/clients.rs, define_clients!) plus scanner.rs is the most complete public map of where they store transcripts. This issue records that map so each collector issue can start from a known path and format instead of re-discovering it. tokscale extracts only usage fields, so its parsers tell us where and what shape, not how to get text, tool calls and edits; each collector still needs its own characterization.

Format legend: J = JSONL transcript, JS = JSON per session/message, DB = SQLite, X = only reachable through a vendor API or a tokscale-specific sync cache (not a local transcript; not a relayhistory candidate).

Harness Root (env override) Files Fmt tokscale parser Notes
Gemini CLI $GEMINI_CLI_HOME else ~/.gemini tmp/*/chats/*.json JS sessions/gemini.rs (1.4k lines) project dir under tmp/
Antigravity CLI same Gemini home antigravity-cli/conversations/*.db DB sessions/antigravity_cli.rs read directly
GitHub Copilot CLI ~/.copilot session-store.db (usage events), otel/*.jsonl (+COPILOT_OTEL_FILE_EXPORTER_PATH), data.db DB + J sessions/copilot.rs (2.2k) session-store.db needs no OTEL setup
VS Code Copilot Chat VS Code globalStorage / workspaceStorage/*/chatSessions chatSessions v3 with kind-1 incremental updates JS sessions/copilot_desktop.rs (1.6k) see scanner.rs:1754 discover_copilot_vscode_sessions
Qwen Code ~/.qwen/projects **/*.jsonl J sessions/qwen.rs Claude-Code-like layout
Kiro ~/.kiro/sessions/cli/*.json(l); ~/.local/share/kiro-cli/data.sqlite3 (macOS ~/Library/Application Support/kiro-cli/); IDE Kiro/User/globalStorage/kiro.kiroagent JS + DB sessions/kiro.rs (6k) three sources
Kimi CLI / Kimi Code ~/.kimi/sessions (wire.jsonl); ~/.kimi-code/sessions ($KIMI_CODE_HOME); desktop app-data J sessions/kimi.rs (1.3k)
Hermes Agent $HERMES_HOME else ~/.hermes state.db, profiles/*/state.db DB sessions/hermes.rs scanner.rs:1061
OpenClaw ~/.openclaw/agents (+ legacy .clawdbot, .moltbot, .moldbot) */agent/openclaw-agent.sqlite (live), */sessions/*.jsonl (legacy) + .jsonl.deleted.*, .jsonl.reset.*, .zst, checkpoint files to exclude DB + J sessions/openclaw.rs (2.2k) scanner.rs:399-432, 506-530 for the suffix rules; Codex-driven threads carry originator: "openclaw" in Codex rollouts
Droid (Factory) ~/.factory/sessions *.settings.json + transcript JS sessions/droid.rs (1.2k)
Pi ~/.pi/agent/sessions *.jsonl J sessions/pi.rs (1.3k)
Oh My Pi ~/.omp/agent/sessions **/*.jsonl J sessions/omp.rs Pi fork
Amp $XDG_DATA_HOME/amp/threads T-*.json JS sessions/amp.rs
Roo Code ~/.config/Code/User/globalStorage/rooveterinaryinc.roo-cline/tasks (+ ~/.vscode-server/data/...) ui_messages.json, api_conversation_history.json JS sessions/roocode.rs macOS ~/Library/Application Support/Code/..., Windows %APPDATA%\Code\...
Kilo Code (VS Code) .../globalStorage/kilocode.kilo-code/tasks same as Roo JS sessions/kilocode.rs
Kilo CLI $XDG_DATA_HOME/kilo/kilo.db DB sessions/kilo.rs OpenCode schema (opencode_schema.rs)
Cline same globalStorage pattern, saoudrizwan.claude-dev/tasks; CLI: $CLINE_SESSION_DATA_DIR, $CLINE_DATA_DIR/sessions, $CLINE_DIR/data/sessions ui_messages.json, api_conversation_history.json JS sessions/cline.rs scanner.rs:1433-1480
Mux ~/.mux/sessions session-usage.json JS sessions/mux.rs
Crush $XDG_DATA_HOME/crush/projects.json registry → per-project data dir DB DB sessions/crush.rs scanner.rs:1345-1430
Goose $XDG_DATA_HOME/goose/sessions/sessions.db (macOS App Support, GOOSE_PATH_ROOT) DB sessions/goose.rs
Zed Agent $XDG_DATA_HOME/zed/threads/threads.db (macOS ~/Library/Application Support/Zed/threads/, Win %LOCALAPPDATA%/Zed/threads/) DB sessions/zed.rs hosted models only
Octofriend ~/.local/share/octofriend/sqlite.db DB via synthetic reattribution
Codebuff / Freebuff $CODEBUFF_DATA_DIR else ~/.config/manicode (+-dev, -staging) chat-messages.json JS sessions/codebuff.rs
Devin CLI / Desktop $XDG_DATA_HOME/devin/cli/sessions.db; desktop ~/Library/Application Support/Devin/User/acp-events/*.ndjson (Linux ~/.config/Devin/...) DB + NDJSON sessions/devin.rs ACP event stream
Junie ~/.junie/sessions/*/events.jsonl J sessions/junie.rs
Augment (Auggie) ~/.augment/sessions/*.json JS sessions/augment.rs chat_history array per session
Cherry Studio %APPDATA%\CherryStudio\Data\Agents\.claude\projects (macOS/Linux equivalents) Claude Code JSONL J reuses claudecode.rs a Claude Code host; PathRoot::AppData
Command Code ~/.commandcode/projects/**/*.jsonl J sessions/commandcode.rs Claude-Code-like, v3 has per-request usage
ZCode ~/.zcode/cli/db/db.sqlite, legacy ~/.zcode/projects/**/*.jsonl DB + J sessions/zcode.rs
CodeBuddy / WorkBuddy ~/.codebuddy/projects/**/*.jsonl; ~/.workbuddy(-ai)/projects/**/*.jsonl + workbuddy.db J / DB sessions/codebuddy.rs, workbuddy.rs
MiMo Code $XDG_DATA_HOME/mimocode/mimocode(-<channel>).db DB sessions/micode.rs OpenCode-style channel DBs
Jcode $JCODE_HOME else ~/.jcode/sessions/session_*.json + session_*.journal.jsonl JS + J sessions/jcode.rs
DeepSeek Harness $DSH_HOME else ~/.dsh/sessions/**/session[.vN].jsonl[.zstd] J (zstd) sessions/dsh.rs
Prime Agent $PRIME_AGENT_CODING_AGENT_DIR else ~/.prime/agent/sessions, session-artifacts/ (RLM children) J sessions/prime_agent.rs (2.6k)
Senpi / Kimchi / gajae-code / Reasonix / Hindsight / LM Studio / Unsloth / fx see README table mixed one module each niche
Cursor IDE ~/.cursor/projects/*/agent-transcripts/<id>/<id>.jsonl (relayhistory) J none tokscale does not parse ~/.cursor; it pulls the dashboard API with a token from state.vscdb (tokscale-cli/src/cursor.rs:337-460). Nothing to borrow for #166
Antigravity IDE, Trae, Warp X vendor API sync caches only

Not in tokscale: Every Code, OpenHands, iFlow, Cortex (from burn's list) — those still need first-hand research.

Cross-cutting rules worth copying when the registry (#177) is designed:

  • Roots are PathRoot::{Home, XdgData, Config, AppData, EnvVar{var, fallback_relative}} resolved once, with a use_env_roots switch for explicit --home runs (clients.rs:1-18, 100-170).
  • VS Code extension stores have four spellings per OS plus ~/.vscode-server/data/User/globalStorage (scanner.rs:1433-1480).
  • Several harnesses adopted OpenCode's SQLite schema (sessions/opencode_schema.rs: Kilo CLI, MiMo Code, and others), so one driver with a config struct serves them all — the same shape [G1] OpenCode: event-level parity across both storage layouts (SQLite opencode.db and legacy JSON storage/), with tokens, model/provider, parts, step-finish, compaction, parentID #168's "common normalizer" proposes.
  • Archive / backup / checkpoint copies of transcripts (.jsonl.deleted.<ts>, .jsonl.reset.<ts>, .checkpoint.<uuid>.jsonl, .bak) exist for OpenClaw and must be excluded or deduplicated explicitly (scanner.rs:399-432, 506-530).

Proposed use

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions