diff --git a/CHANGELOG.md b/CHANGELOG.md index 68ad6f94..3d4fbcd6 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -81,6 +81,23 @@ Notable changes to the native `ai-hist` CLI are documented here. ### Added +- Muse Code (Meta's `muse` CLI) is a first-class source, `muse`. Sessions are + read from `$XDG_DATA_HOME/muse/sessions/YYYY/MM/DD//session.jsonl` + (`~/.local/share/muse/sessions` by default) by discovery, `sync`, + targeted hydration and live capture: typed prompts become `history` rows at + the microsecond time Muse recorded them, and each session gets its prose, + readable thinking, tool calls and results (status from Muse's own + `tool_batch.effect.terminal` outcome, and a non-zero `bash` exit code), + `write_file` / `edit_file` edits, per-model-step token usage (normalized as + `per-request`), models, CLI version and lifecycle markers (`turn_end`, + `session_start`, `session_resumed`, `session_end`, `model_switch`, + `encrypted_reasoning`). Subagent and reminder logs + (`subagent//session.jsonl`, at any depth) are indexed under the child's + own session id and linked to their parent as `delegated` children, typed + from the parent's `task_stream_linked` record; they are not catalogued as + sessions and their objectives are not history rows. A Muse hydration + reports `full`. `ai-hist resume` prints `muse resume `, and the sync + service forwards `XDG_DATA_HOME`. - Read every OpenCode channel database. OpenCode keeps one SQLite store per release channel — `opencode.db` for `latest`/`beta`, and `opencode-stable.db`, `opencode-nightly.db`, ... beside it — and only @@ -406,6 +423,9 @@ Notable changes to the native `ai-hist` CLI are documented here. ### Rust API +- `Source::Muse` and `ProviderRoots::muse` (the Muse Code sessions directory; + `from_env` honours `XDG_DATA_HOME`). Both types are `#[non_exhaustive]`, so + this is additive. - `ProviderRoots` gains `opencode_db_pinned: bool`. `from_env` sets it when `OPENCODE_DB` is set; `from_home` leaves it `false`, so every OpenCode channel database beside `opencode_db` is read. Set it to read `opencode_db` alone. diff --git a/README.md b/README.md index 11073f69..9b905975 100644 --- a/README.md +++ b/README.md @@ -2,7 +2,7 @@ **Search, resume, and hand off every coding-agent session — across every harness on your machine.** -Local memory for **Claude Code**, **Codex**, **Cursor**, **Grok**, **OpenCode**, and [**Agent Relay**](https://github.com/AgentWorkforce/relay). Your agent sessions are indexed into one local SQLite database. Search them all at once, jump back into a session with its harness's native resume command, and hand a compact context pack to another agent — or to a teammate's agent — to continue the work. +Local memory for **Claude Code**, **Codex**, **Cursor**, **Grok**, **Muse Code**, **OpenCode**, and [**Agent Relay**](https://github.com/AgentWorkforce/relay). Your agent sessions are indexed into one local SQLite database. Search them all at once, jump back into a session with its harness's native resume command, and hand a compact context pack to another agent — or to a teammate's agent — to continue the work. ```sh ai-hist search "auth rewrite" @@ -31,7 +31,7 @@ ai-hist search "the thing i was working on" The bare `ai-hist` command bootstraps a searchable database on first use: it discovers your most recent local sessions and indexes their evidence, then tells you what it found. It leaves an already-populated database alone. Run `ai-hist sync` any time you want a full re-ingest rather than the bounded first-run pass. -RelayHistory follows each harness's configured state directory. Set `CLAUDE_CONFIG_DIR`, `CODEX_HOME`, or `GROK_HOME` to relocate Claude Code, Codex, or Grok data; unset or empty values fall back to `~/.claude`, `~/.codex`, and `~/.grok`. OpenCode uses `OPENCODE_DB`, defaulting to `~/.local/share/opencode/opencode.db`. +RelayHistory follows each harness's configured state directory. Set `CLAUDE_CONFIG_DIR`, `CODEX_HOME`, or `GROK_HOME` to relocate Claude Code, Codex, or Grok data; unset or empty values fall back to `~/.claude`, `~/.codex`, and `~/.grok`. Muse Code sessions are read from `$XDG_DATA_HOME/muse/sessions` (`~/.local/share/muse/sessions` when `XDG_DATA_HOME` is unset). OpenCode uses `OPENCODE_DB`, defaulting to `~/.local/share/opencode/opencode.db`. Node.js 20 or 22 is required. `npm install` pulls a prebuilt native addon for macOS (arm64, x64), Linux glibc ≥ 2.28 and musl (arm64, x64), and Windows x64 — no Rust toolchain, compiler, or separate binary download. The glibc floor covers Debian 12, Ubuntu 22.04, Amazon Linux 2023, and RHEL/Alma 9; releases are smoke-tested on `node:22-bookworm-slim` and `ubuntu:22.04`. @@ -52,9 +52,9 @@ ai-hist recent 20 # the last N prompts, newes ai-hist stats # how much history is indexed, by source and project ``` -`` is the session's source — `claude`, `codex`, `cursor`, `grok`, `opencode`, or `relay` — and is required alongside the ID, because session IDs collide across providers. `ai-hist sessions list` prints both. +`` is the session's source — `claude`, `codex`, `cursor`, `grok`, `muse`, `opencode`, or `relay` — and is required alongside the ID, because session IDs collide across providers. `ai-hist sessions list` prints both. -`ai-hist resume` prints a native resume command for Claude Code, Codex, Cursor, and Grok sessions. OpenCode and Agent Relay sessions are searchable and packable, but have no native resume command to print, so use `ai-hist pack` to carry that context forward instead. +`ai-hist resume` prints a native resume command for Claude Code, Codex, Cursor, Grok, and Muse Code sessions. OpenCode and Agent Relay sessions are searchable and packable, but have no native resume command to print, so use `ai-hist pack` to carry that context forward instead. OpenCode is read from whichever of its two stores the machine has: the SQLite `opencode.db` that current releases write (`OPENCODE_DB`), or the older JSON @@ -114,7 +114,7 @@ Team uploads come from the [Agent Relay desktop app](https://agentrelay.com), no ## Why `ai-hist` -- **Every harness, one search.** Claude Code, Codex, Cursor, Grok, OpenCode, Agent Relay — indexed side-by-side. No per-harness silo. +- **Every harness, one search.** Claude Code, Codex, Cursor, Grok, Muse Code, OpenCode, Agent Relay — indexed side-by-side. No per-harness silo. - **Provider-aware evidence.** Prompts, tool calls, and edits are preserved as raw evidence, not summarized away — as much of it as each harness actually exposes. Hydration reports `full`, `partial`, or `shallow_only` per session, so you can tell thin coverage from a thing that never happened. Where a harness records less than the others, the gap is named. Cursor transcripts carry the assistant's prose and every tool call, but no tool output, model id, token usage or timestamp field — those are reported as unavailable, and a turn whose injected `` tag cannot be read is stamped from the file mtime with `CURSOR_TIMESTAMP_FROM_MTIME`. Older Grok builds log no per-turn billing tokens, so their only token fact is a context-window proxy, and hydration says so; newer builds' per-turn `usage` breakdown is recorded beside it, never added to it. The per-field detail is in [the session catalog](docs/session-catalog.md). - **Local by default.** SQLite on your machine. Export requires an explicit selection; remote acquisition requires an installed source plugin. - **Handoff-native.** `pack` and `resume` are first-class commands, not afterthoughts. diff --git a/crates/ai-hist-cli/src/lib.rs b/crates/ai-hist-cli/src/lib.rs index 4bb7d6c9..33dfb682 100644 --- a/crates/ai-hist-cli/src/lib.rs +++ b/crates/ai-hist-cli/src/lib.rs @@ -417,7 +417,7 @@ enum SessionsAction { /// content blocks, agent lifecycle events. Same `(ts_ms IS NULL, ts_ms, /// id)` keyset as tool calls and file edits; undated markers page last. Markers { - /// Coding-agent source (claude, codex, cursor, grok, relay, opencode). + /// Coding-agent source (claude, codex, cursor, grok, muse, relay, opencode). source: String, /// Native session identifier within that source. session_id: String, @@ -441,7 +441,7 @@ enum SessionsAction { /// and cost is never computed: `reported_cost_usd` appears only when the /// source data carried one. Usage { - /// Coding-agent source (claude, codex, cursor, grok, relay, opencode). + /// Coding-agent source (claude, codex, cursor, grok, muse, relay, opencode). source: String, /// Native session identifier within that source. session_id: String, @@ -479,7 +479,7 @@ enum SessionsAction { enum LearnAction { /// Distill local session history into decision/finding/reflection events. Distill { - /// Only distill sessions from this source (claude, codex, cursor, grok, relay, opencode). + /// Only distill sessions from this source (claude, codex, cursor, grok, muse, relay, opencode). #[arg(long)] source: Option, /// Distill one session id. @@ -2901,11 +2901,13 @@ fn service_command_args(spec: &ServiceSpec, args: &[String]) -> Vec { command } -const PROVIDER_ENV_VARS: [&str; 4] = [ +const PROVIDER_ENV_VARS: [&str; 5] = [ "CLAUDE_CONFIG_DIR", "CODEX_HOME", "GROK_HOME", "OPENCODE_DB", + // Muse Code keeps its sessions under `$XDG_DATA_HOME/muse/sessions`. + "XDG_DATA_HOME", ]; fn scheduler_environment_value(name: &str, value: std::ffi::OsString) -> Result> { diff --git a/crates/ai-hist-napi/src/lib.rs b/crates/ai-hist-napi/src/lib.rs index 3b8b8127..a64cf40f 100644 --- a/crates/ai-hist-napi/src/lib.rs +++ b/crates/ai-hist-napi/src/lib.rs @@ -1736,7 +1736,9 @@ pub async fn hydrate_session(options: HydrateSessionOptions) -> napi::Result { pub codex_home: PathBuf, /// Grok state root. pub grok_home: PathBuf, + /// Muse Code session logs. + pub muse_sessions: PathBuf, /// Path to the opencode database. pub opencode_db: PathBuf, /// Whether `opencode_db` is the only OpenCode database to read, rather @@ -341,6 +343,7 @@ impl<'a> DiscoveryEnv<'a> { claude_config_dir: roots.claude, codex_home: roots.codex, grok_home: roots.grok, + muse_sessions: roots.muse, opencode_db: roots.opencode_db, opencode_db_pinned: roots.opencode_db_pinned, opencode_storage_dir: roots.opencode_storage_dir, @@ -369,9 +372,11 @@ impl<'a> DiscoveryEnv<'a> { .parent() .map(|parent| parent.join("storage")) .unwrap_or_else(|| home.join(".local/share/opencode/storage")); + let muse = crate::paths::default_muse_sessions_dir(&home); Self::with_provider_roots( conn, crate::ProviderRoots { + muse, home, claude: claude_config_dir, codex: codex_home, @@ -409,6 +414,7 @@ impl<'a> DiscoveryEnv<'a> { claude_config_dir: &self.claude_config_dir, codex_home: &self.codex_home, grok_home: &self.grok_home, + muse_sessions: &self.muse_sessions, opencode_db: &self.opencode_db, opencode_db_pinned: self.opencode_db_pinned, opencode_storage_dir: &self.opencode_storage_dir, @@ -451,6 +457,8 @@ pub struct ScanEnv<'a> { pub codex_home: &'a Path, /// Grok state root. pub grok_home: &'a Path, + /// Muse Code session logs. + pub muse_sessions: &'a Path, /// Path to the opencode database. pub opencode_db: &'a Path, /// Whether `opencode_db` is the only OpenCode database to read. @@ -500,6 +508,8 @@ pub struct ProviderRoots<'a> { pub codex: &'a Path, /// Grok state root. pub grok: &'a Path, + /// Muse Code session logs. + pub muse: &'a Path, /// Path to the opencode database. pub opencode_db: &'a Path, } @@ -1009,6 +1019,7 @@ pub fn shallow_providers() -> Vec> { Box::new(CodexProvider), Box::new(CursorProvider), Box::new(GrokProvider), + Box::new(MuseProvider), Box::new(OpencodeProvider::default()), Box::new(RelayProvider), ] @@ -2063,6 +2074,141 @@ fn grok_update_bounds( Ok((first, last)) } +// --------------------------------------------------------------------------- +// muse +// --------------------------------------------------------------------------- + +/// Muse Code: one append-only `session.jsonl` per session under +/// `muse/sessions/YYYY/MM/DD//`. Subagent and reminder children +/// write their own transcripts under `subagent/` beside the parent; those are +/// not sessions of their own and are never enumerated. +struct MuseProvider; + +impl ShallowSessionProvider for MuseProvider { + fn acquire( + &self, + _home: &Path, + _observation: &crate::observations::SessionObservation, + ) -> Result { + Ok(crate::sources::AcquiredEvidence::LocalFiles) + } + fn source(&self) -> &'static str { + "muse" + } + /// The transcript parser writes prompts, events, tool calls, file edits + /// and `delegated` edges to the subagent logs beside the session: every + /// kind a full session is made of. + fn evidence_kinds(&self) -> &'static [EvidenceKind] { + FULL_SESSION_KINDS + } + + fn watch_roots(&self, roots: &ProviderRoots<'_>) -> Vec { + vec![WatchRoot::tree(roots.muse.to_path_buf())] + } + + fn enumerate( + &self, + env: &DiscoveryEnv<'_>, + _requested_limit: Option, + ) -> Result> { + file_candidates( + "muse", + crate::collect_muse_transcripts(&env.muse_sessions)?, + crate::file_stamp_and_modified, + ) + } + + /// Enumeration is top-level transcripts only, but a subagent log is + /// evidence too: sync stamps it, and a background subagent keeps writing + /// after its parent has stopped. Left out of the fold, a tick whose only + /// change was a child log would sit behind an unchanged fingerprint and + /// never reach the sweep that reads it. + fn fingerprint_inputs(&self, env: &DiscoveryEnv<'_>) -> Result> { + let mut files = Vec::new(); + for transcript in crate::collect_muse_transcripts(&env.muse_sessions)? { + match crate::muse_session_files(&transcript) { + Ok(tree) => files.extend(tree), + Err(error) if error.is::() => return Err(error), + // One session's child log that cannot be statted must not fail + // the whole fold. Its transcript still counts; the sweep reads + // that session, fails it alone, and — having left a source + // unread — does not cache this fingerprint, so it is retried. + Err(_) => files.push(transcript), + } + } + file_candidates("muse", files, crate::file_stamp_and_modified) + } + + fn read_shallow( + &self, + scan: &ScanEnv<'_>, + _catalog: Option<&Connection>, + candidate: &Candidate, + ) -> Result> { + use crate::ingest::muse; + let path = PathBuf::from(&candidate.locator); + // A subagent's log is never a session of its own, however the + // candidate arrived — enumeration skips them, a watch hit or a + // by-path read need not. + if muse::is_subagent_log(&path) { + return Ok(None); + } + let bounded = read_bounded_jsonl(scan, &path)?; + // A permission frame can precede the metadata record, so the header + // is looked for rather than assumed to be the first line. + let Some(metadata) = bounded + .head_records() + .filter_map(parse_record) + .find_map(|record| muse::parse_metadata(&record)) + else { + return Ok(None); + }; + let own_stream = |record: &Value| { + record.pointer("/stream/id").and_then(Value::as_str) + == Some(metadata.session_id.as_str()) + }; + let mut models = Vec::new(); + push_unique(&mut models, metadata.model_id.as_deref()); + let mut first_prompt = None; + for record in bounded.head_records().filter_map(parse_record) { + if !own_stream(&record) { + continue; + } + match muse::parse_event(&record) { + Some(muse::MuseEvent::Prompt { text }) if first_prompt.is_none() => { + first_prompt = Some(excerpt(&text)); + } + Some(muse::MuseEvent::ModelCompleted { model, .. }) => { + push_unique(&mut models, model.as_deref()); + } + _ => {} + } + if first_prompt.is_some() && models.len() > 1 { + break; + } + } + let last_activity_ms = bounded + .tail_records_rev() + .filter_map(parse_record) + .filter(own_stream) + .find_map(|record| muse::recorded_ms(&record)); + Ok(Some(ShallowSession { + source: "muse".into(), + session_id: metadata.session_id, + cwd: metadata.workspace_root, + first_activity_ms: metadata.recorded_ms, + last_activity_ms: last_activity_ms + .or(metadata.recorded_ms) + .or_else(|| crate::file_modified_ms(&path)), + first_prompt, + models, + agent_version: metadata.cli_version, + raw_path: Some(candidate.locator.clone()), + ..Default::default() + })) + } +} + // --------------------------------------------------------------------------- // opencode // --------------------------------------------------------------------------- @@ -3458,6 +3604,7 @@ pub(crate) fn provider_watch_roots(source: &str, roots: &crate::ProviderRoots) - claude: &roots.claude, codex: &roots.codex, grok: &roots.grok, + muse: &roots.muse, opencode_db: &roots.opencode_db, }, ) diff --git a/crates/ai-hist/src/discover/tests.rs b/crates/ai-hist/src/discover/tests.rs index f192bd64..88bca939 100644 --- a/crates/ai-hist/src/discover/tests.rs +++ b/crates/ai-hist/src/discover/tests.rs @@ -3867,6 +3867,97 @@ fn cancellation_during_row_emission_preserves_the_committed_window() { } } +/// A Muse subagent's log is never a session of its own, however it reaches +/// the shallow reader — enumeration skips it, a by-path read must too. +#[test] +fn muse_shallow_read_refuses_a_subagent_log() { + let home = tempfile::tempdir().unwrap(); + let conn = catalog(); + let session = home + .path() + .join(".local/share/muse/sessions/2026/09/20/parent"); + let header = |id: &str| { + format!( + "{{\"id\":\"r1\",\"stream\":{{\"kind\":\"session\",\"id\":\"{id}\"}},\ + \"recorded_at\":1790337600000000,\"payload_type\":\"runtime.session.metadata\",\ + \"payload\":{{\"kind\":\"metadata\",\"record\":{{\"workspace_root\":\"/w\"}}}}}}\n" + ) + }; + write(&session.join("session.jsonl"), &header("parent")); + let child = session.join("subagent/child/session.jsonl"); + write(&child, &header("child")); + + let env = env_at(&conn, home.path()); + let candidates = MuseProvider.enumerate(&env, None).unwrap(); + assert_eq!(candidates.len(), 1, "only the parent is enumerated"); + let mut by_path = candidates[0].clone(); + by_path.locator = child.to_string_lossy().into_owned(); + assert!(MuseProvider + .read_shallow(&env.scan(), None, &by_path) + .unwrap() + .is_none()); + assert_eq!( + MuseProvider + .read_shallow(&env.scan(), None, &candidates[0]) + .unwrap() + .unwrap() + .session_id, + "parent" + ); +} + +/// One session's child log that cannot be statted leaves that session's +/// transcript in the fingerprint and every other session's tree untouched, +/// rather than failing the fold. +#[cfg(unix)] +#[test] +fn a_muse_child_log_stat_error_does_not_fail_the_fingerprint() { + use std::os::unix::fs::PermissionsExt; + let home = tempfile::tempdir().unwrap(); + let conn = catalog(); + let sessions = home.path().join(".local/share/muse/sessions/2026/09/20"); + let header = |id: &str| { + format!( + "{{\"id\":\"r1\",\"stream\":{{\"kind\":\"session\",\"id\":\"{id}\"}},\ + \"recorded_at\":1790337600000000,\"payload_type\":\"runtime.session.metadata\",\ + \"payload\":{{\"kind\":\"metadata\",\"record\":{{\"workspace_root\":\"/w\"}}}}}}\n" + ) + }; + write(&sessions.join("a/session.jsonl"), &header("a")); + write(&sessions.join("a/subagent/c/session.jsonl"), &header("c")); + write(&sessions.join("b/session.jsonl"), &header("b")); + write(&sessions.join("b/subagent/d/session.jsonl"), &header("d")); + let blocked = sessions.join("a/subagent/c"); + fs::set_permissions(&blocked, fs::Permissions::from_mode(0o644)).unwrap(); + let stat_blocked = fs::metadata(blocked.join("session.jsonl")).is_err(); + + let env = env_at(&conn, home.path()); + let inputs = MuseProvider.fingerprint_inputs(&env); + fs::set_permissions(&blocked, fs::Permissions::from_mode(0o755)).unwrap(); + if !stat_blocked { + return; + } + let mut locators: Vec = inputs + .unwrap() + .into_iter() + .map(|candidate| { + candidate + .locator + .trim_start_matches(&*sessions.to_string_lossy()) + .to_string() + }) + .collect(); + locators.sort(); + assert_eq!( + locators, + vec![ + "/a/session.jsonl".to_string(), + "/b/session.jsonl".to_string(), + "/b/subagent/d/session.jsonl".to_string(), + ] + ); +} + /// A subagent workflow journal shares the `.jsonl` extension and sits inside /// the project tree, but it is not a transcript, so discovery never lists it /// as a candidate session. diff --git a/crates/ai-hist/src/ingest.rs b/crates/ai-hist/src/ingest.rs index 8a8e7928..8e12ee88 100644 --- a/crates/ai-hist/src/ingest.rs +++ b/crates/ai-hist/src/ingest.rs @@ -23,6 +23,7 @@ pub(crate) mod hook; pub(crate) mod hydrate; pub(crate) mod incremental; pub(crate) mod jsonl; +pub(crate) mod muse; pub(crate) mod opencode; pub(crate) mod tool_result_facts; pub(crate) mod transcript_cursor; @@ -61,10 +62,10 @@ pub use hydrate::{ }; pub use tool_result_facts::{ content_hash, stable_stringify, ToolResultFacts, ToolResultIndexer, ERROR_SIGNAL_EXIT_CODE, - ERROR_SIGNAL_MCP_ERR, ERROR_SIGNAL_PATCH_APPLY, ERROR_SIGNAL_SUBAGENT_STATUS, - ERROR_SIGNAL_TOOL_RESULT, ERROR_SIGNAL_TOOL_STATUS, EVENT_SOURCE_FUNCTION_CALL_OUTPUT, - EVENT_SOURCE_SUBAGENT_NOTIFICATION, EVENT_SOURCE_TOOL_RESULT, STATUS_COMPLETED, STATUS_ERRORED, - STATUS_UNKNOWN, + ERROR_SIGNAL_MCP_ERR, ERROR_SIGNAL_MUSE_TOOL_OUTCOME, ERROR_SIGNAL_PATCH_APPLY, + ERROR_SIGNAL_SUBAGENT_STATUS, ERROR_SIGNAL_TOOL_RESULT, ERROR_SIGNAL_TOOL_STATUS, + EVENT_SOURCE_FUNCTION_CALL_OUTPUT, EVENT_SOURCE_SUBAGENT_NOTIFICATION, + EVENT_SOURCE_TOOL_RESULT, STATUS_COMPLETED, STATUS_ERRORED, STATUS_UNKNOWN, }; use std::sync::atomic::{AtomicBool, AtomicU64, Ordering as AtomicOrdering}; @@ -766,7 +767,11 @@ const DESTINATION_MARKER_VERSION: &str = "v4"; /// sync now re-reads a session indexed as prompts-only, so a loss in one is /// repairable on the same terms. Its rows do not come from a flat log, which /// is what kept it out before. -const REPAIRABLE_EVENT_SOURCES: &[&str] = &["claude", "codex", "opencode"]; +/// +/// Muse Code joined with its parser: every session is a whole-transcript +/// read, and a sweep re-reads a session this marker names short, or the +/// parent whose tree holds a short subagent. +const REPAIRABLE_EVENT_SOURCES: &[&str] = &["claude", "codex", "opencode", "muse"]; /// What one session is expected to hold. /// @@ -1045,12 +1050,20 @@ fn destination_shortfall(conn: &Connection, stored: &str) -> Result Result 0 + let expected_subagent_deregistration = before.catalog > 0 && current.catalog == 0 - && codex_delegation_recorded(conn, &session_id)? - && !codex_session_has_remote_presence(conn, &session_id)?; + && match source.as_str() { + "codex" => { + codex_delegation_recorded(conn, &session_id)? + && !codex_session_has_remote_presence(conn, &session_id)? + } + // A Muse subagent is deregistered the same way once its log + // is linked to its parent. + "muse" => { + muse_delegation_recorded(conn, &session_id)? + && !session_has_remote_presence(conn, "muse", &session_id)? + } + _ => false, + }; let covered = current.events >= before.events && current.tool_calls >= before.tool_calls && current.file_edits >= before.file_edits @@ -1516,6 +1539,15 @@ fn sync_basic( total_inserted += inserted; checkpoint_sync_state(&state_path, &state); } + capture_progress("muse", 0, None); + check_capture_cancelled()?; + if let Some(inserted) = report.capture( + "muse", + sync_muse_with_coverage(conn, &mut state, &roots.muse, &repairs, &mut coverage), + ) { + total_inserted += inserted; + checkpoint_sync_state(&state_path, &state); + } capture_progress("trajectory", 0, None); check_capture_cancelled()?; if let Some(inserted) = report.capture( @@ -3262,12 +3294,19 @@ fn cleanup_codex_subagent_registration(conn: &Connection, session_id: &str) -> R } fn codex_session_has_remote_presence(conn: &Connection, session_id: &str) -> Result { + session_has_remote_presence(conn, "codex", session_id) +} + +/// Whether a session is also known remotely. Local and remote evidence share +/// one `(source, session_id)` identity, so a local re-read must not delete +/// rows a remote observation put there. +fn session_has_remote_presence(conn: &Connection, source: &str, session_id: &str) -> Result { conn.query_row( "SELECT EXISTS(\ SELECT 1 FROM session_presences \ - WHERE source = 'codex' AND session_id = ? AND location = 'remote'\ + WHERE source = ? AND session_id = ? AND location = 'remote'\ )", - [session_id], + params![source, session_id], |row| row.get(0), ) .map_err(Into::into) @@ -11243,23 +11282,59 @@ pub(crate) fn decode_cursor_project(name: &str) -> String { format!("/{}", name.replace('-', "/")) } +/// Where plain `sync` remembers the change stamp of each Muse Code transcript +/// it has already read. Renamed on a parser upgrade so unchanged transcripts +/// are read again; see [`GROK_SYNC_STATE_KEY`]. +const MUSE_SYNC_STATE_KEY: &str = "muse_sessions_v1"; + +/// Every top-level Muse Code transcript under `root`, sorted. +/// +/// A transcript under a `subagent/` directory belongs to a subagent or +/// reminder child of the session beside it, not to a session of its own, so +/// it is never returned. +pub(crate) fn collect_muse_transcripts(root: &Path) -> Result> { + Ok( + collect_matching_files(root, muse::SESSION_FILE.trim_end_matches(".jsonl"), "jsonl")? + .into_iter() + .filter(|path| { + path.file_name().and_then(|name| name.to_str()) == Some(muse::SESSION_FILE) + && !muse::is_child_transcript(path, root) + }) + .collect(), + ) +} + #[cfg(test)] -fn sync_grok(conn: &Connection, state: &mut Map, root: &Path) -> Result { - sync_grok_with_coverage(conn, state, root, &mut SweepCoverage::default()) +fn sync_muse(conn: &Connection, state: &mut Map, root: &Path) -> Result { + sync_muse_with_coverage( + conn, + state, + root, + &SweepRepairs::default(), + &mut SweepCoverage::default(), + ) } -fn sync_grok_with_coverage( +/// Index every Muse Code transcript whose stamp changed since the last run. +/// +/// A transcript is append-only, but it is read whole: a session is one file, +/// its records carry their own ids and times, and a whole read is what lets a +/// re-read replace rather than merge. Same state-map discipline as Grok: a +/// stamp is trusted only while the evidence it recorded is still stored, and +/// an unreadable transcript keeps no stamp, so the next run retries it. +fn sync_muse_with_coverage( conn: &Connection, state: &mut Map, root: &Path, + repairs: &SweepRepairs, coverage: &mut SweepCoverage, ) -> Result { if !root.exists() { - sync_note!(" [grok] not found: {} (skipped)", root.display()); + sync_note!(" [muse] not found: {} (skipped)", root.display()); return Ok(0); } - let mut grok_state = state - .get(GROK_SYNC_STATE_KEY) + let mut muse_state = state + .get(MUSE_SYNC_STATE_KEY) .and_then(Value::as_object) .cloned() .unwrap_or_default(); @@ -11267,25 +11342,11 @@ fn sync_grok_with_coverage( let mut scanned = 0; let mut sessions = 0; let mut errors = 0; - // Sessions this run could **account for**: indexed now, or confirmed - // unchanged against a stamp it read successfully. Not the same as - // `sessions`, which counts only what was indexed — a store that is fully - // synced indexes nothing on every later run, and using that count to - // decide whether the source failed would fail it forever over one - // unreadable sibling. let mut accounted = 0; - for chat in capture_files( - "grok", - collect_matching_files(root, "chat_history", "jsonl")?, - ) { + for transcript in capture_files("muse", collect_muse_transcripts(root)?) { check_capture_cancelled()?; - let key = chat.to_string_lossy().to_string(); - // One unreadable session directory does not stop the rest, and it does - // not update its saved stamp either: the next run tries again. It is - // counted as looked at, so the run says so — a stamp that fails - // closed and is then reported as nothing at all is the silence the - // strictness exists to prevent. - let stamp = match grok_session_stamp(&chat) { + let key = transcript.to_string_lossy().to_string(); + let stamp = match muse_session_stamp(&transcript) { Ok(stamp) => stamp, Err(error) => { if error.is::() { @@ -11294,224 +11355,1465 @@ fn sync_grok_with_coverage( scanned += 1; errors += 1; coverage.note_unread(); - sync_note!(" [grok] unreadable session {}: {error:#}", chat.display()); + sync_note!( + " [muse] unreadable session {}: {error:#}", + transcript.display() + ); continue; } }; - let recorded = grok_state.get(&key); - let recorded_stamp = recorded.and_then(grok_state_stamp).map(str::to_string); - let recorded_session = recorded.and_then(grok_state_session).map(str::to_string); - let recorded_evidence = recorded.and_then(grok_state_had_evidence); - if recorded_stamp.as_deref() == Some(stamp.as_str()) { - // The stamp says the file has not changed. That is only half the - // question: `.sync-state.json` lives beside the database, so a - // deleted or rebuilt `history.db` keeps this file, and a stamp - // read on its own would answer "already indexed" for a session - // whose rows are gone -- until some file in the directory changes, - // which for a finished session is never. The Codex and Claude - // walks check the database for the same reason. - // - // The other half of that is knowing what "its rows" means for - // *this* session. Not every Grok session produces `session_events` - // -- one made only of `system` lines, synthetic turns or encrypted - // reasoning is stored entirely as markers, and one with an empty - // transcript and a `subagents/` entry only a relationship -- so - // asking only about events would find nothing and re-read such a - // session on every run, forever. The answer is not to guess from - // the tables but to record what the indexing run actually - // produced, and then to ask about that. - match (recorded_session.as_deref(), recorded_evidence) { - // Written by an older build that saved the stamp alone, so - // there is no session id to look the evidence up by. Trust - // the stamp, as that build did. - (None, _) => { - accounted += 1; - continue; - } - // The run that indexed it wrote no evidence rows at all. It - // still wrote a catalog row -- every ingestion does -- so - // that is what there is to check, and it is enough: a session - // with no evidence *is* its catalog row. Skipping without - // asking anything is what let an empty session disappear for - // good when the database was rebuilt. - (Some(id), Some(false)) if grok_catalog_row_exists(conn, id)? => { - accounted += 1; - continue; - } - (Some(id), Some(true) | None) if grok_evidence_exists(conn, id)? => { - accounted += 1; - continue; - } - // Unchanged, and what it did write is missing: re-index it. - _ => {} + let recorded = muse_state.get(&key); + if recorded.and_then(grok_state_stamp) == Some(stamp.as_str()) { + // The stamp says unchanged; the database has to agree before + // that is taken as "already indexed". Every Muse ingestion writes + // at least its catalog row, and one that wrote events is checked + // against those. + let still_there = match ( + recorded.and_then(grok_state_session), + recorded.and_then(grok_state_had_evidence), + ) { + (None, _) => true, + // The destination marker says rows were lost under this + // unchanged stamp — in the session or in a subagent it links. + // Existence cannot see a partial loss; re-reading repairs it. + (Some(id), _) if muse_tree_needs_repair(conn, repairs, id)? => false, + (Some(id), Some(false)) => muse_catalog_row_exists(conn, id)?, + (Some(id), _) => muse_evidence_exists(conn, id)?, + }; + if still_there { + accounted += 1; + continue; } } scanned += 1; - match scan_grok_session_file(&chat) { - Ok(Some(session)) => { - let raw_path = chat.to_string_lossy().to_string(); - let session_id = session.session_id.clone(); - // One session directory is one transaction: its evidence is - // replaced, not merged, and a reader must never see the gap - // between the two halves of that. + // Everything is read before anything is written. A transcript or a + // subagent log that cannot be read is this session's failure alone: + // it keeps no stamp, so the next run retries it, and the pass goes on. + // A write that fails once the transaction is open is a database + // failure, not this session's, so it ends the pass, as it does for + // Grok. + match read_muse_tree(&transcript) { + Ok(Some((parsed, children))) => { + let session_id = parsed.metadata.session_id.clone(); let tx = conn.unchecked_transaction()?; - let outcome = ingest_grok_session(&tx, &session, &raw_path)?; - inserted += outcome.prompts; + let written = muse_history_rows(&tx, &session_id).and_then(|before| { + let outcome = + ingest_muse_session_tree(&tx, &parsed, &transcript, Some(&children))?; + let after = muse_history_rows(&tx, &session_id)?; + Ok((outcome, after.saturating_sub(before))) + }); + let (outcome, added) = match written { + Ok(written) => written, + Err(error) => { + drop(tx); + return Err(error); + } + }; + // A re-read replaces the session's prompts wholesale, so what + // it added is the difference, not what it wrote. + inserted += added; tx.commit()?; - // Whether this session has any evidence to go looking for on - // a later run. A session stored entirely as markers has no - // events; one whose transcript is empty but whose - // `subagents/` directory names a child has only a - // relationship. Every writer this ingestion drives has to be - // counted here, or a later run asks about a table this - // session never filled and re-reads it for ever. - let had_evidence = - outcome.events > 0 || outcome.markers > 0 || outcome.relationships > 0; sessions += 1; accounted += 1; - // The session id travels with the stamp so the next run can - // ask the database whether this evidence is still there, and - // `evidence` says whether there was any to ask about. - grok_state.insert( + muse_state.insert( key, - json!({ "stamp": stamp, "session": session_id, "evidence": had_evidence }), + json!({ + "stamp": stamp, + "session": session_id, + "evidence": outcome.events > 0 + || outcome.markers > 0 + || outcome.relationships > 0, + }), ); } Ok(None) => { - // Read fine, and there was no session in it -- so there is no - // session id to check any future run against either. accounted += 1; - grok_state.insert(key, json!({ "stamp": stamp })); + muse_state.insert(key, json!({ "stamp": stamp })); } Err(error) => { if error.is::() { return Err(error); } errors += 1; - // The stamp is deliberately not recorded so the next sweep - // retries this directory. Keep the source fingerprint stale - // too, or the next tick would skip before reaching it. coverage.note_unread(); - sync_note!(" [grok] unreadable session {}: {error:#}", chat.display()); + sync_note!( + " [muse] unreadable session {}: {error:#}", + transcript.display() + ); } } } - state.insert(GROK_SYNC_STATE_KEY.to_string(), Value::Object(grok_state)); + state.insert(MUSE_SYNC_STATE_KEY.to_string(), Value::Object(muse_state)); if scanned > 0 { let suffix = if errors > 0 { format!(" ({errors} errors)") } else { String::new() }; - sync_note!(" [grok] +{inserted} rows from {sessions} sessions{suffix}"); + sync_note!(" [muse] +{inserted} rows from {sessions} sessions{suffix}"); } - // A store where nothing could be read is a failed source, not an empty - // one: the caller records it and the run says so. A store where something - // failed and something else was accounted for stays a success, so the - // sessions that did read keep their saved stamps -- otherwise one - // unreadable directory would send every future run back over the whole - // store, and a fully synced store would fail this source on every run - // from then on. if errors > 0 && accounted == 0 { anyhow::bail!( - "{errors} Grok session(s) could not be read and none were indexed; \ - the unreadable directories are named above" + "{errors} Muse Code session(s) could not be read and none were indexed; \ + the unreadable transcripts are named above" ); } Ok(inserted) } -/// What indexing one Grok session directory produced, and what its own records -/// could not establish. -/// -/// The fallback counters are not diagnostics of *this* run: they are how many -/// facts Grok did not write down, which is what a caller has to be told before -/// it trusts a timestamp. -#[derive(Debug, Default, Clone)] -pub(crate) struct GrokIngestOutcome { - pub prompts: usize, - pub events: usize, - pub tool_calls: usize, - pub file_edits: usize, - pub markers: usize, - pub relationships: usize, - /// Subagent metadata files that named no child session id. - pub unlinked_subagents: usize, - /// `Task`-style calls in the transcript. Grok names no child in the call - /// itself, so these produce a tool call and never a relationship row. - pub subagent_calls: usize, - /// Records whose time came from their turn's `turnStartMs`. - pub turn_start_fallbacks: usize, - /// Records that inherited the previous record's time. - pub inherited_fallbacks: usize, - /// Records stamped with the session's `created_at`. - pub session_start_fallbacks: usize, - pub encrypted_reasoning: usize, - pub unread_update_rows: usize, - /// The session directory had no `updates.jsonl` at all. - pub missing_updates: bool, - /// The stream was there and established no timing at all — no turn, no - /// message group, no tool. Distinct from `missing_updates`, because the - /// two want different answers: a session Grok wrote no updates for is - /// normal, and a stream this parser could not use is a gap in coverage - /// worth knowing about. - pub updates_yielded_no_timing: bool, - /// The newest `turn_completed` context snapshot, for the caller's report. - pub context_total_tokens: Option, - /// Turns whose `turn_completed` usage breakdown was stored on one of the - /// turn's assistant rows, and how many turns `updates.jsonl` opened in - /// all. - pub usage_turns: usize, - pub turns: usize, +/// Whether the destination marker names this session, or any subagent its +/// logs link, as having lost rows. +fn muse_tree_needs_repair( + conn: &Connection, + repairs: &SweepRepairs, + session_id: &str, +) -> Result { + if repairs.is_empty() { + return Ok(false); + } + if repairs.contains("muse", session_id) { + return Ok(true); + } + let descendants: Vec = conn + .prepare( + "WITH RECURSIVE tree(id) AS ( \ + SELECT child_session_id FROM session_relationships \ + WHERE source = 'muse' AND parent_session_id = ?1 \ + AND evidence_kind = 'muse_subagent_log' AND child_session_id IS NOT NULL \ + UNION \ + SELECT r.child_session_id FROM session_relationships r JOIN tree \ + ON r.parent_session_id = tree.id \ + WHERE r.source = 'muse' AND r.evidence_kind = 'muse_subagent_log' \ + AND r.child_session_id IS NOT NULL \ + ) SELECT id FROM tree", + )? + .query_map(params![session_id], |row| row.get(0))? + .collect::>()?; + Ok(descendants + .iter() + .any(|child| repairs.contains("muse", child))) } -/// The request span of a Grok assistant row: the `updates.jsonl` turn it -/// belongs to. -/// -/// Grok names none of its API calls, and the only usage it reports is one -/// `turn_completed.usage` per turn, so the turn is the finest unit a request -/// can be measured at. Without a shared span each assistant row of a turn — -/// its thinking, every tool call, its prose — grouped as a request of its -/// own, and a turn that called a tool read as several requests, all but one -/// unmeasured. -fn grok_turn_span(turn: Option) -> Option { - turn.map(|turn| turn.to_string()) +/// Whether a Muse session is a subagent some parent's log links. +fn muse_delegation_recorded(conn: &Connection, child_session_id: &str) -> Result { + let exists: i64 = conn.query_row( + "SELECT EXISTS(SELECT 1 FROM session_relationships \ + WHERE source = 'muse' AND child_session_id = ? \ + AND evidence_kind = 'muse_subagent_log' LIMIT 1)", + params![child_session_id], + |row| row.get(0), + )?; + Ok(exists != 0) } -/// Index one Grok session directory: prompts, events, tools, edits, markers -/// and subagent delegations. -fn ingest_grok_session( - conn: &Connection, - session: &GrokSession, - raw_path: &str, -) -> Result { - const SOURCE: &str = "grok"; - let sid = session.session_id.as_str(); - let project = session.cwd.as_deref(); - let branch = session.git_branch.as_deref(); - replace_grok_session_evidence(conn, sid)?; - let mut outcome = GrokIngestOutcome { - missing_updates: !session.updates_present, - updates_yielded_no_timing: session.updates_present && session.updates.is_empty(), - unread_update_rows: session.updates.unread_rows, - context_total_tokens: session - .updates - .turns - .iter() - .rev() - .find_map(|turn| turn.total_tokens), - usage_turns: session - .updates - .turns - .iter() - .filter(|turn| turn.usage.is_some()) - .count(), - turns: session.updates.turns.len(), - ..Default::default() - }; - - let mut user_ordinal = 0usize; - let mut message_ordinal = 0usize; +fn muse_history_rows(conn: &Connection, session_id: &str) -> Result { + let rows: i64 = conn.query_row( + "SELECT COUNT(*) FROM history WHERE source = 'muse' AND session_id = ?", + params![session_id], + |row| row.get(0), + )?; + Ok(rows as usize) +} + +fn muse_evidence_exists(conn: &Connection, session_id: &str) -> Result { + if session_events_exist(conn, "muse", session_id)? { + return Ok(true); + } + for statement in [ + "SELECT EXISTS(SELECT 1 FROM session_markers \ + WHERE source = 'muse' AND session_id = ? LIMIT 1)", + "SELECT EXISTS(SELECT 1 FROM session_relationships \ + WHERE source = 'muse' AND parent_session_id = ? \ + AND evidence_kind = 'muse_subagent_log' LIMIT 1)", + ] { + let exists: i64 = conn.query_row(statement, params![session_id], |row| row.get(0))?; + if exists != 0 { + return Ok(true); + } + } + Ok(false) +} + +fn muse_catalog_row_exists(conn: &Connection, session_id: &str) -> Result { + let exists: i64 = conn.query_row( + "SELECT EXISTS(SELECT 1 FROM sessions \ + WHERE source = 'muse' AND session_id = ? LIMIT 1)", + params![session_id], + |row| row.get(0), + )?; + Ok(exists != 0) +} + +/// The transcript and every subagent log beneath its session directory, +/// parent first, then children depth-first in sorted order. +pub(crate) fn muse_session_files(transcript: &Path) -> Result> { + let mut files = vec![transcript.to_path_buf()]; + if let Some(dir) = transcript.parent() { + collect_muse_child_logs(dir, &mut files)?; + } + Ok(files) +} + +/// The `subagent//session.jsonl` logs under one session directory, and +/// theirs in turn. +/// +/// No depth limit is needed, and none is imposed — a limit would silently drop +/// the logs beneath it. [`direct_muse_child_logs`] accepts a child only where +/// it resolves beneath the session's *real* directory plus a literal +/// `subagent/` component, so each level is strictly deeper in the real +/// filesystem than the one before and adds at least `/subagent/` to its +/// path; the operating system's own path-length limit therefore bounds every +/// walk, symlinks and all. +fn collect_muse_child_logs(session_dir: &Path, out: &mut Vec) -> Result<()> { + for log in direct_muse_child_logs(session_dir)? { + check_capture_cancelled()?; + out.push(log.clone()); + if let Some(child_dir) = log.parent() { + collect_muse_child_logs(child_dir, out)?; + } + } + Ok(()) +} + +/// The immediate `subagent//session.jsonl` logs of one session +/// directory, sorted. +fn direct_muse_child_logs(session_dir: &Path) -> Result> { + let children = session_dir.join(muse::SUBAGENT_DIR); + let entries = match fs::read_dir(&children) { + Ok(entries) => entries, + Err(error) if error.kind() == io::ErrorKind::NotFound => return Ok(Vec::new()), + Err(error) => { + return Err(error) + .with_context(|| format!("list Muse subagents {}", children.display())) + } + }; + // A child log is read only where it resolves beneath the session's real + // directory plus a literal `subagent/` component. The session directory + // is resolved first and `subagent` joined after, so a `subagent/` that is + // itself a symlink is measured against where it *should* be, not where + // it points. Anything else is not this session's evidence: outside the + // Muse root it would index a stranger's file, and back up into the + // session (`subagent -> .`, `subagent/loop -> ..`) it would read the + // parent's own transcript as its child and recurse. Requiring each level + // to resolve strictly beneath the previous one's real `subagent/` is what + // makes every walk of the tree finite. + let within = fs::canonicalize(session_dir) + .with_context(|| format!("resolve Muse session {}", session_dir.display()))? + .join(muse::SUBAGENT_DIR); + let mut logs = Vec::new(); + for entry in entries { + let entry = entry.with_context(|| format!("list Muse subagents {}", children.display()))?; + let log = entry.path().join(muse::SESSION_FILE); + // Only "not there" means absent. A log that cannot be statted is + // still there, and treating it as gone would forget its evidence; + // the error fails this session's read instead, so it is retried. + match fs::metadata(&log) { + Ok(metadata) if metadata.is_file() => { + let resolved = fs::canonicalize(&log) + .with_context(|| format!("resolve Muse subagent log {}", log.display()))?; + if resolved.starts_with(&within) { + logs.push(log); + } + } + Ok(_) => {} + Err(error) + if matches!( + error.kind(), + io::ErrorKind::NotFound | io::ErrorKind::NotADirectory + ) => {} + Err(error) => { + return Err(error) + .with_context(|| format!("stat Muse subagent log {}", log.display())) + } + } + } + logs.sort(); + Ok(logs) +} + +/// The change stamp of a Muse session: its transcript plus every child log +/// beneath it, because a background subagent keeps writing its own log after +/// the parent's last record, and a stamp over the parent alone would never +/// see that. +pub(crate) fn muse_session_stamp(transcript: &Path) -> Result { + let files = muse_session_files(transcript)?; + let base = transcript.parent().unwrap_or(transcript); + let mut parts = Vec::with_capacity(files.len()); + for (index, file) in files.iter().enumerate() { + let stamp = file_stamp(file)?; + if index == 0 { + parts.push(stamp); + } else { + let relative = file.strip_prefix(base).unwrap_or(file); + parts.push(format!("{}={stamp}", relative.to_string_lossy())); + } + } + Ok(parts.join("|")) +} + +/// Index a Muse session and every subagent it left a log for. +/// +/// Each child is indexed under the session id its **own** metadata record +/// names — never the directory name — and linked to its parent as +/// `delegated`, with the role, label, model and task the parent recorded for +/// it. A child is not a session anybody started: its objective is not a +/// prompt, and it is not a catalog row of its own. So, as for a Codex +/// subagent rollout, its `history` rows and catalog registration are removed +/// and its events stay addressable through the relationship. +/// +/// `children` is the tree [`read_muse_children`] read, or `None` to leave the +/// session's children and edges exactly as they are — a hydration that did +/// not ask for related evidence must not rewrite it. +pub(crate) fn ingest_muse_session_tree( + conn: &Connection, + transcript: &muse::MuseTranscript, + path: &Path, + children: Option<&[MuseChildLog]>, +) -> Result { + // A subagent's log is read as one wherever the read starts — a child + // also known remotely keeps a catalog row, and hydrating it directly must + // not turn its objective into a typed prompt that the parent's next sweep + // would take back out. + let role = if muse::is_subagent_log(path) { + MuseRole::Subagent + } else { + MuseRole::Session + }; + let mut outcome = ingest_muse_session(conn, transcript, &path.to_string_lossy(), role)?; + if role == MuseRole::Subagent { + // Not a catalog row of its own either, unless a remote presence + // stands behind it. + cleanup_subagent_registration(conn, "muse", &transcript.metadata.session_id)?; + } + if let (Some(dir), Some(children)) = (path.parent(), children) { + let mut visited = HashSet::from([transcript.metadata.session_id.clone()]); + link_muse_subagents( + conn, + transcript, + dir, + children, + 1, + &mut visited, + &mut outcome, + )?; + } + Ok(outcome) +} + +/// Whose transcript an ingestion is reading. +#[derive(Clone, Copy, Debug, PartialEq, Eq)] +pub(crate) enum MuseRole { + /// A session someone started: its prompts were typed. + Session, + /// A subagent's log: its prompt is the objective its parent assigned. + Subagent, +} + +/// One subagent log beneath a session, read, with its own children. +pub(crate) struct MuseChildLog { + path: PathBuf, + /// `None` when the log names no session. + transcript: Option, + children: Vec, +} + +/// Read a transcript and every subagent log beneath it. `Ok(None)` when the +/// transcript names no session. +pub(crate) fn read_muse_tree( + transcript: &Path, +) -> Result)>> { + let Some(parsed) = read_muse_transcript(transcript)? else { + return Ok(None); + }; + let children = match transcript.parent() { + Some(dir) => read_muse_children(dir)?, + None => Vec::new(), + }; + Ok(Some((parsed, children))) +} + +/// The subagent logs directly beneath one session directory, each read along +/// with its own children. +/// Finite for the reason [`collect_muse_child_logs`] is. +pub(crate) fn read_muse_children(session_dir: &Path) -> Result> { + let mut out = Vec::new(); + for path in direct_muse_child_logs(session_dir)? { + check_capture_cancelled()?; + let transcript = read_muse_transcript(&path)?; + let children = match path.parent() { + Some(dir) => read_muse_children(dir)?, + None => Vec::new(), + }; + out.push(MuseChildLog { + path, + transcript, + children, + }); + } + Ok(out) +} + +fn link_muse_subagents( + conn: &Connection, + parent: &muse::MuseTranscript, + parent_dir: &Path, + children: &[MuseChildLog], + depth: i64, + visited: &mut HashSet, + outcome: &mut MuseIngestOutcome, +) -> Result<()> { + let parent_id = parent.metadata.session_id.as_str(); + // Replaced, not merged: a child whose log is gone is no longer evidenced. + // The edges are upserted as the children are linked and only the ones + // left over are retired afterwards, so a child whose log is still there + // is never dropped in between. + let previous = muse_child_locators(conn, parent_id)?; + let mut linked = HashSet::new(); + let mut unidentified_logs = HashSet::new(); + let links = muse::subagent_links(parent); + for entry in children { + check_capture_cancelled()?; + let log = &entry.path; + let Some(child) = entry.transcript.as_ref() else { + // A log that names no session yet — Muse is still writing its + // header — is not a log that is gone. + outcome.unidentified_children += 1; + unidentified_logs.insert(log.to_string_lossy().to_string()); + continue; + }; + let child_id = child.metadata.session_id.clone(); + // A log that names its own parent, or an ancestor, would loop. + if !visited.insert(child_id.clone()) { + continue; + } + linked.insert(child_id.clone()); + let locator = log.to_string_lossy().to_string(); + // Read as a subagent: its objective is an event, never `history`, and + // any history it held before is cleared the same way a session's is + // — all of it, or only this log's own rows where the child is also + // known remotely. + ingest_muse_session(conn, child, &locator, MuseRole::Subagent)?; + cleanup_subagent_registration(conn, "muse", &child_id)?; + if let Some(child_dir) = log.parent() { + link_muse_subagents( + conn, + child, + child_dir, + &entry.children, + depth + 1, + visited, + outcome, + )?; + } + let relative = log + .strip_prefix(parent_dir) + .map(|relative| muse::normalize_link_path(&relative.to_string_lossy())) + .unwrap_or_default(); + let (linked_at, link) = links.get(&relative).cloned().unwrap_or_default(); + // Only this session's own spawns are compared with its direct + // children, and only a child it typed as a worker answers one: a + // grandchild answers its own parent's spawn, and a log the parent + // never described is not evidence that a spawn succeeded. + if depth == 1 && link.role.as_deref().is_some_and(|role| role != "reminder") { + outcome.worker_children += 1; + } + let child_model = link + .model + .clone() + .or_else(|| child.metadata.model_id.clone()); + let spawned_at_ms = (linked_at > 0) + .then_some(linked_at) + .or_else(|| child.first_ts()); + record_relationship( + conn, + &ObservedRelationship { + source: "muse", + parent_session_id: parent_id, + child_session_id: Some(&child_id), + child_agent_type: link.role.as_deref(), + child_agent_name: link.label.as_deref(), + child_model: child_model.as_deref(), + spawn_depth: Some(depth), + evidence_kind: "muse_subagent_log", + evidence_locator: Some(&locator), + evidence_ref: link.task_id.as_deref(), + child_has_events: session_events_exist(conn, "muse", &child_id)?, + spawned_at_ms, + ..ObservedRelationship::default() + }, + )?; + // The upsert keeps an optional field the new observation leaves empty, + // so a label, model or task the parent no longer records would stay + // on the edge. What the parent's record owns is assigned from it, + // empty values included; the edge itself — and its first-seen + // `created_ms` — stays in place. + conn.execute( + "UPDATE session_relationships SET child_agent_type = ?, child_agent_name = ?, \ + child_model = ?, evidence_ref = ?, evidence_locator = ?, spawn_depth = ?, \ + spawned_at_ms = ? \ + WHERE source = 'muse' AND parent_session_id = ? AND child_session_id = ? \ + AND evidence_kind = 'muse_subagent_log'", + params![ + link.role.as_deref(), + link.label.as_deref(), + child_model.as_deref(), + link.task_id.as_deref(), + locator, + depth, + spawned_at_ms, + parent_id, + child_id, + ], + )?; + outcome.relationships += 1; + } + for (child, locator) in &previous { + if linked.contains(child) { + continue; + } + // Still on disk, just not identifiable yet: keep what it had. + if locator + .as_deref() + .is_some_and(|locator| unidentified_logs.contains(locator)) + { + continue; + } + conn.execute( + "DELETE FROM session_relationships WHERE source = 'muse' \ + AND parent_session_id = ? AND child_session_id = ? \ + AND evidence_kind = 'muse_subagent_log'", + params![parent_id, child], + )?; + forget_muse_subagent(conn, child)?; + } + Ok(()) +} + +/// The children a parent's subagent logs linked on the last read, with the +/// log each was read from. +fn muse_child_locators( + conn: &Connection, + parent_id: &str, +) -> Result>> { + Ok(conn + .prepare( + "SELECT child_session_id, evidence_locator FROM session_relationships \ + WHERE source = 'muse' AND parent_session_id = ? \ + AND evidence_kind = 'muse_subagent_log' AND child_session_id IS NOT NULL", + )? + .query_map(params![parent_id], |row| Ok((row.get(0)?, row.get(1)?)))? + .collect::>()?) +} + +/// The children a parent's subagent logs linked on the last read. +fn muse_linked_children(conn: &Connection, parent_id: &str) -> Result> { + Ok(conn + .prepare( + "SELECT child_session_id FROM session_relationships \ + WHERE source = 'muse' AND parent_session_id = ? \ + AND evidence_kind = 'muse_subagent_log' AND child_session_id IS NOT NULL", + )? + .query_map(params![parent_id], |row| row.get::<_, String>(0))? + .collect::>()?) +} + +/// Drop the evidence of a child whose log is gone, and of its own children. +/// It was only ever addressable through the edge that no longer exists. +/// +/// A child also known remotely keeps its own events and markers — the rows +/// under its id are not only this log's — but the children it linked from the +/// missing directory, and its edges to them, go all the same. +fn forget_muse_subagent(conn: &Connection, child_id: &str) -> Result<()> { + // Its own children were linked from the same, now missing, directory, + // whatever the child itself is. + for grandchild in muse_linked_children(conn, child_id)? { + forget_muse_subagent(conn, &grandchild)?; + } + conn.execute( + "DELETE FROM session_relationships WHERE source = 'muse' \ + AND parent_session_id = ? AND evidence_kind = 'muse_subagent_log'", + params![child_id], + )?; + if session_has_remote_presence(conn, "muse", child_id)? { + // Its log is gone, so it is no longer present locally; the remote + // presence, the catalog row that stands on it and its history stay. + conn.execute( + "DELETE FROM session_presences \ + WHERE source = 'muse' AND session_id = ? AND location = 'local'", + params![child_id], + )?; + return Ok(()); + } + // Any history still filed under it — from before it was linked as a + // subagent — goes too, or moves to a session that still evidences it. + replace_session_history(conn, "muse", child_id)?; + for statement in [ + "DELETE FROM session_events WHERE source = 'muse' AND session_id = ?", + "DELETE FROM tool_calls WHERE source = 'muse' AND session_id = ?", + "DELETE FROM file_edits WHERE source = 'muse' AND session_id = ?", + "DELETE FROM session_markers WHERE source = 'muse' AND session_id = ?", + "DELETE FROM session_relationships WHERE source = 'muse' \ + AND parent_session_id = ? AND evidence_kind = 'muse_subagent_log'", + ] { + conn.execute(statement, params![child_id])?; + } + Ok(()) +} + +/// Read and interpret one transcript. `Ok(None)` when it names no session. +pub(crate) fn read_muse_transcript(path: &Path) -> Result> { + let bytes = + fs::read(path).with_context(|| format!("read Muse transcript {}", path.display()))?; + Ok(muse::parse_transcript(&String::from_utf8_lossy(&bytes))) +} + +/// What indexing one Muse Code transcript produced, and what its records +/// could not establish. +#[derive(Debug, Default, Clone)] +pub(crate) struct MuseIngestOutcome { + pub prompts: usize, + pub events: usize, + pub tool_calls: usize, + pub file_edits: usize, + pub markers: usize, + /// Reasoning Muse kept only as an encrypted trace. + pub encrypted_reasoning: usize, + /// `model_completed` steps whose usage had no assistant record after them + /// in the same run to carry it. + pub unattached_usage: usize, + /// `subagent_spawn` calls in this session's own transcript. + pub subagent_calls: usize, + /// Child transcripts linked as `delegated` children, at any depth. + pub relationships: usize, + /// The session's direct children that it typed as something other than + /// a reminder: the ones its own `subagent_spawn` calls account for. + pub worker_children: usize, + /// `subagent/` logs that named no session, so there was no child to link. + pub unidentified_children: usize, + /// Tool results with no recorded outcome for their call. + pub results_without_outcome: usize, + /// Lines that did not parse, usually a live session's partial tail. + pub unparsed_lines: usize, +} + +/// What one model step reported, for the assistant record that step owns: +/// its usage verbatim, its model and its `finish_reason`. +struct MuseStep<'a> { + token_json: Option, + model: Option<&'a str>, + finish_reason: Option<&'a str>, +} + +impl<'a> MuseStep<'a> { + fn of(transcript: &'a muse::MuseTranscript, index: usize) -> Self { + match &transcript.records[index].event { + muse::MuseEvent::ModelCompleted { + model, + usage, + finish_reason, + .. + } => Self { + token_json: usage.as_ref().map(Value::to_string), + model: model.as_deref(), + finish_reason: finish_reason.as_deref(), + }, + _ => Self { + token_json: None, + model: None, + finish_reason: None, + }, + } + } +} + +/// Index one Muse Code transcript: prompts, prose, thinking, tool calls and +/// results, file edits, per-step token usage and lifecycle markers. +/// +/// A whole-transcript read replaces the session's evidence rather than +/// merging into it, so a re-read after the file grew never duplicates a row +/// and one that dropped a record (a rewritten or truncated file) leaves no +/// stale one behind. +/// +/// `role` says whose transcript this is. A [`MuseRole::Session`]'s prompts +/// were typed and become `history`; a [`MuseRole::Subagent`]'s run-start +/// prompt is the objective its parent assigned, which is an event but never +/// a prompt anybody typed. +pub(crate) fn ingest_muse_session( + conn: &Connection, + transcript: &muse::MuseTranscript, + raw_path: &str, + role: MuseRole, +) -> Result { + const SOURCE: &str = "muse"; + let sid = transcript.metadata.session_id.as_str(); + let project = transcript.metadata.workspace_root.as_deref(); + let version = transcript.metadata.cli_version.as_deref(); + // History is replaced, not merged, so a prompt the transcript no longer + // has stops being searchable. A session also known remotely shares its + // `history` identity with that evidence, so there only the rows this + // transcript wrote last time — the ones its stored prompt events name — + // are removed; replacing the whole identity would delete prompts it never + // wrote. Either way this runs before the events it reads are replaced. + let scope = if session_has_remote_presence(conn, SOURCE, sid)? { + HistoryScope::OwnPromptEvents + } else { + HistoryScope::Session + }; + replace_session_history_scoped(conn, SOURCE, sid, scope)?; + for statement in [ + "DELETE FROM session_events WHERE source = 'muse' AND session_id = ?", + "DELETE FROM tool_calls WHERE source = 'muse' AND session_id = ?", + "DELETE FROM file_edits WHERE source = 'muse' AND session_id = ?", + "DELETE FROM session_markers WHERE source = 'muse' AND session_id = ?", + ] { + conn.execute(statement, params![sid])?; + } + + let mut outcome = MuseIngestOutcome { + unparsed_lines: transcript.unparsed_lines, + ..Default::default() + }; + let outcomes = muse::tool_outcomes(transcript); + let tool_names = muse::tool_names(transcript); + // Calls whose result this transcript recorded. + let mut answered: HashSet = HashSet::new(); + let steps = muse::pair_model_steps(transcript); + outcome.unattached_usage = steps.orphans; + // The step a record belongs to: its model and finish reason are every one + // of its records', and its usage is its first record's alone, never a + // neighbour's. + let step_of = |index: usize| { + steps.members.get(&index).map(|step| { + let mut step_facts = MuseStep::of(transcript, *step); + if !steps.owners.contains_key(&index) { + step_facts.token_json = None; + } + step_facts + }) + }; + // The model a record ran under when its own step does not name one. + let mut model = transcript.metadata.model_id.clone(); + let mut first_prompt: Option = None; + let mut last_assistant_text: Option = None; + let mut results = ToolResultIndexer::default(); + + for (idx, record) in transcript.records.iter().enumerate() { + check_capture_cancelled()?; + let ts = record.ts_ms; + let run_id = record.run_id.as_deref(); + let record_uid = record + .record_id + .clone() + .unwrap_or_else(|| format!("r{}", record.sequence.unwrap_or(idx as i64))); + let facts = RawMessageFacts { + agent_version: version, + turn_id: run_id, + ..Default::default() + }; + match &record.event { + muse::MuseEvent::Prompt { text } => { + // A subagent's run-start prompt is its assigned objective, + // not a title anybody typed. + if role == MuseRole::Session && first_prompt.is_none() { + first_prompt = Some(crate::discover::excerpt(text)); + } + insert_session_event( + conn, + SOURCE, + sid, + project, + project, + None, + &record_uid, + None, + ts, + "user", + "text", + Some(text), + None, + None, + RequestIdentity::none(), + &record_uid, + None, + facts, + )?; + outcome.events += 1; + if role == MuseRole::Session { + outcome.prompts += insert_history( + conn, + &HistoryEntry { + id: 0, + source: SOURCE.into(), + session_id: Some(sid.to_string()), + project: project.map(str::to_string), + prompt_hash: Some(prompt_hash(text)), + prompt: text.clone(), + timestamp_ms: ts, + }, + )?; + } + } + // Paired with its assistant record up front; only the running + // model is tracked here. + muse::MuseEvent::ModelCompleted { + model: step_model, .. + } => { + if step_model.is_some() { + model = step_model.clone(); + } + } + muse::MuseEvent::Reasoning { + message_id, + text, + encrypted, + } => match text { + Some(text) => { + let step = step_of(idx); + let token_json = step.as_ref().and_then(|step| step.token_json.clone()); + let row_model = step + .as_ref() + .and_then(|step| step.model) + .or(model.as_deref()); + let message_id = message_id.as_deref().unwrap_or(&record_uid); + insert_session_event( + conn, + SOURCE, + sid, + project, + project, + None, + message_id, + None, + ts, + "assistant", + "thinking", + Some(text), + row_model, + token_json.as_deref(), + RequestIdentity::none(), + &record_uid, + None, + RawMessageFacts { + stop_reason: step.as_ref().and_then(|step| step.finish_reason), + ..facts + }, + )?; + outcome.events += 1; + } + None if *encrypted => { + // The trace exists but is opaque: record that Muse thought + // here, never readable thinking for it. + outcome.encrypted_reasoning += 1; + outcome.markers += insert_session_marker( + conn, + SOURCE, + sid, + &NewSessionMarker { + marker_uid: &record_uid, + kind: "encrypted_reasoning", + subkind: Some("reasoning_committed"), + ts_ms: Some(ts), + message_id: message_id.as_deref(), + turn_id: run_id, + ..Default::default() + }, + )?; + } + None => {} + }, + muse::MuseEvent::AssistantText { + message_id, + response_id, + text, + } => { + let step = step_of(idx); + let token_json = step.as_ref().and_then(|step| step.token_json.clone()); + let row_model = step + .as_ref() + .and_then(|step| step.model) + .or(model.as_deref()); + let finish_reason = step.as_ref().and_then(|step| step.finish_reason); + let message_id = message_id.as_deref().unwrap_or(&record_uid); + insert_session_event( + conn, + SOURCE, + sid, + project, + project, + None, + message_id, + None, + ts, + "assistant", + "text", + Some(text), + row_model, + token_json.as_deref(), + RequestIdentity { + request_id: response_id.as_deref(), + provider_message_id: Some(message_id), + }, + &record_uid, + None, + RawMessageFacts { + request_id: response_id.as_deref(), + stop_reason: finish_reason, + ..facts + }, + )?; + outcome.events += 1; + last_assistant_text = Some(text.clone()); + } + muse::MuseEvent::ToolCalls { + message_id, + response_id, + calls, + } => { + let message_id = message_id.as_deref().unwrap_or(&record_uid); + let step = step_of(idx); + let row_model = step + .as_ref() + .and_then(|step| step.model) + .or(model.as_deref()); + let finish_reason = step.as_ref().and_then(|step| step.finish_reason); + // One request, several calls: its usage is stored once, on + // the first call's row. + let mut step_usage = step.as_ref().and_then(|step| step.token_json.clone()); + for call in calls { + let token_json = step_usage.take(); + let target = muse::pick_tool_target(&call.name, &call.arguments); + let uid = format!("tool:{}", call.call_id); + insert_session_event( + conn, + SOURCE, + sid, + project, + project, + None, + message_id, + None, + ts, + "assistant", + "tool_use", + Some(&format_tool_event_text( + &call.name, + target.as_deref(), + &call.arguments, + )), + row_model, + token_json.as_deref(), + RequestIdentity { + request_id: response_id.as_deref(), + provider_message_id: Some(message_id), + }, + &uid, + None, + RawMessageFacts { + request_id: response_id.as_deref(), + stop_reason: finish_reason, + ..facts + }, + )?; + outcome.events += 1; + insert_tool_call( + conn, + SOURCE, + sid, + message_id, + &call.call_id, + &call.name, + target.as_deref(), + &serde_json::to_string(&call.arguments).unwrap_or_default(), + None, + ts, + )?; + outcome.tool_calls += 1; + if muse::is_subagent_tool(&call.name) { + outcome.subagent_calls += 1; + } + if muse::is_file_edit_tool(&call.name) { + if let Some(path) = target.as_deref() { + upsert_file_edit_from_call( + conn, + SOURCE, + sid, + message_id, + &call.call_id, + path, + &call.name, + ts, + None, + project, + )?; + outcome.file_edits += 1; + } + } + } + } + muse::MuseEvent::ToolResults { results: batch } => { + for result in batch { + answered.insert(result.call_id.clone()); + let text = result.text.as_deref(); + let mut result_facts = tool_result_facts::muse_tool_result_facts( + text, + &result.call_id, + tool_names.get(result.call_id.as_str()).copied(), + outcomes.get(result.call_id.as_str()).copied(), + ); + if result_facts.result_status.as_deref() + == Some(tool_result_facts::STATUS_UNKNOWN) + { + outcome.results_without_outcome += 1; + } + let (call_index, event_index) = results.next(&result.call_id); + result_facts = result_facts.with_ordering(call_index, event_index); + let uid = format!("result:{}", result.call_id); + insert_session_event( + conn, + SOURCE, + sid, + project, + project, + None, + &uid, + None, + ts, + "tool_result", + "tool_result", + text, + None, + None, + RequestIdentity::none(), + &uid, + Some(&result_facts), + facts, + )?; + outcome.events += 1; + match result_facts.result_status.as_deref() { + Some(tool_result_facts::STATUS_ERRORED) + | Some(tool_result_facts::STATUS_CANCELLED) => { + set_tool_call_error(conn, SOURCE, sid, &result.call_id, true)? + } + Some(tool_result_facts::STATUS_COMPLETED) => { + set_tool_call_error(conn, SOURCE, sid, &result.call_id, false)? + } + _ => {} + } + } + } + muse::MuseEvent::RunTerminal { status, reason } => { + let payload_json = + marker_payload(vec![("terminal", json!(status)), ("reason", json!(reason))]); + outcome.markers += insert_session_marker( + conn, + SOURCE, + sid, + &NewSessionMarker { + marker_uid: &record_uid, + kind: "turn_end", + subkind: Some("terminal"), + ts_ms: Some(ts), + turn_id: run_id, + text: status.as_deref(), + payload_json: payload_json.as_deref(), + ..Default::default() + }, + )?; + } + muse::MuseEvent::SessionOpened { resume } => { + let payload_json = marker_payload(vec![("resume", json!(resume))]); + outcome.markers += insert_session_marker( + conn, + SOURCE, + sid, + &NewSessionMarker { + marker_uid: &record_uid, + kind: "session_start", + subkind: Some("session.opened.observed"), + ts_ms: Some(ts), + payload_json: payload_json.as_deref(), + ..Default::default() + }, + )?; + } + muse::MuseEvent::SessionResumed => { + outcome.markers += insert_session_marker( + conn, + SOURCE, + sid, + &NewSessionMarker { + marker_uid: &record_uid, + kind: "session_resumed", + subkind: Some("session.resumed"), + ts_ms: Some(ts), + ..Default::default() + }, + )?; + } + muse::MuseEvent::SessionEnd { exit_reason } => { + outcome.markers += insert_session_marker( + conn, + SOURCE, + sid, + &NewSessionMarker { + marker_uid: &record_uid, + kind: "session_end", + subkind: Some("session.end"), + ts_ms: Some(ts), + text: exit_reason.as_deref(), + ..Default::default() + }, + )?; + } + muse::MuseEvent::Metadata(metadata) => { + if metadata.model_id.is_some() { + model = metadata.model_id.clone(); + } + outcome.markers += insert_session_marker( + conn, + SOURCE, + sid, + &NewSessionMarker { + marker_uid: &record_uid, + kind: "model_switch", + subkind: Some(muse::METADATA_PAYLOAD_TYPE), + ts_ms: Some(ts), + text: metadata.model_id.as_deref(), + ..Default::default() + }, + )?; + } + // Folded into the result rows above, by call id. + muse::MuseEvent::ToolOutcome { .. } => {} + // Read by `link_muse_subagents`, which owns the edges. + muse::MuseEvent::SubagentLinked(_) => {} + } + } + // A call can end without a result record — cancelled, or failed before + // producing output. Its outcome is still Muse's word on how it ended, so + // it sets the call's status; a call that has a result was settled above, + // where the result's own evidence (a `bash` exit code) is weighed too. + for (call_id, (status, _)) in &outcomes { + if answered.contains(*call_id) { + continue; + } + let failed = match *status { + "completed" => false, + "failed" | "cancelled" | "canceled" | "skipped" => true, + _ => continue, + }; + set_tool_call_error(conn, SOURCE, sid, call_id, failed)?; + } + + let first_ts = transcript.first_ts().unwrap_or_default(); + let last_ts = transcript.last_ts().unwrap_or(first_ts); + upsert_session( + conn, + sid, + SOURCE, + project, + None, + first_ts, + last_ts, + last_assistant_text.as_deref(), + Some(raw_path), + )?; + // `upsert_session` merges, which suits an append-only log read in pieces. + // This read is the whole transcript, so the fields it owns are assigned + // from it, empty values included: a rewritten file that no longer has a + // reply or a prompt must not keep quoting the old one. A subagent has no + // typed prompt to own, so its read leaves `first_prompt` as it is — a + // remotely known child keeps the title the remote side gave it. + let models = transcript.models(); + conn.execute( + "UPDATE sessions SET first_activity_ms = ?, last_activity_ms = ?, \ + last_assistant_text = ?, models_json = ?, \ + first_prompt = CASE WHEN ? THEN ? ELSE first_prompt END, \ + agent_version = COALESCE(?, agent_version) \ + WHERE source = 'muse' AND session_id = ?", + params![ + first_ts, + last_ts, + last_assistant_text.as_deref(), + (!models.is_empty()).then(|| serde_json::to_string(&models).unwrap_or_default()), + role == MuseRole::Session, + first_prompt.as_deref(), + version, + sid, + ], + )?; + Ok(outcome) +} + +#[cfg(test)] +fn sync_grok(conn: &Connection, state: &mut Map, root: &Path) -> Result { + sync_grok_with_coverage(conn, state, root, &mut SweepCoverage::default()) +} + +fn sync_grok_with_coverage( + conn: &Connection, + state: &mut Map, + root: &Path, + coverage: &mut SweepCoverage, +) -> Result { + if !root.exists() { + sync_note!(" [grok] not found: {} (skipped)", root.display()); + return Ok(0); + } + let mut grok_state = state + .get(GROK_SYNC_STATE_KEY) + .and_then(Value::as_object) + .cloned() + .unwrap_or_default(); + let mut inserted = 0; + let mut scanned = 0; + let mut sessions = 0; + let mut errors = 0; + // Sessions this run could **account for**: indexed now, or confirmed + // unchanged against a stamp it read successfully. Not the same as + // `sessions`, which counts only what was indexed — a store that is fully + // synced indexes nothing on every later run, and using that count to + // decide whether the source failed would fail it forever over one + // unreadable sibling. + let mut accounted = 0; + for chat in capture_files( + "grok", + collect_matching_files(root, "chat_history", "jsonl")?, + ) { + check_capture_cancelled()?; + let key = chat.to_string_lossy().to_string(); + // One unreadable session directory does not stop the rest, and it does + // not update its saved stamp either: the next run tries again. It is + // counted as looked at, so the run says so — a stamp that fails + // closed and is then reported as nothing at all is the silence the + // strictness exists to prevent. + let stamp = match grok_session_stamp(&chat) { + Ok(stamp) => stamp, + Err(error) => { + if error.is::() { + return Err(error); + } + scanned += 1; + errors += 1; + coverage.note_unread(); + sync_note!(" [grok] unreadable session {}: {error:#}", chat.display()); + continue; + } + }; + let recorded = grok_state.get(&key); + let recorded_stamp = recorded.and_then(grok_state_stamp).map(str::to_string); + let recorded_session = recorded.and_then(grok_state_session).map(str::to_string); + let recorded_evidence = recorded.and_then(grok_state_had_evidence); + if recorded_stamp.as_deref() == Some(stamp.as_str()) { + // The stamp says the file has not changed. That is only half the + // question: `.sync-state.json` lives beside the database, so a + // deleted or rebuilt `history.db` keeps this file, and a stamp + // read on its own would answer "already indexed" for a session + // whose rows are gone -- until some file in the directory changes, + // which for a finished session is never. The Codex and Claude + // walks check the database for the same reason. + // + // The other half of that is knowing what "its rows" means for + // *this* session. Not every Grok session produces `session_events` + // -- one made only of `system` lines, synthetic turns or encrypted + // reasoning is stored entirely as markers, and one with an empty + // transcript and a `subagents/` entry only a relationship -- so + // asking only about events would find nothing and re-read such a + // session on every run, forever. The answer is not to guess from + // the tables but to record what the indexing run actually + // produced, and then to ask about that. + match (recorded_session.as_deref(), recorded_evidence) { + // Written by an older build that saved the stamp alone, so + // there is no session id to look the evidence up by. Trust + // the stamp, as that build did. + (None, _) => { + accounted += 1; + continue; + } + // The run that indexed it wrote no evidence rows at all. It + // still wrote a catalog row -- every ingestion does -- so + // that is what there is to check, and it is enough: a session + // with no evidence *is* its catalog row. Skipping without + // asking anything is what let an empty session disappear for + // good when the database was rebuilt. + (Some(id), Some(false)) if grok_catalog_row_exists(conn, id)? => { + accounted += 1; + continue; + } + (Some(id), Some(true) | None) if grok_evidence_exists(conn, id)? => { + accounted += 1; + continue; + } + // Unchanged, and what it did write is missing: re-index it. + _ => {} + } + } + scanned += 1; + match scan_grok_session_file(&chat) { + Ok(Some(session)) => { + let raw_path = chat.to_string_lossy().to_string(); + let session_id = session.session_id.clone(); + // One session directory is one transaction: its evidence is + // replaced, not merged, and a reader must never see the gap + // between the two halves of that. + let tx = conn.unchecked_transaction()?; + let outcome = ingest_grok_session(&tx, &session, &raw_path)?; + inserted += outcome.prompts; + tx.commit()?; + // Whether this session has any evidence to go looking for on + // a later run. A session stored entirely as markers has no + // events; one whose transcript is empty but whose + // `subagents/` directory names a child has only a + // relationship. Every writer this ingestion drives has to be + // counted here, or a later run asks about a table this + // session never filled and re-reads it for ever. + let had_evidence = + outcome.events > 0 || outcome.markers > 0 || outcome.relationships > 0; + sessions += 1; + accounted += 1; + // The session id travels with the stamp so the next run can + // ask the database whether this evidence is still there, and + // `evidence` says whether there was any to ask about. + grok_state.insert( + key, + json!({ "stamp": stamp, "session": session_id, "evidence": had_evidence }), + ); + } + Ok(None) => { + // Read fine, and there was no session in it -- so there is no + // session id to check any future run against either. + accounted += 1; + grok_state.insert(key, json!({ "stamp": stamp })); + } + Err(error) => { + if error.is::() { + return Err(error); + } + errors += 1; + // The stamp is deliberately not recorded so the next sweep + // retries this directory. Keep the source fingerprint stale + // too, or the next tick would skip before reaching it. + coverage.note_unread(); + sync_note!(" [grok] unreadable session {}: {error:#}", chat.display()); + } + } + } + state.insert(GROK_SYNC_STATE_KEY.to_string(), Value::Object(grok_state)); + if scanned > 0 { + let suffix = if errors > 0 { + format!(" ({errors} errors)") + } else { + String::new() + }; + sync_note!(" [grok] +{inserted} rows from {sessions} sessions{suffix}"); + } + // A store where nothing could be read is a failed source, not an empty + // one: the caller records it and the run says so. A store where something + // failed and something else was accounted for stays a success, so the + // sessions that did read keep their saved stamps -- otherwise one + // unreadable directory would send every future run back over the whole + // store, and a fully synced store would fail this source on every run + // from then on. + if errors > 0 && accounted == 0 { + anyhow::bail!( + "{errors} Grok session(s) could not be read and none were indexed; \ + the unreadable directories are named above" + ); + } + Ok(inserted) +} + +/// What indexing one Grok session directory produced, and what its own records +/// could not establish. +/// +/// The fallback counters are not diagnostics of *this* run: they are how many +/// facts Grok did not write down, which is what a caller has to be told before +/// it trusts a timestamp. +#[derive(Debug, Default, Clone)] +pub(crate) struct GrokIngestOutcome { + pub prompts: usize, + pub events: usize, + pub tool_calls: usize, + pub file_edits: usize, + pub markers: usize, + pub relationships: usize, + /// Subagent metadata files that named no child session id. + pub unlinked_subagents: usize, + /// `Task`-style calls in the transcript. Grok names no child in the call + /// itself, so these produce a tool call and never a relationship row. + pub subagent_calls: usize, + /// Records whose time came from their turn's `turnStartMs`. + pub turn_start_fallbacks: usize, + /// Records that inherited the previous record's time. + pub inherited_fallbacks: usize, + /// Records stamped with the session's `created_at`. + pub session_start_fallbacks: usize, + pub encrypted_reasoning: usize, + pub unread_update_rows: usize, + /// The session directory had no `updates.jsonl` at all. + pub missing_updates: bool, + /// The stream was there and established no timing at all — no turn, no + /// message group, no tool. Distinct from `missing_updates`, because the + /// two want different answers: a session Grok wrote no updates for is + /// normal, and a stream this parser could not use is a gap in coverage + /// worth knowing about. + pub updates_yielded_no_timing: bool, + /// The newest `turn_completed` context snapshot, for the caller's report. + pub context_total_tokens: Option, + /// Turns whose `turn_completed` usage breakdown was stored on one of the + /// turn's assistant rows, and how many turns `updates.jsonl` opened in + /// all. + pub usage_turns: usize, + pub turns: usize, +} + +/// The request span of a Grok assistant row: the `updates.jsonl` turn it +/// belongs to. +/// +/// Grok names none of its API calls, and the only usage it reports is one +/// `turn_completed.usage` per turn, so the turn is the finest unit a request +/// can be measured at. Without a shared span each assistant row of a turn — +/// its thinking, every tool call, its prose — grouped as a request of its +/// own, and a turn that called a tool read as several requests, all but one +/// unmeasured. +fn grok_turn_span(turn: Option) -> Option { + turn.map(|turn| turn.to_string()) +} + +/// Index one Grok session directory: prompts, events, tools, edits, markers +/// and subagent delegations. +fn ingest_grok_session( + conn: &Connection, + session: &GrokSession, + raw_path: &str, +) -> Result { + const SOURCE: &str = "grok"; + let sid = session.session_id.as_str(); + let project = session.cwd.as_deref(); + let branch = session.git_branch.as_deref(); + replace_grok_session_evidence(conn, sid)?; + let mut outcome = GrokIngestOutcome { + missing_updates: !session.updates_present, + updates_yielded_no_timing: session.updates_present && session.updates.is_empty(), + unread_update_rows: session.updates.unread_rows, + context_total_tokens: session + .updates + .turns + .iter() + .rev() + .find_map(|turn| turn.total_tokens), + usage_turns: session + .updates + .turns + .iter() + .filter(|turn| turn.usage.is_some()) + .count(), + turns: session.updates.turns.len(), + ..Default::default() + }; + + let mut user_ordinal = 0usize; + let mut message_ordinal = 0usize; let mut thought_ordinal = 0usize; let mut first_prompt: Option = None; // Chat-side turn index: `None` until the first typed prompt, because the @@ -12041,38 +13343,91 @@ fn replace_grok_session_evidence(conn: &Connection, session_id: &str) -> Result< ] { conn.execute(statement, params![session_id])?; } - // `history` is keyed `(source, timestamp_ms, prompt)` and inserted with - // `INSERT OR IGNORE`, so a prompt two sessions both contain is **one row**, - // attributed to whichever session was indexed first. Deleting this - // session's rows outright would therefore delete a prompt that another, - // unchanged session still has -- it would vanish from search with nothing - // to say it had ever been there, and nothing would ever put it back, - // because that session's own files never change again. - // - // So a row this session no longer owns is **re-attributed** rather than - // skipped. Skipping was the other option and is worse: it would leave the - // row filed under a session that no longer contains the prompt, which is - // a false statement about provenance, and would leave it undeletable -- - // the only session that could ever clean it up is the one that no longer - // evidences it. Re-attribution moves the row to a session whose stored - // events actually carry that prompt at that moment, so the row stays true - // and stays owned. + // Every Grok prompt written before this parser carried a timestamp + // synthesized as `created_at + index`; merging would keep the fabricated + // ones forever, so history is replaced, not merged. + replace_session_history(conn, "grok", session_id)?; conn.execute( - "UPDATE history AS h SET session_id = (SELECT e.session_id FROM session_events e WHERE e.source = 'grok' AND e.role = 'user' AND e.kind = 'text' AND e.session_id <> h.session_id AND e.ts_ms = h.timestamp_ms AND e.text = h.prompt ORDER BY e.session_id LIMIT 1), project = (SELECT e.project FROM session_events e WHERE e.source = 'grok' AND e.role = 'user' AND e.kind = 'text' AND e.session_id <> h.session_id AND e.ts_ms = h.timestamp_ms AND e.text = h.prompt ORDER BY e.session_id LIMIT 1) WHERE h.source = 'grok' AND h.session_id = ? AND EXISTS(SELECT 1 FROM session_events e WHERE e.source = 'grok' AND e.role = 'user' AND e.kind = 'text' AND e.session_id <> h.session_id AND e.ts_ms = h.timestamp_ms AND e.text = h.prompt)", - params![session_id], - )?; - for statement in [ - // What is left is this session's alone. A prompt's identity is - // `(source, timestamp_ms, prompt)`, and every Grok prompt written - // before this parser carried a timestamp synthesized as - // `created_at + index`; merging would keep the fabricated ones - // forever. - "DELETE FROM history WHERE source = 'grok' AND session_id = ?", "DELETE FROM session_relationships WHERE source = 'grok' \ AND parent_session_id = ? AND evidence_kind = 'grok_subagent_dir'", - ] { - conn.execute(statement, params![session_id])?; - } + params![session_id], + )?; + Ok(()) +} + +/// Drop the `history` rows one session owns, ahead of a whole-session +/// re-read that writes them again. +/// +/// `history` is keyed `(source, timestamp_ms, prompt)` and inserted with +/// `INSERT OR IGNORE`, so a prompt two sessions both contain is **one row**, +/// attributed to whichever session was indexed first. Deleting this +/// session's rows outright would therefore delete a prompt that another, +/// unchanged session still has -- it would vanish from search with nothing +/// to say it had ever been there, and nothing would ever put it back, +/// because that session's own files never change again. +/// +/// So a row this session no longer owns is **re-attributed** rather than +/// skipped. Skipping was the other option and is worse: it would leave the +/// row filed under a session that no longer contains the prompt, which is +/// a false statement about provenance, and would leave it undeletable -- +/// the only session that could ever clean it up is the one that no longer +/// evidences it. Re-attribution moves the row to a session whose stored +/// events actually carry that prompt at that moment, so the row stays true +/// and stays owned. What is left is this session's alone, and is deleted. +fn replace_session_history(conn: &Connection, source: &str, session_id: &str) -> Result<()> { + replace_session_history_scoped(conn, source, session_id, HistoryScope::Session) +} + +/// Which of a session's `history` rows a re-read owns. +#[derive(Clone, Copy, PartialEq, Eq)] +enum HistoryScope { + /// Every row filed under the session. + Session, + /// Only the rows its own stored prompt events name — what the previous + /// read of the same transcript wrote. For a session whose identity it + /// shares with remote evidence, whose rows it did not write. Must run + /// before those events are replaced. + OwnPromptEvents, +} + +/// [`replace_session_history`] over the rows `scope` names: each is +/// re-attributed to another session that still evidences it, or deleted. +fn replace_session_history_scoped( + conn: &Connection, + source: &str, + session_id: &str, + scope: HistoryScope, +) -> Result<()> { + let owned = match scope { + HistoryScope::Session => "", + HistoryScope::OwnPromptEvents => { + " AND EXISTS(SELECT 1 FROM session_events own \ + WHERE own.source = h.source AND own.session_id = h.session_id \ + AND own.role = 'user' AND own.kind = 'text' \ + AND own.ts_ms = h.timestamp_ms AND own.text = h.prompt)" + } + }; + let update = format!( + "UPDATE history AS h SET \ + session_id = (SELECT e.session_id FROM session_events e \ + WHERE e.source = h.source AND e.role = 'user' AND e.kind = 'text' \ + AND e.session_id <> h.session_id \ + AND e.ts_ms = h.timestamp_ms AND e.text = h.prompt \ + ORDER BY e.session_id LIMIT 1), \ + project = (SELECT e.project FROM session_events e \ + WHERE e.source = h.source AND e.role = 'user' AND e.kind = 'text' \ + AND e.session_id <> h.session_id \ + AND e.ts_ms = h.timestamp_ms AND e.text = h.prompt \ + ORDER BY e.session_id LIMIT 1) \ + WHERE h.source = ? AND h.session_id = ? \ + AND EXISTS(SELECT 1 FROM session_events e \ + WHERE e.source = h.source AND e.role = 'user' AND e.kind = 'text' \ + AND e.session_id <> h.session_id \ + AND e.ts_ms = h.timestamp_ms AND e.text = h.prompt){owned}", + ); + conn.execute(&update, params![source, session_id])?; + let delete = format!("DELETE FROM history AS h WHERE h.source = ? AND h.session_id = ?{owned}"); + conn.execute(&delete, params![source, session_id])?; Ok(()) } @@ -13997,855 +15352,1819 @@ mod tests { save_sync_state(&path, &state).unwrap(); assert_eq!(load_sync_state(&path).unwrap(), state); - // The temp file is renamed away, never left beside the real one. - assert_eq!(leftover_tmp_files(&dir), Vec::::new()); + // The temp file is renamed away, never left beside the real one. + assert_eq!(leftover_tmp_files(&dir), Vec::::new()); + + fs::remove_dir_all(&dir).ok(); + } + + /// Temp files staged by `save_sync_state`, which should never outlive a save. + fn leftover_tmp_files(dir: &std::path::Path) -> Vec { + let mut found: Vec = fs::read_dir(dir) + .unwrap() + .map(|entry| entry.unwrap().file_name().to_string_lossy().into_owned()) + .filter(|name| name.starts_with(".sync-state.json.tmp")) + .collect(); + found.sort(); + found + } + + #[test] + fn stale_sync_state_temps_are_removed_without_touching_a_live_writer() { + let dir = + std::env::temp_dir().join(format!("ai-hist-state-cleanup-{}", std::process::id())); + fs::create_dir_all(&dir).unwrap(); + let path = dir.join(".sync-state.json"); + let stale = dir.join(".sync-state.json.tmp.999999999.0"); + let live = dir.join(format!(".sync-state.json.tmp.{}.0", std::process::id())); + fs::write(&stale, "stale").unwrap(); + fs::write(&live, "live").unwrap(); + + assert_eq!(cleanup_stale_sync_state_temps(&path).unwrap(), 1); + assert!(!stale.exists()); + assert!(live.exists()); + + fs::remove_dir_all(&dir).ok(); + } + + #[test] + fn a_failed_source_does_not_prevent_later_sources_from_completing() { + let db_path = std::path::Path::new("unused.db"); + let mut report = SyncSourceReport::new(db_path); + assert!(report + .capture::("broken", Err(anyhow::anyhow!("bad source"))) + .is_none()); + assert_eq!(report.capture("healthy", Ok(7)), Some(7)); + assert_eq!(report.succeeded, 1); + assert_eq!(report.failures.len(), 1); + assert!(report.finish().is_ok()); + + let mut all_failed = SyncSourceReport::new(db_path); + all_failed.capture::("only", Err(anyhow::anyhow!("still bad"))); + assert!(all_failed.finish().is_err()); + } + + #[test] + fn contention_diagnostics_reprobe_write_capability() { + let dir = std::env::temp_dir().join(format!("ai-hist-contention-{}", std::process::id())); + fs::create_dir_all(&dir).unwrap(); + let path = dir.join("contention.db"); + let holder = open_db(&path).unwrap(); + holder.execute_batch("BEGIN IMMEDIATE").unwrap(); + + let contender = Connection::open(&path).unwrap(); + contender.busy_timeout(std::time::Duration::ZERO).unwrap(); + let busy = contender + .execute_batch("BEGIN IMMEDIATE") + .expect_err("the competing writer must be busy"); + assert!(is_sqlite_contention(&anyhow::Error::new(busy))); + let blocked = write_contention_diagnostic(&path); + assert!(blocked.contains("write capability probe is still blocked")); + + holder.execute_batch("ROLLBACK").unwrap(); + let recovered = write_contention_diagnostic(&path); + assert!(recovered.contains("write capability probe now succeeds")); + + fs::remove_dir_all(&dir).ok(); + } + + #[test] + fn contention_diagnostics_keep_source_paths_and_recovered_wal_wording_accurate() { + let source = std::path::PathBuf::from("/tmp/opencode-source.db"); + let busy = rusqlite::Error::SqliteFailure( + rusqlite::ffi::Error::new(rusqlite::ffi::SQLITE_BUSY), + None, + ); + let error = anyhow::Error::new(SourceDatabaseError::new(&source, busy)); + assert!(is_sqlite_contention(&error)); + assert_eq!(source_database_path(&error), Some(source.as_path())); + + let recovered = wal_contention_line(WAL_WARN_BYTES + 1, false).unwrap(); + assert!( + recovered.contains("write capability recovered"), + "{recovered}" + ); + assert!(!recovered.contains("write path is failing"), "{recovered}"); + let blocked = wal_contention_line(WAL_WARN_BYTES + 1, true).unwrap(); + assert!(blocked.contains("write path is failing"), "{blocked}"); + } + + #[test] + fn process_status_uses_an_absolute_fallback_when_path_lookup_fails() { + let pid = std::process::id().to_string(); + let (state, command) = process_status_with_programs( + &pid, + &["/definitely/missing/ps", "/bin/ps", "/usr/bin/ps"], + ) + .expect("an absolute system ps should describe the current process"); + assert!(!state.is_empty()); + assert!(!command.is_empty()); + } + + #[test] + fn a_slow_run_cannot_rewind_or_clobber_a_faster_runs_cursors() { + let dir = std::env::temp_dir().join(format!("ai-hist-merge-{}", std::process::id())); + fs::create_dir_all(&dir).unwrap(); + let path = dir.join(".sync-state.json"); + + // A fast run finished claude and codex and moved on. + let mut fast = Map::new(); + fast.insert("claude".into(), json!(900)); + fast.insert("codex".into(), json!(500)); + checkpoint_sync_state(&path, &fast); + + // A slow overlapping run only just finished claude, at an older offset. + // Writing its whole map wholesale would rewind claude and delete codex, + // sending the next run back over work that was already done. + let mut slow = Map::new(); + slow.insert("claude".into(), json!(400)); + checkpoint_sync_state(&path, &slow); + + let on_disk = load_sync_state(&path).unwrap(); + assert_eq!(on_disk.get("claude").and_then(Value::as_u64), Some(900)); + assert_eq!(on_disk.get("codex").and_then(Value::as_u64), Some(500)); + + // A genuinely newer cursor still advances. + let mut newer = Map::new(); + newer.insert("claude".into(), json!(1200)); + checkpoint_sync_state(&path, &newer); + assert_eq!( + load_sync_state(&path) + .unwrap() + .get("claude") + .and_then(Value::as_u64), + Some(1200) + ); + + fs::remove_dir_all(&dir).ok(); + } + + #[test] + fn typed_cursor_merges_are_monotonic_within_a_generation_and_replace_stale_generations() { + let dir = tempfile::tempdir().unwrap(); + let path = dir.path().join(".sync-state.json"); + let transcript = "/cursor/session.jsonl"; + + let mut newest = Map::new(); + newest.insert("claude".into(), test_file_cursor(25, 10, 200)); + newest.insert( + super::CURSOR_SYNC_STATE_KEY.into(), + json!({transcript: test_file_cursor(30, 10, 200)}), + ); + checkpoint_sync_state(&path, &newest); + + // A stale writer from the prior inode cannot restore its larger offset. + let mut stale_generation = Map::new(); + stale_generation.insert("claude".into(), test_file_cursor(900, 9, 100)); + stale_generation.insert( + super::CURSOR_SYNC_STATE_KEY.into(), + json!({transcript: test_file_cursor(800, 9, 100)}), + ); + checkpoint_sync_state(&path, &stale_generation); + + // Nor can a slow writer rewind the active generation. + let mut slow_same_generation = Map::new(); + let mut later_open = decode_typed_cursor(test_file_cursor(12, 10, 200)); + later_open.generation.started_mtime_ns += 10; + later_open.generation.started_size += 50; + later_open.generation.observed_at_ns += 10; + slow_same_generation.insert("claude".into(), later_open.to_value()); + slow_same_generation.insert( + super::CURSOR_SYNC_STATE_KEY.into(), + json!({transcript: test_file_cursor(14, 10, 200)}), + ); + checkpoint_sync_state(&path, &slow_same_generation); + + let saved = load_sync_state(&path).unwrap(); + assert_eq!(saved_cursor_offset(&saved["claude"]), 25); + assert_eq!( + saved_cursor_offset(&saved[super::CURSOR_SYNC_STATE_KEY][transcript]), + 30 + ); + } + + #[test] + fn a_same_inode_rewrite_epoch_supersedes_the_old_larger_offset() { + let dir = tempfile::tempdir().unwrap(); + let path = dir.path().join(".sync-state.json"); + let mut old = Map::new(); + old.insert("claude".into(), test_file_cursor(900, 10, 100)); + checkpoint_sync_state(&path, &old); + + let mut reset = decode_typed_cursor(test_file_cursor(0, 10, 200)); + reset.generation.rewrite_epoch = 1; + let mut rewritten = Map::new(); + rewritten.insert("claude".into(), reset.to_value()); + checkpoint_sync_state(&path, &rewritten); + + let saved = load_sync_state(&path).unwrap(); + assert_eq!(saved_cursor_offset(&saved["claude"]), 0); + } + + #[test] + fn a_new_inode_wins_by_observation_time_even_with_a_lower_rewrite_epoch() { + let dir = tempfile::tempdir().unwrap(); + let path = dir.path().join(".sync-state.json"); + let mut old_cursor = decode_typed_cursor(test_file_cursor(900, 10, 100)); + old_cursor.generation.rewrite_epoch = 10; + let mut old = Map::new(); + old.insert("claude".into(), old_cursor.to_value()); + checkpoint_sync_state(&path, &old); + + let mut replacement = decode_typed_cursor(test_file_cursor(0, 11, 200)); + replacement.generation.rewrite_epoch = 2; + let mut new = Map::new(); + new.insert("claude".into(), replacement.to_value()); + checkpoint_sync_state(&path, &new); + + let saved = load_sync_state(&path).unwrap(); + assert_eq!(saved_cursor_offset(&saved["claude"]), 0); + let saved = decode_typed_cursor(saved["claude"].clone()); + assert_eq!(saved.generation.inode, Some(11)); + } + + #[test] + fn sync_state_merge_recurses_through_all_object_maps() { + let dir = tempfile::tempdir().unwrap(); + let path = dir.path().join(".sync-state.json"); + let mut on_disk = Map::new(); + on_disk.insert( + "claude_sessions_v3".into(), + json!({"first.jsonl": "old", "nested": {"left": 1}}), + ); + save_sync_state(&path, &on_disk).unwrap(); + + let mut ours = Map::new(); + ours.insert( + "claude_sessions_v3".into(), + json!({"second.jsonl": "new", "nested": {"right": 2}}), + ); + checkpoint_sync_state(&path, &ours); + + assert_eq!( + load_sync_state(&path).unwrap()["claude_sessions_v3"], + json!({ + "first.jsonl": "old", + "second.jsonl": "new", + "nested": {"left": 1, "right": 2} + }) + ); + } - fs::remove_dir_all(&dir).ok(); + /// A generation migration drops its predecessor from the in-memory map, but + /// the merge only folds in the keys a run *has*, so on its own that deletion + /// never reaches disk: the retired map is reloaded and rewritten forever. + /// A Grok session a previous release already consumed must be re-read + /// once, and must come back with evidence instead of prompts alone. + /// + /// The change stamp of that session is unchanged on disk, so nothing but + /// retiring the sync-state key can make `sync` look at it again — and the + /// prompts it wrote with synthesized timestamps must be replaced, not + /// joined by a second copy carrying the real ones. + /// Build a Grok session directory from raw lines and index it. + fn ingest_grok_lines(home: &Path, chat: &str, updates: &str) -> Connection { + let dir = home.join(".grok/sessions/%2Ftmp%2Ffallback/grok-fb-0001"); + fs::create_dir_all(&dir).unwrap(); + fs::write(dir.join("chat_history.jsonl"), chat).unwrap(); + fs::write(dir.join("updates.jsonl"), updates).unwrap(); + fs::write( + dir.join("summary.json"), + br#"{"info":{"id":"grok-fb-0001","cwd":"/tmp/fallback"},"created_at":"2026-01-01T00:00:00.000Z"}"#, + ) + .unwrap(); + let conn = open_db(&home.join("history.db")).unwrap(); + let chat_path = dir.join("chat_history.jsonl"); + let session = super::scan_grok_session_file(&chat_path).unwrap().unwrap(); + super::ingest_grok_session(&conn, &session, &chat_path.to_string_lossy()).unwrap(); + conn } - /// Temp files staged by `save_sync_state`, which should never outlive a save. - fn leftover_tmp_files(dir: &std::path::Path) -> Vec { - let mut found: Vec = fs::read_dir(dir) - .unwrap() - .map(|entry| entry.unwrap().file_name().to_string_lossy().into_owned()) - .filter(|name| name.starts_with(".sync-state.json.tmp")) - .collect(); - found.sort(); - found + fn grok_event_time(conn: &Connection, text: &str) -> i64 { + conn.query_row( + "SELECT ts_ms FROM session_events WHERE source = 'grok' AND text = ?", + params![text], + |row| row.get(0), + ) + .unwrap() } + /// An assistant message with no recorded time of its own falls back to + /// **its own** turn's start. Resolving against a turn index captured + /// before the message told us which turn it was in hands it the previous + /// turn's start — a time from before the person had even asked. #[test] - fn stale_sync_state_temps_are_removed_without_touching_a_live_writer() { - let dir = - std::env::temp_dir().join(format!("ai-hist-state-cleanup-{}", std::process::id())); - fs::create_dir_all(&dir).unwrap(); - let path = dir.join(".sync-state.json"); - let stale = dir.join(".sync-state.json.tmp.999999999.0"); - let live = dir.join(format!(".sync-state.json.tmp.{}.0", std::process::id())); - fs::write(&stale, "stale").unwrap(); - fs::write(&live, "live").unwrap(); - - assert_eq!(cleanup_stale_sync_state_temps(&path).unwrap(), 1); - assert!(!stale.exists()); - assert!(live.exists()); + fn an_untimed_assistant_message_falls_back_to_its_own_turns_start() { + let home = tempfile::tempdir().unwrap(); + let conn = ingest_grok_lines( + home.path(), + &[ + r#"{"type":"user","content":"first"}"#, + r#"{"type":"assistant","content":"answer one"}"#, + // The second turn is the model continuing on its own: Grok + // wrote no user record for it, so the chat side never learns + // the turn advanced and only the matched group knows. + r#"{"type":"assistant","content":"answer two"}"#, + "", + ] + .join("\n"), + &[ + r#"{"method":"session/update","params":{"update":{"sessionUpdate":"user_message_chunk"},"_meta":{"agentTimestampMs":1000,"turnStartMs":1000}}}"#, + r#"{"method":"session/update","params":{"update":{"sessionUpdate":"agent_message_chunk"},"_meta":{"agentTimestampMs":2000,"turnStartMs":1000}}}"#, + r#"{"method":"_x.ai/session/update","params":{"update":{"sessionUpdate":"turn_completed"},"_meta":{"agentTimestampMs":2500,"turnStartMs":1000}}}"#, + // The second turn opens with the model continuing: the chunk + // says which turn it belongs to, and records no time of its + // own — which is the only reason the fallback is reached. + r#"{"method":"session/update","params":{"update":{"sessionUpdate":"agent_message_chunk"},"_meta":{"eventId":"ev_late","turnStartMs":5000}}}"#, + "", + ] + .join("\n"), + ); + // The positive control: the first turn's events are exact, so the + // second one's fallback is the only thing under test. + assert_eq!(grok_event_time(&conn, "first"), 1000); + assert_eq!(grok_event_time(&conn, "answer one"), 2000); - fs::remove_dir_all(&dir).ok(); + assert_eq!( + grok_event_time(&conn, "answer two"), + 5000, + "an untimed reply belongs to the turn it answered, not the one before it" + ); } + /// A marker is a record with a place in the file: the next record with no + /// time of its own inherits the marker's, not that of whatever preceded + /// it. #[test] - fn a_failed_source_does_not_prevent_later_sources_from_completing() { - let db_path = std::path::Path::new("unused.db"); - let mut report = SyncSourceReport::new(db_path); - assert!(report - .capture::("broken", Err(anyhow::anyhow!("bad source"))) - .is_none()); - assert_eq!(report.capture("healthy", Ok(7)), Some(7)); - assert_eq!(report.succeeded, 1); - assert_eq!(report.failures.len(), 1); - assert!(report.finish().is_ok()); - - let mut all_failed = SyncSourceReport::new(db_path); - all_failed.capture::("only", Err(anyhow::anyhow!("still bad"))); - assert!(all_failed.finish().is_err()); + fn a_marker_carries_its_time_forward_to_the_next_untimed_record() { + let home = tempfile::tempdir().unwrap(); + let conn = ingest_grok_lines( + home.path(), + &[ + r#"{"type":"user","content":"first","timestamp_ms":1000}"#, + r#"{"type":"user","content":"injected","synthetic_reason":"compaction","timestamp_ms":2000}"#, + r#"{"type":"assistant","content":"after the marker"}"#, + "", + ] + .join("\n"), + "", + ); + // The positive control: the marker really was stored at 2000, so the + // event below inherits a time that exists rather than a default. + let marker: i64 = conn + .query_row( + "SELECT ts_ms FROM session_markers WHERE source = 'grok' AND kind = 'synthetic_turn'", + [], + |row| row.get(0), + ) + .unwrap(); + assert_eq!(marker, 2000); + assert_eq!( + grok_event_time(&conn, "after the marker"), + 2000, + "the nearest preceding record is the marker, not the prompt before it" + ); } + /// Reasoning and tools join their update the same way prose does, so they + /// have to adopt the turn that match establishes. A model-initiated turn + /// has no user record to advance the transcript-side cursor, so without + /// that the fallback dates them from before the turn began. #[test] - fn contention_diagnostics_reprobe_write_capability() { - let dir = std::env::temp_dir().join(format!("ai-hist-contention-{}", std::process::id())); - fs::create_dir_all(&dir).unwrap(); - let path = dir.join("contention.db"); - let holder = open_db(&path).unwrap(); - holder.execute_batch("BEGIN IMMEDIATE").unwrap(); - - let contender = Connection::open(&path).unwrap(); - contender.busy_timeout(std::time::Duration::ZERO).unwrap(); - let busy = contender - .execute_batch("BEGIN IMMEDIATE") - .expect_err("the competing writer must be busy"); - assert!(is_sqlite_contention(&anyhow::Error::new(busy))); - let blocked = write_contention_diagnostic(&path); - assert!(blocked.contains("write capability probe is still blocked")); - - holder.execute_batch("ROLLBACK").unwrap(); - let recovered = write_contention_diagnostic(&path); - assert!(recovered.contains("write capability probe now succeeds")); + fn untimed_reasoning_and_tools_fall_back_to_their_own_turns_start() { + let home = tempfile::tempdir().unwrap(); + let conn = ingest_grok_lines( + home.path(), + &[ + r#"{"type":"user","content":"first"}"#, + r#"{"type":"reasoning","summary":"thinking in turn zero"}"#, + r#"{"type":"assistant","content":"answer one"}"#, + // The model continues on its own: no user record opens turn 1. + r#"{"type":"reasoning","summary":"thinking in turn one"}"#, + r#"{"type":"assistant","content":"","tool_calls":[{"id":"call_late","name":"Shell","arguments":{"command":"ls"}}]}"#, + r#"{"type":"tool_result","tool_call_id":"call_late","content":"done"}"#, + "", + ] + .join("\n"), + &[ + r#"{"method":"session/update","params":{"update":{"sessionUpdate":"user_message_chunk"},"_meta":{"agentTimestampMs":1000,"turnStartMs":1000}}}"#, + r#"{"method":"session/update","params":{"update":{"sessionUpdate":"agent_thought_chunk"},"_meta":{"agentTimestampMs":1100,"turnStartMs":1000}}}"#, + r#"{"method":"session/update","params":{"update":{"sessionUpdate":"agent_message_chunk"},"_meta":{"agentTimestampMs":2000,"turnStartMs":1000}}}"#, + r#"{"method":"_x.ai/session/update","params":{"update":{"sessionUpdate":"turn_completed"},"_meta":{"agentTimestampMs":2500,"turnStartMs":1000}}}"#, + // Turn 1 opens on the model's own thinking, and none of its + // rows records a time of its own. + r#"{"method":"session/update","params":{"update":{"sessionUpdate":"agent_thought_chunk"},"_meta":{"eventId":"ev_think","turnStartMs":5000}}}"#, + r#"{"method":"session/update","params":{"update":{"sessionUpdate":"tool_call","toolCallId":"call_late"},"_meta":{"turnStartMs":5000}}}"#, + r#"{"method":"session/update","params":{"update":{"sessionUpdate":"tool_call_update","toolCallId":"call_late","status":"completed"},"_meta":{"turnStartMs":5000}}}"#, + "", + ] + .join("\n"), + ); + // The positive control: turn 0 is exact, so only the fallbacks below + // are under test. + assert_eq!(grok_event_time(&conn, "first"), 1000); + assert_eq!(grok_event_time(&conn, "thinking in turn zero"), 1100); + assert_eq!(grok_event_time(&conn, "answer one"), 2000); - fs::remove_dir_all(&dir).ok(); + assert_eq!( + grok_event_time(&conn, "thinking in turn one"), + 5000, + "a thought belongs to the turn its group names, not the previous one" + ); + let tool_times: Vec = conn + .prepare( + "SELECT ts_ms FROM session_events WHERE source = 'grok' \ + AND event_uid IN ('tool:call_late', 'result:call_late') ORDER BY event_uid", + ) + .unwrap() + .query_map([], |row| row.get(0)) + .unwrap() + .collect::>>() + .unwrap(); + assert_eq!( + tool_times, + vec![5000, 5000], + "a call and its result join by id, so their turn is known exactly" + ); } + /// ACP's terminal status is read case-blind for the call as for its + /// result, so a `Cancelled` call is not stored as a success beside a + /// cancelled result. #[test] - fn contention_diagnostics_keep_source_paths_and_recovered_wal_wording_accurate() { - let source = std::path::PathBuf::from("/tmp/opencode-source.db"); - let busy = rusqlite::Error::SqliteFailure( - rusqlite::ffi::Error::new(rusqlite::ffi::SQLITE_BUSY), - None, + fn a_mixed_case_acp_failure_marks_the_grok_call_failed() { + let home = tempfile::tempdir().unwrap(); + let conn = ingest_grok_lines( + home.path(), + &[ + r#"{"type":"user","content":"first"}"#, + r#"{"type":"assistant","content":"","tool_calls":[{"id":"call_x","name":"Shell","arguments":{"command":"ls"}}]}"#, + r#"{"type":"tool_result","tool_call_id":"call_x","content":"stopped"}"#, + "", + ] + .join("\n"), + &[ + r#"{"method":"session/update","params":{"update":{"sessionUpdate":"user_message_chunk"},"_meta":{"agentTimestampMs":1000,"turnStartMs":1000}}}"#, + r#"{"method":"session/update","params":{"update":{"sessionUpdate":"tool_call","toolCallId":"call_x"},"_meta":{"agentTimestampMs":1100,"turnStartMs":1000}}}"#, + r#"{"method":"session/update","params":{"update":{"sessionUpdate":"tool_call_update","toolCallId":"call_x","status":"Cancelled"},"_meta":{"agentTimestampMs":1200,"turnStartMs":1000}}}"#, + "", + ] + .join("\n"), ); - let error = anyhow::Error::new(SourceDatabaseError::new(&source, busy)); - assert!(is_sqlite_contention(&error)); - assert_eq!(source_database_path(&error), Some(source.as_path())); - - let recovered = wal_contention_line(WAL_WARN_BYTES + 1, false).unwrap(); - assert!( - recovered.contains("write capability recovered"), - "{recovered}" + let (is_error, result_status): (Option, Option) = conn + .query_row( + "SELECT (SELECT is_error FROM tool_calls WHERE source = 'grok' AND tool_use_id = 'call_x'), \ + (SELECT result_status FROM session_events \ + WHERE source = 'grok' AND event_uid = 'result:call_x')", + [], + |row| Ok((row.get(0)?, row.get(1)?)), + ) + .unwrap(); + assert_eq!( + (is_error, result_status.as_deref()), + (Some(1), Some("cancelled")) ); - assert!(!recovered.contains("write path is failing"), "{recovered}"); - let blocked = wal_contention_line(WAL_WARN_BYTES + 1, true).unwrap(); - assert!(blocked.contains("write path is failing"), "{blocked}"); } + /// Encrypted-only reasoning still occupies an `agent_thoughts` ordinal. + /// Leaving that cursor unmoved handed the next readable thought the + /// encrypted record's timestamp and event id. #[test] - fn process_status_uses_an_absolute_fallback_when_path_lookup_fails() { - let pid = std::process::id().to_string(); - let (state, command) = process_status_with_programs( - &pid, - &["/definitely/missing/ps", "/bin/ps", "/usr/bin/ps"], - ) - .expect("an absolute system ps should describe the current process"); - assert!(!state.is_empty()); - assert!(!command.is_empty()); + fn encrypted_reasoning_consumes_its_thought_chunk_ordinal() { + let home = tempfile::tempdir().unwrap(); + let conn = ingest_grok_lines( + home.path(), + &[ + r#"{"type":"user","content":"first"}"#, + r#"{"type":"reasoning","encrypted_content":"opaque"}"#, + r#"{"type":"reasoning","summary":"visible thought"}"#, + r#"{"type":"assistant","content":"done"}"#, + "", + ] + .join("\n"), + &[ + r#"{"method":"session/update","params":{"update":{"sessionUpdate":"user_message_chunk"},"_meta":{"agentTimestampMs":1000,"turnStartMs":1000}}}"#, + r#"{"method":"session/update","params":{"update":{"sessionUpdate":"agent_thought_chunk"},"_meta":{"agentTimestampMs":1100,"turnStartMs":1000}}}"#, + // A new turnStartMs starts a second thought group. Consecutive + // chunks in the same turn would merge into one group, which + // would not exercise the ordinal join this test is about. + r#"{"method":"session/update","params":{"update":{"sessionUpdate":"agent_thought_chunk"},"_meta":{"eventId":"ev_visible","agentTimestampMs":2000,"turnStartMs":2000}}}"#, + r#"{"method":"session/update","params":{"update":{"sessionUpdate":"agent_message_chunk"},"_meta":{"agentTimestampMs":3000,"turnStartMs":2000}}}"#, + "", + ] + .join("\n"), + ); + let encrypted: i64 = conn + .query_row( + "SELECT ts_ms FROM session_markers WHERE source = 'grok' AND kind = 'encrypted_reasoning'", + [], + |row| row.get(0), + ) + .unwrap(); + assert_eq!( + encrypted, 1100, + "the opaque trace uses its own chunk's time" + ); + assert_eq!( + grok_event_time(&conn, "visible thought"), + 2000, + "the next thought must not inherit the encrypted chunk" + ); + let uid: String = conn + .query_row( + "SELECT event_uid FROM session_events WHERE source = 'grok' AND text = 'visible thought'", + [], + |row| row.get(0), + ) + .unwrap(); + assert_eq!(uid, "ev:ev_visible"); } + /// `sessions.first_prompt` is a bounded catalog excerpt. Hydration must + /// not overwrite discovery's 4,096-character field with the full prompt. #[test] - fn a_slow_run_cannot_rewind_or_clobber_a_faster_runs_cursors() { - let dir = std::env::temp_dir().join(format!("ai-hist-merge-{}", std::process::id())); - fs::create_dir_all(&dir).unwrap(); - let path = dir.join(".sync-state.json"); - - // A fast run finished claude and codex and moved on. - let mut fast = Map::new(); - fast.insert("claude".into(), json!(900)); - fast.insert("codex".into(), json!(500)); - checkpoint_sync_state(&path, &fast); - - // A slow overlapping run only just finished claude, at an older offset. - // Writing its whole map wholesale would rewind claude and delete codex, - // sending the next run back over work that was already done. - let mut slow = Map::new(); - slow.insert("claude".into(), json!(400)); - checkpoint_sync_state(&path, &slow); - - let on_disk = load_sync_state(&path).unwrap(); - assert_eq!(on_disk.get("claude").and_then(Value::as_u64), Some(900)); - assert_eq!(on_disk.get("codex").and_then(Value::as_u64), Some(500)); - - // A genuinely newer cursor still advances. - let mut newer = Map::new(); - newer.insert("claude".into(), json!(1200)); - checkpoint_sync_state(&path, &newer); + fn grok_first_prompt_is_the_bounded_catalog_excerpt() { + let home = tempfile::tempdir().unwrap(); + let long = "x".repeat(crate::discover::EXCERPT_MAX_CHARS + 80); + let conn = ingest_grok_lines( + home.path(), + &format!("{{\"type\":\"user\",\"content\":\"{long}\"}}\n"), + concat!( + r#"{"method":"session/update","params":{"update":{"sessionUpdate":"user_message_chunk"},"_meta":{"agentTimestampMs":1000,"turnStartMs":1000}}}"#, + "\n", + ), + ); + let stored: String = conn + .query_row( + "SELECT first_prompt FROM sessions WHERE source = 'grok'", + [], + |row| row.get(0), + ) + .unwrap(); + assert_eq!(stored.chars().count(), crate::discover::EXCERPT_MAX_CHARS); + let history: String = conn + .query_row( + "SELECT prompt FROM history WHERE source = 'grok'", + [], + |row| row.get(0), + ) + .unwrap(); assert_eq!( - load_sync_state(&path) - .unwrap() - .get("claude") - .and_then(Value::as_u64), - Some(1200) + history.chars().count(), + crate::discover::EXCERPT_MAX_CHARS + 80, + "the transcript keeps the full prompt; only the catalog excerpt is bounded" ); - - fs::remove_dir_all(&dir).ok(); } + /// A directory that cannot be read is not an empty directory. + /// + /// The read is a replacement: an unreadable `compaction_checkpoints/` + /// taken as empty would stamp the session as having no checkpoints and + /// then delete the ones already stored, reporting success. It has to fail + /// so the transaction rolls back. #[test] - fn typed_cursor_merges_are_monotonic_within_a_generation_and_replace_stale_generations() { - let dir = tempfile::tempdir().unwrap(); - let path = dir.path().join(".sync-state.json"); - let transcript = "/cursor/session.jsonl"; + fn an_unreadable_sidecar_directory_fails_instead_of_erasing_the_evidence() { + let home = tempfile::tempdir().unwrap(); + let dir = home + .path() + .join(".grok/sessions/%2Ftmp%2Fbroken/grok-brk-0001"); + fs::create_dir_all(dir.join("compaction_checkpoints")).unwrap(); + fs::write( + dir.join("chat_history.jsonl"), + b"{\"type\":\"user\",\"content\":\"hello\"}\n", + ) + .unwrap(); + fs::write( + dir.join("summary.json"), + br#"{"info":{"id":"grok-brk-0001","cwd":"/tmp/broken"},"created_at":"2026-01-01T00:00:00.000Z"}"#, + ) + .unwrap(); + fs::write( + dir.join("compaction_checkpoints/1700000000000.json"), + br#"{"created_at":"2026-01-01T00:10:00.000Z"}"#, + ) + .unwrap(); + let chat = dir.join("chat_history.jsonl"); - let mut newest = Map::new(); - newest.insert("claude".into(), test_file_cursor(25, 10, 200)); - newest.insert( - super::CURSOR_SYNC_STATE_KEY.into(), - json!({transcript: test_file_cursor(30, 10, 200)}), - ); - checkpoint_sync_state(&path, &newest); + let conn = open_db(&home.path().join("history.db")).unwrap(); + let session = super::scan_grok_session_file(&chat).unwrap().unwrap(); + super::ingest_grok_session(&conn, &session, &chat.to_string_lossy()).unwrap(); + let markers = || -> i64 { + conn.query_row( + "SELECT COUNT(*) FROM session_markers WHERE source = 'grok' AND kind = 'compaction_boundary'", + [], + |row| row.get(0), + ) + .unwrap() + }; + let prompts = || -> i64 { + conn.query_row( + "SELECT COUNT(*) FROM history WHERE source = 'grok'", + [], + |row| row.get(0), + ) + .unwrap() + }; + assert_eq!(markers(), 1); + assert_eq!(prompts(), 1); - // A stale writer from the prior inode cannot restore its larger offset. - let mut stale_generation = Map::new(); - stale_generation.insert("claude".into(), test_file_cursor(900, 9, 100)); - stale_generation.insert( - super::CURSOR_SYNC_STATE_KEY.into(), - json!({transcript: test_file_cursor(800, 9, 100)}), - ); - checkpoint_sync_state(&path, &stale_generation); + // An entry that `read_dir` lists and `stat` cannot resolve: a symlink + // to itself is ELOOP on every platform that has symlinks. + let loop_entry = dir.join("compaction_checkpoints/loop.json"); + #[cfg(unix)] + std::os::unix::fs::symlink(&loop_entry, &loop_entry).unwrap(); + #[cfg(not(unix))] + return; - // Nor can a slow writer rewind the active generation. - let mut slow_same_generation = Map::new(); - let mut later_open = decode_typed_cursor(test_file_cursor(12, 10, 200)); - later_open.generation.started_mtime_ns += 10; - later_open.generation.started_size += 50; - later_open.generation.observed_at_ns += 10; - slow_same_generation.insert("claude".into(), later_open.to_value()); - slow_same_generation.insert( - super::CURSOR_SYNC_STATE_KEY.into(), - json!({transcript: test_file_cursor(14, 10, 200)}), - ); - checkpoint_sync_state(&path, &slow_same_generation); + // The stamp fails rather than describing a directory it could not read. + assert!(super::grok_session_stamp(&chat).is_err()); - let saved = load_sync_state(&path).unwrap(); - assert_eq!(saved_cursor_offset(&saved["claude"]), 25); - assert_eq!( - saved_cursor_offset(&saved[super::CURSOR_SYNC_STATE_KEY][transcript]), - 30 - ); + // And so does the read, so nothing is replaced. + let scanned = super::scan_grok_session_file(&chat); + assert!(scanned.is_err(), "an unreadable entry must not scan clean"); + assert_eq!(markers(), 1, "the stored checkpoint marker must survive"); + assert_eq!(prompts(), 1); + + // Positive control: with the entry readable the same paths succeed and + // the second checkpoint is indexed, so the failure above is the + // unreadable entry and not the extra file. + fs::remove_file(&loop_entry).unwrap(); + fs::write(&loop_entry, br#"{"created_at":"2026-01-01T00:20:00.000Z"}"#).unwrap(); + assert!(super::grok_session_stamp(&chat).is_ok()); + let session = super::scan_grok_session_file(&chat).unwrap().unwrap(); + super::ingest_grok_session(&conn, &session, &chat.to_string_lossy()).unwrap(); + assert_eq!(markers(), 2); } - #[test] - fn a_same_inode_rewrite_epoch_supersedes_the_old_larger_offset() { - let dir = tempfile::tempdir().unwrap(); - let path = dir.path().join(".sync-state.json"); - let mut old = Map::new(); - old.insert("claude".into(), test_file_cursor(900, 10, 100)); - checkpoint_sync_state(&path, &old); - - let mut reset = decode_typed_cursor(test_file_cursor(0, 10, 200)); - reset.generation.rewrite_epoch = 1; - let mut rewritten = Map::new(); - rewritten.insert("claude".into(), reset.to_value()); - checkpoint_sync_state(&path, &rewritten); + /// The stamp sync saved for one Grok session file. + fn grok_saved_stamp<'a>(state: &'a Map, key: &str) -> Option<&'a str> { + state + .get(super::GROK_SYNC_STATE_KEY)? + .get(key) + .and_then(super::grok_state_stamp) + } - let saved = load_sync_state(&path).unwrap(); - assert_eq!(saved_cursor_offset(&saved["claude"]), 0); + /// Write a Grok session directory with no `updates.jsonl`, and answer with + /// the transcript and the directory. + fn grok_stream_fixture(home: &Path, id: &str) -> (PathBuf, PathBuf) { + let dir = home.join(".grok/sessions/%2Ftmp%2Fstream").join(id); + fs::create_dir_all(&dir).unwrap(); + fs::write( + dir.join("chat_history.jsonl"), + format!("{{\"type\":\"user\",\"content\":\"hello from {id}\"}}\n"), + ) + .unwrap(); + fs::write( + dir.join("summary.json"), + format!( + r#"{{"info":{{"id":"{id}","cwd":"/tmp/stream"}},"created_at":"2026-01-01T00:00:00.000Z"}}"# + ), + ) + .unwrap(); + (dir.join("chat_history.jsonl"), dir) } + /// A Grok session indexed by a build that stored no `turn_completed.usage` + /// has not changed on disk, so only the retired `grok_events_v1` key can + /// send plain `sync` back over it. With that key the unchanged directory + /// is re-read and gains the breakdown and its turn's request span. #[test] - fn a_new_inode_wins_by_observation_time_even_with_a_lower_rewrite_epoch() { - let dir = tempfile::tempdir().unwrap(); - let path = dir.path().join(".sync-state.json"); - let mut old_cursor = decode_typed_cursor(test_file_cursor(900, 10, 100)); - old_cursor.generation.rewrite_epoch = 10; + fn an_upgraded_sync_rereads_unchanged_grok_sessions_for_turn_usage() { + let home = tempfile::tempdir().unwrap(); + let (chat, dir) = grok_stream_fixture(home.path(), "grok-upg-0001"); + fs::write( + &chat, + concat!( + r#"{"type":"user","content":"hi"}"#, + "\n", + r#"{"type":"assistant","content":"hello","model_id":"grok-4-build"}"#, + "\n" + ), + ) + .unwrap(); + fs::write( + dir.join("updates.jsonl"), + concat!( + r#"{"method":"session/update","params":{"update":{"sessionUpdate":"user_message_chunk"},"_meta":{"eventId":"u1","agentTimestampMs":1789560000000,"turnStartMs":1789560000000}}}"#, + "\n", + r#"{"method":"session/update","params":{"update":{"sessionUpdate":"agent_message_chunk"},"_meta":{"eventId":"a1","agentTimestampMs":1789560001000,"turnStartMs":1789560000000}}}"#, + "\n", + r#"{"method":"_x.ai/session/update","params":{"update":{"sessionUpdate":"turn_completed","totalTokens":1800,"usage":{"inputTokens":1000,"outputTokens":100}},"_meta":{"agentTimestampMs":1789560002000,"turnStartMs":1789560000000}}}"#, + "\n" + ), + ) + .unwrap(); + let conn = open_db(&home.path().join("history.db")).unwrap(); + let root = home.path().join(".grok/sessions"); + let mut state = Map::new(); + assert_eq!(super::sync_grok(&conn, &mut state, &root).unwrap(), 1); + let stamps = state[super::GROK_SYNC_STATE_KEY].clone(); + + // What the previous build left: proxy-only token facts, no span, and + // its stamps under the key it used. + conn.execute_batch( + "UPDATE session_events SET request_span = NULL, \ + token_json = json_remove(token_json, '$.usage') WHERE source = 'grok';", + ) + .unwrap(); let mut old = Map::new(); - old.insert("claude".into(), old_cursor.to_value()); - checkpoint_sync_state(&path, &old); + old.insert("grok_events_v1".into(), stamps.clone()); + let mut current = Map::new(); + current.insert(super::GROK_SYNC_STATE_KEY.into(), stamps); + // The positive control: under the current key an unchanged session is + // skipped, which is exactly why the key had to move. + assert_eq!(super::sync_grok(&conn, &mut current, &root).unwrap(), 0); - let mut replacement = decode_typed_cursor(test_file_cursor(0, 11, 200)); - replacement.generation.rewrite_epoch = 2; - let mut new = Map::new(); - new.insert("claude".into(), replacement.to_value()); - checkpoint_sync_state(&path, &new); + assert_eq!(super::sync_grok(&conn, &mut old, &root).unwrap(), 1); + let (usage, span): (Option, Option) = conn + .query_row( + "SELECT json_extract(token_json, '$.usage.inputTokens'), request_span \ + FROM session_events WHERE source = 'grok' AND token_json IS NOT NULL", + [], + |row| Ok((row.get(0)?, row.get(1)?)), + ) + .unwrap(); + assert_eq!((usage, span.as_deref()), (Some(1000), Some("0"))); + } - let saved = load_sync_state(&path).unwrap(); - assert_eq!(saved_cursor_offset(&saved["claude"]), 0); - let saved = decode_typed_cursor(saved["claude"].clone()); - assert_eq!(saved.generation.inode, Some(11)); + /// A turn whose only readable output is its thinking still stores its + /// `turn_completed.usage`, on that thinking row, and counts as measured. + #[test] + fn a_thinking_only_grok_turn_keeps_its_usage() { + let home = tempfile::tempdir().unwrap(); + let (chat, dir) = grok_stream_fixture(home.path(), "grok-think-0001"); + fs::write( + &chat, + concat!( + r#"{"type":"user","content":"hi"}"#, + "\n", + r#"{"type":"reasoning","summary":"Working through it"}"#, + "\n" + ), + ) + .unwrap(); + fs::write( + dir.join("updates.jsonl"), + concat!( + r#"{"method":"session/update","params":{"update":{"sessionUpdate":"user_message_chunk"},"_meta":{"eventId":"u1","agentTimestampMs":1789560000000,"turnStartMs":1789560000000}}}"#, + "\n", + r#"{"method":"session/update","params":{"update":{"sessionUpdate":"agent_thought_chunk"},"_meta":{"eventId":"t1","agentTimestampMs":1789560001000,"turnStartMs":1789560000000}}}"#, + "\n", + r#"{"method":"_x.ai/session/update","params":{"update":{"sessionUpdate":"turn_completed","usage":{"inputTokens":10,"outputTokens":3}},"_meta":{"agentTimestampMs":1789560002000,"turnStartMs":1789560000000}}}"#, + "\n" + ), + ) + .unwrap(); + let conn = open_db(&home.path().join("history.db")).unwrap(); + let session = super::scan_grok_session_file(&chat).unwrap().unwrap(); + let outcome = super::ingest_grok_session(&conn, &session, &chat.to_string_lossy()).unwrap(); + let (kind, input): (String, Option) = conn + .query_row( + "SELECT kind, json_extract(token_json, '$.usage.inputTokens') \ + FROM session_events WHERE source = 'grok' AND token_json IS NOT NULL", + [], + |row| Ok((row.get(0)?, row.get(1)?)), + ) + .unwrap(); + assert_eq!((kind.as_str(), input), ("thinking", Some(10))); + assert_eq!((outcome.usage_turns, outcome.turns), (1, 1)); } + /// An `updates.jsonl` that exists and cannot be read is a failure, not an + /// absent stream. + /// + /// Taking it as absent would replace every exact event time with a + /// fallback and drop the token snapshots — and the change stamp, computed + /// from metadata that is still perfectly readable, would then checkpoint + /// that loss as the session's settled state until some file changes again. #[test] - fn sync_state_merge_recurses_through_all_object_maps() { - let dir = tempfile::tempdir().unwrap(); - let path = dir.path().join(".sync-state.json"); - let mut on_disk = Map::new(); - on_disk.insert( - "claude_sessions_v3".into(), - json!({"first.jsonl": "old", "nested": {"left": 1}}), - ); - save_sync_state(&path, &on_disk).unwrap(); + fn an_unreadable_update_stream_fails_instead_of_dating_events_from_fallbacks() { + let home = tempfile::tempdir().unwrap(); + let (chat, dir) = grok_stream_fixture(home.path(), "grok-str-0001"); - let mut ours = Map::new(); - ours.insert( - "claude_sessions_v3".into(), - json!({"second.jsonl": "new", "nested": {"right": 2}}), - ); - checkpoint_sync_state(&path, &ours); + // The positive control first: with no stream at all the session reads + // clean and simply reports that there is none. + let absent = super::scan_grok_session_file(&chat).unwrap().unwrap(); + assert!(!absent.updates_present); - assert_eq!( - load_sync_state(&path).unwrap()["claude_sessions_v3"], - json!({ - "first.jsonl": "old", - "second.jsonl": "new", - "nested": {"left": 1, "right": 2} - }) + let stream = dir.join("updates.jsonl"); + #[cfg(unix)] + std::os::unix::fs::symlink(&stream, &stream).unwrap(); + #[cfg(not(unix))] + return; + + let scanned = super::scan_grok_session_file(&chat); + assert!( + scanned.is_err(), + "an unreadable update stream must not scan as an absent one" ); + + // And once it is readable, the same path succeeds and the stream is + // present — so the failure was the unreadable file, not its existence. + fs::remove_file(&stream).unwrap(); + fs::write( + &stream, + br#"{"method":"session/update","params":{"update":{"sessionUpdate":"user_message_chunk"},"_meta":{"agentTimestampMs":1000,"turnStartMs":1000}}} +"#, + ) + .unwrap(); + let read = super::scan_grok_session_file(&chat).unwrap().unwrap(); + assert!(read.updates_present); + assert_eq!(read.first_ts, 1000); } - /// A generation migration drops its predecessor from the in-memory map, but - /// the merge only folds in the keys a run *has*, so on its own that deletion - /// never reaches disk: the retired map is reloaded and rewritten forever. - /// A Grok session a previous release already consumed must be re-read - /// once, and must come back with evidence instead of prompts alone. - /// - /// The change stamp of that session is unchanged on disk, so nothing but - /// retiring the sync-state key can make `sync` look at it again — and the - /// prompts it wrote with synthesized timestamps must be replaced, not - /// joined by a second copy carrying the real ones. - /// Build a Grok session directory from raw lines and index it. - fn ingest_grok_lines(home: &Path, chat: &str, updates: &str) -> Connection { - let dir = home.join(".grok/sessions/%2Ftmp%2Ffallback/grok-fb-0001"); - fs::create_dir_all(&dir).unwrap(); - fs::write(dir.join("chat_history.jsonl"), chat).unwrap(); - fs::write(dir.join("updates.jsonl"), updates).unwrap(); + /// "Nothing in it parsed" is not "it is not there". A stream of rows this + /// parser does not interpret is a stream that exists, and a caller told it + /// is missing would look for a file that is sitting right there. + #[test] + fn a_stream_of_rows_the_parser_cannot_use_is_present_not_missing() { + let home = tempfile::tempdir().unwrap(); + let (chat, dir) = grok_stream_fixture(home.path(), "grok-str-0002"); fs::write( - dir.join("summary.json"), - br#"{"info":{"id":"grok-fb-0001","cwd":"/tmp/fallback"},"created_at":"2026-01-01T00:00:00.000Z"}"#, + dir.join("updates.jsonl"), + b"{\"hello\":\"world\"}\n{\"another\":\"row\"}\n", + ) + .unwrap(); + let conn = open_db(&home.path().join("history.db")).unwrap(); + let session = super::scan_grok_session_file(&chat).unwrap().unwrap(); + let outcome = super::ingest_grok_session(&conn, &session, &chat.to_string_lossy()).unwrap(); + assert!( + !outcome.missing_updates, + "the file exists; it just said nothing this parser understands" + ); + assert!(outcome.updates_yielded_no_timing); + assert_eq!(outcome.unread_update_rows, 2); + + // The positive control: with the file removed, it really is missing. + fs::remove_file(dir.join("updates.jsonl")).unwrap(); + let session = super::scan_grok_session_file(&chat).unwrap().unwrap(); + let outcome = super::ingest_grok_session(&conn, &session, &chat.to_string_lossy()).unwrap(); + assert!(outcome.missing_updates); + assert!(!outcome.updates_yielded_no_timing); + assert_eq!(outcome.unread_update_rows, 0); + } + + /// Copy the checked-in Muse fixture into `home` and answer with the + /// sessions root and the transcript. + fn muse_fixture(home: &Path) -> (PathBuf, PathBuf) { + let fixture = Path::new(env!("CARGO_MANIFEST_DIR")) + .join("tests/fixtures/muse/tools-session/.local/share/muse/sessions"); + let root = home.join(".local/share/muse/sessions"); + let day = "2026/09/20/muse-a001"; + for file in super::muse_session_files(&fixture.join(day).join("session.jsonl")).unwrap() { + let target = root.join(file.strip_prefix(&fixture).unwrap()); + fs::create_dir_all(target.parent().unwrap()).unwrap(); + fs::copy(&file, &target).unwrap(); + } + let transcript = root.join(day).join("session.jsonl"); + (root, transcript) + } + + /// Rows one session owns in `table`. + fn muse_session_count(conn: &Connection, table: &str, session_id: &str) -> i64 { + conn.query_row( + &format!("SELECT COUNT(*) FROM {table} WHERE source = 'muse' AND session_id = ?"), + [session_id], + |row| row.get(0), ) - .unwrap(); - let conn = open_db(&home.join("history.db")).unwrap(); - let chat_path = dir.join("chat_history.jsonl"); - let session = super::scan_grok_session_file(&chat_path).unwrap().unwrap(); - super::ingest_grok_session(&conn, &session, &chat_path.to_string_lossy()).unwrap(); - conn + .unwrap() } - fn grok_event_time(conn: &Connection, text: &str) -> i64 { + const MUSE_PARENT: &str = "muse-a001"; + const MUSE_WORKER: &str = "muse-c001"; + + fn muse_count(conn: &Connection, table: &str) -> i64 { conn.query_row( - "SELECT ts_ms FROM session_events WHERE source = 'grok' AND text = ?", - params![text], + &format!("SELECT COUNT(*) FROM {table} WHERE source = 'muse'"), + [], |row| row.get(0), ) .unwrap() } - /// An assistant message with no recorded time of its own falls back to - /// **its own** turn's start. Resolving against a turn index captured - /// before the message told us which turn it was in hands it the previous - /// turn's start — a time from before the person had even asked. + /// Plain `sync` indexes the transcript once, re-reads it only when it + /// changes, replaces rather than duplicates on that re-read, and never + /// turns the `subagent/` child into a session. #[test] - fn an_untimed_assistant_message_falls_back_to_its_own_turns_start() { + fn muse_sync_is_idempotent_and_rereads_a_grown_transcript() { let home = tempfile::tempdir().unwrap(); - let conn = ingest_grok_lines( - home.path(), - &[ - r#"{"type":"user","content":"first"}"#, - r#"{"type":"assistant","content":"answer one"}"#, - // The second turn is the model continuing on its own: Grok - // wrote no user record for it, so the chat side never learns - // the turn advanced and only the matched group knows. - r#"{"type":"assistant","content":"answer two"}"#, - "", - ] - .join("\n"), - &[ - r#"{"method":"session/update","params":{"update":{"sessionUpdate":"user_message_chunk"},"_meta":{"agentTimestampMs":1000,"turnStartMs":1000}}}"#, - r#"{"method":"session/update","params":{"update":{"sessionUpdate":"agent_message_chunk"},"_meta":{"agentTimestampMs":2000,"turnStartMs":1000}}}"#, - r#"{"method":"_x.ai/session/update","params":{"update":{"sessionUpdate":"turn_completed"},"_meta":{"agentTimestampMs":2500,"turnStartMs":1000}}}"#, - // The second turn opens with the model continuing: the chunk - // says which turn it belongs to, and records no time of its - // own — which is the only reason the fallback is reached. - r#"{"method":"session/update","params":{"update":{"sessionUpdate":"agent_message_chunk"},"_meta":{"eventId":"ev_late","turnStartMs":5000}}}"#, - "", - ] - .join("\n"), - ); - // The positive control: the first turn's events are exact, so the - // second one's fallback is the only thing under test. - assert_eq!(grok_event_time(&conn, "first"), 1000); - assert_eq!(grok_event_time(&conn, "answer one"), 2000); + let (root, transcript) = muse_fixture(home.path()); + let conn = open_db(&home.path().join("history.db")).unwrap(); + let mut state = Map::new(); + + assert_eq!(super::sync_muse(&conn, &mut state, &root).unwrap(), 2); + assert_eq!(muse_count(&conn, "sessions"), 1, "a child is not a session"); + assert_eq!(muse_count(&conn, "session_relationships"), 3); + let events = muse_count(&conn, "session_events"); + assert_eq!(muse_session_count(&conn, "session_events", MUSE_PARENT), 15); + + // Unchanged: nothing is read again and nothing is duplicated. + assert_eq!(super::sync_muse(&conn, &mut state, &root).unwrap(), 0); + assert_eq!(muse_count(&conn, "session_events"), events); + // The session grows by one turn: the re-read adds exactly that turn. + let mut file = fs::OpenOptions::new() + .append(true) + .open(&transcript) + .unwrap(); + for (sequence, event) in [ + (40, json!({"kind": "started", "prompt": "Fix the test."})), + ( + 41, + json!({"kind": "assistant_message_committed", "message_id": "msg-5", "text": "Done."}), + ), + ] { + let record = json!({ + "id": format!("grown-{sequence}"), + "stream": {"kind": "session", "id": "muse-a001"}, + "sequence": sequence, + "recorded_at": 1_790_337_700_000_000i64 + sequence, + "payload_type": "runtime.session", + "payload": {"kind": "run", "run_id": "run-c", "event": event}, + }); + writeln!(file, "{record}").unwrap(); + } + drop(file); + assert_eq!(super::sync_muse(&conn, &mut state, &root).unwrap(), 1); + assert_eq!(muse_count(&conn, "session_events"), events + 2); + assert_eq!(muse_count(&conn, "history"), 3); + } + + /// `.sync-state.json` outlives a rebuilt database. An unchanged stamp is + /// trusted only while the evidence it recorded is still there. + #[test] + fn muse_sync_reindexes_an_unchanged_transcript_whose_rows_are_gone() { + let home = tempfile::tempdir().unwrap(); + let (root, _) = muse_fixture(home.path()); + let conn = open_db(&home.path().join("history.db")).unwrap(); + let mut state = Map::new(); + super::sync_muse(&conn, &mut state, &root).unwrap(); + + let rebuilt = open_db(&home.path().join("rebuilt.db")).unwrap(); + assert_eq!(super::sync_muse(&rebuilt, &mut state, &root).unwrap(), 2); assert_eq!( - grok_event_time(&conn, "answer two"), - 5000, - "an untimed reply belongs to the turn it answered, not the one before it" + muse_session_count(&rebuilt, "session_events", MUSE_PARENT), + 15 ); + assert_eq!(muse_count(&rebuilt, "session_relationships"), 3); } - /// A marker is a record with a place in the file: the next record with no - /// time of its own inherits the marker's, not that of whatever preceded - /// it. + /// A background subagent keeps writing its own log after the parent's + /// last record. The parent's stamp covers the child logs, so that growth + /// is read even though the parent file never changes again. #[test] - fn a_marker_carries_its_time_forward_to_the_next_untimed_record() { + fn muse_sync_rereads_a_subagent_log_that_grew_on_its_own() { let home = tempfile::tempdir().unwrap(); - let conn = ingest_grok_lines( - home.path(), - &[ - r#"{"type":"user","content":"first","timestamp_ms":1000}"#, - r#"{"type":"user","content":"injected","synthetic_reason":"compaction","timestamp_ms":2000}"#, - r#"{"type":"assistant","content":"after the marker"}"#, - "", - ] - .join("\n"), - "", - ); - // The positive control: the marker really was stored at 2000, so the - // event below inherits a time that exists rather than a default. - let marker: i64 = conn - .query_row( - "SELECT ts_ms FROM session_markers WHERE source = 'grok' AND kind = 'synthetic_turn'", - [], - |row| row.get(0), - ) + let (root, transcript) = muse_fixture(home.path()); + let conn = open_db(&home.path().join("history.db")).unwrap(); + let mut state = Map::new(); + super::sync_muse(&conn, &mut state, &root).unwrap(); + let before = muse_session_count(&conn, "session_events", MUSE_WORKER); + + let child_log = transcript + .parent() + .unwrap() + .join(format!("subagent/{MUSE_WORKER}/session.jsonl")); + let record = json!({ + "id": "worker-late", + "stream": {"kind": "session", "id": MUSE_WORKER}, + "sequence": 99, + "recorded_at": 1_790_337_700_000_000i64, + "payload_type": "runtime.session", + "payload": {"kind": "run", "run_id": "child-run", "event": { + "kind": "assistant_message_committed", "message_id": "late", "text": "Also: no docs." + }}, + }); + let mut file = fs::OpenOptions::new() + .append(true) + .open(&child_log) .unwrap(); - assert_eq!(marker, 2000); + writeln!(file, "{record}").unwrap(); + drop(file); + + super::sync_muse(&conn, &mut state, &root).unwrap(); assert_eq!( - grok_event_time(&conn, "after the marker"), - 2000, - "the nearest preceding record is the marker, not the prompt before it" + muse_session_count(&conn, "session_events", MUSE_WORKER), + before + 1 + ); + assert_eq!(muse_count(&conn, "session_relationships"), 3); + assert_eq!( + muse_session_count(&conn, "history", MUSE_WORKER), + 0, + "a child's objective never becomes a typed prompt" ); } - /// Reasoning and tools join their update the same way prose does, so they - /// have to adopt the turn that match establishes. A model-initiated turn - /// has no user record to advance the transcript-side cursor, so without - /// that the fallback dates them from before the turn began. + /// Roots under `home` with nothing read from the environment, for the + /// fingerprint-gated sync path. + fn muse_roots(home: &Path) -> crate::ProviderRoots { + crate::ProviderRoots::from_home(home.to_path_buf(), home.join("opencode.db")) + } + + fn append_muse_record(log: &Path, session_id: &str, uid: &str, text: &str) { + let record = json!({ + "id": uid, + "stream": {"kind": "session", "id": session_id}, + "sequence": 99, + "recorded_at": 1_790_337_700_000_000i64, + "payload_type": "runtime.session", + "payload": {"kind": "run", "run_id": "late", "event": { + "kind": "assistant_message_committed", "message_id": uid, "text": text + }}, + }); + let mut file = fs::OpenOptions::new().append(true).open(log).unwrap(); + writeln!(file, "{record}").unwrap(); + } + + /// A subagent that keeps writing after its parent has stopped changes + /// only its own log. That has to move the source fingerprint, or a + /// plain `sync` takes the fast path and never reads it. #[test] - fn untimed_reasoning_and_tools_fall_back_to_their_own_turns_start() { + fn a_growing_muse_subagent_log_moves_the_sync_fast_path() { let home = tempfile::tempdir().unwrap(); - let conn = ingest_grok_lines( - home.path(), - &[ - r#"{"type":"user","content":"first"}"#, - r#"{"type":"reasoning","summary":"thinking in turn zero"}"#, - r#"{"type":"assistant","content":"answer one"}"#, - // The model continues on its own: no user record opens turn 1. - r#"{"type":"reasoning","summary":"thinking in turn one"}"#, - r#"{"type":"assistant","content":"","tool_calls":[{"id":"call_late","name":"Shell","arguments":{"command":"ls"}}]}"#, - r#"{"type":"tool_result","tool_call_id":"call_late","content":"done"}"#, - "", - ] - .join("\n"), - &[ - r#"{"method":"session/update","params":{"update":{"sessionUpdate":"user_message_chunk"},"_meta":{"agentTimestampMs":1000,"turnStartMs":1000}}}"#, - r#"{"method":"session/update","params":{"update":{"sessionUpdate":"agent_thought_chunk"},"_meta":{"agentTimestampMs":1100,"turnStartMs":1000}}}"#, - r#"{"method":"session/update","params":{"update":{"sessionUpdate":"agent_message_chunk"},"_meta":{"agentTimestampMs":2000,"turnStartMs":1000}}}"#, - r#"{"method":"_x.ai/session/update","params":{"update":{"sessionUpdate":"turn_completed"},"_meta":{"agentTimestampMs":2500,"turnStartMs":1000}}}"#, - // Turn 1 opens on the model's own thinking, and none of its - // rows records a time of its own. - r#"{"method":"session/update","params":{"update":{"sessionUpdate":"agent_thought_chunk"},"_meta":{"eventId":"ev_think","turnStartMs":5000}}}"#, - r#"{"method":"session/update","params":{"update":{"sessionUpdate":"tool_call","toolCallId":"call_late"},"_meta":{"turnStartMs":5000}}}"#, - r#"{"method":"session/update","params":{"update":{"sessionUpdate":"tool_call_update","toolCallId":"call_late","status":"completed"},"_meta":{"turnStartMs":5000}}}"#, - "", - ] - .join("\n"), + let (_, transcript) = muse_fixture(home.path()); + let db = home.path().join("history.db"); + let roots = muse_roots(home.path()); + assert!( + super::sync_exclusive_with_roots(&db, &roots, false) + .unwrap() + .swept ); - // The positive control: turn 0 is exact, so only the fallbacks below - // are under test. - assert_eq!(grok_event_time(&conn, "first"), 1000); - assert_eq!(grok_event_time(&conn, "thinking in turn zero"), 1100); - assert_eq!(grok_event_time(&conn, "answer one"), 2000); - - assert_eq!( - grok_event_time(&conn, "thinking in turn one"), - 5000, - "a thought belongs to the turn its group names, not the previous one" + assert!( + !super::sync_exclusive_with_roots(&db, &roots, false) + .unwrap() + .swept, + "an unchanged store takes the fast path" ); - let tool_times: Vec = conn - .prepare( - "SELECT ts_ms FROM session_events WHERE source = 'grok' \ - AND event_uid IN ('tool:call_late', 'result:call_late') ORDER BY event_uid", - ) - .unwrap() - .query_map([], |row| row.get(0)) + let conn = open_db(&db).unwrap(); + let before = muse_session_count(&conn, "session_events", MUSE_WORKER); + + let child_log = transcript + .parent() .unwrap() - .collect::>>() - .unwrap(); + .join(format!("subagent/{MUSE_WORKER}/session.jsonl")); + append_muse_record(&child_log, MUSE_WORKER, "worker-late", "Also: no docs."); + assert!( + super::sync_exclusive_with_roots(&db, &roots, false) + .unwrap() + .swept + ); assert_eq!( - tool_times, - vec![5000, 5000], - "a call and its result join by id, so their turn is known exactly" + muse_session_count(&conn, "session_events", MUSE_WORKER), + before + 1 ); } - /// ACP's terminal status is read case-blind for the call as for its - /// result, so a `Cancelled` call is not stored as a success beside a - /// cancelled result. + /// Rows lost under an unchanged stamp — in a session, or in a subagent + /// it links — are put back by the next plain `sync`, not only by a forced + /// one: Muse is a source the destination marker guards. #[test] - fn a_mixed_case_acp_failure_marks_the_grok_call_failed() { + fn lost_muse_evidence_is_repaired_by_the_next_sync() { let home = tempfile::tempdir().unwrap(); - let conn = ingest_grok_lines( - home.path(), - &[ - r#"{"type":"user","content":"first"}"#, - r#"{"type":"assistant","content":"","tool_calls":[{"id":"call_x","name":"Shell","arguments":{"command":"ls"}}]}"#, - r#"{"type":"tool_result","tool_call_id":"call_x","content":"stopped"}"#, - "", - ] - .join("\n"), - &[ - r#"{"method":"session/update","params":{"update":{"sessionUpdate":"user_message_chunk"},"_meta":{"agentTimestampMs":1000,"turnStartMs":1000}}}"#, - r#"{"method":"session/update","params":{"update":{"sessionUpdate":"tool_call","toolCallId":"call_x"},"_meta":{"agentTimestampMs":1100,"turnStartMs":1000}}}"#, - r#"{"method":"session/update","params":{"update":{"sessionUpdate":"tool_call_update","toolCallId":"call_x","status":"Cancelled"},"_meta":{"agentTimestampMs":1200,"turnStartMs":1000}}}"#, - "", - ] - .join("\n"), + let _ = muse_fixture(home.path()); + let db = home.path().join("history.db"); + let roots = muse_roots(home.path()); + super::sync_exclusive_with_roots(&db, &roots, false).unwrap(); + let conn = open_db(&db).unwrap(); + let parent_events = muse_session_count(&conn, "session_events", MUSE_PARENT); + let worker_events = muse_session_count(&conn, "session_events", MUSE_WORKER); + + conn.execute( + "DELETE FROM session_events WHERE source = 'muse' AND session_id = ? \ + AND event_uid = 'tool:call_read'", + [MUSE_PARENT], + ) + .unwrap(); + conn.execute( + "DELETE FROM session_events WHERE source = 'muse' AND session_id = ? \ + AND role = 'assistant'", + [MUSE_WORKER], + ) + .unwrap(); + super::sync_exclusive_with_roots(&db, &roots, false).unwrap(); + assert_eq!( + muse_session_count(&conn, "session_events", MUSE_PARENT), + parent_events ); - let (is_error, result_status): (Option, Option) = conn + assert_eq!( + muse_session_count(&conn, "session_events", MUSE_WORKER), + worker_events + ); + } + + /// Only the parent's own direct children that it typed as workers answer + /// its `subagent_spawn` calls; the worker's nested child does not. + #[test] + fn muse_worker_children_count_only_direct_typed_workers() { + let home = tempfile::tempdir().unwrap(); + let (_, transcript) = muse_fixture(home.path()); + let conn = open_db(&home.path().join("history.db")).unwrap(); + let (parsed, children) = super::read_muse_tree(&transcript).unwrap().unwrap(); + let outcome = + super::ingest_muse_session_tree(&conn, &parsed, &transcript, Some(&children)).unwrap(); + assert_eq!(outcome.subagent_calls, 1); + assert_eq!(outcome.worker_children, 1, "{outcome:?}"); + assert_eq!(outcome.relationships, 3); + } + + /// A subagent also known remotely shares its `history` identity with that + /// evidence. A local re-read removes the child's local objective only + /// when nothing remote stands behind the id. + #[test] + fn a_remote_muse_childs_history_survives_a_local_reread() { + let home = tempfile::tempdir().unwrap(); + let (root, transcript) = muse_fixture(home.path()); + let conn = open_db(&home.path().join("history.db")).unwrap(); + let mut state = Map::new(); + super::sync_muse(&conn, &mut state, &root).unwrap(); + crate::mark_session_presence(&conn, "muse", MUSE_WORKER, super::SessionLocation::Remote) + .unwrap(); + insert_history( + &conn, + &HistoryEntry { + id: 0, + source: "muse".into(), + session_id: Some(MUSE_WORKER.to_string()), + project: None, + prompt: "remote prompt".into(), + prompt_hash: Some(prompt_hash("remote prompt")), + timestamp_ms: 42, + }, + ) + .unwrap(); + + let child_log = transcript + .parent() + .unwrap() + .join(format!("subagent/{MUSE_WORKER}/session.jsonl")); + append_muse_record(&child_log, MUSE_WORKER, "worker-late", "Also: no docs."); + super::sync_muse(&conn, &mut state, &root).unwrap(); + let remote: i64 = conn .query_row( - "SELECT (SELECT is_error FROM tool_calls WHERE source = 'grok' AND tool_use_id = 'call_x'), \ - (SELECT result_status FROM session_events \ - WHERE source = 'grok' AND event_uid = 'result:call_x')", - [], - |row| Ok((row.get(0)?, row.get(1)?)), + "SELECT COUNT(*) FROM history WHERE source = 'muse' AND session_id = ? \ + AND prompt = 'remote prompt'", + [MUSE_WORKER], + |row| row.get(0), + ) + .unwrap(); + assert_eq!(remote, 1, "a local re-read does not erase remote history"); + let objectives: i64 = conn + .query_row( + "SELECT COUNT(*) FROM history WHERE source = 'muse' AND session_id = ? \ + AND prompt LIKE 'Role:%'", + [MUSE_WORKER], + |row| row.get(0), ) .unwrap(); assert_eq!( - (is_error, result_status.as_deref()), - (Some(1), Some("cancelled")) + objectives, 0, + "a subagent's objective is never a typed prompt" ); } - /// Encrypted-only reasoning still occupies an `agent_thoughts` ordinal. - /// Leaving that cursor unmoved handed the next readable thought the - /// encrypted record's timestamp and event id. + /// A remotely known session whose transcript drops a prompt stops + /// offering it, while the prompts only the remote side holds stay. #[test] - fn encrypted_reasoning_consumes_its_thought_chunk_ordinal() { + fn a_remote_muse_session_drops_prompts_its_transcript_no_longer_has() { let home = tempfile::tempdir().unwrap(); - let conn = ingest_grok_lines( - home.path(), - &[ - r#"{"type":"user","content":"first"}"#, - r#"{"type":"reasoning","encrypted_content":"opaque"}"#, - r#"{"type":"reasoning","summary":"visible thought"}"#, - r#"{"type":"assistant","content":"done"}"#, - "", - ] - .join("\n"), - &[ - r#"{"method":"session/update","params":{"update":{"sessionUpdate":"user_message_chunk"},"_meta":{"agentTimestampMs":1000,"turnStartMs":1000}}}"#, - r#"{"method":"session/update","params":{"update":{"sessionUpdate":"agent_thought_chunk"},"_meta":{"agentTimestampMs":1100,"turnStartMs":1000}}}"#, - // A new turnStartMs starts a second thought group. Consecutive - // chunks in the same turn would merge into one group, which - // would not exercise the ordinal join this test is about. - r#"{"method":"session/update","params":{"update":{"sessionUpdate":"agent_thought_chunk"},"_meta":{"eventId":"ev_visible","agentTimestampMs":2000,"turnStartMs":2000}}}"#, - r#"{"method":"session/update","params":{"update":{"sessionUpdate":"agent_message_chunk"},"_meta":{"agentTimestampMs":3000,"turnStartMs":2000}}}"#, - "", - ] - .join("\n"), - ); - let encrypted: i64 = conn - .query_row( - "SELECT ts_ms FROM session_markers WHERE source = 'grok' AND kind = 'encrypted_reasoning'", - [], - |row| row.get(0), + let (root, transcript) = muse_fixture(home.path()); + let conn = open_db(&home.path().join("history.db")).unwrap(); + let mut state = Map::new(); + super::sync_muse(&conn, &mut state, &root).unwrap(); + crate::mark_session_presence(&conn, "muse", MUSE_PARENT, super::SessionLocation::Remote) + .unwrap(); + insert_history( + &conn, + &HistoryEntry { + id: 0, + source: "muse".into(), + session_id: Some(MUSE_PARENT.to_string()), + project: None, + prompt: "remote prompt".into(), + prompt_hash: Some(prompt_hash("remote prompt")), + timestamp_ms: 42, + }, + ) + .unwrap(); + + // Drop the second typed prompt from the transcript. + let kept: String = fs::read_to_string(&transcript) + .unwrap() + .lines() + .filter(|line| !line.contains("Why did it fail?")) + .map(|line| format!("{line}\n")) + .collect(); + fs::write(&transcript, kept).unwrap(); + super::sync_muse(&conn, &mut state, &root).unwrap(); + let prompts: Vec = conn + .prepare( + "SELECT prompt FROM history WHERE source = 'muse' AND session_id = ? \ + ORDER BY timestamp_ms", ) + .unwrap() + .query_map([MUSE_PARENT], |row| row.get(0)) + .unwrap() + .collect::>() .unwrap(); assert_eq!( - encrypted, 1100, - "the opaque trace uses its own chunk's time" - ); - assert_eq!( - grok_event_time(&conn, "visible thought"), - 2000, - "the next thought must not inherit the encrypted chunk" + prompts, + vec![ + "remote prompt".to_string(), + "Add a retry helper to src/http.rs and run the tests.".to_string() + ] ); - let uid: String = conn - .query_row( - "SELECT event_uid FROM session_events WHERE source = 'grok' AND text = 'visible thought'", - [], - |row| row.get(0), - ) - .unwrap(); - assert_eq!(uid, "ev:ev_visible"); } - /// `sessions.first_prompt` is a bounded catalog excerpt. Hydration must - /// not overwrite discovery's 4,096-character field with the full prompt. + /// A removed child that is also known remotely keeps its own evidence, + /// but the local-only children it linked from the missing directory do + /// not linger, and neither do its edges to them. #[test] - fn grok_first_prompt_is_the_bounded_catalog_excerpt() { + fn a_removed_remote_muse_child_still_forgets_its_local_children() { let home = tempfile::tempdir().unwrap(); - let long = "x".repeat(crate::discover::EXCERPT_MAX_CHARS + 80); - let conn = ingest_grok_lines( - home.path(), - &format!("{{\"type\":\"user\",\"content\":\"{long}\"}}\n"), - concat!( - r#"{"method":"session/update","params":{"update":{"sessionUpdate":"user_message_chunk"},"_meta":{"agentTimestampMs":1000,"turnStartMs":1000}}}"#, - "\n", - ), + let (root, transcript) = muse_fixture(home.path()); + let conn = open_db(&home.path().join("history.db")).unwrap(); + let mut state = Map::new(); + super::sync_muse(&conn, &mut state, &root).unwrap(); + crate::mark_session_presence(&conn, "muse", MUSE_WORKER, super::SessionLocation::Remote) + .unwrap(); + let worker_events = muse_session_count(&conn, "session_events", MUSE_WORKER); + fs::remove_dir_all( + transcript + .parent() + .unwrap() + .join(format!("subagent/{MUSE_WORKER}")), + ) + .unwrap(); + super::sync_muse(&conn, &mut state, &root).unwrap(); + assert_eq!( + muse_session_count(&conn, "session_events", MUSE_WORKER), + worker_events ); - let stored: String = conn + assert_eq!(muse_session_count(&conn, "session_events", "muse-c002"), 0); + let edges_from_worker: i64 = conn .query_row( - "SELECT first_prompt FROM sessions WHERE source = 'grok'", - [], + "SELECT COUNT(*) FROM session_relationships \ + WHERE source = 'muse' AND parent_session_id = ?", + [MUSE_WORKER], |row| row.get(0), ) .unwrap(); - assert_eq!(stored.chars().count(), crate::discover::EXCERPT_MAX_CHARS); - let history: String = conn + assert_eq!(edges_from_worker, 0); + let local: i64 = conn .query_row( - "SELECT prompt FROM history WHERE source = 'grok'", - [], + "SELECT COUNT(*) FROM session_presences \ + WHERE source = 'muse' AND session_id = ? AND location = 'local'", + [MUSE_WORKER], |row| row.get(0), ) .unwrap(); - assert_eq!( - history.chars().count(), - crate::discover::EXCERPT_MAX_CHARS + 80, - "the transcript keeps the full prompt; only the catalog excerpt is bounded" - ); + assert_eq!(local, 0, "a child whose log is gone is not present locally"); } - /// A directory that cannot be read is not an empty directory. - /// - /// The read is a replacement: an unreadable `compaction_checkpoints/` - /// taken as empty would stamp the session as having no checkpoints and - /// then delete the ones already stored, reporting success. It has to fail - /// so the transaction rolls back. + /// A subagent with no catalog row and no events left can still be named + /// for repair — by its parent's edge — so the next plain `sync` re-reads + /// the parent's tree and restores it. #[test] - fn an_unreadable_sidecar_directory_fails_instead_of_erasing_the_evidence() { + fn a_muse_subagent_that_lost_every_event_is_repaired() { let home = tempfile::tempdir().unwrap(); - let dir = home - .path() - .join(".grok/sessions/%2Ftmp%2Fbroken/grok-brk-0001"); - fs::create_dir_all(dir.join("compaction_checkpoints")).unwrap(); - fs::write( - dir.join("chat_history.jsonl"), - b"{\"type\":\"user\",\"content\":\"hello\"}\n", - ) - .unwrap(); - fs::write( - dir.join("summary.json"), - br#"{"info":{"id":"grok-brk-0001","cwd":"/tmp/broken"},"created_at":"2026-01-01T00:00:00.000Z"}"#, + let _ = muse_fixture(home.path()); + let db = home.path().join("history.db"); + let roots = muse_roots(home.path()); + super::sync_exclusive_with_roots(&db, &roots, false).unwrap(); + let conn = open_db(&db).unwrap(); + let worker_events = muse_session_count(&conn, "session_events", MUSE_WORKER); + conn.execute( + "DELETE FROM session_events WHERE source = 'muse' AND session_id = ?", + [MUSE_WORKER], ) .unwrap(); + super::sync_exclusive_with_roots(&db, &roots, false).unwrap(); + assert_eq!( + muse_session_count(&conn, "session_events", MUSE_WORKER), + worker_events + ); + } + + /// A `subagent/` entry that is a symlink out of the session directory is + /// not the session's evidence: it is never read, stamped or linked. + #[cfg(unix)] + #[test] + fn a_muse_child_log_symlinked_outside_its_session_is_not_read() { + let home = tempfile::tempdir().unwrap(); + let (root, transcript) = muse_fixture(home.path()); + let outside = home.path().join("elsewhere/secret"); + fs::create_dir_all(&outside).unwrap(); fs::write( - dir.join("compaction_checkpoints/1700000000000.json"), - br#"{"created_at":"2026-01-01T00:10:00.000Z"}"#, + outside.join("session.jsonl"), + "{\"id\":\"x\",\"stream\":{\"kind\":\"session\",\"id\":\"escaped\"},\ + \"recorded_at\":1790337600000000,\"payload_type\":\"runtime.session.metadata\",\ + \"payload\":{\"kind\":\"metadata\",\"record\":{\"workspace_root\":\"/w\"}}}\n", ) .unwrap(); - let chat = dir.join("chat_history.jsonl"); + let session_dir = transcript.parent().unwrap(); + std::os::unix::fs::symlink(&outside, session_dir.join("subagent/escaped")).unwrap(); + let files = super::muse_session_files(&transcript).unwrap(); + assert!( + files + .iter() + .all(|file| !file.to_string_lossy().contains("escaped")), + "{files:?}" + ); let conn = open_db(&home.path().join("history.db")).unwrap(); - let session = super::scan_grok_session_file(&chat).unwrap().unwrap(); - super::ingest_grok_session(&conn, &session, &chat.to_string_lossy()).unwrap(); - let markers = || -> i64 { + let mut state = Map::new(); + super::sync_muse(&conn, &mut state, &root).unwrap(); + assert_eq!(muse_session_count(&conn, "session_events", "escaped"), 0); + assert_eq!(muse_count(&conn, "session_relationships"), 3); + } + + /// Muse commits tool results in their own records, not inside a user + /// message, so a user turn is the typed prompt alone: no tool output is + /// counted among its blocks. + #[test] + fn muse_user_turns_are_the_prompts_alone() { + let home = tempfile::tempdir().unwrap(); + let (root, _) = muse_fixture(home.path()); + let conn = open_db(&home.path().join("history.db")).unwrap(); + let mut state = Map::new(); + super::sync_muse(&conn, &mut state, &root).unwrap(); + let page = crate::session_user_turns_page(&conn, "muse", MUSE_PARENT, 100, None).unwrap(); + assert_eq!(page.user_turns.len(), 2, "{:?}", page.user_turns); + for turn in &page.user_turns { + assert!( + turn.blocks.iter().all(|block| block.kind != "tool_result"), + "{turn:?}" + ); + } + let sources: Vec = conn + .prepare( + "SELECT DISTINCT event_source FROM session_events \ + WHERE source = 'muse' AND role = 'tool_result'", + ) + .unwrap() + .query_map([], |row| row.get(0)) + .unwrap() + .collect::>() + .unwrap(); + assert_eq!(sources, vec!["function_call_output".to_string()]); + } + + /// A call that failed or was cancelled before any result was recorded + /// still takes its status from Muse's recorded outcome. + #[test] + fn a_muse_call_that_ended_without_a_result_keeps_its_outcome() { + let home = tempfile::tempdir().unwrap(); + let (root, transcript) = muse_fixture(home.path()); + let conn = open_db(&home.path().join("history.db")).unwrap(); + let mut state = Map::new(); + // Drop the result record that answered the four calls of step 2, + // keeping their `tool_batch.effect.terminal` outcomes; make the + // write call's outcome a cancellation. + let rewritten: String = fs::read_to_string(&transcript) + .unwrap() + .lines() + .filter(|line| !line.contains("\"batch_id\": \"msg-2\"")) + .map(|line| { + if line.contains("\"call_id\": \"call_write\"") && line.contains("tool_batch") { + line.replace("\"kind\": \"completed\"", "\"kind\": \"cancelled\"") + } else { + line.to_string() + } + }) + .map(|line| format!("{line}\n")) + .collect(); + fs::write(&transcript, rewritten).unwrap(); + super::sync_muse(&conn, &mut state, &root).unwrap(); + let status = |call: &str| -> Option { conn.query_row( - "SELECT COUNT(*) FROM session_markers WHERE source = 'grok' AND kind = 'compaction_boundary'", - [], + "SELECT is_error FROM tool_calls WHERE source = 'muse' AND tool_use_id = ?", + [call], |row| row.get(0), ) .unwrap() }; - let prompts = || -> i64 { + assert_eq!(status("call_write"), Some(1), "cancelled without a result"); + assert_eq!(status("call_edit"), Some(0), "completed without a result"); + } + + /// An edge reflects what the parent records now — a label the parent no + /// longer gives its child does not linger — while keeping the time it + /// was first observed. + #[test] + fn a_muse_delegation_drops_details_the_parent_no_longer_records() { + let home = tempfile::tempdir().unwrap(); + let (root, transcript) = muse_fixture(home.path()); + let conn = open_db(&home.path().join("history.db")).unwrap(); + let mut state = Map::new(); + super::sync_muse(&conn, &mut state, &root).unwrap(); + let label = |conn: &Connection| -> Option { conn.query_row( - "SELECT COUNT(*) FROM history WHERE source = 'grok'", - [], + "SELECT child_agent_name FROM session_relationships \ + WHERE source = 'muse' AND child_session_id = ?", + [MUSE_WORKER], |row| row.get(0), ) .unwrap() }; - assert_eq!(markers(), 1); - assert_eq!(prompts(), 1); - - // An entry that `read_dir` lists and `stat` cannot resolve: a symlink - // to itself is ELOOP on every platform that has symlinks. - let loop_entry = dir.join("compaction_checkpoints/loop.json"); - #[cfg(unix)] - std::os::unix::fs::symlink(&loop_entry, &loop_entry).unwrap(); - #[cfg(not(unix))] - return; - - // The stamp fails rather than describing a directory it could not read. - assert!(super::grok_session_stamp(&chat).is_err()); - - // And so does the read, so nothing is replaced. - let scanned = super::scan_grok_session_file(&chat); - assert!(scanned.is_err(), "an unreadable entry must not scan clean"); - assert_eq!(markers(), 1, "the stored checkpoint marker must survive"); - assert_eq!(prompts(), 1); + assert_eq!(label(&conn).as_deref(), Some("reviewer")); - // Positive control: with the entry readable the same paths succeed and - // the second checkpoint is indexed, so the failure above is the - // unreadable entry and not the extra file. - fs::remove_file(&loop_entry).unwrap(); - fs::write(&loop_entry, br#"{"created_at":"2026-01-01T00:20:00.000Z"}"#).unwrap(); - assert!(super::grok_session_stamp(&chat).is_ok()); - let session = super::scan_grok_session_file(&chat).unwrap().unwrap(); - super::ingest_grok_session(&conn, &session, &chat.to_string_lossy()).unwrap(); - assert_eq!(markers(), 2); + let rewritten = fs::read_to_string(&transcript) + .unwrap() + .replace("\"label\": \"reviewer\", ", ""); + let created = |conn: &Connection| -> i64 { + conn.query_row( + "SELECT created_ms FROM session_relationships \ + WHERE source = 'muse' AND child_session_id = ?", + [MUSE_WORKER], + |row| row.get(0), + ) + .unwrap() + }; + let first_seen = created(&conn); + std::thread::sleep(std::time::Duration::from_millis(5)); + fs::write(&transcript, rewritten).unwrap(); + super::sync_muse(&conn, &mut state, &root).unwrap(); + assert_eq!(label(&conn), None); + assert_eq!( + created(&conn), + first_seen, + "the edge keeps the time it was first observed" + ); } - /// The stamp sync saved for one Grok session file. - fn grok_saved_stamp<'a>(state: &'a Map, key: &str) -> Option<&'a str> { - state - .get(super::GROK_SYNC_STATE_KEY)? - .get(key) - .and_then(super::grok_state_stamp) + /// A `subagent/` entry that points back up into its own session is not a + /// child: it is never read as one, and the walk ends rather than + /// recursing through it forever. + #[cfg(unix)] + #[test] + fn a_muse_subagent_symlink_back_into_its_session_is_not_followed() { + let home = tempfile::tempdir().unwrap(); + let (root, transcript) = muse_fixture(home.path()); + let session_dir = transcript.parent().unwrap(); + std::os::unix::fs::symlink("..", session_dir.join("subagent/loop")).unwrap(); + + let files = super::muse_session_files(&transcript).unwrap(); + assert_eq!( + files.len(), + 4, + "the parent and its three real child logs: {files:?}" + ); + let conn = open_db(&home.path().join("history.db")).unwrap(); + let mut state = Map::new(); + super::sync_muse(&conn, &mut state, &root).unwrap(); + assert_eq!(muse_count(&conn, "session_relationships"), 3); + let self_edges: i64 = conn + .query_row( + "SELECT COUNT(*) FROM session_relationships \ + WHERE source = 'muse' AND child_session_id = ?", + [MUSE_PARENT], + |row| row.get(0), + ) + .unwrap(); + assert_eq!(self_edges, 0, "the parent is never its own child"); } - /// Write a Grok session directory with no `updates.jsonl`, and answer with - /// the transcript and the directory. - fn grok_stream_fixture(home: &Path, id: &str) -> (PathBuf, PathBuf) { - let dir = home.join(".grok/sessions/%2Ftmp%2Fstream").join(id); - fs::create_dir_all(&dir).unwrap(); - fs::write( - dir.join("chat_history.jsonl"), - format!("{{\"type\":\"user\",\"content\":\"hello from {id}\"}}\n"), - ) - .unwrap(); - fs::write( - dir.join("summary.json"), - format!( - r#"{{"info":{{"id":"{id}","cwd":"/tmp/stream"}},"created_at":"2026-01-01T00:00:00.000Z"}}"# - ), - ) - .unwrap(); - (dir.join("chat_history.jsonl"), dir) + /// A `subagent/` directory that is itself a symlink — back into its own + /// session, or out of it — is not the session's subagents: nothing under + /// it is read, and the walk ends. + #[cfg(unix)] + #[test] + fn a_symlinked_muse_subagent_directory_is_not_followed() { + let home = tempfile::tempdir().unwrap(); + let (_, transcript) = muse_fixture(home.path()); + let session_dir = transcript.parent().unwrap(); + + // Back into the session. + let real = session_dir.join("subagent"); + let moved = session_dir.join("subagent-real"); + fs::rename(&real, &moved).unwrap(); + std::os::unix::fs::symlink(".", &real).unwrap(); + assert_eq!( + super::muse_session_files(&transcript).unwrap(), + vec![transcript.clone()] + ); + + // Out of the session, to a real subagent tree somewhere else. + fs::remove_file(&real).unwrap(); + let elsewhere = home.path().join("elsewhere"); + fs::rename(&moved, &elsewhere).unwrap(); + std::os::unix::fs::symlink(&elsewhere, &real).unwrap(); + assert_eq!( + super::muse_session_files(&transcript).unwrap(), + vec![transcript.clone()] + ); } - /// A Grok session indexed by a build that stored no `turn_completed.usage` - /// has not changed on disk, so only the retired `grok_events_v1` key can - /// send plain `sync` back over it. With that key the unchanged directory - /// is re-read and gains the breakdown and its turn's request span. + /// A child log that names no session yet — Muse is still writing its + /// header — is not a child that is gone: its evidence and edge stay, and + /// it is linked again once it can be read. #[test] - fn an_upgraded_sync_rereads_unchanged_grok_sessions_for_turn_usage() { + fn an_unidentified_muse_child_log_keeps_its_evidence() { + const REMINDER: &str = "muse-e001"; let home = tempfile::tempdir().unwrap(); - let (chat, dir) = grok_stream_fixture(home.path(), "grok-upg-0001"); - fs::write( - &chat, - concat!( - r#"{"type":"user","content":"hi"}"#, - "\n", - r#"{"type":"assistant","content":"hello","model_id":"grok-4-build"}"#, - "\n" - ), - ) - .unwrap(); - fs::write( - dir.join("updates.jsonl"), - concat!( - r#"{"method":"session/update","params":{"update":{"sessionUpdate":"user_message_chunk"},"_meta":{"eventId":"u1","agentTimestampMs":1789560000000,"turnStartMs":1789560000000}}}"#, - "\n", - r#"{"method":"session/update","params":{"update":{"sessionUpdate":"agent_message_chunk"},"_meta":{"eventId":"a1","agentTimestampMs":1789560001000,"turnStartMs":1789560000000}}}"#, - "\n", - r#"{"method":"_x.ai/session/update","params":{"update":{"sessionUpdate":"turn_completed","totalTokens":1800,"usage":{"inputTokens":1000,"outputTokens":100}},"_meta":{"agentTimestampMs":1789560002000,"turnStartMs":1789560000000}}}"#, - "\n" - ), - ) - .unwrap(); + let (root, transcript) = muse_fixture(home.path()); let conn = open_db(&home.path().join("history.db")).unwrap(); - let root = home.path().join(".grok/sessions"); let mut state = Map::new(); - assert_eq!(super::sync_grok(&conn, &mut state, &root).unwrap(), 1); - let stamps = state[super::GROK_SYNC_STATE_KEY].clone(); + super::sync_muse(&conn, &mut state, &root).unwrap(); + let reminder_events = muse_session_count(&conn, "session_events", REMINDER); + assert!(reminder_events > 0); - // What the previous build left: proxy-only token facts, no span, and - // its stamps under the key it used. - conn.execute_batch( - "UPDATE session_events SET request_span = NULL, \ - token_json = json_remove(token_json, '$.usage') WHERE source = 'grok';", - ) - .unwrap(); - let mut old = Map::new(); - old.insert("grok_events_v1".into(), stamps.clone()); - let mut current = Map::new(); - current.insert(super::GROK_SYNC_STATE_KEY.into(), stamps); - // The positive control: under the current key an unchanged session is - // skipped, which is exactly why the key had to move. - assert_eq!(super::sync_grok(&conn, &mut current, &root).unwrap(), 0); + let log = transcript + .parent() + .unwrap() + .join(format!("subagent/{REMINDER}/session.jsonl")); + let complete = fs::read_to_string(&log).unwrap(); + // Mid-write: the header record is not finished. + fs::write(&log, &complete[..complete.find('\n').unwrap() / 2]).unwrap(); + append_muse_record(&transcript, MUSE_PARENT, "parent-late", "More."); + super::sync_muse(&conn, &mut state, &root).unwrap(); + assert_eq!( + muse_session_count(&conn, "session_events", REMINDER), + reminder_events + ); + assert_eq!(muse_count(&conn, "session_relationships"), 3); - assert_eq!(super::sync_grok(&conn, &mut old, &root).unwrap(), 1); - let (usage, span): (Option, Option) = conn - .query_row( - "SELECT json_extract(token_json, '$.usage.inputTokens'), request_span \ - FROM session_events WHERE source = 'grok' AND token_json IS NOT NULL", - [], - |row| Ok((row.get(0)?, row.get(1)?)), - ) - .unwrap(); - assert_eq!((usage, span.as_deref()), (Some(1000), Some("0"))); + fs::write(&log, &complete).unwrap(); + super::sync_muse(&conn, &mut state, &root).unwrap(); + assert_eq!( + muse_session_count(&conn, "session_events", REMINDER), + reminder_events + ); + assert_eq!(muse_count(&conn, "session_relationships"), 3); } - /// A turn whose only readable output is its thinking still stores its - /// `turn_completed.usage`, on that thinking row, and counts as measured. + /// A remotely known subagent keeps its catalog row; a local read of its + /// log must not turn its assigned objective into that row's typed title, + /// nor clear the title the remote side gave it. #[test] - fn a_thinking_only_grok_turn_keeps_its_usage() { + fn a_muse_subagent_read_leaves_its_catalog_title_alone() { let home = tempfile::tempdir().unwrap(); - let (chat, dir) = grok_stream_fixture(home.path(), "grok-think-0001"); - fs::write( - &chat, - concat!( - r#"{"type":"user","content":"hi"}"#, - "\n", - r#"{"type":"reasoning","summary":"Working through it"}"#, - "\n" - ), + let (_, transcript) = muse_fixture(home.path()); + let conn = open_db(&home.path().join("history.db")).unwrap(); + crate::mark_session_presence(&conn, "muse", MUSE_WORKER, super::SessionLocation::Remote) + .unwrap(); + conn.execute( + "INSERT OR IGNORE INTO sessions (source, session_id, first_prompt) \ + VALUES ('muse', ?, 'remote title')", + [MUSE_WORKER], ) .unwrap(); - fs::write( - dir.join("updates.jsonl"), - concat!( - r#"{"method":"session/update","params":{"update":{"sessionUpdate":"user_message_chunk"},"_meta":{"eventId":"u1","agentTimestampMs":1789560000000,"turnStartMs":1789560000000}}}"#, - "\n", - r#"{"method":"session/update","params":{"update":{"sessionUpdate":"agent_thought_chunk"},"_meta":{"eventId":"t1","agentTimestampMs":1789560001000,"turnStartMs":1789560000000}}}"#, - "\n", - r#"{"method":"_x.ai/session/update","params":{"update":{"sessionUpdate":"turn_completed","usage":{"inputTokens":10,"outputTokens":3}},"_meta":{"agentTimestampMs":1789560002000,"turnStartMs":1789560000000}}}"#, - "\n" - ), + conn.execute( + "UPDATE sessions SET first_prompt = 'remote title' \ + WHERE source = 'muse' AND session_id = ?", + [MUSE_WORKER], ) .unwrap(); - let conn = open_db(&home.path().join("history.db")).unwrap(); - let session = super::scan_grok_session_file(&chat).unwrap().unwrap(); - let outcome = super::ingest_grok_session(&conn, &session, &chat.to_string_lossy()).unwrap(); - let (kind, input): (String, Option) = conn + let log = transcript + .parent() + .unwrap() + .join(format!("subagent/{MUSE_WORKER}/session.jsonl")); + let child = super::read_muse_transcript(&log).unwrap().unwrap(); + super::ingest_muse_session_tree(&conn, &child, &log, None).unwrap(); + let title: Option = conn .query_row( - "SELECT kind, json_extract(token_json, '$.usage.inputTokens') \ - FROM session_events WHERE source = 'grok' AND token_json IS NOT NULL", - [], - |row| Ok((row.get(0)?, row.get(1)?)), + "SELECT first_prompt FROM sessions WHERE source = 'muse' AND session_id = ?", + [MUSE_WORKER], + |row| row.get(0), ) .unwrap(); - assert_eq!((kind.as_str(), input), ("thinking", Some(10))); - assert_eq!((outcome.usage_turns, outcome.turns), (1, 1)); + assert_eq!(title.as_deref(), Some("remote title")); } - /// An `updates.jsonl` that exists and cannot be read is a failure, not an - /// absent stream. - /// - /// Taking it as absent would replace every exact event time with a - /// fallback and drop the token snapshots — and the change stamp, computed - /// from metadata that is still perfectly readable, would then checkpoint - /// that loss as the session's settled state until some file changes again. + /// Every record of a model step shares its model and finish reason; only + /// its usage is the first record's alone. A step that thinks before it + /// calls tools keeps `finish_reason` on the thinking row and on the calls. #[test] - fn an_unreadable_update_stream_fails_instead_of_dating_events_from_fallbacks() { + fn every_record_of_a_muse_step_carries_its_finish_reason() { let home = tempfile::tempdir().unwrap(); - let (chat, dir) = grok_stream_fixture(home.path(), "grok-str-0001"); - - // The positive control first: with no stream at all the session reads - // clean and simply reports that there is none. - let absent = super::scan_grok_session_file(&chat).unwrap().unwrap(); - assert!(!absent.updates_present); - - let stream = dir.join("updates.jsonl"); - #[cfg(unix)] - std::os::unix::fs::symlink(&stream, &stream).unwrap(); - #[cfg(not(unix))] - return; - - let scanned = super::scan_grok_session_file(&chat); + let (root, _) = muse_fixture(home.path()); + let conn = open_db(&home.path().join("history.db")).unwrap(); + let mut state = Map::new(); + super::sync_muse(&conn, &mut state, &root).unwrap(); + let row = |uid: &str| -> (Option, Option) { + conn.query_row( + "SELECT stop_reason, token_json FROM session_events \ + WHERE source = 'muse' AND session_id = ? AND event_uid = ?", + params![MUSE_PARENT, uid], + |row| Ok((row.get(0)?, row.get(1)?)), + ) + .unwrap() + }; + let (thinking_stop, thinking_usage) = row("a001-rec-006"); + assert_eq!(thinking_stop.as_deref(), Some("tool_calls")); assert!( - scanned.is_err(), - "an unreadable update stream must not scan as an absent one" + thinking_usage.is_some(), + "the step's first record owns its usage" ); + let (call_stop, call_usage) = row("tool:call_read"); + assert_eq!(call_stop.as_deref(), Some("tool_calls")); + assert!(call_usage.is_none(), "…and only the first"); + } - // And once it is readable, the same path succeeds and the stream is - // present — so the failure was the unreadable file, not its existence. - fs::remove_file(&stream).unwrap(); - fs::write( - &stream, - br#"{"method":"session/update","params":{"update":{"sessionUpdate":"user_message_chunk"},"_meta":{"agentTimestampMs":1000,"turnStartMs":1000}}} -"#, + /// A subagent log that is there but cannot be read is not a log that is + /// gone: the session's read fails and is retried, and the child keeps + /// its evidence and its edge. + #[cfg(unix)] + #[test] + fn an_unreadable_muse_subagent_log_is_not_forgotten() { + use std::os::unix::fs::PermissionsExt; + const REMINDER: &str = "muse-e001"; + let home = tempfile::tempdir().unwrap(); + let (root, transcript) = muse_fixture(home.path()); + let conn = open_db(&home.path().join("history.db")).unwrap(); + let mut state = Map::new(); + super::sync_muse(&conn, &mut state, &root).unwrap(); + let reminder_events = muse_session_count(&conn, "session_events", REMINDER); + assert!(reminder_events > 0); + + // The parent changes, so its tree is read again; the reminder's + // directory stays listable but its log can no longer be statted. + append_muse_record(&transcript, MUSE_PARENT, "parent-late", "More."); + let reminder_dir = transcript + .parent() + .unwrap() + .join(format!("subagent/{REMINDER}")); + fs::set_permissions(&reminder_dir, fs::Permissions::from_mode(0o644)).unwrap(); + let stat_blocked = fs::metadata(reminder_dir.join("session.jsonl")).is_err(); + let result = super::sync_muse(&conn, &mut state, &root); + fs::set_permissions(&reminder_dir, fs::Permissions::from_mode(0o755)).unwrap(); + if !stat_blocked { + // Running as a user permissions do not bind (root in a + // container): there is nothing unreadable to test. + return; + } + assert!(result.is_err(), "the only session could not be read"); + assert_eq!( + muse_session_count(&conn, "session_events", REMINDER), + reminder_events, + "an unreadable log is not a deleted one" + ); + assert_eq!(muse_count(&conn, "session_relationships"), 3); + } + + /// A removed child's leftover history — filed under it before it was + /// linked as a subagent — goes with the rest of its evidence. + #[test] + fn a_removed_muse_child_takes_its_leftover_history_with_it() { + let home = tempfile::tempdir().unwrap(); + let (root, transcript) = muse_fixture(home.path()); + let conn = open_db(&home.path().join("history.db")).unwrap(); + let mut state = Map::new(); + super::sync_muse(&conn, &mut state, &root).unwrap(); + insert_history( + &conn, + &HistoryEntry { + id: 0, + source: "muse".into(), + session_id: Some(MUSE_WORKER.to_string()), + project: None, + prompt: "indexed as a session once".into(), + prompt_hash: Some(prompt_hash("indexed as a session once")), + timestamp_ms: 42, + }, ) .unwrap(); - let read = super::scan_grok_session_file(&chat).unwrap().unwrap(); - assert!(read.updates_present); - assert_eq!(read.first_ts, 1000); + fs::remove_dir_all( + transcript + .parent() + .unwrap() + .join(format!("subagent/{MUSE_WORKER}")), + ) + .unwrap(); + super::sync_muse(&conn, &mut state, &root).unwrap(); + assert_eq!(muse_session_count(&conn, "history", MUSE_WORKER), 0); } - /// "Nothing in it parsed" is not "it is not there". A stream of rows this - /// parser does not interpret is a stream that exists, and a caller told it - /// is missing would look for a file that is sitting right there. + /// Hydrating a subagent's log directly reads it as a subagent: its + /// objective is an event, never history. #[test] - fn a_stream_of_rows_the_parser_cannot_use_is_present_not_missing() { + fn a_muse_subagent_log_read_directly_writes_no_history() { let home = tempfile::tempdir().unwrap(); - let (chat, dir) = grok_stream_fixture(home.path(), "grok-str-0002"); - fs::write( - dir.join("updates.jsonl"), - b"{\"hello\":\"world\"}\n{\"another\":\"row\"}\n", + let (_, transcript) = muse_fixture(home.path()); + let conn = open_db(&home.path().join("history.db")).unwrap(); + let log = transcript + .parent() + .unwrap() + .join(format!("subagent/{MUSE_WORKER}/session.jsonl")); + let child = super::read_muse_transcript(&log).unwrap().unwrap(); + super::ingest_muse_session_tree(&conn, &child, &log, None).unwrap(); + assert!(muse_session_count(&conn, "session_events", MUSE_WORKER) > 0); + assert_eq!(muse_session_count(&conn, "history", MUSE_WORKER), 0); + assert_eq!( + muse_session_count(&conn, "sessions", MUSE_WORKER), + 0, + "nor a catalog row of its own" + ); + } + + /// Scoped to its own prompt events, a remote session's history cleanup + /// still re-attributes a row another session evidences rather than + /// deleting it, and leaves rows it never wrote alone. + #[test] + fn scoped_history_cleanup_reattributes_shared_prompts() { + let conn = Connection::open_in_memory().unwrap(); + init_db(&conn).unwrap(); + for (session, text) in [("a", "shared"), ("b", "shared"), ("a", "only a")] { + conn.execute( + "INSERT INTO session_events (source, session_id, ts_ms, role, kind, text, event_uid) \ + VALUES ('muse', ?, 7, 'user', 'text', ?, ?)", + params![session, text, format!("{session}:{text}")], + ) + .unwrap(); + } + for (session, text) in [("a", "shared"), ("a", "only a"), ("a", "remote")] { + conn.execute( + "INSERT INTO history (source, session_id, prompt, timestamp_ms) \ + VALUES ('muse', ?, ?, 7)", + params![session, text], + ) + .unwrap(); + } + super::replace_session_history_scoped( + &conn, + "muse", + "a", + super::HistoryScope::OwnPromptEvents, ) .unwrap(); - let conn = open_db(&home.path().join("history.db")).unwrap(); - let session = super::scan_grok_session_file(&chat).unwrap().unwrap(); - let outcome = super::ingest_grok_session(&conn, &session, &chat.to_string_lossy()).unwrap(); - assert!( - !outcome.missing_updates, - "the file exists; it just said nothing this parser understands" + let rows: Vec<(String, String)> = conn + .prepare("SELECT session_id, prompt FROM history ORDER BY prompt") + .unwrap() + .query_map([], |row| Ok((row.get(0)?, row.get(1)?))) + .unwrap() + .collect::>() + .unwrap(); + assert_eq!( + rows, + vec![ + ("a".to_string(), "remote".to_string()), + ("b".to_string(), "shared".to_string()), + ] ); - assert!(outcome.updates_yielded_no_timing); - assert_eq!(outcome.unread_update_rows, 2); + } - // The positive control: with the file removed, it really is missing. - fs::remove_file(dir.join("updates.jsonl")).unwrap(); - let session = super::scan_grok_session_file(&chat).unwrap().unwrap(); - let outcome = super::ingest_grok_session(&conn, &session, &chat.to_string_lossy()).unwrap(); - assert!(outcome.missing_updates); - assert!(!outcome.updates_yielded_no_timing); - assert_eq!(outcome.unread_update_rows, 0); + /// A rewritten transcript that no longer has a reply or a prompt does not + /// keep quoting the old ones in the catalog. + #[test] + fn a_rewritten_muse_transcript_replaces_its_catalog_text() { + let home = tempfile::tempdir().unwrap(); + let (root, transcript) = muse_fixture(home.path()); + let conn = open_db(&home.path().join("history.db")).unwrap(); + let mut state = Map::new(); + super::sync_muse(&conn, &mut state, &root).unwrap(); + let catalog = |conn: &Connection| -> (Option, Option) { + conn.query_row( + "SELECT first_prompt, last_assistant_text FROM sessions \ + WHERE source = 'muse' AND session_id = ?", + [MUSE_PARENT], + |row| Ok((row.get(0)?, row.get(1)?)), + ) + .unwrap() + }; + let (prompt, reply) = catalog(&conn); + assert!(prompt.is_some() && reply.is_some()); + + // Keep only the permission frame, metadata and the open record. + let head: String = fs::read_to_string(&transcript) + .unwrap() + .lines() + .take(3) + .map(|line| format!("{line}\n")) + .collect(); + fs::write(&transcript, head).unwrap(); + super::sync_muse(&conn, &mut state, &root).unwrap(); + assert_eq!(catalog(&conn), (None, None)); + } + + /// A child whose log is gone is no longer evidenced: the edge and the + /// child's events go with it. + #[test] + fn muse_resync_drops_the_edge_to_a_removed_subagent_log() { + let home = tempfile::tempdir().unwrap(); + let (root, transcript) = muse_fixture(home.path()); + let conn = open_db(&home.path().join("history.db")).unwrap(); + let mut state = Map::new(); + super::sync_muse(&conn, &mut state, &root).unwrap(); + fs::remove_dir_all(transcript.parent().unwrap().join("subagent/muse-e001")).unwrap(); + super::sync_muse(&conn, &mut state, &root).unwrap(); + let children: Vec = conn + .prepare( + "SELECT child_session_id FROM session_relationships \ + WHERE source = 'muse' ORDER BY child_session_id", + ) + .unwrap() + .query_map([], |row| row.get(0)) + .unwrap() + .collect::>() + .unwrap(); + assert_eq!( + children, + vec![MUSE_WORKER.to_string(), "muse-c002".to_string()] + ); + assert_eq!( + muse_session_count(&conn, "session_events", "muse-e001"), + 0, + "the removed child's events go with its edge" + ); } /// A stamp that fails closed has to be *reported*. Round four stopped one diff --git a/crates/ai-hist/src/ingest/hydrate.rs b/crates/ai-hist/src/ingest/hydrate.rs index ace67301..e84a565f 100644 --- a/crates/ai-hist/src/ingest/hydrate.rs +++ b/crates/ai-hist/src/ingest/hydrate.rs @@ -1364,7 +1364,7 @@ fn validate_options(options: &HydrateSessionOptions) -> Result<()> { } if !matches!( options.source.as_str(), - "claude" | "codex" | "cursor" | "grok" | "relay" | "opencode" + "claude" | "codex" | "cursor" | "grok" | "relay" | "opencode" | "muse" ) { return Err(hydration_error( "INVALID_ARGUMENT", @@ -1621,6 +1621,22 @@ fn source_snapshot( SnapshotRecords::DeferredGrok(path.clone()), inventory.stamp, ) + } else if options.source == "muse" && options.include_related { + // With related evidence, a Muse session is its transcript plus the + // subagent logs beside it; the stamp and the counts cover all of them, + // so a child that grew after the parent's last record is still a + // change. Without it, the transcript alone, like any other file. + let mut bytes = 0i64; + let mut records = 0i64; + for file in muse_session_files(&path)? { + bytes += file.metadata()?.len() as i64; + records += complete_jsonl_records(&file)?; + } + ( + bytes, + SnapshotRecords::Counted(records), + muse_session_stamp(&path)?, + ) } else if let Some(snapshot) = captured_claude.as_ref() { // The hook already read these bytes; nothing here opens the file. ( @@ -1937,6 +1953,7 @@ pub(crate) fn validate_provider_path( ], "cursor" => vec![provider_roots.home.join(".cursor/projects")], "grok" => vec![provider_roots.grok.join("sessions")], + "muse" => vec![provider_roots.muse.clone()], _ => Vec::new(), }; let canonical = fs::canonicalize(path)?; @@ -2181,6 +2198,20 @@ fn ingest_selected( } "grok" => ingest_grok(conn, options, path.unwrap()) .map(|diagnostics| (whole_file(), diagnostics, None)), + "muse" => { + let path = path.unwrap(); + let diagnostics = ingest_muse(conn, options, path)?; + let mut outcome = whole_file(); + // With related evidence the read covers every subagent log too. + if options.include_related { + outcome.bytes_read = muse_session_files(path)? + .iter() + .filter_map(|file| file.metadata().ok()) + .map(|metadata| metadata.len() as i64) + .sum(); + } + Ok((outcome, diagnostics, None)) + } "opencode" => { let path = path.unwrap(); // Whichever layout `source_snapshot` validated this locator @@ -3010,6 +3041,114 @@ fn ingest_grok( Ok(grok_diagnostics(&outcome)) } +fn ingest_muse( + conn: &Connection, + options: &HydrateSessionOptions, + path: &Path, +) -> Result> { + // Subagent logs are related evidence: read and linked only when the + // caller asked for it, and otherwise left exactly as they are. + let tree = if options.include_related { + read_muse_tree(path)? + } else { + read_muse_transcript(path)?.map(|transcript| (transcript, Vec::new())) + }; + let (transcript, children) = tree.ok_or_else(|| { + hydration_error( + "SESSION_SOURCE_MISMATCH", + "Muse Code transcript has no session metadata", + ) + })?; + if transcript.metadata.session_id != options.session_id { + return Err(hydration_error( + "SESSION_SOURCE_MISMATCH", + "Muse Code transcript identity does not match the catalog row", + )); + } + let outcome = ingest_muse_session_tree( + conn, + &transcript, + path, + options.include_related.then_some(children.as_slice()), + )?; + Ok(muse_diagnostics(&outcome)) +} + +/// What a Muse Code transcript could not establish on its own. Every code +/// describes an absence in Muse's records, not a failure of this run. +fn muse_diagnostics(outcome: &MuseIngestOutcome) -> Vec { + let diagnostic = |code: &str, message: String| HydrationDiagnostic { + code: code.to_string(), + message, + duration_ms: None, + source_bytes: None, + records_parsed: None, + }; + let mut diagnostics = Vec::new(); + if outcome.encrypted_reasoning > 0 { + diagnostics.push(diagnostic( + "MUSE_REASONING_ENCRYPTED", + format!( + "{} reasoning record(s) carried only an encrypted trace; each is recorded as \ + an encrypted_reasoning marker rather than as thinking text", + outcome.encrypted_reasoning + ), + )); + } + if outcome.unattached_usage > 0 { + diagnostics.push(diagnostic( + "MUSE_USAGE_UNATTACHED", + format!( + "{} model step(s) reported usage but committed no assistant record in the \ + same run to carry it; that usage is not stored on any event", + outcome.unattached_usage + ), + )); + } + if outcome.subagent_calls > outcome.worker_children { + diagnostics.push(diagnostic( + "MUSE_SUBAGENT_LOG_MISSING", + format!( + "{} subagent_spawn call(s) but {} subagent log(s) beside the session; a \ + spawn with no log is visible as a tool call and has no child to link", + outcome.subagent_calls, outcome.worker_children + ), + )); + } + if outcome.unidentified_children > 0 { + diagnostics.push(diagnostic( + "MUSE_SUBAGENT_LOG_UNIDENTIFIED", + format!( + "{} subagent log(s) carried no session metadata, so there is no child \ + session id to link them under", + outcome.unidentified_children + ), + )); + } + if outcome.results_without_outcome > 0 { + diagnostics.push(diagnostic( + "MUSE_TOOL_OUTCOME_MISSING", + format!( + "{} tool result(s) had no tool_batch.effect.terminal record for their call; \ + their status is unknown rather than assumed", + outcome.results_without_outcome + ), + )); + } + if outcome.unparsed_lines > 0 { + diagnostics.push(diagnostic( + "MUSE_LINES_UNPARSED", + format!( + "{} complete line(s) of session.jsonl were not JSON records and were \ + skipped; a live session's unterminated last line is not counted, it is \ + read once Muse finishes it", + outcome.unparsed_lines + ), + )); + } + diagnostics +} + /// How much of a Grok session's usage is billing evidence, parsed or cached. /// /// Every code here describes an absence in Grok's records, not a failure of @@ -4089,6 +4228,46 @@ mod tests { found.pop().expect("a staged chat_history.jsonl") } + /// A hydration that did not ask for related evidence indexes the session + /// and leaves its subagent logs alone; asking for it links them. + #[test] + fn muse_hydration_links_subagents_only_when_related_evidence_is_asked_for() { + let dir = tempfile::tempdir().unwrap(); + copy_tree( + &Path::new(env!("CARGO_MANIFEST_DIR")).join("tests/fixtures/muse/tools-session"), + dir.path(), + ); + let transcript = dir + .path() + .join(".local/share/muse/sessions/2026/09/20/muse-a001/session.jsonl"); + let db = dir.path().join("history.db"); + let conn = open_db(&db).unwrap(); + catalog_row(&conn, "muse", "muse-a001", Some(&transcript)); + let counts = |conn: &Connection| -> (i64, i64) { + conn.query_row( + "SELECT (SELECT COUNT(*) FROM session_relationships WHERE source = 'muse'), \ + (SELECT COUNT(*) FROM session_events \ + WHERE source = 'muse' AND session_id <> 'muse-a001')", + [], + |row| Ok((row.get(0)?, row.get(1)?)), + ) + .unwrap() + }; + + let mut without = options("muse", "muse-a001"); + without.include_related = false; + let result = hydrate_session_at_with_home(&db, &without, dir.path()).unwrap(); + assert!(result.related_session_ids.is_empty()); + assert_eq!(counts(&conn), (0, 0), "no child evidence was written"); + + let result = + hydrate_session_at_with_home(&db, &options("muse", "muse-a001"), dir.path()).unwrap(); + assert_eq!(result.related_session_ids.len(), 3); + let (edges, child_events) = counts(&conn); + assert_eq!(edges, 3); + assert!(child_events > 0); + } + fn copy_tree(from: &Path, to: &Path) { for entry in fs::read_dir(from).unwrap().flatten() { let target = to.join(entry.file_name()); diff --git a/crates/ai-hist/src/ingest/muse.rs b/crates/ai-hist/src/ingest/muse.rs new file mode 100644 index 00000000..3c237a26 --- /dev/null +++ b/crates/ai-hist/src/ingest/muse.rs @@ -0,0 +1,958 @@ +//! Muse Code (Meta's `muse` CLI) session record parsing. +//! +//! One place for the interpretation of +//! `$XDG_DATA_HOME/muse/sessions/YYYY/MM/DD//session.jsonl` +//! (`~/.local/share/muse/sessions/…` when `XDG_DATA_HOME` is unset), so +//! shallow discovery, full sync and targeted hydration cannot drift apart. +//! +//! ## What Muse actually writes +//! +//! `session.jsonl` is an append-only, event-sourced log. Every line is one +//! envelope: +//! +//! ```json +//! {"schema_version": 1, "id": "", +//! "stream": {"kind": "session", "id": ""}, "sequence": 1, +//! "recorded_at": 1788223110587169, "payload_type": "runtime.session", …, +//! "payload": { … }} +//! ``` +//! +//! * `recorded_at` is **microseconds** since the epoch, on every record, so +//! no timestamp here is ever inferred. +//! * `runtime.session.metadata` (normally the first record, though a +//! permission frame can precede it) names the session: `stream.id` is the +//! session id and `payload.record` carries `workspace_root`, `model_id`, +//! `provider_id` and `build.semver`. +//! * `runtime.session` with `payload.kind == "run"` is the conversation, keyed +//! by `payload.event.kind`: `started` (`prompt`, the typed message), +//! `reasoning_committed`, `assistant_message_committed`, +//! `assistant_tool_calls_committed` (`tool_calls[].args` is a JSON +//! **string**), `tool_result_batch_committed` (results keyed by +//! `tool_call_id`), `model_completed` (per model step: `model`, `usage`, +//! `duration_ms`, `finish_reason`) and `terminal` (run end). +//! `payload.kind == "task"` is execution bookkeeping and is not read. +//! * `tool_batch.effect.terminal` carries the authoritative outcome of each +//! call (`outcome.kind` = `completed` / `failed` / …) keyed by `call_id`. +//! The result text itself carries no error flag. +//! * `session.opened.observed`, `session.resumed` and `session.end` are +//! process lifecycle; they become markers. +//! * Subagent and reminder runs write their own +//! `subagent//session.jsonl` beside the parent, and their task +//! streams are also mirrored into the parent file under a **different** +//! `stream.id`. Only records on the session's own stream are read, and only +//! top-level transcripts are sessions. +//! +//! The shapes were characterized from a transcript captured by the real CLI +//! (Muse 0.2.1, checked in under `tests/fixtures/muse/`) and cross-checked +//! against the published Muse Code SDK and independent readers. + +use serde_json::Value; +use std::collections::HashMap; +use std::path::Path; + +/// The `payload_type` of the record that names the session. +pub(crate) const METADATA_PAYLOAD_TYPE: &str = "runtime.session.metadata"; + +/// The directory name a subagent (or reminder child) transcript lives under. +pub(crate) const SUBAGENT_DIR: &str = "subagent"; + +/// The file name every Muse transcript has. +pub(crate) const SESSION_FILE: &str = "session.jsonl"; + +/// What the metadata record says about a session. +#[derive(Debug, Clone, Default, PartialEq, Eq)] +pub(crate) struct MuseMetadata { + pub session_id: String, + pub workspace_root: Option, + pub model_id: Option, + pub provider_id: Option, + pub cli_version: Option, + pub recorded_ms: Option, +} + +/// One tool call as `assistant_tool_calls_committed` records it. +#[derive(Debug, Clone, PartialEq)] +pub(crate) struct MuseToolCall { + /// The provider call id results and outcomes join on. + pub call_id: String, + pub name: String, + /// `args` parsed from its JSON string; the raw string when it does not + /// parse, so nothing Muse wrote is dropped. + pub arguments: Value, +} + +/// One tool result as `tool_result_batch_committed` records it. +#[derive(Debug, Clone, PartialEq, Eq)] +pub(crate) struct MuseToolResult { + pub call_id: String, + pub text: Option, +} + +/// The conversation-level meaning of one record on the session's own stream. +#[derive(Debug, Clone, PartialEq)] +pub(crate) enum MuseEvent { + /// A typed prompt that opened a run. + Prompt { + text: String, + }, + /// Thinking. `text` is `None` when Muse wrote only the encrypted trace. + Reasoning { + message_id: Option, + text: Option, + encrypted: bool, + }, + AssistantText { + message_id: Option, + response_id: Option, + text: String, + }, + ToolCalls { + message_id: Option, + response_id: Option, + calls: Vec, + }, + ToolResults { + results: Vec, + }, + /// One model step finished. `usage` is Muse's own object, verbatim. + ModelCompleted { + model: Option, + usage: Option, + finish_reason: Option, + duration_ms: Option, + }, + /// The run ended. + RunTerminal { + status: Option, + reason: Option, + }, + /// The authoritative outcome of one tool call. + ToolOutcome { + call_id: String, + status: String, + reason: Option, + }, + SessionOpened { + resume: bool, + }, + SessionResumed, + SessionEnd { + exit_reason: Option, + }, + /// A metadata record after the first: a model or provider switch. + Metadata(MuseMetadata), + /// The parent linked a child agent's log: a subagent's task stream + /// (`task_stream_linked`) or a reminder child + /// (`memory_reminder_child_session_linked`). Describes the child; its + /// identity is always read from the child's own transcript. + SubagentLinked(MuseSubagentLink), +} + +/// What a parent records about one child agent it started. +#[derive(Debug, Clone, Default, PartialEq, Eq)] +pub(crate) struct MuseSubagentLink { + /// The child's log, relative to the parent's session directory: + /// `subagent//session.jsonl`. + pub path: Option, + /// `worker`, `reminder`, …: what kind of child this is. + pub role: Option, + /// The display label the parent gave it. + pub label: Option, + /// The child's model, when the parent named a concrete one. + pub model: Option, + pub task_id: Option, + /// The child's session id, where the parent states it outright. + pub child_session_id: Option, +} + +/// One interpreted record, with the envelope facts every event carries. +#[derive(Debug, Clone, PartialEq)] +pub(crate) struct MuseRecord { + /// The envelope `id`: Muse's own, stable record identity. + pub record_id: Option, + pub sequence: Option, + pub ts_ms: i64, + pub run_id: Option, + pub event: MuseEvent, +} + +/// A whole transcript, read. +#[derive(Debug, Clone, Default)] +pub(crate) struct MuseTranscript { + pub metadata: MuseMetadata, + pub records: Vec, + /// Lines that were not JSON objects. A trailing partial line of a live + /// session lands here too. + pub unparsed_lines: usize, + /// Records on another stream (mirrored subagent / reminder tasks). + pub foreign_stream_records: usize, +} + +impl MuseTranscript { + pub fn first_ts(&self) -> Option { + self.metadata + .recorded_ms + .into_iter() + .chain(self.records.iter().map(|record| record.ts_ms)) + .min() + } + + pub fn last_ts(&self) -> Option { + self.metadata + .recorded_ms + .into_iter() + .chain(self.records.iter().map(|record| record.ts_ms)) + .max() + } + + /// Every model the session names, metadata first, in first-seen order. + pub fn models(&self) -> Vec { + let mut models: Vec = Vec::new(); + let mut push = |model: Option<&str>| { + if let Some(model) = model.filter(|model| !model.is_empty()) { + if !models.iter().any(|seen| seen == model) { + models.push(model.to_string()); + } + } + }; + push(self.metadata.model_id.as_deref()); + for record in &self.records { + match &record.event { + MuseEvent::ModelCompleted { model, .. } => push(model.as_deref()), + MuseEvent::Metadata(metadata) => push(metadata.model_id.as_deref()), + _ => {} + } + } + models + } +} + +/// `recorded_at` microseconds as epoch milliseconds. +pub(crate) fn recorded_ms(value: &Value) -> Option { + value + .get("recorded_at") + .and_then(Value::as_i64) + .filter(|micros| *micros > 0) + .map(|micros| micros / 1000) +} + +fn text_field(value: &Value, key: &str) -> Option { + value + .get(key) + .and_then(Value::as_str) + .filter(|text| !text.trim().is_empty()) + .map(str::to_string) +} + +fn stream_id(value: &Value) -> Option<&str> { + value.pointer("/stream/id").and_then(Value::as_str) +} + +/// The session a metadata record names, or `None` when the record is not one +/// (or names no session). Identity comes only from `stream.id` on a +/// `session`-kind stream, never from the directory name. +pub(crate) fn parse_metadata(value: &Value) -> Option { + if value.get("payload_type").and_then(Value::as_str) != Some(METADATA_PAYLOAD_TYPE) { + return None; + } + if value.pointer("/stream/kind").and_then(Value::as_str) != Some("session") { + return None; + } + let session_id = stream_id(value).filter(|id| !id.is_empty())?; + let record = value.pointer("/payload/record").unwrap_or(&Value::Null); + Some(MuseMetadata { + session_id: session_id.to_string(), + workspace_root: text_field(record, "workspace_root"), + model_id: text_field(record, "model_id"), + provider_id: text_field(record, "provider_id"), + cli_version: record + .pointer("/build/semver") + .and_then(Value::as_str) + .filter(|version| !version.is_empty()) + .map(str::to_string), + recorded_ms: recorded_ms(value), + }) +} + +/// Interpret one record that is already known to be on the session's stream. +/// `None` for the bookkeeping and diagnostics this reader does not keep. +pub(crate) fn parse_event(value: &Value) -> Option { + let payload = value.get("payload")?; + match value.get("payload_type").and_then(Value::as_str)? { + METADATA_PAYLOAD_TYPE => parse_metadata(value).map(MuseEvent::Metadata), + "runtime.session" => { + if payload.get("kind").and_then(Value::as_str) != Some("run") { + return None; + } + parse_run_event(payload.get("event")?) + } + "tool_batch.effect.terminal" => { + let record = payload.get("record")?; + let call_id = text_field(record, "call_id")?; + let outcome = record.get("outcome")?; + Some(MuseEvent::ToolOutcome { + call_id, + status: text_field(outcome, "kind")?, + reason: text_field(outcome, "reason"), + }) + } + "session.opened.observed" => Some(MuseEvent::SessionOpened { + resume: payload + .pointer("/record/resume") + .and_then(Value::as_bool) + .unwrap_or(false), + }), + "session.resumed" => Some(MuseEvent::SessionResumed), + "session.end" => Some(MuseEvent::SessionEnd { + exit_reason: payload + .get("record") + .and_then(|record| text_field(record, "exit_reason")), + }), + _ => None, + } +} + +fn parse_run_event(event: &Value) -> Option { + match event.get("kind").and_then(Value::as_str)? { + "started" => text_field(event, "prompt").map(|text| MuseEvent::Prompt { text }), + "reasoning_committed" => Some(MuseEvent::Reasoning { + message_id: text_field(event, "message_id"), + text: text_field(event, "text"), + encrypted: event + .get("encrypted_content") + .is_some_and(|content| !content.is_null() && content != ""), + }), + "assistant_message_committed" => Some(MuseEvent::AssistantText { + message_id: text_field(event, "message_id"), + response_id: text_field(event, "response_id"), + text: text_field(event, "text")?, + }), + "assistant_tool_calls_committed" => { + let calls: Vec = event + .get("tool_calls") + .and_then(Value::as_array) + .map(|calls| calls.iter().filter_map(parse_tool_call).collect()) + .unwrap_or_default(); + (!calls.is_empty()).then(|| MuseEvent::ToolCalls { + message_id: text_field(event, "message_id"), + response_id: text_field(event, "response_id"), + calls, + }) + } + "tool_result_batch_committed" => { + let results: Vec = event + .get("results") + .and_then(Value::as_array) + .map(|results| { + results + .iter() + .filter_map(|result| { + Some(MuseToolResult { + call_id: text_field(result, "tool_call_id")?, + text: result + .get("text") + .and_then(Value::as_str) + .map(str::to_string), + }) + }) + .collect() + }) + .unwrap_or_default(); + (!results.is_empty()).then_some(MuseEvent::ToolResults { results }) + } + "model_completed" => Some(MuseEvent::ModelCompleted { + model: text_field(event, "model"), + usage: event + .get("usage") + .filter(|usage| usage.is_object()) + .cloned(), + finish_reason: text_field(event, "finish_reason"), + duration_ms: event.get("duration_ms").and_then(Value::as_i64), + }), + "task_stream_linked" => { + let display = event.get("display").unwrap_or(&Value::Null); + Some(MuseEvent::SubagentLinked(MuseSubagentLink { + path: text_field(display, "path"), + role: text_field(display, "role"), + label: text_field(display, "label"), + // `same-as-main` is a policy, not a model name. + model: text_field(display, "model").filter(|model| model != "same-as-main"), + task_id: text_field(event, "task_id"), + child_session_id: None, + })) + } + "memory_reminder_child_session_linked" => { + Some(MuseEvent::SubagentLinked(MuseSubagentLink { + path: text_field(event, "child_session_log_path"), + role: Some("reminder".to_string()), + label: text_field(event, "reminder_agent_id"), + model: None, + task_id: text_field(event, "task_id"), + child_session_id: text_field(event, "child_session_id"), + })) + } + "terminal" => Some(MuseEvent::RunTerminal { + status: text_field(event, "terminal"), + reason: text_field(event, "reason"), + }), + _ => None, + } +} + +fn parse_tool_call(call: &Value) -> Option { + // `call_id` is what results and outcomes name; `id` is the provider's + // response item id and only the fallback. + let call_id = text_field(call, "call_id").or_else(|| text_field(call, "id"))?; + let name = text_field(call, "name")?; + let arguments = match call.get("args") { + Some(Value::String(raw)) => { + serde_json::from_str(raw).unwrap_or_else(|_| Value::String(raw.clone())) + } + Some(other) => other.clone(), + None => Value::Null, + }; + Some(MuseToolCall { + call_id, + name, + arguments, + }) +} + +/// Read a whole transcript. `None` when it names no session — no metadata +/// record on a `session` stream — which is "not a session", not an error. +pub(crate) fn parse_transcript(contents: &str) -> Option { + let mut values = Vec::new(); + let mut unparsed_lines = 0; + // Only newline-terminated records are committed. A live session's last + // line may still be mid-write, and a prefix of a record can itself be + // valid JSON, so the unterminated tail is left for the next read. + let complete = match contents.rfind('\n') { + Some(end) => &contents[..=end], + None => "", + }; + for line in complete.lines() { + if line.trim().is_empty() { + continue; + } + match serde_json::from_str::(line) { + Ok(value) if value.is_object() => values.push(value), + _ => unparsed_lines += 1, + } + } + let metadata = values.iter().find_map(parse_metadata)?; + let mut transcript = MuseTranscript { + metadata, + unparsed_lines, + ..Default::default() + }; + let mut seen_metadata = false; + for value in &values { + // Mirrored subagent / reminder task records share the file but not + // the stream. Reading them would present every child objective as a + // prompt the person typed. + if stream_id(value) != Some(transcript.metadata.session_id.as_str()) { + transcript.foreign_stream_records += 1; + continue; + } + let Some(event) = parse_event(value) else { + continue; + }; + if matches!(event, MuseEvent::Metadata(_)) && !seen_metadata { + // The first metadata record is the session's header, not a switch. + seen_metadata = true; + continue; + } + let Some(ts_ms) = recorded_ms(value) else { + continue; + }; + transcript.records.push(MuseRecord { + record_id: text_field(value, "id"), + sequence: value.get("sequence").and_then(Value::as_i64), + ts_ms, + run_id: value + .pointer("/payload/run_id") + .and_then(Value::as_str) + .map(str::to_string), + event, + }); + } + Some(transcript) +} + +/// Which assistant record each model step's `model_completed` belongs to. +/// +/// Muse does not write the two in one order. A step that calls tools logs +/// `model_completed` *before* `assistant_tool_calls_committed`; a step that +/// answers in prose logs `assistant_message_committed` *before* its +/// `model_completed` (both are in the real 0.2.1 capture). A step can also +/// commit several records — readable reasoning and then its tool calls. +/// +/// What does delimit a step is what the model is called *after*: a typed +/// prompt, or the tool results it asked for. So each run is cut into +/// segments at `started`, `tool_result_batch_committed` and `terminal`. A +/// segment is one model call: its first assistant record owns that call's +/// `model_completed`, whichever of the two came first, and every other +/// record of the step is part of it and owns no usage of its own. A second +/// usage-bearing `model_completed` in one segment (a retried call) has no +/// record of its own to carry it, and is counted rather than attached to a +/// neighbour. +/// +/// Indexes are into `transcript.records`. +pub(crate) fn pair_model_steps(transcript: &MuseTranscript) -> ModelSteps { + #[derive(Default)] + struct Segment { + commits: Vec, + steps: Vec, + } + let has_usage = |index: usize| { + matches!( + &transcript.records[index].event, + MuseEvent::ModelCompleted { usage: Some(_), .. } + ) + }; + let mut paired = ModelSteps::default(); + let mut close = |segment: Segment| { + let owner = segment + .steps + .iter() + .copied() + .find(|step| has_usage(*step)) + .or_else(|| segment.steps.first().copied()); + match (segment.commits.first().copied(), owner) { + (Some(commit), Some(step)) => { + paired.owners.insert(commit, step); + for member in &segment.commits { + paired.members.insert(*member, step); + } + paired.orphans += segment + .steps + .iter() + .filter(|other| **other != step && has_usage(**other)) + .count(); + } + _ => { + paired.orphans += segment + .steps + .iter() + .filter(|step| has_usage(**step)) + .count(); + } + } + }; + let mut runs: HashMap, Segment> = HashMap::new(); + for (index, record) in transcript.records.iter().enumerate() { + let run = record.run_id.as_deref(); + match &record.event { + MuseEvent::AssistantText { .. } + | MuseEvent::ToolCalls { .. } + | MuseEvent::Reasoning { text: Some(_), .. } => { + runs.entry(run).or_default().commits.push(index); + } + MuseEvent::ModelCompleted { .. } => runs.entry(run).or_default().steps.push(index), + MuseEvent::Prompt { .. } + | MuseEvent::ToolResults { .. } + | MuseEvent::RunTerminal { .. } => { + if let Some(segment) = runs.remove(&run) { + close(segment); + } + } + _ => {} + } + } + for (_, segment) in runs.drain() { + close(segment); + } + paired +} + +/// How [`pair_model_steps`] assigned each model step. +#[derive(Debug, Default)] +pub(crate) struct ModelSteps { + /// The one assistant record that carries each step's usage, mapped to + /// that step's `model_completed`. + pub owners: HashMap, + /// Every assistant record of a step — its owner and the rest (reasoning + /// before tool calls) — mapped to the step's `model_completed`, whose + /// model and `finish_reason` all of them share. + pub members: HashMap, + /// Steps that reported usage but committed no assistant record to carry + /// it. + pub orphans: usize, +} + +/// Every call's tool name, by call id. +pub(crate) fn tool_names(transcript: &MuseTranscript) -> HashMap<&str, &str> { + transcript + .records + .iter() + .filter_map(|record| match &record.event { + MuseEvent::ToolCalls { calls, .. } => Some(calls), + _ => None, + }) + .flatten() + .map(|call| (call.call_id.as_str(), call.name.as_str())) + .collect() +} + +/// Every call's authoritative outcome, by call id. +pub(crate) fn tool_outcomes(transcript: &MuseTranscript) -> HashMap<&str, (&str, Option<&str>)> { + transcript + .records + .iter() + .filter_map(|record| match &record.event { + MuseEvent::ToolOutcome { + call_id, + status, + reason, + } => Some((call_id.as_str(), (status.as_str(), reason.as_deref()))), + _ => None, + }) + .collect() +} + +/// Whether a transcript path is a subagent or reminder child rather than a +/// session of its own: any `subagent` directory between it and the root. +pub(crate) fn is_child_transcript(path: &Path, root: &Path) -> bool { + path.strip_prefix(root) + .unwrap_or(path) + .components() + .any(|component| component.as_os_str() == SUBAGENT_DIR) +} + +/// Everything the parent recorded about its children, keyed by the child +/// log's path relative to the parent's session directory. A child linked by +/// both a task stream and a reminder event keeps every field either named. +pub(crate) fn subagent_links( + transcript: &MuseTranscript, +) -> HashMap { + let mut links: HashMap = HashMap::new(); + for record in &transcript.records { + let MuseEvent::SubagentLinked(link) = &record.event else { + continue; + }; + let Some(path) = link.path.as_deref() else { + continue; + }; + let entry = links + .entry(normalize_link_path(path)) + .or_insert_with(|| (record.ts_ms, MuseSubagentLink::default())); + let merged = &mut entry.1; + merged.path = merged.path.take().or_else(|| link.path.clone()); + merged.role = merged.role.take().or_else(|| link.role.clone()); + merged.label = merged.label.take().or_else(|| link.label.clone()); + merged.model = merged.model.take().or_else(|| link.model.clone()); + merged.task_id = merged.task_id.take().or_else(|| link.task_id.clone()); + merged.child_session_id = merged + .child_session_id + .take() + .or_else(|| link.child_session_id.clone()); + } + links +} + +/// A child log path as a lookup key: `/`-separated, no leading `./`. +pub(crate) fn normalize_link_path(path: &str) -> String { + path.replace('\\', "/").trim_start_matches("./").to_string() +} + +/// Whether a transcript is a subagent's own log by where it sits: +/// `…/subagent//session.jsonl`. A session's transcript sits under its +/// date directories (`…/YYYY/MM/DD//session.jsonl`), never there. +pub(crate) fn is_subagent_log(path: &Path) -> bool { + path.parent() + .and_then(Path::parent) + .and_then(Path::file_name) + .is_some_and(|name| name == SUBAGENT_DIR) +} + +/// Tools that write a file. +pub(crate) fn is_file_edit_tool(name: &str) -> bool { + matches!(name, "write_file" | "edit_file") +} + +/// Tools that start another agent. +pub(crate) fn is_subagent_tool(name: &str) -> bool { + name == "subagent_spawn" +} + +/// What a tool call acts on: the file, the command, the pattern. +pub(crate) fn pick_tool_target(name: &str, arguments: &Value) -> Option { + let object = arguments.as_object()?; + let get = |key: &str| { + object + .get(key) + .and_then(Value::as_str) + .map(str::trim) + .filter(|value| !value.is_empty()) + .map(str::to_string) + }; + match name { + "bash" => get("command"), + "bash_input" => get("chars"), + "search" => get("pattern"), + "web_search" => get("query"), + "subagent_spawn" => get("role") + .or_else(|| get("objective")) + .or_else(|| get("prompt")), + _ => get("path") + .or_else(|| get("file_path")) + .or_else(|| get("url")) + .or_else(|| get("query")), + } +} + +/// A `bash` result is a JSON object string; its `exit_code` is the one +/// in-band failure signal Muse writes. +pub(crate) fn bash_exit_code(text: &str) -> Option { + serde_json::from_str::(text) + .ok()? + .get("exit_code") + .and_then(Value::as_i64) +} + +#[cfg(test)] +mod tests { + use super::*; + use serde_json::json; + + const SID: &str = "74747474-7474-4747-8747-747474747474"; + + fn line(sequence: i64, payload_type: &str, payload: Value) -> String { + json!({ + "schema_version": 1, + "id": format!("rec-{sequence}"), + "stream": {"kind": "session", "id": SID}, + "sequence": sequence, + "recorded_at": 1_788_223_110_000_000i64 + sequence * 1000, + "payload_type": payload_type, + "payload": payload, + }) + .to_string() + } + + fn run(sequence: i64, event: Value) -> String { + line( + sequence, + "runtime.session", + json!({"kind": "run", "run_id": "run-1", "event": event}), + ) + } + + fn metadata() -> String { + line( + 1, + METADATA_PAYLOAD_TYPE, + json!({"kind": "metadata", "record": { + "workspace_root": "/work", "model_id": "meta/muse-spark", + "provider_id": "meta", "build": {"semver": "1.3.0"}}}), + ) + } + + #[test] + fn a_file_without_metadata_is_not_a_session() { + assert!(parse_transcript(&run(2, json!({"kind": "started", "prompt": "hi"}))).is_none()); + assert!(parse_transcript("not json\n").is_none()); + } + + #[test] + fn metadata_may_follow_a_permission_frame() { + let contents = format!( + "{}\n{}\n", + r#"{"retained_frame":"session_permission_transaction","children":[]}"#, + metadata() + ); + let transcript = parse_transcript(&contents).unwrap(); + assert_eq!(transcript.metadata.session_id, SID); + assert_eq!(transcript.metadata.workspace_root.as_deref(), Some("/work")); + assert_eq!(transcript.metadata.cli_version.as_deref(), Some("1.3.0")); + assert_eq!(transcript.metadata.recorded_ms, Some(1_788_223_110_001)); + assert_eq!(transcript.unparsed_lines, 0); + } + + #[test] + fn reads_the_conversation_and_skips_task_bookkeeping() { + let contents = [ + metadata(), + run(2, json!({"kind": "started", "prompt": "fix it"})), + line( + 3, + "runtime.session", + json!({"kind": "task", "run_id": "run-1", "event": {"kind": "started", "task_id": "t"}}), + ), + run( + 4, + json!({"kind": "model_completed", "model": "meta/muse-spark", "duration_ms": 5, + "usage": {"input_tokens": 10, "output_tokens": 2}}), + ), + run( + 5, + json!({"kind": "assistant_tool_calls_committed", "message_id": "m1", + "tool_calls": [{"id": "fc_1", "call_id": "call_1", "name": "write_file", + "args": "{\"path\":\"a.rs\",\"content\":\"x\"}"}]}), + ), + line( + 6, + "tool_batch.effect.terminal", + json!({"kind": "tool_batch_effect", "record": {"call_id": "call_1", + "outcome": {"kind": "failed", "reason": "denied"}}}), + ), + run( + 7, + json!({"kind": "tool_result_batch_committed", + "results": [{"tool_call_id": "call_1", "text": "tool failed: denied"}]}), + ), + run(8, json!({"kind": "terminal", "terminal": "completed"})), + ] + .join("\n") + + "\n"; + let transcript = parse_transcript(&contents).unwrap(); + let kinds: Vec<_> = transcript + .records + .iter() + .map(|record| std::mem::discriminant(&record.event)) + .collect(); + assert_eq!(kinds.len(), 6, "{:#?}", transcript.records); + assert_eq!( + transcript.records[0].event, + MuseEvent::Prompt { + text: "fix it".into() + } + ); + assert_eq!(transcript.records[0].ts_ms, 1_788_223_110_002); + assert_eq!(transcript.records[0].run_id.as_deref(), Some("run-1")); + let MuseEvent::ToolCalls { calls, .. } = &transcript.records[2].event else { + panic!("expected tool calls"); + }; + assert_eq!(calls[0].call_id, "call_1"); + assert_eq!(calls[0].arguments["path"], "a.rs"); + assert_eq!( + pick_tool_target(&calls[0].name, &calls[0].arguments).as_deref(), + Some("a.rs") + ); + assert_eq!( + tool_outcomes(&transcript).get("call_1"), + Some(&("failed", Some("denied"))) + ); + assert_eq!(transcript.models(), vec!["meta/muse-spark".to_string()]); + } + + /// Tool steps log usage before their calls; prose steps after their + /// reply. Both pair with the right record, and neither leaks into the + /// next step or the next run. + #[test] + fn model_steps_pair_with_their_commit_in_either_order() { + let usage = json!({"input_tokens": 1, "output_tokens": 1}); + let calls = |id: &str| { + json!({"kind": "assistant_tool_calls_committed", + "tool_calls": [{"call_id": id, "name": "bash", "args": "{}"}]}) + }; + let results = |id: &str| { + json!({"kind": "tool_result_batch_committed", + "results": [{"tool_call_id": id, "text": "ok"}]}) + }; + let contents = [ + metadata(), + run(2, json!({"kind": "started", "prompt": "go"})), + // Step 1: usage first, then readable reasoning *and* tool calls. + run(3, json!({"kind": "model_completed", "usage": usage})), + run(4, json!({"kind": "reasoning_committed", "text": "think"})), + run(5, calls("c1")), + run(6, results("c1")), + // Step 2: usage first, tool calls. + run(7, json!({"kind": "model_completed", "usage": usage})), + run(8, calls("c2")), + run(9, results("c2")), + // Step 3: the reply first, then its usage. + run( + 10, + json!({"kind": "assistant_message_committed", "text": "done"}), + ), + run(11, json!({"kind": "model_completed", "usage": usage})), + // A second usage in the same step has no record to carry it. + run(12, json!({"kind": "model_completed", "usage": usage})), + run(13, json!({"kind": "terminal", "terminal": "completed"})), + ] + .join("\n") + + "\n"; + let transcript = parse_transcript(&contents).unwrap(); + let steps = pair_model_steps(&transcript); + let (owners, orphans) = (&steps.owners, steps.orphans); + // Record indexes: 0 prompt, 1 usage, 2 reasoning, 3 calls, 4 results, + // 5 usage, 6 calls, 7 results, 8 reply, 9 usage, 10 usage, 11 terminal. + assert_eq!( + owners.get(&2), + Some(&1), + "step 1 is owned by its first commit" + ); + assert_eq!(owners.get(&3), None, "the rest of step 1 owns nothing"); + assert_eq!( + steps.members.get(&3), + Some(&1), + "…but still belongs to step 1, whose model and finish reason it shares" + ); + assert_eq!(owners.get(&6), Some(&5)); + assert_eq!(owners.get(&8), Some(&9)); + assert_eq!(owners.len(), 3); + assert_eq!(orphans, 1); + } + + #[test] + fn an_unterminated_last_line_is_not_read_yet() { + let contents = format!( + "{}\n{}", + metadata(), + run(2, json!({"kind": "started", "prompt": "hi"})) + ); + let transcript = parse_transcript(&contents).unwrap(); + assert!(transcript.records.is_empty(), "{:?}", transcript.records); + let transcript = parse_transcript(&format!("{contents}\n")).unwrap(); + assert_eq!(transcript.records.len(), 1); + } + + #[test] + fn records_on_another_stream_are_not_the_sessions() { + let foreign = json!({ + "id": "x", "stream": {"kind": "task", "id": "task-1"}, "recorded_at": 1_788_223_110_000_000i64, + "payload_type": "runtime.session", + "payload": {"kind": "run", "event": {"kind": "started", "prompt": "Role: worker Objective: …"}}, + }) + .to_string(); + let transcript = parse_transcript(&format!("{}\n{foreign}\n", metadata())).unwrap(); + assert!(transcript.records.is_empty()); + assert_eq!(transcript.foreign_stream_records, 1); + } + + #[test] + fn unparseable_arguments_are_kept_as_the_raw_string() { + let call = + parse_tool_call(&json!({"call_id": "c", "name": "bash", "args": "{oops"})).unwrap(); + assert_eq!(call.arguments, json!("{oops")); + } + + #[test] + fn child_transcripts_are_recognized_by_their_subagent_directory() { + let root = Path::new("/data/muse/sessions"); + assert!(is_child_transcript( + &root.join("2026/09/09/p/subagent/c/session.jsonl"), + root + )); + assert!(!is_child_transcript( + &root.join("2026/09/09/p/session.jsonl"), + root + )); + } + + #[test] + fn a_subagent_log_is_recognized_by_where_it_sits() { + assert!(is_subagent_log(Path::new( + "/s/2026/09/09/p/subagent/c/session.jsonl" + ))); + assert!(!is_subagent_log(Path::new("/s/2026/09/09/p/session.jsonl"))); + } + + #[test] + fn bash_results_expose_their_exit_code() { + assert_eq!(bash_exit_code(r#"{"exit_code":2,"output":""}"#), Some(2)); + assert_eq!(bash_exit_code("plain text"), None); + } +} diff --git a/crates/ai-hist/src/ingest/tool_result_facts.rs b/crates/ai-hist/src/ingest/tool_result_facts.rs index 38b96949..5a8bec26 100644 --- a/crates/ai-hist/src/ingest/tool_result_facts.rs +++ b/crates/ai-hist/src/ingest/tool_result_facts.rs @@ -59,6 +59,9 @@ pub const ERROR_SIGNAL_EXIT_CODE: &str = "exit_code"; pub const ERROR_SIGNAL_PATCH_APPLY: &str = "patch_apply"; /// A Codex `mcp_tool_call_end` whose `result` carries `Err`. pub const ERROR_SIGNAL_MCP_ERR: &str = "mcp_err"; +/// A Muse Code `tool_batch.effect.terminal` whose `outcome.kind` is not +/// `completed`. +pub const ERROR_SIGNAL_MUSE_TOOL_OUTCOME: &str = "tool_batch.effect"; /// A harness subagent notification reporting a failed or cancelled child. pub const ERROR_SIGNAL_SUBAGENT_STATUS: &str = "subagent_status"; /// The provider's own terminal status on the tool call: an OpenCode tool @@ -354,6 +357,53 @@ pub fn codex_output_facts(output: &Value, call_id: &str) -> ToolResultFacts { facts } +/// Facts for one Muse Code tool result. +/// +/// Muse writes no error flag on the result itself. The call's outcome is a +/// separate `tool_batch.effect.terminal` record, passed here as +/// `(outcome.kind, reason)`; without one the status stays `unknown`. A `bash` +/// result that completed as a call but whose command exited non-zero is an +/// error too, the way Codex reads `exit_code`. Only a `bash` result: any +/// other tool's output is the tool's data, and a file that happens to hold +/// `{"exit_code": 1}` is not a failed read. +pub fn muse_tool_result_facts( + text: Option<&str>, + call_id: &str, + tool_name: Option<&str>, + outcome: Option<(&str, Option<&str>)>, +) -> ToolResultFacts { + let payload = text.map_or(Value::Null, |text| Value::String(text.to_string())); + let mut facts = ToolResultFacts::from_payload(&payload); + // Muse commits results in their own `tool_result_batch_committed` + // record, not inside a user message, so they are standalone outputs — a + // user turn is the prompt alone, as for Grok and OpenCode. + facts.event_source = Some(EVENT_SOURCE_FUNCTION_CALL_OUTPUT.to_string()); + facts.tool_use_id = (!call_id.is_empty()).then(|| call_id.to_string()); + let status = match outcome.map(|(kind, _)| kind) { + Some("completed") => STATUS_COMPLETED, + Some("failed") => { + facts.error_signal = Some(ERROR_SIGNAL_MUSE_TOOL_OUTCOME.to_string()); + STATUS_ERRORED + } + Some("cancelled" | "canceled" | "skipped") => { + facts.error_signal = Some(ERROR_SIGNAL_MUSE_TOOL_OUTCOME.to_string()); + STATUS_CANCELLED + } + _ => STATUS_UNKNOWN, + }; + facts.result_status = Some(status.to_string()); + if status == STATUS_COMPLETED + && tool_name == Some("bash") + && text + .and_then(crate::ingest::muse::bash_exit_code) + .is_some_and(|code| code != 0) + { + facts.result_status = Some(STATUS_ERRORED.to_string()); + facts.error_signal = Some(ERROR_SIGNAL_EXIT_CODE.to_string()); + } + facts +} + /// Facts for one Cursor `tool_result` content block. /// /// Cursor writes Claude-shaped blocks (`tool_use_id`, `content`, `is_error`) @@ -442,6 +492,20 @@ mod tests { use super::*; use serde_json::json; + /// Only a `bash` result's `exit_code` is a failure signal; any other + /// tool's output is data that may happen to contain the same key. + #[test] + fn muse_exit_codes_fail_only_bash_results() { + let text = Some(r#"{"exit_code":101}"#); + let completed = Some(("completed", None)); + let bash = muse_tool_result_facts(text, "c1", Some("bash"), completed); + assert_eq!(bash.result_status.as_deref(), Some(STATUS_ERRORED)); + assert_eq!(bash.error_signal.as_deref(), Some(ERROR_SIGNAL_EXIT_CODE)); + let read = muse_tool_result_facts(text, "c2", Some("read_file"), completed); + assert_eq!(read.result_status.as_deref(), Some(STATUS_COMPLETED)); + assert_eq!(read.error_signal, None); + } + #[test] fn stable_stringify_sorts_object_keys_and_keeps_array_order() { let value = json!({ "b": 1, "a": 2, "c": [3, { "y": 1, "x": 2 }] }); diff --git a/crates/ai-hist/src/paths.rs b/crates/ai-hist/src/paths.rs index 802a0792..79c453af 100644 --- a/crates/ai-hist/src/paths.rs +++ b/crates/ai-hist/src/paths.rs @@ -24,6 +24,21 @@ pub fn grok_home(home: &Path) -> PathBuf { env_dir("GROK_HOME").unwrap_or_else(|| home.join(".grok")) } +/// Where Muse Code keeps its session logs: `$XDG_DATA_HOME/muse/sessions`, +/// or `~/.local/share/muse/sessions` when `XDG_DATA_HOME` is unset. Muse uses +/// the same XDG layout on every platform. +pub fn muse_sessions_dir(home: &Path) -> PathBuf { + match env_dir("XDG_DATA_HOME") { + Some(data) => data.join("muse/sessions"), + None => default_muse_sessions_dir(home), + } +} + +/// [`muse_sessions_dir`] with nothing read from the environment. +pub(crate) fn default_muse_sessions_dir(home: &Path) -> PathBuf { + home.join(".local/share/muse/sessions") +} + pub fn opencode_db_path(home: &Path) -> PathBuf { env_dir("OPENCODE_DB") .unwrap_or_else(|| home.join(".local/share/opencode/opencode.db")) @@ -166,7 +181,7 @@ pub fn opencode_storage_dir(home: &Path) -> PathBuf { /// watched, or a session the sweep catalogued cannot be hydrated afterwards. /// Build it with [`ProviderRoots::from_env`] (the CLI's rules: the process /// environment's `CLAUDE_CONFIG_DIR`, `CODEX_HOME`, `GROK_HOME`, -/// `OPENCODE_DB`, `OPENCODE_STORAGE_DIR` and `TRAJECTORY_ROOT` override the +/// `XDG_DATA_HOME` (for Muse Code), `OPENCODE_DB`, `OPENCODE_STORAGE_DIR` and `TRAJECTORY_ROOT` override the /// defaults under `home`) or [`ProviderRoots::from_home`] (the defaults under /// `home`, with nothing read from the environment — what a test or an /// embedder with its own layout wants). The environment is read **once**, @@ -182,6 +197,8 @@ pub struct ProviderRoots { pub codex: PathBuf, /// Grok state root (`~/.grok`). pub grok: PathBuf, + /// Muse Code session logs (`~/.local/share/muse/sessions`). + pub muse: PathBuf, /// The OpenCode SQLite store. Unless [`Self::opencode_db_pinned`], every /// channel database beside it (`opencode-stable.db`, /// `opencode-nightly.db`, ...) is read as well. @@ -216,6 +233,7 @@ impl ProviderRoots { claude: claude_config_dir(&home), codex: codex_home(&home), grok: grok_home(&home), + muse: muse_sessions_dir(&home), opencode_db: opencode_db_path(&home), opencode_db_pinned: env_dir("OPENCODE_DB").is_some(), opencode_storage_dir: opencode_storage_dir(&home), @@ -236,6 +254,7 @@ impl ProviderRoots { claude: home.join(".claude"), codex: home.join(".codex"), grok: home.join(".grok"), + muse: default_muse_sessions_dir(&home), home, opencode_db, opencode_db_pinned: false, @@ -379,7 +398,12 @@ mod tests { ("ingest/hydrate.rs", include_str!("ingest/hydrate.rs")), ] { let production = source.split("\n#[cfg(test)]").next().unwrap(); - for literal in ["join(\".claude", "join(\".codex", "join(\".grok"] { + for literal in [ + "join(\".claude", + "join(\".codex", + "join(\".grok", + "join(\".local/share/muse", + ] { assert!( !production.contains(literal), "{name} builds a provider root directly with {literal}" diff --git a/crates/ai-hist/src/relationship_graph.rs b/crates/ai-hist/src/relationship_graph.rs index 834c906c..9560278b 100644 --- a/crates/ai-hist/src/relationship_graph.rs +++ b/crates/ai-hist/src/relationship_graph.rs @@ -295,6 +295,11 @@ pub fn relationship_capabilities(source: &str) -> RelationshipCapabilities { // the normal sessions tree. The id is never taken from the file name, // so an entry without one stays unlinked evidence. "grok" => ("sometimes", true, true, true), + // A Muse subagent writes its own `subagent//session.jsonl`, whose + // metadata record names the child session, so a linked child's + // identity is always its own. The parent's `task_stream_linked` + // names its role, label and model. + "muse" => ("always", true, true, true), _ => ("never", false, false, false), }; RelationshipCapabilities { diff --git a/crates/ai-hist/src/session_store.rs b/crates/ai-hist/src/session_store.rs index 97ad6487..d6fa24da 100644 --- a/crates/ai-hist/src/session_store.rs +++ b/crates/ai-hist/src/session_store.rs @@ -317,6 +317,7 @@ pub enum Source { Trajectory, #[serde(rename = "opencode")] OpenCode, + Muse, } impl Source { @@ -335,6 +336,7 @@ impl Source { Self::Relay, Self::Trajectory, Self::OpenCode, + Self::Muse, ]; /// Canonical lowercase identifier stored in the ledger. @@ -347,6 +349,7 @@ impl Source { Self::Relay => "relay", Self::Trajectory => "trajectory", Self::OpenCode => "opencode", + Self::Muse => "muse", } } @@ -383,7 +386,7 @@ impl Source { // write them are the ones an embedder needs listed. if matches!( self, - Self::Claude | Self::Codex | Self::Grok | Self::OpenCode + Self::Claude | Self::Codex | Self::Grok | Self::OpenCode | Self::Muse ) && !evidence_kinds.contains(&EvidenceKind::SessionMarker) { evidence_kinds.push(EvidenceKind::SessionMarker); @@ -403,6 +406,9 @@ impl Source { // Cursor: the message `id` when the build wrote one, else // `cursor:{record offset}`. Self::Cursor => MessageIdOrigin::Mixed, + // Muse: the record's own envelope `id` for prose and + // lifecycle, `tool:{call id}` / `result:{call id}` for tools. + Self::Muse => MessageIdOrigin::Mixed, Self::Relay | Self::Trajectory => MessageIdOrigin::None, }, hydrates_by_path: HOOK_HARNESSES.contains(&name), @@ -2691,7 +2697,8 @@ mod tests { | Source::Grok | Source::Relay | Source::Trajectory - | Source::OpenCode => Source::ALL.contains(source), + | Source::OpenCode + | Source::Muse => Source::ALL.contains(source), }; assert!(listed, "{source:?} is missing from Source::ALL"); } diff --git a/crates/ai-hist/src/store.rs b/crates/ai-hist/src/store.rs index d6230cea..b64f44d4 100644 --- a/crates/ai-hist/src/store.rs +++ b/crates/ai-hist/src/store.rs @@ -30,6 +30,7 @@ pub const SOURCE_CHOICES: &[&str] = &[ "relay", "trajectory", "opencode", + "muse", ]; #[derive(Debug, Clone, Serialize, Deserialize, PartialEq, Eq)] @@ -4322,6 +4323,12 @@ pub fn resume_command(entry: &HistoryEntry) -> Option { || format!("grok resume {}", shell_quote(sid)), |p| format!("cd {} && grok resume {}", shell_quote(p), shell_quote(sid)), )), + // `muse resume ` finds the session by id; the `cd` puts the + // resumed agent back in the workspace it recorded. + "muse" => Some(entry.project.as_ref().map_or_else( + || format!("muse resume {}", shell_quote(sid)), + |p| format!("cd {} && muse resume {}", shell_quote(p), shell_quote(sid)), + )), _ => None, } } diff --git a/crates/ai-hist/src/usage.rs b/crates/ai-hist/src/usage.rs index 5277b9ac..6528fe7e 100644 --- a/crates/ai-hist/src/usage.rs +++ b/crates/ai-hist/src/usage.rs @@ -356,7 +356,7 @@ impl std::error::Error for UsageError {} /// Sources this build can normalize. Anything else is an explicit /// [`UsageError::UnknownSource`] rather than a silent zero. -pub const NORMALIZABLE_SOURCES: &[&str] = &["claude", "codex", "grok"]; +pub const NORMALIZABLE_SOURCES: &[&str] = &["claude", "codex", "grok", "muse"]; /// Whether one model request can be spread across several stored records for /// this source. @@ -384,6 +384,10 @@ pub fn source_accounting(source: &str) -> Option { // that turn's last assistant event. The context-window snapshot // stored beside it is never normalized. "grok" => Some(UsageAccounting::PerRequest), + // Muse writes one `model_completed` usage object per model step, and + // the parser stores it once, on the first assistant row that step + // committed. + "muse" => Some(UsageAccounting::PerRequest), _ => None, } } @@ -424,7 +428,13 @@ pub fn normalize_usage( let mut usage = NormalizedUsage::empty(accounting); let output = counter("output_tokens")?; - let reasoning = counter("reasoning_output_tokens")?; + // Muse names the reasoning counter `reasoning_tokens`; Codex spells it + // `reasoning_output_tokens`. Either way it is a subset of output. + let reasoning = counter(if source == "muse" { + "reasoning_tokens" + } else { + "reasoning_output_tokens" + })?; usage.output_tokens = output.unwrap_or(0); usage.reasoning_tokens = reasoning; usage.coverage.has_output_tokens = output.is_some(); @@ -457,6 +467,34 @@ pub fn normalize_usage( usage.coverage.has_cache_read_tokens = cached.is_some(); usage.coverage.has_cache_write_tokens = cache_write.is_some(); } + "muse" => { + // Responses-shaped: `input_tokens` includes the cached prefix, + // which Muse reports as `cache_read_tokens` (and again as + // `cached_tokens`, the same count). Input is emitted exclusive of + // it, as for Codex, so summing the categories counts each token + // once. + let input = counter("input_tokens")?; + let cache_read = match counter("cache_read_tokens")? { + Some(value) => Some(value), + None => counter("cached_tokens")?, + }; + let cache_write = counter("cache_write_tokens")?; + let inclusive = input.unwrap_or(0); + let cached_value = cache_read.unwrap_or(0); + usage.input_tokens = + inclusive + .checked_sub(cached_value) + .ok_or(UsageError::CounterRegressed { + field: "input_tokens", + value: inclusive, + subtracted: cached_value, + })?; + usage.cache_read_tokens = cached_value; + usage.cache_write_tokens = cache_write.unwrap_or(0); + usage.coverage.has_input_tokens = input.is_some(); + usage.coverage.has_cache_read_tokens = cache_read.is_some(); + usage.coverage.has_cache_write_tokens = cache_write.is_some(); + } "claude" => { // Claude's native `message.usage` already excludes cache reads // and writes from `input_tokens`; subtracting them again would @@ -1170,6 +1208,41 @@ mod tests { assert_eq!(usage.reported_cost_usd, None); } + /// Muse's `model_completed` usage, as the real CLI writes it: input + /// inclusive of the cached prefix, reported under two keys. + #[test] + fn muse_input_is_made_cache_exclusive_and_counted_once() { + let usage = normalize_usage( + "muse", + &json!({ + "input_tokens": 26964, + "output_tokens": 379, + "cached_tokens": 5105, + "cache_read_tokens": 5105, + "cache_write_tokens": 0, + "reasoning_tokens": 278, + }), + ) + .unwrap() + .unwrap(); + assert_eq!(usage.input_tokens, 21859); + assert_eq!(usage.cache_read_tokens, 5105); + assert_eq!(usage.cache_write_tokens, 0); + assert_eq!(usage.output_tokens, 379); + assert_eq!(usage.reasoning_tokens, Some(278)); + assert_eq!(usage.accounting, UsageAccounting::PerRequest); + + // An older record that spells the cache read only as `cached_tokens`. + let legacy = normalize_usage( + "muse", + &json!({"input_tokens": 10, "output_tokens": 1, "cached_tokens": 4}), + ) + .unwrap() + .unwrap(); + assert_eq!(legacy.input_tokens, 6); + assert_eq!(legacy.cache_read_tokens, 4); + } + #[test] fn codex_input_is_made_cache_exclusive() { let usage = codex(json!({ diff --git a/crates/ai-hist/tests/fixture_corpus.rs b/crates/ai-hist/tests/fixture_corpus.rs index 870ea4c1..57e17a4f 100644 --- a/crates/ai-hist/tests/fixture_corpus.rs +++ b/crates/ai-hist/tests/fixture_corpus.rs @@ -63,6 +63,9 @@ enum Origin { Burn, /// Authored here, for a log shape burn's corpus does not cover. RelayHistory, + /// Derived from `xhluca/session-migrate`'s native corpus (MIT): a + /// transcript the real harness CLI wrote, kept verbatim line by line. + SessionMigrate, } struct Fixture { @@ -637,6 +640,23 @@ const CORPUS: &[Fixture] = &[ files: &["grok/events-session"], quirk: "documented Grok Build layout: `chat_history.jsonl` with `tool_calls[]`, ACP `updates.jsonl` with real `agentTimestampMs` times, `compaction_checkpoints/`, `subagents/`, `signals.json` and `prompt_context.json`", }, + // -- muse -------------------------------------------------------------- + Fixture { + source: "muse", + name: "cli-capture", + layout: Layout::HomeTree, + origin: Origin::SessionMigrate, + files: &["muse/cli-capture"], + quirk: "a transcript the real `muse` CLI (0.2.1) wrote, trimmed to its conversation, tool and lifecycle records: three runs across two resumes, `read_file` calls with one failed outcome, per-step `model_completed` usage, and mirrored reminder task records", + }, + Fixture { + source: "muse", + name: "tools-session", + layout: Layout::HomeTree, + origin: Origin::RelayHistory, + files: &["muse/tools-session"], + quirk: "authored from the documented shape: a permission frame before the metadata, encrypted and readable reasoning, `edit_file`/`write_file` edits, a `bash` result that exits 101, a mirrored subagent task stream, a mid-session model switch, and `subagent/` logs — a worker with its own nested child, and a reminder — linked as delegated children rather than catalogued as sessions", + }, // -- opencode ---------------------------------------------------------- Fixture { source: "opencode", @@ -878,6 +898,9 @@ fn capture(fixture: &Fixture, home: &Path) -> Value { std::env::set_var("USERPROFILE", home); std::env::set_var("OPENCODE_DB", &opencode_db); std::env::remove_var("AI_HIST_DB"); + // Muse Code's root follows `XDG_DATA_HOME`; the fixture's own `HOME` + // layout has to win over whatever the machine running the tests sets. + std::env::remove_var("XDG_DATA_HOME"); let db = home.join("ai-history.db"); let mut notes: Vec = Vec::new(); @@ -2511,3 +2534,150 @@ fn grok_events_and_real_timestamps_reach_session_events() { "prompt timestamps come from the record, not from `created_at + index`: {prompts:?}" ); } + +/// A transcript the real Muse CLI wrote: every prompt is a history row at the +/// microsecond time Muse recorded, every read is a tool call joined to its +/// result by call id, and the one call whose `tool_batch.effect.terminal` +/// failed is the one marked as an error. +#[test] +fn muse_cli_capture_reaches_history_tools_and_recorded_times() { + let prompts = rows("muse/cli-capture", "history"); + assert_eq!(prompts.len(), 3, "{prompts:?}"); + assert_eq!( + field(&prompts[0], "timestamp_ms").as_i64(), + Some(1_788_223_110_680), + "the prompt's own `recorded_at`, in milliseconds: {prompts:?}" + ); + let calls = rows("muse/cli-capture", "tool_calls"); + let failed: Vec<&str> = calls + .iter() + .filter(|call| field(call, "is_error").as_i64() == Some(1)) + .map(|call| text(call, "tool_use_id")) + .collect(); + assert_eq!(failed, vec!["call_muse_missing"], "{calls:?}"); + let events = rows("muse/cli-capture", "session_events"); + assert!( + events + .iter() + .filter(|event| text(event, "role") == "user") + .all(|event| !text(event, "text").starts_with("Role:")), + "a mirrored task stream is never a prompt: {events:?}" + ); + // Tool steps log `model_completed` before their calls and prose steps + // after their reply; every one of the six steps keeps its usage. + let with_usage: Vec<&str> = events + .iter() + .filter(|event| !field(event, "token_json").is_null()) + .map(|event| text(event, "kind")) + .collect(); + assert_eq!( + with_usage, + vec!["tool_use", "tool_use", "tool_use", "text", "text", "text"], + "{events:?}" + ); +} + +/// The authored Muse session: edits reach `file_edits`, a completed `bash` +/// call whose command exited non-zero is an error, usage lands once per model +/// step, and every `subagent/` log — a worker, the worker's own child and a +/// reminder — is linked as a `delegated` child under the id its own metadata +/// names, without becoming a catalog row or a typed prompt. +#[test] +fn muse_tools_session_records_edits_errors_usage_and_linked_subagents() { + const PARENT: &str = "muse-a001"; + const WORKER: &str = "muse-c001"; + const NESTED: &str = "muse-c002"; + const REMINDER: &str = "muse-e001"; + let edits = rows("muse/tools-session", "file_edits"); + let mut paths: Vec<&str> = edits.iter().map(|edit| text(edit, "file_path")).collect(); + paths.sort_unstable(); + assert_eq!(paths, vec!["src/http.rs", "tests/retry.rs"], "{edits:?}"); + let calls = rows("muse/tools-session", "tool_calls"); + let bash = calls + .iter() + .find(|call| text(call, "tool_use_id") == "call_bash") + .expect("bash call"); + assert_eq!(field(bash, "is_error").as_i64(), Some(1), "{bash:?}"); + let events = rows("muse/tools-session", "session_events"); + let parent_usage_rows = events + .iter() + .filter(|event| text(event, "session_id") == PARENT) + .filter(|event| !field(event, "token_json").is_null()) + .count(); + assert_eq!( + parent_usage_rows, 4, + "one row per model_completed step: {events:?}" + ); + // Each step's usage sits on that step's first record: step 1's on its + // readable reasoning (not on the tool calls the same step committed), + // step 2's on its first call, the replies' on the replies. + let owners: Vec<(String, i64)> = events + .iter() + .filter(|event| text(event, "session_id") == PARENT) + .filter_map(|event| { + let usage: serde_json::Value = + serde_json::from_str(field(event, "token_json").as_str()?).ok()?; + Some(( + text(event, "event_uid").to_string(), + usage["input_tokens"].as_i64()?, + )) + }) + .collect(); + assert_eq!( + owners, + vec![ + ("a001-rec-006".to_string(), 1200), + ("tool:call_edit".to_string(), 1500), + ("a001-rec-020".to_string(), 1800), + ("a001-rec-029".to_string(), 2000), + ] + ); + + let sessions = rows("muse/tools-session", "sessions"); + assert_eq!( + sessions.len(), + 1, + "children are not catalog rows: {sessions:?}" + ); + assert_eq!( + text(&sessions[0], "models_json"), + r#"["meta/muse-spark-1.3","meta/muse-spark-1.3-contributor"]"# + ); + let history = rows("muse/tools-session", "history"); + assert!( + history.iter().all(|row| text(row, "session_id") == PARENT), + "a child's objective is not a typed prompt: {history:?}" + ); + + let edges = rows("muse/tools-session", "session_relationships"); + let edge = |child: &str| { + edges + .iter() + .find(|edge| text(edge, "child_session_id") == child) + .unwrap_or_else(|| panic!("no edge to {child}: {edges:?}")) + }; + for (child, parent, agent_type, depth) in [ + (WORKER, PARENT, "worker", 1), + (REMINDER, PARENT, "reminder", 1), + (NESTED, WORKER, "worker", 2), + ] { + let edge = edge(child); + assert_eq!(text(edge, "parent_session_id"), parent, "{edge:?}"); + assert_eq!(text(edge, "relationship"), "delegated", "{edge:?}"); + assert_eq!(text(edge, "child_agent_type"), agent_type, "{edge:?}"); + assert_eq!(field(edge, "spawn_depth").as_i64(), Some(depth), "{edge:?}"); + assert_eq!( + field(edge, "child_has_events").as_i64(), + Some(1), + "{edge:?}" + ); + assert!( + events + .iter() + .any(|event| text(event, "session_id") == child), + "the child's events are stored under its own id: {child}" + ); + } + assert_eq!(text(edge(NESTED), "child_model"), "meta/muse-glimmer-30b"); + assert_eq!(text(edge(WORKER), "child_agent_name"), "reviewer"); +} diff --git a/crates/ai-hist/tests/fixtures/README.md b/crates/ai-hist/tests/fixtures/README.md index 8b0eed9a..c94d9f0b 100644 --- a/crates/ai-hist/tests/fixtures/README.md +++ b/crates/ai-hist/tests/fixtures/README.md @@ -44,6 +44,15 @@ and the raw-fact assertions among them are ported into `fixture_corpus.rs`. Everything marked **relayhistory** was authored here, for a log shape burn's corpus does not cover. +The fixture marked **sessionmigrate** is derived from the native corpus in +[`xhluca/session-migrate`](https://github.com/xhluca/session-migrate) +(`tests/native_corpus/v1/sources/muse/0.2.1/portable-rich`), licensed **MIT**. +That transcript was written by the real Muse Code 0.2.1 CLI against a +credential-free loopback provider and sanitized upstream (system +instructions and capture paths). Here it is trimmed to the records a parser +reads plus a sample of the rest; every kept line is byte-identical to the +upstream file. + ## Determinism Every staged file's mtime is pinned (base `2026-04-20T00:00:00Z`, plus one @@ -167,6 +176,13 @@ source without deciding which one fails the test. | `grok/full-session` | relayhistory | `grok/full-session` | older Claude-shaped grok directory: per-record timestamps, `tool_use` blocks in `content`, `updates.jsonl` as `file_changed` rows, plus `prompt_context.json`, `signals.json` and `subagents/` | | `grok/events-session` | relayhistory | `grok/events-session` | documented Grok Build layout: `chat_history.jsonl` with `tool_calls[]`, ACP `updates.jsonl` with real `agentTimestampMs` times, `compaction_checkpoints/`, `subagents/`, `signals.json` and `prompt_context.json` | +### `muse` + +| Fixture | Origin | Corpus files | Quirk it encodes | +| --- | --- | --- | --- | +| `muse/cli-capture` | sessionmigrate | `muse/cli-capture` | a transcript the real `muse` CLI (0.2.1) wrote, trimmed to its conversation, tool and lifecycle records: three runs across two resumes, `read_file` calls with one failed outcome, per-step `model_completed` usage, and mirrored reminder task records | +| `muse/tools-session` | relayhistory | `muse/tools-session` | authored from the documented shape: a permission frame before the metadata, encrypted and readable reasoning, `edit_file`/`write_file` edits, a `bash` result that exits 101, a mirrored subagent task stream, a mid-session model switch, and `subagent/` logs — a worker with its own nested child, and a reminder — linked as delegated children rather than catalogued as sessions | + ### `opencode` | Fixture | Origin | Corpus files | Quirk it encodes | diff --git a/crates/ai-hist/tests/fixtures/muse/cli-capture/.local/share/muse/sessions/2026/09/01/74747474-7474-4747-8747-747474747474/session.jsonl b/crates/ai-hist/tests/fixtures/muse/cli-capture/.local/share/muse/sessions/2026/09/01/74747474-7474-4747-8747-747474747474/session.jsonl new file mode 100644 index 00000000..220612ae --- /dev/null +++ b/crates/ai-hist/tests/fixtures/muse/cli-capture/.local/share/muse/sessions/2026/09/01/74747474-7474-4747-8747-747474747474/session.jsonl @@ -0,0 +1,46 @@ +{"schema_version":1,"id":"7016ff17-1131-49a4-860c-e02f9734d074","stream":{"kind":"session","id":"74747474-7474-4747-8747-747474747474"},"sequence":1,"recorded_at":1788223110587169,"record_type":"event","durability":"durable","causation_id":null,"payload_type":"runtime.session.metadata","payload_schema_version":1,"payload":{"kind":"metadata","record":{"build":{"semver":"0.2.1","sha":"b3170a534f"},"model_id":"meta/muse-glimmer-30b","provider_id":"meta","tool_surface_version":"2","web_search_mode":"client","workspace_root":"/fixture/work"}}} +{"schema_version":1,"id":"b4818b8e-d40f-4b81-a13e-4157ab0f6761","stream":{"kind":"session","id":"74747474-7474-4747-8747-747474747474"},"sequence":2,"recorded_at":1788223110589012,"record_type":"event","durability":"durable","causation_id":null,"payload_type":"runtime.session.route_facts","payload_schema_version":2,"payload":{"kind":"route_facts","record":{"cwd":"/fixture/work","pid":3392377,"terminal_title_identity":"747474747474"}}} +{"schema_version":1,"id":"8b83613d-78d1-4298-b2a1-8181328e6ec9","stream":{"kind":"session","id":"74747474-7474-4747-8747-747474747474"},"sequence":3,"recorded_at":1788223110669852,"record_type":"event","durability":"durable","causation_id":null,"payload_type":"session.opened.observed","payload_schema_version":1,"payload":{"kind":"session_opened","record":{"data_dir_free_bytes":2217422848,"resume":false,"schema_version":1,"security_mode":"yolo","session_id":"74747474-7474-4747-8747-747474747474","workspace_free_bytes":2217422848}}} +{"schema_version":1,"id":"e3e35fae-9c8c-40a9-b961-bc81ff200b79","stream":{"kind":"session","id":"74747474-7474-4747-8747-747474747474"},"sequence":10,"recorded_at":1788223110680073,"record_type":"event","durability":"durable","causation_id":null,"payload_type":"runtime.session","payload_schema_version":1,"payload":{"event":{"kind":"started","prompt":"SM_CORPUS_7319 Inspect timeline.py and CORPUS_NOTE.txt with the native file tool. Then try missing-corpus-file.txt so the native log includes a failed tool result. Explain the interval boundary and quote the note marker."},"kind":"run","run_id":"ffa8d311-da04-4cfc-b100-89c18a58f823","source_run_record_id":"5b4e9462-2d01-4fc3-a92e-905ecc88ef05","source_run_record_sequence":1}} +{"schema_version":1,"id":"cd4bd2e0-5e0d-4905-88d1-e2699728fec7","stream":{"kind":"session","id":"74747474-7474-4747-8747-747474747474"},"sequence":17,"recorded_at":1788223110685440,"record_type":"event","durability":"durable","causation_id":null,"payload_type":"runtime.session","payload_schema_version":1,"payload":{"event":{"display":{"label":"plugin:tbh-reminders:skill-reminder reminder","model":"same-as-main","path":"subagent/962066db-24fe-4e3e-9031-be5b7139678a/session.jsonl","role":"reminder"},"execution_mode":"background","kind":"task_stream_linked","task_id":"4a6c4ebc-aac7-4cbf-88ff-055f1ca8b0be","task_stream":{"id":"4a6c4ebc-aac7-4cbf-88ff-055f1ca8b0be","kind":"task"}},"kind":"run","run_id":"ffa8d311-da04-4cfc-b100-89c18a58f823","source_run_record_id":"91eea925-a942-436a-9972-d8e24b2d587e","source_run_record_sequence":7}} +{"schema_version":1,"id":"a3d106c8-7cdd-4401-8003-8c2020331d62","stream":{"kind":"session","id":"74747474-7474-4747-8747-747474747474"},"sequence":18,"recorded_at":1788223110685796,"record_type":"event","durability":"durable","causation_id":null,"payload_type":"runtime.session","payload_schema_version":1,"payload":{"event":{"kind":"proposed","task_id":"4a6c4ebc-aac7-4cbf-88ff-055f1ca8b0be","task_kind":"reminder.agent.plugin:tbh-reminders:skill-reminder"},"kind":"task","run_id":"ffa8d311-da04-4cfc-b100-89c18a58f823","task_id":"4a6c4ebc-aac7-4cbf-88ff-055f1ca8b0be"}} +{"schema_version":1,"id":"2b803dd4-fe46-44c1-912b-37617f1189e4","stream":{"kind":"session","id":"74747474-7474-4747-8747-747474747474"},"sequence":19,"recorded_at":1788223110686044,"record_type":"event","durability":"durable","causation_id":null,"payload_type":"runtime.session","payload_schema_version":1,"payload":{"event":{"kind":"accepted","task_id":"4a6c4ebc-aac7-4cbf-88ff-055f1ca8b0be"},"kind":"task","run_id":"ffa8d311-da04-4cfc-b100-89c18a58f823","task_id":"4a6c4ebc-aac7-4cbf-88ff-055f1ca8b0be"}} +{"schema_version":1,"id":"7b501901-d80a-4612-8309-c1b59d101bd9","stream":{"kind":"session","id":"74747474-7474-4747-8747-747474747474"},"sequence":20,"recorded_at":1788223110686305,"record_type":"event","durability":"durable","causation_id":null,"payload_type":"runtime.session","payload_schema_version":1,"payload":{"event":{"child_session_id":"962066db-24fe-4e3e-9031-be5b7139678a","child_session_log_path":"subagent/962066db-24fe-4e3e-9031-be5b7139678a/session.jsonl","generation_id":1,"kind":"memory_reminder_child_session_linked","parent_run_id":"ffa8d311-da04-4cfc-b100-89c18a58f823","parent_session_id":"74747474-7474-4747-8747-747474747474","reminder_agent_id":"plugin:tbh-reminders:skill-reminder","task_id":"4a6c4ebc-aac7-4cbf-88ff-055f1ca8b0be","task_stream":{"id":"4a6c4ebc-aac7-4cbf-88ff-055f1ca8b0be","kind":"task"}},"kind":"run","run_id":"ffa8d311-da04-4cfc-b100-89c18a58f823","source_run_record_id":"ff53d032-2fe6-4875-a5b7-a810af1018b4","source_run_record_sequence":8}} +{"schema_version":1,"id":"92a27a4d-2ed8-4354-bd59-189423374b6a","stream":{"kind":"session","id":"74747474-7474-4747-8747-747474747474"},"sequence":21,"recorded_at":1788223110686582,"record_type":"event","durability":"durable","causation_id":null,"payload_type":"runtime.session","payload_schema_version":1,"payload":{"event":{"kind":"started","task_id":"4a6c4ebc-aac7-4cbf-88ff-055f1ca8b0be"},"kind":"task","run_id":"ffa8d311-da04-4cfc-b100-89c18a58f823","task_id":"4a6c4ebc-aac7-4cbf-88ff-055f1ca8b0be"}} +{"schema_version":1,"id":"6cf77b19-1934-47cb-a5d4-630195ce4613","stream":{"kind":"session","id":"74747474-7474-4747-8747-747474747474"},"sequence":22,"recorded_at":1788223110686971,"record_type":"event","durability":"durable","causation_id":null,"payload_type":"runtime.session","payload_schema_version":1,"payload":{"event":{"display":{"label":"plugin:tbh-reminders:scope-reminder reminder","model":"same-as-main","path":"subagent/cbfd5023-3d9e-44e2-ba65-2feea5d64182/session.jsonl","role":"reminder"},"execution_mode":"background","kind":"task_stream_linked","task_id":"71860a9b-1b61-42cd-9a5d-9eb8c904e1da","task_stream":{"id":"71860a9b-1b61-42cd-9a5d-9eb8c904e1da","kind":"task"}},"kind":"run","run_id":"ffa8d311-da04-4cfc-b100-89c18a58f823","source_run_record_id":"43797131-a514-468c-bdea-15b18fc7d8bc","source_run_record_sequence":9}} +{"schema_version":1,"id":"106ac6fc-b12c-493d-b8b2-10e7efb4de19","stream":{"kind":"session","id":"74747474-7474-4747-8747-747474747474"},"sequence":25,"recorded_at":1788223110688365,"record_type":"event","durability":"durable","causation_id":null,"payload_type":"runtime.session","payload_schema_version":1,"payload":{"event":{"child_session_id":"cbfd5023-3d9e-44e2-ba65-2feea5d64182","child_session_log_path":"subagent/cbfd5023-3d9e-44e2-ba65-2feea5d64182/session.jsonl","generation_id":1,"kind":"memory_reminder_child_session_linked","parent_run_id":"ffa8d311-da04-4cfc-b100-89c18a58f823","parent_session_id":"74747474-7474-4747-8747-747474747474","reminder_agent_id":"plugin:tbh-reminders:scope-reminder","task_id":"71860a9b-1b61-42cd-9a5d-9eb8c904e1da","task_stream":{"id":"71860a9b-1b61-42cd-9a5d-9eb8c904e1da","kind":"task"}},"kind":"run","run_id":"ffa8d311-da04-4cfc-b100-89c18a58f823","source_run_record_id":"c2f6c229-274b-4050-833e-6130bbd429df","source_run_record_sequence":10}} +{"schema_version":1,"id":"59809fb7-64dc-409c-bc85-de8c3ded5947","stream":{"kind":"session","id":"74747474-7474-4747-8747-747474747474"},"sequence":39,"recorded_at":1788223110715023,"record_type":"event","durability":"durable","causation_id":null,"payload_type":"runtime.session","payload_schema_version":1,"payload":{"event":{"duration_ms":22,"finish_reason":"tool_calls","kind":"model_completed","model":"meta/muse-glimmer-30b","usage":{"cache_read_tokens":0,"cache_write_tokens":0,"cached_tokens":0,"input_tokens":32,"output_tokens":8,"reasoning_tokens":0}},"kind":"run","run_id":"ffa8d311-da04-4cfc-b100-89c18a58f823","source_run_record_id":"1b5fee19-e670-4f75-8ed0-4a93e7164d53","source_run_record_sequence":16}} +{"schema_version":1,"id":"b2b45785-4827-4593-aac7-17692fa8c621","stream":{"kind":"session","id":"74747474-7474-4747-8747-747474747474"},"sequence":41,"recorded_at":1788223110715815,"record_type":"event","durability":"durable","causation_id":null,"payload_type":"runtime.session","payload_schema_version":1,"payload":{"event":{"kind":"assistant_tool_calls_committed","message_id":"80749233-420e-4c1c-b567-a9047d47c4be","response_id":"resp_call_muse_timeline","tool_calls":[{"args":"{\"path\":\"timeline.py\"}","call_id":"call_muse_timeline","id":"fc_call_muse_timeline","name":"read_file"}]},"kind":"run","run_id":"ffa8d311-da04-4cfc-b100-89c18a58f823","source_run_record_id":"1ac33300-ebfe-451e-886e-13a798999c26","source_run_record_sequence":17}} +{"schema_version":1,"id":"f1b6e377-4ba2-4c76-9a0b-c5d01921d7db","stream":{"kind":"session","id":"74747474-7474-4747-8747-747474747474"},"sequence":48,"recorded_at":1788223110725128,"record_type":"event","durability":"durable","causation_id":null,"payload_type":"tool_batch.effect.started","payload_schema_version":1,"payload":{"kind":"tool_batch_effect","record":{"call_id":"call_muse_timeline","effect_id":"d61d8e5e-6134-4f02-a7a7-ff2b485e5db9","kind":"started","model_call_index":0,"parallel_profile":{"kind":"file_read","subject":"workspace:timeline.py"},"task_id":"d61d8e5e-6134-4f02-a7a7-ff2b485e5db9","task_stream":{"id":"d61d8e5e-6134-4f02-a7a7-ff2b485e5db9","kind":"task"},"tool_name":"read_file"},"run_id":"ffa8d311-da04-4cfc-b100-89c18a58f823"}} +{"schema_version":1,"id":"989fcf96-e492-4253-937a-1e7dfdd94dc9","stream":{"kind":"session","id":"74747474-7474-4747-8747-747474747474"},"sequence":49,"recorded_at":1788223110734322,"record_type":"event","durability":"durable","causation_id":null,"payload_type":"tool_batch.effect.terminal","payload_schema_version":1,"payload":{"kind":"tool_batch_effect","record":{"call_id":"call_muse_timeline","effect_id":"d61d8e5e-6134-4f02-a7a7-ff2b485e5db9","kind":"terminal","model_call_index":0,"outcome":{"kind":"completed","output_ref_count":0,"task_completion":{"kind":"complete"}},"task_id":"d61d8e5e-6134-4f02-a7a7-ff2b485e5db9","task_stream":{"id":"d61d8e5e-6134-4f02-a7a7-ff2b485e5db9","kind":"task"}},"run_id":"ffa8d311-da04-4cfc-b100-89c18a58f823"}} +{"schema_version":1,"id":"ed665062-ac23-45dc-bc48-c021bc62e845","stream":{"kind":"session","id":"74747474-7474-4747-8747-747474747474"},"sequence":53,"recorded_at":1788223110736076,"record_type":"event","durability":"durable","causation_id":null,"payload_type":"runtime.session","payload_schema_version":1,"payload":{"event":{"batch_id":"80749233-420e-4c1c-b567-a9047d47c4be","kind":"tool_result_batch_committed","results":[{"text":"Read text file `timeline.py`.\n1|from dataclasses import dataclass\n2|\n3|\n4|@dataclass(frozen=True)\n5|class Event:\n6| start_ms: int\n7| end_ms: int\n8|\n9|\n10|def touches(previous: Event, current: Event) -> bool:\n11| \"\"\"Return whether two closed intervals overlap or touch.\"\"\"\n12| return current.start_ms < previous.end_ms","tool_call_id":"call_muse_timeline","tool_call_index":0}]},"kind":"run","run_id":"ffa8d311-da04-4cfc-b100-89c18a58f823"}} +{"schema_version":1,"id":"87573844-f5ec-4127-a6b3-9a9668d35351","stream":{"kind":"session","id":"74747474-7474-4747-8747-747474747474"},"sequence":72,"recorded_at":1788223110756121,"record_type":"event","durability":"durable","causation_id":null,"payload_type":"runtime.session","payload_schema_version":1,"payload":{"event":{"duration_ms":13,"finish_reason":"tool_calls","kind":"model_completed","model":"meta/muse-glimmer-30b","usage":{"cache_read_tokens":0,"cache_write_tokens":0,"cached_tokens":0,"input_tokens":32,"output_tokens":8,"reasoning_tokens":0}},"kind":"run","run_id":"ffa8d311-da04-4cfc-b100-89c18a58f823","source_run_record_id":"f793d0ef-2591-423a-b403-861519ab7e6c","source_run_record_sequence":28}} +{"schema_version":1,"id":"c6176039-f878-4842-bc83-d47e1bc4b2e3","stream":{"kind":"session","id":"74747474-7474-4747-8747-747474747474"},"sequence":74,"recorded_at":1788223110756715,"record_type":"event","durability":"durable","causation_id":null,"payload_type":"runtime.session","payload_schema_version":1,"payload":{"event":{"kind":"assistant_tool_calls_committed","message_id":"cb7c9354-f80e-410c-8739-89807a688bed","response_id":"resp_call_muse_note","tool_calls":[{"args":"{\"path\":\"CORPUS_NOTE.txt\"}","call_id":"call_muse_note","id":"fc_call_muse_note","name":"read_file"}]},"kind":"run","run_id":"ffa8d311-da04-4cfc-b100-89c18a58f823","source_run_record_id":"ad593dc6-7c3f-4881-bc31-464b8a24cadb","source_run_record_sequence":29}} +{"schema_version":1,"id":"c26acff4-669b-456d-91b0-fa2f5cf69bb7","stream":{"kind":"session","id":"74747474-7474-4747-8747-747474747474"},"sequence":81,"recorded_at":1788223110759340,"record_type":"event","durability":"durable","causation_id":null,"payload_type":"tool_batch.effect.started","payload_schema_version":1,"payload":{"kind":"tool_batch_effect","record":{"call_id":"call_muse_note","effect_id":"f36e3f72-c5ac-4e7f-b4d3-98d660b8d37d","kind":"started","model_call_index":0,"parallel_profile":{"kind":"file_read","subject":"workspace:CORPUS_NOTE.txt"},"task_id":"f36e3f72-c5ac-4e7f-b4d3-98d660b8d37d","task_stream":{"id":"f36e3f72-c5ac-4e7f-b4d3-98d660b8d37d","kind":"task"},"tool_name":"read_file"},"run_id":"ffa8d311-da04-4cfc-b100-89c18a58f823"}} +{"schema_version":1,"id":"7f3c0eb5-3726-4c56-90cc-fb6fabac470b","stream":{"kind":"session","id":"74747474-7474-4747-8747-747474747474"},"sequence":82,"recorded_at":1788223110760000,"record_type":"event","durability":"durable","causation_id":null,"payload_type":"tool_batch.effect.terminal","payload_schema_version":1,"payload":{"kind":"tool_batch_effect","record":{"call_id":"call_muse_note","effect_id":"f36e3f72-c5ac-4e7f-b4d3-98d660b8d37d","kind":"terminal","model_call_index":0,"outcome":{"kind":"completed","output_ref_count":0,"task_completion":{"kind":"complete"}},"task_id":"f36e3f72-c5ac-4e7f-b4d3-98d660b8d37d","task_stream":{"id":"f36e3f72-c5ac-4e7f-b4d3-98d660b8d37d","kind":"task"}},"run_id":"ffa8d311-da04-4cfc-b100-89c18a58f823"}} +{"schema_version":1,"id":"18d71b3c-ef85-47d3-a58f-a9ea2ed1aef3","stream":{"kind":"session","id":"74747474-7474-4747-8747-747474747474"},"sequence":86,"recorded_at":1788223110761286,"record_type":"event","durability":"durable","causation_id":null,"payload_type":"runtime.session","payload_schema_version":1,"payload":{"event":{"batch_id":"cb7c9354-f80e-410c-8739-89807a688bed","kind":"tool_result_batch_committed","results":[{"text":"Read text file `CORPUS_NOTE.txt`.\n1|Native corpus file marker: COPPER_4821\n2|\n3|This file belongs to a disposable session-migrate test workspace. It contains\n4|no credentials, user data, or repository source.","tool_call_id":"call_muse_note","tool_call_index":0}]},"kind":"run","run_id":"ffa8d311-da04-4cfc-b100-89c18a58f823"}} +{"schema_version":1,"id":"941e480a-c553-4976-8e0b-03614d4b4607","stream":{"kind":"session","id":"74747474-7474-4747-8747-747474747474"},"sequence":115,"recorded_at":1788223110798214,"record_type":"event","durability":"durable","causation_id":null,"payload_type":"runtime.session","payload_schema_version":1,"payload":{"event":{"duration_ms":26,"finish_reason":"tool_calls","kind":"model_completed","model":"meta/muse-glimmer-30b","usage":{"cache_read_tokens":0,"cache_write_tokens":0,"cached_tokens":0,"input_tokens":32,"output_tokens":8,"reasoning_tokens":0}},"kind":"run","run_id":"ffa8d311-da04-4cfc-b100-89c18a58f823","source_run_record_id":"3b491522-5317-47aa-87e6-d4bbc73d8d4c","source_run_record_sequence":48}} +{"schema_version":1,"id":"e0ed554b-6119-45d9-9d22-584f0c720419","stream":{"kind":"session","id":"74747474-7474-4747-8747-747474747474"},"sequence":117,"recorded_at":1788223110799120,"record_type":"event","durability":"durable","causation_id":null,"payload_type":"runtime.session","payload_schema_version":1,"payload":{"event":{"kind":"assistant_tool_calls_committed","message_id":"d9a98768-7eaf-4496-b382-be2ce4614480","response_id":"resp_call_muse_missing","tool_calls":[{"args":"{\"path\":\"missing-corpus-file.txt\"}","call_id":"call_muse_missing","id":"fc_call_muse_missing","name":"read_file"}]},"kind":"run","run_id":"ffa8d311-da04-4cfc-b100-89c18a58f823","source_run_record_id":"df1f5f4a-e57e-435d-81de-5528e29c7273","source_run_record_sequence":49}} +{"schema_version":1,"id":"ead09371-c41f-4d02-b1a6-4b05a8115bcc","stream":{"kind":"session","id":"74747474-7474-4747-8747-747474747474"},"sequence":124,"recorded_at":1788223110801824,"record_type":"event","durability":"durable","causation_id":null,"payload_type":"tool_batch.effect.started","payload_schema_version":1,"payload":{"kind":"tool_batch_effect","record":{"call_id":"call_muse_missing","effect_id":"017f8c0d-b268-4d99-a73b-3391f126516c","kind":"started","model_call_index":0,"parallel_profile":{"kind":"file_read","subject":"workspace:missing-corpus-file.txt"},"task_id":"017f8c0d-b268-4d99-a73b-3391f126516c","task_stream":{"id":"017f8c0d-b268-4d99-a73b-3391f126516c","kind":"task"},"tool_name":"read_file"},"run_id":"ffa8d311-da04-4cfc-b100-89c18a58f823"}} +{"schema_version":1,"id":"00e19237-0051-4a9e-b1a2-bfd411d97d6d","stream":{"kind":"session","id":"74747474-7474-4747-8747-747474747474"},"sequence":125,"recorded_at":1788223110809647,"record_type":"event","durability":"durable","causation_id":null,"payload_type":"tool_batch.effect.terminal","payload_schema_version":1,"payload":{"kind":"tool_batch_effect","record":{"call_id":"call_muse_missing","effect_id":"017f8c0d-b268-4d99-a73b-3391f126516c","kind":"terminal","model_call_index":0,"outcome":{"kind":"failed","reason":"No such file or directory (os error 2)"},"task_id":"017f8c0d-b268-4d99-a73b-3391f126516c","task_stream":{"id":"017f8c0d-b268-4d99-a73b-3391f126516c","kind":"task"}},"run_id":"ffa8d311-da04-4cfc-b100-89c18a58f823"}} +{"schema_version":1,"id":"9d2dfe2a-4215-4112-9f14-e5cd68c0a786","stream":{"kind":"session","id":"74747474-7474-4747-8747-747474747474"},"sequence":128,"recorded_at":1788223110811068,"record_type":"event","durability":"durable","causation_id":null,"payload_type":"runtime.session","payload_schema_version":1,"payload":{"event":{"batch_id":"d9a98768-7eaf-4496-b382-be2ce4614480","kind":"tool_result_batch_committed","results":[{"text":"tool failed: No such file or directory (os error 2)","tool_call_id":"call_muse_missing","tool_call_index":0}]},"kind":"run","run_id":"ffa8d311-da04-4cfc-b100-89c18a58f823"}} +{"schema_version":1,"id":"698a9efd-8497-477a-9730-7436a0511a92","stream":{"kind":"session","id":"74747474-7474-4747-8747-747474747474"},"sequence":145,"recorded_at":1788223110828867,"record_type":"event","durability":"durable","causation_id":null,"payload_type":"runtime.session","payload_schema_version":1,"payload":{"event":{"kind":"assistant_message_committed","message_id":"3a5a3d69-0f3b-4971-af76-38b3e569c76d","provider_item_id":"msg_resp_muse_summary","response_id":"resp_muse_summary","text":"The boundary comparison is strict, the note marker is COPPER_4821, and the missing-file failure was retained."},"kind":"run","run_id":"ffa8d311-da04-4cfc-b100-89c18a58f823","source_run_record_id":"1d420b0d-e170-4aae-83e5-9e79ffebc93f","source_run_record_sequence":60}} +{"schema_version":1,"id":"3a0f7bf4-093f-4903-96ff-52ee79b624e6","stream":{"kind":"session","id":"74747474-7474-4747-8747-747474747474"},"sequence":148,"recorded_at":1788223110833646,"record_type":"event","durability":"durable","causation_id":null,"payload_type":"runtime.session","payload_schema_version":1,"payload":{"event":{"duration_ms":14,"kind":"model_completed","model":"meta/muse-glimmer-30b","usage":{"cache_read_tokens":0,"cache_write_tokens":0,"cached_tokens":0,"input_tokens":32,"output_tokens":8,"reasoning_tokens":0}},"kind":"run","run_id":"ffa8d311-da04-4cfc-b100-89c18a58f823","source_run_record_id":"5f694ca3-9d3a-448e-8507-2e7c3a077dd0","source_run_record_sequence":62}} +{"schema_version":1,"id":"bcf0bf7c-5a65-4ef3-a337-1e99fc6dec57","stream":{"kind":"session","id":"74747474-7474-4747-8747-747474747474"},"sequence":165,"recorded_at":1788223110922656,"record_type":"event","durability":"durable","causation_id":null,"payload_type":"runtime.session","payload_schema_version":1,"payload":{"event":{"eot_gate_ms":87,"kind":"terminal","reason":null,"terminal":"completed","time_to_first_token_ms":148,"turn_duration_ms":242},"kind":"run","run_id":"ffa8d311-da04-4cfc-b100-89c18a58f823","source_run_record_id":"c42f3009-b225-4a1b-8a0b-f4bd584355f1","source_run_record_sequence":68}} +{"schema_version":1,"id":"c6f0de74-d6c5-4890-8796-a14b97a7409d","stream":{"kind":"session","id":"74747474-7474-4747-8747-747474747474"},"sequence":166,"recorded_at":1788223110936073,"record_type":"event","durability":"durable","causation_id":null,"payload_type":"session.end","payload_schema_version":1,"payload":{"kind":"session_end","record":{"exit_reason":"clean","resource_usage":{"cpu_children_ms":2,"cpu_self_ms":231,"data_dir_free_bytes":2216562688,"load_avg_1m_hundredths":3185,"rss_self_peak_bytes":101261312,"rss_tree_peak_bytes":104226816,"session_log_bytes":805849,"threads_live":9,"workspace_free_bytes":2216562688},"schema_version":1,"session_id":"74747474-7474-4747-8747-747474747474","uptime_ms":257}}} +{"schema_version":1,"id":"c1ac942b-c0d6-4e32-b446-96645277a4d4","stream":{"kind":"session","id":"74747474-7474-4747-8747-747474747474"},"sequence":167,"recorded_at":1788223111117411,"record_type":"event","durability":"durable","causation_id":null,"payload_type":"runtime.session.route_facts","payload_schema_version":2,"payload":{"kind":"route_facts","record":{"cwd":"/fixture/work","pid":3392533,"terminal_title_identity":"747474747474"}}} +{"schema_version":1,"id":"7c4404b5-7678-499d-b77e-dcfa66c0dd04","stream":{"kind":"session","id":"74747474-7474-4747-8747-747474747474"},"sequence":168,"recorded_at":1788223111260389,"record_type":"event","durability":"durable","causation_id":null,"payload_type":"session.opened.observed","payload_schema_version":1,"payload":{"kind":"session_opened","record":{"data_dir_free_bytes":2216615936,"resume":true,"schema_version":1,"security_mode":"yolo","session_id":"74747474-7474-4747-8747-747474747474","workspace_free_bytes":2216615936}}} +{"schema_version":1,"id":"6c14d49a-a545-4649-a13d-22fc84499c99","stream":{"kind":"session","id":"74747474-7474-4747-8747-747474747474"},"sequence":170,"recorded_at":1788223111261294,"record_type":"event","durability":"durable","causation_id":null,"payload_type":"session.resumed","payload_schema_version":1,"payload":{"kind":"session_resumed","record":{"prior_turn_count":1,"resumed_from_record_id":"bcf0bf7c-5a65-4ef3-a337-1e99fc6dec57","resumed_from_sequence":165,"schema_version":1,"session_id":"74747474-7474-4747-8747-747474747474"}}} +{"schema_version":1,"id":"13488036-3b77-462c-a65d-d2217076ce71","stream":{"kind":"session","id":"74747474-7474-4747-8747-747474747474"},"sequence":172,"recorded_at":1788223111288863,"record_type":"event","durability":"durable","causation_id":null,"payload_type":"runtime.session","payload_schema_version":1,"payload":{"event":{"kind":"started","prompt":"Inspect the attached image and identify its shape and visible nonce."},"kind":"run","run_id":"28adbeba-47af-4425-b362-0261f207b380","source_run_record_id":"fead892d-ca45-45e5-b2a8-b14e53bb3eeb","source_run_record_sequence":1}} +{"schema_version":1,"id":"d55187fd-db48-4af5-8fb1-6dc1d455fe49","stream":{"kind":"session","id":"74747474-7474-4747-8747-747474747474"},"sequence":198,"recorded_at":1788223111332818,"record_type":"event","durability":"durable","causation_id":null,"payload_type":"runtime.session","payload_schema_version":1,"payload":{"event":{"kind":"assistant_message_committed","message_id":"ab7a934d-2bd5-4915-a1ce-daa50835418a","provider_item_id":"msg_resp_muse_image","response_id":"resp_muse_image","text":"The image contains a blue triangle labeled BLUE_TRIANGLE_7319."},"kind":"run","run_id":"28adbeba-47af-4425-b362-0261f207b380","source_run_record_id":"a40a2f8b-69a5-43e7-939c-70f438d481ae","source_run_record_sequence":16}} +{"schema_version":1,"id":"44383b5e-aa7a-443b-a228-d6965b655109","stream":{"kind":"session","id":"74747474-7474-4747-8747-747474747474"},"sequence":201,"recorded_at":1788223111334229,"record_type":"event","durability":"durable","causation_id":null,"payload_type":"runtime.session","payload_schema_version":1,"payload":{"event":{"duration_ms":25,"kind":"model_completed","model":"meta/muse-glimmer-30b","usage":{"cache_read_tokens":0,"cache_write_tokens":0,"cached_tokens":0,"input_tokens":32,"output_tokens":8,"reasoning_tokens":0}},"kind":"run","run_id":"28adbeba-47af-4425-b362-0261f207b380","source_run_record_id":"a37bb5c8-dcf3-4d20-ba09-f33171d3d16f","source_run_record_sequence":18}} +{"schema_version":1,"id":"32f03605-c293-454a-a3f8-c19657fa8ddb","stream":{"kind":"session","id":"74747474-7474-4747-8747-747474747474"},"sequence":218,"recorded_at":1788223111458161,"record_type":"event","durability":"durable","causation_id":null,"payload_type":"runtime.session","payload_schema_version":1,"payload":{"event":{"eot_gate_ms":122,"kind":"terminal","reason":null,"terminal":"completed","time_to_first_token_ms":43,"turn_duration_ms":169},"kind":"run","run_id":"28adbeba-47af-4425-b362-0261f207b380","source_run_record_id":"002b4af6-2f42-4bc9-973b-33f12fbadeed","source_run_record_sequence":24}} +{"schema_version":1,"id":"0ea590be-17c7-490c-ac76-b4de0b5ff0e3","stream":{"kind":"session","id":"74747474-7474-4747-8747-747474747474"},"sequence":219,"recorded_at":1788223111470232,"record_type":"event","durability":"durable","causation_id":null,"payload_type":"session.end","payload_schema_version":1,"payload":{"kind":"session_end","record":{"exit_reason":"clean","resource_usage":{"cpu_children_ms":1,"cpu_self_ms":354,"data_dir_free_bytes":2215981056,"load_avg_1m_hundredths":3185,"rss_self_peak_bytes":103186432,"rss_tree_peak_bytes":106086400,"session_log_bytes":1341049,"threads_live":10,"workspace_free_bytes":2215981056},"schema_version":1,"session_id":"74747474-7474-4747-8747-747474747474","uptime_ms":187}}} +{"schema_version":1,"id":"84d32ee7-3ded-48cd-ab9c-9da6cbf1abcb","stream":{"kind":"session","id":"74747474-7474-4747-8747-747474747474"},"sequence":220,"recorded_at":1788223111679190,"record_type":"event","durability":"durable","causation_id":null,"payload_type":"runtime.session.route_facts","payload_schema_version":2,"payload":{"kind":"route_facts","record":{"cwd":"/fixture/work","pid":3392695,"terminal_title_identity":"747474747474"}}} +{"schema_version":1,"id":"65b0a57d-b549-468e-bc86-b3b6cbcb02da","stream":{"kind":"session","id":"74747474-7474-4747-8747-747474747474"},"sequence":221,"recorded_at":1788223111841906,"record_type":"event","durability":"durable","causation_id":null,"payload_type":"session.opened.observed","payload_schema_version":1,"payload":{"kind":"session_opened","record":{"data_dir_free_bytes":2216005632,"resume":true,"schema_version":1,"security_mode":"yolo","session_id":"74747474-7474-4747-8747-747474747474","workspace_free_bytes":2216005632}}} +{"schema_version":1,"id":"d0a1e421-1b6c-4630-8486-a02081af0847","stream":{"kind":"session","id":"74747474-7474-4747-8747-747474747474"},"sequence":223,"recorded_at":1788223111842797,"record_type":"event","durability":"durable","causation_id":null,"payload_type":"session.resumed","payload_schema_version":1,"payload":{"kind":"session_resumed","record":{"prior_turn_count":2,"resumed_from_record_id":"32f03605-c293-454a-a3f8-c19657fa8ddb","resumed_from_sequence":218,"schema_version":1,"session_id":"74747474-7474-4747-8747-747474747474"}}} +{"schema_version":1,"id":"123e15fd-ff2f-4065-818a-ae2019998fab","stream":{"kind":"session","id":"74747474-7474-4747-8747-747474747474"},"sequence":228,"recorded_at":1788223111867428,"record_type":"event","durability":"durable","causation_id":null,"payload_type":"runtime.session","payload_schema_version":1,"payload":{"event":{"kind":"started","prompt":"Without tools, recall the task marker, note marker, image marker, and failed filename."},"kind":"run","run_id":"0bbf256f-b87e-4cf9-99bb-3d462715e1d8","source_run_record_id":"7e2ac1a3-bea1-40da-b88d-48c77b9f89e6","source_run_record_sequence":1}} +{"schema_version":1,"id":"ef1c2327-4418-45cd-8259-e4e173983fbf","stream":{"kind":"session","id":"74747474-7474-4747-8747-747474747474"},"sequence":255,"recorded_at":1788223111916704,"record_type":"event","durability":"durable","causation_id":null,"payload_type":"runtime.session","payload_schema_version":1,"payload":{"event":{"kind":"assistant_message_committed","message_id":"5726f698-6d80-4a72-bf94-ac460d454acd","provider_item_id":"msg_resp_muse_recall","response_id":"resp_muse_recall","text":"Recall: SM_CORPUS_7319, COPPER_4821, BLUE_TRIANGLE_7319, and missing-corpus-file.txt."},"kind":"run","run_id":"0bbf256f-b87e-4cf9-99bb-3d462715e1d8","source_run_record_id":"8dedd56b-5b37-462f-913b-815d110ff272","source_run_record_sequence":16}} +{"schema_version":1,"id":"c1ddabad-cf2c-4995-9957-ba2a1dc54230","stream":{"kind":"session","id":"74747474-7474-4747-8747-747474747474"},"sequence":258,"recorded_at":1788223111925271,"record_type":"event","durability":"durable","causation_id":null,"payload_type":"runtime.session","payload_schema_version":1,"payload":{"event":{"duration_ms":29,"kind":"model_completed","model":"meta/muse-glimmer-30b","usage":{"cache_read_tokens":0,"cache_write_tokens":0,"cached_tokens":0,"input_tokens":32,"output_tokens":8,"reasoning_tokens":0}},"kind":"run","run_id":"0bbf256f-b87e-4cf9-99bb-3d462715e1d8","source_run_record_id":"b401006d-9749-4f27-8125-eff1a81574a4","source_run_record_sequence":18}} +{"schema_version":1,"id":"61dbbf75-ac99-4c6b-aa83-d5317b32e8b3","stream":{"kind":"session","id":"74747474-7474-4747-8747-747474747474"},"sequence":275,"recorded_at":1788223112041092,"record_type":"event","durability":"durable","causation_id":null,"payload_type":"runtime.session","payload_schema_version":1,"payload":{"event":{"eot_gate_ms":114,"kind":"terminal","reason":null,"terminal":"completed","time_to_first_token_ms":47,"turn_duration_ms":173},"kind":"run","run_id":"0bbf256f-b87e-4cf9-99bb-3d462715e1d8","source_run_record_id":"c7fb0944-688d-4ce7-913e-3c4d73d7d853","source_run_record_sequence":24}} +{"schema_version":1,"id":"05e6ada9-1d31-42e9-a030-685d7adef551","stream":{"kind":"session","id":"74747474-7474-4747-8747-747474747474"},"sequence":276,"recorded_at":1788223112057691,"record_type":"event","durability":"durable","causation_id":null,"payload_type":"session.end","payload_schema_version":1,"payload":{"kind":"session_end","record":{"exit_reason":"clean","resource_usage":{"cpu_children_ms":1,"cpu_self_ms":403,"data_dir_free_bytes":2215403520,"load_avg_1m_hundredths":3185,"rss_self_peak_bytes":104079360,"rss_tree_peak_bytes":106979328,"session_log_bytes":1864612,"threads_live":9,"workspace_free_bytes":2215403520},"schema_version":1,"session_id":"74747474-7474-4747-8747-747474747474","uptime_ms":194}}} diff --git a/crates/ai-hist/tests/fixtures/muse/tools-session/.local/share/muse/sessions/2026/09/20/muse-a001/session.jsonl b/crates/ai-hist/tests/fixtures/muse/tools-session/.local/share/muse/sessions/2026/09/20/muse-a001/session.jsonl new file mode 100644 index 00000000..7749f0c5 --- /dev/null +++ b/crates/ai-hist/tests/fixtures/muse/tools-session/.local/share/muse/sessions/2026/09/20/muse-a001/session.jsonl @@ -0,0 +1,31 @@ +{"retained_frame": "session_permission_transaction", "children": []} +{"schema_version": 1, "id": "a001-rec-001", "stream": {"kind": "session", "id": "muse-a001"}, "sequence": 1, "recorded_at": 1790337600250000, "record_type": "event", "durability": "durable", "causation_id": null, "payload_type": "runtime.session.metadata", "payload_schema_version": 1, "payload": {"kind": "metadata", "record": {"build": {"semver": "1.3.0", "sha": "0000000000"}, "model_id": "meta/muse-spark-1.3", "provider_id": "meta", "tool_surface_version": "2", "workspace_root": "/tmp/muse-demo"}}} +{"schema_version": 1, "id": "a001-rec-002", "stream": {"kind": "session", "id": "muse-a001"}, "sequence": 2, "recorded_at": 1790337600500000, "record_type": "event", "durability": "durable", "causation_id": null, "payload_type": "session.opened.observed", "payload_schema_version": 1, "payload": {"kind": "session_opened", "record": {"resume": false, "session_id": "muse-a001"}}} +{"schema_version": 1, "id": "a001-rec-003", "stream": {"kind": "session", "id": "muse-a001"}, "sequence": 3, "recorded_at": 1790337600750000, "record_type": "event", "durability": "durable", "causation_id": null, "payload_type": "runtime.session", "payload_schema_version": 1, "payload": {"kind": "run", "run_id": "run-a", "event": {"kind": "started", "prompt": "Add a retry helper to src/http.rs and run the tests."}}} +{"schema_version": 1, "id": "a001-rec-004", "stream": {"kind": "session", "id": "muse-a001"}, "sequence": 4, "recorded_at": 1790337601000000, "record_type": "event", "durability": "durable", "causation_id": null, "payload_type": "runtime.session", "payload_schema_version": 1, "payload": {"kind": "run", "run_id": "run-a", "event": {"kind": "model_completed", "model": "meta/muse-spark-1.3", "duration_ms": 900, "finish_reason": "tool_calls", "usage": {"input_tokens": 1200, "output_tokens": 90, "cached_tokens": 800, "cache_read_tokens": 800, "cache_write_tokens": 0, "reasoning_tokens": 40}}}} +{"schema_version": 1, "id": "a001-rec-005", "stream": {"kind": "session", "id": "muse-a001"}, "sequence": 5, "recorded_at": 1790337601250000, "record_type": "event", "durability": "durable", "causation_id": null, "payload_type": "runtime.session", "payload_schema_version": 1, "payload": {"kind": "run", "run_id": "run-a", "event": {"kind": "reasoning_committed", "message_id": "msg-r1", "text": "", "encrypted_content": "gAAAAB-opaque"}}} +{"schema_version": 1, "id": "a001-rec-006", "stream": {"kind": "session", "id": "muse-a001"}, "sequence": 6, "recorded_at": 1790337601500000, "record_type": "event", "durability": "durable", "causation_id": null, "payload_type": "runtime.session", "payload_schema_version": 1, "payload": {"kind": "run", "run_id": "run-a", "event": {"kind": "reasoning_committed", "message_id": "msg-r2", "text": "Read the file before editing it."}}} +{"schema_version": 1, "id": "a001-rec-007", "stream": {"kind": "session", "id": "muse-a001"}, "sequence": 7, "recorded_at": 1790337601750000, "record_type": "event", "durability": "durable", "causation_id": null, "payload_type": "runtime.session", "payload_schema_version": 1, "payload": {"kind": "run", "run_id": "run-a", "event": {"kind": "assistant_tool_calls_committed", "message_id": "msg-1", "response_id": "resp-1", "tool_calls": [{"id": "fc_read", "call_id": "call_read", "name": "read_file", "args": "{\"path\": \"src/http.rs\"}"}]}}} +{"schema_version": 1, "id": "a001-rec-008", "stream": {"kind": "session", "id": "muse-a001"}, "sequence": 8, "recorded_at": 1790337602000000, "record_type": "event", "durability": "durable", "causation_id": null, "payload_type": "tool_batch.effect.terminal", "payload_schema_version": 1, "payload": {"kind": "tool_batch_effect", "run_id": "run-a", "record": {"call_id": "call_read", "kind": "terminal", "outcome": {"kind": "completed"}}}} +{"schema_version": 1, "id": "a001-rec-009", "stream": {"kind": "session", "id": "muse-a001"}, "sequence": 9, "recorded_at": 1790337602250000, "record_type": "event", "durability": "durable", "causation_id": null, "payload_type": "runtime.session", "payload_schema_version": 1, "payload": {"kind": "run", "run_id": "run-a", "event": {"kind": "tool_result_batch_committed", "batch_id": "msg-1", "results": [{"tool_call_id": "call_read", "tool_call_index": 0, "text": "Read text file `src/http.rs`.\n1|pub fn get() {}"}]}}} +{"schema_version": 1, "id": "a001-rec-010", "stream": {"kind": "session", "id": "muse-a001"}, "sequence": 10, "recorded_at": 1790337602500000, "record_type": "event", "durability": "durable", "causation_id": null, "payload_type": "runtime.session", "payload_schema_version": 1, "payload": {"kind": "run", "run_id": "run-a", "event": {"kind": "model_completed", "model": "meta/muse-spark-1.3", "duration_ms": 700, "finish_reason": "tool_calls", "usage": {"input_tokens": 1500, "output_tokens": 210, "cached_tokens": 1100, "cache_read_tokens": 1100, "cache_write_tokens": 0, "reasoning_tokens": 0}}}} +{"schema_version": 1, "id": "a001-rec-011", "stream": {"kind": "session", "id": "muse-a001"}, "sequence": 11, "recorded_at": 1790337602750000, "record_type": "event", "durability": "durable", "causation_id": null, "payload_type": "runtime.session", "payload_schema_version": 1, "payload": {"kind": "run", "run_id": "run-a", "event": {"kind": "assistant_tool_calls_committed", "message_id": "msg-2", "response_id": "resp-2", "tool_calls": [{"id": "fc_edit", "call_id": "call_edit", "name": "edit_file", "args": "{\"path\": \"src/http.rs\", \"find\": \"pub fn get() {}\", \"replace\": \"pub fn get() {}\\npub fn retry() {}\"}"}, {"id": "fc_write", "call_id": "call_write", "name": "write_file", "args": "{\"path\": \"tests/retry.rs\", \"content\": \"#[test] fn retries() {}\"}"}, {"id": "fc_bash", "call_id": "call_bash", "name": "bash", "args": "{\"command\": \"cargo test\", \"workdir\": \"/tmp/muse-demo\"}"}, {"id": "fc_spawn", "call_id": "call_spawn", "name": "subagent_spawn", "args": "{\"role\": \"reviewer\", \"objective\": \"review src/http.rs\"}"}]}}} +{"schema_version": 1, "id": "a001-rec-012", "stream": {"kind": "session", "id": "muse-a001"}, "sequence": 12, "recorded_at": 1790337603000000, "record_type": "event", "durability": "durable", "causation_id": null, "payload_type": "tool_batch.effect.terminal", "payload_schema_version": 1, "payload": {"kind": "tool_batch_effect", "run_id": "run-a", "record": {"call_id": "call_edit", "kind": "terminal", "outcome": {"kind": "completed"}}}} +{"schema_version": 1, "id": "a001-rec-013", "stream": {"kind": "session", "id": "muse-a001"}, "sequence": 13, "recorded_at": 1790337603250000, "record_type": "event", "durability": "durable", "causation_id": null, "payload_type": "tool_batch.effect.terminal", "payload_schema_version": 1, "payload": {"kind": "tool_batch_effect", "run_id": "run-a", "record": {"call_id": "call_write", "kind": "terminal", "outcome": {"kind": "completed"}}}} +{"schema_version": 1, "id": "a001-rec-014", "stream": {"kind": "session", "id": "muse-a001"}, "sequence": 14, "recorded_at": 1790337603500000, "record_type": "event", "durability": "durable", "causation_id": null, "payload_type": "tool_batch.effect.terminal", "payload_schema_version": 1, "payload": {"kind": "tool_batch_effect", "run_id": "run-a", "record": {"call_id": "call_bash", "kind": "terminal", "outcome": {"kind": "completed"}}}} +{"schema_version": 1, "id": "a001-rec-015", "stream": {"kind": "session", "id": "muse-a001"}, "sequence": 15, "recorded_at": 1790337603750000, "record_type": "event", "durability": "durable", "causation_id": null, "payload_type": "tool_batch.effect.terminal", "payload_schema_version": 1, "payload": {"kind": "tool_batch_effect", "run_id": "run-a", "record": {"call_id": "call_spawn", "kind": "terminal", "outcome": {"kind": "completed"}}}} +{"schema_version": 1, "id": "a001-rec-016", "stream": {"kind": "session", "id": "muse-a001"}, "sequence": 16, "recorded_at": 1790337604000000, "record_type": "event", "durability": "durable", "causation_id": null, "payload_type": "runtime.session", "payload_schema_version": 1, "payload": {"kind": "run", "run_id": "run-a", "event": {"kind": "task_stream_linked", "execution_mode": "background", "task_id": "muse-task-d001", "task_stream": {"id": "muse-task-d001", "kind": "task"}, "display": {"label": "reviewer", "model": "same-as-main", "path": "subagent/muse-c001/session.jsonl", "role": "worker"}}}} +{"schema_version": 1, "id": "a001-rec-017", "stream": {"kind": "task", "id": "muse-task-d001"}, "sequence": 17, "recorded_at": 1790337604250000, "record_type": "event", "durability": "durable", "causation_id": null, "payload_type": "runtime.session", "payload_schema_version": 1, "payload": {"kind": "run", "run_id": "task-run", "event": {"kind": "started", "prompt": "Role: reviewer Objective: review src/http.rs"}}} +{"schema_version": 1, "id": "a001-rec-018", "stream": {"kind": "session", "id": "muse-a001"}, "sequence": 18, "recorded_at": 1790337604500000, "record_type": "event", "durability": "durable", "causation_id": null, "payload_type": "runtime.session", "payload_schema_version": 1, "payload": {"kind": "run", "run_id": "run-a", "event": {"kind": "tool_result_batch_committed", "batch_id": "msg-2", "results": [{"tool_call_id": "call_edit", "tool_call_index": 0, "text": "edited \u2014 diff: -pub fn get() {} +pub fn retry() {}"}, {"tool_call_id": "call_write", "tool_call_index": 1, "text": "wrote 22 bytes to tests/retry.rs"}, {"tool_call_id": "call_bash", "tool_call_index": 2, "text": "{\"command\": \"cargo test\", \"description\": null, \"exit_code\": 101, \"terminal_status\": \"exited\", \"output\": \"test retries ... FAILED\", \"truncated\": false}"}, {"tool_call_id": "call_spawn", "tool_call_index": 3, "text": "{\"ok\": true, \"task_id\": \"muse-task-d001\", \"status\": \"running\"}"}]}}} +{"schema_version": 1, "id": "a001-rec-019", "stream": {"kind": "session", "id": "muse-a001"}, "sequence": 19, "recorded_at": 1790337604750000, "record_type": "event", "durability": "durable", "causation_id": null, "payload_type": "runtime.session", "payload_schema_version": 1, "payload": {"kind": "run", "run_id": "run-a", "event": {"kind": "model_completed", "model": "meta/muse-spark-1.3", "duration_ms": 400, "finish_reason": "stop", "usage": {"input_tokens": 1800, "output_tokens": 35, "cached_tokens": 1500, "cache_read_tokens": 1500, "cache_write_tokens": 0, "reasoning_tokens": 0}}}} +{"schema_version": 1, "id": "a001-rec-020", "stream": {"kind": "session", "id": "muse-a001"}, "sequence": 20, "recorded_at": 1790337605000000, "record_type": "event", "durability": "durable", "causation_id": null, "payload_type": "runtime.session", "payload_schema_version": 1, "payload": {"kind": "run", "run_id": "run-a", "event": {"kind": "assistant_message_committed", "message_id": "msg-3", "response_id": "resp-3", "text": "Added `retry` and a test; `cargo test` fails with exit code 101."}}} +{"schema_version": 1, "id": "a001-rec-021", "stream": {"kind": "session", "id": "muse-a001"}, "sequence": 21, "recorded_at": 1790337605250000, "record_type": "event", "durability": "durable", "causation_id": null, "payload_type": "runtime.session", "payload_schema_version": 1, "payload": {"kind": "run", "run_id": "run-a", "event": {"kind": "memory_reminder_child_session_linked", "child_session_id": "muse-e001", "child_session_log_path": "subagent/muse-e001/session.jsonl", "generation_id": 1, "parent_run_id": "run-a", "parent_session_id": "muse-a001", "reminder_agent_id": "plugin:tbh-reminders:goal-reminder", "task_id": "muse-task-d002", "task_stream": {"id": "muse-task-d002", "kind": "task"}}}} +{"schema_version": 1, "id": "a001-rec-022", "stream": {"kind": "session", "id": "muse-a001"}, "sequence": 22, "recorded_at": 1790337605500000, "record_type": "event", "durability": "durable", "causation_id": null, "payload_type": "runtime.session", "payload_schema_version": 1, "payload": {"kind": "run", "run_id": "run-a", "event": {"kind": "terminal", "terminal": "completed", "reason": null, "turn_duration_ms": 2500}}} +{"schema_version": 1, "id": "a001-rec-023", "stream": {"kind": "session", "id": "muse-a001"}, "sequence": 23, "recorded_at": 1790337605750000, "record_type": "event", "durability": "durable", "causation_id": null, "payload_type": "session.end", "payload_schema_version": 1, "payload": {"kind": "session_end", "record": {"exit_reason": "clean", "session_id": "muse-a001"}}} +{"schema_version": 1, "id": "a001-rec-024", "stream": {"kind": "session", "id": "muse-a001"}, "sequence": 24, "recorded_at": 1790337606000000, "record_type": "event", "durability": "durable", "causation_id": null, "payload_type": "session.opened.observed", "payload_schema_version": 1, "payload": {"kind": "session_opened", "record": {"resume": true, "session_id": "muse-a001"}}} +{"schema_version": 1, "id": "a001-rec-025", "stream": {"kind": "session", "id": "muse-a001"}, "sequence": 25, "recorded_at": 1790337606250000, "record_type": "event", "durability": "durable", "causation_id": null, "payload_type": "session.resumed", "payload_schema_version": 1, "payload": {"kind": "session_resumed", "record": {"prior_turn_count": 1, "session_id": "muse-a001"}}} +{"schema_version": 1, "id": "a001-rec-026", "stream": {"kind": "session", "id": "muse-a001"}, "sequence": 26, "recorded_at": 1790337606500000, "record_type": "event", "durability": "durable", "causation_id": null, "payload_type": "runtime.session.metadata", "payload_schema_version": 1, "payload": {"kind": "metadata", "record": {"build": {"semver": "1.3.0", "sha": "0000000000"}, "model_id": "meta/muse-spark-1.3-contributor", "provider_id": "meta", "tool_surface_version": "2", "workspace_root": "/tmp/muse-demo"}}} +{"schema_version": 1, "id": "a001-rec-027", "stream": {"kind": "session", "id": "muse-a001"}, "sequence": 27, "recorded_at": 1790337606750000, "record_type": "event", "durability": "durable", "causation_id": null, "payload_type": "runtime.session", "payload_schema_version": 1, "payload": {"kind": "run", "run_id": "run-b", "event": {"kind": "started", "prompt": "Why did it fail?"}}} +{"schema_version": 1, "id": "a001-rec-028", "stream": {"kind": "session", "id": "muse-a001"}, "sequence": 28, "recorded_at": 1790337607000000, "record_type": "event", "durability": "durable", "causation_id": null, "payload_type": "runtime.session", "payload_schema_version": 1, "payload": {"kind": "run", "run_id": "run-b", "event": {"kind": "model_completed", "model": "meta/muse-spark-1.3-contributor", "duration_ms": 300, "finish_reason": "stop", "usage": {"input_tokens": 2000, "output_tokens": 20, "cached_tokens": 1800, "cache_read_tokens": 1800, "cache_write_tokens": 0, "reasoning_tokens": 0}}}} +{"schema_version": 1, "id": "a001-rec-029", "stream": {"kind": "session", "id": "muse-a001"}, "sequence": 29, "recorded_at": 1790337607250000, "record_type": "event", "durability": "durable", "causation_id": null, "payload_type": "runtime.session", "payload_schema_version": 1, "payload": {"kind": "run", "run_id": "run-b", "event": {"kind": "assistant_message_committed", "message_id": "msg-4", "response_id": "resp-4", "text": "`retry` is declared but the test expects it to return a value."}}} +{"schema_version": 1, "id": "a001-rec-030", "stream": {"kind": "session", "id": "muse-a001"}, "sequence": 30, "recorded_at": 1790337607500000, "record_type": "event", "durability": "durable", "causation_id": null, "payload_type": "runtime.session", "payload_schema_version": 1, "payload": {"kind": "run", "run_id": "run-b", "event": {"kind": "terminal", "terminal": "interrupted", "reason": "user_cancelled", "turn_duration_ms": 400}}} diff --git a/crates/ai-hist/tests/fixtures/muse/tools-session/.local/share/muse/sessions/2026/09/20/muse-a001/subagent/muse-c001/session.jsonl b/crates/ai-hist/tests/fixtures/muse/tools-session/.local/share/muse/sessions/2026/09/20/muse-a001/subagent/muse-c001/session.jsonl new file mode 100644 index 00000000..284a0015 --- /dev/null +++ b/crates/ai-hist/tests/fixtures/muse/tools-session/.local/share/muse/sessions/2026/09/20/muse-a001/subagent/muse-c001/session.jsonl @@ -0,0 +1,6 @@ +{"schema_version": 1, "id": "c001-rec-001", "stream": {"kind": "session", "id": "muse-c001"}, "sequence": 1, "recorded_at": 1790337604350000, "record_type": "event", "durability": "durable", "causation_id": null, "payload_type": "runtime.session.metadata", "payload_schema_version": 1, "payload": {"kind": "metadata", "record": {"build": {"semver": "1.3.0", "sha": "0000000000"}, "model_id": "meta/muse-spark-1.3", "provider_id": "meta", "tool_surface_version": "2", "workspace_root": "/tmp/muse-demo"}}} +{"schema_version": 1, "id": "c001-rec-002", "stream": {"kind": "session", "id": "muse-c001"}, "sequence": 2, "recorded_at": 1790337604600000, "record_type": "event", "durability": "durable", "causation_id": null, "payload_type": "runtime.session", "payload_schema_version": 1, "payload": {"kind": "run", "run_id": "child-run", "event": {"kind": "started", "prompt": "Role: reviewer Objective: review src/http.rs"}}} +{"schema_version": 1, "id": "c001-rec-003", "stream": {"kind": "session", "id": "muse-c001"}, "sequence": 3, "recorded_at": 1790337604850000, "record_type": "event", "durability": "durable", "causation_id": null, "payload_type": "runtime.session", "payload_schema_version": 1, "payload": {"kind": "run", "run_id": "child-run", "event": {"kind": "model_completed", "model": "meta/muse-spark-1.3", "duration_ms": 200, "finish_reason": "stop", "usage": {"input_tokens": 600, "output_tokens": 25, "cached_tokens": 0, "cache_read_tokens": 0, "cache_write_tokens": 0, "reasoning_tokens": 0}}}} +{"schema_version": 1, "id": "c001-rec-004", "stream": {"kind": "session", "id": "muse-c001"}, "sequence": 4, "recorded_at": 1790337605100000, "record_type": "event", "durability": "durable", "causation_id": null, "payload_type": "runtime.session", "payload_schema_version": 1, "payload": {"kind": "run", "run_id": "child-run", "event": {"kind": "assistant_message_committed", "message_id": "child-msg-1", "text": "`retry` has no body; the test cannot pass."}}} +{"schema_version": 1, "id": "c001-rec-005", "stream": {"kind": "session", "id": "muse-c001"}, "sequence": 5, "recorded_at": 1790337605350000, "record_type": "event", "durability": "durable", "causation_id": null, "payload_type": "runtime.session", "payload_schema_version": 1, "payload": {"kind": "run", "run_id": "child-run", "event": {"kind": "task_stream_linked", "task_id": "muse-task-d003", "display": {"label": "linter", "model": "meta/muse-glimmer-30b", "path": "subagent/muse-c002/session.jsonl", "role": "worker"}}}} +{"schema_version": 1, "id": "c001-rec-006", "stream": {"kind": "session", "id": "muse-c001"}, "sequence": 6, "recorded_at": 1790337605600000, "record_type": "event", "durability": "durable", "causation_id": null, "payload_type": "runtime.session", "payload_schema_version": 1, "payload": {"kind": "run", "run_id": "child-run", "event": {"kind": "terminal", "terminal": "completed"}}} diff --git a/crates/ai-hist/tests/fixtures/muse/tools-session/.local/share/muse/sessions/2026/09/20/muse-a001/subagent/muse-c001/subagent/muse-c002/session.jsonl b/crates/ai-hist/tests/fixtures/muse/tools-session/.local/share/muse/sessions/2026/09/20/muse-a001/subagent/muse-c001/subagent/muse-c002/session.jsonl new file mode 100644 index 00000000..6a206cbb --- /dev/null +++ b/crates/ai-hist/tests/fixtures/muse/tools-session/.local/share/muse/sessions/2026/09/20/muse-a001/subagent/muse-c001/subagent/muse-c002/session.jsonl @@ -0,0 +1,3 @@ +{"schema_version": 1, "id": "c002-rec-001", "stream": {"kind": "session", "id": "muse-c002"}, "sequence": 1, "recorded_at": 1790337605850000, "record_type": "event", "durability": "durable", "causation_id": null, "payload_type": "runtime.session.metadata", "payload_schema_version": 1, "payload": {"kind": "metadata", "record": {"build": {"semver": "1.3.0", "sha": "0000000000"}, "model_id": "meta/muse-glimmer-30b", "provider_id": "meta", "tool_surface_version": "2", "workspace_root": "/tmp/muse-demo"}}} +{"schema_version": 1, "id": "c002-rec-002", "stream": {"kind": "session", "id": "muse-c002"}, "sequence": 2, "recorded_at": 1790337606100000, "record_type": "event", "durability": "durable", "causation_id": null, "payload_type": "runtime.session", "payload_schema_version": 1, "payload": {"kind": "run", "run_id": "grand-run", "event": {"kind": "started", "prompt": "Role: linter Objective: lint src/http.rs"}}} +{"schema_version": 1, "id": "c002-rec-003", "stream": {"kind": "session", "id": "muse-c002"}, "sequence": 3, "recorded_at": 1790337606350000, "record_type": "event", "durability": "durable", "causation_id": null, "payload_type": "runtime.session", "payload_schema_version": 1, "payload": {"kind": "run", "run_id": "grand-run", "event": {"kind": "assistant_message_committed", "message_id": "grand-msg-1", "text": "No lint findings."}}} diff --git a/crates/ai-hist/tests/fixtures/muse/tools-session/.local/share/muse/sessions/2026/09/20/muse-a001/subagent/muse-e001/session.jsonl b/crates/ai-hist/tests/fixtures/muse/tools-session/.local/share/muse/sessions/2026/09/20/muse-a001/subagent/muse-e001/session.jsonl new file mode 100644 index 00000000..fe0f31fc --- /dev/null +++ b/crates/ai-hist/tests/fixtures/muse/tools-session/.local/share/muse/sessions/2026/09/20/muse-a001/subagent/muse-e001/session.jsonl @@ -0,0 +1,3 @@ +{"schema_version": 1, "id": "e001-rec-001", "stream": {"kind": "session", "id": "muse-e001"}, "sequence": 1, "recorded_at": 1790337605650000, "record_type": "event", "durability": "durable", "causation_id": null, "payload_type": "runtime.session.metadata", "payload_schema_version": 1, "payload": {"kind": "metadata", "record": {"build": {"semver": "1.3.0", "sha": "0000000000"}, "model_id": "meta/muse-spark-1.3", "provider_id": "meta", "tool_surface_version": "2", "workspace_root": "/tmp/muse-demo"}}} +{"schema_version": 1, "id": "e001-rec-002", "stream": {"kind": "session", "id": "muse-e001"}, "sequence": 2, "recorded_at": 1790337605900000, "record_type": "event", "durability": "durable", "causation_id": null, "payload_type": "runtime.session", "payload_schema_version": 1, "payload": {"kind": "run", "run_id": "remind-run", "event": {"kind": "started", "prompt": "Is the user's task finished?"}}} +{"schema_version": 1, "id": "e001-rec-003", "stream": {"kind": "session", "id": "muse-e001"}, "sequence": 3, "recorded_at": 1790337606150000, "record_type": "event", "durability": "durable", "causation_id": null, "payload_type": "runtime.session", "payload_schema_version": 1, "payload": {"kind": "run", "run_id": "remind-run", "event": {"kind": "assistant_message_committed", "message_id": "remind-msg-1", "text": "No: the test still fails."}}} diff --git a/crates/ai-hist/tests/live_capture.rs b/crates/ai-hist/tests/live_capture.rs index 51eb4eb9..785f034d 100644 --- a/crates/ai-hist/tests/live_capture.rs +++ b/crates/ai-hist/tests/live_capture.rs @@ -32,6 +32,7 @@ struct HomeLayout { claude: PathBuf, codex: PathBuf, grok: PathBuf, + muse: PathBuf, opencode_db: PathBuf, } @@ -43,6 +44,7 @@ impl HomeLayout { claude: home.join(".claude"), codex: home.join(".codex"), grok: home.join(".grok"), + muse: home.join(".local/share/muse/sessions"), opencode_db: home.join(".local/share/opencode/opencode.db"), } } @@ -53,6 +55,7 @@ impl HomeLayout { claude: &self.claude, codex: &self.codex, grok: &self.grok, + muse: &self.muse, opencode_db: &self.opencode_db, } } @@ -535,7 +538,8 @@ fn every_file_backed_provider_contributes_a_watch_root() { /// A relocated provider root moves the watch with it. /// -/// `CLAUDE_CONFIG_DIR`, `CODEX_HOME` and `GROK_HOME` already move what a sweep +/// `CLAUDE_CONFIG_DIR`, `CODEX_HOME`, `GROK_HOME` and (for Muse Code) +/// `XDG_DATA_HOME` already move what a sweep /// reads. A watch root rebuilt from `$HOME` instead would leave live capture /// staring at a directory the provider never writes to: every sweep correct, /// every one of them waiting out the backstop. @@ -547,6 +551,7 @@ fn configured_provider_roots_move_the_watch_roots() { claude: PathBuf::from("/tmp/relayhistory-relocated/claude"), codex: PathBuf::from("/tmp/relayhistory-relocated/codex"), grok: PathBuf::from("/tmp/relayhistory-relocated/grok"), + muse: PathBuf::from("/tmp/relayhistory-relocated/xdg-data/muse/sessions"), opencode_db: PathBuf::from("/tmp/relayhistory-relocated/opencode/opencode.db"), }; let roots = ai_hist::discover::watch_roots(&shallow_providers(), &layout.roots()); @@ -556,6 +561,7 @@ fn configured_provider_roots_move_the_watch_roots() { layout.codex.join("sessions"), layout.codex.join("archived_sessions"), layout.grok.join("sessions"), + layout.muse.clone(), layout .opencode_db .parent() @@ -572,7 +578,8 @@ fn configured_provider_roots_move_the_watch_roots() { .iter() .any(|root| root.path.starts_with(home.join(".claude")) || root.path.starts_with(home.join(".codex")) - || root.path.starts_with(home.join(".grok"))), + || root.path.starts_with(home.join(".grok")) + || root.path.starts_with(home.join(".local/share/muse"))), "a configured root must not leave a $HOME-relative watch behind: {roots:?}" ); } diff --git a/crates/ai-hist/tests/snapshots/muse/cli-capture.json b/crates/ai-hist/tests/snapshots/muse/cli-capture.json new file mode 100644 index 00000000..35597720 --- /dev/null +++ b/crates/ai-hist/tests/snapshots/muse/cli-capture.json @@ -0,0 +1,584 @@ +{ + "corpus_files": [ + "muse/cli-capture" + ], + "discovery_diagnostics": [], + "file_edits": [], + "fixture": "muse/cli-capture", + "history": [ + { + "project": "/fixture/work", + "prompt": "SM_CORPUS_7319 Inspect timeline.py and CORPUS_NOTE.txt with the native file tool. Then try missing-corpus-file.txt so the native log includes a failed tool result. Explain the interval boundary and quote the note marker.", + "session_id": "74747474-7474-4747-8747-747474747474", + "source": "muse", + "timestamp_ms": 1788223110680 + }, + { + "project": "/fixture/work", + "prompt": "Inspect the attached image and identify its shape and visible nonce.", + "session_id": "74747474-7474-4747-8747-747474747474", + "source": "muse", + "timestamp_ms": 1788223111288 + }, + { + "project": "/fixture/work", + "prompt": "Without tools, recall the task marker, note marker, image marker, and failed filename.", + "session_id": "74747474-7474-4747-8747-747474747474", + "source": "muse", + "timestamp_ms": 1788223111867 + } + ], + "hydration": [ + { + "related_session_ids": [], + "session_id": "74747474-7474-4747-8747-747474747474", + "source": "muse", + "status": "hydrated" + } + ], + "layout": "HomeTree", + "notes": [], + "origin": "sessionmigrate", + "quirk": "a transcript the real `muse` CLI (0.2.1) wrote, trimmed to its conversation, tool and lifecycle records: three runs across two resumes, `read_file` calls with one failed outcome, per-step `model_completed` usage, and mirrored reminder task records", + "session_events": [ + { + "agent_id": null, + "call_index": null, + "control_kind": null, + "cwd": "/fixture/work", + "error_signal": null, + "event_index": null, + "event_source": null, + "event_uid": "e3e35fae-9c8c-40a9-b961-bc81ff200b79", + "git_branch": null, + "kind": "text", + "message_id": "e3e35fae-9c8c-40a9-b961-bc81ff200b79", + "model": null, + "parent_id": null, + "payload_bytes": null, + "payload_hash": null, + "payload_truncated": null, + "project": "/fixture/work", + "result_status": null, + "role": "user", + "session_id": "74747474-7474-4747-8747-747474747474", + "source": "muse", + "subagent_session_id": null, + "text": "SM_CORPUS_7319 Inspect timeline.py and CORPUS_NOTE.txt with the native file tool. Then try missing-corpus-file.txt so the native log includes a failed tool result. Explain the interval boundary and quote the note marker.", + "token_json": null, + "tool_use_id": null, + "ts_ms": 1788223110680 + }, + { + "agent_id": null, + "call_index": null, + "control_kind": null, + "cwd": "/fixture/work", + "error_signal": null, + "event_index": null, + "event_source": null, + "event_uid": "tool:call_muse_timeline", + "git_branch": null, + "kind": "tool_use", + "message_id": "80749233-420e-4c1c-b567-a9047d47c4be", + "model": "meta/muse-glimmer-30b", + "parent_id": null, + "payload_bytes": null, + "payload_hash": null, + "payload_truncated": null, + "project": "/fixture/work", + "result_status": null, + "role": "assistant", + "session_id": "74747474-7474-4747-8747-747474747474", + "source": "muse", + "subagent_session_id": null, + "text": "read_file timeline.py", + "token_json": "{\"cache_read_tokens\":0,\"cache_write_tokens\":0,\"cached_tokens\":0,\"input_tokens\":32,\"output_tokens\":8,\"reasoning_tokens\":0}", + "tool_use_id": null, + "ts_ms": 1788223110715 + }, + { + "agent_id": null, + "call_index": 0, + "control_kind": null, + "cwd": "/fixture/work", + "error_signal": null, + "event_index": 0, + "event_source": "function_call_output", + "event_uid": "result:call_muse_timeline", + "git_branch": null, + "kind": "tool_result", + "message_id": "result:call_muse_timeline", + "model": null, + "parent_id": null, + "payload_bytes": 329, + "payload_hash": "5eafafc4b93a21d1", + "payload_truncated": 0, + "project": "/fixture/work", + "result_status": "completed", + "role": "tool_result", + "session_id": "74747474-7474-4747-8747-747474747474", + "source": "muse", + "subagent_session_id": null, + "text": "Read text file `timeline.py`.\n1|from dataclasses import dataclass\n2|\n3|\n4|@dataclass(frozen=True)\n5|class Event:\n6| start_ms: int\n7| end_ms: int\n8|\n9|\n10|def touches(previous: Event, current: Event) -> bool:\n11| \"\"\"Return whether two closed intervals overlap or touch.\"\"\"\n12| return current.start_ms < previous.end_ms", + "token_json": null, + "tool_use_id": "call_muse_timeline", + "ts_ms": 1788223110736 + }, + { + "agent_id": null, + "call_index": null, + "control_kind": null, + "cwd": "/fixture/work", + "error_signal": null, + "event_index": null, + "event_source": null, + "event_uid": "tool:call_muse_note", + "git_branch": null, + "kind": "tool_use", + "message_id": "cb7c9354-f80e-410c-8739-89807a688bed", + "model": "meta/muse-glimmer-30b", + "parent_id": null, + "payload_bytes": null, + "payload_hash": null, + "payload_truncated": null, + "project": "/fixture/work", + "result_status": null, + "role": "assistant", + "session_id": "74747474-7474-4747-8747-747474747474", + "source": "muse", + "subagent_session_id": null, + "text": "read_file CORPUS_NOTE.txt", + "token_json": "{\"cache_read_tokens\":0,\"cache_write_tokens\":0,\"cached_tokens\":0,\"input_tokens\":32,\"output_tokens\":8,\"reasoning_tokens\":0}", + "tool_use_id": null, + "ts_ms": 1788223110756 + }, + { + "agent_id": null, + "call_index": 0, + "control_kind": null, + "cwd": "/fixture/work", + "error_signal": null, + "event_index": 1, + "event_source": "function_call_output", + "event_uid": "result:call_muse_note", + "git_branch": null, + "kind": "tool_result", + "message_id": "result:call_muse_note", + "model": null, + "parent_id": null, + "payload_bytes": 208, + "payload_hash": "2a81dde7d9a0949f", + "payload_truncated": 0, + "project": "/fixture/work", + "result_status": "completed", + "role": "tool_result", + "session_id": "74747474-7474-4747-8747-747474747474", + "source": "muse", + "subagent_session_id": null, + "text": "Read text file `CORPUS_NOTE.txt`.\n1|Native corpus file marker: COPPER_4821\n2|\n3|This file belongs to a disposable session-migrate test workspace. It contains\n4|no credentials, user data, or repository source.", + "token_json": null, + "tool_use_id": "call_muse_note", + "ts_ms": 1788223110761 + }, + { + "agent_id": null, + "call_index": null, + "control_kind": null, + "cwd": "/fixture/work", + "error_signal": null, + "event_index": null, + "event_source": null, + "event_uid": "tool:call_muse_missing", + "git_branch": null, + "kind": "tool_use", + "message_id": "d9a98768-7eaf-4496-b382-be2ce4614480", + "model": "meta/muse-glimmer-30b", + "parent_id": null, + "payload_bytes": null, + "payload_hash": null, + "payload_truncated": null, + "project": "/fixture/work", + "result_status": null, + "role": "assistant", + "session_id": "74747474-7474-4747-8747-747474747474", + "source": "muse", + "subagent_session_id": null, + "text": "read_file missing-corpus-file.txt", + "token_json": "{\"cache_read_tokens\":0,\"cache_write_tokens\":0,\"cached_tokens\":0,\"input_tokens\":32,\"output_tokens\":8,\"reasoning_tokens\":0}", + "tool_use_id": null, + "ts_ms": 1788223110799 + }, + { + "agent_id": null, + "call_index": 0, + "control_kind": null, + "cwd": "/fixture/work", + "error_signal": "tool_batch.effect", + "event_index": 2, + "event_source": "function_call_output", + "event_uid": "result:call_muse_missing", + "git_branch": null, + "kind": "tool_result", + "message_id": "result:call_muse_missing", + "model": null, + "parent_id": null, + "payload_bytes": 51, + "payload_hash": "70d253561a4288c1", + "payload_truncated": 0, + "project": "/fixture/work", + "result_status": "errored", + "role": "tool_result", + "session_id": "74747474-7474-4747-8747-747474747474", + "source": "muse", + "subagent_session_id": null, + "text": "tool failed: No such file or directory (os error 2)", + "token_json": null, + "tool_use_id": "call_muse_missing", + "ts_ms": 1788223110811 + }, + { + "agent_id": null, + "call_index": null, + "control_kind": null, + "cwd": "/fixture/work", + "error_signal": null, + "event_index": null, + "event_source": null, + "event_uid": "698a9efd-8497-477a-9730-7436a0511a92", + "git_branch": null, + "kind": "text", + "message_id": "3a5a3d69-0f3b-4971-af76-38b3e569c76d", + "model": "meta/muse-glimmer-30b", + "parent_id": null, + "payload_bytes": null, + "payload_hash": null, + "payload_truncated": null, + "project": "/fixture/work", + "result_status": null, + "role": "assistant", + "session_id": "74747474-7474-4747-8747-747474747474", + "source": "muse", + "subagent_session_id": null, + "text": "The boundary comparison is strict, the note marker is COPPER_4821, and the missing-file failure was retained.", + "token_json": "{\"cache_read_tokens\":0,\"cache_write_tokens\":0,\"cached_tokens\":0,\"input_tokens\":32,\"output_tokens\":8,\"reasoning_tokens\":0}", + "tool_use_id": null, + "ts_ms": 1788223110828 + }, + { + "agent_id": null, + "call_index": null, + "control_kind": null, + "cwd": "/fixture/work", + "error_signal": null, + "event_index": null, + "event_source": null, + "event_uid": "13488036-3b77-462c-a65d-d2217076ce71", + "git_branch": null, + "kind": "text", + "message_id": "13488036-3b77-462c-a65d-d2217076ce71", + "model": null, + "parent_id": null, + "payload_bytes": null, + "payload_hash": null, + "payload_truncated": null, + "project": "/fixture/work", + "result_status": null, + "role": "user", + "session_id": "74747474-7474-4747-8747-747474747474", + "source": "muse", + "subagent_session_id": null, + "text": "Inspect the attached image and identify its shape and visible nonce.", + "token_json": null, + "tool_use_id": null, + "ts_ms": 1788223111288 + }, + { + "agent_id": null, + "call_index": null, + "control_kind": null, + "cwd": "/fixture/work", + "error_signal": null, + "event_index": null, + "event_source": null, + "event_uid": "d55187fd-db48-4af5-8fb1-6dc1d455fe49", + "git_branch": null, + "kind": "text", + "message_id": "ab7a934d-2bd5-4915-a1ce-daa50835418a", + "model": "meta/muse-glimmer-30b", + "parent_id": null, + "payload_bytes": null, + "payload_hash": null, + "payload_truncated": null, + "project": "/fixture/work", + "result_status": null, + "role": "assistant", + "session_id": "74747474-7474-4747-8747-747474747474", + "source": "muse", + "subagent_session_id": null, + "text": "The image contains a blue triangle labeled BLUE_TRIANGLE_7319.", + "token_json": "{\"cache_read_tokens\":0,\"cache_write_tokens\":0,\"cached_tokens\":0,\"input_tokens\":32,\"output_tokens\":8,\"reasoning_tokens\":0}", + "tool_use_id": null, + "ts_ms": 1788223111332 + }, + { + "agent_id": null, + "call_index": null, + "control_kind": null, + "cwd": "/fixture/work", + "error_signal": null, + "event_index": null, + "event_source": null, + "event_uid": "123e15fd-ff2f-4065-818a-ae2019998fab", + "git_branch": null, + "kind": "text", + "message_id": "123e15fd-ff2f-4065-818a-ae2019998fab", + "model": null, + "parent_id": null, + "payload_bytes": null, + "payload_hash": null, + "payload_truncated": null, + "project": "/fixture/work", + "result_status": null, + "role": "user", + "session_id": "74747474-7474-4747-8747-747474747474", + "source": "muse", + "subagent_session_id": null, + "text": "Without tools, recall the task marker, note marker, image marker, and failed filename.", + "token_json": null, + "tool_use_id": null, + "ts_ms": 1788223111867 + }, + { + "agent_id": null, + "call_index": null, + "control_kind": null, + "cwd": "/fixture/work", + "error_signal": null, + "event_index": null, + "event_source": null, + "event_uid": "ef1c2327-4418-45cd-8259-e4e173983fbf", + "git_branch": null, + "kind": "text", + "message_id": "5726f698-6d80-4a72-bf94-ac460d454acd", + "model": "meta/muse-glimmer-30b", + "parent_id": null, + "payload_bytes": null, + "payload_hash": null, + "payload_truncated": null, + "project": "/fixture/work", + "result_status": null, + "role": "assistant", + "session_id": "74747474-7474-4747-8747-747474747474", + "source": "muse", + "subagent_session_id": null, + "text": "Recall: SM_CORPUS_7319, COPPER_4821, BLUE_TRIANGLE_7319, and missing-corpus-file.txt.", + "token_json": "{\"cache_read_tokens\":0,\"cache_write_tokens\":0,\"cached_tokens\":0,\"input_tokens\":32,\"output_tokens\":8,\"reasoning_tokens\":0}", + "tool_use_id": null, + "ts_ms": 1788223111916 + } + ], + "session_markers": [ + { + "kind": "session_start", + "marker_uid": "8b83613d-78d1-4298-b2a1-8181328e6ec9", + "message_id": null, + "parent_id": null, + "payload_json": "{\"resume\":false}", + "session_id": "74747474-7474-4747-8747-747474747474", + "source": "muse", + "subkind": "session.opened.observed", + "text": null, + "ts_ms": 1788223110669, + "turn_id": null + }, + { + "kind": "turn_end", + "marker_uid": "bcf0bf7c-5a65-4ef3-a337-1e99fc6dec57", + "message_id": null, + "parent_id": null, + "payload_json": "{\"terminal\":\"completed\"}", + "session_id": "74747474-7474-4747-8747-747474747474", + "source": "muse", + "subkind": "terminal", + "text": "completed", + "ts_ms": 1788223110922, + "turn_id": "ffa8d311-da04-4cfc-b100-89c18a58f823" + }, + { + "kind": "session_end", + "marker_uid": "c6f0de74-d6c5-4890-8796-a14b97a7409d", + "message_id": null, + "parent_id": null, + "payload_json": null, + "session_id": "74747474-7474-4747-8747-747474747474", + "source": "muse", + "subkind": "session.end", + "text": "clean", + "ts_ms": 1788223110936, + "turn_id": null + }, + { + "kind": "session_start", + "marker_uid": "7c4404b5-7678-499d-b77e-dcfa66c0dd04", + "message_id": null, + "parent_id": null, + "payload_json": "{\"resume\":true}", + "session_id": "74747474-7474-4747-8747-747474747474", + "source": "muse", + "subkind": "session.opened.observed", + "text": null, + "ts_ms": 1788223111260, + "turn_id": null + }, + { + "kind": "session_resumed", + "marker_uid": "6c14d49a-a545-4649-a13d-22fc84499c99", + "message_id": null, + "parent_id": null, + "payload_json": null, + "session_id": "74747474-7474-4747-8747-747474747474", + "source": "muse", + "subkind": "session.resumed", + "text": null, + "ts_ms": 1788223111261, + "turn_id": null + }, + { + "kind": "turn_end", + "marker_uid": "32f03605-c293-454a-a3f8-c19657fa8ddb", + "message_id": null, + "parent_id": null, + "payload_json": "{\"terminal\":\"completed\"}", + "session_id": "74747474-7474-4747-8747-747474747474", + "source": "muse", + "subkind": "terminal", + "text": "completed", + "ts_ms": 1788223111458, + "turn_id": "28adbeba-47af-4425-b362-0261f207b380" + }, + { + "kind": "session_end", + "marker_uid": "0ea590be-17c7-490c-ac76-b4de0b5ff0e3", + "message_id": null, + "parent_id": null, + "payload_json": null, + "session_id": "74747474-7474-4747-8747-747474747474", + "source": "muse", + "subkind": "session.end", + "text": "clean", + "ts_ms": 1788223111470, + "turn_id": null + }, + { + "kind": "session_start", + "marker_uid": "65b0a57d-b549-468e-bc86-b3b6cbcb02da", + "message_id": null, + "parent_id": null, + "payload_json": "{\"resume\":true}", + "session_id": "74747474-7474-4747-8747-747474747474", + "source": "muse", + "subkind": "session.opened.observed", + "text": null, + "ts_ms": 1788223111841, + "turn_id": null + }, + { + "kind": "session_resumed", + "marker_uid": "d0a1e421-1b6c-4630-8486-a02081af0847", + "message_id": null, + "parent_id": null, + "payload_json": null, + "session_id": "74747474-7474-4747-8747-747474747474", + "source": "muse", + "subkind": "session.resumed", + "text": null, + "ts_ms": 1788223111842, + "turn_id": null + }, + { + "kind": "turn_end", + "marker_uid": "61dbbf75-ac99-4c6b-aa83-d5317b32e8b3", + "message_id": null, + "parent_id": null, + "payload_json": "{\"terminal\":\"completed\"}", + "session_id": "74747474-7474-4747-8747-747474747474", + "source": "muse", + "subkind": "terminal", + "text": "completed", + "ts_ms": 1788223112041, + "turn_id": "0bbf256f-b87e-4cf9-99bb-3d462715e1d8" + }, + { + "kind": "session_end", + "marker_uid": "05e6ada9-1d31-42e9-a030-685d7adef551", + "message_id": null, + "parent_id": null, + "payload_json": null, + "session_id": "74747474-7474-4747-8747-747474747474", + "source": "muse", + "subkind": "session.end", + "text": "clean", + "ts_ms": 1788223112057, + "turn_id": null + } + ], + "session_relationships": [], + "sessions": [ + { + "agent_version": "0.2.1", + "cwd": "/fixture/work", + "discovery_state": "full", + "first_activity_ms": 1788223110587, + "first_prompt": "SM_CORPUS_7319 Inspect timeline.py and CORPUS_NOTE.txt with the native file tool. Then try missing-corpus-file.txt so the native log includes a failed tool result. Explain the interval boundary and quote the note marker.", + "git_branch": null, + "initial_commit": null, + "last_activity_ms": 1788223112057, + "last_assistant_text": "Recall: SM_CORPUS_7319, COPPER_4821, BLUE_TRIANGLE_7319, and missing-corpus-file.txt.", + "models_json": "[\"meta/muse-glimmer-30b\"]", + "originator": null, + "raw_path": "/.local/share/muse/sessions/2026/09/01/74747474-7474-4747-8747-747474747474/session.jsonl", + "repo_url": null, + "session_id": "74747474-7474-4747-8747-747474747474", + "source": "muse", + "source_stamp": ":31189", + "workspace_roots_json": null + } + ], + "source": "muse", + "tool_calls": [ + { + "args_json": "{\"path\":\"timeline.py\"}", + "is_error": 0, + "message_id": "80749233-420e-4c1c-b567-a9047d47c4be", + "name": "read_file", + "session_id": "74747474-7474-4747-8747-747474747474", + "source": "muse", + "target": "timeline.py", + "tool_use_id": "call_muse_timeline", + "ts_ms": 1788223110715 + }, + { + "args_json": "{\"path\":\"CORPUS_NOTE.txt\"}", + "is_error": 0, + "message_id": "cb7c9354-f80e-410c-8739-89807a688bed", + "name": "read_file", + "session_id": "74747474-7474-4747-8747-747474747474", + "source": "muse", + "target": "CORPUS_NOTE.txt", + "tool_use_id": "call_muse_note", + "ts_ms": 1788223110756 + }, + { + "args_json": "{\"path\":\"missing-corpus-file.txt\"}", + "is_error": 1, + "message_id": "d9a98768-7eaf-4496-b382-be2ce4614480", + "name": "read_file", + "session_id": "74747474-7474-4747-8747-747474747474", + "source": "muse", + "target": "missing-corpus-file.txt", + "tool_use_id": "call_muse_missing", + "ts_ms": 1788223110799 + } + ] +} diff --git a/crates/ai-hist/tests/snapshots/muse/tools-session.json b/crates/ai-hist/tests/snapshots/muse/tools-session.json new file mode 100644 index 00000000..b469de4a --- /dev/null +++ b/crates/ai-hist/tests/snapshots/muse/tools-session.json @@ -0,0 +1,899 @@ +{ + "corpus_files": [ + "muse/tools-session" + ], + "discovery_diagnostics": [], + "file_edits": [ + { + "cwd": "/tmp/muse-demo", + "file_path": "src/http.rs", + "git_branch": null, + "lines_added": null, + "lines_removed": null, + "message_id": "msg-2", + "session_id": "muse-a001", + "source": "muse", + "structured_patch_json": null, + "tool_name": "edit_file", + "tool_use_id": "call_edit", + "ts_ms": 1790337602750, + "user_modified": null + }, + { + "cwd": "/tmp/muse-demo", + "file_path": "tests/retry.rs", + "git_branch": null, + "lines_added": null, + "lines_removed": null, + "message_id": "msg-2", + "session_id": "muse-a001", + "source": "muse", + "structured_patch_json": null, + "tool_name": "write_file", + "tool_use_id": "call_write", + "ts_ms": 1790337602750, + "user_modified": null + } + ], + "fixture": "muse/tools-session", + "history": [ + { + "project": "/tmp/muse-demo", + "prompt": "Add a retry helper to src/http.rs and run the tests.", + "session_id": "muse-a001", + "source": "muse", + "timestamp_ms": 1790337600750 + }, + { + "project": "/tmp/muse-demo", + "prompt": "Why did it fail?", + "session_id": "muse-a001", + "source": "muse", + "timestamp_ms": 1790337606750 + } + ], + "hydration": [ + { + "related_session_ids": [ + "muse-c001", + "muse-c002", + "muse-e001" + ], + "session_id": "muse-a001", + "source": "muse", + "status": "hydrated" + } + ], + "layout": "HomeTree", + "notes": [], + "origin": "relayhistory", + "quirk": "authored from the documented shape: a permission frame before the metadata, encrypted and readable reasoning, `edit_file`/`write_file` edits, a `bash` result that exits 101, a mirrored subagent task stream, a mid-session model switch, and `subagent/` logs — a worker with its own nested child, and a reminder — linked as delegated children rather than catalogued as sessions", + "session_events": [ + { + "agent_id": null, + "call_index": null, + "control_kind": null, + "cwd": "/tmp/muse-demo", + "error_signal": null, + "event_index": null, + "event_source": null, + "event_uid": "a001-rec-003", + "git_branch": null, + "kind": "text", + "message_id": "a001-rec-003", + "model": null, + "parent_id": null, + "payload_bytes": null, + "payload_hash": null, + "payload_truncated": null, + "project": "/tmp/muse-demo", + "result_status": null, + "role": "user", + "session_id": "muse-a001", + "source": "muse", + "subagent_session_id": null, + "text": "Add a retry helper to src/http.rs and run the tests.", + "token_json": null, + "tool_use_id": null, + "ts_ms": 1790337600750 + }, + { + "agent_id": null, + "call_index": null, + "control_kind": null, + "cwd": "/tmp/muse-demo", + "error_signal": null, + "event_index": null, + "event_source": null, + "event_uid": "a001-rec-006", + "git_branch": null, + "kind": "thinking", + "message_id": "msg-r2", + "model": "meta/muse-spark-1.3", + "parent_id": null, + "payload_bytes": null, + "payload_hash": null, + "payload_truncated": null, + "project": "/tmp/muse-demo", + "result_status": null, + "role": "assistant", + "session_id": "muse-a001", + "source": "muse", + "subagent_session_id": null, + "text": "Read the file before editing it.", + "token_json": "{\"cache_read_tokens\":800,\"cache_write_tokens\":0,\"cached_tokens\":800,\"input_tokens\":1200,\"output_tokens\":90,\"reasoning_tokens\":40}", + "tool_use_id": null, + "ts_ms": 1790337601500 + }, + { + "agent_id": null, + "call_index": null, + "control_kind": null, + "cwd": "/tmp/muse-demo", + "error_signal": null, + "event_index": null, + "event_source": null, + "event_uid": "tool:call_read", + "git_branch": null, + "kind": "tool_use", + "message_id": "msg-1", + "model": "meta/muse-spark-1.3", + "parent_id": null, + "payload_bytes": null, + "payload_hash": null, + "payload_truncated": null, + "project": "/tmp/muse-demo", + "result_status": null, + "role": "assistant", + "session_id": "muse-a001", + "source": "muse", + "subagent_session_id": null, + "text": "read_file src/http.rs", + "token_json": null, + "tool_use_id": null, + "ts_ms": 1790337601750 + }, + { + "agent_id": null, + "call_index": 0, + "control_kind": null, + "cwd": "/tmp/muse-demo", + "error_signal": null, + "event_index": 0, + "event_source": "function_call_output", + "event_uid": "result:call_read", + "git_branch": null, + "kind": "tool_result", + "message_id": "result:call_read", + "model": null, + "parent_id": null, + "payload_bytes": 47, + "payload_hash": "6b971e600bea89fd", + "payload_truncated": 0, + "project": "/tmp/muse-demo", + "result_status": "completed", + "role": "tool_result", + "session_id": "muse-a001", + "source": "muse", + "subagent_session_id": null, + "text": "Read text file `src/http.rs`.\n1|pub fn get() {}", + "token_json": null, + "tool_use_id": "call_read", + "ts_ms": 1790337602250 + }, + { + "agent_id": null, + "call_index": null, + "control_kind": null, + "cwd": "/tmp/muse-demo", + "error_signal": null, + "event_index": null, + "event_source": null, + "event_uid": "tool:call_bash", + "git_branch": null, + "kind": "tool_use", + "message_id": "msg-2", + "model": "meta/muse-spark-1.3", + "parent_id": null, + "payload_bytes": null, + "payload_hash": null, + "payload_truncated": null, + "project": "/tmp/muse-demo", + "result_status": null, + "role": "assistant", + "session_id": "muse-a001", + "source": "muse", + "subagent_session_id": null, + "text": "bash cargo test", + "token_json": null, + "tool_use_id": null, + "ts_ms": 1790337602750 + }, + { + "agent_id": null, + "call_index": null, + "control_kind": null, + "cwd": "/tmp/muse-demo", + "error_signal": null, + "event_index": null, + "event_source": null, + "event_uid": "tool:call_edit", + "git_branch": null, + "kind": "tool_use", + "message_id": "msg-2", + "model": "meta/muse-spark-1.3", + "parent_id": null, + "payload_bytes": null, + "payload_hash": null, + "payload_truncated": null, + "project": "/tmp/muse-demo", + "result_status": null, + "role": "assistant", + "session_id": "muse-a001", + "source": "muse", + "subagent_session_id": null, + "text": "edit_file src/http.rs", + "token_json": "{\"cache_read_tokens\":1100,\"cache_write_tokens\":0,\"cached_tokens\":1100,\"input_tokens\":1500,\"output_tokens\":210,\"reasoning_tokens\":0}", + "tool_use_id": null, + "ts_ms": 1790337602750 + }, + { + "agent_id": null, + "call_index": null, + "control_kind": null, + "cwd": "/tmp/muse-demo", + "error_signal": null, + "event_index": null, + "event_source": null, + "event_uid": "tool:call_spawn", + "git_branch": null, + "kind": "tool_use", + "message_id": "msg-2", + "model": "meta/muse-spark-1.3", + "parent_id": null, + "payload_bytes": null, + "payload_hash": null, + "payload_truncated": null, + "project": "/tmp/muse-demo", + "result_status": null, + "role": "assistant", + "session_id": "muse-a001", + "source": "muse", + "subagent_session_id": null, + "text": "subagent_spawn reviewer", + "token_json": null, + "tool_use_id": null, + "ts_ms": 1790337602750 + }, + { + "agent_id": null, + "call_index": null, + "control_kind": null, + "cwd": "/tmp/muse-demo", + "error_signal": null, + "event_index": null, + "event_source": null, + "event_uid": "tool:call_write", + "git_branch": null, + "kind": "tool_use", + "message_id": "msg-2", + "model": "meta/muse-spark-1.3", + "parent_id": null, + "payload_bytes": null, + "payload_hash": null, + "payload_truncated": null, + "project": "/tmp/muse-demo", + "result_status": null, + "role": "assistant", + "session_id": "muse-a001", + "source": "muse", + "subagent_session_id": null, + "text": "write_file tests/retry.rs", + "token_json": null, + "tool_use_id": null, + "ts_ms": 1790337602750 + }, + { + "agent_id": null, + "call_index": 0, + "control_kind": null, + "cwd": "/tmp/muse-demo", + "error_signal": "exit_code", + "event_index": 3, + "event_source": "function_call_output", + "event_uid": "result:call_bash", + "git_branch": null, + "kind": "tool_result", + "message_id": "result:call_bash", + "model": null, + "parent_id": null, + "payload_bytes": 150, + "payload_hash": "91b19b6592840e54", + "payload_truncated": 0, + "project": "/tmp/muse-demo", + "result_status": "errored", + "role": "tool_result", + "session_id": "muse-a001", + "source": "muse", + "subagent_session_id": null, + "text": "{\"command\": \"cargo test\", \"description\": null, \"exit_code\": 101, \"terminal_status\": \"exited\", \"output\": \"test retries ... FAILED\", \"truncated\": false}", + "token_json": null, + "tool_use_id": "call_bash", + "ts_ms": 1790337604500 + }, + { + "agent_id": null, + "call_index": 0, + "control_kind": null, + "cwd": "/tmp/muse-demo", + "error_signal": null, + "event_index": 1, + "event_source": "function_call_output", + "event_uid": "result:call_edit", + "git_branch": null, + "kind": "tool_result", + "message_id": "result:call_edit", + "model": null, + "parent_id": null, + "payload_bytes": 52, + "payload_hash": "50b57ec4569c319e", + "payload_truncated": 0, + "project": "/tmp/muse-demo", + "result_status": "completed", + "role": "tool_result", + "session_id": "muse-a001", + "source": "muse", + "subagent_session_id": null, + "text": "edited — diff: -pub fn get() {} +pub fn retry() {}", + "token_json": null, + "tool_use_id": "call_edit", + "ts_ms": 1790337604500 + }, + { + "agent_id": null, + "call_index": 0, + "control_kind": null, + "cwd": "/tmp/muse-demo", + "error_signal": null, + "event_index": 4, + "event_source": "function_call_output", + "event_uid": "result:call_spawn", + "git_branch": null, + "kind": "tool_result", + "message_id": "result:call_spawn", + "model": null, + "parent_id": null, + "payload_bytes": 62, + "payload_hash": "03fcd057c573e487", + "payload_truncated": 0, + "project": "/tmp/muse-demo", + "result_status": "completed", + "role": "tool_result", + "session_id": "muse-a001", + "source": "muse", + "subagent_session_id": null, + "text": "{\"ok\": true, \"task_id\": \"muse-task-d001\", \"status\": \"running\"}", + "token_json": null, + "tool_use_id": "call_spawn", + "ts_ms": 1790337604500 + }, + { + "agent_id": null, + "call_index": 0, + "control_kind": null, + "cwd": "/tmp/muse-demo", + "error_signal": null, + "event_index": 2, + "event_source": "function_call_output", + "event_uid": "result:call_write", + "git_branch": null, + "kind": "tool_result", + "message_id": "result:call_write", + "model": null, + "parent_id": null, + "payload_bytes": 32, + "payload_hash": "72f50bf5459f1f2b", + "payload_truncated": 0, + "project": "/tmp/muse-demo", + "result_status": "completed", + "role": "tool_result", + "session_id": "muse-a001", + "source": "muse", + "subagent_session_id": null, + "text": "wrote 22 bytes to tests/retry.rs", + "token_json": null, + "tool_use_id": "call_write", + "ts_ms": 1790337604500 + }, + { + "agent_id": null, + "call_index": null, + "control_kind": null, + "cwd": "/tmp/muse-demo", + "error_signal": null, + "event_index": null, + "event_source": null, + "event_uid": "a001-rec-020", + "git_branch": null, + "kind": "text", + "message_id": "msg-3", + "model": "meta/muse-spark-1.3", + "parent_id": null, + "payload_bytes": null, + "payload_hash": null, + "payload_truncated": null, + "project": "/tmp/muse-demo", + "result_status": null, + "role": "assistant", + "session_id": "muse-a001", + "source": "muse", + "subagent_session_id": null, + "text": "Added `retry` and a test; `cargo test` fails with exit code 101.", + "token_json": "{\"cache_read_tokens\":1500,\"cache_write_tokens\":0,\"cached_tokens\":1500,\"input_tokens\":1800,\"output_tokens\":35,\"reasoning_tokens\":0}", + "tool_use_id": null, + "ts_ms": 1790337605000 + }, + { + "agent_id": null, + "call_index": null, + "control_kind": null, + "cwd": "/tmp/muse-demo", + "error_signal": null, + "event_index": null, + "event_source": null, + "event_uid": "a001-rec-027", + "git_branch": null, + "kind": "text", + "message_id": "a001-rec-027", + "model": null, + "parent_id": null, + "payload_bytes": null, + "payload_hash": null, + "payload_truncated": null, + "project": "/tmp/muse-demo", + "result_status": null, + "role": "user", + "session_id": "muse-a001", + "source": "muse", + "subagent_session_id": null, + "text": "Why did it fail?", + "token_json": null, + "tool_use_id": null, + "ts_ms": 1790337606750 + }, + { + "agent_id": null, + "call_index": null, + "control_kind": null, + "cwd": "/tmp/muse-demo", + "error_signal": null, + "event_index": null, + "event_source": null, + "event_uid": "a001-rec-029", + "git_branch": null, + "kind": "text", + "message_id": "msg-4", + "model": "meta/muse-spark-1.3-contributor", + "parent_id": null, + "payload_bytes": null, + "payload_hash": null, + "payload_truncated": null, + "project": "/tmp/muse-demo", + "result_status": null, + "role": "assistant", + "session_id": "muse-a001", + "source": "muse", + "subagent_session_id": null, + "text": "`retry` is declared but the test expects it to return a value.", + "token_json": "{\"cache_read_tokens\":1800,\"cache_write_tokens\":0,\"cached_tokens\":1800,\"input_tokens\":2000,\"output_tokens\":20,\"reasoning_tokens\":0}", + "tool_use_id": null, + "ts_ms": 1790337607250 + }, + { + "agent_id": null, + "call_index": null, + "control_kind": null, + "cwd": "/tmp/muse-demo", + "error_signal": null, + "event_index": null, + "event_source": null, + "event_uid": "c001-rec-002", + "git_branch": null, + "kind": "text", + "message_id": "c001-rec-002", + "model": null, + "parent_id": null, + "payload_bytes": null, + "payload_hash": null, + "payload_truncated": null, + "project": "/tmp/muse-demo", + "result_status": null, + "role": "user", + "session_id": "muse-c001", + "source": "muse", + "subagent_session_id": null, + "text": "Role: reviewer Objective: review src/http.rs", + "token_json": null, + "tool_use_id": null, + "ts_ms": 1790337604600 + }, + { + "agent_id": null, + "call_index": null, + "control_kind": null, + "cwd": "/tmp/muse-demo", + "error_signal": null, + "event_index": null, + "event_source": null, + "event_uid": "c001-rec-004", + "git_branch": null, + "kind": "text", + "message_id": "child-msg-1", + "model": "meta/muse-spark-1.3", + "parent_id": null, + "payload_bytes": null, + "payload_hash": null, + "payload_truncated": null, + "project": "/tmp/muse-demo", + "result_status": null, + "role": "assistant", + "session_id": "muse-c001", + "source": "muse", + "subagent_session_id": null, + "text": "`retry` has no body; the test cannot pass.", + "token_json": "{\"cache_read_tokens\":0,\"cache_write_tokens\":0,\"cached_tokens\":0,\"input_tokens\":600,\"output_tokens\":25,\"reasoning_tokens\":0}", + "tool_use_id": null, + "ts_ms": 1790337605100 + }, + { + "agent_id": null, + "call_index": null, + "control_kind": null, + "cwd": "/tmp/muse-demo", + "error_signal": null, + "event_index": null, + "event_source": null, + "event_uid": "c002-rec-002", + "git_branch": null, + "kind": "text", + "message_id": "c002-rec-002", + "model": null, + "parent_id": null, + "payload_bytes": null, + "payload_hash": null, + "payload_truncated": null, + "project": "/tmp/muse-demo", + "result_status": null, + "role": "user", + "session_id": "muse-c002", + "source": "muse", + "subagent_session_id": null, + "text": "Role: linter Objective: lint src/http.rs", + "token_json": null, + "tool_use_id": null, + "ts_ms": 1790337606100 + }, + { + "agent_id": null, + "call_index": null, + "control_kind": null, + "cwd": "/tmp/muse-demo", + "error_signal": null, + "event_index": null, + "event_source": null, + "event_uid": "c002-rec-003", + "git_branch": null, + "kind": "text", + "message_id": "grand-msg-1", + "model": "meta/muse-glimmer-30b", + "parent_id": null, + "payload_bytes": null, + "payload_hash": null, + "payload_truncated": null, + "project": "/tmp/muse-demo", + "result_status": null, + "role": "assistant", + "session_id": "muse-c002", + "source": "muse", + "subagent_session_id": null, + "text": "No lint findings.", + "token_json": null, + "tool_use_id": null, + "ts_ms": 1790337606350 + }, + { + "agent_id": null, + "call_index": null, + "control_kind": null, + "cwd": "/tmp/muse-demo", + "error_signal": null, + "event_index": null, + "event_source": null, + "event_uid": "e001-rec-002", + "git_branch": null, + "kind": "text", + "message_id": "e001-rec-002", + "model": null, + "parent_id": null, + "payload_bytes": null, + "payload_hash": null, + "payload_truncated": null, + "project": "/tmp/muse-demo", + "result_status": null, + "role": "user", + "session_id": "muse-e001", + "source": "muse", + "subagent_session_id": null, + "text": "Is the user's task finished?", + "token_json": null, + "tool_use_id": null, + "ts_ms": 1790337605900 + }, + { + "agent_id": null, + "call_index": null, + "control_kind": null, + "cwd": "/tmp/muse-demo", + "error_signal": null, + "event_index": null, + "event_source": null, + "event_uid": "e001-rec-003", + "git_branch": null, + "kind": "text", + "message_id": "remind-msg-1", + "model": "meta/muse-spark-1.3", + "parent_id": null, + "payload_bytes": null, + "payload_hash": null, + "payload_truncated": null, + "project": "/tmp/muse-demo", + "result_status": null, + "role": "assistant", + "session_id": "muse-e001", + "source": "muse", + "subagent_session_id": null, + "text": "No: the test still fails.", + "token_json": null, + "tool_use_id": null, + "ts_ms": 1790337606150 + } + ], + "session_markers": [ + { + "kind": "session_start", + "marker_uid": "a001-rec-002", + "message_id": null, + "parent_id": null, + "payload_json": "{\"resume\":false}", + "session_id": "muse-a001", + "source": "muse", + "subkind": "session.opened.observed", + "text": null, + "ts_ms": 1790337600500, + "turn_id": null + }, + { + "kind": "encrypted_reasoning", + "marker_uid": "a001-rec-005", + "message_id": "msg-r1", + "parent_id": null, + "payload_json": null, + "session_id": "muse-a001", + "source": "muse", + "subkind": "reasoning_committed", + "text": null, + "ts_ms": 1790337601250, + "turn_id": "run-a" + }, + { + "kind": "turn_end", + "marker_uid": "a001-rec-022", + "message_id": null, + "parent_id": null, + "payload_json": "{\"terminal\":\"completed\"}", + "session_id": "muse-a001", + "source": "muse", + "subkind": "terminal", + "text": "completed", + "ts_ms": 1790337605500, + "turn_id": "run-a" + }, + { + "kind": "session_end", + "marker_uid": "a001-rec-023", + "message_id": null, + "parent_id": null, + "payload_json": null, + "session_id": "muse-a001", + "source": "muse", + "subkind": "session.end", + "text": "clean", + "ts_ms": 1790337605750, + "turn_id": null + }, + { + "kind": "session_start", + "marker_uid": "a001-rec-024", + "message_id": null, + "parent_id": null, + "payload_json": "{\"resume\":true}", + "session_id": "muse-a001", + "source": "muse", + "subkind": "session.opened.observed", + "text": null, + "ts_ms": 1790337606000, + "turn_id": null + }, + { + "kind": "session_resumed", + "marker_uid": "a001-rec-025", + "message_id": null, + "parent_id": null, + "payload_json": null, + "session_id": "muse-a001", + "source": "muse", + "subkind": "session.resumed", + "text": null, + "ts_ms": 1790337606250, + "turn_id": null + }, + { + "kind": "model_switch", + "marker_uid": "a001-rec-026", + "message_id": null, + "parent_id": null, + "payload_json": null, + "session_id": "muse-a001", + "source": "muse", + "subkind": "runtime.session.metadata", + "text": "meta/muse-spark-1.3-contributor", + "ts_ms": 1790337606500, + "turn_id": null + }, + { + "kind": "turn_end", + "marker_uid": "a001-rec-030", + "message_id": null, + "parent_id": null, + "payload_json": "{\"reason\":\"user_cancelled\",\"terminal\":\"interrupted\"}", + "session_id": "muse-a001", + "source": "muse", + "subkind": "terminal", + "text": "interrupted", + "ts_ms": 1790337607500, + "turn_id": "run-b" + } + ], + "session_relationships": [ + { + "child_agent_name": "reviewer", + "child_agent_type": "worker", + "child_has_events": 1, + "child_model": "meta/muse-spark-1.3", + "child_session_id": "muse-c001", + "evidence_kind": "muse_subagent_log", + "evidence_locator": "/.local/share/muse/sessions/2026/09/20/muse-a001/subagent/muse-c001/session.jsonl", + "evidence_ref": "muse-task-d001", + "identity_status": "observed", + "parent_session_id": "muse-a001", + "relationship": "delegated", + "relationship_uid": "child:muse-c001", + "source": "muse", + "spawn_depth": 1, + "spawned_at_ms": 1790337604000 + }, + { + "child_agent_name": "plugin:tbh-reminders:goal-reminder", + "child_agent_type": "reminder", + "child_has_events": 1, + "child_model": "meta/muse-spark-1.3", + "child_session_id": "muse-e001", + "evidence_kind": "muse_subagent_log", + "evidence_locator": "/.local/share/muse/sessions/2026/09/20/muse-a001/subagent/muse-e001/session.jsonl", + "evidence_ref": "muse-task-d002", + "identity_status": "observed", + "parent_session_id": "muse-a001", + "relationship": "delegated", + "relationship_uid": "child:muse-e001", + "source": "muse", + "spawn_depth": 1, + "spawned_at_ms": 1790337605250 + }, + { + "child_agent_name": "linter", + "child_agent_type": "worker", + "child_has_events": 1, + "child_model": "meta/muse-glimmer-30b", + "child_session_id": "muse-c002", + "evidence_kind": "muse_subagent_log", + "evidence_locator": "/.local/share/muse/sessions/2026/09/20/muse-a001/subagent/muse-c001/subagent/muse-c002/session.jsonl", + "evidence_ref": "muse-task-d003", + "identity_status": "observed", + "parent_session_id": "muse-c001", + "relationship": "delegated", + "relationship_uid": "child:muse-c002", + "source": "muse", + "spawn_depth": 2, + "spawned_at_ms": 1790337605350 + } + ], + "sessions": [ + { + "agent_version": "1.3.0", + "cwd": "/tmp/muse-demo", + "discovery_state": "full", + "first_activity_ms": 1790337600250, + "first_prompt": "Add a retry helper to src/http.rs and run the tests.", + "git_branch": null, + "initial_commit": null, + "last_activity_ms": 1790337607500, + "last_assistant_text": "`retry` is declared but the test expects it to return a value.", + "models_json": "[\"meta/muse-spark-1.3\",\"meta/muse-spark-1.3-contributor\"]", + "originator": null, + "raw_path": "/.local/share/muse/sessions/2026/09/20/muse-a001/session.jsonl", + "repo_url": null, + "session_id": "muse-a001", + "source": "muse", + "source_stamp": ":15581|subagent/muse-c001/session.jsonl=:2880|subagent/muse-c001/subagent/muse-c002/session.jsonl=:1355|subagent/muse-e001/session.jsonl=:1352", + "workspace_roots_json": null + } + ], + "source": "muse", + "tool_calls": [ + { + "args_json": "{\"path\":\"src/http.rs\"}", + "is_error": 0, + "message_id": "msg-1", + "name": "read_file", + "session_id": "muse-a001", + "source": "muse", + "target": "src/http.rs", + "tool_use_id": "call_read", + "ts_ms": 1790337601750 + }, + { + "args_json": "{\"command\":\"cargo test\",\"workdir\":\"/tmp/muse-demo\"}", + "is_error": 1, + "message_id": "msg-2", + "name": "bash", + "session_id": "muse-a001", + "source": "muse", + "target": "cargo test", + "tool_use_id": "call_bash", + "ts_ms": 1790337602750 + }, + { + "args_json": "{\"find\":\"pub fn get() {}\",\"path\":\"src/http.rs\",\"replace\":\"pub fn get() {}\\npub fn retry() {}\"}", + "is_error": 0, + "message_id": "msg-2", + "name": "edit_file", + "session_id": "muse-a001", + "source": "muse", + "target": "src/http.rs", + "tool_use_id": "call_edit", + "ts_ms": 1790337602750 + }, + { + "args_json": "{\"objective\":\"review src/http.rs\",\"role\":\"reviewer\"}", + "is_error": 0, + "message_id": "msg-2", + "name": "subagent_spawn", + "session_id": "muse-a001", + "source": "muse", + "target": "reviewer", + "tool_use_id": "call_spawn", + "ts_ms": 1790337602750 + }, + { + "args_json": "{\"content\":\"#[test] fn retries() {}\",\"path\":\"tests/retry.rs\"}", + "is_error": 0, + "message_id": "msg-2", + "name": "write_file", + "session_id": "muse-a001", + "source": "muse", + "target": "tests/retry.rs", + "tool_use_id": "call_write", + "ts_ms": 1790337602750 + } + ] +} diff --git a/docs/architecture.md b/docs/architecture.md index 451a9586..d32b204f 100644 --- a/docs/architecture.md +++ b/docs/architecture.md @@ -242,6 +242,7 @@ so every result reports it: |---|---|---|---|---| | `codex` | always | yes | yes | yes | | `claude` | sometimes (only versions that emit a per-child `agentId`) | yes | yes | yes | +| `muse` | always | yes | yes | yes | | `cursor`, `grok`, `opencode`, `relay` | never | no | no | no | A linked child's events are stored under the child's own session id and are diff --git a/docs/decisions/2026-09-19-relayhistory-owns-session-sourcing.md b/docs/decisions/2026-09-19-relayhistory-owns-session-sourcing.md index 6746aede..936d98a2 100644 --- a/docs/decisions/2026-09-19-relayhistory-owns-session-sourcing.md +++ b/docs/decisions/2026-09-19-relayhistory-owns-session-sourcing.md @@ -18,6 +18,7 @@ Two repositories parse the same harness logs today, with no shared code. | OpenCode | `sync_opencode_session` (`src/store.rs`) | `reader/opencode.rs` | | Cursor | `ingest_cursor_transcript` (`src/ingest.rs`) | not supported | | Grok | `ingest_grok_session` (`src/ingest.rs`) | not supported | +| Muse Code | `ingest_muse_session` (`src/ingest.rs`) | not supported | burn has no dependency on relayhistory. relayhistory already defers cost to burn by name (the upload plugin's `convergence.rs` — "Input excludes @@ -127,29 +128,29 @@ provider sessions") that lands in the `trajectories` table and is filtered out of `sessions list`. It is listed for completeness; every session-evidence row is `—` for it by construction. -| Record type | claude | codex | cursor | grok | opencode | relay | trajectory | Owner / closes | -| ------------------------------------------------------------------ | ------ | ----- | ------ | ---- | -------- | ----- | ---------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| Prompt / `history` row | ✓ | ✓ | ✓ | ◐ | ✓ | ✓ | — | grok timestamps: [#167](https://github.com/AgentWorkforce/relayhistory/issues/167) | -| `session_events` — `text` | ✓ | ✓ | ✓ | ✗ | ✓ | ✗ | — | [#166](https://github.com/AgentWorkforce/relayhistory/issues/166) / [#167](https://github.com/AgentWorkforce/relayhistory/issues/167) / [#168](https://github.com/AgentWorkforce/relayhistory/issues/168) / [#177](https://github.com/AgentWorkforce/relayhistory/issues/177) | -| `session_events` — `thinking` | ✓ | ✓ | — | ✗ | ✗ | ✗ | — | [#166](https://github.com/AgentWorkforce/relayhistory/issues/166) / [#167](https://github.com/AgentWorkforce/relayhistory/issues/167) / [#168](https://github.com/AgentWorkforce/relayhistory/issues/168) | -| `session_events` — `tool_use` | ✓ | ✓ | ✓ | ✗ | ✓ | ✗ | — | [#166](https://github.com/AgentWorkforce/relayhistory/issues/166) / [#167](https://github.com/AgentWorkforce/relayhistory/issues/167) / [#168](https://github.com/AgentWorkforce/relayhistory/issues/168) | -| `session_events` — `tool_result` | ✓ | ✓ | — | ✗ | ✓ | ✗ | — | [#166](https://github.com/AgentWorkforce/relayhistory/issues/166) / [#167](https://github.com/AgentWorkforce/relayhistory/issues/167) / [#168](https://github.com/AgentWorkforce/relayhistory/issues/168) | -| Model, per event | ✓ | ✓ | — | ◐ | ✓ | ✗ | — | [#166](https://github.com/AgentWorkforce/relayhistory/issues/166) / [#167](https://github.com/AgentWorkforce/relayhistory/issues/167) / [#168](https://github.com/AgentWorkforce/relayhistory/issues/168) | -| Token usage | ✓ | ◐ | — | ◐ | ✓ | ✗ | — | [#172](https://github.com/AgentWorkforce/relayhistory/issues/172) (supersedes #99) / [#168](https://github.com/AgentWorkforce/relayhistory/issues/168) / grok: [#212](https://github.com/AgentWorkforce/relayhistory/issues/212) | -| `request_id` | ✗ | — | — | — | — | — | — | [#164](https://github.com/AgentWorkforce/relayhistory/issues/164) | -| `stop_reason` | ✗ | — | — | — | ✓ | — | — | [#164](https://github.com/AgentWorkforce/relayhistory/issues/164) / [#168](https://github.com/AgentWorkforce/relayhistory/issues/168) | -| Codex `turn_id` | — | ✗ | — | — | — | — | — | [#164](https://github.com/AgentWorkforce/relayhistory/issues/164) | -| Sidechain / meta flags | ✗ | ✗ | — | — | — | — | — | [#164](https://github.com/AgentWorkforce/relayhistory/issues/164) | -| Tool calls (`tool_calls`) | ✓ | ✓ | ✓ | ✗ | ✓ | ✗ | — | [#166](https://github.com/AgentWorkforce/relayhistory/issues/166) / [#167](https://github.com/AgentWorkforce/relayhistory/issues/167) / [#168](https://github.com/AgentWorkforce/relayhistory/issues/168) | -| Tool-result fidelity (identity, bytes, truncation, hash, ordering) | ✓ | ✓ | — | ✗ | ✗ | ✗ | — | [#171](https://github.com/AgentWorkforce/relayhistory/issues/171) | -| File edits (`file_edits`) | ✓ | ✓ | ✓ | ✗ | ✓ | ✗ | — | [#166](https://github.com/AgentWorkforce/relayhistory/issues/166) / [#167](https://github.com/AgentWorkforce/relayhistory/issues/167) / [#168](https://github.com/AgentWorkforce/relayhistory/issues/168) | -| Compaction / summary markers | ✗ | ✗ | ✗ | ✗ | ✓ | ✗ | — | [#165](https://github.com/AgentWorkforce/relayhistory/issues/165) / [#168](https://github.com/AgentWorkforce/relayhistory/issues/168) | -| Control / lifecycle rows | ✓ | ✓ | ✗ | ✗ | ✗ | ✗ | — | [#165](https://github.com/AgentWorkforce/relayhistory/issues/165), [#180](https://github.com/AgentWorkforce/relayhistory/issues/180) | -| Relationship — delegated | ◐ | ✓ | — | — | ✓ | — | — | [#170](https://github.com/AgentWorkforce/relayhistory/issues/170) / [#168](https://github.com/AgentWorkforce/relayhistory/issues/168) | -| Relationship — fork / resume / continuation | ✗ | ◐ | ✗ | ✗ | ✗ | ✗ | — | [#170](https://github.com/AgentWorkforce/relayhistory/issues/170) / codex replay gate: [#210](https://github.com/AgentWorkforce/relayhistory/issues/210) | -| Session metadata (cwd, branch, versions) | ◐ | ✓ | ◐ | ◐ | ◐ | ✗ | — | [#164](https://github.com/AgentWorkforce/relayhistory/issues/164), [#177](https://github.com/AgentWorkforce/relayhistory/issues/177) | -| Canonical `project_key` | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | — | [#175](https://github.com/AgentWorkforce/relayhistory/issues/175) | -| Declared hydration `capability` | ◐ | ◐ | ◐ | ◐ | ◐ | ✗ | — | [#169](https://github.com/AgentWorkforce/relayhistory/issues/169) | +| Record type | claude | codex | cursor | grok | muse | opencode | relay | trajectory | Owner / closes | +| ------------------------------------------------------------------ | ------ | ----- | ------ | ---- | ---- | -------- | ----- | ---------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| Prompt / `history` row | ✓ | ✓ | ✓ | ◐ | ✓ | ✓ | ✓ | — | grok timestamps: [#167](https://github.com/AgentWorkforce/relayhistory/issues/167) | +| `session_events` — `text` | ✓ | ✓ | ✓ | ✗ | ✓ | ✓ | ✗ | — | [#166](https://github.com/AgentWorkforce/relayhistory/issues/166) / [#167](https://github.com/AgentWorkforce/relayhistory/issues/167) / [#168](https://github.com/AgentWorkforce/relayhistory/issues/168) / [#177](https://github.com/AgentWorkforce/relayhistory/issues/177) | +| `session_events` — `thinking` | ✓ | ✓ | — | ✗ | ◐ | ✗ | ✗ | — | [#166](https://github.com/AgentWorkforce/relayhistory/issues/166) / [#167](https://github.com/AgentWorkforce/relayhistory/issues/167) / [#168](https://github.com/AgentWorkforce/relayhistory/issues/168) | +| `session_events` — `tool_use` | ✓ | ✓ | ✓ | ✗ | ✓ | ✓ | ✗ | — | [#166](https://github.com/AgentWorkforce/relayhistory/issues/166) / [#167](https://github.com/AgentWorkforce/relayhistory/issues/167) / [#168](https://github.com/AgentWorkforce/relayhistory/issues/168) | +| `session_events` — `tool_result` | ✓ | ✓ | — | ✗ | ✓ | ✓ | ✗ | — | [#166](https://github.com/AgentWorkforce/relayhistory/issues/166) / [#167](https://github.com/AgentWorkforce/relayhistory/issues/167) / [#168](https://github.com/AgentWorkforce/relayhistory/issues/168) | +| Model, per event | ✓ | ✓ | — | ◐ | ✓ | ✓ | ✗ | — | [#166](https://github.com/AgentWorkforce/relayhistory/issues/166) / [#167](https://github.com/AgentWorkforce/relayhistory/issues/167) / [#168](https://github.com/AgentWorkforce/relayhistory/issues/168) | +| Token usage | ✓ | ◐ | — | ◐ | ✓ | ✓ | ✗ | — | [#172](https://github.com/AgentWorkforce/relayhistory/issues/172) (supersedes #99) / [#168](https://github.com/AgentWorkforce/relayhistory/issues/168) / grok: [#212](https://github.com/AgentWorkforce/relayhistory/issues/212) | +| `request_id` | ✗ | — | — | — | ✓ | — | — | — | [#164](https://github.com/AgentWorkforce/relayhistory/issues/164) | +| `stop_reason` | ✗ | — | — | — | ✓ | ✓ | — | — | [#164](https://github.com/AgentWorkforce/relayhistory/issues/164) / [#168](https://github.com/AgentWorkforce/relayhistory/issues/168) | +| Codex `turn_id` | — | ✗ | — | — | — | — | — | — | [#164](https://github.com/AgentWorkforce/relayhistory/issues/164) | +| Sidechain / meta flags | ✗ | ✗ | — | — | — | — | — | — | [#164](https://github.com/AgentWorkforce/relayhistory/issues/164) | +| Tool calls (`tool_calls`) | ✓ | ✓ | ✓ | ✗ | ✓ | ✓ | ✗ | — | [#166](https://github.com/AgentWorkforce/relayhistory/issues/166) / [#167](https://github.com/AgentWorkforce/relayhistory/issues/167) / [#168](https://github.com/AgentWorkforce/relayhistory/issues/168) | +| Tool-result fidelity (identity, bytes, truncation, hash, ordering) | ✓ | ✓ | — | ✗ | ✓ | ✗ | ✗ | — | [#171](https://github.com/AgentWorkforce/relayhistory/issues/171) | +| File edits (`file_edits`) | ✓ | ✓ | ✓ | ✗ | ✓ | ✓ | ✗ | — | [#166](https://github.com/AgentWorkforce/relayhistory/issues/166) / [#167](https://github.com/AgentWorkforce/relayhistory/issues/167) / [#168](https://github.com/AgentWorkforce/relayhistory/issues/168) | +| Compaction / summary markers | ✗ | ✗ | ✗ | ✗ | ✗ | ✓ | ✗ | — | [#165](https://github.com/AgentWorkforce/relayhistory/issues/165) / [#168](https://github.com/AgentWorkforce/relayhistory/issues/168) | +| Control / lifecycle rows | ✓ | ✓ | ✗ | ✗ | ✓ | ✗ | ✗ | — | [#165](https://github.com/AgentWorkforce/relayhistory/issues/165), [#180](https://github.com/AgentWorkforce/relayhistory/issues/180) | +| Relationship — delegated | ◐ | ✓ | — | — | ✓ | ✓ | — | — | [#170](https://github.com/AgentWorkforce/relayhistory/issues/170) / [#168](https://github.com/AgentWorkforce/relayhistory/issues/168) | +| Relationship — fork / resume / continuation | ✗ | ◐ | ✗ | ✗ | ✗ | ✗ | ✗ | — | [#170](https://github.com/AgentWorkforce/relayhistory/issues/170) / codex replay gate: [#210](https://github.com/AgentWorkforce/relayhistory/issues/210) | +| Session metadata (cwd, branch, versions) | ◐ | ✓ | ◐ | ◐ | ◐ | ◐ | ✗ | — | [#164](https://github.com/AgentWorkforce/relayhistory/issues/164), [#177](https://github.com/AgentWorkforce/relayhistory/issues/177) | +| Canonical `project_key` | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | — | [#175](https://github.com/AgentWorkforce/relayhistory/issues/175) | +| Declared hydration `capability` | ◐ | ◐ | ◐ | ◐ | ◐ | ◐ | ✗ | — | [#169](https://github.com/AgentWorkforce/relayhistory/issues/169) | ### Why each non-`✓` cell reads the way it does @@ -186,6 +187,26 @@ different reason — a `Task` block is captured as an ordinary `tool_calls` row, so the spawn is visible, but it names no child transcript, so there is nothing to write a relationship to. +**Muse Code.** `ingest_muse_session` (`src/ingest.rs`, record interpretation +in `src/ingest/muse.rs`) reads `session.jsonl`, an event-sourced log whose +every record carries its own `recorded_at` in microseconds, so no timestamp is +inferred. `thinking` is `◐` because Muse usually keeps reasoning only as +`encrypted_content`; that becomes an `encrypted_reasoning` marker, never +thinking text. Token usage is one `model_completed` object per model step, +stored once on the assistant row that step committed (`per-request`). Tool +steps log it before their calls and prose steps after their reply, so the +pairing works in both directions within a run; a step that committed nothing +is reported as `MUSE_USAGE_UNATTACHED`, not attached to a neighbour. `request_id` is Muse's `response_id` and +`stop_reason` its `finish_reason`. A tool call's status comes from the +separate `tool_batch.effect.terminal` record, joined by call id, and a +`bash` call that completed with a non-zero `exit_code` is an error. Session +metadata is `◐`: `workspace_root`, model and CLI version, but Muse records no +branch. Delegation is `✓`: each `subagent//session.jsonl` beside a +session is indexed under the id its own metadata names and linked as +`delegated` (`evidence_kind = "muse_subagent_log"`), typed from the parent's +`task_stream_linked` record, at any nesting depth. The fixtures include a transcript the real CLI +wrote (`tests/fixtures/muse/cli-capture`). + **Token usage.** Claude is `◐`: `ingest_claude_transcript_as` serializes `message.usage` once and passes the same `token_json` to every block of the message, so one model request is written N times, once per content block. diff --git a/docs/session-catalog.md b/docs/session-catalog.md index d947aac3..42bebe25 100644 --- a/docs/session-catalog.md +++ b/docs/session-catalog.md @@ -367,7 +367,7 @@ contract: | Field | Kind | Notes | |---|---|---| -| `source` | observed | `claude`, `codex`, `cursor`, `grok`, `opencode`, `relay` | +| `source` | observed | `claude`, `codex`, `cursor`, `grok`, `muse`, `opencode`, `relay` | | `session_id` | observed | provider-native; `(source, session_id)` is the primary key, so the same native id under two providers is two rows | | `cwd` | observed | working directory the provider recorded | | `git_branch` | observed | last branch the provider recorded | @@ -463,6 +463,7 @@ which this table must stay consistent with. | **codex** | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | | **cursor** | ✓ (dir name) | ✓ (decoded path) | – (never) | ✓ (injected ``) | ✓ (injected ``, else mtime) | ✓ | ✓ (if a build writes `message.model`) | – | – | – | – | – | | **grok** | ✓ | ✓ | ✓ | ✓ (`updates.jsonl`, else `summary.json`) | ✓ (`updates.jsonl`, else `summary.json`) | ✓ | ✓ | – | – | – | – | – | +| **muse** | ✓ (metadata `stream.id`) | ✓ (`workspace_root`) | – (never) | ✓ (metadata `recorded_at`) | ✓ (tail `recorded_at`) | ✓ (first `started` prompt) | ✓ (metadata, then `model_completed`) | – | ✓ (`build.semver`) | – | – | – | | **opencode** | ✓ | ✓ (directory / message `path.cwd`) | – | ✓ | ✓ | ✓ | ✓ (`providerID/modelID`) | – | – | – | – | – | | **relay** | ✓ | – (never) | – | ✓ (synced min ts) | ✓ (synced max ts) | ✓ (earliest synced prompt) | – | – | – | – | – | – | @@ -491,26 +492,31 @@ keys would silently drop whatever it adds next, which is the failure this table exists to end. Either way the bound is the promise the column makes, and it holds for every kind. -| `kind` | claude | codex | grok | `subkind` examples | -|---|---|---|---|---| -| `compaction_boundary` | ✓ `type:"system"`, `subtype:"compact_boundary"` | ✓ top-level `compacted`, `context_compacted` | ✓ | `compact_boundary`, `compacted` | -| `summary` | ✓ `type:"summary"` | – | – | `summary` | -| `subagent_notification` | ✓ system rows with `parent_tool_use_id`; tool results carrying `toolUseResult.agentId` | ✓ `subagent_*` | – | `subagent_completed`, `tool_use_result_agent_id`, `subagent_message_complete` | -| `task_started` | – | ✓ | – | `task_started` | -| `task_complete` | – | ✓ | – | `task_complete` | -| `turn_diff` | – | ✓ | – | `turn_diff` | -| `stream_error` | – | ✓ | – | `stream_error` | -| `tool_begin` | – | ✓ any `*_begin` | – | `exec_command_begin`, `patch_apply_begin`, `mcp_tool_call_begin` | -| `review_mode` | – | ✓ | – | `entered_review_mode`, `exited_review_mode` | -| `unsupported_block` | ✓ any content block with no event `kind`, plus thinking signatures | – | – | `image`, `document`, `redacted_thinking`, `server_tool_use`, `thinking_signature` | -| `encrypted_reasoning` | – | ✓ `response_item/reasoning` | ✓ an opaque reasoning trace with no summary | `reasoning` | -| `tool_replacement` | ✓ `_meta.replaces` / `_meta.collapsedCalls` | – | – | `tool_result` | -| `system` | – | – | ✓ a system preamble, in `text` | – | -| `synthetic_turn` | – | – | ✓ a turn the harness wrote, in `text` | – | -| `signals` | – | – | ✓ | – | -| `prompt_context` | – | – | ✓ | – | -| `local_notice` | ✓ an `assistant` record whose model is ``: a notice Claude Code wrote itself (API error, expired login), text in `text`, `error` and `is_api_error_message` in `payload_json` | – | – | `synthetic` | -| `unknown` | ✓ any unclassified record type, plus a `user`/`assistant` record that produced no event at all | ✓ any unclassified payload type, including `agent_reasoning_raw_content` and `agent_reasoning_section_break` | – | the provider type, verbatim | +| `kind` | claude | codex | grok | muse | `subkind` examples | +|---|---|---|---|---|---| +| `compaction_boundary` | ✓ `type:"system"`, `subtype:"compact_boundary"` | ✓ top-level `compacted`, `context_compacted` | ✓ | – | `compact_boundary`, `compacted` | +| `summary` | ✓ `type:"summary"` | – | – | – | `summary` | +| `subagent_notification` | ✓ system rows with `parent_tool_use_id`; tool results carrying `toolUseResult.agentId` | ✓ `subagent_*` | – | – | `subagent_completed`, `tool_use_result_agent_id`, `subagent_message_complete` | +| `task_started` | – | ✓ | – | – | `task_started` | +| `task_complete` | – | ✓ | – | – | `task_complete` | +| `turn_diff` | – | ✓ | – | – | `turn_diff` | +| `stream_error` | – | ✓ | – | – | `stream_error` | +| `tool_begin` | – | ✓ any `*_begin` | – | – | `exec_command_begin`, `patch_apply_begin`, `mcp_tool_call_begin` | +| `review_mode` | – | ✓ | – | – | `entered_review_mode`, `exited_review_mode` | +| `unsupported_block` | ✓ any content block with no event `kind`, plus thinking signatures | – | – | – | `image`, `document`, `redacted_thinking`, `server_tool_use`, `thinking_signature` | +| `encrypted_reasoning` | – | ✓ `response_item/reasoning` | ✓ an opaque reasoning trace with no summary | ✓ `reasoning_committed` with only `encrypted_content` | `reasoning` | +| `tool_replacement` | ✓ `_meta.replaces` / `_meta.collapsedCalls` | – | – | – | `tool_result` | +| `system` | – | – | ✓ a system preamble, in `text` | – | – | +| `synthetic_turn` | – | – | ✓ a turn the harness wrote, in `text` | – | – | +| `signals` | – | – | ✓ | – | – | +| `prompt_context` | – | – | ✓ | – | – | +| `local_notice` | ✓ an `assistant` record whose model is ``: a notice Claude Code wrote itself (API error, expired login), text in `text`, `error` and `is_api_error_message` in `payload_json` | – | – | – | `synthetic` | +| `unknown` | ✓ any unclassified record type, plus a `user`/`assistant` record that produced no event at all | ✓ any unclassified payload type, including `agent_reasoning_raw_content` and `agent_reasoning_section_break` | – | – | the provider type, verbatim | +| `turn_end` | – | – | – | ✓ run `terminal` (`completed`, `interrupted`, …) in `text` | `terminal` | +| `session_start` | – | – | – | ✓ `session.opened.observed`, `resume` in the payload | `session.opened.observed` | +| `session_resumed` | – | – | – | ✓ | `session.resumed` | +| `session_end` | – | – | – | ✓ `exit_reason` in `text` | `session.end` | +| `model_switch` | – | – | – | ✓ a later metadata record, the new model in `text` | `runtime.session.metadata` | `text` is the provider's own readable prose for a marker, and `payload_json` its structure; a marker may carry either, both or neither. Grok records a @@ -596,6 +602,7 @@ status, because a fabricated measurement reads exactly like a real one: | **cursor** | ✓ (raw block `content`) | ✓ (harness markers) | ✓ | ✓ | `tool_result` | `tool_result.is_error` | – (no notification rail) | | **grok** | ✓ (raw line `content`) | ✓ (harness markers) | ✓ | ✓ (`unknown` with neither signal) | `function_call_output` | `tool_result.is_error`, `tool_status` | – (no notification rail) | | **opencode** | ✓ (raw part `output`) | ✓ (harness markers) | ✓ | ✓ | `function_call_output` | `exit_code`, `tool_status` | – (no notification rail) | +| **muse** | ✓ (result `text`) | ✓ (harness markers) | ✓ | ✓ (`tool_batch.effect.terminal`, joined by call id) | `function_call_output` | `tool_batch.effect`, `exit_code` (`bash`) | – | | **relay** | – | – | – | – | – | – | – | `payload_bytes` is the raw UTF-8 length of what the provider handed back — @@ -680,6 +687,7 @@ follows. | **codex** | ✓ | ✓ | ✓ | ✓ | ✓ when the bounded child search is complete | `full` or `partial` | | **cursor** | ✓ | ✓ | ✓ | ✓ | – (never: a `Task` block names no child transcript) | `partial` | | **grok** | ✓ | ✓ | ✓ | ✓ | ✓ | `full` | +| **muse** | ✓ | ✓ | ✓ | ✓ | ✓ | `full` | | **opencode** | ✓ | ✓ | ✓ | ✓ | ✓ | `full` | | **relay** | – | – | – | – | – | targeted hydration unsupported | @@ -696,6 +704,7 @@ in flight. | **claude** | ✓ (`requestId`) | ✓ (`message.stop_reason`) | ✓ (`version` / `sourceVersion`) | ✓ (`isSidechain`) | ✓ (`isMeta`) | – | | **codex** | – | – | – | – | – | ✓ (`turn_context.turn_id`, carried to the next `turn_context`) | | **opencode** | – | ✓ (`step-finish.reason`, pending event-level parity) | – | – | – | – | +| **muse** | ✓ (`response_id`) | ✓ (the step's `finish_reason`) | ✓ (`build.semver`) | – | – | ✓ (the run's `run_id`) | | **cursor**, **grok**, **relay** | – | – | – | – | – | – | A null is "the provider did not record it", which is not the same as `false` @@ -748,6 +757,7 @@ Delegation is a separate capability, reported on every relationship result as | **opencode** | always | – | ✓ | ✓ | | **claude** | sometimes | ✓ | ✓ | ✓ | | **grok** | sometimes | ✓ | ✓ | ✓ | +| **muse** | always | ✓ | ✓ | ✓ | | **cursor**, **relay** | never | – | – | – | OpenCode is `always` because a subagent session is a session in its own right @@ -767,6 +777,13 @@ sessions tree, and one that does not is stored as unlinked evidence. The id is never taken from the entry's file name. A `Task` call inside the transcript names no child at all. See ["grok"](#grok). +Muse is `always` because every subagent writes its own +`subagent//session.jsonl` beside the parent, and that log's metadata +record names the child session. The edge is recorded with `evidence_kind = +"muse_subagent_log"`; the agent type, name and model come from the parent's +`task_stream_linked` (or `memory_reminder_child_session_linked`) record for +that log. See ["muse"](#muse). + Cursor is `never` for a different reason from grok, opencode and relay: it does record the spawn — a `Task` / `functions.Subagent` tool call, preserved as a `tool_calls` row — but the block names no child transcript, so there is no @@ -1470,6 +1487,85 @@ How each adapter works: `totalTokens` context proxy that decreases on compaction, the models `grok-composer-2.5-fast` and `grok-build`, and the tool-name list. + + +- **muse** — Muse Code (Meta's `muse` CLI) keeps one append-only log per + session at `$XDG_DATA_HOME/muse/sessions/YYYY/MM/DD//session.jsonl` + (`~/.local/share/muse/sessions/…` when `XDG_DATA_HOME` is unset, on every + platform). Record interpretation lives in `crates/ai-hist/src/ingest/muse.rs` + so discovery, sync and hydration read it the same way. + + Every line is an envelope `{id, stream: {kind, id}, sequence, recorded_at, + payload_type, payload}`, and `recorded_at` is **microseconds** on every + record, so no time here is inferred. Identity is the `stream.id` of the + `runtime.session.metadata` record — searched for rather than assumed to be + line one, because a permission frame can precede it — never the directory + name. That record also gives `workspace_root` (the `cwd`), `model_id` and + `build.semver`. + + The conversation is `payload_type: "runtime.session"` with + `payload.kind: "run"`, keyed by `payload.event.kind`: `started` (the typed + prompt, one `history` row), `reasoning_committed`, + `assistant_message_committed`, `assistant_tool_calls_committed` (`args` is a + JSON *string*), `tool_result_batch_committed`, `model_completed` (per model + step: `model`, `usage`, `finish_reason`) and `terminal`. `payload.kind: + "task"` is execution bookkeeping and is not read. A call's outcome is the + separate `tool_batch.effect.terminal` record, joined by call id; the result + text carries no error flag of its own. Only records on the session's own + stream are read: subagent and reminder task streams are mirrored into the + parent file under a different `stream.id`, and reading them would present + every child objective as a prompt somebody typed. + + Usage is recorded as Muse wrote it: each `model_completed.usage` object + (`input_tokens` inclusive of the cached prefix, `cache_read_tokens` / + `cached_tokens`, `cache_write_tokens`, `output_tokens`, `reasoning_tokens`) + goes verbatim into `token_json` on the assistant row that step committed, + with its `model` and `finish_reason`, and normalizes as `per-request`. Muse + does not write the two in one order — a step that calls tools logs + `model_completed` *before* its calls, a step that answers in prose logs it + *after* the reply, and one step can commit readable reasoning and then its + tool calls. A step is what lies between the model being called and the + next `started`, `tool_result_batch_committed` or `terminal` in its run; its + first assistant record owns its usage, in whichever direction the step + wrote the two. A step that committed + nothing keeps its usage off every row and is reported as + `MUSE_USAGE_UNATTACHED`. Only newline-terminated records are read, so a + live session's half-written last line waits for the next read, and Muse is + one of the sources the sweep's destination marker guards: rows lost under + an unchanged stamp are restored by the next `sync`. + Reasoning Muse kept only encrypted is an `encrypted_reasoning` marker + (`MUSE_REASONING_ENCRYPTED`). + + A sync or hydration reads the whole file and **replaces** the session's + evidence, so a re-read after the log grew adds exactly the new turns and + duplicates nothing. + + **Subagents.** Every child agent — a `subagent_spawn` worker, or a reminder + child when Muse is set to save those — writes + `subagent//session.jsonl` beside its parent, and may nest its own + `subagent/` below that. These are never enumerated as sessions. Reading a + session reads its whole tree: each child is indexed under the session id + its **own** metadata record names (never the directory name) and linked to + its parent as `delegated`, with `spawn_depth`, the parent's + `task_stream_linked` `display.role` (`worker`, `reminder`, …) as + `child_agent_type`, its `display.label` as `child_agent_name`, a concrete + `display.model` (else the child's own `model_id`) as `child_model`, and the + task id as `evidence_ref`. As for a Codex subagent rollout, a child's + `history` rows and catalog row are removed — its objective is not a prompt + anybody typed — and its events stay addressable through the edge + (`sessions tree`, `related_session_ids`). The change stamp covers every + child log, so a background subagent that keeps writing after the parent's + last record is still re-read; a child whose log disappears loses its edge + and its evidence. A reminder linked by `memory_reminder_child_session_linked` + but saved only in memory (Muse's default for reminders) has no log and no + edge. `MUSE_SUBAGENT_LOG_MISSING` reports `subagent_spawn` calls with no log + to link, and `MUSE_SUBAGENT_LOG_UNIDENTIFIED` a log with no session + metadata. The shapes were characterized from a transcript + the real CLI wrote (`tests/fixtures/muse/cli-capture`, from + [xhluca/session-migrate](https://github.com/xhluca/session-migrate)) and + cross-checked against the published [Muse Code + SDK](https://github.com/meta-models/muse-code-sdk) and independent readers. + - **opencode** — **two storage layouts**, because both are in the field. RelayHistory prefers `opencode.db` when the host has it and falls back to the legacy JSON tree when it does not; it never reads both, because a host that diff --git a/docs/sourcing-sdk.md b/docs/sourcing-sdk.md index 191ce06d..e30042d2 100644 --- a/docs/sourcing-sdk.md +++ b/docs/sourcing-sdk.md @@ -660,6 +660,7 @@ handle. | codex | ✓ | ✓ | ✓ | ✓ | ✓ | cumulative-delta | | cursor | ✓ | ✓ | ✓ | ✓ | — | none | | grok | ✓ | ✓ | ✓ | ✓ | ✓ | per-request | +| muse | ✓ | ✓ | ✓ | ✓ | ✓ | per-request | | opencode | ✓ | ✓ | ✓ | ✓ | ✓ | none | | relay | — | — | — | — | — | none | | trajectory | — | — | — | — | — | none | diff --git a/sdk-ts/src/helpers.test.ts b/sdk-ts/src/helpers.test.ts index 0c88e269..f53fd6cc 100644 --- a/sdk-ts/src/helpers.test.ts +++ b/sdk-ts/src/helpers.test.ts @@ -12,7 +12,7 @@ import { } from './index.js'; test('exports immutable, aligned runtime source registries and matching type guards', () => { - assert.deepEqual(SOURCES, ['claude', 'codex', 'cursor', 'grok', 'relay', 'trajectory', 'opencode']); + assert.deepEqual(SOURCES, ['claude', 'codex', 'cursor', 'grok', 'relay', 'trajectory', 'opencode', 'muse']); assert.deepEqual(CATALOG_SOURCES, SOURCES.filter((source) => source !== 'trajectory')); assert.equal(Object.isFrozen(SOURCES), true); assert.equal(Object.isFrozen(CATALOG_SOURCES), true); @@ -49,4 +49,5 @@ test('resumeCommand preserves source and project-aware commands', () => { assert.equal(resumeCommand({ source: 'cursor', sessionId: 's3', project: '/work/app', locations: ['local', 'remote'] }), 'cd /work/app && cursor-agent --resume=s3'); assert.equal(resumeCommand({ source: 'grok', sessionId: 's4', project: null, locations: ['remote'] }), null); assert.equal(resumeCommand({ source: 'relay', sessionId: 's5', project: null, locations: ['local'] }), null); + assert.equal(resumeCommand({ source: 'muse', sessionId: 's6', project: '/work/app', locations: ['local'] }), 'cd /work/app && muse resume s6'); }); diff --git a/sdk-ts/src/mcp-server.ts b/sdk-ts/src/mcp-server.ts index 5c3415ac..02daebdd 100644 --- a/sdk-ts/src/mcp-server.ts +++ b/sdk-ts/src/mcp-server.ts @@ -21,8 +21,8 @@ const READ = { readOnlyHint: true, idempotentHint: true, openWorldHint: false } // (claude.ai/code web sessions, Codex cloud tasks), so it is open-world. const ACQUIRE = { readOnlyHint: false, idempotentHint: true, openWorldHint: true } as const; const LOCAL_ACQUIRE = { readOnlyHint: false, idempotentHint: true, openWorldHint: false } as const; -const SOURCE = z.enum(['claude', 'codex', 'cursor', 'grok', 'relay', 'trajectory', 'opencode']); -const CATALOG_SOURCE = z.enum(['claude', 'codex', 'cursor', 'grok', 'relay', 'opencode']); +const SOURCE = z.enum(['claude', 'codex', 'cursor', 'grok', 'relay', 'trajectory', 'opencode', 'muse']); +const CATALOG_SOURCE = z.enum(['claude', 'codex', 'cursor', 'grok', 'relay', 'opencode', 'muse']); const SESSION_SCOPE = z.enum(['local', 'remote', 'all']); const SOURCE_CONNECTORS = z.array(z.string().min(1)).optional().describe('Explicit configured source-plugin IDs; [] disables remote acquisition.'); const packageVersion = JSON.parse( diff --git a/sdk-ts/src/operations.ts b/sdk-ts/src/operations.ts index 51491dbb..02e7ce96 100644 --- a/sdk-ts/src/operations.ts +++ b/sdk-ts/src/operations.ts @@ -884,6 +884,7 @@ export function formatSessionRow( relay: '↔', opencode: '⌘', trajectory: '↗', + muse: '✧', }; const ageMs = session.lastActivityMs === null @@ -1073,6 +1074,7 @@ export function resumeCommand( if (entry.source === 'codex') return `codex resume ${shellQuote(entry.sessionId)}`; if (entry.source === 'cursor') return `cursor-agent --resume=${shellQuote(entry.sessionId)}`; if (entry.source === 'grok') return `grok resume ${shellQuote(entry.sessionId)}`; + if (entry.source === 'muse') return `muse resume ${shellQuote(entry.sessionId)}`; return null; })(); return resume && entry.project ? `cd ${shellQuote(entry.project)} && ${resume}` : resume; diff --git a/sdk-ts/src/sdk-common.ts b/sdk-ts/src/sdk-common.ts index 525368c5..fbac7ed3 100644 --- a/sdk-ts/src/sdk-common.ts +++ b/sdk-ts/src/sdk-common.ts @@ -21,6 +21,7 @@ export const SOURCES = Object.freeze([ 'relay', 'trajectory', 'opencode', + 'muse', ] as const); export type Source = (typeof SOURCES)[number];