diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index 5b643e6..632cd45 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -51,6 +51,7 @@ jobs: - muse - omp - opencode + - opencode-v2 - openhands - pi - qwen diff --git a/CHANGELOG.md b/CHANGELOG.md index 9d75a84..0a38eb6 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -6,6 +6,66 @@ here. Native format compatibility is documented separately in ## Unreleased +- Add version-aware OpenCode 2.0 public session transfer support, retaining the + legacy 1.17.20 adapter and native no-overwrite/dry-run controls. Preserve + portable tools, inline media, and completed compaction summaries, and account + for incomplete tools and provider-private state. + +## 0.12.1 - 2026-10-16 + +- Accept the compaction loop Claude Code records when the compact summary is not + the boundary's direct child. The declaration is unchanged, but how many + attachment records the CLI interleaves varies, so the summary can sit three or + four parent hops below its boundary, and `preservedMessages` also names ids + from the summarized branch that are no longer on the active path. A readable + 73 MB transcript with two compactions was refused as `Claude active graph + contains an ancestry cycle`; the loop is now validated as declared (the tail + walks up to the summary, the summary up to the boundary, every hop already on + the walked path) and a declaration that does not close still fails. +- Import a bundle the target store caps at 5 MiB per session by raising the + budget to what the bundle needs — never past the 25 MiB that store's own web + import accepts — and reporting `target_import_budget_raised` in the manifest. + A session carrying pasted screenshots (inline base64 images) produced a + 6.88 MB bundle and was refused with `session exceeds the import size limit`, + although the store keeps that content. + +## 0.12.0 - 2026-10-16 + +- Resolve the Hermes home the way the CLI does: `HERMES_HOME` when set, + otherwise `%LOCALAPPDATA%\hermes` on Windows and `~/.hermes` elsewhere. A bare + `~/.hermes` on Windows named a directory the CLI never writes to, so + `--to hermes` (and catalog discovery) missed the real store unless the caller + exported `HERMES_HOME` by hand. +- Move the Hermes import bundle across the process boundary as UTF-8 bytes + (`sys.stdin.buffer`, explicit stdout/stderr decoding, `-X utf8` child) instead + of locale-encoded text. On a Windows console whose code page is not UTF-8 the + old path replaced every non-ASCII character in the imported conversation — + title and message bodies — with `U+FFFD`. +- Install a Hermes session untitled instead of refusing the migration when the + target store already holds that title: Hermes enforces a partial unique index + over non-null titles, and its own importer treats the field as cosmetic. The + omission is counted as `title:duplicate_in_target` and reported as a + `target_title_omitted` manifest warning. +- Promote the Hermes Agent adapter to 0.22.2 (tag `v2026.9.24`) and SQLite + schema 31, keeping `SessionDB.import_sessions` as the native write boundary. +- Resolve the 0.22.2 runtime through `hermes --print-runtime-command` and + reproduce the launcher's `hermes_bootstrap` prelude, so the installer works + with a store Python that ships without third-party packages. The install root + comes from `hermes --version` (or the launcher's own bootstrap), and the + launcher shebang remains the fallback for builds predating the flag. +- Fix Hermes installation on Windows: keep the Windows essentials in the + sanitized child environment, initialize a missing `state.db` through the + native `SessionDB` constructor, and close the manifest identity guard before a + rollback unlinks it (`WinError 32`). +- Add first-class support for Antigravity Desktop (macOS/Electron) 2.18.1 + with bidirectional conversion, SQLite schema adaptation, Hub Summaries + Protobuf cache synchronization (`agyhub_summaries_proto.pb`), project-scoped + sidebar discovery, and verified native Go language server integration. +- Document the clean-room Antigravity Desktop storage architecture, Protobuf + wire formats, Connect-RPC interfaces, and collision invariants in + `docs/antigravity-desktop-format.md`. + + ## 0.11.0 - 2026-09-11 - Read root Codex rollouts written with `history_mode: "paginated"` by using diff --git a/docs/antigravity-desktop-format.md b/docs/antigravity-desktop-format.md new file mode 100644 index 0000000..471e0be --- /dev/null +++ b/docs/antigravity-desktop-format.md @@ -0,0 +1,189 @@ +# Antigravity Desktop session format + +`session-migrate` supports Antigravity Desktop (macOS, Linux, and Windows) +through a clean-room adapter pinned to the 2.18.1 release: + +| Property | Pinned value | +| --- | --- | +| Application | Antigravity Desktop `2.18.1` (Electron `44.3.0`) | +| macOS arm64 binary | `148,990,608` bytes, SHA-256 `300ee20f3108a511be1149602b91ba84cbbdcc42213067151e255584129be30f` | +| Linux x64 binary | `181,432,528` bytes, SHA-256 `ab4937445fa3817bc374a1db71062de7daecdb093cbfb6bd36b80f443198670e` | +| Windows x64 binary | `163,640,320` bytes, SHA-256 `1ed84e6a1d1e51064d80c9f382ab3a519eb2775cb91d552c63064a18cfdf3cf2` | +| Conversation store | `/.gemini/antigravity/conversations/.db` | +| Picker index | `/.gemini/antigravity/conversation_summaries.db` (`PRAGMA user_version = 3`) | +| Hub summary cache | `/.gemini/antigravity/agyhub_summaries_proto.pb` | +| UI project catalog | `/.gemini/config/projects/.json` + `app_storage.json` | + +This is deliberately not a claim of compatibility with unreleased or arbitrary +Antigravity versions. Automatic installation checks executable digests and +schema invariants when native verification is requested. + +No vendor code, binaries, descriptors, credentials, or user transcripts are +included in this repository. All schemas, wire mappings, and RPC endpoints were +established through clean-room differential probing and synthetic test fixtures. + +## How the format was established + +The adapter was derived from independent observations of synthetic sessions and +package analysis across macOS, Linux, and Windows distributions: + +1. generate synthetic conversations with unique marker text in isolated environments; +2. inspect SQLite schema, columns, and Protobuf wire field numbers and types; +3. analyze Electron application resources (`app.asar`), verifying identical JavaScript + (`3c03ce352dc3c1b43f357c27ead1f73c74d198a8ded89b1b3d6a2539e7a6ac5c`) across all three OSes; +4. inspect native Go language server (`language_server`) symbols and Connect-RPC endpoints; +5. verify round-trip serialization and cold-reload step retrieval via Connect-RPC + (`GetCascadeTrajectorySteps`); and +6. test real-world conversation migration end-to-end with active multi-turn sessions. + +## SQLite layout + +Each conversation is stored as an independent SQLite 3 database with write-ahead +logging (`WAL`) containing these tables: + +```text +trajectory_meta +steps +gen_metadata +executor_metadata +parent_references +trajectory_metadata_blob +battle_mode_infos +``` + +While the table names match Antigravity CLI, Desktop enforces distinct wire and +metadata invariants: + +| Property | Antigravity CLI (`antigravity`) | Antigravity Desktop (`antigravity-desktop`) | +| --- | --- | --- | +| App data subdirectory | `~/.gemini/antigravity-cli` | `~/.gemini/antigravity` | +| `trajectory_meta.source` | `17` (`CLI`) | `1` (`IDE` / `Hub`) | +| `trajectory_metadata_blob` project ID | `"default-cli-project"` | `"outside-of-project"` (or workspace UUID) | +| `conversation_summaries.source` | `"antigravity-cli"` | `""` (empty string) | +| `conversation_summaries.app_data_dir` | `"antigravity-cli"` | `"antigravity"` | +| `conversation_summaries.status` | `""` (empty string) | `"CASCADE_RUN_STATUS_IDLE"` | +| `conversation_summaries.project_id` | `"default-cli-project"` | `"outside-of-project"` | +| Summary columns | 19 or 21 columns | 21 columns (includes `raw_summary`, `group_id`) | +| Subagent trajectory roots | root ID == conversation ID | root ID points to root cascade ID | + +The main `trajectory_meta` row stores a fresh trajectory UUID, conversation UUID, +trajectory type `4` (cascade), and source `1` (`IDE`/`Hub`). `steps.idx` is a +contiguous zero-based sequence. Every step duplicates its type and status in +both the SQLite row columns and the serialized `Step` blob. + +### Sidebar discovery and project grouping + +Antigravity Desktop filters sidebar conversations strictly by active projects: + +1. **Active Projects List**: Tracked in `app_storage.json` under `projectsOrder`. + - macOS: `~/Library/Application Support/Antigravity/app_storage.json` + - Linux: `$XDG_CONFIG_HOME/Antigravity/app_storage.json` (or `~/.config/Antigravity`) + - Windows: `%APPDATA%\Antigravity\app_storage.json` +2. **Project Configurations**: Stored in `~/.gemini/config/projects/.json`, + defining `project_id`, workspace paths, and display labels. +3. Unassociated conversations (`project_id = "outside-of-project"`) are not + grouped in the project tree; assigning a registered project ID ensures instant + sidebar visibility. + +## Protobuf subset and Hub cache + +The adapter contains a bounded wire codec without generated vendor classes. It +accepts canonical varints, length-delimited bytes, and fixed 32/64-bit wire types. + +The `Step` envelope uses: + +| Field | Meaning | +| --- | --- | +| `1` | step type enum | +| `4` | step status enum | +| `19` | user-input payload when type is `14` (`CORTEX_STEP_TYPE_USER_INPUT`) | +| `20` | planner-response payload when type is `15` (`CORTEX_STEP_TYPE_PLANNER_RESPONSE`) | +| `140` | generic-tool payload when type is `132` (`CORTEX_STEP_TYPE_GENERIC_TOOL`) | + +Status `3` is done, status `5` is cleared, and status `7` is error. User-input +field `2` stores visible user text. Planner-response field `1` stores visible +assistant text, field `6` stores a fresh message ID, and repeated field `7` +stores tool calls. + +### Hub summary cache (`agyhub_summaries_proto.pb`) + +Antigravity Desktop caches summaries in `agyhub_summaries_proto.pb`. On startup, +the native Go process reconciles SQLite records with this protobuf cache. If a +conversation exists in SQLite but has no entry in the cache or `raw_summary` +column, metadata fields reset to zero defaults (`0001-01-01`). The adapter +synchronizes both stores during installation. + +## Conversion policy + +| Portable event | Antigravity Desktop representation | +| --- | --- | +| User text | done user-input step | +| Assistant text | done planner-response step | +| Tool call | planner-response tool call | +| Tool result | following generic-tool step | +| Thinking/reasoning | omitted and counted as `thinking:private` | +| Compaction | omitted and counted | +| System/developer message | omitted as privileged context | +| Images and other context | omitted by type and counted | +| Unknown/opaque source event | omitted by reason and counted | + +Private chain-of-thought is not portable. Thinking bytes and model signatures are +omitted from generated outputs, and source reading emits only a content-free +`thinking` marker. + +A tool result without a preceding call receives a fresh synthetic call and an +explicit orphan counter. Duplicate or out-of-order IDs are rewritten or counted +to prevent corrupted native trajectories. + +## Reading and installation + +Source databases may have active write-ahead logs (`.db-wal`). The reader uses +SQLite's backup API to obtain a transactionally consistent snapshot, then hashes +and parses that snapshot. It validates: + +- exact table, column, and schema invariants; +- SQLite page bounds and `integrity_check`; +- valid canonical UUIDv4 IDs and source `1` / `17` discrimination; +- contiguous step indices and supported step types; and +- structurally valid, bounded protobuf values. + +Native installation creates target directories privately, reserves the summary +ID with `BEGIN IMMEDIATE`, writes the database atomically, and commits the +summary transaction. If a transaction fails, only newly created files are +unlinked. Existing sessions are never overwritten. + +## Native validation evidence + +1. **Synthetic Round-Trip Invariants**: Synthetic multi-turn sessions with tool + invocations, results, and thinking events serialize and re-parse with exact + fidelity, asserting that thinking text is removed. +2. **Native Language Server Verification**: The official Go binary + (`language_server`) was executed in an isolated sandbox, queried over Connect-RPC + (`GetCascadeTrajectorySteps`), and asserted to return all imported steps with + identical structure. +3. **Live User Session Migration**: A live ESPHome voice satellite conversation + with 256 steps was migrated from Antigravity CLI to Antigravity Desktop, + confirming complete rendering and sidebar searchability. + +Run the test suite with: + +```console +uv run pytest -q tests/test_antigravity_desktop_format.py +``` + +Run the native language server oracle (macOS with Antigravity Desktop installed): + +```console +uv run pytest -q tests/test_antigravity_desktop_native.py +``` + +## Known boundaries + +- Native oracle hash validation is pinned to official release builds. Unvalidated + binaries can be tested with `SESSION_MIGRATE_UNVALIDATED_DESKTOP_BIN=1`. +- Generic tool transport is visually render-proven; internal vendor-privileged tools + are preserved as generic tool steps to maintain auditability. +- Private reasoning tokens, system instructions, and ephemeral UI selections are + not transferred. +- Subagent conversations are mapped to independent portable sessions with their + parent reference preserved in metadata. diff --git a/docs/cli-reference.md b/docs/cli-reference.md index 98d95b0..26811cd 100644 --- a/docs/cli-reference.md +++ b/docs/cli-reference.md @@ -1,6 +1,6 @@ # CLI reference -This page documents `session-migrate` 0.11.0. `smigrate` is an exact shorthand +This page documents `session-migrate` 0.12.1. `smigrate` is an exact shorthand for the same executable. ## Commands @@ -175,7 +175,7 @@ installation still requires the exact pinned version. | Grok | `$GROK_HOME`, otherwise `~/.grok` | | Kilo Code | official CLI under its normal XDG data root | | OpenHands | `$OPENHANDS_CONVERSATIONS_DIR`, otherwise `~/.openhands/conversations` | -| Hermes Agent | `$HERMES_HOME`, otherwise `~/.hermes` | +| Hermes Agent | `$HERMES_HOME`, otherwise the CLI's platform default: `%LOCALAPPDATA%\hermes` on Windows, `~/.hermes` elsewhere | | MastraCode | `$MASTRA_DB_PATH`; otherwise `$MASTRA_APP_DATA_DIR/mastra.db` or the platform data directory | | Devin CLI | `$XDG_DATA_HOME/devin/cli`, otherwise `~/.local/share/devin/cli` on Linux | diff --git a/docs/credential-free-native-testing.md b/docs/credential-free-native-testing.md index e8e834f..92b00ea 100644 --- a/docs/credential-free-native-testing.md +++ b/docs/credential-free-native-testing.md @@ -49,7 +49,7 @@ No production credential is read, translated, or written by this workflow. | Cursor `2026.03.20-44cb435` | Shipped backend loads blobs and the real TUI renders imported text; no offline model append | | Devin 3000.6.7 | Native list/resume selection and ACP `session/load`; stops at vendor authentication | | Grok 1.0.5 | OpenAI-compatible override; native creation, reload, TUI replay, and append | -| Hermes 0.20.6 | OpenAI-compatible override; replay, append, and compaction | +| Hermes 0.22.2 | OpenAI-compatible override; replay, append, and compaction | | Kilo 7.5.0 | Official import/export plus OpenAI-compatible replay and append | | Kimi 0.38.0 | OpenAI-compatible override; imported history replays and the reply persists | | MastraCode 0.37.1 | OpenAI-compatible override; replay and append | @@ -82,3 +82,11 @@ harness and installations do not compete for disk space. The normal Python job separately runs the deterministic 18 by 18 route oracle. The native matrix is the independent acceptance layer: generated files must be accepted by code owned and shipped by each harness. + +## OpenCode v2 variant + +The additional `opencode-v2` job pins `@opencode/cli@2.0.23` and sets +`SESSION_MIGRATE_TEST_OPENCODE_V2`. Its runner requires every assigned test to +run without skips, exercising synthetic import/export, collisions, cold reopen +and localhost-only continuation. Legacy `opencode` remains pinned to 1.17.20. +See [the v2 transfer contract](opencode-v2.md). diff --git a/docs/development.md b/docs/development.md index e91554b..972ccd1 100644 --- a/docs/development.md +++ b/docs/development.md @@ -39,7 +39,7 @@ pytest and Ruff are development dependencies locked by `uv.lock`. | `src/session_migrate/formats/grok.py` | Grok 1.0.5 ACP-update adapter | | `src/session_migrate/formats/kilo.py` | Kilo Code 7.5.0 official-bundle adapter | | `src/session_migrate/formats/openhands.py` | OpenHands 1.16.0 SDK-event adapter | -| `src/session_migrate/formats/hermes.py` | Hermes Agent 0.20.6 shared-state adapter | +| `src/session_migrate/formats/hermes.py` | Hermes Agent 0.22.2 shared-state adapter | | `src/session_migrate/formats/mastracode.py` | MastraCode 0.37.1 LibSQL adapter | | `src/session_migrate/formats/devin.py` | Devin CLI 3000.6.7 active-forest adapter | | `src/session_migrate/formats/common.py` | Shared timestamps, text, and image validation | diff --git a/docs/format-compatibility.md b/docs/format-compatibility.md index 25e070d..3e68c96 100644 --- a/docs/format-compatibility.md +++ b/docs/format-compatibility.md @@ -20,9 +20,11 @@ to separately installed host binaries: | Codex CLI | `0.144.4` | | Pi source and target | `0.80.6` | | Oh My Pi source and target | `18.0.5` | -| OpenCode source and target | `1.17.20` | +| OpenCode legacy source and target | `1.17.20` | +| OpenCode v2 transfer adapter | Linux gateway build `2.0.23` | | GitHub Copilot CLI source and target | `1.0.70` | | Antigravity CLI source and target | `1.1.16` | +| Antigravity Desktop source and target | `2.18.1` | | Cursor Agent experimental text adapter | `2026.03.20-44cb435` | | Mistral Vibe source and target | `2.24.3` | | Muse Code source and target | `0.2.1 (0.2.1-R1215.1)` | @@ -31,7 +33,7 @@ to separately installed host binaries: | Grok source and target | `1.0.5` | | Kilo Code source and target | `7.5.0` | | OpenHands source and target | `1.16.0` (SDK `1.21.0`) | -| Hermes Agent source and target | `0.20.6` (tag `v2026.8.27`) | +| Hermes Agent source and target | `0.22.2` (tag `v2026.9.24`, SQLite schema `31`) | | MastraCode source and target | `0.37.1` | | Devin CLI source and target | `3000.6.7` (`260a97c8`) | @@ -54,6 +56,13 @@ rendering without a model append; Devin proves ACP loading and the vendor-login boundary. See [Credential-free native client testing](credential-free-native-testing.md). +OpenCode 2.0.x has a separate public-transfer adapter. Its local synthetic tests +cover native import/export, tool linkage, completed compaction, dry-run and +identity conflicts; this does not extend the historical eighteen-client corpus +validation claim. See [OpenCode v2 transfers](opencode-v2.md) for exact scope and +limitations. No real user transcript migration or native Windows validation has +been performed for this adapter. + All eighteen formats are sources and targets. Their mappings, native probes, and loss keys are specified in [Additional native formats](additional-target-formats.md), [OpenCode source research](opencode-source-exploration.md), diff --git a/docs/hermes-format.md b/docs/hermes-format.md index 5e0e293..f8ea647 100644 --- a/docs/hermes-format.md +++ b/docs/hermes-format.md @@ -9,16 +9,18 @@ so the adapter fails closed on any other Hermes or schema version. | Item | Validated value | | --- | --- | | Project | [`NousResearch/hermes-agent`](https://github.com/NousResearch/hermes-agent) | -| Release tag | `v2026.8.27` | -| Python package | `hermes-agent==0.20.6` | -| Source commit | `5fc308a70719a83cccdbba4c0e39c23f5a8239d5` | -| Source-archive SHA-256 | `0ca18fcf97a69bad808ce4a0807694b25237afc61f7a13efab4f0aa26092a171` | -| Native SQLite schema | `26` | -| Observed CLI version | `Hermes Agent v0.20.6 (2026.8.27)` | - -The source archive hash is the SHA-256 of `git archive` at the pinned commit. -The editable development launcher is deliberately not used as an artifact pin: -its shebang contains the temporary installation path. +| Release tag | `v2026.9.24` (also published as `v0.22.2`) | +| Python package | `hermes-agent==0.22.2` | +| Source commit | `d923a5eb487a17c0a68801bba9ef0f56c81040a8` | +| Native SQLite schema | `31` | +| Observed CLI version | `Hermes Agent v0.22.2 (2026.9.24) · stable · local d923a5eb487a` | + +The adapter now pins the same release the published launcher reports, instead of +carrying a source-archive hash: a 0.22.2 git install reports its own build +provenance on the version line, so the release identity is read from the +launcher itself (`Hermes Agent v ()`, with any trailing +`· stable · local ` suffix tolerated) rather than from an artifact hash +that a rebuild would invalidate. ## Native store and discovery @@ -28,7 +30,11 @@ Hermes stores all CLI sessions in one SQLite database: $HERMES_HOME/state.db ``` -`HERMES_HOME` defaults to `~/.hermes`. Unlike file-per-session harnesses, a +`HERMES_HOME` wins when it is set; otherwise the home is the CLI's own platform +default — `%LOCALAPPDATA%\hermes` on Windows (`~/AppData/Local/hermes` when that +variable is unset) and `~/.hermes` elsewhere. Resolving a bare `~/.hermes` on +Windows used to name a directory the CLI never writes to. +Unlike file-per-session harnesses, a Hermes catalog entry is virtual: its identity is the pair `state.db` plus the native session ID. Native IDs have the form: @@ -43,7 +49,7 @@ stored message count. `cli_version` is `None` because Hermes does not persist a per-session CLI version; database schema validation supplies the compatibility gate instead. -The schema-26 tables used by the adapter are: +The schema-31 tables used by the adapter are: | Table | Fields used | | --- | --- | @@ -51,10 +57,14 @@ The schema-26 tables used by the adapter are: | `sessions` | identity, title, CWD, model/provider, start/end/activity time, message count | | `messages` | role/content, tool linkage, timestamp, private reasoning markers, active/compacted state | +Schema 31 keeps every column the adapter requires from schema 26, so the +projection and the required-column gates are unchanged; only the pinned version +value moved. + Discovery and parsing first take a bounded SQLite backup. This includes committed WAL frames and gives the catalog a stable SHA-256 fingerprint. A symlink, non-regular file, oversized database, integrity failure, missing -columns, or schema other than 26 is rejected. +columns, or schema other than 31 is rejected. ## Source projection @@ -102,16 +112,97 @@ install_bundle( ) -> InstalledHermesSession ``` -The installer verifies the exact CLI version, initializes a missing -`state.db` through the CLI, refuses an existing session ID, invokes the native -importer, and reparses the installed row before returning. Target directories -are private and symlink traversal is rejected. +The installer verifies the exact CLI version, creates a missing `state.db` +through the native `SessionDB` initializer, refuses an existing session ID, +invokes the native importer, and reparses the installed row before returning. +Target directories are private and symlink traversal is rejected. Hermes's user-facing `hermes sessions import` command currently accepts Claude -and Codex sources, not Hermes's own export. `SessionDB.import_sessions` is the -native export/import transaction used internally and is therefore the pinned +and Codex sources, not Hermes's own export. `SessionDB.import_sessions` remains +the native export/import transaction in 0.22.2 and is therefore still the pinned installation boundary here. +## The native runtime behind the launcher + +0.22.2 publishes its own launch command, and the adapter uses it instead of +guessing a Python from the launcher's file head: + +```bash +hermes --print-runtime-command # JSON argv: [interpreter, -I, -c, bootstrap, ...] +hermes --version # also prints "Install directory: " +``` + +That interpreter is *store Python*: it deliberately ships with no third-party +packages. The published launcher reaches the dependency generation by importing +`hermes_bootstrap` after putting the install root on `sys.path`, so the helper +the installer runs reproduces exactly that prelude (clear `PYTHONHOME`, +`PYTHONPATH` and `VIRTUAL_ENV`; insert the install root; import +`hermes_bootstrap`). Without it, every `hermes_state` import fails on a missing +`ruamel`, not on anything session-related. Builds predating +`--print-runtime-command` keep the older behaviour: the adapter derives the +interpreter from the launcher's `#!` line. + +Two platform facts are handled explicitly: + +- **Windows launchers are compiled zipapps.** `hermes.exe` is a PE stub with the + launcher's `__main__.py` appended, so there is no `#!` line the old code could + read; `--print-runtime-command` is the only portable source. The install + directory is recovered from `Install directory:`, or from the launcher's own + `sys.path.insert(0, ...)` bootstrap when that line is absent. +- **A sanitized child environment must keep the Windows essentials.** The CLI + refuses to start when it cannot resolve a home directory, and a Windows child + needs `SYSTEMROOT`, `PATHEXT` and `COMSPEC` to load at all. The allowlist + therefore carries `USERPROFILE`, `HOMEDRIVE`, `HOMEPATH`, `APPDATA`, + `LOCALAPPDATA`, `PROGRAMDATA`, `SYSTEMROOT`, `SYSTEMDRIVE`, `WINDIR`, + `PATHEXT`, `COMSPEC`, `TEMP`, `TMP` and `NUMBER_OF_PROCESSORS` alongside the + POSIX variables; `HOME` is mirrored into `USERPROFILE` when the caller + redirects the home through `HOME` only. +- **The bundle crosses the process boundary as bytes.** The helper reads + `sys.stdin.buffer` and the caller decodes stdout/stderr explicitly as UTF-8; + the child interpreter is started with `-X utf8`. Passing the payload as *text* + encoded it with the caller's locale codec while the store's interpreter always + decodes UTF-8, so a Windows console whose code page is not UTF-8 replaced + every non-ASCII character in the imported conversation — title and message + bodies alike — with `U+FFFD`. Both directions are now codec-independent. + +Installation summary: `hermes sessions list` no longer creates `state.db` on a +read of an absent store, so initialization moved to the native `SessionDB` +constructor; and the identity-guard descriptor that reserves the manifest is +closed before a rollback unlinks it, because Windows refuses to delete a file +that still has an open handle (`WinError 32`). + +### Titles are unique per store + +Hermes enforces a partial unique index over non-null titles: + +```sql +CREATE UNIQUE INDEX idx_sessions_title_unique ON sessions(title) WHERE title IS NOT NULL +``` + +A second session therefore cannot reuse a title, and the native importer treats +the field as cosmetic: it wraps its own title write in a suppressed exception +and leaves the conflicting session untitled. Before this change the migrator +inherited the raw refusal, so migrating the same conversation twice into one +home — or migrating two sessions whose first message produced the same title — +failed the whole bundle with `UNIQUE constraint failed: sessions.title`. + +The installer now samples the target store first and, when the title is already +taken, drops it from the bundle before the native import. The migration +succeeds, the manifest records the omission as the `title:duplicate_in_target` +counter plus a `target_title_omitted` warning, and the session appears in the +native picker without a title. Only the cosmetic field is removed: the envelope, +its messages, and the record count are unchanged. + +The migration manifest is the record of truth here. The CLI's content-free +stdout summary is built from the conversion result before any target installer +runs (true for every harness), so a real run reports the omission in +`/session-migrate/manifests/hermes-.json` rather than in the +printed JSON, and a `--dry-run` — which writes no manifest — reports it nowhere. + +The sample is advisory. It runs before the import transaction, which still +enforces the constraint itself, so a title created by a concurrent writer in +between fails closed exactly as before, with no partial session. + Resume uses Hermes's native command: ```bash @@ -132,6 +223,7 @@ unambiguous session ID. | Compaction boundary/replacement metadata | summary kept; metadata counted | Hermes stores the active summary, not portable boundary semantics | | Rewound or already compacted bodies | opaque accounting only | not part of Hermes's active model replay | | Token/cost/runtime counters | reset | derived native runtime state | +| Session title already held by the target store | installed untitled; counted | Hermes allows one session per non-null title | ## Native research trajectories @@ -139,6 +231,31 @@ Research and validation ran in a unique mode-`0700` temporary root. The pinned source was installed into its own virtual environment; no user Hermes home or credential store was read or copied. +The 0.20.6 trajectories below remain the historical record of the original +adapter research. The 0.22.2 promotion was re-validated on a live Windows +install: + +1. Read side, against the real store, read-only: a 31 MB `state.db` at schema + `31` was snapshotted (bounded SQLite backup), all required session and + message columns were present, and inventory returned every session row. +2. Write side, in an isolated scratch home: a generated bundle installed through + `SessionDB.import_sessions`, was reparsed (title, CWD, both message events), + appeared in `list_sessions` with the right record count, and a second install + of the same native ID was refused with the session store byte-identical + afterwards. +3. End-to-end route: a real Claude Code session (23 records) transferred with + `smigrate transfer --from claude --to hermes`. The native CLI listed the + result — `message_count` 22, `tool_call_count` 9, nine linked `tool` rows + carrying `tool_call_id` and `tool_name`, the source title and CWD preserved, + `source=session-migrate` — which is the fidelity the text-only native + importer does not provide: `hermes sessions import` writes `[ran tool: Bash]` + markers into the assistant body and leaves `tool_call_count=0`. + +Not verified in this round: a live `hermes chat --resume` against a provider, +native compaction after import, and WAL-snapshot behaviour while another writer +is appending. Those remain covered by the opt-in native test rather than by +this promotion. + Three native trajectories were inspected: 1. A real Hermes CLI session containing a user turn, assistant function call, @@ -165,7 +282,7 @@ the new summary and accounted for archived rows. One observed native nuance is documented rather than papered over: the first CLI-created row stored a title but left `cwd` and `model` null despite launch options, while imported sessions retained an explicit CWD. Both are valid -schema-26 states, so inventory and parsing treat native CWD/model as optional. +schema states, so inventory and parsing treat native CWD/model as optional. ## Test contract @@ -184,8 +301,8 @@ and its installed CLI. It still uses no provider credential and binds its fake provider only to loopback: ```bash -SESSION_MIGRATE_HERMES_SOURCE=/path/to/hermes-agent-v2026.8.27 \ -SESSION_MIGRATE_HERMES_BIN=/path/to/hermes-agent-v2026.8.27/.venv/bin/hermes \ +SESSION_MIGRATE_HERMES_SOURCE=/path/to/hermes-agent-v2026.9.24 \ +SESSION_MIGRATE_HERMES_BIN=/path/to/hermes-agent-v2026.9.24/.venv/bin/hermes \ uv run pytest -q tests/test_hermes_native.py -vv ``` diff --git a/docs/native-corpus-validation.md b/docs/native-corpus-validation.md index d7781fe..23d0e12 100644 --- a/docs/native-corpus-validation.md +++ b/docs/native-corpus-validation.md @@ -51,7 +51,7 @@ and sanitization mutations live beside each fixture in `provenance.json`. | Cursor | 2026.03.20-44cb435 | vendor | dropped | — | dropped | lossy | rejected | rejected | dropped | dropped | | Devin | 3000.6.7 | vendor | preserved | preserved | — | lossy | lossy | lossy | lossy | — | | Grok | 1.0.5 | loopback | preserved | — | preserved | lossy | rejected | rejected | — | — | -| Hermes | 0.20.6 | loopback | preserved | lossy | — | rejected | rejected | rejected | — | — | +| Hermes | 0.22.2 | loopback | preserved | lossy | — | rejected | rejected | rejected | — | — | | Kilo | 7.5.0 | loopback | preserved | preserved | — | lossy | rejected | rejected | — | — | | Kimi | 0.38.0 | OpenRouter | preserved | unsupported | — | — | — | — | lossy | — | | MastraCode | 0.37.1 | loopback | preserved | preserved | — | rejected | rejected | rejected | — | — | diff --git a/docs/opencode-v2.md b/docs/opencode-v2.md new file mode 100644 index 0000000..8b8ee03 --- /dev/null +++ b/docs/opencode-v2.md @@ -0,0 +1,97 @@ +# OpenCode 2.0 public session transfers + +OpenCode 1.17.20 uses `import/export --pure` and nested message/part bundles. +OpenCode 2.0 uses `session import/export --standalone` and flat typed messages. +Native import detects the installed client and selects its transfer schema. +Convert-only output remains legacy unless `--target-cli-version 2.0.23` is set. +Unsupported schema series fail closed. The `opencode v` version prefix and +custom suffixes are accepted; unvalidated exact releases receive schema-aware +warnings. Native acceptance is pinned to stock 2.0.23, not every 2.0.x release. + +The stock 2.0.22 release is also recognized by the native tests; recognition is +not evidence that a particular local test run used that release. + +The v2 adapter preserves ordered user/assistant text, linked tool inputs and +results, inline user/tool images, and readable completed compaction summaries. +User text blocks in one native message become their exact newline join. +Private-only or whitespace-only checkpoints do not retire readable history. +Encrypted provider checkpoints, private traces, source reasoning variants and +unsupported control/metadata fields are explicitly counted as omissions. + +Completed tool results retain ordered portable `content_blocks`, including +caption/image interleaving, adjacent text blocks and empty text blocks. Targets +that support ordered blocks retain them. Legacy OpenCode and Kilo flatten text +and attachments; `tool_result:content_block_order` records any resulting loss of +order or segmentation rather than silently treating that projection as exact. + +Structured tool errors retain their message. Nondefault error types, HTTP status +and response bodies are counted separately as `opencode_v2_tool_error_type`, +`opencode_v2_tool_error_status` and `opencode_v2_tool_error_response` omissions. +Opaque events and manifests contain reason labels, not diagnostic response bodies. + +Original native session fields and all accepted control variants are validated +before projection/defaults, including required and nested fields and the +distinction between absent and null optionals. Unsupported readable control +records are accounted for as omissions; malformed native records are rejected, +not repaired by discarding them. + +Catalog discovery reads legacy `session` and `session_v2` metadata in one +read-only transaction, preferring v2 for duplicate identities. Flat v2 bundles +can be autodetected; legacy OpenCode/Kilo bundles remain ambiguous. Kilo keeps +its legacy nested transfer contract, even with a v2-looking version override. + +## Review follow-up and attribution + +The focused implementation originated in tudstudent's PR #15, commit +`0f32c8f377f9667fac8be9325ca3065314b9f0ef`. PR #17 by TheViniAlmeida explicitly +builds on that implementation and adds catalog discovery and Kilo separation. +This branch incorporates those adaptations and the review fixes and regression +coverage from PR #17 commit `2e3671dcf402159bcb456380409a103cfaaf19cf` +(TheViniAlmeida, `eu@thevinialmeida.com`), addressing xhluca's three findings in +review `5477456070`. This records both directions of reuse; it does not claim +sole authorship of the follow-up work or upstream acceptance. + +A streamed tool input can be an incomplete JSON string. The source projection +keeps it unchanged inside `{ "input": ORIGINAL_STRING }`; it does not replace +it with `{}` or attempt to parse/complete it. On conversion, an unfinished call +is archived as an `ImportedIncompleteTool` error with the same ID and input, +because native transfer discards unsettled assistant records. This is historical +input preservation, not an executable resumed partial tool invocation. The +source→portable→target regression includes an unterminated string input. + +OpenCode owns its database import. No private SQLite writes, credentials or +machine-specific launchers are part of the adapter. Dry run performs public +native preflight without importing or writing a migration manifest; native +preflight may initialize the client's ordinary store. Imports enforce global +identity collisions and explicit success confirmation, then verify the imported +ID using public export. A conflict can have native exit status zero and still +fails import confirmation. + +## Reproducible native and route checks + +```sh +./scripts/install-native-test-clis.sh /tmp/session-migrate-native opencode-v2 +./scripts/run-native-test-client.sh \ + opencode-v2 /tmp/session-migrate-native/session-migrate-native.env +uv run pytest -q tests/test_opencode_v2.py tests/test_opencode_v2_review.py tests/test_route_matrix.py +``` + +The `opencode-v2` CI matrix entry installs **@opencode/cli@2.0.23** (the legacy +package is opencode-ai), supplies `SESSION_MIGRATE_TEST_OPENCODE_V2`, and rejects +skipped assigned tests. Native tests use synthetic transcripts, isolated HOME +and XDG directories, an explicit credential-free config, and a deterministic +localhost provider. They verify import/export, dry run, collisions, cold process +reopening, compaction/tool replay and a new persisted reply under the same ID. +They do not use real accounts or supplier inference credits. The default suite +can skip native tests when that exact executable is unavailable; the selected +CI job cannot silently pass those skips. + +Route tests cover legacy and v2 OpenCode source and target variants against the +other supported formats. The 19 variants remain eighteen harnesses. Existing +legacy wire behavior and native tests remain separate. + +This change is independent of the Windows filesystem-cleanup and Codex child +projection changes split from PR #10. Those fixes complement Windows execution +and additional Codex sources, but neither is required to review this schema or +run the Linux v2 gate. Native Windows acceptance and all historical client +versions are not established by this branch's Linux gates. diff --git a/docs/specification.md b/docs/specification.md index 88c856f..09694c5 100644 --- a/docs/specification.md +++ b/docs/specification.md @@ -1,6 +1,6 @@ # Session migration specification -This is the user-facing contract for `session-migrate` 0.11.0. +This is the user-facing contract for `session-migrate` 0.12.1. ## Scope @@ -21,7 +21,7 @@ The migrator reads and writes native sessions for: - Grok 1.0.5 - Kilo Code 7.5.0 - OpenHands 1.16.0 -- Hermes Agent 0.20.6 +- Hermes Agent 0.22.2 - MastraCode 0.37.1 - Devin CLI 3000.6.7 @@ -128,7 +128,7 @@ OMP's current fixed-title-slot v3 writer is pinned to 18.0.5. Vibe's writer and append-boundary fingerprint are pinned to 2.24.3. Muse, Qwen, and Kimi writers are pinned to 0.2.1, 0.22.1, and 0.38.0. Grok, Kilo, and OpenHands are pinned to 1.0.5, 7.5.0, and 1.16.0. Hermes, MastraCode, and -Devin are pinned to 0.20.6, 0.37.1, and 3000.6.7. Installation +Devin are pinned to 0.22.2, 0.37.1, and 3000.6.7. Installation does not invoke Vibe; the exact CLI is exercised by the credential-free native resume gate. diff --git a/docs/troubleshooting.md b/docs/troubleshooting.md index a3e3bdc..c49cbdc 100644 --- a/docs/troubleshooting.md +++ b/docs/troubleshooting.md @@ -248,7 +248,7 @@ smigrate catalog refresh --mastracode-root /path/to/mastra.db smigrate catalog refresh --devin-root /path/to/devin-home ``` -Hermes `0.20.6`, MastraCode `0.37.1`, and Devin `3000.6.7` are strict native +Hermes `0.22.2`, MastraCode `0.37.1`, and Devin `3000.6.7` are strict native pins. Use `--source-cli` or `--target-cli` when the exact binary is not on `PATH`; a version override cannot bypass a schema or binary mismatch. Hermes installation uses the pinned official Python importer. MastraCode and Devin diff --git a/pyproject.toml b/pyproject.toml index 79d226c..6e4925a 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -4,7 +4,7 @@ build-backend = "hatchling.build" [project] name = "session-migrate" -version = "0.11.0" +version = "0.12.1" description = "Migrate native coding-agent sessions across eighteen terminal harnesses" readme = "README.md" requires-python = ">=3.11" diff --git a/scripts/install-native-test-clis.sh b/scripts/install-native-test-clis.sh index 08d7787..47b9009 100755 --- a/scripts/install-native-test-clis.sh +++ b/scripts/install-native-test-clis.sh @@ -77,6 +77,10 @@ if want opencode; then npm_cli opencode 'opencode-ai@1.17.20' \ SESSION_MIGRATE_OPENCODE_BIN node_modules/.bin/opencode fi +if want opencode-v2; then + npm_cli opencode-v2 '@opencode/cli@2.0.23' \ + SESSION_MIGRATE_TEST_OPENCODE_V2 node_modules/@opencode/cli-linux-x64/bin/opencode +fi if want pi; then npm_cli pi '@earendil-works/pi-coding-agent@0.80.6' \ SESSION_MIGRATE_PI_BIN node_modules/.bin/pi @@ -174,22 +178,32 @@ if want vibe; then fi if want hermes; then - hermes_archive="$native_root/hermes-agent-5fc308a7.tar.gz" - if ! verify 8710e0017792e78369a7da3d96c16141f0787374285f1a6cf6f80d29b7b9ea2c \ + hermes_archive="$native_root/hermes-agent-d923a5eb.tar.gz" + if ! verify 616649f8ac7e3bb186999d000d77511e733b3930bd2b5921be4a85673320ef8b \ "$hermes_archive" 2>/dev/null; then curl --fail --location --silent --show-error --retry 3 --output "$hermes_archive" \ - 'https://codeload.github.com/NousResearch/hermes-agent/tar.gz/5fc308a70719a83cccdbba4c0e39c23f5a8239d5' + 'https://codeload.github.com/NousResearch/hermes-agent/tar.gz/d923a5eb487a17c0a68801bba9ef0f56c81040a8' fi - verify 8710e0017792e78369a7da3d96c16141f0787374285f1a6cf6f80d29b7b9ea2c \ + verify 616649f8ac7e3bb186999d000d77511e733b3930bd2b5921be4a85673320ef8b \ "$hermes_archive" mkdir -p "$native_root/hermes" tar -xzf "$hermes_archive" --strip-components=1 -C "$native_root/hermes" - # Hermes' revision-3 lockfile uses options first supported by uv 0.10. + # The pinned revision resolves its own version from the source git tree, so + # the gitless tarball report would be "vunknown"; reconstruct a minimal + # checkout carrying the pinned release tag. + git -C "$native_root/hermes" init -q + git -C "$native_root/hermes" add -A + git -C "$native_root/hermes" -c user.name=session-migrate \ + -c user.email=ci@invalid -c commit.gpgsign=false \ + commit -q -m "session-migrate: pin hermes agent d923a5eb" + git -C "$native_root/hermes" tag v0.22.2 + # Hermes' revision-3 lockfile uses options first supported by uv 0.10 + # and requires Python >= 3.14. uvx --from 'uv==0.10.9' uv sync \ - --directory "$native_root/hermes" --frozen --no-dev --python 3.13 + --directory "$native_root/hermes" --frozen --no-dev --python 3.14 record SESSION_MIGRATE_HERMES_SOURCE "$native_root/hermes" record SESSION_MIGRATE_HERMES_BIN "$native_root/hermes/.venv/bin/hermes" - record SESSION_MIGRATE_HERMES_REVISION 5fc308a70719a83cccdbba4c0e39c23f5a8239d5 + record SESSION_MIGRATE_HERMES_REVISION d923a5eb487a17c0a68801bba9ef0f56c81040a8 fi path_entries=() diff --git a/scripts/native-corpus/capture-hermes.py b/scripts/native-corpus/capture-hermes.py index b660dbf..ced1b03 100644 --- a/scripts/native-corpus/capture-hermes.py +++ b/scripts/native-corpus/capture-hermes.py @@ -1,5 +1,5 @@ #!/usr/bin/env python3 -"""Capture an exact Hermes 0.20.6 source trajectory without credentials.""" +"""Capture an exact Hermes 0.22.2 source trajectory without credentials.""" from __future__ import annotations @@ -22,11 +22,18 @@ ASSETS = REPO_ROOT / "tests/native_corpus/v1/assets" SCENARIO = json.loads((REPO_ROOT / "tests/native_corpus/v1/scenario.json").read_text()) TITLE = str(SCENARIO["session_title"]) -VERSION = "0.20.6" -RELEASE = "2026.8.27" -SOURCE_COMMIT = "5fc308a70719a83cccdbba4c0e39c23f5a8239d5" -BINARY_BYTES = 339 -BINARY_SHA256 = "060277e4ac1caa59ea6e0c551c7627f8fea0989a4aace9f73cdc1f09425f34c2" +VERSION = "0.22.2" +RELEASE = "2026.9.24" +SOURCE_COMMIT = "d923a5eb487a17c0a68801bba9ef0f56c81040a8" +# The entry-point wrapper embeds the install root, so its bytes are per-install +# rather than per-release. A capture on a new install must re-record both +# values here (the defaults below are the wrapper recorded on POSIX for the +# previous pin and will not match another tree). +BINARY_BYTES = int(os.environ.get("SESSION_MIGRATE_HERMES_WRAPPER_BYTES", "341")) +BINARY_SHA256 = os.environ.get( + "SESSION_MIGRATE_HERMES_WRAPPER_SHA256", + "d2c74d2575bc2cbfd9ddbb7b1d3d110cc68c1897e8dbf4882260d856b26cf104", +) @dataclass(frozen=True, slots=True) @@ -237,7 +244,9 @@ def verify_client(source: Path, binary: Path) -> dict[str, Any]: timeout=20, ) expected_line = f"Hermes Agent v{VERSION} ({RELEASE})" - if version.returncode or expected_line not in version.stdout: + # 0.22.2 appends build provenance ("· stable · local ") to the release. + first_line = version.stdout.splitlines()[0].strip() if version.stdout.splitlines() else "" + if version.returncode or not first_line.startswith(expected_line): raise SystemExit(f"expected {expected_line!r}, got {version.stdout!r}") size = binary.stat().st_size digest = hashlib.sha256(binary.read_bytes()).hexdigest() @@ -283,7 +292,7 @@ def write_config(home: Path, port: int) -> None: " default_model: fixture-model", " models: [fixture-model]", " discover_models: false", - " context_length: 32768", + " context_length: 131072", "display:", " interface: cli", " resume_display: minimal", diff --git a/scripts/native-corpus/sanitize-hermes.py b/scripts/native-corpus/sanitize-hermes.py index 8fab621..7bbbc50 100644 --- a/scripts/native-corpus/sanitize-hermes.py +++ b/scripts/native-corpus/sanitize-hermes.py @@ -19,6 +19,7 @@ "actual_cost_usd", "api_call_count", "archived", + "auto_archived", "billing_base_url", "billing_mode", "billing_provider", @@ -30,8 +31,11 @@ "compression_failure_error", "compression_fallback_streak", "compression_ineffective_count", + "compression_overload_streak", + "compression_recovery_deadline", "cost_source", "cost_status", + "created_source", "cwd", "display_name", "end_reason", @@ -69,14 +73,18 @@ "system_prompt", "system_prompt_hash", "thread_id", + "timings", "title", "title_source", "tool_call_count", + "tool_names", + "transport_profile", "user_id", } ) MESSAGE_FIELDS = frozenset( { + "absorbed_message_uids", "active", "api_content", "codex_message_items", @@ -88,6 +96,7 @@ "effect_disposition", "finish_reason", "id", + "message_uid", "observed", "platform_message_id", "reasoning", @@ -98,6 +107,8 @@ "timestamp", "token_count", "tool_call_id", + "tool_call_uid", + "tool_call_uids", "tool_calls", "tool_name", } @@ -122,6 +133,7 @@ "system_prompt", "system_prompt_hash", "thread_id", + "tool_names", "user_id", ) PUBLIC_CWD = "/fixture/work" @@ -139,9 +151,14 @@ class Result: def _scrub(value: Any, private_cwd: str, mutations: Counter[str]) -> Any: if isinstance(value, str): - replaced = value.replace(private_cwd, PUBLIC_CWD) - if replaced != value: - mutations["message path"] += value.count(private_cwd) + # On Windows the private cwd contains separators json.dumps also + # escapes inside tool_calls arguments; scrub both the plain and the + # JSON-escaped variant. + escaped = private_cwd.replace("\\", "\\\\") + replaced = value.replace(escaped, PUBLIC_CWD) + replaced = replaced.replace(private_cwd, PUBLIC_CWD) + if value.count(escaped) or value.count(private_cwd): + mutations["message path"] += replaced.count(PUBLIC_CWD) - value.count(PUBLIC_CWD) return replaced if isinstance(value, list): return [_scrub(item, private_cwd, mutations) for item in value] diff --git a/scripts/run-native-test-client.sh b/scripts/run-native-test-client.sh index d2c3dc7..cf5188e 100755 --- a/scripts/run-native-test-client.sh +++ b/scripts/run-native-test-client.sh @@ -99,6 +99,13 @@ case "$client" in 'tests/test_opencode_kilo_corpus_native.py::test_exact_client_cold_import_export_and_continuation_preserve_prefix[opencode]' ) ;; + opencode-v2) + tests=( + tests/test_opencode_v2.py::test_native_v2_isolated_import_export_dry_run_and_collision + tests/test_opencode_v2_native.py + tests/test_opencode_v2_review.py::test_stock_v2_accepts_and_cold_exports_review_sources_and_fixed_targets + ) + ;; openhands) tests=( tests/test_grok_kilo_openhands_native.py::test_openhands_1160_creates_native_tool_source_from_empty_state diff --git a/src/session_migrate/__init__.py b/src/session_migrate/__init__.py index f9319e8..b830747 100644 --- a/src/session_migrate/__init__.py +++ b/src/session_migrate/__init__.py @@ -1,3 +1,3 @@ """Native coding-agent session discovery and migration.""" -__version__ = "0.11.0" +__version__ = "0.12.1" diff --git a/src/session_migrate/catalog.py b/src/session_migrate/catalog.py index 2c34fe1..aaa2ecb 100644 --- a/src/session_migrate/catalog.py +++ b/src/session_migrate/catalog.py @@ -25,6 +25,7 @@ from session_migrate.errors import JsonlError, SessionMigrateError from session_migrate.formats import ( antigravity, + antigravity_desktop, codex, devin, hermes, @@ -237,6 +238,11 @@ def auto_roots( user_home / ".gemini" / "antigravity-cli", "default", ), + ( + AgentFormat.ANTIGRAVITY_DESKTOP, + user_home / ".gemini" / "antigravity", + "default", + ), ( AgentFormat.CURSOR, cursor_home, @@ -336,6 +342,11 @@ def auto_roots( antigravity_home = directory / ".gemini" / "antigravity-cli" if (antigravity_home / "conversations").is_dir(): candidates.append((AgentFormat.ANTIGRAVITY, antigravity_home, "project")) + antigravity_desktop_home = directory / ".gemini" / "antigravity" + if (antigravity_desktop_home / "conversations").is_dir(): + candidates.append( + (AgentFormat.ANTIGRAVITY_DESKTOP, antigravity_desktop_home, "project") + ) cursor_home = directory / ".cursor" if (cursor_home / "chats").is_dir(): candidates.append((AgentFormat.CURSOR, cursor_home, "project")) @@ -887,6 +898,7 @@ def add_root( AgentFormat.OPENCODE, AgentFormat.COPILOT, AgentFormat.ANTIGRAVITY, + AgentFormat.ANTIGRAVITY_DESKTOP, AgentFormat.CURSOR, AgentFormat.VIBE, AgentFormat.MUSE, @@ -946,6 +958,7 @@ def refresh( opencode_roots: Sequence[Path] = (), copilot_roots: Sequence[Path] = (), antigravity_roots: Sequence[Path] = (), + antigravity_desktop_roots: Sequence[Path] = (), cursor_roots: Sequence[Path] = (), vibe_roots: Sequence[Path] = (), muse_roots: Sequence[Path] = (), @@ -981,6 +994,8 @@ def refresh( self.add_root(AgentFormat.COPILOT, path) for path in antigravity_roots: self.add_root(AgentFormat.ANTIGRAVITY, path) + for path in antigravity_desktop_roots: + self.add_root(AgentFormat.ANTIGRAVITY_DESKTOP, path) for path in cursor_roots: self.add_root(AgentFormat.CURSOR, path) for path in vibe_roots: @@ -1110,7 +1125,10 @@ def _refresh_root(self, root: CatalogRoot, *, validate: bool) -> dict[str, int]: metadata = _codex_native_metadata(root_path) elif root.format == AgentFormat.COPILOT.value: metadata = _copilot_native_metadata(root_path) - elif root.format == AgentFormat.ANTIGRAVITY.value: + elif root.format in { + AgentFormat.ANTIGRAVITY.value, + AgentFormat.ANTIGRAVITY_DESKTOP.value, + }: metadata = _antigravity_native_metadata(root_path) else: metadata = _NativeMetadata({}, {}, {}, False) @@ -1156,6 +1174,7 @@ def _refresh_root(self, root: CatalogRoot, *, validate: bool) -> dict[str, int]: continue elif root.format in { AgentFormat.ANTIGRAVITY.value, + AgentFormat.ANTIGRAVITY_DESKTOP.value, AgentFormat.CURSOR.value, }: try: @@ -1175,8 +1194,8 @@ def _refresh_root(self, root: CatalogRoot, *, validate: bool) -> dict[str, int]: unchanged = bool( previous and previous["status"] != "missing" - and int(previous["device"]) == before.device - and int(previous["inode"]) == before.inode + and int(previous["device"]) == _sqlite_stat_int(before.device) + and int(previous["inode"]) == _sqlite_stat_int(before.inode) and int(previous["bytes"]) == before.size and int(previous["modified_ns"]) == before.modified_ns and previous["source_fingerprint"] == getattr(before, "fingerprint", None) @@ -1544,8 +1563,8 @@ def _upsert_session( scan.status, scan.reason, scan.records, - snapshot.device, - snapshot.inode, + _sqlite_stat_int(snapshot.device), + _sqlite_stat_int(snapshot.inode), snapshot.size, snapshot.modified_ns, getattr(snapshot, "fingerprint", None), @@ -1625,8 +1644,8 @@ def _upsert_virtual_session( scan.status, scan.reason, scan.records, - snapshot.device, - snapshot.inode, + _sqlite_stat_int(snapshot.device), + _sqlite_stat_int(snapshot.inode), snapshot.size, snapshot.modified_ns, snapshot.fingerprint, @@ -1922,7 +1941,7 @@ def _candidate_files(agent_format: AgentFormat, root: Path) -> Iterable[Path]: ] yield from sorted(candidates) return - if agent_format == AgentFormat.ANTIGRAVITY: + if agent_format in {AgentFormat.ANTIGRAVITY, AgentFormat.ANTIGRAVITY_DESKTOP}: conversations = root / "conversations" if not conversations.is_dir(): return @@ -2039,6 +2058,8 @@ def _raise_walk_error(error: OSError) -> None: def _scan_file(path: Path, agent_format: AgentFormat, root: Path) -> _Scan: if agent_format == AgentFormat.ANTIGRAVITY: return _scan_antigravity_file(path, root) + if agent_format == AgentFormat.ANTIGRAVITY_DESKTOP: + return _scan_antigravity_desktop_file(path, root) if agent_format == AgentFormat.CURSOR: return _scan_cursor_file(path, root) if agent_format == AgentFormat.VIBE: @@ -2450,6 +2471,7 @@ def _base_scan( lifecycle = "project" elif agent_format in { AgentFormat.ANTIGRAVITY, + AgentFormat.ANTIGRAVITY_DESKTOP, AgentFormat.CURSOR, AgentFormat.VIBE, AgentFormat.MUSE, @@ -2513,6 +2535,39 @@ def _scan_antigravity_file(path: Path, root: Path) -> _Scan: ) +def _scan_antigravity_desktop_file(path: Path, root: Path) -> _Scan: + base = _base_scan(path, AgentFormat.ANTIGRAVITY_DESKTOP, root, "candidate", None) + try: + parsed = antigravity_desktop.parse(path) + except SessionMigrateError: + return _replace_scan_status(base, "corrupt", "invalid_antigravity_database") + labels: list[_Label] = [] + if parsed.title: + title = _bounded(parsed.title, LABEL_LIMIT) + if title: + labels.append(_Label("native_title", title, 0, 110)) + has_conversation = any( + event.kind.value in {"message", "tool_call", "tool_result"} for event in parsed.events + ) + status = "candidate" if has_conversation else "corrupt" + reason = None if has_conversation else "no_conversation_records" + return _Scan( + session_id=parsed.session_id, + filename_session_id=base.filename_session_id, + cwd=_bounded(str(parsed.cwd), PATH_VALUE_LIMIT) if parsed.cwd else None, + started_at=parsed.started_at, + cli_version=parsed.cli_version, + history_mode=None, + kind=base.kind, + lifecycle=base.lifecycle, + parent_session_id=None, + status=status, + reason=reason, + records=parsed.raw_record_count, + labels=tuple(labels), + ) + + def _scan_cursor_file(path: Path, root: Path) -> _Scan: base = _base_scan(path, AgentFormat.CURSOR, root, "candidate", None) try: @@ -2948,9 +3003,14 @@ def _opencode_inventory( connection.execute("PRAGMA query_only = ON") connection.execute("PRAGMA busy_timeout = 1000") connection.execute("BEGIN") - columns = { - str(row[1]) for row in connection.execute("PRAGMA table_info(session)").fetchall() + tables = ("session_v2", "session") if label == "opencode" else ("session",) + schemas = { + table: { + str(row[1]) for row in connection.execute(f"PRAGMA table_info({table})").fetchall() + } + for table in tables } + schemas = {table: columns for table, columns in schemas.items() if columns} required = { "id", "directory", @@ -2959,23 +3019,38 @@ def _opencode_inventory( "time_created", "time_updated", } - if not required.issubset(columns): + if not schemas or any(not required.issubset(columns) for columns in schemas.values()): raise _OpenCodeInventoryError(f"{label}_schema_unsupported") - selected = [ - "id", - f"substr(directory, 1, {PATH_VALUE_LIMIT}) AS directory", - f"substr(title, 1, {LABEL_LIMIT}) AS title", - f"substr(version, 1, {LABEL_LIMIT}) AS version", - "time_created", - "time_updated", - ( - f"substr(parent_id, 1, {LABEL_LIMIT}) AS parent_id" - if "parent_id" in columns - else "NULL AS parent_id" - ), - "time_archived" if "time_archived" in columns else "NULL AS time_archived", - ] - rows = connection.execute(f"SELECT {', '.join(selected)} FROM session ORDER BY id") # noqa: S608 + queries = [] + for table, columns in schemas.items(): + selected = [ + "id", + f"substr(directory, 1, {PATH_VALUE_LIMIT}) AS directory", + ( + f"substr(COALESCE(title, 'Untitled session'), 1, {LABEL_LIMIT}) AS title" + if table == "session_v2" + else f"substr(title, 1, {LABEL_LIMIT}) AS title" + ), + f"substr(version, 1, {LABEL_LIMIT}) AS version", + "time_created", + "time_updated", + ( + f"substr(parent_id, 1, {LABEL_LIMIT}) AS parent_id" + if "parent_id" in columns + else "NULL AS parent_id" + ), + "time_archived" if "time_archived" in columns else "NULL AS time_archived", + ] + # Upgraded stores can retain the same ID in both schemas. The v2 + # projection is authoritative for migrated sessions; legacy-only + # sessions remain discoverable without reading message bodies. + where = ( + " WHERE NOT EXISTS (SELECT 1 FROM session_v2 WHERE session_v2.id = session.id)" + if table == "session" and "session_v2" in schemas + else "" + ) + queries.append(f"SELECT {', '.join(selected)} FROM {table}{where}") # noqa: S608 + rows = connection.execute(" UNION ALL ".join(queries) + " ORDER BY id") except _OpenCodeInventoryError: if connection is not None: connection.close() @@ -3087,6 +3162,21 @@ def _is_non_negative_sqlite_int(value: Any) -> bool: return isinstance(value, int) and not isinstance(value, bool) and value >= 0 +def _sqlite_stat_int(value: int) -> int: + """Fold one ``os.stat`` identity field into SQLite's signed 64-bit range. + + Windows reports ``st_dev`` as an unsigned 64-bit volume serial + (``15332589137577146414`` for an NTFS system volume), which SQLite cannot + bind at all: the INSERT raises ``OverflowError`` and the entire root scan + fails closed. The same 64 bits read signed identify the volume just as + uniquely, and both the stored value and every comparison go through here, + so refresh reuse keeps matching. + """ + + folded = int(value) & ((1 << 64) - 1) + return folded - (1 << 64) if folded > (1 << 63) - 1 else folded + + def _copilot_native_metadata(root: Path) -> _NativeMetadata: """Read only bounded picker names, never event/message content.""" diff --git a/src/session_migrate/cli.py b/src/session_migrate/cli.py index b2cc420..8f4fb36 100644 --- a/src/session_migrate/cli.py +++ b/src/session_migrate/cli.py @@ -18,8 +18,10 @@ content_free_result, convert_session, default_target_home, + detect_opencode_version, ensure_target_paths_available, install_antigravity_artifact, + install_antigravity_desktop_artifact, install_copilot_artifact, install_cursor_artifact, install_devin_artifact, @@ -239,6 +241,13 @@ def build_parser() -> argparse.ArgumentParser: default=[], help="register and scan an additional Antigravity CLI data home (repeatable)", ) + refresh_parser.add_argument( + "--antigravity-desktop-root", + type=_expanded_path, + action="append", + default=[], + help="register and scan an additional Antigravity Desktop data home (repeatable)", + ) refresh_parser.add_argument( "--cursor-root", type=_expanded_path, @@ -558,6 +567,7 @@ def main(argv: Sequence[str] | None = None) -> int: TargetFormat.OPENCODE, TargetFormat.KILO, TargetFormat.ANTIGRAVITY, + TargetFormat.ANTIGRAVITY_DESKTOP, TargetFormat.CURSOR, TargetFormat.HERMES, } @@ -567,6 +577,12 @@ def main(argv: Sequence[str] | None = None) -> int: "--target-cli only applies to OpenCode/Kilo/Antigravity/Cursor import " "and transfer, or Hermes import and transfer" ) + if ( + target_format == TargetFormat.OPENCODE + and args.command != "convert" + and not args.target_cli_version + ): + args.target_cli_version = detect_opencode_version(args.target_cli) artifact = convert_session( session, ConversionOptions( @@ -632,6 +648,13 @@ def main(argv: Sequence[str] | None = None) -> int: target_cli=args.target_cli, dry_run=dry_run, ) + elif target_format == TargetFormat.ANTIGRAVITY_DESKTOP and args.command != "convert": + install_antigravity_desktop_artifact( + artifact, + target_home=home, + target_cli=args.target_cli, + dry_run=dry_run, + ) elif target_format == TargetFormat.CURSOR and args.command != "convert": install_cursor_artifact( artifact, @@ -739,7 +762,10 @@ def _add_conversion_arguments( ) parser.add_argument( "--target-cli-version", - help="metadata version only; the writer schema remains pinned", + help=( + "target release; selects legacy or 2.0 transfer schema for OpenCode, " + "metadata only for other targets" + ), ) parser.add_argument( "--target-cli", @@ -811,6 +837,7 @@ def _run_catalog(args: argparse.Namespace) -> int: opencode_roots=args.opencode_root, copilot_roots=args.copilot_root, antigravity_roots=args.antigravity_root, + antigravity_desktop_roots=args.antigravity_desktop_root, cursor_roots=args.cursor_root, vibe_roots=args.vibe_root, muse_roots=args.muse_root, diff --git a/src/session_migrate/conversion.py b/src/session_migrate/conversion.py index 46d9497..4160472 100644 --- a/src/session_migrate/conversion.py +++ b/src/session_migrate/conversion.py @@ -21,6 +21,7 @@ from session_migrate.errors import FormatDetectionError, JsonlError, SessionMigrateError from session_migrate.formats import ( antigravity, + antigravity_desktop, claude, codex, copilot, @@ -34,6 +35,7 @@ muse, omp, opencode, + opencode_v2, openhands, pi, qwen, @@ -132,6 +134,8 @@ def load_session(path: Path, source_format: AgentFormat | None = None) -> Sessio # The adapter makes a transactionally consistent SQLite backup that # includes live WAL state; a plain file stat/hash check would not. return antigravity.parse_session(path) + if source_format == AgentFormat.ANTIGRAVITY_DESKTOP: + return antigravity_desktop.parse_session(path) if source_format == AgentFormat.CURSOR: # Cursor is also a live SQLite store. Its adapter takes a consistent # backup including committed WAL state before projecting the graph. @@ -193,10 +197,10 @@ def load_opencode_session( values.setdefault("OPENCODE_DISABLE_PRUNE", "true") cli = _resolve_opencode_cli(source_cli, values) observed_version = _opencode_version(cli, values) - if observed_version != opencode.PINNED_OPENCODE_VERSION: + if not opencode.supported_version(observed_version): raise SessionMigrateError( "OpenCode source CLI version mismatch: expected " - f"{opencode.PINNED_OPENCODE_VERSION}, observed {observed_version}" + f"{opencode.PINNED_OPENCODE_VERSION} or supported 2.0.x, observed {observed_version}" ) temporary_root = values.get("TMPDIR") with tempfile.TemporaryDirectory( @@ -209,7 +213,9 @@ def load_opencode_session( session = load_session(export_path, AgentFormat.OPENCODE) if session.session_id != session_id: raise SessionMigrateError("OpenCode export metadata does not match the requested session") - return replace(session, source_path=Path(f"opencode:{session_id}")) + return replace( + session, source_path=Path(f"opencode:{session_id}"), cli_version=observed_version + ) def load_kilo_session( @@ -390,6 +396,18 @@ def convert_session(session: Session, options: ConversionOptions) -> ConversionA model=options.model, timestamp=timestamp, ) + elif target_format == TargetFormat.ANTIGRAVITY_DESKTOP: + target_version = ( + options.target_cli_version or antigravity_desktop.PINNED_ANTIGRAVITY_DESKTOP_VERSION + ) + native_bytes, dropped = antigravity_desktop.serialize( + session, + session_id=target_id, + cwd=target_cwd, + cli_version=target_version, + model=options.model, + timestamp=timestamp, + ) elif target_format == TargetFormat.CURSOR: target_version = options.target_cli_version or cursor.PINNED_CURSOR_VERSION native_bytes, dropped = cursor.serialize( @@ -522,13 +540,25 @@ def convert_session(session: Session, options: ConversionOptions) -> ConversionA timestamp=timestamp, ) pinned_target = _pinned_target_version(target_format) - if target_version != pinned_target: + observed_target = target_version + version_message = "target metadata version differs but the writer schema remains pinned" + if target_format == TargetFormat.OPENCODE and opencode.is_v2(target_version): + pinned_target = opencode_v2.VALIDATED_VERSION + observed_target = opencode_v2.version(target_version) + version_message = ( + "selected OpenCode 2.0 transfer schema; exact target release differs " + "from the validated native integration release" + ) + target_validated = observed_target == pinned_target or ( + target_format == TargetFormat.OPENCODE and observed_target in opencode_v2.VALIDATED_VERSIONS + ) + if not target_validated: warnings.append( { "code": "unvalidated_target_version", "observed": target_version, "validated": pinned_target, - "message": "target metadata version differs but the writer schema remains pinned", + "message": version_message, } ) if not native_bytes: @@ -578,6 +608,9 @@ def convert_session(session: Session, options: ConversionOptions) -> ConversionA AgentFormat.OPENCODE: opencode.PINNED_OPENCODE_VERSION, AgentFormat.COPILOT: copilot.PINNED_COPILOT_VERSION, AgentFormat.ANTIGRAVITY: antigravity.PINNED_ANTIGRAVITY_VERSION, + AgentFormat.ANTIGRAVITY_DESKTOP: ( + antigravity_desktop.PINNED_ANTIGRAVITY_DESKTOP_VERSION + ), AgentFormat.CURSOR: cursor.PINNED_CURSOR_VERSION, AgentFormat.VIBE: vibe.PINNED_VIBE_VERSION, AgentFormat.MUSE: muse.PINNED_MUSE_VERSION, @@ -590,7 +623,13 @@ def convert_session(session: Session, options: ConversionOptions) -> ConversionA AgentFormat.MASTRACODE: mastracode.PINNED_MASTRACODE_VERSION, AgentFormat.DEVIN: devin.PINNED_DEVIN_VERSION, }[session.source_format] - if session.cli_version != pinned_source: + if session.source_format == AgentFormat.OPENCODE and opencode.is_v2(session.cli_version): + pinned_source = opencode_v2.VALIDATED_VERSION + source_validated = session.cli_version == pinned_source or ( + session.source_format == AgentFormat.OPENCODE + and opencode_v2.version(session.cli_version) in opencode_v2.VALIDATED_VERSIONS + ) + if not source_validated: warnings.append( { "code": "unvalidated_source_version", @@ -638,6 +677,8 @@ def target_import_paths(artifact: ConversionArtifact, target_home: Path) -> tupl native_path = target_home / copilot.session_relative_path(artifact.session_id) elif artifact.target_format == TargetFormat.ANTIGRAVITY: native_path = target_home / antigravity.session_relative_path(artifact.session_id) + elif artifact.target_format == TargetFormat.ANTIGRAVITY_DESKTOP: + native_path = target_home / antigravity_desktop.session_relative_path(artifact.session_id) elif artifact.target_format == TargetFormat.CURSOR: native_path = target_home / cursor.session_relative_path(artifact.session_id, artifact.cwd) elif artifact.target_format == TargetFormat.VIBE: @@ -682,6 +723,8 @@ def default_target_home(target_format: TargetFormat | AgentFormat) -> Path: return Path(configured).expanduser() if configured else Path.home() / ".copilot" if target_format.value == TargetFormat.ANTIGRAVITY.value: return antigravity.app_data_home() + if target_format.value == TargetFormat.ANTIGRAVITY_DESKTOP.value: + return antigravity_desktop.app_data_home() if target_format.value == TargetFormat.CURSOR.value: return cursor.config_home() if target_format.value == TargetFormat.VIBE.value: @@ -786,16 +829,16 @@ def write_artifact(artifact: ConversionArtifact, *, output_path: Path, manifest_ ): raise JsonlError("conversion artifact changed during installation") except BaseException: + output_guard = _close_guard(output_guard) + manifest_guard = _close_guard(manifest_guard) if output_identity is not None: _unlink_if_identity_matches(output_path, output_identity) if manifest_identity is not None: _unlink_if_identity_matches(manifest_path, manifest_identity) raise finally: - if output_guard is not None: - os.close(output_guard) - if manifest_guard is not None: - os.close(manifest_guard) + output_guard = _close_guard(output_guard) + manifest_guard = _close_guard(manifest_guard) def install_copilot_artifact( @@ -848,6 +891,8 @@ def install_copilot_artifact( if not all(_path_matches_identity(path, identity) for path, identity in identities): raise JsonlError("Copilot artifact changed during installation") except BaseException: + while guards: + _close_guard(guards.pop()) for path, identity in reversed(identities): _unlink_if_identity_matches(path, identity) if created_directory: @@ -855,8 +900,8 @@ def install_copilot_artifact( session_directory.rmdir() raise finally: - for descriptor in guards: - os.close(descriptor) + while guards: + _close_guard(guards.pop()) return events_path, manifest_path @@ -925,6 +970,7 @@ def install_antigravity_artifact( manifest_bytes, ) except BaseException as exc: + reservation_guard = _close_guard(reservation_guard) if reservation_identity is not None: _unlink_if_identity_matches(manifest_path, reservation_identity) if install_succeeded: @@ -934,8 +980,92 @@ def install_antigravity_artifact( ) from exc raise finally: - if reservation_guard is not None: - os.close(reservation_guard) + reservation_guard = _close_guard(reservation_guard) + return native_path, manifest_path + + +def install_antigravity_desktop_artifact( + artifact: ConversionArtifact, + *, + target_home: Path, + target_cli: Path | None = None, + dry_run: bool = False, + environ: Mapping[str, str] | None = None, +) -> tuple[Path, Path]: + """Install a pinned Antigravity Desktop DB and finalize a content-free manifest.""" + + if artifact.target_format != TargetFormat.ANTIGRAVITY_DESKTOP: + raise SessionMigrateError( + "Antigravity Desktop installation requires an Antigravity Desktop artifact" + ) + if artifact.target_cli_version != antigravity_desktop.PINNED_ANTIGRAVITY_DESKTOP_VERSION: + raise SessionMigrateError( + "automatic Antigravity Desktop import requires target metadata version " + f"{antigravity_desktop.PINNED_ANTIGRAVITY_DESKTOP_VERSION}; " + "convert-only artifacts may opt into unvalidated metadata versions" + ) + native_path, manifest_path = target_import_paths(artifact, target_home) + ensure_target_paths_available(manifest_path) + if dry_run: + installed = antigravity_desktop.install_database( + artifact.native_bytes, + session_id=artifact.session_id, + cwd=artifact.cwd, + timestamp=artifact.timestamp, + title=artifact.source.title, + target_home=target_home, + target_cli=target_cli, + dry_run=True, + environ=environ, + ) + if installed.conversation_path != native_path: + raise SessionMigrateError( + "Antigravity Desktop installer resolved an unexpected target path" + ) + return native_path, manifest_path + + manifest_bytes = ( + json.dumps(artifact.manifest(output_path=native_path), indent=2, sort_keys=True) + "\n" + ).encode() + reservation_identity: tuple[int, int] | None = None + reservation_guard: int | None = None + install_succeeded = False + try: + reservation_identity = write_private_atomic(manifest_path, b"") + reservation_guard = _open_identity_guard(manifest_path, reservation_identity, writable=True) + installed = antigravity_desktop.install_database( + artifact.native_bytes, + session_id=artifact.session_id, + cwd=artifact.cwd, + timestamp=artifact.timestamp, + title=artifact.source.title, + target_home=target_home, + target_cli=target_cli, + environ=environ, + ) + install_succeeded = True + if installed.conversation_path != native_path: + raise SessionMigrateError( + "Antigravity Desktop installer resolved an unexpected target path" + ) + _write_reserved_file( + reservation_guard, + manifest_path, + reservation_identity, + manifest_bytes, + ) + except BaseException as exc: + reservation_guard = _close_guard(reservation_guard) + if reservation_identity is not None: + _unlink_if_identity_matches(manifest_path, reservation_identity) + if install_succeeded: + raise SessionMigrateError( + "Antigravity Desktop import succeeded but migrator manifest finalization failed; " + f"the native session may already exist as {artifact.session_id}" + ) from exc + raise + finally: + reservation_guard = _close_guard(reservation_guard) return native_path, manifest_path @@ -1000,6 +1130,7 @@ def install_cursor_artifact( manifest_bytes, ) except BaseException as exc: + reservation_guard = _close_guard(reservation_guard) if reservation_identity is not None: _unlink_if_identity_matches(manifest_path, reservation_identity) if install_succeeded: @@ -1009,8 +1140,7 @@ def install_cursor_artifact( ) from exc raise finally: - if reservation_guard is not None: - os.close(reservation_guard) + reservation_guard = _close_guard(reservation_guard) return native_path, manifest_path @@ -1063,6 +1193,8 @@ def install_vibe_artifact( if not all(_path_matches_identity(path, identity) for path, identity in identities): raise JsonlError("Vibe artifact changed during installation") except BaseException: + while guards: + _close_guard(guards.pop()) for path, identity in reversed(identities): _unlink_if_identity_matches(path, identity) if created_directory: @@ -1070,8 +1202,8 @@ def install_vibe_artifact( session_directory.rmdir() raise finally: - for descriptor in guards: - os.close(descriptor) + while guards: + _close_guard(guards.pop()) return messages_path, manifest_path @@ -1253,6 +1385,47 @@ def install_openhands_artifact( return events_path, manifest_path +def _resolve_hermes_title_conflict( + artifact: ConversionArtifact, *, target_home: Path +) -> ConversionArtifact: + """Install without the title when the target store already holds it. + + Hermes enforces ``idx_sessions_title_unique``, a partial unique index over + non-null session titles, so a second session cannot reuse one; its own + importer treats the title as cosmetic and leaves a conflicting session + untitled. The migrator does the same and counts the omission, instead of + letting a cosmetic field refuse the whole migration. + + The store is sampled before the import transaction, which still enforces the + constraint itself: a title created by a concurrent writer in between fails + closed, with no partial write. + """ + + title = hermes.bundle_title(artifact.native_bytes) + if not title: + return artifact + database = target_home.expanduser() / hermes.HERMES_STATE_FILENAME + if not hermes.title_taken(database, title): + return artifact + dropped = dict(artifact.dropped) + dropped["title:duplicate_in_target"] = 1 + return replace( + artifact, + native_bytes=hermes.without_title(artifact.native_bytes), + dropped=dropped, + warnings=( + *artifact.warnings, + { + "code": "target_title_omitted", + "title": title, + "message": ( + "target store already holds this title; the session was installed without one" + ), + }, + ), + ) + + def install_hermes_artifact( artifact: ConversionArtifact, *, @@ -1266,6 +1439,27 @@ def install_hermes_artifact( if artifact.target_format != TargetFormat.HERMES: raise SessionMigrateError("Hermes installation requires a Hermes artifact") + artifact = _resolve_hermes_title_conflict(artifact, target_home=target_home) + measured = hermes.measured_session_bytes(json.loads(artifact.native_bytes)["session"]) + if measured > hermes.PINNED_HERMES_IMPORT_SESSION_BYTES: + artifact = replace( + artifact, + warnings=( + *artifact.warnings, + { + "code": "target_import_budget_raised", + "session_bytes": measured, + "default_bytes": hermes.PINNED_HERMES_IMPORT_SESSION_BYTES, + "budget_bytes": hermes.import_budget_bytes(measured), + "message": ( + "the target store's per-session import cap was raised to fit this " + "bundle, which carries inline image data; the store's own web import " + "applies the same ceiling" + ), + }, + ), + ) + def install(simulate: bool) -> Path: result = hermes.install_bundle( artifact.native_bytes, @@ -1375,6 +1569,7 @@ def _install_shared_database_artifact( manifest_bytes, ) except BaseException as exc: + reservation_guard = _close_guard(reservation_guard) if reservation_identity is not None: _unlink_if_identity_matches(manifest_path, reservation_identity) if import_succeeded: @@ -1384,8 +1579,7 @@ def _install_shared_database_artifact( ) from exc raise finally: - if reservation_guard is not None: - os.close(reservation_guard) + reservation_guard = _close_guard(reservation_guard) return database, manifest_path @@ -1405,18 +1599,19 @@ def install_opencode_artifact( values = dict(os.environ if environ is None else environ) values.setdefault("OPENCODE_DISABLE_AUTOUPDATE", "true") values.setdefault("OPENCODE_DISABLE_PRUNE", "true") - if artifact.target_cli_version != opencode.PINNED_OPENCODE_VERSION: + if not opencode.supported_version(artifact.target_cli_version): raise SessionMigrateError( - "automatic OpenCode import requires target metadata version " - f"{opencode.PINNED_OPENCODE_VERSION}; convert-only artifacts may opt into " - "unvalidated metadata versions" + "automatic OpenCode import requires target metadata version 1.17.20 or supported 2.0.x" ) cli = _resolve_opencode_cli(target_cli, values) observed_version = _opencode_version(cli, values) - if observed_version != opencode.PINNED_OPENCODE_VERSION: + if not opencode.supported_version(observed_version): raise SessionMigrateError( - "OpenCode CLI version mismatch: expected " - f"{opencode.PINNED_OPENCODE_VERSION}, observed {observed_version}" + f"OpenCode CLI version mismatch: unsupported observed version {observed_version}" + ) + if opencode.is_v2(artifact.target_cli_version) != opencode.is_v2(observed_version): + raise SessionMigrateError( + "OpenCode CLI version mismatch: artifact and CLI transfer schemas differ" ) if artifact.session_id in _opencode_session_ids(cli, values): raise SessionMigrateError( @@ -1456,12 +1651,25 @@ def install_opencode_artifact( os.chmod(directory, 0o700) bundle_path = directory / "import.json" write_private_atomic(bundle_path, artifact.native_bytes) - _invoke_opencode_import(cli, bundle_path, values) + if opencode.is_v2(observed_version): + _invoke_opencode_import(cli, bundle_path, values, cwd=artifact.cwd) + else: + _invoke_opencode_import(cli, bundle_path, values) # From this point onward, any local cleanup/finalization failure # must warn that the native session may already exist. import_succeeded = True - if artifact.session_id not in _opencode_session_ids(cli, values): + if opencode.is_v2(observed_version): + with tempfile.TemporaryDirectory( + prefix="session-migrate-verify-", dir=temporary_root + ) as name: + _invoke_opencode_export( + cli, artifact.session_id, Path(name) / "verify.json", values + ) + exported = opencode.parse_import(Path(name) / "verify.json") + if exported.session_id != artifact.session_id: + raise SessionMigrateError("OpenCode imported a different session ID") + elif artifact.session_id not in _opencode_session_ids(cli, values): raise SessionMigrateError( "OpenCode import returned success but the session was not discoverable afterward" ) @@ -1472,6 +1680,11 @@ def install_opencode_artifact( manifest_bytes, ) except BaseException as exc: + # Windows refuses to delete a file that still has an open handle, so release + # the identity guard *before* the rollback unlink. Otherwise the cleanup + # error replaces the failure that triggered the rollback and the user never + # learns why finalization stopped, although the native import already ran. + reservation_guard = _close_guard(reservation_guard) if reservation_identity is not None: _unlink_if_identity_matches(manifest_path, reservation_identity) if import_succeeded: @@ -1564,6 +1777,8 @@ def install_kilo_artifact( manifest_bytes, ) except BaseException as exc: + # Same Windows rule as the OpenCode path: close the guard before unlinking. + reservation_guard = _close_guard(reservation_guard) if reservation_identity is not None: _unlink_if_identity_matches(manifest_path, reservation_identity) if import_succeeded: @@ -1649,6 +1864,9 @@ def _validate_native_bytes(data: bytes, target_format: TargetFormat, session_id: if target_format == TargetFormat.ANTIGRAVITY: antigravity.validate_native_bytes(data, session_id) return + if target_format == TargetFormat.ANTIGRAVITY_DESKTOP: + antigravity_desktop.validate_native_bytes(data, session_id) + return if target_format == TargetFormat.CURSOR: cursor.validate_native_bytes(data, session_id) return @@ -1703,6 +1921,7 @@ def _pinned_target_version(target_format: TargetFormat) -> str: TargetFormat.OPENCODE: opencode.PINNED_OPENCODE_VERSION, TargetFormat.COPILOT: copilot.PINNED_COPILOT_VERSION, TargetFormat.ANTIGRAVITY: antigravity.PINNED_ANTIGRAVITY_VERSION, + TargetFormat.ANTIGRAVITY_DESKTOP: (antigravity_desktop.PINNED_ANTIGRAVITY_DESKTOP_VERSION), TargetFormat.CURSOR: cursor.PINNED_CURSOR_VERSION, TargetFormat.VIBE: vibe.PINNED_VIBE_VERSION, TargetFormat.MUSE: muse.PINNED_MUSE_VERSION, @@ -1736,6 +1955,8 @@ def _native_record_count(data: bytes, target_format: TargetFormat) -> int: return kimi.native_record_count(data) if target_format == TargetFormat.ANTIGRAVITY: return antigravity.native_record_count(data) + if target_format == TargetFormat.ANTIGRAVITY_DESKTOP: + return antigravity_desktop.native_record_count(data) if target_format == TargetFormat.CURSOR: return cursor.native_record_count(data) if target_format == TargetFormat.VIBE: @@ -1744,6 +1965,8 @@ def _native_record_count(data: bytes, target_format: TargetFormat) -> int: return data.count(b"\n") value = json.loads(data) messages = value.get("messages", []) if isinstance(value, dict) else [] + if opencode_v2.is_bundle(value): + return opencode_v2.record_count(value) return ( 1 + len(messages) @@ -1799,11 +2022,28 @@ def _opencode_version(cli: Path, environ: Mapping[str, str]) -> str: version = completed.stdout.strip() if not version or "\n" in version: raise SessionMigrateError("OpenCode CLI returned an invalid version string") + if version.startswith("opencode v"): + version = version.removeprefix("opencode v") return version +def detect_opencode_version(target_cli: Path | None = None) -> str: + values = dict(os.environ) + values.setdefault("OPENCODE_DISABLE_AUTOUPDATE", "true") + values.setdefault("OPENCODE_DISABLE_PRUNE", "true") + observed = _opencode_version(_resolve_opencode_cli(target_cli, values), values) + if not opencode.supported_version(observed): + raise SessionMigrateError(f"unsupported OpenCode CLI version: {observed}") + return observed + + def _opencode_session_ids(cli: Path, environ: Mapping[str, str]) -> set[str]: - completed = _run_opencode([str(cli), "session", "list", "--format", "json", "--pure"], environ) + flags = ( + ["--standalone", "--max-count", "1000000"] + if opencode.is_v2(_opencode_version(cli, environ)) + else ["--pure"] + ) + completed = _run_opencode([str(cli), "session", "list", "--format", "json", *flags], environ) if len(completed.stdout.encode()) > 64 * 1024 * 1024: raise SessionMigrateError("OpenCode session list exceeded the safety limit") # Pinned OpenCode 1.17.20 emits an empty stream, rather than ``[]``, when @@ -1824,8 +2064,28 @@ def _opencode_session_ids(cli: Path, environ: Mapping[str, str]) -> set[str]: return result -def _invoke_opencode_import(cli: Path, bundle_path: Path, environ: Mapping[str, str]) -> None: - _run_opencode([str(cli), "import", str(bundle_path), "--pure"], environ) +def _invoke_opencode_import( + cli: Path, bundle_path: Path, environ: Mapping[str, str], *, cwd: Path | None = None +) -> None: + if opencode.is_v2(_opencode_version(cli, environ)): + result = _run_opencode( + [ + str(cli), + "session", + "import", + str(bundle_path), + "--directory", + str(cwd or Path.cwd()), + "--standalone", + ], + environ, + ) + if not result.stdout.startswith("Imported session: "): + raise SessionMigrateError( + "OpenCode native import refused the session; no overwrite attempted" + ) + else: + _run_opencode([str(cli), "import", str(bundle_path), "--pure"], environ) def _invoke_opencode_export( @@ -1843,7 +2103,11 @@ def _invoke_opencode_export( try: descriptor = os.open(bundle_path, flags, 0o600) completed = subprocess.run( - [str(cli), "export", session_id, "--pure"], + ( + [str(cli), "session", "export", session_id, "--standalone"] + if opencode.is_v2(_opencode_version(cli, environ)) + else [str(cli), "export", session_id, "--pure"] + ), env=dict(environ), check=False, stdout=descriptor, @@ -2056,6 +2320,8 @@ def _write_reserved_file( def _fsync_directory(path: Path) -> None: + if os.name == "nt": + return descriptor = os.open(path, os.O_RDONLY | getattr(os, "O_DIRECTORY", 0)) try: os.fsync(descriptor) @@ -2080,6 +2346,22 @@ def _unlink_if_identity_matches(path: Path, identity: tuple[int, int]) -> None: path.unlink() +def _close_guard(descriptor: int | None) -> None: + """Close one identity-guard descriptor and hand back ``None``. + + Windows refuses to delete a file while any handle still has it open, so a + rollback path must release its own guards *before* unlinking; the guard is + therefore closed in the failure path and re-closed (as a no-op) in the + surrounding ``finally``. + """ + + if descriptor is None: + return None + with suppress(OSError): + os.close(descriptor) + return None + + def _open_identity_guard(path: Path, identity: tuple[int, int], *, writable: bool = False) -> int: try: descriptor = os.open( diff --git a/src/session_migrate/discovery.py b/src/session_migrate/discovery.py index 0545fed..0f9c31a 100644 --- a/src/session_migrate/discovery.py +++ b/src/session_migrate/discovery.py @@ -57,7 +57,7 @@ def locate_session( "--source-cwd applies only to Claude/Pi/OMP/Cursor/Vibe/Qwen/Kimi discovery" ) matches = [home / "session-state" / normalized_id / "events.jsonl"] - elif source_format == AgentFormat.ANTIGRAVITY: + elif source_format in {AgentFormat.ANTIGRAVITY, AgentFormat.ANTIGRAVITY_DESKTOP}: if cwd is not None: raise SessionMigrateError( "--source-cwd applies only to Claude/Pi/OMP/Cursor/Vibe/Qwen/Kimi discovery" diff --git a/src/session_migrate/formats/__init__.py b/src/session_migrate/formats/__init__.py index 1600409..dc3e932 100644 --- a/src/session_migrate/formats/__init__.py +++ b/src/session_migrate/formats/__init__.py @@ -2,6 +2,7 @@ from session_migrate.formats import ( antigravity, + antigravity_desktop, claude, codex, copilot, @@ -23,6 +24,7 @@ __all__ = [ "antigravity", + "antigravity_desktop", "claude", "codex", "copilot", diff --git a/src/session_migrate/formats/antigravity.py b/src/session_migrate/formats/antigravity.py index c98e56f..b735857 100644 --- a/src/session_migrate/formats/antigravity.py +++ b/src/session_migrate/formats/antigravity.py @@ -44,13 +44,16 @@ ) PINNED_ANTIGRAVITY_LINUX_X86_64_SIZE = 205_545_512 -MAX_NATIVE_BYTES = 256 * 1024 * 1024 +MAX_NATIVE_BYTES = 512 * 1024 * 1024 MAX_STEPS = 100_000 MAX_PROTO_FIELDS = 100_000 MAX_PROTO_VALUE_BYTES = 64 * 1024 * 1024 MAX_JSON_NODES = 100_000 MAX_TEXT_BYTES = 64 * 1024 * 1024 PROJECT_ID = "default-cli-project" +PROJECT_ID_DESKTOP = "outside-of-project" +SOURCE_CLI = 17 +SOURCE_DESKTOP = 1 STEP_STATUS_DONE = 3 STEP_STATUS_ERROR = 7 @@ -415,34 +418,41 @@ def validate_native_bytes(data: bytes, session_id: str) -> None: def snapshot_database_bytes(path: Path) -> bytes: - """Return a bounded, transactionally consistent copy, including live WAL state.""" + """Create a consistent SQLite snapshot and return its bytes.""" - source = path.expanduser() - try: - info = source.lstat() - except OSError as exc: - raise SessionMigrateError("cannot inspect Antigravity conversation database") from exc - if stat.S_ISLNK(info.st_mode) or not stat.S_ISREG(info.st_mode): - raise SessionMigrateError("Antigravity conversation source must be a regular file") - if info.st_size > MAX_NATIVE_BYTES: - raise SessionMigrateError("Antigravity conversation exceeds the database safety limit") + path = Path(path) + + if not path.is_file(): + raise SessionMigrateError(f"Antigravity database not found: {path}") with tempfile.TemporaryDirectory(prefix="session-migrate-antigravity-snapshot-") as directory: - target = Path(directory) / "snapshot.db" - uri = f"file:{quote(str(source.resolve()), safe='/')}?mode=ro" + snapshot_path = Path(directory) / "snapshot.db" + + source_db = None + snapshot_db = None + try: - with sqlite3.connect(uri, uri=True, timeout=5) as origin: - origin.execute("PRAGMA trusted_schema=OFF") - with sqlite3.connect(target) as destination: - origin.backup(destination) - data = target.read_bytes() - except (OSError, sqlite3.Error) as exc: + source_db = sqlite3.connect(path) + snapshot_db = sqlite3.connect(snapshot_path) + + source_db.backup(snapshot_db) + snapshot_db.commit() + + except sqlite3.Error as exc: raise SessionMigrateError( - "cannot make a consistent Antigravity database snapshot" + f"failed to snapshot Antigravity database {path}: {exc}" ) from exc - if not data or len(data) > MAX_NATIVE_BYTES: - raise SessionMigrateError("Antigravity database snapshot violates the safety limit") - return data + + finally: + if snapshot_db is not None: + snapshot_db.close() + snapshot_db = None + + if source_db is not None: + source_db.close() + source_db = None + + return snapshot_path.read_bytes() def session_relative_path(session_id: str) -> Path: @@ -632,6 +642,12 @@ def install_database( raise SessionMigrateError("Antigravity install paths changed during transaction") summaries.execute("COMMIT") except BaseException: + if conversation_guard is not None: + os.close(conversation_guard) + conversation_guard = None + if summary_guard is not None: + os.close(summary_guard) + summary_guard = None if created_identity is not None: _unlink_if_same_file(conversation_path, created_identity) raise @@ -649,85 +665,123 @@ def _build_database( conversation_id: str, trajectory_id: str, started_at: str, + source: int = SOURCE_CLI, + project_id: str = PROJECT_ID, ) -> bytes: with tempfile.TemporaryDirectory(prefix="session-migrate-antigravity-build-") as directory: path = Path(directory) / "conversation.db" + + db = None try: - with sqlite3.connect(path) as db: - db.execute("PRAGMA journal_mode=DELETE") - db.executescript( - """ - CREATE TABLE trajectory_meta ( - trajectory_id text, cascade_id text, trajectory_type integer, source integer, - PRIMARY KEY (trajectory_id) - ); - CREATE TABLE steps ( - idx integer, step_type integer NOT NULL DEFAULT 0, - status integer NOT NULL DEFAULT 0, - has_subtrajectory numeric NOT NULL DEFAULT false, - metadata blob, error_details blob, permissions blob, task_details blob, - render_info blob, step_payload blob, - step_format integer NOT NULL DEFAULT 0, - PRIMARY KEY (idx) - ); - CREATE INDEX idx_steps_status ON steps(status); - CREATE INDEX idx_steps_step_type ON steps(step_type); - CREATE TABLE gen_metadata ( - idx integer, data blob, size integer NOT NULL DEFAULT 0, PRIMARY KEY (idx) - ); - CREATE TABLE executor_metadata (idx integer, data blob, PRIMARY KEY (idx)); - CREATE TABLE parent_references (idx integer, data blob, PRIMARY KEY (idx)); - CREATE TABLE trajectory_metadata_blob ( - id text DEFAULT "main", data blob, PRIMARY KEY (id) - ); - CREATE TABLE battle_mode_infos (idx integer, data blob, PRIMARY KEY (idx)); - """ - ) - db.execute( - "INSERT INTO trajectory_meta VALUES(?,?,4,17)", - (trajectory_id, conversation_id), - ) - db.executemany( - "INSERT INTO steps(" - "idx,step_type,status,has_subtrajectory,metadata,step_payload,step_format" - ") VALUES(?,?,?,0,?,?,0)", - [ - (row.index, row.step_type, row.status, row.metadata, row.payload) - for row in rows - ], - ) - seconds = int(_parse_timestamp(started_at).timestamp()) - trajectory_metadata = ( - _field_bytes(2, _field_varint(1, seconds)) - + _field_text(6, conversation_id) - + _field_text(18, PROJECT_ID) - ) - db.execute( - "INSERT INTO trajectory_metadata_blob(id,data) VALUES('main',?)", - (trajectory_metadata,), - ) - result = db.execute("PRAGMA integrity_check").fetchone() - if result != ("ok",): - raise SessionMigrateError( - "generated Antigravity database failed integrity check" + db = sqlite3.connect(path) + + db.execute("PRAGMA journal_mode=DELETE") + db.executescript( + """ + CREATE TABLE trajectory_meta ( + trajectory_id text, cascade_id text, trajectory_type integer, source integer, + PRIMARY KEY (trajectory_id) + ); + CREATE TABLE steps ( + idx integer, step_type integer NOT NULL DEFAULT 0, + status integer NOT NULL DEFAULT 0, + has_subtrajectory numeric NOT NULL DEFAULT false, + metadata blob, error_details blob, permissions blob, task_details blob, + render_info blob, step_payload blob, + step_format integer NOT NULL DEFAULT 0, + PRIMARY KEY (idx) + ); + CREATE INDEX idx_steps_status ON steps(status); + CREATE INDEX idx_steps_step_type ON steps(step_type); + CREATE TABLE gen_metadata ( + idx integer, data blob, size integer NOT NULL DEFAULT 0, PRIMARY KEY (idx) + ); + CREATE TABLE executor_metadata (idx integer, data blob, PRIMARY KEY (idx)); + CREATE TABLE parent_references (idx integer, data blob, PRIMARY KEY (idx)); + CREATE TABLE trajectory_metadata_blob ( + id text DEFAULT "main", data blob, PRIMARY KEY (id) + ); + CREATE TABLE battle_mode_infos (idx integer, data blob, PRIMARY KEY (idx)); + """ + ) + + db.execute( + "INSERT INTO trajectory_meta VALUES(?,?,4,?)", + (trajectory_id, conversation_id, source), + ) + + db.executemany( + "INSERT INTO steps(" + "idx,step_type,status,has_subtrajectory,metadata,step_payload,step_format" + ") VALUES(?,?,?,0,?,?,0)", + [ + ( + row.index, + row.step_type, + row.status, + row.metadata, + row.payload, ) - data = path.read_bytes() + for row in rows + ], + ) + + seconds = int(_parse_timestamp(started_at).timestamp()) + + trajectory_metadata = ( + _field_bytes(2, _field_varint(1, seconds)) + + _field_text(6, conversation_id) + + _field_text(18, project_id) + ) + + db.execute( + "INSERT INTO trajectory_metadata_blob(id,data) VALUES('main',?)", + (trajectory_metadata,), + ) + + result = db.execute("PRAGMA integrity_check").fetchone() + + if result != ("ok",): + raise SessionMigrateError("generated Antigravity database failed integrity check") + + db.commit() + except sqlite3.Error as exc: raise SessionMigrateError("cannot construct Antigravity conversation database") from exc + + finally: + if db is not None: + db.close() + + data = path.read_bytes() + if not data or len(data) > MAX_NATIVE_BYTES: raise SessionMigrateError("generated Antigravity database violates the safety limit") + return data def _parse_database_bytes( - data: bytes, *, expected_session_id: str | None, generated: bool = False + data: bytes, + *, + expected_session_id: str | None, + generated: bool = False, + expected_source: int = SOURCE_CLI, + expected_project_id: str = PROJECT_ID, + is_desktop: bool = False, + cli_version: str = PINNED_ANTIGRAVITY_VERSION, ) -> ParsedAntigravitySession: if not data or len(data) > MAX_NATIVE_BYTES: raise SessionMigrateError("Antigravity conversation violates the database safety limit") digest = hashlib.sha256(data).hexdigest() with _database_from_bytes(data) as db: conversation_id, trajectory_id, rows, started_at, metadata_losses = _validate_database( - db, expected_session_id=expected_session_id, generated=generated + db, + expected_session_id=expected_session_id, + generated=generated, + expected_source=expected_source, + expected_project_id=expected_project_id, + is_desktop=is_desktop, ) events = list(_project_events(rows)) for reason, count in metadata_losses.items(): @@ -744,7 +798,7 @@ def _parse_database_bytes( trajectory_id=trajectory_id, cwd=None, started_at=started_at, - cli_version=PINNED_ANTIGRAVITY_VERSION, + cli_version=cli_version, model=None, title=None, events=tuple(events), @@ -780,7 +834,13 @@ def _database_from_bytes(data: bytes) -> Iterator[sqlite3.Connection]: def _validate_database( - db: sqlite3.Connection, *, expected_session_id: str | None, generated: bool + db: sqlite3.Connection, + *, + expected_session_id: str | None, + generated: bool, + expected_source: int = SOURCE_CLI, + expected_project_id: str = PROJECT_ID, + is_desktop: bool = False, ) -> tuple[str, str, tuple[_StepRow, ...], str | None, dict[str, int]]: try: page_size = int(db.execute("PRAGMA page_size").fetchone()[0]) @@ -794,10 +854,11 @@ def _validate_database( trajectory_rows = db.execute( "SELECT trajectory_id,cascade_id,trajectory_type,source FROM trajectory_meta" ).fetchall() - main_rows = [row for row in trajectory_rows if row[2:] == (4, 17)] + main_rows = [row for row in trajectory_rows if row[2:] == (4, expected_source)] if len(main_rows) != 1: + desc = "Desktop" if is_desktop else "CLI" raise SessionMigrateError( - "Antigravity database must contain one CLI cascade trajectory" + f"Antigravity database must contain one {desc} cascade trajectory" ) if generated and len(trajectory_rows) != 1: raise SessionMigrateError("generated Antigravity database has auxiliary trajectories") @@ -808,8 +869,9 @@ def _validate_database( _require_uuid4(conversation_id, "Antigravity conversation ID") if expected_session_id is not None and conversation_id != expected_session_id: raise SessionMigrateError("Antigravity database conversation ID does not match target") - if trajectory_type != 4 or source != 17: - raise SessionMigrateError("Antigravity trajectory is not a pinned CLI cascade") + if trajectory_type != 4 or source != expected_source: + desc = "Desktop" if is_desktop else "CLI" + raise SessionMigrateError(f"Antigravity trajectory is not a pinned {desc} cascade") raw_metadata = db.execute( "SELECT id,data FROM trajectory_metadata_blob ORDER BY id" @@ -821,7 +883,11 @@ def _validate_database( ): raise SessionMigrateError("Antigravity trajectory metadata is missing or ambiguous") started_at, trajectory_metadata_loss = _validate_trajectory_metadata( - raw_metadata[0][1], conversation_id, generated=generated + raw_metadata[0][1], + conversation_id, + generated=generated, + expected_project_id=expected_project_id, + is_desktop=is_desktop, ) count = int(db.execute("SELECT count(*) FROM steps").fetchone()[0]) @@ -912,17 +978,29 @@ def _validate_conversation_schema(db: sqlite3.Connection) -> None: def _validate_trajectory_metadata( - data: bytes, conversation_id: str, *, generated: bool + data: bytes, + conversation_id: str, + *, + generated: bool, + expected_project_id: str = PROJECT_ID, + is_desktop: bool = False, ) -> tuple[str | None, bool]: fields = _decode_message(data) root = _optional_text(fields, 6) - if root is not None and root != conversation_id: - raise SessionMigrateError("Antigravity trajectory metadata root ID is inconsistent") + if root is not None: + try: + _require_uuid4(root, "Antigravity trajectory metadata root ID") + except SessionMigrateError: + raise SessionMigrateError( + "Antigravity trajectory metadata root ID is inconsistent" + ) from None + if generated and root != conversation_id: + raise SessionMigrateError("Antigravity trajectory metadata root ID is inconsistent") incomplete = root is None if generated and incomplete: raise SessionMigrateError("generated Antigravity trajectory metadata is incomplete") project = _optional_text(fields, 18) - if project is not None and project != PROJECT_ID: + if project is not None and project != expected_project_id: if generated: raise SessionMigrateError("Antigravity trajectory metadata project is unsupported") incomplete = True @@ -951,6 +1029,9 @@ def _validate_step(data: bytes, *, step_type: int, status: int, generated: bool) raise SessionMigrateError("Antigravity step payload status disagrees with its SQLite row") if status not in {1, 2, 3, 4, 5, 6, 7, 8, 9, 11, 12}: raise SessionMigrateError("Antigravity step status is outside pinned 1.1.16") + if status == 5: + # CLEARED status (compacted/cleared step): payload is empty or metadata only + return expected_payload_field = _STEP_PAYLOAD_FIELDS.get(step_type) if generated and step_type not in { STEP_TYPE_USER_INPUT, @@ -968,7 +1049,7 @@ def _validate_step(data: bytes, *, step_type: int, status: int, generated: bool) def _validate_known_step_payload(step_type: int, data: bytes, *, generated: bool) -> None: fields = _decode_message(data) if step_type == STEP_TYPE_USER_INPUT: - if _optional_text(fields, 2) is None and _optional_text(fields, 1) is None: + if generated and _optional_text(fields, 2) is None and _optional_text(fields, 1) is None: raise SessionMigrateError("Antigravity user step contains no readable text") elif step_type == STEP_TYPE_PLANNER_RESPONSE: response = _optional_text(fields, 1) @@ -981,14 +1062,19 @@ def _validate_known_step_payload(step_type: int, data: bytes, *, generated: bool for entry in _bytes_values(fields, 1): entry_fields = _decode_message(entry) _required_text(entry_fields, 1, "generic argument key") - _required_text(entry_fields, 2, "generic argument value") - result = _required_bytes(fields, 2, "generic tool result") - result_fields = _decode_message(result) - _required_text(result_fields, 1, "generic result text", allow_empty=True) - for entry in _bytes_values(result_fields, 2): - metadata_fields = _decode_message(entry) - _required_text(metadata_fields, 1, "generic result metadata key") - _required_text(metadata_fields, 2, "generic result metadata value", allow_empty=True) + _required_text(entry_fields, 2, "generic argument value", allow_empty=True) + result = _optional_bytes(fields, 2) + if result is not None: + result_fields = _decode_message(result) + _required_text(result_fields, 1, "generic result text", allow_empty=True) + for entry in _bytes_values(result_fields, 2): + metadata_fields = _decode_message(entry) + _required_text(metadata_fields, 1, "generic result metadata key") + _required_text( + metadata_fields, 2, "generic result metadata value", allow_empty=True + ) + elif generated: + raise SessionMigrateError("Antigravity generic tool result is missing") elif step_type == STEP_TYPE_MCP_TOOL: call = _optional_bytes(fields, 2) if call is not None: @@ -1002,6 +1088,10 @@ def _project_events(rows: Sequence[_StepRow]) -> tuple[Event, ...]: pending: deque[tuple[str, str]] = deque() emitted_calls: Counter[str] = Counter() for row in rows: + if row.status == 5: + timestamp = _step_timestamp(row.metadata) + events.append(_opaque_event(row, "antigravity_cleared_step", timestamp)) + continue outer = _decode_message(row.payload) payload_number = _STEP_PAYLOAD_FIELDS.get(row.step_type) payload = _optional_bytes(outer, payload_number) if payload_number else None @@ -1020,6 +1110,8 @@ def _project_events(rows: Sequence[_StepRow]) -> tuple[Event, ...]: provenance=provenance, ) ) + else: + events.append(_opaque_event(row, "antigravity_user_action", timestamp)) if _bytes_values(fields, 3) or _bytes_values(fields, 9): events.append(_opaque_event(row, "antigravity_user_context", timestamp)) continue @@ -1073,9 +1165,17 @@ def _project_events(rows: Sequence[_StepRow]) -> tuple[Event, ...]: continue if row.step_type == STEP_TYPE_GENERIC and payload is not None: fields = _decode_message(payload) - result_fields = _decode_message(_required_bytes(fields, 2, "generic result")) - result_text = _required_text(result_fields, 1, "generic result text", allow_empty=True) - linked_call_id = _generic_result_call_id(result_fields) + result_bytes = _optional_bytes(fields, 2) + if result_bytes is not None: + result_fields = _decode_message(result_bytes) + result_text = _required_text( + result_fields, 1, "generic result text", allow_empty=True + ) + linked_call_id = _generic_result_call_id(result_fields) + else: + result_fields = () + result_text = None + linked_call_id = None pending_index = next( ( index @@ -1532,6 +1632,10 @@ def _summary_values( timestamp: str, cwd: Path, last_user_input_index: int, + status: str = "", + source: str = "antigravity-cli", + project_id: str = PROJECT_ID, + app_data_dir: str = "antigravity-cli", ) -> tuple[Any, ...]: bounded_title = title[:4_096] bounded_preview = preview[:16_384] @@ -1543,9 +1647,9 @@ def _summary_values( step_count, _parse_timestamp(timestamp).isoformat(sep=" "), json.dumps([workspace_uri], ensure_ascii=False, separators=(",", ":")), - "", - "antigravity-cli", - PROJECT_ID, + status, + source, + project_id, "", "", 0, @@ -1555,7 +1659,7 @@ def _summary_values( 0, _parse_timestamp(timestamp).isoformat(sep=" "), last_user_input_index, - "antigravity-cli", + app_data_dir, ) @@ -1588,7 +1692,15 @@ def _read_summary_metadata(path: Path, session_id: str) -> tuple[str | None, Pat if isinstance(uris, list) and uris and isinstance(uris[0], str): parsed = urlparse(uris[0]) if parsed.scheme == "file" and not parsed.netloc: - cwd = Path(unquote(parsed.path)) + raw_path = unquote(parsed.path) + if ( + os.name == "nt" + and len(raw_path) >= 3 + and raw_path[0] == "/" + and raw_path[2] == ":" + ): + raw_path = raw_path[1:] + cwd = Path(raw_path) except (TypeError, ValueError, json.JSONDecodeError): pass return title, cwd @@ -1625,43 +1737,50 @@ def _ensure_summary_database(path: Path) -> None: prefix=".session-migrate-summary-", dir=path.parent ) as directory: staged = Path(directory) / "conversation_summaries.db" + db = None try: - with sqlite3.connect(staged) as db: - db.executescript( - """ - CREATE TABLE conversation_summaries ( - conversation_id text, - title text NOT NULL DEFAULT "", - preview text NOT NULL DEFAULT "", - step_count integer NOT NULL DEFAULT 0, - last_modified_time datetime NOT NULL, - workspace_uris text NOT NULL, - status text NOT NULL DEFAULT "", - source text NOT NULL DEFAULT "", - project_id text NOT NULL DEFAULT "", - agent_name text NOT NULL DEFAULT "", - parent_conversation_id text NOT NULL DEFAULT "", - nesting_depth integer NOT NULL DEFAULT 0, - battle_id text NOT NULL DEFAULT "", - winning_conversation_id text NOT NULL DEFAULT "", - not_fully_idle numeric NOT NULL DEFAULT false, - killed numeric NOT NULL DEFAULT false, - last_user_input_time datetime NOT NULL, - last_user_input_step_index integer NOT NULL DEFAULT -1, - app_data_dir text NOT NULL DEFAULT "", - PRIMARY KEY (conversation_id) - ); - CREATE INDEX idx_conversation_summaries_last_user_input_time - ON conversation_summaries(last_user_input_time); - CREATE INDEX idx_conversation_summaries_last_modified_time - ON conversation_summaries(last_modified_time); - """ - ) - if db.execute("PRAGMA integrity_check").fetchone() != ("ok",): - raise SessionMigrateError("generated Antigravity summary DB failed integrity") + db = sqlite3.connect(staged) + db.executescript( + """ + CREATE TABLE conversation_summaries ( + conversation_id text, + title text NOT NULL DEFAULT "", + preview text NOT NULL DEFAULT "", + step_count integer NOT NULL DEFAULT 0, + last_modified_time datetime NOT NULL, + workspace_uris text NOT NULL, + status text NOT NULL DEFAULT "", + source text NOT NULL DEFAULT "", + project_id text NOT NULL DEFAULT "", + agent_name text NOT NULL DEFAULT "", + parent_conversation_id text NOT NULL DEFAULT "", + nesting_depth integer NOT NULL DEFAULT 0, + battle_id text NOT NULL DEFAULT "", + winning_conversation_id text NOT NULL DEFAULT "", + not_fully_idle numeric NOT NULL DEFAULT false, + killed numeric NOT NULL DEFAULT false, + last_user_input_time datetime NOT NULL, + last_user_input_step_index integer NOT NULL DEFAULT -1, + app_data_dir text NOT NULL DEFAULT "", + PRIMARY KEY (conversation_id) + ); + CREATE INDEX idx_conversation_summaries_last_user_input_time + ON conversation_summaries(last_user_input_time); + CREATE INDEX idx_conversation_summaries_last_modified_time + ON conversation_summaries(last_modified_time); + """ + ) + if db.execute("PRAGMA integrity_check").fetchone() != ("ok",): + raise SessionMigrateError("generated Antigravity summary DB failed integrity") + db.commit() + db.close() + db = None data = staged.read_bytes() except sqlite3.Error as exc: raise SessionMigrateError("cannot create Antigravity summary database") from exc + finally: + if db is not None: + db.close() try: write_private_atomic(path, data) except Exception: @@ -1685,7 +1804,7 @@ def _validate_summary_schema(db: sqlite3.Connection) -> None: if object_row != ("table",): raise SessionMigrateError("Antigravity summary table is missing") columns = tuple(row[1] for row in db.execute("PRAGMA table_info('conversation_summaries')")) - if columns != _SUMMARY_COLUMNS: + if columns not in (_SUMMARY_COLUMNS, _SUMMARY_COLUMNS + ("raw_summary", "group_id")): raise SessionMigrateError("Antigravity summary columns do not match pinned 1.1.16") for index_name, expected_column in ( ("idx_conversation_summaries_last_user_input_time", "last_user_input_time"), diff --git a/src/session_migrate/formats/antigravity_desktop.py b/src/session_migrate/formats/antigravity_desktop.py new file mode 100644 index 0000000..17d9095 --- /dev/null +++ b/src/session_migrate/formats/antigravity_desktop.py @@ -0,0 +1,616 @@ +"""Antigravity Desktop 2.18.1 SQLite/protobuf session adapter. + +This module adapts conversations to and from the Antigravity Desktop application +(macOS/Electron), which stores individual cascade conversations as SQLite databases +under ``~/.gemini/antigravity/conversations/.db`` and registers metadata in +``~/.gemini/antigravity/conversation_summaries.db``. + +While the table schemas match the Antigravity CLI format, Desktop differs in: + - ``trajectory_meta.source``: 1 (IDE / Hub) rather than 17 (CLI). + - ``trajectory_metadata_blob.project_id``: ``"outside-of-project"`` (or project UUID) + rather than ``"default-cli-project"``. + - ``conversation_summaries``: + - ``source``: ``""`` + - ``app_data_dir``: ``"antigravity"`` + - ``project_id``: ``"outside-of-project"`` + - ``status``: ``"CASCADE_RUN_STATUS_IDLE"`` + - Subagents: field 6 of ``trajectory_metadata_blob`` points to the root cascade ID, + so subagent conversations have ``root != conversation_id`` and ``nesting_depth >= 1``. +""" + +from __future__ import annotations + +import os +import shutil +import sqlite3 +import stat +import sys +from collections import Counter +from collections.abc import Mapping +from dataclasses import replace +from pathlib import Path +from typing import Any +from urllib.parse import quote + +from session_migrate.errors import SessionMigrateError +from session_migrate.formats import antigravity +from session_migrate.formats.antigravity import ( + _SUMMARY_COLUMNS, + STEP_STATUS_DONE, + STEP_STATUS_ERROR, + STEP_TYPE_GENERIC, + STEP_TYPE_PLANNER_RESPONSE, + STEP_TYPE_USER_INPUT, + InstalledAntigravitySession, + ParsedAntigravitySession, + _absolute_no_follow, + _database_from_bytes, + _ensure_private_directory, + _ensure_summary_database, + _event_timestamp, + _field_bytes, + _field_text, + _field_varint, + _generic_argument_entries, + _guard_matches_path, + _last_user_input_index, + _omission_key, + _open_identity_guard, + _read_summary_metadata, + _require_uuid4, + _session_id_from_path, + _StepRow, + _stream_sha256, + _summary_values, + _unlink_if_same_file, + _utc_now, + _validate_summary_schema, + content_text, + valid_rfc3339, +) +from session_migrate.jsonl import write_private_atomic +from session_migrate.model import AgentFormat, EventKind, Role, Session + +PINNED_ANTIGRAVITY_DESKTOP_VERSION = "2.18.1" + +# macOS arm64 +PINNED_ANTIGRAVITY_DESKTOP_MACOS_ARM64_SHA256 = ( + "300ee20f3108a511be1149602b91ba84cbbdcc42213067151e255584129be30f" +) +PINNED_ANTIGRAVITY_DESKTOP_MACOS_ARM64_SIZE = 148_990_608 + +# Linux x64 +PINNED_ANTIGRAVITY_DESKTOP_LINUX_X64_SHA256 = ( + "ab4937445fa3817bc374a1db71062de7daecdb093cbfb6bd36b80f443198670e" +) +PINNED_ANTIGRAVITY_DESKTOP_LINUX_X64_SIZE = 181_432_528 + +# Windows x64 +PINNED_ANTIGRAVITY_DESKTOP_WINDOWS_X64_SHA256 = ( + "1ed84e6a1d1e51064d80c9f382ab3a519eb2775cb91d552c63064a18cfdf3cf2" +) +PINNED_ANTIGRAVITY_DESKTOP_WINDOWS_X64_SIZE = 163_640_320 + +PINNED_DESKTOP_SPECS: dict[str, tuple[int, str]] = { + "darwin": ( + PINNED_ANTIGRAVITY_DESKTOP_MACOS_ARM64_SIZE, + PINNED_ANTIGRAVITY_DESKTOP_MACOS_ARM64_SHA256, + ), + "linux": ( + PINNED_ANTIGRAVITY_DESKTOP_LINUX_X64_SIZE, + PINNED_ANTIGRAVITY_DESKTOP_LINUX_X64_SHA256, + ), + "win32": ( + PINNED_ANTIGRAVITY_DESKTOP_WINDOWS_X64_SIZE, + PINNED_ANTIGRAVITY_DESKTOP_WINDOWS_X64_SHA256, + ), +} + +DEFAULT_MACOS_LANGUAGE_SERVER_PATH = Path( + "/Applications/Antigravity.app/Contents/Resources/bin/language_server" +) +DEFAULT_LINUX_LANGUAGE_SERVER_PATH = Path("/opt/Antigravity/resources/bin/language_server") +DEFAULT_WINDOWS_LANGUAGE_SERVER_PATH = Path( + "Programs/antigravity/resources/bin/language_server.exe" +) + + +def default_language_server_path( + platform: str | None = None, + environ: Mapping[str, str] | None = None, + home: Path | None = None, +) -> Path: + """Resolve the default language server binary path for the target platform. + + Can be overridden via the ``SESSION_MIGRATE_ANTIGRAVITY_DESKTOP_BIN`` + environment variable. + """ + env = os.environ if environ is None else environ + override = env.get("SESSION_MIGRATE_ANTIGRAVITY_DESKTOP_BIN") + if override: + return Path(override).expanduser() + + current_platform = sys.platform if platform is None else platform + if current_platform == "darwin": + return DEFAULT_MACOS_LANGUAGE_SERVER_PATH + if current_platform == "win32": + local_app_data = env.get("LOCALAPPDATA") + if local_app_data: + for sub in ( + "Programs/antigravity", + "Programs/Antigravity", + "programs/antigravity", + "programs/Antigravity", + ): + candidate = Path(local_app_data) / sub / "resources" / "bin" / "language_server.exe" + if candidate.is_file(): + return candidate + prog_files = env.get("PROGRAMFILES") + if prog_files: + for sub in ("antigravity", "Antigravity"): + candidate = Path(prog_files) / sub / "resources" / "bin" / "language_server.exe" + if candidate.is_file(): + return candidate + if local_app_data: + return Path(local_app_data) / DEFAULT_WINDOWS_LANGUAGE_SERVER_PATH + return Path("C:\\Program Files\\antigravity\\resources\\bin\\language_server.exe") + + # Linux / other Unix: check PATH launcher, then common installation directories + launcher = shutil.which("antigravity", path=env.get("PATH")) + if launcher: + cand = Path(launcher).resolve().parent / "resources" / "bin" / "language_server" + if cand.is_file(): + return cand + user_home = (home or Path.home()).expanduser() + for cand in ( + Path("/opt/Antigravity/resources/bin/language_server"), + Path("/opt/Antigravity-x64/resources/bin/language_server"), + Path("/opt/antigravity/resources/bin/language_server"), + user_home / "Antigravity-x64/resources/bin/language_server", + user_home / ".local/share/Antigravity/resources/bin/language_server", + user_home / ".local/share/antigravity/resources/bin/language_server", + Path("/usr/lib/antigravity/resources/bin/language_server"), + ): + if cand.is_file(): + return cand + return DEFAULT_LINUX_LANGUAGE_SERVER_PATH + + +def electron_user_data_dir( + platform: str | None = None, + environ: Mapping[str, str] | None = None, + home: Path | None = None, +) -> Path: + """Return the Electron userData directory where app_storage.json resides.""" + env = os.environ if environ is None else environ + current_platform = sys.platform if platform is None else platform + user_home = (home or Path.home()).expanduser() + + if current_platform == "darwin": + return user_home / "Library" / "Application Support" / "Antigravity" + if current_platform == "win32": + app_data = env.get("APPDATA") + if app_data: + return Path(app_data) / "Antigravity" + return user_home / "AppData" / "Roaming" / "Antigravity" + + # Linux / XDG standard + xdg_config = env.get("XDG_CONFIG_HOME") + if xdg_config: + return Path(xdg_config) / "Antigravity" + return user_home / ".config" / "Antigravity" + + +MAX_NATIVE_BYTES = antigravity.MAX_NATIVE_BYTES +PROJECT_ID = antigravity.PROJECT_ID_DESKTOP # "outside-of-project" +SOURCE_DESKTOP = antigravity.SOURCE_DESKTOP # 1 + +session_relative_path = antigravity.session_relative_path +snapshot_database_bytes = antigravity.snapshot_database_bytes + + +def app_data_home(home: Path | None = None) -> Path: + """Return the Antigravity Desktop store below a chosen process HOME.""" + return (home or Path.home()).expanduser() / ".gemini" / "antigravity" + + +def serialize( + session: Session, + *, + session_id: str, + cwd: Path, + cli_version: str = PINNED_ANTIGRAVITY_DESKTOP_VERSION, + model: str | None = None, + timestamp: str | None = None, + trajectory_id: str | None = None, +) -> tuple[bytes, dict[str, int]]: + """Serialize portable conversation history as an Antigravity Desktop SQLite DB.""" + del cli_version, model + _require_uuid4(session_id, "Antigravity conversation ID") + import uuid + + target_trajectory_id = trajectory_id or str(uuid.uuid4()) + _require_uuid4(target_trajectory_id, "Antigravity trajectory ID") + if target_trajectory_id == session_id: + raise SessionMigrateError("Antigravity conversation and trajectory IDs must differ") + + started_at = valid_rfc3339(timestamp) or valid_rfc3339(session.started_at) or _utc_now() + dropped: Counter[str] = Counter() + rows: list[_StepRow] = [] + pending_calls: list[tuple[str | None, str, str, Any]] = [] + generated_call_number = 0 + + def append_step(step_type: int, status: int, native_payload: bytes) -> None: + outer = _field_varint(1, step_type) + _field_varint(4, status) + outer += _field_bytes(antigravity._STEP_PAYLOAD_FIELDS[step_type], native_payload) + rows.append(_StepRow(len(rows), step_type, status, None, outer)) + + for event in session.events: + _event_timestamp(event, started_at, dropped) + if event.kind == EventKind.MESSAGE and event.role == Role.USER: + if event.text: + append_step(STEP_TYPE_USER_INPUT, STEP_STATUS_DONE, _field_text(2, event.text)) + continue + if event.kind == EventKind.MESSAGE and event.role == Role.ASSISTANT: + if event.text: + planner = _field_text(1, event.text) + _field_text(6, str(uuid.uuid4())) + append_step(STEP_TYPE_PLANNER_RESPONSE, STEP_STATUS_DONE, planner) + continue + if event.kind == EventKind.TOOL_CALL and event.role == Role.ASSISTANT: + call_id = event.tool_call_id or f"session-migrate-{session_id}-{len(rows)}" + name = event.tool_name or "unknown_tool" + raw_input = event.payload.get("input", {}) + import json + + arguments_json = json.dumps( + raw_input if isinstance(raw_input, dict) else {}, + ensure_ascii=False, + separators=(",", ":"), + ) + native_call = ( + _field_text(1, call_id) + _field_text(2, name) + _field_text(3, arguments_json) + ) + planner = _field_text(6, str(uuid.uuid4())) + _field_bytes(7, native_call) + pending_calls.append((event.tool_call_id, call_id, name, raw_input)) + append_step(STEP_TYPE_PLANNER_RESPONSE, STEP_STATUS_DONE, planner) + continue + if event.kind == EventKind.TOOL_RESULT and event.role == Role.TOOL: + match_index: int | None = None + if event.tool_call_id: + matching_indices = [ + index + for index, pending in enumerate(pending_calls) + if pending[0] == event.tool_call_id + ] + if event.tool_name: + named_indices = [ + index + for index in matching_indices + if pending_calls[index][2] == event.tool_name + ] + if named_indices: + matching_indices = named_indices + match_index = next(iter(matching_indices), None) + elif pending_calls: + match_index = 0 + dropped["tool_result:missing_id"] += 1 + if match_index is not None: + _, call_id, tool_name, tool_input = pending_calls.pop(match_index) + else: + generated_call_number += 1 + call_id = event.tool_call_id or ( + f"session-migrate-{session_id}-{generated_call_number}" + ) + tool_name = event.tool_name or "unknown_tool" + tool_input = {} + import json + + native_call = ( + _field_text(1, call_id) + _field_text(2, tool_name) + _field_text(3, "{}") + ) + append_step( + STEP_TYPE_PLANNER_RESPONSE, + STEP_STATUS_DONE, + _field_text(6, str(uuid.uuid4())) + _field_bytes(7, native_call), + ) + dropped["tool_result:orphan_id"] += 1 + generic = b"".join(_generic_argument_entries(tool_input, dropped)) + result_text = event.text or content_text(event.payload.get("content_blocks")) + linkage_entry = _field_text(1, "session_migrate_call_id") + _field_text(2, call_id) + generic_result = _field_text(1, result_text or "") + _field_bytes(2, linkage_entry) + generic += _field_bytes(2, generic_result) + append_step( + STEP_TYPE_GENERIC, + STEP_STATUS_ERROR if event.payload.get("is_error") is True else STEP_STATUS_DONE, + generic, + ) + blocks = event.payload.get("content_blocks") + if isinstance(blocks, list): + for block in blocks: + if not isinstance(block, dict) or block.get("type") != "text": + dropped["tool_result:non_text_block"] += 1 + continue + if event.kind == EventKind.THINKING: + dropped["thinking:private"] += 1 + elif event.kind == EventKind.COMPACTION: + dropped["compaction:no_stored_native_equivalent"] += 1 + elif event.kind == EventKind.MESSAGE: + dropped["message:privileged_role"] += 1 + else: + dropped[_omission_key(event)] += 1 + + if not any(row.step_type == STEP_TYPE_USER_INPUT for row in rows): + raise SessionMigrateError( + "Antigravity Desktop target requires at least one portable user message" + ) + + data = antigravity._build_database( + rows, + conversation_id=session_id, + trajectory_id=target_trajectory_id, + started_at=started_at, + source=SOURCE_DESKTOP, + project_id=PROJECT_ID, + ) + validate_native_bytes(data, session_id) + return data, dict(sorted(dropped.items())) + + +def parse(path: Path) -> ParsedAntigravitySession: + """Read one Desktop conversation DB through a consistent SQLite backup snapshot.""" + snapshot = snapshot_database_bytes(path) + parsed = antigravity._parse_database_bytes( + snapshot, + expected_session_id=_session_id_from_path(path), + expected_source=SOURCE_DESKTOP, + expected_project_id=PROJECT_ID, + is_desktop=True, + cli_version=PINNED_ANTIGRAVITY_DESKTOP_VERSION, + ) + title, cwd = _read_summary_metadata(path, parsed.session_id) + return replace(parsed, title=title, cwd=cwd) + + +def parse_session(path: Path) -> Session: + """Parse Antigravity Desktop as a first-class source.""" + parsed = parse(path) + return Session( + source_format=AgentFormat.ANTIGRAVITY_DESKTOP, + source_path=path.resolve(), + source_sha256=parsed.snapshot_sha256, + session_id=parsed.session_id, + cwd=parsed.cwd, + started_at=parsed.started_at, + cli_version=parsed.cli_version, + model=parsed.model, + title=parsed.title, + events=parsed.events, + raw_record_count=parsed.raw_record_count, + model_provider="google", + ) + + +def validate_native_bytes(data: bytes, session_id: str) -> None: + """Strictly validate a generated Antigravity Desktop conversation database.""" + _require_uuid4(session_id, "Antigravity conversation ID") + parsed = antigravity._parse_database_bytes( + data, + expected_session_id=session_id, + generated=True, + expected_source=SOURCE_DESKTOP, + expected_project_id=PROJECT_ID, + is_desktop=True, + cli_version=PINNED_ANTIGRAVITY_DESKTOP_VERSION, + ) + if not any( + event.kind == EventKind.MESSAGE and event.role == Role.USER for event in parsed.events + ): + raise SessionMigrateError("generated Antigravity Desktop session contains no user message") + + +def native_record_count(data: bytes) -> int: + """Return the number of stored steps after full byte validation.""" + with _database_from_bytes(data) as db: + antigravity._validate_database( + db, + expected_session_id=None, + generated=True, + expected_source=SOURCE_DESKTOP, + expected_project_id=PROJECT_ID, + is_desktop=True, + ) + return int(db.execute("SELECT count(*) FROM steps").fetchone()[0]) + + +def verify_pinned_desktop( + executable: Path | None = None, + *, + platform: str | None = None, + environ: Mapping[str, str] | None = None, +) -> Path: + """Resolve and verify the Antigravity Desktop language server binary.""" + values = dict(os.environ if environ is None else environ) + current_platform = sys.platform if platform is None else platform + candidate: str | None = None + if executable: + candidate = str(executable) + elif values.get("SESSION_MIGRATE_ANTIGRAVITY_DESKTOP_BIN"): + candidate = values["SESSION_MIGRATE_ANTIGRAVITY_DESKTOP_BIN"] + elif values.get("ANTIGRAVITY_DESKTOP_BIN"): + candidate = values["ANTIGRAVITY_DESKTOP_BIN"] + elif values.get("ANTIGRAVITY_LANGUAGE_SERVER_BIN"): + candidate = values["ANTIGRAVITY_LANGUAGE_SERVER_BIN"] + elif default_language_server_path(platform=current_platform, environ=values).is_file(): + candidate = str(default_language_server_path(platform=current_platform, environ=values)) + else: + candidate = shutil.which("language_server", path=values.get("PATH")) + + if not candidate: + raise SessionMigrateError("Antigravity Desktop executable 'language_server' was not found") + path = Path(candidate).expanduser().resolve() + try: + info = path.stat() + except OSError as exc: + raise SessionMigrateError("cannot inspect the Antigravity Desktop executable") from exc + if not stat.S_ISREG(info.st_mode): + raise SessionMigrateError("Antigravity Desktop executable is not a regular file") + + if values.get("SESSION_MIGRATE_UNVALIDATED_DESKTOP_BIN") == "1": + return path + + if current_platform not in PINNED_DESKTOP_SPECS: + raise SessionMigrateError( + f"Antigravity Desktop native binary verification is not supported on " + f"{current_platform}; supported platforms are darwin, linux, win32. " + "Set SESSION_MIGRATE_UNVALIDATED_DESKTOP_BIN=1 to proceed with an unvalidated binary." + ) + + expected_size, expected_digest = PINNED_DESKTOP_SPECS[current_platform] + if info.st_size != expected_size: + raise SessionMigrateError( + f"Antigravity Desktop binary size mismatch on {current_platform}: expected " + f"{expected_size} bytes, observed {info.st_size}" + ) + digest = _stream_sha256(path, maximum=expected_size) + if digest != expected_digest: + raise SessionMigrateError( + f"Antigravity Desktop binary digest mismatch on {current_platform}; " + "refusing private-store installation" + ) + return path + + +def install_database( + data: bytes, + *, + session_id: str, + cwd: Path, + timestamp: str, + title: str | None, + target_home: Path, + target_cli: Path | None = None, + dry_run: bool = False, + environ: Mapping[str, str] | None = None, +) -> InstalledAntigravitySession: + """Install a validated conversation and its desktop summary without overwrite.""" + validate_native_bytes(data, session_id) + if target_cli is not None or os.environ.get("SESSION_MIGRATE_VERIFY_DESKTOP_BIN") == "1": + verify_pinned_desktop(target_cli, environ=environ) + root = _absolute_no_follow(target_home) + conversation_path = root / session_relative_path(session_id) + summaries_path = root / "conversation_summaries.db" + cwd = cwd.expanduser().resolve() + summary_timestamp = valid_rfc3339(timestamp) or _utc_now() + parsed = antigravity._parse_database_bytes( + data, + expected_session_id=session_id, + generated=True, + expected_source=SOURCE_DESKTOP, + expected_project_id=PROJECT_ID, + is_desktop=True, + cli_version=PINNED_ANTIGRAVITY_DESKTOP_VERSION, + ) + preview = next( + ( + event.text + for event in parsed.events + if event.kind == EventKind.MESSAGE and event.role == Role.USER and event.text + ), + "Imported conversation", + ) + summary = _summary_values( + session_id=session_id, + title=title or "Imported conversation", + preview=preview, + step_count=parsed.raw_record_count, + timestamp=summary_timestamp, + cwd=cwd, + last_user_input_index=_last_user_input_index(data), + status="CASCADE_RUN_STATUS_IDLE", + source="", + project_id=PROJECT_ID, + app_data_dir="antigravity", + ) + + if dry_run: + if os.path.lexists(conversation_path): + raise SessionMigrateError( + "Antigravity Desktop conversation ID already exists; refusing to overwrite it" + ) + if os.path.lexists(summaries_path): + if stat.S_ISLNK(summaries_path.lstat().st_mode): + raise SessionMigrateError( + "Antigravity Desktop summary database must not be a symlink" + ) + try: + uri = f"file:{quote(str(summaries_path), safe='/')}?mode=ro" + with sqlite3.connect(uri, uri=True, timeout=5) as summaries: + summaries.execute("PRAGMA trusted_schema=OFF") + _validate_summary_schema(summaries) + collision = summaries.execute( + "SELECT 1 FROM conversation_summaries WHERE conversation_id=?", + (session_id,), + ).fetchone() + except sqlite3.Error as exc: + raise SessionMigrateError( + "Antigravity Desktop summary database cannot be checked safely" + ) from exc + if collision: + raise SessionMigrateError( + "Antigravity Desktop conversation ID already exists; refusing to overwrite it" + ) + return InstalledAntigravitySession(conversation_path, summaries_path) + + _ensure_private_directory(root) + _ensure_private_directory(conversation_path.parent) + _ensure_summary_database(summaries_path) + created_identity: tuple[int, int] | None = None + conversation_guard: int | None = None + summary_guard: int | None = None + try: + summary_guard = _open_identity_guard(summaries_path, writable=True) + with sqlite3.connect(summaries_path, timeout=15, isolation_level=None) as summaries: + summaries.execute("PRAGMA trusted_schema=OFF") + _validate_summary_schema(summaries) + summaries.execute("BEGIN IMMEDIATE") + collision = summaries.execute( + "SELECT 1 FROM conversation_summaries WHERE conversation_id=?", (session_id,) + ).fetchone() + if collision or os.path.lexists(conversation_path): + summaries.execute("ROLLBACK") + raise SessionMigrateError( + "Antigravity Desktop conversation ID already exists; refusing to overwrite it" + ) + created_identity = write_private_atomic(conversation_path, data) + conversation_guard = _open_identity_guard( + conversation_path, expected_identity=created_identity + ) + summaries.execute( + f"INSERT INTO conversation_summaries({','.join(_SUMMARY_COLUMNS)}) " + f"VALUES({','.join('?' for _ in _SUMMARY_COLUMNS)})", + summary, + ) + if not _guard_matches_path(conversation_guard, conversation_path) or not ( + _guard_matches_path(summary_guard, summaries_path) + ): + summaries.execute("ROLLBACK") + raise SessionMigrateError( + "Antigravity Desktop install paths changed during transaction" + ) + summaries.execute("COMMIT") + except BaseException: + if conversation_guard is not None: + os.close(conversation_guard) + conversation_guard = None + if summary_guard is not None: + os.close(summary_guard) + summary_guard = None + if created_identity is not None: + _unlink_if_same_file(conversation_path, created_identity) + raise + finally: + if conversation_guard is not None: + os.close(conversation_guard) + if summary_guard is not None: + os.close(summary_guard) + return InstalledAntigravitySession(conversation_path, summaries_path) diff --git a/src/session_migrate/formats/claude.py b/src/session_migrate/formats/claude.py index 5a58e58..22e9d21 100644 --- a/src/session_migrate/formats/claude.py +++ b/src/session_migrate/formats/claude.py @@ -436,7 +436,23 @@ def _valid_preserved_compaction_back_edge( seen: set[str], by_uuid: dict[str, JsonlRecord], ) -> bool: - """Recognize Claude's metadata-declared preserved-segment loop.""" + """Recognize Claude's metadata-declared preserved-segment loop. + + The declaration is stable, the shape under it is not: the number of records + between the boundary and the compact summary varies with the attachment + records the CLI interleaves (an environment snapshot, a file read), so the + summary is sometimes the boundary's direct child and sometimes three or four + parent hops below it. One 73 MB session with two compactions carried both + shapes. ``preservedMessages`` likewise names ids from the summarized branch + that are no longer on the active path, next to the preserved ones. + + Accept the edge when the declaration closes over the walked path: the tail + must lead up to the summary, the summary must lead up to the boundary, and + every hop must already be on the walked path. Requiring the summary to be + the boundary's direct child, or every declared id to be on the path, refuses + readable transcripts — that session raised "ancestry cycle" for exactly this + reason. + """ metadata = boundary.value.get("compactMetadata") if not isinstance(metadata, dict): @@ -454,30 +470,39 @@ def _valid_preserved_compaction_back_edge( assert anchor is not None and head is not None and tail is not None assert boundary_uuid is not None anchor_record = by_uuid.get(anchor) - if ( - anchor_record is None - or anchor_record.value.get("isCompactSummary") is not True - or string(anchor_record.value.get("parentUuid")) != boundary_uuid - ): + if anchor_record is None or anchor_record.value.get("isCompactSummary") is not True: return False declared = messages.get("allUuids", messages.get("uuids")) if not isinstance(declared, list) or not declared: return False declared_ids = {value for value in declared if isinstance(value, str)} - if not {head, tail}.issubset(declared_ids) or not declared_ids.issubset(seen): + if not {head, tail}.issubset(declared_ids): return False - cursor: str | None = tail - preserved_path: set[str] = set() - while cursor and cursor != anchor: - if cursor in preserved_path or cursor not in seen: + return _walks_up_within_seen(tail, anchor, seen, by_uuid) and _walks_up_within_seen( + anchor, boundary_uuid, seen, by_uuid + ) + + +def _walks_up_within_seen( + start: str, + target: str, + seen: set[str], + by_uuid: dict[str, JsonlRecord], +) -> bool: + """Whether ``start`` reaches ``target`` through parents that are already walked.""" + + cursor: str | None = start + visited: set[str] = set() + while cursor and cursor != target: + if cursor in visited or cursor not in seen: return False - preserved_path.add(cursor) + visited.add(cursor) record = by_uuid.get(cursor) if record is None: return False cursor = string(record.value.get("parentUuid")) - return cursor == anchor and head in preserved_path + return cursor == target def _events_from_content(record: JsonlRecord, role: Role, content: Any) -> Iterable[Event]: diff --git a/src/session_migrate/formats/cursor.py b/src/session_migrate/formats/cursor.py index a18c555..ca98cf4 100644 --- a/src/session_migrate/formats/cursor.py +++ b/src/session_migrate/formats/cursor.py @@ -322,14 +322,26 @@ def snapshot_database_bytes(path: Path) -> bytes: with tempfile.TemporaryDirectory(prefix="session-migrate-cursor-snapshot-") as directory: target = Path(directory) / "snapshot.db" uri = f"file:{quote(str(source), safe='/')}?mode=ro" + origin = None + destination = None try: - with sqlite3.connect(uri, uri=True, timeout=5) as origin: - origin.execute("PRAGMA trusted_schema=OFF") - with sqlite3.connect(target) as destination: - origin.backup(destination) + origin = sqlite3.connect(uri, uri=True, timeout=5) + origin.execute("PRAGMA trusted_schema=OFF") + destination = sqlite3.connect(target) + origin.backup(destination) + destination.commit() + origin.close() + origin = None + destination.close() + destination = None data = target.read_bytes() except (OSError, sqlite3.Error) as exc: raise SessionMigrateError("cannot make a consistent Cursor store snapshot") from exc + finally: + if origin is not None: + origin.close() + if destination is not None: + destination.close() if not data or len(data) > MAX_NATIVE_BYTES: raise SessionMigrateError("Cursor store snapshot violates the database size limit") return data @@ -482,28 +494,35 @@ def install_database( def _build_database(metadata: Mapping[str, Any], blobs: Mapping[str, bytes]) -> bytes: with tempfile.TemporaryDirectory(prefix="session-migrate-cursor-build-") as directory: path = Path(directory) / "store.db" + db = None try: - with sqlite3.connect(path) as db: - db.execute("PRAGMA journal_mode=DELETE") - db.execute("PRAGMA synchronous=FULL") - db.execute("PRAGMA user_version=1") - db.executescript( - """ - CREATE TABLE blobs (id TEXT PRIMARY KEY, data BLOB); - CREATE TABLE meta (key TEXT PRIMARY KEY, value TEXT); - """ - ) - db.executemany( - "INSERT INTO blobs(id,data) VALUES(?,?)", - sorted((blob_id, value) for blob_id, value in blobs.items()), - ) - metadata_bytes = json.dumps( - dict(metadata), ensure_ascii=False, separators=(",", ":"), allow_nan=False - ).encode() - db.execute("INSERT INTO meta(key,value) VALUES('0',?)", (metadata_bytes.hex(),)) + db = sqlite3.connect(path) + db.execute("PRAGMA journal_mode=DELETE") + db.execute("PRAGMA synchronous=FULL") + db.execute("PRAGMA user_version=1") + db.executescript( + """ + CREATE TABLE blobs (id TEXT PRIMARY KEY, data BLOB); + CREATE TABLE meta (key TEXT PRIMARY KEY, value TEXT); + """ + ) + db.executemany( + "INSERT INTO blobs(id,data) VALUES(?,?)", + sorted((blob_id, value) for blob_id, value in blobs.items()), + ) + metadata_bytes = json.dumps( + dict(metadata), ensure_ascii=False, separators=(",", ":"), allow_nan=False + ).encode() + db.execute("INSERT INTO meta(key,value) VALUES('0',?)", (metadata_bytes.hex(),)) + db.commit() + db.close() + db = None data = path.read_bytes() except (OSError, sqlite3.Error, TypeError, ValueError) as exc: raise SessionMigrateError("cannot build Cursor store database") from exc + finally: + if db is not None: + db.close() if not data or len(data) > MAX_NATIVE_BYTES: raise SessionMigrateError("generated Cursor store violates the database size limit") return data diff --git a/src/session_migrate/formats/devin.py b/src/session_migrate/formats/devin.py index f53865e..e476a8d 100644 --- a/src/session_migrate/formats/devin.py +++ b/src/session_migrate/formats/devin.py @@ -1475,6 +1475,8 @@ def _check_safe_existing_prefix(path: Path) -> None: def _fsync_directory(path: Path) -> None: + if os.name == "nt": + return descriptor = os.open(path, os.O_RDONLY | getattr(os, "O_DIRECTORY", 0)) try: os.fsync(descriptor) diff --git a/src/session_migrate/formats/hermes.py b/src/session_migrate/formats/hermes.py index e94990f..f4838f7 100644 --- a/src/session_migrate/formats/hermes.py +++ b/src/session_migrate/formats/hermes.py @@ -1,11 +1,12 @@ -"""Hermes Agent 0.20.6 SQLite session adapter. +"""Hermes Agent 0.22.2 SQLite session adapter. Hermes keeps every CLI conversation in ``$HERMES_HOME/state.db`` (normally -``~/.hermes/state.db``). A single database contains many sessions, so source -selection is always the pair ``(state.db, native session id)``. Target bytes -use Hermes's own exported-session shape inside a small versioned envelope; -installation delegates the final transaction to the pinned Hermes -``SessionDB.import_sessions`` implementation. +``~/.hermes``; on Windows ``%LOCALAPPDATA%\\hermes``). A single database +contains many sessions, so source selection is always the pair ``(state.db, +native session id)``. Target bytes use Hermes's own exported-session shape +inside a small versioned envelope; installation delegates the final +transaction to the pinned Hermes ``SessionDB.import_sessions`` implementation, +which is still the native portability boundary in 0.22.2. """ from __future__ import annotations @@ -19,9 +20,10 @@ import sqlite3 import stat import subprocess +import sys import tempfile from collections import Counter -from collections.abc import Iterator, Mapping +from collections.abc import Iterator, Mapping, Sequence from contextlib import contextmanager from dataclasses import dataclass from datetime import UTC, datetime @@ -34,10 +36,18 @@ from session_migrate.jsonl import DEFAULT_MAX_RECORDS, DEFAULT_MAX_TOTAL_BYTES from session_migrate.model import AgentFormat, Event, EventKind, Provenance, Role, Session -PINNED_HERMES_VERSION = "0.20.6" -PINNED_HERMES_RELEASE_TAG = "v2026.8.27" -PINNED_HERMES_SOURCE_COMMIT = "5fc308a70719a83cccdbba4c0e39c23f5a8239d5" -PINNED_HERMES_SCHEMA_VERSION = 26 +PINNED_HERMES_VERSION = "0.22.2" +PINNED_HERMES_RELEASE_TAG = "v2026.9.24" +PINNED_HERMES_SOURCE_COMMIT = "d923a5eb487a17c0a68801bba9ef0f56c81040a8" +PINNED_HERMES_SCHEMA_VERSION = 31 +# ``SessionDB`` caps one session at 5 MiB on import while Hermes' own web import +# accepts 25 MiB (``hermes_cli/web_routers/sessions.py``). A portable session +# carrying a pasted screenshot (inline base64 image) trips the lower bound: the +# bundle is valid and the store accepts stores of that size everywhere else, so +# the installer raises the per-session budget to what the bundle needs — never +# past the vendor's own ceiling — and reports it as a manifest warning. +PINNED_HERMES_IMPORT_SESSION_BYTES = 5 * 1024 * 1024 +PINNED_HERMES_IMPORT_CEILING_BYTES = 25 * 1024 * 1024 HERMES_BUNDLE_SCHEMA = "session-migrate.hermes.v1" HERMES_STATE_FILENAME = "state.db" MAX_DATABASE_BYTES = 512 * 1024 * 1024 @@ -48,6 +58,10 @@ MAX_JSON_NODES = 1_000_000 _NATIVE_ID = re.compile(r"(?P\d{8})_(?P