diff --git a/.claude-plugin/marketplace.json b/.claude-plugin/marketplace.json index 45a5fb210..05cac08ed 100644 --- a/.claude-plugin/marketplace.json +++ b/.claude-plugin/marketplace.json @@ -73,7 +73,7 @@ { "name": "aidd-telemetry", "source": "./plugins/aidd-telemetry", - "description": "Measurement: counts the tokens of Claude Code sessions per task, ticket, model and day, locally. The hooks ask for a task on plain node; its two skills reach the `aidd` CLI to turn it on and answer.", + "description": "Measurement: counts the tokens of Claude Code and Codex sessions per task, ticket, model and day, locally. The hooks ask for a task on plain node; its two skills reach the `aidd` CLI to turn it on and answer.", "strict": true, "metadata": { "recommended": false diff --git a/.gitignore b/.gitignore index dd4852527..a13297832 100644 --- a/.gitignore +++ b/.gitignore @@ -60,3 +60,6 @@ temp/ tmp/ setup-test/ tmp-test/ + +# Session journals the previous telemetry plugin wrote into a project: private, never tracked. +aidd_docs/runs/ diff --git a/README.md b/README.md index ef30eb62d..e5496c38a 100644 --- a/README.md +++ b/README.md @@ -317,7 +317,7 @@ UI / UX design — smoke-test only, not ready for use. `2 skills` · **beta** -Counts the tokens of Claude Code sessions per task, ticket, model and day. Opt-in, and local to your machine. +Counts the tokens of Claude Code and Codex sessions per task, ticket, model and day. Opt-in, and local to your machine. diff --git a/aidd_docs/memory/cli.md b/aidd_docs/memory/cli.md index 426453040..577da0a7d 100644 --- a/aidd_docs/memory/cli.md +++ b/aidd_docs/memory/cli.md @@ -7,7 +7,7 @@ The `aidd` binary, built from `cli/` and published to npm as `@ai-driven-dev/cli ## Commands - Eleven groups: `setup`, `framework`, `translate`, `plugin`, `marketplace`, `auth`, `sync`, `update`, `doctor`, `clean`, `telemetry`. -- `telemetry` counts the tokens of Claude Code sessions locally: [usage contract](../product/usage-contract.md). +- `telemetry` counts the tokens of Claude Code and Codex sessions locally: [usage contract](../product/usage-contract.md). - No list here. The surface moves; `aidd --help` is the only reading that stays true. ## Interface diff --git a/aidd_docs/product/usage-contract.md b/aidd_docs/product/usage-contract.md index a9896c938..2cb0d843e 100644 --- a/aidd_docs/product/usage-contract.md +++ b/aidd_docs/product/usage-contract.md @@ -1,6 +1,6 @@ # Usage contract -> For: maintainers, hook authors, a later destination. What `aidd telemetry` records, stores and reports for Claude Code, release 1. How to use it: [plugin README](../../plugins/aidd-telemetry/README.md). Where the code is: [`cli/aidd_docs/memory/telemetry.md`](../../cli/aidd_docs/memory/telemetry.md). +> For: maintainers, hook authors, a later destination. What `aidd telemetry` records, stores and reports for Claude Code and Codex. How to use it: [plugin README](../../plugins/aidd-telemetry/README.md). Where the code is: [`cli/aidd_docs/memory/telemetry.md`](../../cli/aidd_docs/memory/telemetry.md). The code is the authority. A change to a format below changes this page in the same commit. @@ -9,11 +9,11 @@ The code is the authority. A change to a format below changes this page in the s | CLI, `cli/src/contexts/telemetry/` | reading, storing, attributing, reporting | | Plugin hooks, `plugins/aidd-telemetry/hooks/`, plain Node | asking for a task, answering it, recording session facts | -Only Claude Code is measured. The record admits other tools; no reader for them exists. +Claude Code and Codex are measured, each from the files it writes itself. Reading is the only part written per tool: resolving, consent, storing, attribution and the report are written once for both. ```mermaid flowchart LR - T["Claude Code transcripts"] -->|"ingest"| L[("ledger")] + T["Claude Code transcripts, Codex session files"] -->|"ingest"| L[("ledger")] C[("consent log + git key")] -->|"gates storing"| L H["hooks"] -->|"session facts, carries"| B[("bindings")] H -->|"close lines"| C @@ -49,7 +49,7 @@ The ledger stores `StoredUsage`: the record plus `repository_id`, resolved at in **Null is not zero.** - Unknown is `null`, never `0`. A `null` counter is left out of its sum and counted in that counter's `unknown_records`. - A usage line missing a counter is still a record, that counter `null`. -- A line of a shape the reader does not recognise is never a record: one that is not JSON or not an object, or an `assistant` line whose `message.usage` is not an object or that lacks `message.id`, `sessionId` or `timestamp`. It counts in `run.unrecognised_shapes` of the report that read it, and, when it names a clone that consents, in `coverage.left_out.unrecognised` of every report (see [Left out](#left-out)). A line that is not `assistant`, has no `message.usage` or names the model `` is no call at all, and counts nowhere. +- A line of a shape the reader does not recognise is never a record. For Claude Code: one that is not JSON or not an object, or an `assistant` line whose `message.usage` is not an object or that lacks `message.id`, `sessionId` or `timestamp`. It counts in `run.unrecognised_shapes` of the report that read it, and, when it names a clone that consents, in `coverage.left_out.unrecognised` of every report (see [Left out](#left-out)). A line that is not `assistant`, has no `message.usage` or names the model `` is no call at all, and counts nowhere. For Codex, see [Codex](#codex). - Ingest and report say: "not recognised by this read: a call missing a counter is kept with that counter unknown; a line that is no usage line is never a call." (`UNRECOGNISED_WORDS`, `presentation/display/telemetry-display.ts`). What is kept of such lines is said apart: "Kept from every read: …". **Key and fold** (`domain/formats/claude-transcript-usage.ts`). @@ -64,6 +64,33 @@ The ledger stores `StoredUsage`: the record plus `repository_id`, resolved at in | Advisor | each `advisor_message` iteration is a record, `agent: advisor`, own model | | Sub-agent | `isSidechain` or an `agentId` | +### Codex + +Read from `CODEX_HOME`, else `~/.codex`: every `rollout-*.jsonl` and `rollout-*.jsonl.zst` under `sessions/` and `archived_sessions/`, at any depth (`domain/formats/codex-transcript-usage.ts`, `infrastructure/transcripts/codex-transcript-source-adapter.ts`). Archiving moves a file, and Codex compresses one it no longer writes to `.jsonl.zst` and removes the `.jsonl`: either way its calls are the same keys, so the ledger takes them once. + +Only a `token_usage_record` line is a call. Every other line is context or nothing, and never left out. + +| Record field | From | +| --- | --- | +| `key` | `payload.response_id` | +| `session_id` | `payload.session_id`: the root session, also on a sub-agent's call | +| `agent` | `subagent` when `payload.thread_id` differs from `payload.session_id`, else `main` | +| `at` | the line's `timestamp` | +| `input` | `usage.input_tokens − usage.cached_input_tokens − usage.cache_write_input_tokens`; `null` unless all three are counts | +| `cache_read`, `cache_write` | `usage.cached_input_tokens`, `usage.cache_write_input_tokens` | +| `cache_write_1h` | `null`: Codex does not say | +| `output`, `reasoning` | `usage.output_tokens` (reasoning included), `usage.reasoning_output_tokens` | +| `model`, `cwd` | of the latest `turn_context` before the line; `cwd` else of the file's first `session_meta`; `model` else `null` | +| `tool_version`, `git_branch` | `cli_version` and `git.branch` of the file's first `session_meta`: a sub-agent's file carries its parent's after its own | + +- Only `payload.usage` is read. `turn_token_usage`, `thread_token_usage` and `event_msg` `token_count` are running totals: reading them would count a call many times. +- A missing or malformed counter is `null` and said, as for Claude Code. +- A `token_usage_record` with no `response_id`, `session_id`, `thread_id`, `timestamp` or `usage` object, or a line that is not JSON and holds the text `token_usage_record`, is unrecognised, named with the directory its context gives and its time. +- Lines are cut on byte `0x0a` only: a line holds U+2028 and U+2029 unescaped. +- A call's directory and model are written before it. A `.jsonl` that moved is read from its start: the lines before the saved offset only build its context, the lines after it give calls and unrecognised shapes exactly as a read from that offset would. For a `.jsonl`, `offsets.json` keeps its meaning, the end of the last complete line consumed. +- A `.jsonl.zst` is read whole, once, decompressed as a stream: its offset is its size, and its size and identity keep it read. Its bytes are counted against the content size its frame header declares, since a Node 22 `zlib` ends a cut frame quietly, short. A Node whose `zlib` has no zstd reads none, and one that does not decompress, or decompresses to another size than its frame declares, is not read: each counts in that run's unrecognised shapes, saying why, keeps its position and is tried again by the next run. Nothing else stops for it. +- A `.jsonl` and a `.jsonl.zst` of one session may lie side by side, or one turn into the other: both are read and the ledger takes each call once, `left_out` each thing once. Only the run that reads the second counts its unrecognised shapes and calls not stored again. + **Repository.** The call's working directory resolved at ingest: its git root, named by the hash of the `origin` address (`host/owner/repo`), else the root commit. The address is never kept. Every form (https, `ssh://`, `git://`, scp-like) gives one address: scheme, credentials, port, `.git` and trailing `/` dropped, host case-folded, `ssh.github.com` read as `github.com`. Owner and repo are case-folded on `github.com` only, which documents them as not case sensitive; any other host keeps their case, so two spellings there may still give two ids. **Not stored.** Storing follows [Consent](#consent). A call read and left out counts in `run.not_stored` of the report that read it, under one reason, and nothing of it is kept, except an `undated` call of a clone that consents (see [Left out](#left-out)). A call with no `requestId` is not stored either, and counts under no reason here: only in `coverage.left_out.requestless`. @@ -147,7 +174,7 @@ Directory: `AIDD_TELEMETRY_DIR`, else `AIDD_USER_CONFIG_DIR/telemetry`, else `XD | `ledger/.jsonl` | `ingest`, `report` | `report`, `forget` | `StoredUsage` by UTC day of `at`, in time then key order, one call per line. Ingest reads its calls' days and the day on each side, and rewrites only a changed day, whole, by rename; no new bytes, no write. `report` rewrites, folded, the partitions it read that share a call, or hold a line that is not a record; a report with no period reads every partition, so after it each call is on disk once. A line that is not a record (a newer version's, an older one's call with no `requestId`, a hand edit, a torn one) is never removed: a rewrite keeps it in its day, after the records, in text order and once, and counts it in `skipped_ledger_lines` on every read. A day holding only such lines stays | | `ledger/left-out.txt` | `ingest` | `report`, `forget` | what ingests measured and could not store, by hash, see [Left out](#left-out). cli-v5.5.0 never reads it | | `ledger/repository-ids.json` | `ingest`, `report` | every reader of the ledger, `roots.json` and `branches.jsonl` | `{version: 1, complete, aliases, pending}`: the id an earlier version gave a repository, to its id now, both hashes or root commits. Made from each root `roots.json` names, by its `origin` now. Every read re-ids by this table; the first ingest or report also rewrites `roots.json` and every ledger partition so, then marks it `complete`. A run stopped part way is finished by the next with the same table. A root that could not be read, or was not there, stays in `pending`: every later ingest looks at those roots alone and, when one adds an id, rewrites the stores again. Until then its stored calls keep the earlier id while its new calls get the id of now: two ids for one repository. An earlier version reading migrated data reads it as written; what it stores afterwards is re-ided when its repository is in the table, at every read. `branches.jsonl` is never rewritten, so it relies on this table for good: a damaged table is made again from the clones still there, and a snapshot of a repository none of whose clones remains keeps its earlier id, so the calls of now no longer find its declaration. An `undated` hash in `left-out.txt` keeps its earlier key | -| `ledger/offsets.json` | `ingest`, `report` | same | per transcript: bytes consumed, size, file identity. Shrunk or changed identity: read whole | +| `ledger/offsets.json` | `ingest`, `report` | same | per transcript: bytes consumed (for a `.jsonl.zst`, its compressed size), size, file identity. Shrunk or changed identity: read whole | | `ledger/roots.json` | `ingest`, `on` | `ingest`, `forget` | `{version: 2, directories}`: per directory and per clone seen alive there, `repository_id`, `root`, `clone`, `seen_at`, for every directory a read call or line named, in any clone. Other format holds nothing: a live directory is remembered again, a gone one counts `never-seen-alive`. `on` records its own root for `forget` | | `ledger/consents.jsonl` | `on`, `off`, `ingest`, hooks (close only) | same, plus `forget` | consent intervals, see [Consent](#consent). Outlives the clone; removed with the ledger. Hooks parse it strictly, pinned by the shared fixture | | `ledger/.lock` | CLI | CLI | `{pid, created_at}`; dead process or older than ten minutes: cleared | @@ -327,7 +354,19 @@ None observable by the CLI. | Call read while git config, `.git` or git dir unreadable | `unreadable-consent`. A clone remembered there that consented at the time: the transcript read again by every ingest until its consent can be read, then stored. Otherwise: said only by that run | repair the clone, or `forget --yes`, or remove the clone; not `off` | | pid reused within one boot by a Claude process whose `startup` was never recorded (e.g. started before `on`) | can take the dead process's last session as predecessor, so `/clear` carries its task | announced, replaced by first declaration | | One transcript line longer than the longest string the runtime holds (512 MB) | every ingest fails when it reaches that line (inferred, not run). A transcript is otherwise read a block at a time: an ingest holds 1 MiB, up to about three copies of the longest line, and what the reading keeps (every record, every unrecognised line naming a directory), whatever the transcript's size | - | -| Transcript rewritten in place, to the same size or larger, while an ingest reads it | not seen: a line may be spliced from the two versions and the offset saved inside the new one. Shrunk: that read counts nothing and the transcript is read again from where it stood. Replaced at its path: the read finishes the file it opened, and the next one reads the new file whole. Claude Code only appends | - | +| Transcript rewritten in place, to the same size or larger, while an ingest reads it | not seen: a line may be spliced from the two versions and the offset saved inside the new one. Shrunk: that read counts nothing and the transcript is read again from where it stood. Replaced at its path: the read finishes the file it opened, and the next one reads the new file whole. Claude Code only appends; Codex is assumed to only append to a `.jsonl` | - | +| Task and ticket of a Codex call | come from its branch's declaration only: the plugin's hooks run in Claude Code, so nothing asks a Codex session for its task, and a session declaration never binds one | declare the branch | +| Branch of a Codex call | the branch its session started on: a branch changed during the session is not seen | - | +| A person who only uses Codex | measured when `aidd telemetry ingest` or `report` runs, which the plugin does at a Claude Code session start | run a report | +| Codex cache writes | part of `input_tokens`, which `input` subtracts them from, as OpenAI's own formula does; billed only on GPT-5.6 and later. OpenAI has one write rate, so `cache_write_1h` is `null`. A cached read and write above `input_tokens` was never seen: its `input` would be below zero, which the ledger does not read as a record | - | +| Codex versions | per-call records ship from Codex CLI 0.153. An earlier version writes calls only as running totals (`token_count`), which are not read: a session it wrote counts nothing. Observed: 0.149 pre-releases to 0.163 | - | +| Codex calls written with no record | not counted. Off by default: background remote compaction, memory extraction, the Guardian scorer. Always: title generation, `codex exec --ephemeral` (no session file), realtime and voice conversations, failed, aborted or dropped-stream responses, and, before 0.163, a response cut short by its output limit | - | +| Codex cost fields not in the file | the service tier actually served, and audio or image token detail. The counts are tokens, not money | - | +| A Codex fork that copies its parent's records | the copies carry the parent's `response_id`, so the ledger takes each call once. Not seen | - | +| A Codex thread resumed from a history that holds no per-call records | only the calls written as `token_usage_record` count: what the thread consumed before them is in no record. Seen as a thread whose running total is already large on its first call | - | +| A Codex `.jsonl.zst` that does not decompress | not seen. The calls decoded before the bad frame are stored once, the ledger being keyed, but every run tries the file again and counts them again in its own figures, with one unrecognised shape | remove or repair the file | +| A Codex `.jsonl.zst` whose last line is unterminated | not seen: that line is never read, the file being read once | - | +| A Codex `.jsonl.zst` whose frame declares no content size, or that holds several frames | one compressed before Codex 0.152 may declare no size: it is read as `zlib` decompresses it, unchecked. Several frames were not seen: of several, `zlib` reads the first alone | - | | `AIDD_TELEMETRY=0` | stops hooks too; a key turned off meanwhile is observed at the next run without it | - | | Declaration keys edited by hand: every key including the stamp, `aiddTask` and `aiddDeclaredAt` alone (read as unstamped), or a stamp removed or emptied | the branch reads as declared: no reader can tell who wrote a git config key | - | | Copy or rename of an unstamped (cli-v5.5.0) declaration | the new name inherits it | declare on the new name, or on the original to stamp it before copying or renaming | diff --git a/aidd_docs/tasks/2026_10/2026_10_19_codex-is-measured/contract.md b/aidd_docs/tasks/2026_10/2026_10_19_codex-is-measured/contract.md new file mode 100644 index 000000000..cf50573fc --- /dev/null +++ b/aidd_docs/tasks/2026_10/2026_10_19_codex-is-measured/contract.md @@ -0,0 +1,89 @@ +--- +status: done +--- + +# Codex is measured + +## Need + +Release 1 measures Claude Code only: the usage record admits `codex`, no reader for it exists. A person who works with Codex in a clone that opted in sees none of that work in `aidd telemetry report`. The measured facts behind every rule below are in `evidence.md`. + +## Decision + +```mermaid +flowchart LR + S[("CODEX_HOME sessions and archived_sessions: rollout-*.jsonl, .jsonl.zst")] -->|"from the start, a block at a time"| A["Codex transcript source"] + A -->|"each line, and whether it is past the saved offset"| F["Codex format, one state per file read"] + F -->|"session_meta, turn_context: context only"| F + F -->|"token_usage_record past the offset"| R["UsageRecord, tool codex"] + R --> I["ingest: resolve, store, as for Claude Code"] +``` + +One more reader in the ingest list. Resolving, consent, storing, attribution and the report are unchanged and written once for every tool. + +### Files + +- Root: `CODEX_HOME`, else `~/.codex`, read by the composition root. Under `sessions/` and `archived_sessions/`, every `rollout-*.jsonl` and `rollout-*.jsonl.zst`, at any depth. Archiving moves a file there: its calls already stored are the same keys. +- Codex compresses a session file to `.jsonl.zst` about ten days after it was last written, and removes the `.jsonl`. A compressed file is read whole, decompressed as a stream (never held whole), once: it does not change, so its size and identity keep it read. Its calls already stored under the `.jsonl` are the same keys, so the ledger takes them once. +- A runtime whose `node:zlib` has no zstd reads no compressed file, and each one it lists is counted in that run's unrecognised shapes, naming why. + +### What a line becomes + +Only a `token_usage_record` line is a call. Every other line is context or nothing, and never named as dropped. + +| Record field | From | +| --- | --- | +| `tool` | `codex` | +| `key` | `payload.response_id` | +| `session_id` | `payload.session_id`: the root session, also on a sub-agent's call | +| `agent` | `subagent` when `payload.thread_id` differs from `payload.session_id`, else `main` | +| `at` | the line's `timestamp` | +| `input` | `usage.input_tokens − usage.cached_input_tokens − usage.cache_write_input_tokens` | +| `cache_read` | `usage.cached_input_tokens` | +| `cache_write` | `usage.cache_write_input_tokens` | +| `cache_write_1h` | `null`: Codex does not say | +| `output` | `usage.output_tokens`, reasoning included | +| `reasoning` | `usage.reasoning_output_tokens` | +| `model` | `model` of the latest `turn_context` before the line, else `null` | +| `cwd` | `cwd` of the latest `turn_context` before the line, else of the file's first `session_meta` | +| `tool_version` | `cli_version` of the file's first `session_meta` | +| `git_branch` | `git.branch` of the file's first `session_meta` | + +- Only `payload.usage` is read. `turn_token_usage`, `thread_token_usage` and `event_msg` `token_count` are running totals: reading them would count a call many times. +- The file's **first** `session_meta` is its own. A sub-agent's file carries its parent's after it. +- A missing or malformed counter is `null` and named, as for Claude Code. An `input` whose parts are not all counts is `null`. +- A `token_usage_record` with no `response_id`, `session_id`, `thread_id`, `timestamp` or `usage` object is unrecognised, named with its context's directory and its time, and never a record. So is a line that is not JSON but holds the text `token_usage_record`. + +### Reading from where a read stopped + +A call's directory and model are written earlier in the file than the call. A read that started at the saved offset would see calls with no directory, counted `no-cwd` and never stored. + +- When a Codex file moved (size or identity), it is read from its start. The lines before the saved offset only build the file's context; the lines after it give calls and unrecognised shapes, exactly as a read from that offset would. So a run counts each line once, whatever the number of ingests. +- `offsets.json` keeps its format. For a `.jsonl`, its meaning too: the end of the last complete line consumed. For a `.jsonl.zst`, read whole once, the offset is its compressed size. +- A file that did not move is not opened again. One last written before the first consent interval is skipped, as for Claude Code. + +## Properties + +| # | Property | Tested by | +| --- | --- | --- | +| P1 | Any split of a Codex file's growth across ingests stores the same ledger, byte for byte, as one ingest of the whole file, and the runs' unrecognised and not-stored counts add up to the single run's | fast-check over cut points | +| P2 | Every record's `input + output + cache_read + cache_write` equals the line's `usage.total_tokens` | unit, and over the real corpus in `evidence.md` | +| P3 | The same session read from its `.jsonl`, then from its `.jsonl.zst`, stores each call once | integration | +| P4 | The existing ledger and report properties hold with Codex records in their generators | the existing property suites | + +## Limits + +These are written into the usage contract's residual limits; none is code. + +- Task and ticket of a Codex call come from its branch's declaration only. The plugin's hooks run in Claude Code, so nothing asks a Codex session for its task, and a session declaration never binds one. +- `git_branch` is the branch the session started on: a branch changed during the session is not seen. +- Codex is read by `aidd telemetry ingest` and `report`, which the plugin runs at a Claude Code session start. A person who only uses Codex is measured when they run a report. +- Cache writes are part of `input_tokens`, and `input` subtracts them as OpenAI's own formula does; they are billed only on GPT-5.6 and later. +- Per-call records ship from Codex CLI 0.153; an earlier version writes only running totals, which are not read. Observed: 0.149 pre-releases to 0.163. +- Codex is assumed to only append to a `.jsonl`. A rewrite in place behaves as for Claude Code. + +## Out of scope + +- A tool axis in the report: the model and session axes already tell Codex calls apart. Proposed, not built. +- Codex hooks: asking a Codex session for its task. +- A price: the report counts tokens. diff --git a/aidd_docs/tasks/2026_10/2026_10_19_codex-is-measured/evidence.md b/aidd_docs/tasks/2026_10/2026_10_19_codex-is-measured/evidence.md new file mode 100644 index 000000000..fd05b5f4f --- /dev/null +++ b/aidd_docs/tasks/2026_10/2026_10_19_codex-is-measured/evidence.md @@ -0,0 +1,171 @@ +# Evidence + +## Census of real Codex session files + +Read-only, over one machine's `~/.codex/sessions`, Codex CLI 0.149 to 0.162. Aggregates only: no id, path or content is recorded here. The probe scripts stayed in the scratchpad. + +| Fact | Measured | +| --- | --- | +| Plain session files, size | 476 `.jsonl`, 2,161 MB, the largest 723 MB | +| Longest line | 6.9 MB | +| Lines that are not JSON, split strictly on byte `0x0a` | 0 of 470,948. Node `readline` also splits on U+2028 and U+2029 and turned 224 lines into fragments: the reader splits on `0x0a` only | +| `token_usage_record` lines | 56,317 | +| ... with `response_id`, `session_id`, `thread_id`, `usage` | all | +| ... `response_id` repeated, in one file or across files | 0 | +| ... `total_tokens = input_tokens + output_tokens` | all | +| ... `cached_input_tokens ≤ input_tokens` | all | +| ... `reasoning_output_tokens ≤ output_tokens` | all | +| ... `cache_write_input_tokens > 0` | 0 | +| ... `thread_id ≠ session_id` | 17,904, all in sub-agent files (`session_meta.source.subagent.thread_spawn`); 0 elsewhere | +| ... with a `turn_context` of the same turn before it | 56,304; the other 13 have an earlier turn's | +| ... latest `turn_context` before it agrees with the same turn's, model and cwd | all, 0 differences | +| Files with two `session_meta` | 186, all sub-agent files: the first is the file's own, the second its parent's | +| `session_meta` with `git.branch` | 367 of 473 | +| Running totals (`event_msg` `token_count`) | 60,190 lines: not calls. Their last value differs from the sum of calls in 65 files, 64 of them compacted | +| Full `JSON.parse` of every line of every file | 2.9 s, the largest file 0.8 s, 263 MB resident | +| Compressed files `rollout-*.jsonl.zst` | 1,056, holding 19,147 calls; none has a `.jsonl` twin | +| When a file is compressed | by day folder: none of the last ten days, most of the older ones, a few older `.jsonl` left (sessions still resumed) | +| Archived sessions (`~/.codex/archived_sessions`) | 5 `.zst` files, flat | + +## Red run + +Each test was run red first, failing on the assertion it names, before the code it covers existed or with that code broken on purpose. + +| Guard | Broken how | What went red | +| --- | --- | --- | +| The format, every row of the table (`codex-transcript-usage.unit.test.ts`) | reader stubbed to give nothing | 29 of 32 tests failed on their assertions; the 3 that passed assert "gives nothing" | +| A moved `.jsonl` read from its start | read from the saved offset, as Claude Code's is | adapter "reads a grown file from its start…", use case "gives a call written after a resumed read the directory and model…", P1 | +| Lines before the offset counted once | the use case keeps outcomes of uncounted lines | use case resume test, P1 (counts and ledger) | +| `counted` computed by the shared splitter | `counted = true` | adapter grown-file test, use case resume test | +| A `.zst` read once | re-read whatever its position | adapter "keeps the file read by its size and identity" | +| The no-zstd shape reaches the run | the source's `unrecognised` dropped by the use case | use case "names each compressed file a Node with no zstd cannot read" | + +Review round 1, red first: + +| Guard | Broken how | What went red | +| --- | --- | --- | +| A `.zst` that does not decompress stops nothing (adapter, and an ingest beside a Claude Code transcript) | before the fix | both tests, on `Error: Unknown frame descriptor` | +| Only zlib's own codes are passed over | the code check removed | adapter "fails the read: only a stream that does not decompress is named and passed over" (an `EACCES` open) | +| The file keeps its position, so it is tried again | a made-up position saved | adapter "…when it does not decompress, and is read again next time", and the no-zstd test | +| The sandbox points `CODEX_HOME` inside it | the relocation removed from `sandboxedEnv` | "points Codex's home inside the sandbox, even when the runner sets its own" | + +P3 (plain then compressed) passes under the re-read-every-time break: the ledger takes a key once whatever reads it, which is what P3 states; the re-read itself is killed by the adapter test. + +## Verification + +### Real corpus, read-only + +The CLI's Codex source and format, bundled into a scratchpad script, read `~/.codex/sessions` and `~/.codex/archived_sessions` from byte 0 with no saved position; nothing was written there. The independent probe shares no code with the CLI. Both were run back to back, twice, on a corpus still being written: + +| Measure | Independent probe | CLI reader | +| --- | --- | --- | +| Files | 1,537 | 1,537 (1,061 `.zst`) | +| Distinct calls | 75,584 | 75,584 records after the fold | +| Tokens (`total_tokens` / `input + output + cache_read + cache_write`) | 9,313,648,283 | 9,313,648,283 (P2 over the corpus) | +| Threads whose per-call sum equals Codex's own `thread_token_usage` on the last call | 617 of 618 | 617 of 618, all five counters | +| The one other thread | running total already 370,167,606 on its first call | same thread: a history with no per-call records, a written limit | +| Unrecognised shapes | - | 0 | + +The coordinator's earlier count (75,579 calls, 9,313,057,550 tokens) was taken before the files grew: the CLI read 75,581 and then 75,582 records on the way, and the two agreed on every run made back to back. + +### Gates + +| Gate | Decisive line | +| --- | --- | +| `pnpm exec lefthook run pre-commit` | every job green on each commit; `cli-architecture` once timed out (`mutation-covers-source.arch.test.ts`, 6 s against the 5 s default while `scripts-tests` ran beside it) and passed alone and on the retried commit | +| `pnpm lint` (cli) | `No fixes applied.`, the one warning is in `uninstall-use-case.ts`, untouched | +| `pnpm typecheck` | `tsc --noEmit`, no error | +| `pnpm test:arch` | `Tests 168 passed (168)` | +| `node scripts/check-cli-type-honesty.mjs` | `No type is widened through unknown, any or never` | +| `pnpm knip` | no finding | +| `pnpm jscpd` | exit 0, `1.55%` duplicated, threshold 3.3, no clone in a changed file | +| `pnpm test` | `Test Files 525 passed \| 1 skipped`, `Tests 6834 passed \| 1 skipped`; the one skipped file is none of the telemetry ones | +| `pnpm build` | `Bundle size: 704.7 KB / budget: 709 KB` | +| `pnpm smoke` | `PASS: 132 FAIL: 0 SKIP: 1 · coverage 100%` | +| `check-claude-tokens-end-to-end.cjs` | `2 calls, 393 tokens, all on task demo, ticket T-1` | +| `check-claude-tokens-end-to-end.cjs --attended` | same, `asked, answered by the hook, kept across /clear` | +| `node scripts/check-no-local-paths.js` | `0 leak(s)` | +| `run-mutation.mjs --changed` | score 96.5 over 21 changed ranges, survivors below | +| `run-mutation.mjs telemetry` | score 97.2, floor 96 | + +### Mutants left alive on the changed lines + +The first changed-line run scored 95.0. The five survivors in Codex code were killed by tests added for them: the rollout name pattern's `^`, `$` and `.*`, the source's `oldestModified`, and a `session_meta` after a `turn_context`. What survives is equivalent, or lies in code moved unchanged out of the Claude Code adapter: + +| Where | Mutant | Why it lives | +| --- | --- | --- | +| `transcript-files.ts` `identityOf` | the `ino === 0` branch | Windows only: an inode is never 0 on the platforms the suite runs on | +| `lastLineEnd` | `end > 0` to `>=`; `end - from` to `+` | equivalent: at 0 the block is empty and the loop ends; a larger block only reads bytes already searched or past the end | +| `LineSplitter.push` | the one-piece fast path off; an empty remainder kept | equivalent: the joined bytes decode to the same line | +| `eachLine` | `size - from` to `+`; `at < size` to `<=` | equivalent: each read is capped at what is left, and a read at the end returns 0 bytes and stops | +| `oldestModifiedOf` | `<` to `<=` | equivalent: an equal time replaces itself | +| `readTranscript` | the `finally` that closes the handle emptied | not observable through the port: the handle is an own property, not on a prototype a test can spy on, and Node closes a leaked one on collection | +| `line-values.ts` `counter` | `typeof value === "number"` dropped | equivalent: `Number.isInteger` is false for every non-number | + +### Review round 1 + +The gates above were run again after the fixes, in one pass: + +| Gate | Decisive line | +| --- | --- | +| `pnpm lint`, `pnpm typecheck`, `pnpm test:arch` | `No fixes applied.`; no error; `Tests 168 passed (168)` | +| `check-cli-type-honesty.mjs` (repository root) | `No type is widened through unknown, any or never` | +| `pnpm knip`, `pnpm jscpd` | exit 0; `1.55%` duplicated | +| `pnpm test` | `Tests 6840 passed \| 1 skipped` | +| `pnpm build` | `Bundle size: 704.9 KB / budget: 709 KB` | +| `pnpm smoke` | `PASS: 132 FAIL: 0 SKIP: 1 · coverage 100%` | +| `check-claude-tokens-end-to-end.cjs`, plain and `--attended` | `2 calls, 393 tokens, all on task demo, ticket T-1` in both, the second `asked, answered by the hook, kept across /clear` | +| `check-no-local-paths.js` | `0 leak(s)` | +| `run-mutation.mjs --changed` | score 96.3 over 22 changed ranges | +| `run-mutation.mjs telemetry` | score 96.6, floor 96 | +| The checker's case, against the built binary in a sandbox | 100 random bytes as a `rollout-*.jsonl.zst` in a clone that opted in: `report --json` exits 0 with `unrecognised_shapes` 1, twice in a row; `ingest` exits 0 and names it | + +New survivor on the changed lines, beside the ones justified above: `UNDECOMPRESSABLE` without its `^` anchor. It is equivalent for every code a read meets: an I/O code never holds `Z_` or `ZSTD_`. The `ZSTD_` and `Z_` prefixes are each held by a test (`ZSTD_error_prefix_unknown`, `Z_BUF_ERROR`). + +The telemetry scope fell from 97.2 to 96.6 over the round, still above its floor of 96. + +### Review round 2: a cut frame on Node 22 + +CI runs Node 22, where `createZstdDecompress` ends a cut frame with no error and short output: a truncated `.zst` was read as whole, its position saved, its missing calls lost. Node 24 fails the same stream with `Z_BUF_ERROR`. Every real compressed file is one frame declaring its content size, equal to its decompressed length (measured by the coordinator, read-only), so the reader counts the decompressed bytes against that size. + +Red first, with the frame header reader stubbed to declare nothing: + +| Node | Red | +| --- | --- | +| v24.20.0 | the 7 header-reading unit tests, and "names a stream that ends short of the size its frame declares" (a decompressor that ends one byte short, as Node 22's does) | +| v22.18.0 | the same 8, plus "names a stream cut short and reads it again next time…": the CI failure, reproduced | + +The size check removed on purpose: "ends short" goes red on v24.20.0, "ends short" and "cut short" on v22.18.0. + +Green after the fix, `npx vitest run tests/contexts/telemetry`: + +| Node | Result | +| --- | --- | +| v24.20.0 | `Test Files 78 passed (78)`, `Tests 1388 passed (1388)` | +| v22.18.0 | `Test Files 78 passed (78)`, `Tests 1388 passed (1388)` | + +Real corpus again, read-only, after the fix: the independent probe and the CLI reader give 1,537 files, 75,588 calls, 9,314,170,585 tokens, the same on both Node versions, with 0 unrecognised shapes: no real file fails the size check. + +A file of several frames: `zlib` decodes the first alone on both versions, so the reader cannot see more than the first frame declares. Never seen; a written limit. + +Gates after round 2, in one pass: + +| Gate | Decisive line | +| --- | --- | +| `pnpm lint`, `pnpm typecheck`, `pnpm test:arch` | `No fixes applied.`; no error; `Tests 168 passed (168)` | +| `check-cli-type-honesty.mjs` (repository root) | `No type is widened through unknown, any or never` | +| `pnpm knip`, `pnpm jscpd` | exit 0; `1.55%` duplicated | +| `pnpm test`, Node v24.20.0 | `Test Files 526 passed \| 1 skipped`, `Tests 6851 passed \| 1 skipped` | +| `pnpm test`, Node v22.18.0 | `Test Files 526 passed \| 1 skipped`, `Tests 6851 passed \| 1 skipped` | +| `pnpm build` | `Bundle size: 706.0 KB / budget: 709 KB` | +| `pnpm smoke` | `PASS: 132 FAIL: 0 SKIP: 1 · coverage 100%` | +| `check-claude-tokens-end-to-end.cjs`, plain and `--attended` | `2 calls, 393 tokens, all on task demo, ticket T-1` in both | +| `check-no-local-paths.js` | `0 leak(s)` | +| `run-mutation.mjs --changed` | score 96.8 over 23 changed ranges; the frame reader's survivors were killed by tests added for them, the one left is the anchor-less `UNDECOMPRESSABLE` justified above | +| `run-mutation.mjs telemetry` | score 96.7, floor 96 | + +`Z_BUF_ERROR` is held by a test on Node 24 only: on Node 22 a cut frame reaches the size check instead, so a mutation run on Node 22 leaves the `Z_` prefix of `UNDECOMPRESSABLE` to the code alone. + +### Review round 3: Codex's own source + +A `compacted` line embeds a copy of the latest record as `payload.latest_token_usage_record`. A unit test pins that it gives no record, no unrecognised shape and no `named`; it passed as written, and goes red, with two other "gives nothing" tests, when the top-level type check is dropped. Format tests: `Tests 69 passed (69)` on Node v24.20.0 and on v22.18.0. The usage contract's Codex limits now follow Codex's source: per-call records from 0.153, the calls Codex writes no record for, the cost fields not in the file, cache writes, a frame with no size before 0.152, coexisting copies, forks. diff --git a/aidd_docs/tasks/2026_10/2026_10_19_codex-is-measured/plan.md b/aidd_docs/tasks/2026_10/2026_10_19_codex-is-measured/plan.md new file mode 100644 index 000000000..5496769c1 --- /dev/null +++ b/aidd_docs/tasks/2026_10/2026_10_19_codex-is-measured/plan.md @@ -0,0 +1,28 @@ +--- +status: done +--- + +# Plan: Codex is measured + +Contract: `contract.md`. Each phase starts with its tests, run red first, the red run recorded in `evidence.md`. + +## Phase 1: the format + +- `contexts/telemetry/domain/formats/codex-transcript-usage.ts`: a reader made once per file read, holding that file's context, mapping a line to records and unrecognised shapes as the contract's table says. +- Unit tests over synthetic lines: the mapping, first `session_meta` wins, latest `turn_context` wins, a line before the offset builds context and gives nothing, the unrecognised cases, the running totals ignored, P2. +- Acceptance: every row of the contract's table has a test that fails when its rule is broken. + +## Phase 2: the source and the read from the start + +- The Claude Code source's block reader shared, not copied, with a Codex source that lists both roots, reads `.jsonl` from the start once moved, and reads `.jsonl.zst` through a streamed decompression. +- `ReadUsageUseCase` hands the format whether each line is past the saved offset; the Claude Code format ignores it. +- Integration tests: resume with context (the ingest, append to the turn and to a new turn, ingest again case), compressed file, no-zstd runtime, P3. +- Acceptance: `offsets.json` unchanged in format; the Claude Code adapter's tests unchanged and green. + +## Phase 3: wiring, properties, docs + +- One reader more in `runtime/wiring/telemetry.ts`, root from `CODEX_HOME` or the home directory. +- Every test and script that runs the CLI against a relocated home also relocates `CODEX_HOME`. +- P1 as a fast-check property; P4 by adding Codex records to the existing generators. +- Docs: the usage contract (scope, record source, limits), the plugin README, the 01-usage skill and its envelope reference, `cli/aidd_docs/memory/telemetry.md`, both `cli.md`, both project briefs where they say Claude Code only. +- Acceptance: the full gate list in `evidence.md` green. diff --git a/cli/aidd_docs/memory/cli.md b/cli/aidd_docs/memory/cli.md index a45792cf9..600aebf2e 100644 --- a/cli/aidd_docs/memory/cli.md +++ b/cli/aidd_docs/memory/cli.md @@ -11,7 +11,7 @@ Thirty leaf commands. Read them live: `aidd --help`, then each group's. `scripts - `plugin install | list | remove | search | update`. - `marketplace add | list | remove | refresh | check`. - `auth login | logout | status`. -- `telemetry on | off | forget | ingest | task | status | report | identity`: local token measurement for Claude Code, mapped in `telemetry.md`. +- `telemetry on | off | forget | ingest | task | status | report | identity`: local token measurement for Claude Code and Codex, mapped in `telemetry.md`. - `translate `: author-side, converts a source into a target-native plugin tree. - `update`, aliased `upgrade`: the CLI itself. diff --git a/cli/aidd_docs/memory/codebase-map.md b/cli/aidd_docs/memory/codebase-map.md index 1078b4c55..86da7d334 100644 --- a/cli/aidd_docs/memory/codebase-map.md +++ b/cli/aidd_docs/memory/codebase-map.md @@ -77,7 +77,8 @@ src/ │ │ ├── identity/ # the identity file in the telemetry dir │ │ ├── legacy/ # the table of ids an earlier version gave repositories, beside the ledger │ │ ├── left-out/ # what ingests measured and could not store, by hash, beside the ledger -│ │ └── switch/ # the project config, what an earlier measurement left in a repository, and Claude settings; the clone's consent in git config is `git-consent-adapter.ts`, beside this folder +│ │ ├── switch/ # the project config, what an earlier measurement left in a repository, and Claude settings; the clone's consent in git config is `git-consent-adapter.ts`, beside this folder +│ │ └── transcripts/ # reading each tool's session files a block at a time; the Codex source │ └── translate/ # canonical source to target-native content │ ├── application/ │ │ └── strategies/ diff --git a/cli/aidd_docs/memory/project-brief.md b/cli/aidd_docs/memory/project-brief.md index df0b82749..2cf0f0cfa 100644 --- a/cli/aidd_docs/memory/project-brief.md +++ b/cli/aidd_docs/memory/project-brief.md @@ -33,4 +33,4 @@ What this package is, the problem it solves, and its domain language. - Install, update and remove the framework for a tool, and repair what drifted. - Install plugins from a marketplace, driving a tool's own CLI where its project files are inert. - Translate an arbitrary source into a target-native plugin tree, recording nothing. -- Count what Claude Code sessions consume, from local files, with no service and no upload. +- Count what Claude Code and Codex sessions consume, from local files, with no service and no upload. diff --git a/cli/aidd_docs/memory/telemetry.md b/cli/aidd_docs/memory/telemetry.md index d5e8a57eb..8c7f05dd0 100644 --- a/cli/aidd_docs/memory/telemetry.md +++ b/cli/aidd_docs/memory/telemetry.md @@ -6,10 +6,11 @@ | Edit | Where | | --- | --- | -| its line format | `contexts/telemetry/domain/formats/` | -| its transcript source | `contexts/telemetry/infrastructure/` | +| its line format, made once per file read when a line needs what came before it | `contexts/telemetry/domain/formats/` | +| its transcript source, on the block reader in `infrastructure/transcripts/transcript-files.ts` | `contexts/telemetry/infrastructure/` | | one reader in the ingest list | `runtime/wiring/telemetry.ts` | | its name | `USAGE_TOOLS` in `domain/usage-record.ts` | +| its home relocated in every sandbox that runs the CLI | `tests/e2e/helpers.ts`, `tests/helpers/telemetry-sandbox.ts`, `scripts/check-claude-tokens-end-to-end.cjs`, `scripts/probe-identifier-join.cjs` | Storage, attribution and the report need nothing. Task and ticket come only from `aidd telemetry task`, whoever runs it; the plugin's hooks only invoke it from Claude Code, so another tool's calls are attributed by branch. Rules: [usage contract](../../../aidd_docs/product/usage-contract.md#attribution). @@ -60,7 +61,8 @@ flowchart LR | What a clone is | `domain/consent/clone-identity.ts` | | When a clone consented | `domain/consent/consent-history.ts`, `application/consent-log.ts` | | Which transcripts are read | `application/ingest-usage-use-case.ts`, `application/read-usage-use-case.ts` | -| How one is read: from its saved offset, a block at a time, up to its last complete line | `infrastructure/claude-transcript-source-adapter.ts`, `domain/transcript-position.ts` | +| How one is read: from its saved offset, a block at a time, up to its last complete line | `infrastructure/claude-transcript-source-adapter.ts`, `infrastructure/transcripts/transcript-files.ts` (shared by every tool), `domain/transcript-position.ts` | +| What a Codex line becomes, and why a Codex file that moved is read from its start | `domain/formats/codex-transcript-usage.ts`, `infrastructure/transcripts/codex-transcript-source-adapter.ts` | | How the ledger lies on disk, and what an ingest or a report reads | `infrastructure/usage-ledger-adapter.ts`, `domain/stored-usage.ts`, `application/ledger-upkeep.ts`, `domain/report/period.ts` | | What an ingest measured and could not store, kept for every report (`ledger/left-out.txt`), and why a ledger line no reader knows is kept | `infrastructure/left-out/left-out-adapter.ts`, `infrastructure/usage-ledger-adapter.ts`, `application/ingest-usage-use-case.ts`; rules in the [usage contract](../../../aidd_docs/product/usage-contract.md#left-out) | | What `on` and `forget` clean up from an earlier measurement | `domain/legacy/`, `application/switch/`, `application/forget/` | diff --git a/cli/aidd_docs/memory/testing.md b/cli/aidd_docs/memory/testing.md index e13f24e9b..a48f68392 100644 --- a/cli/aidd_docs/memory/testing.md +++ b/cli/aidd_docs/memory/testing.md @@ -26,7 +26,7 @@ How this package is tested: the layers, the tools, the conventions. - A golden snapshot is machine-independent, never derived from an absolute path: [`golden-machine-independence.md`](../../.claude/skills/test/references/golden-machine-independence.md). - `describe.concurrent` appears in no unit or integration file; in e2e the golden and command-matrix suites use it. An observed split, not a rule. - A bug fix's review reproduces the user's scenario against the built binary: [`bug-empirical-reproduction.md`](../../.claude/skills/test/references/bug-empirical-reproduction.md). -- A sandboxed run reaches no tool binary and no real profile: `tests/e2e/helpers.ts` narrows `PATH`, relocates `HOME`/`XDG_CONFIG_HOME`/`USERPROFILE`/`APPDATA`; `sandbox-reaches-no-tool-binary.e2e.test.ts` holds it. `CODEX_HOME` is never overridden: a machine setting it points a real `codex` at its own profile regardless. +- A sandboxed run reaches no tool binary and no real profile: `tests/e2e/helpers.ts` narrows `PATH`, relocates `HOME`/`XDG_CONFIG_HOME`/`USERPROFILE`/`APPDATA` and `CODEX_HOME`, which Codex and the telemetry reader of its sessions read before `HOME`; `sandbox-reaches-no-tool-binary.e2e.test.ts` holds both. - Records land under `AppData\Roaming` on Windows, `.config` elsewhere. Assert through the helper. - Green unit and integration tests prove output shape, never that a tool consumes it: that is `pnpm smoke`. diff --git a/cli/src/contexts/telemetry/application/read-usage-use-case.ts b/cli/src/contexts/telemetry/application/read-usage-use-case.ts index 6fa8b4f2a..50f91d1a1 100644 --- a/cli/src/contexts/telemetry/application/read-usage-use-case.ts +++ b/cli/src/contexts/telemetry/application/read-usage-use-case.ts @@ -12,6 +12,9 @@ export type ReadUsageLine = (line: string) => { readonly named?: LineNames; }; +/** A tool's format, made once per file read: it may hold what earlier lines of the file said. */ +export type UsageFormat = () => ReadUsageLine; + /** A line that gave no call, the directory and time it names, and its file. */ export interface DroppedLine extends LineNames { readonly line: string; @@ -40,7 +43,7 @@ export interface UsageReading { export class ReadUsageUseCase { constructor( private readonly source: TranscriptSource, - private readonly readLine: ReadUsageLine + private readonly format: UsageFormat ) {} /** `writtenFrom`: files last written before it are not read; none, by default. */ @@ -59,13 +62,22 @@ export class ReadUsageUseCase { const records: UsageRecord[] = []; const lines: DroppedLine[] = []; const reasons: string[] = []; - const read = await this.source.read(path, since.get(path) ?? null, writtenFrom, (line) => { - const outcome = this.readLine(line); - records.push(...outcome.records); - // Only a line that gives no call names anything. - if (outcome.named?.cwd) lines.push({ line, path, ...outcome.named }); - reasons.push(...outcome.unrecognised.map((reason) => `${path}: ${reason}`)); - }); + const readLine = this.format(); + const read = await this.source.read( + path, + since.get(path) ?? null, + writtenFrom, + (line, counted) => { + const outcome = readLine(line); + // A line read again only to know what came before: its calls were counted already. + if (!counted) return; + records.push(...outcome.records); + // Only a line that gives no call names anything. + if (outcome.named?.cwd) lines.push({ line, path, ...outcome.named }); + reasons.push(...outcome.unrecognised.map((reason) => `${path}: ${reason}`)); + } + ); + reasons.push(...(read.unrecognised ?? []).map((reason) => `${path}: ${reason}`)); if (read.skipped) skipped.add(path); // Changed under the read: what it handed over may be spliced from two versions, so none // of it counts and the file keeps its position, to be read again. diff --git a/cli/src/contexts/telemetry/domain/formats/claude-transcript-usage.ts b/cli/src/contexts/telemetry/domain/formats/claude-transcript-usage.ts index c782cc9bb..59df10824 100644 --- a/cli/src/contexts/telemetry/domain/formats/claude-transcript-usage.ts +++ b/cli/src/contexts/telemetry/domain/formats/claude-transcript-usage.ts @@ -1,5 +1,6 @@ import { asPlainObject } from "../../../../kernel/reading/plain-object.js"; import type { LineNames, UsageRecord } from "../usage-record.js"; +import { counter, expected, type Json, text } from "./line-values.js"; export interface ClaudeUsageLine { readonly records: readonly UsageRecord[]; @@ -9,34 +10,17 @@ export interface ClaudeUsageLine { readonly named?: LineNames; } -type Json = Record; - const SYNTHETIC_MODEL = ""; /** What an advisor iteration adds to its message's key. Anchored for clarity only: the format * writes it as a suffix alone, and message and session ids hold no `#`. */ const ADVISOR = /#advisor\d+$/; const NONE: ClaudeUsageLine = { records: [], unrecognised: [] }; -function counter(value: unknown): number | null { - return typeof value === "number" && Number.isInteger(value) && value >= 0 ? value : null; -} - -function text(value: unknown): string | null { - return typeof value === "string" && value !== "" ? value : null; -} - function unrecognised(reason: string, root?: Json): ClaudeUsageLine { const named = root && { cwd: text(root.cwd), at: text(root.timestamp) }; return { records: [], unrecognised: [reason], named }; } -/** A counter that must be there: absent or malformed is unknown, and said so. */ -function expected(source: Json, field: string, where: string, notes: string[]): number | null { - const value = counter(source[field]); - if (value === null) notes.push(`${where}.${field} is missing or not a count`); - return value; -} - function messageIterations(usage: Json, notes: string[]): Json[] { const raw = usage.iterations; if (raw === undefined) return []; diff --git a/cli/src/contexts/telemetry/domain/formats/codex-transcript-usage.ts b/cli/src/contexts/telemetry/domain/formats/codex-transcript-usage.ts new file mode 100644 index 000000000..e24c3efc9 --- /dev/null +++ b/cli/src/contexts/telemetry/domain/formats/codex-transcript-usage.ts @@ -0,0 +1,83 @@ +import { asPlainObject } from "../../../../kernel/reading/plain-object.js"; +import type { LineNames, UsageRecord } from "../usage-record.js"; +import { expected, type Json, text } from "./line-values.js"; + +export interface CodexUsageLine { + readonly records: readonly UsageRecord[]; + /** One entry per shape not recognised, never turned into a record. */ + readonly unrecognised: readonly string[]; + /** For a call line that gave no record: its context's directory and its time. */ + readonly named?: LineNames; +} + +const CALL = "token_usage_record"; +const NONE: CodexUsageLine = { records: [], unrecognised: [] }; + +/** A reader for one Codex session file, read from its start: a call's model and directory are + * written before it, in the latest `turn_context`, and its version and branch in the file's + * first `session_meta` (a sub-agent's file carries its parent's after its own). Only a + * `token_usage_record` is a call: the turn and thread totals beside it, and `token_count` + * events, are running totals that would count a call many times. */ +export function codexUsageReader(): (line: string) => CodexUsageLine { + let session: Json | null = null; + let turn: Json | null = null; + const cwd = () => text(turn?.cwd) ?? text(session?.cwd); + const unrecognised = (reason: string, at: string | null): CodexUsageLine => ({ + records: [], + unrecognised: [reason], + named: { cwd: cwd(), at }, + }); + + return (line) => { + let parsed: unknown; + try { + parsed = JSON.parse(line); + } catch { + return line.includes(CALL) ? unrecognised(`a line holding ${CALL} is not JSON`, null) : NONE; + } + const root = asPlainObject(parsed); + const payload = asPlainObject(root?.payload); + if (root?.type === "session_meta") session ??= payload ?? {}; + if (root?.type === "turn_context") turn = payload; + if (root?.type !== CALL) return NONE; + + const usage = asPlainObject(payload?.usage); + const key = text(payload?.response_id); + const sessionId = text(payload?.session_id); + const thread = text(payload?.thread_id); + const at = text(root.timestamp); + if (usage === null || key === null || sessionId === null || thread === null || at === null) { + return unrecognised( + `a ${CALL} lacks response_id, session_id, thread_id, timestamp or usage`, + at + ); + } + + const notes: string[] = []; + const count = (field: string) => expected(usage, field, "usage", notes); + const input = count("input_tokens"); + const cacheRead = count("cached_input_tokens"); + const cacheWrite = count("cache_write_input_tokens"); + const record: UsageRecord = { + tool: "codex", + tool_version: text(session?.cli_version), + key, + session_id: sessionId, + agent: thread === sessionId ? "main" : "subagent", + at, + model: text(turn?.model), + input: + input === null || cacheRead === null || cacheWrite === null + ? null + : input - cacheRead - cacheWrite, + output: count("output_tokens"), + cache_read: cacheRead, + cache_write: cacheWrite, + cache_write_1h: null, + reasoning: count("reasoning_output_tokens"), + cwd: cwd(), + git_branch: text(asPlainObject(session?.git)?.branch), + }; + return { records: [record], unrecognised: notes }; + }; +} diff --git a/cli/src/contexts/telemetry/domain/formats/line-values.ts b/cli/src/contexts/telemetry/domain/formats/line-values.ts new file mode 100644 index 000000000..924039557 --- /dev/null +++ b/cli/src/contexts/telemetry/domain/formats/line-values.ts @@ -0,0 +1,22 @@ +/** Reading a transcript line's values the same way in every tool's format. */ +export type Json = Record; + +export function counter(value: unknown): number | null { + return typeof value === "number" && Number.isInteger(value) && value >= 0 ? value : null; +} + +export function text(value: unknown): string | null { + return typeof value === "string" && value !== "" ? value : null; +} + +/** A counter that must be there: absent or malformed is unknown, and said so. */ +export function expected( + source: Json, + field: string, + where: string, + notes: string[] +): number | null { + const value = counter(source[field]); + if (value === null) notes.push(`${where}.${field} is missing or not a count`); + return value; +} diff --git a/cli/src/contexts/telemetry/domain/ports/transcript-source.ts b/cli/src/contexts/telemetry/domain/ports/transcript-source.ts index c0d34967f..f158fd9ca 100644 --- a/cli/src/contexts/telemetry/domain/ports/transcript-source.ts +++ b/cli/src/contexts/telemetry/domain/ports/transcript-source.ts @@ -9,6 +9,9 @@ export interface TranscriptRead { readonly restarted: boolean; /** Last written before `writtenFrom`: not read, and its position is its end. */ readonly skipped: boolean; + /** Shapes met that could not be handed on as lines, such as a file this runtime cannot + * decompress. */ + readonly unrecognised?: readonly string[]; } export interface TranscriptSource { @@ -19,12 +22,14 @@ export interface TranscriptSource { * still be read. */ oldestModified(paths: readonly string[]): Promise; /** Hands `each` every complete line after the position, in order, one at a time so a long - * transcript is never held whole; an unterminated last line is left for next time. + * transcript is never held whole; an unterminated last line is left for next time. A tool + * whose lines need what came before them may hand earlier lines too, as not `counted`: they + * only build the context of the counted ones. * `writtenFrom`: a file last written before this instant, in milliseconds, is not read. */ read( path: string, since: TranscriptPosition | null, writtenFrom: number, - each: (line: string) => void + each: (line: string, counted: boolean) => void ): Promise; } diff --git a/cli/src/contexts/telemetry/infrastructure/claude-transcript-source-adapter.ts b/cli/src/contexts/telemetry/infrastructure/claude-transcript-source-adapter.ts index 4b7d11c41..790a54d73 100644 --- a/cli/src/contexts/telemetry/infrastructure/claude-transcript-source-adapter.ts +++ b/cli/src/contexts/telemetry/infrastructure/claude-transcript-source-adapter.ts @@ -1,78 +1,18 @@ -import { constants, type Stats } from "node:fs"; -import { type FileHandle, open, readdir } from "node:fs/promises"; -import { join } from "node:path"; -import { isErrnoException } from "../../../kernel/reading/json-file.js"; -import { modifiedAtIfPresent } from "../../../kernel/reading/text-file.js"; import type { TranscriptRead, TranscriptSource } from "../domain/ports/transcript-source.js"; -import { resumeOffset, type TranscriptPosition } from "../domain/transcript-position.js"; +import type { TranscriptPosition } from "../domain/transcript-position.js"; +import { + type EachLine, + eachLine, + lastLineEnd, + listTranscripts, + oldestModifiedOf, + readTranscript, +} from "./transcripts/transcript-files.js"; -const NEWLINE = 0x0a; -/** How far back from its end a skipped file is searched for its last complete line. */ -const TAIL_BYTES = 1 << 16; const TRANSCRIPT_NAME = /\.jsonl(\.superseded-.+)?$/; -function absentIsNothing(error: unknown): null { - if (isErrnoException(error) && error.code === "ENOENT") return null; - throw error; -} - -/** Device and inode name the file, not its path. Where the inode is not meaningful the birth - * time stands in for it. */ -function identityOf(stats: Stats): string { - return stats.ino === 0 ? `${stats.dev}:${stats.birthtimeMs}` : `${stats.dev}:${stats.ino}`; -} - -/** The end of the file's last complete line, searched back from its end a block at a time. */ -async function lastLineEnd(handle: FileHandle, size: number): Promise { - for (let end = size; end > 0; end -= TAIL_BYTES) { - const from = Math.max(0, end - TAIL_BYTES); - const block = Buffer.alloc(end - from); - const { bytesRead } = await handle.read(block, 0, block.length, from); - const last = block.subarray(0, bytesRead).lastIndexOf(NEWLINE); - if (last !== -1) return from + last + 1; - } - return 0; -} - -/** Hands `each` every complete line between `from` and `size`, a block at a time, and returns - * the end of the last one. A line split across blocks is kept in pieces and decoded once whole: - * byte 0x0a is never inside a UTF-8 sequence, so this decodes as the whole range would. Memory is - * a block and up to about three copies of the longest line (its pieces, the joined bytes, the - * decoded string), whatever the range. */ -async function eachLine( - handle: FileHandle, - from: number, - size: number, - blockBytes: number, - each: (line: string) => void -): Promise { - const block = Buffer.alloc(Math.min(blockBytes, Math.max(0, size - from))); - let pieces: Buffer[] = []; - let consumed = from; - for (let at = from; at < size; ) { - const { bytesRead } = await handle.read(block, 0, Math.min(block.length, size - at), at); - // Shorter than it was when stat'd: the caller sees it and discards this read. - if (bytesRead === 0) break; - const read = block.subarray(0, bytesRead); - let start = 0; - for (let end = read.indexOf(NEWLINE); end !== -1; end = read.indexOf(NEWLINE, start)) { - const line = - pieces.length === 0 - ? read.toString("utf8", start, end) - : Buffer.concat([...pieces, read.subarray(start, end)]).toString("utf8"); - each(line.endsWith("\r") ? line.slice(0, -1) : line); - pieces = []; - start = end + 1; - consumed = at + start; - } - // A copy: the block is read into again. - if (start < bytesRead) pieces.push(Buffer.from(read.subarray(start))); - at += bytesRead; - } - return consumed; -} - -/** Reads the transcripts Claude Code keeps under `/projects`. */ +/** Reads the transcripts Claude Code keeps under `/projects`, each from where the + * last read stopped: every Claude Code line carries its own directory. */ export class ClaudeTranscriptSourceAdapter implements TranscriptSource { constructor( private readonly projectsRoot: string, @@ -80,70 +20,24 @@ export class ClaudeTranscriptSourceAdapter implements TranscriptSource { private readonly blockBytes = 1 << 20 ) {} - async list(): Promise { - const found: string[] = []; - await this.walk(this.projectsRoot, found); - return found.sort(); - } - - private async walk(dir: string, found: string[]): Promise { - const entries = await readdir(dir, { withFileTypes: true }).catch(absentIsNothing); - if (entries === null) return; - for (const entry of entries) { - const path = join(dir, entry.name); - if (entry.isDirectory()) await this.walk(path, found); - else if (entry.isFile() && TRANSCRIPT_NAME.test(entry.name)) found.push(path); - } + list(): Promise { + return listTranscripts([this.projectsRoot], TRANSCRIPT_NAME); } - async oldestModified(paths: readonly string[]): Promise { - let oldest: number | null = null; - for (const path of paths) { - const modified = await modifiedAtIfPresent(path); - if (modified !== null && (oldest === null || modified < oldest)) oldest = modified; - } - return oldest === null ? null : new Date(oldest).toISOString(); + oldestModified(paths: readonly string[]): Promise { + return oldestModifiedOf(paths); } - async read( + read( path: string, since: TranscriptPosition | null, writtenFrom: number, - each: (line: string) => void + each: EachLine ): Promise { - const handle = await open(path, constants.O_RDONLY).catch(absentIsNothing); - if (handle === null) { - return { - position: since ?? { offset: 0, size: 0, identity: "" }, - restarted: false, - skipped: false, - }; - } - try { - const stats = await handle.stat(); - // Windows opens a directory and reports it empty: never read one as a transcript. - if (!stats.isFile()) throw new Error(`${path} is not a file`); - const now = { size: stats.size, identity: identityOf(stats) }; - if (stats.mtimeMs < writtenFrom) { - // Nothing in it is covered: every line was written before the first interval opened. - const offset = await lastLineEnd(handle, now.size); - return { position: { offset, ...now }, restarted: false, skipped: true }; - } - const { offset, restarted } = resumeOffset(since, now); - const end = await eachLine(handle, offset, now.size, this.blockBytes, each); - // A transcript only grows. Shorter now: it was rewritten under the read, a line may be - // spliced from two versions, and no position past it can be trusted. Another file put at - // its path is no harm: the handle still reads the one it opened, whole. - if ((await handle.stat()).size < now.size) { - return { position: null, restarted, skipped: false }; - } - return { - position: { offset: end, size: now.size, identity: now.identity }, - restarted, - skipped: false, - }; - } finally { - await handle.close(); - } + return readTranscript(path, since, writtenFrom, { + skippedEnd: lastLineEnd, + lines: (handle, now, { offset }) => + eachLine(handle, offset, offset, now.size, this.blockBytes, each), + }); } } diff --git a/cli/src/contexts/telemetry/infrastructure/transcripts/codex-transcript-source-adapter.ts b/cli/src/contexts/telemetry/infrastructure/transcripts/codex-transcript-source-adapter.ts new file mode 100644 index 000000000..6197092ac --- /dev/null +++ b/cli/src/contexts/telemetry/infrastructure/transcripts/codex-transcript-source-adapter.ts @@ -0,0 +1,115 @@ +import { join } from "node:path"; +import type { Duplex } from "node:stream"; +import { pipeline } from "node:stream/promises"; +import * as zlib from "node:zlib"; +import { isErrnoException } from "../../../../kernel/reading/json-file.js"; +import type { TranscriptRead, TranscriptSource } from "../../domain/ports/transcript-source.js"; +import type { TranscriptPosition } from "../../domain/transcript-position.js"; +import { + type EachLine, + eachLine, + LineSplitter, + lastLineEnd, + listTranscripts, + NOWHERE, + oldestModifiedOf, + readTranscript, +} from "./transcript-files.js"; +import { declaredContentSize, FRAME_HEADER_BYTES } from "./zstd-frame.js"; + +const SESSION_FILE = /^rollout-.*\.jsonl(\.zst)?$/; +const COMPRESSED = ".zst"; +/** What zlib names a stream it cannot decompress by: `Z_…` or `ZSTD_error_…`, never an I/O code. */ +const UNDECOMPRESSABLE = /^Z(STD)?_/; + +/** A compressed file not read, named once per run: its position stays, so it is tried again. */ +/** A stream that decompressed to another size than its frame declares. */ +class FrameSizeMismatch extends Error {} + +function unread(since: TranscriptPosition | null, reason: string): TranscriptRead { + return { position: since ?? NOWHERE, restarted: false, skipped: false, unrecognised: [reason] }; +} + +/** Where Codex keeps its session files. `codexHome` is `CODEX_HOME`; reading the environment is + * the composition root's job. Archiving moves a file from the first to the second. */ +export function codexSessionRoots(codexHome: string | undefined, home: string): string[] { + const dir = codexHome === undefined || codexHome === "" ? join(home, ".codex") : codexHome; + return [join(dir, "sessions"), join(dir, "archived_sessions")]; +} + +/** Reads the session files Codex keeps. A call's directory and model are written earlier in the + * file than the call, so a file that moved is read from its start, the lines before the saved + * offset handed on as not counted. Codex compresses a file it no longer writes to + * `.jsonl.zst`: that one is read whole, once, as a stream. */ +export class CodexTranscriptSourceAdapter implements TranscriptSource { + constructor( + private readonly roots: readonly string[], + /** Absent from a Node whose zlib has no zstd. */ + private readonly decompress: (() => Duplex) | null = zlib.createZstdDecompress ?? null, + /** How much of a plain file is read at once. */ + private readonly blockBytes = 1 << 20 + ) {} + + list(): Promise { + return listTranscripts(this.roots, SESSION_FILE); + } + + oldestModified(paths: readonly string[]): Promise { + return oldestModifiedOf(paths); + } + + async read( + path: string, + since: TranscriptPosition | null, + writtenFrom: number, + each: EachLine + ): Promise { + if (!path.endsWith(COMPRESSED)) { + return readTranscript(path, since, writtenFrom, { + skippedEnd: lastLineEnd, + lines: (handle, now, { offset }) => { + const from = since?.size === now.size ? offset : 0; + return eachLine(handle, from, offset, now.size, this.blockBytes, each); + }, + }); + } + const decompress = this.decompress; + if (decompress === null) { + return unread(since, "a compressed session file is not read: this Node has no zstd"); + } + // Its offset is its size: it does not change, so its size and identity keep it read. + return readTranscript(path, since, writtenFrom, { + skippedEnd: async (_handle, size) => size, + lines: async (handle, now, { restarted }) => { + if (!restarted && since?.size === now.size) return now.size; + const header = Buffer.alloc(FRAME_HEADER_BYTES); + const { bytesRead } = await handle.read(header, 0, header.length, 0); + const declared = declaredContentSize(header.subarray(0, bytesRead)); + const splitter = new LineSplitter(0, 0, each); + let decompressed = 0; + await pipeline( + handle.createReadStream({ start: 0, autoClose: false }), + decompress(), + async (chunks: AsyncIterable) => { + for await (const chunk of chunks) { + decompressed += chunk.length; + splitter.push(chunk); + } + } + ); + if (declared !== null && decompressed !== declared) { + throw new FrameSizeMismatch( + `${decompressed} of the ${declared} bytes its frame declares` + ); + } + return now.size; + }, + }).catch((error: unknown) => { + if (error instanceof FrameSizeMismatch) { + return unread(since, `a compressed session file does not decompress: ${error.message}`); + } + if (!isErrnoException(error) || !UNDECOMPRESSABLE.test(String(error.code))) throw error; + return unread(since, `a compressed session file does not decompress: ${error.code}`); + }); + } +} diff --git a/cli/src/contexts/telemetry/infrastructure/transcripts/transcript-files.ts b/cli/src/contexts/telemetry/infrastructure/transcripts/transcript-files.ts new file mode 100644 index 000000000..70e15f898 --- /dev/null +++ b/cli/src/contexts/telemetry/infrastructure/transcripts/transcript-files.ts @@ -0,0 +1,176 @@ +import { constants, type Stats } from "node:fs"; +import { type FileHandle, open, readdir } from "node:fs/promises"; +import { join } from "node:path"; +import { isErrnoException } from "../../../../kernel/reading/json-file.js"; +import { modifiedAtIfPresent } from "../../../../kernel/reading/text-file.js"; +import type { TranscriptRead } from "../../domain/ports/transcript-source.js"; +import { + resumeOffset, + type TranscriptPosition, + type TranscriptStat, +} from "../../domain/transcript-position.js"; + +const NEWLINE = 0x0a; +/** How far back from its end a skipped file is searched for its last complete line. */ +const TAIL_BYTES = 1 << 16; +/** Where a file that cannot be read stands: no position a real file could match. */ +export const NOWHERE: TranscriptPosition = { offset: 0, size: 0, identity: "" }; + +/** One complete line, and whether it starts at or past the offset lines are counted from. */ +export type EachLine = (line: string, counted: boolean) => void; + +function absentIsNothing(error: unknown): null { + if (isErrnoException(error) && error.code === "ENOENT") return null; + throw error; +} + +/** Device and inode name the file, not its path. Where the inode is not meaningful the birth + * time stands in for it. */ +function identityOf(stats: Stats): string { + return stats.ino === 0 ? `${stats.dev}:${stats.birthtimeMs}` : `${stats.dev}:${stats.ino}`; +} + +/** The end of the file's last complete line, searched back from its end a block at a time. */ +export async function lastLineEnd(handle: FileHandle, size: number): Promise { + for (let end = size; end > 0; end -= TAIL_BYTES) { + const from = Math.max(0, end - TAIL_BYTES); + const block = Buffer.alloc(end - from); + const { bytesRead } = await handle.read(block, 0, block.length, from); + const last = block.subarray(0, bytesRead).lastIndexOf(NEWLINE); + if (last !== -1) return from + last + 1; + } + return 0; +} + +/** Cuts bytes pushed in order into complete lines, split on byte 0x0a only. A line split + * across pushes is kept in pieces and decoded once whole: byte 0x0a is never inside a UTF-8 + * sequence, so this decodes as the whole range would. Memory is up to about three copies of the + * longest line (its pieces, the joined bytes, the decoded string). */ +export class LineSplitter { + private pieces: Buffer[] = []; + private at: number; + /** The end of the last complete line. */ + consumed: number; + + constructor( + from: number, + private readonly countFrom: number, + private readonly each: EachLine + ) { + this.at = from; + this.consumed = from; + } + + push(read: Buffer): void { + let start = 0; + for (let end = read.indexOf(NEWLINE); end !== -1; end = read.indexOf(NEWLINE, start)) { + const line = + this.pieces.length === 0 + ? read.toString("utf8", start, end) + : Buffer.concat([...this.pieces, read.subarray(start, end)]).toString("utf8"); + const counted = this.consumed >= this.countFrom; + this.each(line.endsWith("\r") ? line.slice(0, -1) : line, counted); + this.pieces = []; + start = end + 1; + this.consumed = this.at + start; + } + // A copy: the caller may read into the same buffer again. + if (start < read.length) this.pieces.push(Buffer.from(read.subarray(start))); + this.at += read.length; + } +} + +/** Hands `each` every complete line between `from` and `size`, a block at a time, and returns + * the end of the last one. Memory is a block and what the splitter keeps, whatever the range. */ +export async function eachLine( + handle: FileHandle, + from: number, + countFrom: number, + size: number, + blockBytes: number, + each: EachLine +): Promise { + const block = Buffer.alloc(Math.min(blockBytes, Math.max(0, size - from))); + const splitter = new LineSplitter(from, countFrom, each); + for (let at = from; at < size; ) { + const { bytesRead } = await handle.read(block, 0, Math.min(block.length, size - at), at); + // Shorter than it was when stat'd: the caller sees it and discards this read. + if (bytesRead === 0) break; + splitter.push(block.subarray(0, bytesRead)); + at += bytesRead; + } + return splitter.consumed; +} + +async function walk(dir: string, name: RegExp, found: string[]): Promise { + const entries = await readdir(dir, { withFileTypes: true }).catch(absentIsNothing); + if (entries === null) return; + for (const entry of entries) { + const path = join(dir, entry.name); + if (entry.isDirectory()) await walk(path, name, found); + else if (entry.isFile() && name.test(entry.name)) found.push(path); + } +} + +/** Every file under the roots, at any depth, whose name matches. */ +export async function listTranscripts( + roots: readonly string[], + name: RegExp +): Promise { + const found: string[] = []; + for (const root of roots) await walk(root, name, found); + return found.sort(); +} + +export async function oldestModifiedOf(paths: readonly string[]): Promise { + let oldest: number | null = null; + for (const path of paths) { + const modified = await modifiedAtIfPresent(path); + if (modified !== null && (oldest === null || modified < oldest)) oldest = modified; + } + return oldest === null ? null : new Date(oldest).toISOString(); +} + +/** How one tool's file is read once opened: where a skipped one ends, and its lines. */ +export interface TranscriptReading { + skippedEnd(handle: FileHandle, size: number): Promise; + /** Hands the lines on and returns the end of the last one. */ + lines( + handle: FileHandle, + now: TranscriptStat, + start: ReturnType + ): Promise; +} + +/** Opens a transcript and reads it as `reading` says, from where `since` left it. */ +export async function readTranscript( + path: string, + since: TranscriptPosition | null, + writtenFrom: number, + reading: TranscriptReading +): Promise { + const handle = await open(path, constants.O_RDONLY).catch(absentIsNothing); + if (handle === null) return { position: since ?? NOWHERE, restarted: false, skipped: false }; + try { + const stats = await handle.stat(); + // Windows opens a directory and reports it empty: never read one as a transcript. + if (!stats.isFile()) throw new Error(`${path} is not a file`); + const now = { size: stats.size, identity: identityOf(stats) }; + if (stats.mtimeMs < writtenFrom) { + // Nothing in it is covered: every line was written before the first interval opened. + const offset = await reading.skippedEnd(handle, now.size); + return { position: { offset, ...now }, restarted: false, skipped: true }; + } + const start = resumeOffset(since, now); + const end = await reading.lines(handle, now, start); + // A transcript only grows. Shorter now: it was rewritten under the read, a line may be + // spliced from two versions, and no position past it can be trusted. Another file put at + // its path is no harm: the handle still reads the one it opened, whole. + if ((await handle.stat()).size < now.size) { + return { position: null, restarted: start.restarted, skipped: false }; + } + return { position: { offset: end, ...now }, restarted: start.restarted, skipped: false }; + } finally { + await handle.close(); + } +} diff --git a/cli/src/contexts/telemetry/infrastructure/transcripts/zstd-frame.ts b/cli/src/contexts/telemetry/infrastructure/transcripts/zstd-frame.ts new file mode 100644 index 000000000..498ad0301 --- /dev/null +++ b/cli/src/contexts/telemetry/infrastructure/transcripts/zstd-frame.ts @@ -0,0 +1,25 @@ +const MAGIC = 0xfd2fb528; +/** Bytes of the dictionary id, by its two-bit flag. */ +const DICTIONARY_ID_BYTES = [0, 1, 2, 4] as const; +/** Bytes of the content size, by its two-bit flag; flag 0 means one byte on a single segment. */ +const CONTENT_SIZE_BYTES = [0, 2, 4, 8] as const; +/** The longest header up to the end of its content size: magic 4, descriptor 1, window 1, + * dictionary id 4, size 8. */ +export const FRAME_HEADER_BYTES = 18; + +/** The decompressed size the first zstd frame of `header` declares (RFC 8878, 3.1.1.1), `null` + * when it declares none or the bytes hold no complete header. zlib is not trusted to say a + * frame was cut: Node 22's ends one quietly, short. */ +export function declaredContentSize(header: Buffer): number | null { + if (header.length < 6 || header.readUInt32LE(0) !== MAGIC) return null; + const descriptor = header.readUInt8(4); + const singleSegment = (descriptor & 0b0010_0000) !== 0; + const flag = descriptor >> 6; + const width = flag === 0 && singleSegment ? 1 : CONTENT_SIZE_BYTES[flag]; + const at = 5 + (singleSegment ? 0 : 1) + DICTIONARY_ID_BYTES[descriptor & 0b11]; + if (width === 0 || header.length < at + width) return null; + if (width === 1) return header.readUInt8(at); + if (width === 2) return header.readUInt16LE(at) + 256; + if (width === 4) return header.readUInt32LE(at); + return Number(header.readBigUInt64LE(at)); +} diff --git a/cli/src/presentation/commands/telemetry.ts b/cli/src/presentation/commands/telemetry.ts index a2ce7cde1..b940269b9 100644 --- a/cli/src/presentation/commands/telemetry.ts +++ b/cli/src/presentation/commands/telemetry.ts @@ -127,7 +127,7 @@ function filterOption( export function registerTelemetryCommand(program: Command): void { const telemetry = program .command("telemetry") - .description("Measure what Claude Code sessions consume, locally"); + .description("Measure what Claude Code and Codex sessions consume, locally"); telemetry .command("on") diff --git a/cli/src/runtime/wiring/telemetry.ts b/cli/src/runtime/wiring/telemetry.ts index 275701108..8910bc2a1 100644 --- a/cli/src/runtime/wiring/telemetry.ts +++ b/cli/src/runtime/wiring/telemetry.ts @@ -22,6 +22,7 @@ import { claudeProjectsRoot, } from "../../contexts/telemetry/domain/claude-projects-root.js"; import { readClaudeUsageLine } from "../../contexts/telemetry/domain/formats/claude-transcript-usage.js"; +import { codexUsageReader } from "../../contexts/telemetry/domain/formats/codex-transcript-usage.js"; import { legacyLocations } from "../../contexts/telemetry/domain/legacy/legacy-locations.js"; import { refusedByEnvironment as refusedByEnvironmentValue } from "../../contexts/telemetry/domain/telemetry-consent.js"; import { BindingSnapshotStoreAdapter } from "../../contexts/telemetry/infrastructure/binding-snapshot-store-adapter.js"; @@ -43,6 +44,10 @@ import { ClaudeSettingsAdapter } from "../../contexts/telemetry/infrastructure/s import { LegacyHookAdapter } from "../../contexts/telemetry/infrastructure/switch/legacy-hook-adapter.js"; import { ProjectConfigAdapter } from "../../contexts/telemetry/infrastructure/switch/project-config-adapter.js"; import { RunJournalAdapter } from "../../contexts/telemetry/infrastructure/switch/run-journal-adapter.js"; +import { + CodexTranscriptSourceAdapter, + codexSessionRoots, +} from "../../contexts/telemetry/infrastructure/transcripts/codex-transcript-source-adapter.js"; import { UsageLedgerAdapter } from "../../contexts/telemetry/infrastructure/usage-ledger-adapter.js"; import { PrivateStorageAdapter } from "../filesystem/private-storage-adapter.js"; import { environmentWithoutGitVariables } from "../git/git-environment.js"; @@ -140,7 +145,11 @@ export function wireTelemetry(homedir: () => string): TelemetryDeps { new ClaudeTranscriptSourceAdapter( claudeProjectsRoot(process.env.CLAUDE_CONFIG_DIR, homedir()) ), - readClaudeUsageLine + () => readClaudeUsageLine + ), + new ReadUsageUseCase( + new CodexTranscriptSourceAdapter(codexSessionRoots(process.env.CODEX_HOME, homedir())), + codexUsageReader ), ], ledger, diff --git a/cli/tests/architecture/tool-addition-cost.arch.test.ts b/cli/tests/architecture/tool-addition-cost.arch.test.ts index c2f591f38..e14ca8aa1 100644 --- a/cli/tests/architecture/tool-addition-cost.arch.test.ts +++ b/cli/tests/architecture/tool-addition-cost.arch.test.ts @@ -83,6 +83,13 @@ const BASELINE: readonly { readonly path: string; readonly named: number }[] = [ { path: "src/contexts/telemetry/domain/usage-record.ts", named: 1 }, // Where Claude Code keeps its transcripts, named for the one tool whose directory it is. { path: "src/contexts/telemetry/domain/claude-projects-root.ts", named: 1 }, + // Codex's own session-file format and where Codex keeps those files, each named for the one + // tool whose files it reads: a tool measured is its reader, not a list a new tool joins. + { path: "src/contexts/telemetry/domain/formats/codex-transcript-usage.ts", named: 1 }, + { + path: "src/contexts/telemetry/infrastructure/transcripts/codex-transcript-source-adapter.ts", + named: 1, + }, // Claude Code's own settings files, read to say how long it keeps a transcript. { path: "src/contexts/telemetry/infrastructure/switch/claude-settings-adapter.ts", named: 1 }, // Cursor's project hooks file, named after the tool whose file it is: the directory it diff --git a/cli/tests/contexts/telemetry/application/codex-transcript-ingest.property.integration.test.ts b/cli/tests/contexts/telemetry/application/codex-transcript-ingest.property.integration.test.ts new file mode 100644 index 000000000..e3e363bf0 --- /dev/null +++ b/cli/tests/contexts/telemetry/application/codex-transcript-ingest.property.integration.test.ts @@ -0,0 +1,291 @@ +import { mkdir, mkdtemp, readdir, readFile, rm, writeFile } from "node:fs/promises"; +import { tmpdir } from "node:os"; +import { join } from "node:path"; +import * as zlib from "node:zlib"; +import * as fc from "fast-check"; +import { afterEach, beforeEach, describe, expect, it } from "vitest"; +import { DirectoryResolver } from "../../../../src/contexts/telemetry/application/directory-resolver.js"; +import { + type IngestResult, + IngestUsageUseCase, +} from "../../../../src/contexts/telemetry/application/ingest-usage-use-case.js"; +import { ReadUsageUseCase } from "../../../../src/contexts/telemetry/application/read-usage-use-case.js"; +import { SnapshotBindingsUseCase } from "../../../../src/contexts/telemetry/application/snapshot-bindings-use-case.js"; +import { readClaudeUsageLine } from "../../../../src/contexts/telemetry/domain/formats/claude-transcript-usage.js"; +import { codexUsageReader } from "../../../../src/contexts/telemetry/domain/formats/codex-transcript-usage.js"; +import { ClaudeTranscriptSourceAdapter } from "../../../../src/contexts/telemetry/infrastructure/claude-transcript-source-adapter.js"; +import { CodexTranscriptSourceAdapter } from "../../../../src/contexts/telemetry/infrastructure/transcripts/codex-transcript-source-adapter.js"; +import { UsageLedgerAdapter } from "../../../../src/contexts/telemetry/infrastructure/usage-ledger-adapter.js"; +import { PrivateStorageAdapter } from "../../../../src/runtime/filesystem/private-storage-adapter.js"; +import { + ALREADY_MIGRATED, + cloneOf, + FakeBindings, + FakeConsents, + FakeLocator, + grantedBy, + InMemoryBindingsLock, + InMemoryConsentHistory, + InMemoryResolutions, + InMemorySnapshots, +} from "../../../helpers/ports/in-memory-telemetry.js"; + +let root: string; +let runs = 0; + +beforeEach(async () => { + root = await mkdtemp(join(tmpdir(), "aidd-codex-ingest-")); +}); +afterEach(async () => { + await rm(root, { recursive: true, force: true }); +}); + +const NOW = new Date("2026-10-09T00:00:00.000Z"); +const SESSION = "44444444-4444-4444-8444-444444444444"; +const THREAD = "55555555-5555-4555-8555-555555555555"; +/** A clone that consents, one that does not, outside any repository, and none. */ +const PLACES = ["/work/in", "/work/out", "/work/none", undefined] as const; + +function machine(sessions: string, projects?: string) { + runs += 1; + const dir = join(root, `ledger-${runs}`); + const ledger = new UsageLedgerAdapter(dir, new PrivateStorageAdapter()); + const locator = new FakeLocator(); + const consents = new FakeConsents(); + const history = new InMemoryConsentHistory(); + for (const place of ["/work/in", "/work/out"]) { + const clone = cloneOf(`${place}/.git`, { ino: place }); + locator.directories.set(place, { + status: "repository", + root: place, + mainRoot: place, + clone, + remote: `https://github.com/acme${place}.git`, + rootCommit: "c0ffee", + }); + if (place === "/work/in") { + consents.cloneSays(clone, grantedBy(clone)); + history.consented(clone); + } + } + locator.directories.set("/work/none", { status: "outside-repository" }); + const ingest = new IngestUsageUseCase( + [ + new ReadUsageUseCase(new CodexTranscriptSourceAdapter([sessions]), codexUsageReader), + ...(projects === undefined + ? [] + : [ + new ReadUsageUseCase( + new ClaudeTranscriptSourceAdapter(projects), + () => readClaudeUsageLine + ), + ]), + ], + ledger, + new DirectoryResolver(locator, consents, new InMemoryResolutions(), history, { + caseInsensitiveFileSystem: false, + now: () => NOW, + }), + new SnapshotBindingsUseCase( + new FakeBindings(), + new InMemorySnapshots(), + new InMemoryBindingsLock(), + () => NOW + ), + { refusedByEnvironment: false }, + ALREADY_MIGRATED + ); + return { dir, ingest }; +} + +/** Every file under the ledger directory, offsets included, the lock left out. */ +async function bytes(dir: string): Promise> { + const found: Record = {}; + for (const entry of await readdir(dir, { recursive: true, withFileTypes: true })) { + if (!entry.isFile() || entry.name === ".lock") continue; + const path = join(entry.parentPath, entry.name); + found[path.slice(dir.length)] = await readFile(path, "latin1"); + } + return Object.fromEntries(Object.entries(found).sort(([a], [b]) => (a < b ? -1 : 1))); +} + +/** What runs say that adds up across them. */ +function counts(results: readonly IngestResult[]) { + return { + unrecognised: results.reduce((all, r) => all + r.unrecognised, 0), + notStored: Object.fromEntries( + Object.keys(results[0]?.notStored ?? {}).map((reason) => [ + reason, + results.reduce((all, r) => all + r.notStored[reason as keyof IngestResult["notStored"]], 0), + ]) + ), + }; +} + +const entry = (type: string, payload: Record, at = "2026-10-08T12:00:00.000Z") => + JSON.stringify({ timestamp: at, type, payload }); + +const call = (n: number, sub: boolean, at: string, total: number) => + entry( + "token_usage_record", + { + response_id: `resp_${n}`, + session_id: SESSION, + thread_id: sub ? THREAD : SESSION, + usage: { + input_tokens: total, + cached_input_tokens: 1, + cache_write_input_tokens: 0, + output_tokens: 2, + reasoning_output_tokens: 1, + total_tokens: total + 2, + }, + }, + at + ); + +const line = fc.oneof( + { + weight: 1, + arbitrary: fc + .constantFrom(...PLACES) + .map((cwd) => entry("session_meta", { session_id: SESSION, cwd, cli_version: "0.160.0" })), + }, + { + weight: 2, + arbitrary: fc + .tuple(fc.constantFrom(...PLACES), fc.constantFrom("gpt-5.5", "gpt-5.6")) + .map(([cwd, model]) => entry("turn_context", { cwd, model })), + }, + { + weight: 4, + arbitrary: fc + .record({ + sub: fc.boolean(), + at: fc.constantFrom("2026-10-07T23:59:59.000Z", "2026-10-08T12:00:00.000Z", "not a time"), + total: fc.integer({ min: 1, max: 9 }), + }) + .map((c) => (n: number) => call(n, c.sub, c.at, c.total)), + }, + { + weight: 1, + arbitrary: fc.constantFrom( + "{torn", + "", + '{"type":"token_usage_record",torn', + entry("token_usage_record", { session_id: SESSION }), + entry("event_msg", { type: "token_count" }) + ), + } +); + +/** A session file whose calls each have their own response id, as every one observed did. */ +const sessionFile = fc + .array(fc.record({ line, end: fc.constantFrom("\n", "\r\n") }), { minLength: 1, maxLength: 10 }) + .chain((lines) => { + const bytes = Buffer.from( + lines.map(({ line, end }, n) => `${typeof line === "string" ? line : line(n)}${end}`).join("") + ); + return fc.record({ + bytes: fc.constant(bytes), + cuts: fc + .array(fc.nat(bytes.length), { maxLength: 3 }) + .map((cuts) => cuts.sort((a, b) => a - b)), + }); + }); + +describe("ingesting a Codex session file as it grows (P1)", () => { + it("stores the same ledger, offsets and left-out bytes as one ingest of the whole file, and its runs' counts add up to that one's", async () => { + await fc.assert( + fc.asyncProperty(sessionFile, async ({ bytes: content, cuts }) => { + const sessions = join(root, `sessions-${runs}`); + await mkdir(sessions, { recursive: true }); + const path = join(sessions, "rollout-a.jsonl"); + const growing = machine(sessions); + const results: IngestResult[] = []; + for (const cut of [...cuts, content.length]) { + await writeFile(path, content.subarray(0, cut)); + results.push(await growing.ingest.execute()); + } + const once = machine(sessions); + const whole = await once.ingest.execute(); + expect(await bytes(growing.dir)).toEqual(await bytes(once.dir)); + expect(counts(results)).toEqual(counts([whole])); + }), + { numRuns: 80 } + ); + }, 240_000); +}); + +describe.skipIf(typeof zlib.zstdCompressSync !== "function")( + "a session read plain, then compressed (P3)", + () => { + it("stores each of its calls once", async () => { + const sessions = join(root, "sessions"); + const day = join(sessions, "2026", "10", "08"); + await mkdir(day, { recursive: true }); + const content = [ + entry("session_meta", { session_id: SESSION, cwd: "/work/in" }), + entry("turn_context", { cwd: "/work/in", model: "gpt-5.5" }), + call(1, false, "2026-10-08T12:00:00.000Z", 5), + call(2, true, "2026-10-08T12:00:01.000Z", 7), + "", + ].join("\n"); + const { dir, ingest } = machine(sessions); + await writeFile(join(day, "rollout-a.jsonl"), content); + const plain = await ingest.execute(); + const stored = await bytes(dir); + await rm(join(day, "rollout-a.jsonl")); + await writeFile( + join(day, "rollout-a.jsonl.zst"), + zlib.zstdCompressSync(Buffer.from(content)) + ); + const compressed = await ingest.execute(); + expect([plain.added, compressed.added, compressed.updated]).toEqual([2, 0, 0]); + const after = await bytes(dir); + const ledgerOnly = (files: Record) => + Object.fromEntries( + Object.entries(files).filter(([path]) => !path.endsWith("offsets.json")) + ); + expect(ledgerOnly(after)).toEqual(ledgerOnly(stored)); + }); + } +); + +describe("a compressed session file that does not decompress", () => { + it("stops no other measurement, and is named again by the next run", async () => { + const sessions = join(root, "sessions"); + const projects = join(root, "projects", "p"); + await mkdir(sessions, { recursive: true }); + await mkdir(projects, { recursive: true }); + await writeFile( + join(sessions, "rollout-b.jsonl.zst"), + Buffer.from("not a zstd frame, only bytes") + ); + await writeFile( + join(projects, "s.jsonl"), + `${JSON.stringify({ + type: "assistant", + sessionId: "s-1", + requestId: "req_A", + timestamp: "2026-10-08T12:00:00.000Z", + cwd: "/work/in", + message: { + id: "msg_A", + model: "m", + usage: { + input_tokens: 1, + output_tokens: 2, + cache_read_input_tokens: 0, + cache_creation_input_tokens: 0, + }, + }, + })}\n` + ); + const { ingest } = machine(sessions, join(root, "projects")); + const first = await ingest.execute(); + const second = await ingest.execute(); + expect([first.added, first.unrecognised]).toEqual([1, 1]); + expect([second.added, second.unrecognised]).toEqual([0, 1]); + }); +}); diff --git a/cli/tests/contexts/telemetry/application/ingest-usage-use-case.unit.test.ts b/cli/tests/contexts/telemetry/application/ingest-usage-use-case.unit.test.ts index 1bbac04c5..b9938e1b9 100644 --- a/cli/tests/contexts/telemetry/application/ingest-usage-use-case.unit.test.ts +++ b/cli/tests/contexts/telemetry/application/ingest-usage-use-case.unit.test.ts @@ -101,9 +101,9 @@ function setup(options: { refused?: boolean; caseInsensitive?: boolean } = {}) { ); const ingest = new IngestUsageUseCase( [ - new ReadUsageUseCase(transcripts, readClaudeUsageLine), + new ReadUsageUseCase(transcripts, () => readClaudeUsageLine), // Another tool's format: the same lines, its own name on every record. - new ReadUsageUseCase(other, (text) => { + new ReadUsageUseCase(other, () => (text) => { const read = readClaudeUsageLine(text); return { ...read, records: read.records.map((r) => ({ ...r, tool: "codex" as const })) }; }), @@ -128,7 +128,7 @@ function setup(options: { refused?: boolean; caseInsensitive?: boolean } = {}) { const ingestReadingAll = () => new IngestUsageUseCase( [transcripts, other].map((source) => { - const reader = new ReadUsageUseCase(source, readClaudeUsageLine); + const reader = new ReadUsageUseCase(source, () => readClaudeUsageLine); return { execute: (since) => reader.execute(since) } as ReadUsageUseCase; }), ledger, diff --git a/cli/tests/contexts/telemetry/application/legacy/stored-usage-migration.property.integration.test.ts b/cli/tests/contexts/telemetry/application/legacy/stored-usage-migration.property.integration.test.ts index c6c729d0d..4c8984c49 100644 --- a/cli/tests/contexts/telemetry/application/legacy/stored-usage-migration.property.integration.test.ts +++ b/cli/tests/contexts/telemetry/application/legacy/stored-usage-migration.property.integration.test.ts @@ -242,7 +242,7 @@ function machine(name: string, storage: PrivateStorage = new PrivateStorageAdapt } const snapshots = new BindingSnapshotStoreAdapter(bindings, storage, aliases); const ingest = new IngestUsageUseCase( - [new ReadUsageUseCase(transcripts, readClaudeUsageLine)], + [new ReadUsageUseCase(transcripts, () => readClaudeUsageLine)], ledger, new DirectoryResolver(locator, consents, resolutions, history, { caseInsensitiveFileSystem: false, diff --git a/cli/tests/contexts/telemetry/application/read-usage-use-case.integration.test.ts b/cli/tests/contexts/telemetry/application/read-usage-use-case.integration.test.ts index e2537e60a..be6566ccf 100644 --- a/cli/tests/contexts/telemetry/application/read-usage-use-case.integration.test.ts +++ b/cli/tests/contexts/telemetry/application/read-usage-use-case.integration.test.ts @@ -5,8 +5,10 @@ import { join, resolve } from "node:path"; import { afterEach, beforeEach, describe, expect, it, vi } from "vitest"; import { ReadUsageUseCase } from "../../../../src/contexts/telemetry/application/read-usage-use-case.js"; import { readClaudeUsageLine } from "../../../../src/contexts/telemetry/domain/formats/claude-transcript-usage.js"; +import { codexUsageReader } from "../../../../src/contexts/telemetry/domain/formats/codex-transcript-usage.js"; import type { TranscriptSource } from "../../../../src/contexts/telemetry/domain/ports/transcript-source.js"; import { ClaudeTranscriptSourceAdapter } from "../../../../src/contexts/telemetry/infrastructure/claude-transcript-source-adapter.js"; +import { CodexTranscriptSourceAdapter } from "../../../../src/contexts/telemetry/infrastructure/transcripts/codex-transcript-source-adapter.js"; const FIXTURES = resolve(import.meta.dirname, "../../../fixtures/claude-usage/projects"); @@ -36,7 +38,7 @@ describe("reading Claude usage from the fixture transcripts", () => { it("yields exactly the expected records, field for field", async () => { const reading = await new ReadUsageUseCase( new ClaudeTranscriptSourceAdapter(projects), - readClaudeUsageLine + () => readClaudeUsageLine ).execute(new Map()); const summary = reading.records.map((r) => ({ @@ -88,17 +90,20 @@ describe("reading Claude usage from the fixture transcripts", () => { it("gives identical records when the files are read in the opposite order", async () => { const source = new ClaudeTranscriptSourceAdapter(projects); - const forward = await new ReadUsageUseCase(source, readClaudeUsageLine).execute(new Map()); - const backward = await new ReadUsageUseCase(reversed(source), readClaudeUsageLine).execute( + const forward = await new ReadUsageUseCase(source, () => readClaudeUsageLine).execute( new Map() ); + const backward = await new ReadUsageUseCase( + reversed(source), + () => readClaudeUsageLine + ).execute(new Map()); expect(backward.records).toEqual(forward.records); }); it("folds a key across files: the resumed copy never beats the larger original", async () => { const reading = await new ReadUsageUseCase( new ClaudeTranscriptSourceAdapter(projects), - readClaudeUsageLine + () => readClaudeUsageLine ).execute(new Map()); const a = reading.records.find((r) => r.key === "msg_A:req_A"); expect([a?.session_id, a?.output]).toEqual(["s-1", 250]); @@ -112,7 +117,7 @@ describe("reading Claude usage from the fixture transcripts", () => { ); const reading = await new ReadUsageUseCase( new ClaudeTranscriptSourceAdapter(projects), - readClaudeUsageLine + () => readClaudeUsageLine ).execute(new Map()); expect(reading.unrecognised).toHaveLength(1); expect(reading.records).toHaveLength(5); @@ -131,7 +136,7 @@ describe("reading Claude usage from the fixture transcripts", () => { ); const reading = await new ReadUsageUseCase( new ClaudeTranscriptSourceAdapter(projects), - readClaudeUsageLine + () => readClaudeUsageLine ).execute(new Map()); expect(reading.unrecognised).toHaveLength(3); expect(reading.dropped).toEqual([ @@ -149,14 +154,14 @@ describe("reading Claude usage from the fixture transcripts", () => { await appendFile(file, "{broken\n"); const reading = await new ReadUsageUseCase( new ClaudeTranscriptSourceAdapter(projects), - readClaudeUsageLine + () => readClaudeUsageLine ).execute(new Map()); expect(reading.unrecognised).toEqual([`${file}: a complete line is not JSON`]); }); it("continues from the positions it returned and reads only what was appended", async () => { const source = new ClaudeTranscriptSourceAdapter(projects); - const useCase = new ReadUsageUseCase(source, readClaudeUsageLine); + const useCase = new ReadUsageUseCase(source, () => readClaudeUsageLine); const first = await useCase.execute(new Map()); await mkdir(join(projects, "proj-b"), { recursive: true }); await writeFile( @@ -193,7 +198,7 @@ describe("reading a long transcript", () => { await probe.close(); let readsBeforeFirstLine: number | null = null; let lines = 0; - await new ReadUsageUseCase(new ClaudeTranscriptSourceAdapter(alone), () => { + await new ReadUsageUseCase(new ClaudeTranscriptSourceAdapter(alone), () => () => { readsBeforeFirstLine ??= reads.mock.calls.length; lines += 1; return { records: [], unrecognised: [] }; @@ -230,7 +235,7 @@ describe("reading a transcript that changes under the read", () => { let seen = 0; const reading = await new ReadUsageUseCase( new ClaudeTranscriptSourceAdapter(alone, 16), - (line) => { + () => (line) => { if (++seen === 2) truncateSync(path, 0); return readClaudeUsageLine(line); } @@ -242,3 +247,63 @@ describe("reading a transcript that changes under the read", () => { expect(reading.positions.has(path)).toBe(false); }); }); + +describe("reading Codex usage", () => { + const SESSION = "33333333-3333-4333-8333-333333333333"; + const entry = (type: string, payload: Record) => + `${JSON.stringify({ timestamp: "2026-10-08T12:00:00.000Z", type, payload })}\n`; + const meta = entry("session_meta", { + session_id: SESSION, + cwd: "/work/meta", + cli_version: "0.160.0", + }); + const turn = (model: string, cwd: string) => entry("turn_context", { model, cwd }); + const call = (id: string) => + entry("token_usage_record", { + response_id: id, + session_id: SESSION, + thread_id: SESSION, + usage: { + input_tokens: 10, + cached_input_tokens: 4, + cache_write_input_tokens: 0, + output_tokens: 2, + reasoning_output_tokens: 1, + total_tokens: 12, + }, + }); + + it("gives a call written after a resumed read the directory and model written before it", async () => { + const sessions = join(root, "codex", "sessions"); + await mkdir(sessions, { recursive: true }); + const path = join(sessions, "rollout-a.jsonl"); + await writeFile(path, meta + turn("gpt-5.5", "/work/one") + call("resp_1")); + const useCase = new ReadUsageUseCase( + new CodexTranscriptSourceAdapter([sessions]), + codexUsageReader + ); + const first = await useCase.execute(new Map()); + await appendFile(path, call("resp_2") + turn("gpt-5.6", "/work/two") + call("resp_3")); + const second = await useCase.execute(first.positions); + expect(second.records.map((r) => [r.key, r.model, r.cwd])).toEqual([ + ["resp_2", "gpt-5.5", "/work/one"], + ["resp_3", "gpt-5.6", "/work/two"], + ]); + expect(second.unrecognised).toEqual([]); + }); + + it("names each compressed file a Node with no zstd cannot read, with its file", async () => { + const sessions = join(root, "codex", "sessions"); + await mkdir(sessions, { recursive: true }); + const path = join(sessions, "rollout-a.jsonl.zst"); + await writeFile(path, ""); + const reading = await new ReadUsageUseCase( + new CodexTranscriptSourceAdapter([sessions], null), + codexUsageReader + ).execute(new Map()); + expect(reading.unrecognised).toEqual([ + `${path}: a compressed session file is not read: this Node has no zstd`, + ]); + expect(reading.records).toEqual([]); + }); +}); diff --git a/cli/tests/contexts/telemetry/application/report/left-out.property.integration.test.ts b/cli/tests/contexts/telemetry/application/report/left-out.property.integration.test.ts index 27d0d7985..f2baa7588 100644 --- a/cli/tests/contexts/telemetry/application/report/left-out.property.integration.test.ts +++ b/cli/tests/contexts/telemetry/application/report/left-out.property.integration.test.ts @@ -194,7 +194,7 @@ function machine() { () => NOW ); const ingest = new IngestUsageUseCase( - [new ReadUsageUseCase(transcripts, readClaudeUsageLine)], + [new ReadUsageUseCase(transcripts, () => readClaudeUsageLine)], ledger, new DirectoryResolver(locator, consents, new InMemoryResolutions(), history, { caseInsensitiveFileSystem: false, diff --git a/cli/tests/contexts/telemetry/application/report/report-figures.property.integration.test.ts b/cli/tests/contexts/telemetry/application/report/report-figures.property.integration.test.ts index c406fa6c5..5c9b92908 100644 --- a/cli/tests/contexts/telemetry/application/report/report-figures.property.integration.test.ts +++ b/cli/tests/contexts/telemetry/application/report/report-figures.property.integration.test.ts @@ -168,7 +168,7 @@ function recordsOf(calls: Scenario["calls"]): StoredUsage[] { return calls.flatMap((c, index) => c.snapshots.map( (s): StoredUsage => ({ - tool: "claude-code", + tool: index % 2 === 0 ? "claude-code" : "codex", tool_version: "1.0.0", key: `k${index}`, session_id: c.session, diff --git a/cli/tests/contexts/telemetry/application/report/report-filters.property.integration.test.ts b/cli/tests/contexts/telemetry/application/report/report-filters.property.integration.test.ts index 83f7a9204..5bba11a13 100644 --- a/cli/tests/contexts/telemetry/application/report/report-filters.property.integration.test.ts +++ b/cli/tests/contexts/telemetry/application/report/report-filters.property.integration.test.ts @@ -88,7 +88,7 @@ async function reporterOf({ calls, declarations, person }: Ledger) { const ledger = new InMemoryLedger(); ledger.records = calls.map( (c, index): StoredUsage => ({ - tool: "claude-code", + tool: index % 2 === 0 ? "claude-code" : "codex", tool_version: null, key: `k${index}`, session_id: c.session, diff --git a/cli/tests/contexts/telemetry/application/report/report-usage-use-case.unit.test.ts b/cli/tests/contexts/telemetry/application/report/report-usage-use-case.unit.test.ts index 4d2734e93..e800a22ce 100644 --- a/cli/tests/contexts/telemetry/application/report/report-usage-use-case.unit.test.ts +++ b/cli/tests/contexts/telemetry/application/report/report-usage-use-case.unit.test.ts @@ -75,7 +75,7 @@ function setup(options: { refused?: boolean } = {}) { const sessions = new InMemorySessions(events); const identity = new InMemoryIdentity(); const ingest = new IngestUsageUseCase( - [new ReadUsageUseCase(transcripts, readClaudeUsageLine)], + [new ReadUsageUseCase(transcripts, () => readClaudeUsageLine)], ledger, new DirectoryResolver(locator, consents, new InMemoryResolutions(), history, { caseInsensitiveFileSystem: false, diff --git a/cli/tests/contexts/telemetry/application/transcript-stream-ingest.property.integration.test.ts b/cli/tests/contexts/telemetry/application/transcript-stream-ingest.property.integration.test.ts index 4275ffdb8..29d4c5e3f 100644 --- a/cli/tests/contexts/telemetry/application/transcript-stream-ingest.property.integration.test.ts +++ b/cli/tests/contexts/telemetry/application/transcript-stream-ingest.property.integration.test.ts @@ -133,7 +133,7 @@ function machine(source: TranscriptSource) { } locator.directories.set("/work/none", { status: "outside-repository" }); const ingest = new IngestUsageUseCase( - [new ReadUsageUseCase(source, readClaudeUsageLine)], + [new ReadUsageUseCase(source, () => readClaudeUsageLine)], ledger, new DirectoryResolver(locator, consents, new InMemoryResolutions(), history, { caseInsensitiveFileSystem: false, diff --git a/cli/tests/contexts/telemetry/domain/formats/codex-transcript-usage.unit.test.ts b/cli/tests/contexts/telemetry/domain/formats/codex-transcript-usage.unit.test.ts new file mode 100644 index 000000000..9914a0af8 --- /dev/null +++ b/cli/tests/contexts/telemetry/domain/formats/codex-transcript-usage.unit.test.ts @@ -0,0 +1,303 @@ +import * as fc from "fast-check"; +import { describe, expect, it } from "vitest"; +import { codexUsageReader } from "../../../../../src/contexts/telemetry/domain/formats/codex-transcript-usage.js"; +import type { UsageRecord } from "../../../../../src/contexts/telemetry/domain/usage-record.js"; + +const SESSION = "11111111-1111-4111-8111-111111111111"; +const THREAD = "22222222-2222-4222-8222-222222222222"; + +const USAGE = { + input_tokens: 1200, + cached_input_tokens: 1000, + cache_write_input_tokens: 0, + output_tokens: 90, + reasoning_output_tokens: 40, + total_tokens: 1290, +}; + +function meta(payload: Record = {}): string { + return JSON.stringify({ + timestamp: "2026-10-07T09:59:00.000Z", + type: "session_meta", + payload: { + id: SESSION, + session_id: SESSION, + cwd: "/work/meta", + cli_version: "0.160.0", + git: { branch: "feat/a" }, + ...payload, + }, + }); +} + +function turn(payload: Record = {}): string { + return JSON.stringify({ + timestamp: "2026-10-07T09:59:30.000Z", + type: "turn_context", + payload: { turn_id: "t-1", cwd: "/work/turn", model: "gpt-5.5", ...payload }, + }); +} + +function call(payload: Record = {}, root: Record = {}): string { + return JSON.stringify({ + timestamp: "2026-10-07T10:00:01.000Z", + type: "token_usage_record", + payload: { + thread_id: SESSION, + session_id: SESSION, + turn_id: "t-1", + response_id: "resp_a", + usage: USAGE, + turn_token_usage: { ...USAGE, input_tokens: 99999, output_tokens: 99999 }, + thread_token_usage: { ...USAGE, input_tokens: 88888, output_tokens: 88888 }, + ...payload, + }, + ...root, + }); +} + +/** What a file read in order gives, line by line. */ +function readAll(lines: readonly string[]) { + const read = codexUsageReader(); + return lines.map((line) => read(line)); +} + +function only(lines: readonly string[]): UsageRecord { + const records = readAll(lines).flatMap((outcome) => outcome.records); + expect(records).toHaveLength(1); + return records[0] as UsageRecord; +} + +describe("a Codex session file to usage records", () => { + it("maps every contract field of a call", () => { + expect(only([meta(), turn(), call()])).toEqual({ + tool: "codex", + tool_version: "0.160.0", + key: "resp_a", + session_id: SESSION, + agent: "main", + at: "2026-10-07T10:00:01.000Z", + model: "gpt-5.5", + input: 200, + output: 90, + cache_read: 1000, + cache_write: 0, + cache_write_1h: null, + reasoning: 40, + cwd: "/work/turn", + git_branch: "feat/a", + }); + }); + + it("takes the cache write out of the input, as the cached read", () => { + const record = only([ + meta(), + turn(), + call({ usage: { ...USAGE, cached_input_tokens: 600, cache_write_input_tokens: 300 } }), + ]); + expect([record.input, record.cache_read, record.cache_write]).toEqual([300, 600, 300]); + }); + + it("is a sub-agent's call when its thread is not its session", () => { + expect(only([meta(), turn(), call({ thread_id: THREAD })]).agent).toBe("subagent"); + }); + + it("keeps the root session on a sub-agent's call", () => { + expect(only([meta(), turn(), call({ thread_id: THREAD })]).session_id).toBe(SESSION); + }); + + it("takes the version, branch and fallback directory from the file's first session_meta", () => { + const record = only([ + meta(), + meta({ cli_version: "0.149.0", git: { branch: "parent" }, cwd: "/work/parent" }), + turn({ cwd: undefined }), + call(), + ]); + expect([record.tool_version, record.git_branch, record.cwd]).toEqual([ + "0.160.0", + "feat/a", + "/work/meta", + ]); + }); + + it("takes the file's own session_meta, even after a turn_context", () => { + const record = only([turn({ cwd: undefined }), meta(), call()]); + expect([record.cwd, record.tool_version]).toEqual(["/work/meta", "0.160.0"]); + }); + + it("takes the model and directory of the latest turn_context before the call", () => { + const records = readAll([ + meta(), + turn({ model: "gpt-5.5", cwd: "/work/one" }), + call({ response_id: "resp_1" }), + turn({ model: "gpt-5.6", cwd: "/work/two" }), + call({ response_id: "resp_2" }), + ]).flatMap((outcome) => outcome.records); + expect(records.map((r) => [r.key, r.model, r.cwd])).toEqual([ + ["resp_1", "gpt-5.5", "/work/one"], + ["resp_2", "gpt-5.6", "/work/two"], + ]); + }); + + it("has no model, and the session's directory, before any turn_context", () => { + const record = only([meta(), call()]); + expect([record.model, record.cwd]).toEqual([null, "/work/meta"]); + }); + + it("has no version, branch or directory with no session_meta and no turn_context", () => { + const record = only([call()]); + expect([record.tool_version, record.git_branch, record.cwd, record.model]).toEqual([ + null, + null, + null, + null, + ]); + }); + + it("starts each file read with no context", () => { + const read = codexUsageReader(); + read(meta()); + read(turn()); + const fresh = codexUsageReader()(call()); + expect(fresh.records[0]?.cwd).toBeNull(); + }); + + it("reads only the call's own usage, never the running totals", () => { + const record = only([meta(), turn(), call()]); + expect([record.input, record.output]).toEqual([200, 90]); + }); + + it("gives nothing for a running total line", () => { + const tokenCount = JSON.stringify({ + timestamp: "2026-10-07T10:00:02.000Z", + type: "event_msg", + payload: { type: "token_count", info: { total_token_usage: USAGE } }, + }); + expect(readAll([meta(), turn(), tokenCount]).at(-1)).toEqual({ + records: [], + unrecognised: [], + }); + }); + + it("gives nothing, and names nothing, for every line that is no call", () => { + for (const outcome of readAll([meta(), turn(), "", "[]", "7", '{"type":"response_item"}'])) { + expect(outcome).toEqual({ records: [], unrecognised: [] }); + } + }); + + it("gives nothing, and names nothing, for a compacted line holding a copy of the latest call", () => { + const compacted = JSON.stringify({ + timestamp: "2026-10-07T10:00:03.000Z", + type: "compacted", + payload: { + message: "summary", + latest_token_usage_record: JSON.parse(call()).payload, + }, + }); + expect(readAll([meta(), turn(), compacted]).at(-1)).toEqual({ records: [], unrecognised: [] }); + }); + + it("gives nothing for a line that is not JSON and holds no call", () => { + expect(readAll(["{torn"])[0]).toEqual({ records: [], unrecognised: [] }); + }); + + it("names a line that is not JSON and holds token_usage_record, with its context's directory", () => { + const outcome = readAll([meta(), turn(), '{"type":"token_usage_record",torn'])[2]; + expect(outcome).toEqual({ + records: [], + unrecognised: ["a line holding token_usage_record is not JSON"], + named: { cwd: "/work/turn", at: null }, + }); + }); + + it.each([ + ["response_id", { response_id: undefined }, {}], + ["session_id", { session_id: "" }, {}], + ["thread_id", { thread_id: 7 }, {}], + ["usage", { usage: 7 }, {}], + ["usage", { usage: undefined }, {}], + ["timestamp", {}, { timestamp: undefined }], + ])("names a call that lacks its %s and makes no record of it", (_field, payload, root) => { + const outcome = readAll([meta(), turn(), call(payload, root)])[2]; + expect(outcome).toEqual({ + records: [], + unrecognised: [ + "a token_usage_record lacks response_id, session_id, thread_id, timestamp or usage", + ], + named: { + cwd: "/work/turn", + at: "timestamp" in root ? null : "2026-10-07T10:00:01.000Z", + }, + }); + }); + + it("names a call with no payload, with the session's directory", () => { + const line = JSON.stringify({ + timestamp: "2026-10-07T10:00:01.000Z", + type: "token_usage_record", + }); + expect(readAll([meta(), line])[1]?.named).toEqual({ + cwd: "/work/meta", + at: "2026-10-07T10:00:01.000Z", + }); + }); + + it.each([ + "input_tokens", + "cached_input_tokens", + "cache_write_input_tokens", + "output_tokens", + "reasoning_output_tokens", + ])("keeps a call whose %s is missing, that counter unknown and said so", (field) => { + const outcome = readAll([meta(), turn(), call({ usage: { ...USAGE, [field]: -1 } })])[2]; + expect(outcome?.records).toHaveLength(1); + expect(outcome?.unrecognised).toEqual([`usage.${field} is missing or not a count`]); + expect(outcome?.named).toBeUndefined(); + }); + + it.each([ + ["input_tokens", { input: null, cache_read: 1000, cache_write: 0 }], + ["cached_input_tokens", { input: null, cache_read: null, cache_write: 0 }], + ["cache_write_input_tokens", { input: null, cache_read: 1000, cache_write: null }], + ])("has no input when %s is not a count", (field, expected) => { + const record = only([meta(), turn(), call({ usage: { ...USAGE, [field]: 1.5 } })]); + expect({ + input: record.input, + cache_read: record.cache_read, + cache_write: record.cache_write, + }).toEqual(expected); + }); + + it.each([ + ["output_tokens", "output"], + ["reasoning_output_tokens", "reasoning"], + ] as const)("has no %s when it is not a count", (field, key) => { + expect(only([meta(), turn(), call({ usage: { ...USAGE, [field]: "9" } })])[key]).toBeNull(); + }); + + it("adds up to the call's total_tokens, whatever its counters (P2)", () => { + fc.assert( + fc.property( + fc.nat(10_000), + fc.nat(10_000), + fc.nat(10_000), + fc.nat(10_000), + (cached, written, rest, output) => { + const input = cached + written + rest; + const usage = { + input_tokens: input, + cached_input_tokens: cached, + cache_write_input_tokens: written, + output_tokens: output, + reasoning_output_tokens: 0, + total_tokens: input + output, + }; + const r = only([meta(), turn(), call({ usage })]); + expect( + (r.input ?? 0) + (r.output ?? 0) + (r.cache_read ?? 0) + (r.cache_write ?? 0) + ).toBe(usage.total_tokens); + } + ) + ); + }); +}); diff --git a/cli/tests/contexts/telemetry/domain/report/usage-report.property.unit.test.ts b/cli/tests/contexts/telemetry/domain/report/usage-report.property.unit.test.ts index dc7ac1760..ee309f677 100644 --- a/cli/tests/contexts/telemetry/domain/report/usage-report.property.unit.test.ts +++ b/cli/tests/contexts/telemetry/domain/report/usage-report.property.unit.test.ts @@ -32,6 +32,7 @@ const usage = fc session_id: fc.constantFrom(...SESSIONS), repository_id: fc.constantFrom(...REPOS), git_branch: fc.option(fc.constantFrom(...BRANCHES), { nil: null }), + tool: fc.constantFrom("claude-code" as const, "codex" as const), model: fc.option(fc.constantFrom("m-1", "m-2"), { nil: null }), at: instant, input: counter, @@ -42,7 +43,6 @@ const usage = fc .map( (r): StoredUsage => ({ ...r, - tool: "claude-code", tool_version: null, agent: "main", cache_write_1h: null, diff --git a/cli/tests/contexts/telemetry/infrastructure/transcripts/codex-transcript-source-adapter.integration.test.ts b/cli/tests/contexts/telemetry/infrastructure/transcripts/codex-transcript-source-adapter.integration.test.ts new file mode 100644 index 000000000..a7980a802 --- /dev/null +++ b/cli/tests/contexts/telemetry/infrastructure/transcripts/codex-transcript-source-adapter.integration.test.ts @@ -0,0 +1,321 @@ +import { appendFile, chmod, mkdir, mkdtemp, rename, rm, utimes, writeFile } from "node:fs/promises"; +import { tmpdir } from "node:os"; +import { join } from "node:path"; +import { PassThrough, Transform } from "node:stream"; +import * as zlib from "node:zlib"; +import { afterEach, beforeEach, describe, expect, it } from "vitest"; +import type { TranscriptPosition } from "../../../../../src/contexts/telemetry/domain/transcript-position.js"; +import { + CodexTranscriptSourceAdapter, + codexSessionRoots, +} from "../../../../../src/contexts/telemetry/infrastructure/transcripts/codex-transcript-source-adapter.js"; + +const EVERYTHING = Number.NEGATIVE_INFINITY; + +let root: string; +let sessions: string; +let archived: string; + +beforeEach(async () => { + root = await mkdtemp(join(tmpdir(), "aidd-codex-source-")); + sessions = join(root, "sessions"); + archived = join(root, "archived_sessions"); + await mkdir(join(sessions, "2026", "10", "07"), { recursive: true }); + await mkdir(archived, { recursive: true }); +}); + +afterEach(async () => { + await rm(root, { recursive: true, force: true }); +}); + +function source(blockBytes?: number) { + return new CodexTranscriptSourceAdapter([sessions, archived], undefined, blockBytes); +} + +async function readLines( + adapter: CodexTranscriptSourceAdapter, + path: string, + since: TranscriptPosition | null, + writtenFrom = EVERYTHING +) { + const lines: [string, boolean][] = []; + const read = await adapter.read(path, since, writtenFrom, (line, counted) => { + lines.push([line, counted]); + }); + return { lines, read }; +} + +describe("where Codex keeps its session files", () => { + it("is sessions/ and archived_sessions/ under CODEX_HOME when it is set", () => { + expect(codexSessionRoots("/cfg/codex", "/home/p")).toEqual([ + join("/cfg/codex", "sessions"), + join("/cfg/codex", "archived_sessions"), + ]); + }); + + it("is under .codex in the home directory otherwise, an empty CODEX_HOME being none", () => { + for (const unset of [undefined, ""]) { + expect(codexSessionRoots(unset, "/home/p")).toEqual([ + join("/home/p", ".codex", "sessions"), + join("/home/p", ".codex", "archived_sessions"), + ]); + } + }); +}); + +describe("listing Codex session files", () => { + it("lists every rollout file, plain or compressed, at any depth, under both roots", async () => { + const day = join(sessions, "2026", "10", "07"); + const names = [ + "rollout-2026-10-07T10-00-00-aaaa.jsonl", + "rollout-2026-10-07T11-00-00-bbbb.jsonl.zst", + "notes.jsonl", + "old-rollout-c.jsonl", + "rollout-d.jsonl.bak", + "rollout-e.json", + ]; + for (const name of names) await writeFile(join(day, name), ""); + await writeFile(join(archived, "rollout-f.jsonl.zst"), ""); + expect(await source().list()).toEqual( + [ + join(day, "rollout-2026-10-07T10-00-00-aaaa.jsonl"), + join(day, "rollout-2026-10-07T11-00-00-bbbb.jsonl.zst"), + join(archived, "rollout-f.jsonl.zst"), + ].sort() + ); + }); + + it("lists nothing when no root exists", async () => { + await rm(root, { recursive: true, force: true }); + expect(await source().list()).toEqual([]); + }); + + it("says when the oldest of its files was last written", async () => { + const [older, newer] = [ + join(sessions, "rollout-a.jsonl"), + join(archived, "rollout-b.jsonl.zst"), + ]; + for (const [path, at] of [ + [older, "2026-09-01T00:00:00.000Z"], + [newer, "2026-10-01T00:00:00.000Z"], + ] as const) { + await writeFile(path, ""); + await utimes(path, new Date(at), new Date(at)); + } + expect(await source().oldestModified([newer, older])).toBe("2026-09-01T00:00:00.000Z"); + }); +}); + +describe("reading a plain Codex session file", () => { + it("hands every line of a new file, each counted", async () => { + const path = join(sessions, "rollout-a.jsonl"); + await writeFile(path, "a\nb\npart"); + const { lines, read } = await readLines(source(), path, null); + expect(lines).toEqual([ + ["a", true], + ["b", true], + ]); + expect(read.position?.offset).toBe(4); + }); + + it("reads a grown file from its start, counting only the lines past the saved offset", async () => { + const path = join(sessions, "rollout-a.jsonl"); + await writeFile(path, "a\nb\n"); + const adapter = source(3); + const first = await readLines(adapter, path, null); + await appendFile(path, "c\nd\n"); + const { lines, read } = await readLines(adapter, path, first.read.position); + expect(lines).toEqual([ + ["a", false], + ["b", false], + ["c", true], + ["d", true], + ]); + expect(read).toMatchObject({ position: { offset: 8, size: 8 }, restarted: false }); + }); + + it("hands no line of a file that did not move", async () => { + const path = join(sessions, "rollout-a.jsonl"); + await writeFile(path, "a\nb\npart"); + const adapter = source(); + const first = await readLines(adapter, path, null); + const again = await readLines(adapter, path, first.read.position); + expect(again.lines).toEqual([]); + expect(again.read.position).toEqual(first.read.position); + }); + + it("counts every line of a file that is no longer the one the position describes", async () => { + const path = join(sessions, "rollout-a.jsonl"); + await writeFile(path, "a\n"); + const adapter = source(); + const first = await readLines(adapter, path, null); + await writeFile(join(sessions, "other"), "x\ny\nz\n"); + await rename(join(sessions, "other"), path); + const { lines, read } = await readLines(adapter, path, first.read.position); + expect(lines.map(([, counted]) => counted)).toEqual([true, true, true]); + expect(read.restarted).toBe(true); + }); + + it("does not read a file last written before the given instant, and keeps its end", async () => { + const path = join(sessions, "rollout-a.jsonl"); + await writeFile(path, "a\nb\npart"); + await utimes(path, new Date("2026-01-01"), new Date("2026-01-01")); + const { lines, read } = await readLines(source(), path, null, Date.parse("2026-06-01")); + expect(lines).toEqual([]); + expect(read).toMatchObject({ skipped: true, position: { offset: 4 } }); + }); +}); + +/** A Node from before zstd still loads this file, and skips what needs zstd to write a file. */ +const NO_ZSTD = typeof zlib.zstdCompressSync !== "function"; + +describe.skipIf(NO_ZSTD)("reading a compressed Codex session file", () => { + const compressed = (text: string) => zlib.zstdCompressSync(Buffer.from(text)); + + it("hands every line, each counted, and keeps the file read by its size and identity", async () => { + const path = join(archived, "rollout-a.jsonl.zst"); + const bytes = compressed("a\nb\nc\n"); + await writeFile(path, bytes); + const adapter = source(); + const { lines, read } = await readLines(adapter, path, null); + expect(lines).toEqual([ + ["a", true], + ["b", true], + ["c", true], + ]); + expect(read).toMatchObject({ + position: { offset: bytes.length, size: bytes.length }, + skipped: false, + }); + const again = await readLines(adapter, path, read.position); + expect(again.lines).toEqual([]); + expect(again.read.position).toEqual(read.position); + }); + + it("reads it whole again once it is another file", async () => { + const path = join(archived, "rollout-a.jsonl.zst"); + await writeFile(path, compressed("a\n")); + const adapter = source(); + const first = await readLines(adapter, path, null); + await writeFile(join(archived, "other"), compressed("a\nb\n")); + await rename(join(archived, "other"), path); + const { lines } = await readLines(adapter, path, first.read.position); + expect(lines).toEqual([ + ["a", true], + ["b", true], + ]); + }); + + it("does not decompress one last written before the given instant", async () => { + const path = join(archived, "rollout-a.jsonl.zst"); + const bytes = compressed("a\n"); + await writeFile(path, bytes); + await utimes(path, new Date("2026-01-01"), new Date("2026-01-01")); + const { lines, read } = await readLines(source(), path, null, Date.parse("2026-06-01")); + expect(lines).toEqual([]); + expect(read).toMatchObject({ skipped: true, position: { offset: bytes.length } }); + }); + + it("names a stream cut short and reads it again next time, whichever of zlib or the frame's size says so", async () => { + // Node 24's zlib fails a cut frame with Z_BUF_ERROR; Node 22's ends it quietly, short. + const path = join(archived, "rollout-a.jsonl.zst"); + await writeFile(path, compressed("a\nb\n").subarray(0, 8)); + const adapter = source(); + const first = await readLines(adapter, path, null); + expect(first.read.position).toEqual({ offset: 0, size: 0, identity: "" }); + expect(first.read.unrecognised).toEqual([ + expect.stringMatching( + /^a compressed session file does not decompress: (Z_BUF_ERROR|\d+ of the 4 bytes its frame declares)$/ + ), + ]); + expect((await readLines(adapter, path, first.read.position)).read).toEqual(first.read); + }); + + it("reads whole, unchecked, a stream whose frame declares no size", async () => { + const path = join(archived, "rollout-a.jsonl.zst"); + await writeFile(path, "a\nb\n"); + const passesThrough = () => new PassThrough(); + const { lines, read } = await readLines( + new CodexTranscriptSourceAdapter([sessions, archived], passesThrough), + path, + null + ); + expect(lines).toEqual([ + ["a", true], + ["b", true], + ]); + expect(read.unrecognised).toBeUndefined(); + }); + + it("names a stream that ends short of the size its frame declares", async () => { + const path = join(archived, "rollout-a.jsonl.zst"); + await writeFile(path, compressed("a\nb\n")); + // A decompressor that ends quietly one byte short, as Node 22's does on a cut frame. + const dropsLastByte = () => { + const input: Buffer[] = []; + return new Transform({ + transform(chunk: Buffer, _encoding, done) { + input.push(chunk); + done(); + }, + flush(done) { + done(null, zlib.zstdDecompressSync(Buffer.concat(input)).subarray(0, -1)); + }, + }); + }; + const { read } = await readLines( + new CodexTranscriptSourceAdapter([sessions, archived], dropsLastByte), + path, + null + ); + expect(read).toMatchObject({ + position: { offset: 0, size: 0, identity: "" }, + unrecognised: [ + "a compressed session file does not decompress: 3 of the 4 bytes its frame declares", + ], + }); + }); + + it("is one unrecognised shape, naming why, when it does not decompress, and is read again next time", async () => { + const path = join(archived, "rollout-a.jsonl.zst"); + await writeFile(path, Buffer.from("not a zstd frame, only bytes")); + const adapter = source(); + const first = await readLines(adapter, path, null); + const expected = { + position: { offset: 0, size: 0, identity: "" }, + restarted: false, + skipped: false, + unrecognised: ["a compressed session file does not decompress: ZSTD_error_prefix_unknown"], + }; + expect(first.read).toEqual(expected); + expect((await readLines(adapter, path, first.read.position)).read).toEqual(expected); + }); +}); + +describe("a compressed file that cannot be opened", () => { + it.skipIf(process.platform === "win32" || process.getuid?.() === 0)( + "fails the read: only a stream that does not decompress is named and passed over", + async () => { + const path = join(archived, "rollout-a.jsonl.zst"); + await writeFile(path, ""); + await chmod(path, 0o000); + await expect(readLines(source(), path, null)).rejects.toMatchObject({ code: "EACCES" }); + } + ); +}); + +describe("a Node with no zstd", () => { + it("is one unrecognised shape, naming why, where Node has no zstd, and stays unread", async () => { + const path = join(archived, "rollout-a.jsonl.zst"); + await writeFile(path, "not read"); + const adapter = new CodexTranscriptSourceAdapter([sessions, archived], null); + const { lines, read } = await readLines(adapter, path, null); + expect(lines).toEqual([]); + expect(read).toEqual({ + position: { offset: 0, size: 0, identity: "" }, + restarted: false, + skipped: false, + unrecognised: ["a compressed session file is not read: this Node has no zstd"], + }); + }); +}); diff --git a/cli/tests/contexts/telemetry/infrastructure/transcripts/zstd-frame.unit.test.ts b/cli/tests/contexts/telemetry/infrastructure/transcripts/zstd-frame.unit.test.ts new file mode 100644 index 000000000..8b05adae2 --- /dev/null +++ b/cli/tests/contexts/telemetry/infrastructure/transcripts/zstd-frame.unit.test.ts @@ -0,0 +1,50 @@ +import { describe, expect, it } from "vitest"; +import { declaredContentSize } from "../../../../../src/contexts/telemetry/infrastructure/transcripts/zstd-frame.js"; + +const MAGIC = [0x28, 0xb5, 0x2f, 0xfd]; + +/** A frame header: the magic, the descriptor byte, then the fields it announces. */ +function header(descriptor: number, ...fields: number[]): Buffer { + return Buffer.from([...MAGIC, descriptor, ...fields]); +} + +describe("the content size a zstd frame declares", () => { + it("reads a one-byte size, on a single-segment frame with no window byte", () => { + expect(declaredContentSize(header(0b0010_0000, 13))).toBe(13); + }); + + it("reads a two-byte size, offset by 256, after the window byte", () => { + expect(declaredContentSize(header(0b0100_0000, 0x58, 0x01, 0x02))).toBe(0x0201 + 256); + }); + + it("reads a two-byte size on a single segment, where the flag alone gives its width", () => { + expect(declaredContentSize(header(0b0110_0000, 0x01, 0x02))).toBe(0x0201 + 256); + }); + + it("reads a four-byte size", () => { + expect(declaredContentSize(header(0b1000_0000, 0x58, 0x04, 0x03, 0x02, 0x01))).toBe(0x01020304); + }); + + it("reads an eight-byte size", () => { + const size = [0x06, 0x05, 0x04, 0x03, 0x02, 0x01, 0x00, 0x00]; + expect(declaredContentSize(header(0b1100_0000, 0x58, ...size))).toBe(0x010203040506); + }); + + it.each([ + [1, [0xaa]], + [2, [0xaa, 0xbb]], + [3, [0xaa, 0xbb, 0xcc, 0xdd]], + ])("skips a dictionary id of flag %s", (flag, id) => { + expect(declaredContentSize(header(0b0010_0000 | flag, ...id, 42))).toBe(42); + }); + + it("is unknown when the frame declares no size", () => { + expect(declaredContentSize(header(0b0000_0000, 0x58, 0x07))).toBeNull(); + }); + + it("is unknown for bytes that are no zstd frame, or too few to hold the size", () => { + expect(declaredContentSize(Buffer.from("not a frame"))).toBeNull(); + expect(declaredContentSize(header(0b1000_0000, 0x58, 0x04))).toBeNull(); + expect(declaredContentSize(Buffer.from(MAGIC))).toBeNull(); + }); +}); diff --git a/cli/tests/e2e/helpers.ts b/cli/tests/e2e/helpers.ts index 01fdd7741..b44dc993c 100644 --- a/cli/tests/e2e/helpers.ts +++ b/cli/tests/e2e/helpers.ts @@ -259,6 +259,8 @@ export function sandboxedEnv( // The check asks GitHub for the latest release and prints a notice from the answer, so // captured stderr would change with every publish. AIDD_SKIP_UPDATE_CHECK: "1", + // Codex reads `CODEX_HOME` before `HOME`, and so does the telemetry reader of its sessions. + CODEX_HOME: join(fakeHome, ".codex"), ...extra, HOME: fakeHome, XDG_CONFIG_HOME: join(fakeHome, ".config"), diff --git a/cli/tests/e2e/sandbox-reaches-no-tool-binary.e2e.test.ts b/cli/tests/e2e/sandbox-reaches-no-tool-binary.e2e.test.ts index 35bab5a7a..f2c7ae6af 100644 --- a/cli/tests/e2e/sandbox-reaches-no-tool-binary.e2e.test.ts +++ b/cli/tests/e2e/sandbox-reaches-no-tool-binary.e2e.test.ts @@ -99,4 +99,16 @@ describe("E2E: the sandbox a test spawns into", () => { await cleanup(); } }); + + it("points Codex's home inside the sandbox, even when the runner sets its own", () => { + const set = process.env.CODEX_HOME; + process.env.CODEX_HOME = "/the/runner/own/codex"; + try { + const home = join("/sandbox", "home"); + expect(sandboxedEnv(home).CODEX_HOME).toBe(join(home, ".codex")); + } finally { + if (set === undefined) delete process.env.CODEX_HOME; + else process.env.CODEX_HOME = set; + } + }); }); diff --git a/cli/tests/golden/snapshots/help/surface.json b/cli/tests/golden/snapshots/help/surface.json index b32664730..bbd228da5 100644 --- a/cli/tests/golden/snapshots/help/surface.json +++ b/cli/tests/golden/snapshots/help/surface.json @@ -2,7 +2,7 @@ { "invocation": "aidd", "exitCode": 0, - "help": "Usage: aidd [options] [command]\n\nGenerate AI coding assistant configurations from the AIDD framework\n\nOptions:\n -V, --version Show version number\n --verbose Show detailed diagnostic output (default: false)\n -h, --help display help for command\n\nCommands:\n setup [options] Set up or update the project to a correct state\n — bootstraps the whole project (marketplace,\n framework, tools, plugins); see `framework\n install`, which acts on the framework alone\n framework Manage the framework's lifecycle on installed\n tools\n translate [options] Convert an arbitrary source into a target-native\n plugin tree — records nothing (see `sync` for\n the manifest-driven, tracked version)\n plugin Manage plugins for AI tools\n marketplace Manage plugin marketplaces\n auth Manage authentication\n sync [options] [files...] Rewrite owned files from what is already there —\n regenerate tracked files, driven by the manifest\n (see `translate`, which converts a source\n without recording anything)\n update|upgrade [options] Update the aidd CLI itself to the latest version\n doctor [options] Detected and equipped tools, plugins, drift, and\n problems — across all tools or one\n clean [options] Remove all AIDD-managed files from the project —\n retires every part of AIDD; see `framework\n remove`, which removes the framework only\n telemetry Measure what Claude Code sessions consume,\n locally\n help [command] display help for command" + "help": "Usage: aidd [options] [command]\n\nGenerate AI coding assistant configurations from the AIDD framework\n\nOptions:\n -V, --version Show version number\n --verbose Show detailed diagnostic output (default: false)\n -h, --help display help for command\n\nCommands:\n setup [options] Set up or update the project to a correct state\n — bootstraps the whole project (marketplace,\n framework, tools, plugins); see `framework\n install`, which acts on the framework alone\n framework Manage the framework's lifecycle on installed\n tools\n translate [options] Convert an arbitrary source into a target-native\n plugin tree — records nothing (see `sync` for\n the manifest-driven, tracked version)\n plugin Manage plugins for AI tools\n marketplace Manage plugin marketplaces\n auth Manage authentication\n sync [options] [files...] Rewrite owned files from what is already there —\n regenerate tracked files, driven by the manifest\n (see `translate`, which converts a source\n without recording anything)\n update|upgrade [options] Update the aidd CLI itself to the latest version\n doctor [options] Detected and equipped tools, plugins, drift, and\n problems — across all tools or one\n clean [options] Remove all AIDD-managed files from the project —\n retires every part of AIDD; see `framework\n remove`, which removes the framework only\n telemetry Measure what Claude Code and Codex sessions\n consume, locally\n help [command] display help for command" }, { "invocation": "aidd auth", @@ -137,7 +137,7 @@ { "invocation": "aidd telemetry", "exitCode": 0, - "help": "Usage: aidd telemetry [options] [command]\n\nMeasure what Claude Code sessions consume, locally\n\nOptions:\n -h, --help display help for command\n\nCommands:\n on [options] Measure this clone, and remove what the previous\n version left in it\n off Stop measuring this clone; what was measured stays\n forget [options] Show everything measurement keeps on this machine, or\n remove it with --yes\n ingest [options] Read new transcripts into the local ledger, for\n clones that opted in\n task [options] [name] Declare the task the current work belongs to\n status Show if this clone is measured, since when, and its\n task\n report [options] Show what the work consumed, split along one axis,\n after reading new transcripts\n identity [options] [id] Choose to be named on your own measurement, or show\n or remove that choice\n help [command] display help for command" + "help": "Usage: aidd telemetry [options] [command]\n\nMeasure what Claude Code and Codex sessions consume, locally\n\nOptions:\n -h, --help display help for command\n\nCommands:\n on [options] Measure this clone, and remove what the previous\n version left in it\n off Stop measuring this clone; what was measured stays\n forget [options] Show everything measurement keeps on this machine, or\n remove it with --yes\n ingest [options] Read new transcripts into the local ledger, for\n clones that opted in\n task [options] [name] Declare the task the current work belongs to\n status Show if this clone is measured, since when, and its\n task\n report [options] Show what the work consumed, split along one axis,\n after reading new transcripts\n identity [options] [id] Choose to be named on your own measurement, or show\n or remove that choice\n help [command] display help for command" }, { "invocation": "aidd telemetry forget", diff --git a/cli/tests/helpers/ports/in-memory-telemetry.ts b/cli/tests/helpers/ports/in-memory-telemetry.ts index 8c5b70ec9..d3482c940 100644 --- a/cli/tests/helpers/ports/in-memory-telemetry.ts +++ b/cli/tests/helpers/ports/in-memory-telemetry.ts @@ -83,7 +83,7 @@ export class InMemoryTranscripts implements TranscriptSource { path: string, since: TranscriptPosition | null, writtenFrom: number, - each: (line: string) => void + each: (line: string, counted: boolean) => void ): Promise { const all = this.files.get(path) ?? []; const modified = this.modified.get(path); @@ -91,7 +91,7 @@ export class InMemoryTranscripts implements TranscriptSource { const position = { offset: all.length, size: all.length, identity: "inode" }; return { position, restarted: false, skipped: true }; } - for (const line of all.slice(since?.offset ?? 0)) each(line); + for (const line of all.slice(since?.offset ?? 0)) each(line, true); return { position: { offset: all.length, diff --git a/cli/tests/helpers/telemetry-sandbox.ts b/cli/tests/helpers/telemetry-sandbox.ts index ffcf1619d..f943e5f8a 100644 --- a/cli/tests/helpers/telemetry-sandbox.ts +++ b/cli/tests/helpers/telemetry-sandbox.ts @@ -63,6 +63,7 @@ export function createTelemetrySandbox(): TelemetrySandbox { APPDATA: join(home, "AppData", "Roaming"), USERPROFILE: home, CLAUDE_CONFIG_DIR: claude, + CODEX_HOME: join(home, ".codex"), AIDD_TELEMETRY: "", CLAUDE_CODE_SESSION_ID: "", CODEX_THREAD_ID: "", diff --git a/cli/tests/helpers/whole-transcript-reader.ts b/cli/tests/helpers/whole-transcript-reader.ts index 3d1ee4986..9ae00ac93 100644 --- a/cli/tests/helpers/whole-transcript-reader.ts +++ b/cli/tests/helpers/whole-transcript-reader.ts @@ -16,7 +16,7 @@ export class WholeTranscriptReader extends ClaudeTranscriptSourceAdapter { path: string, since: TranscriptPosition | null, writtenFrom: number, - each: (line: string) => void + each: (line: string, counted: boolean) => void ): Promise { const handle = await open(path, constants.O_RDONLY).catch(() => null); if (handle === null) return super.read(path, since, writtenFrom, each); @@ -36,7 +36,7 @@ export class WholeTranscriptReader extends ClaudeTranscriptSourceAdapter { .split("\n") .slice(0, -1) .map((line) => (line.endsWith("\r") ? line.slice(0, -1) : line)); - for (const line of lines) each(line); + for (const line of lines) each(line, true); return { position: { offset: offset + end, size: now.size, identity: now.identity }, restarted, diff --git a/docs/ARCHITECTURE.md b/docs/ARCHITECTURE.md index 73399f5bf..a8976344d 100644 --- a/docs/ARCHITECTURE.md +++ b/docs/ARCHITECTURE.md @@ -78,7 +78,7 @@ Place each capability in its owning concern and delegate to it. ### Measurement boundary -`aidd-telemetry` counts the tokens of Claude Code sessions deterministically and attributes them to the task a person declared. The CLI reads, stores, attributes and reports; the plugin's hooks only ask, answer and record session facts. The contract is [`aidd_docs/product/usage-contract.md`](../aidd_docs/product/usage-contract.md). +`aidd-telemetry` counts the tokens of Claude Code and Codex sessions deterministically and attributes them to the task a person declared. The CLI reads, stores, attributes and reports; the plugin's hooks only ask, answer and record session facts. The contract is [`aidd_docs/product/usage-contract.md`](../aidd_docs/product/usage-contract.md). ## Execution model diff --git a/docs/CATALOG.md b/docs/CATALOG.md index 1f7b71f73..ece5b41c7 100644 --- a/docs/CATALOG.md +++ b/docs/CATALOG.md @@ -9,7 +9,7 @@ The exhaustive list of AIDD plugins, skills, and actions. Skills are invoked thr - [aidd-vcs](#-aidd-vcs) - version control workflows - [aidd-orchestrator](#-aidd-orchestrator) - async orchestration (optional) - [aidd-ui](#-aidd-ui) - UI / UX (🚧 alpha, not ready) -- [aidd-telemetry](#-aidd-telemetry) - token measurement for Claude Code (🧪 beta, off the curated path) +- [aidd-telemetry](#-aidd-telemetry) - token measurement for Claude Code and Codex (🧪 beta, off the curated path) - [aidd-qa](#-aidd-qa) - acceptance QA (🆕 new, off the curated path) --- @@ -112,7 +112,7 @@ Runs synchronous feature delivery, optional async issue automation, and the prod ## 📈 aidd-telemetry -🧪 **Beta.** Measurement: counts the tokens of Claude Code sessions, per person, session, model, day, repository, task and ticket, and keeps everything on your machine. +🧪 **Beta.** Measurement: counts the tokens of Claude Code and Codex sessions, per person, session, model, day, repository, task and ticket, and keeps everything on your machine. | Skill | Role | Actions | | ----- | ---- | ------- | diff --git a/docs/FAQ.md b/docs/FAQ.md index aa36ef9c1..b9b96d47d 100644 --- a/docs/FAQ.md +++ b/docs/FAQ.md @@ -46,7 +46,7 @@ You can write your own Claude Code skills — nothing stops you. AIDD exists bec ## 📊 Measurement -**Nothing is measured until you opt in.** `aidd telemetry on` turns it on for one project, and `aidd telemetry forget --yes` removes everything it kept. It counts the tokens of Claude Code sessions and sends nothing anywhere. What it asks, what it keeps and what it needs are in the plugin's [README](../plugins/aidd-telemetry/README.md). +**Nothing is measured until you opt in.** `aidd telemetry on` turns it on for one project, and `aidd telemetry forget --yes` removes everything it kept. It counts the tokens of Claude Code and Codex sessions and sends nothing anywhere. What it asks, what it keeps and what it needs are in the plugin's [README](../plugins/aidd-telemetry/README.md). ## 🆘 Still stuck? diff --git a/plugins/aidd-telemetry/.claude-plugin/plugin.json b/plugins/aidd-telemetry/.claude-plugin/plugin.json index 5d4862a2f..807fe9686 100644 --- a/plugins/aidd-telemetry/.claude-plugin/plugin.json +++ b/plugins/aidd-telemetry/.claude-plugin/plugin.json @@ -2,7 +2,7 @@ "$schema": "https://json.schemastore.org/claude-code-plugin-manifest.json", "name": "aidd-telemetry", "version": "0.3.0", - "description": "Measurement: counts the tokens of Claude Code sessions per task, ticket, model and day, locally. Opt in with the 00-init skill; ask with 01-usage.", + "description": "Measurement: counts the tokens of Claude Code and Codex sessions per task, ticket, model and day, locally. Opt in with the 00-init skill; ask with 01-usage.", "author": { "name": "AI-Driven Dev", "url": "https://github.com/ai-driven-dev" diff --git a/plugins/aidd-telemetry/README.md b/plugins/aidd-telemetry/README.md index 96cb09bc8..17e3ae7bc 100644 --- a/plugins/aidd-telemetry/README.md +++ b/plugins/aidd-telemetry/README.md @@ -2,7 +2,7 @@ # aidd-telemetry -Counts the tokens your Claude Code sessions consume, each model call once, by person, session, model, agent, day, repository, task and ticket. Local only: nothing is sent anywhere. +Counts the tokens your Claude Code and Codex sessions consume, each model call once, by person, session, model, agent, day, repository, task and ticket. Local only: nothing is sent anywhere. > For: people who measure their work. Formats, rules and limits: [usage contract](../../aidd_docs/product/usage-contract.md). @@ -10,7 +10,7 @@ Counts the tokens your Claude Code sessions consume, each model call once, by pe ```mermaid flowchart LR - Claude["Claude Code sessions"] -->|"write"| Files["session files"] + Claude["Claude Code and Codex sessions"] -->|"write"| Files["session files"] Files -->|"read, each call once"| Ledger[("local ledger")] You["you"] -->|"aidd telemetry task"| Tasks[("task declarations")] Hooks["plugin hooks"] -->|"ask when none"| You @@ -21,6 +21,7 @@ flowchart LR - Only clones that ran `aidd telemetry on` are measured, from that moment on. - A task and a ticket come only from what you declare; nothing is guessed. - Every axis splits the same total. +- Codex is read with the rest, from `CODEX_HOME` or `~/.codex`, whenever `aidd telemetry ingest` or `report` runs. The plugin's hooks run in Claude Code, so nothing asks a Codex session for its task: a Codex call takes it from its branch's declaration only. ## Start diff --git a/plugins/aidd-telemetry/skills/00-init/actions/start.md b/plugins/aidd-telemetry/skills/00-init/actions/start.md index cfe8a7897..680e71052 100644 --- a/plugins/aidd-telemetry/skills/00-init/actions/start.md +++ b/plugins/aidd-telemetry/skills/00-init/actions/start.md @@ -13,7 +13,7 @@ Measurement is on for the project, or nothing changed and the reason is stated. ## Process 1. **Explain.** Tell the person, before asking anything: - - what is measured: the token counts of Claude Code sessions run in this project, read from Claude Code's own session files, split into input, output, cache read and cache write; + - what is measured: the token counts of Claude Code and Codex sessions run in this project, read from each tool's own session files (Codex's under `CODEX_HOME`, else `~/.codex`), split into input, output, cache read and cache write; - what is asked: one question per working branch, a task name and optionally a ticket, only when a person is at the keyboard; - when it starts: now. Measurement starts at the moment it is turned on, and what this project's sessions did before is not counted, whatever Claude Code still holds of it; - where it stays: on this machine, and nothing is sent anywhere; @@ -31,7 +31,7 @@ Measurement is on for the project, or nothing changed and the reason is stated. 5. **Report.** Relay everything the command printed, the removals and the warnings included, in the person's words rather than pasted. - When it warned that Claude Code keeps session files only briefly, say that a call is measured only if it is read before the files are removed, and give the setting it named. 6. **Prove.** Run `aidd telemetry report --days 1`, and state what it shows. - - No calls yet is a normal answer for a project that has just opted in: say so, and that the next Claude Code session in this project will be counted. + - No calls yet is a normal answer for a project that has just opted in: say so, and that the next Claude Code or Codex session in this project will be counted. 7. **Tell the rest.** Say that the plugin is what asks about the task and that it needs `node`, and that a task can always be declared by hand with `aidd telemetry task [--ticket ]` or `aidd telemetry task --none`, and that `aidd telemetry status` shows whether a clone is measured and its task. ## Test diff --git a/plugins/aidd-telemetry/skills/00-init/actions/stop.md b/plugins/aidd-telemetry/skills/00-init/actions/stop.md index 68e914e12..4d601a9be 100644 --- a/plugins/aidd-telemetry/skills/00-init/actions/stop.md +++ b/plugins/aidd-telemetry/skills/00-init/actions/stop.md @@ -12,7 +12,7 @@ Measurement is off for the project, and the person knows the measured data is st ## Process -1. **Confirm.** Say what stopping does: new Claude Code work in this project is no longer counted or asked about, and what was measured stays. Wait for a yes. +1. **Confirm.** Say what stopping does: new Claude Code or Codex work in this project is no longer counted, nor asked about, and what was measured stays. Wait for a yes. 2. **Turn off.** Run `aidd telemetry off` and relay its message. 3. **Offer the rest.** Say that deleting what was measured is a separate step, and that it needs its own confirmation. diff --git a/plugins/aidd-telemetry/skills/01-usage/actions/answer.md b/plugins/aidd-telemetry/skills/01-usage/actions/answer.md index 0f633c58e..83c7f66a0 100644 --- a/plugins/aidd-telemetry/skills/01-usage/actions/answer.md +++ b/plugins/aidd-telemetry/skills/01-usage/actions/answer.md @@ -27,7 +27,7 @@ The figures the report holds for the question, each counter apart, followed by e - any `coverage.left_out` entry above `0`: that many calls or lines were left out, from any day, not only the period; - `coverage.records` of `0`: that no call is recorded for the period, and why that may be, from the reference. - `totals.records` of `0` with `coverage.records` above `0`: that a filter matched no call of the period, naming each as `filters` prints it, from the reference. -5. **Bound the past.** When the period starts before `run.oldest_transcript_at`, say that Claude Code may have removed older session files, so earlier figures can be incomplete. +5. **Bound the past.** When the period starts before `run.oldest_transcript_at`, say that a tool may have removed older session files, so earlier figures can be incomplete. 6. **Offer the fix.** For work with no task, say it can be attributed afterwards by declaring the task from that branch with `aidd telemetry task [--ticket ]`. A branch renamed or copied since it was declared is undeclared under its new name (unless declared by `aidd` 5.5.0 or earlier), and is declared the same way. Declare nothing yourself. ## Test diff --git a/plugins/aidd-telemetry/skills/01-usage/references/envelope.md b/plugins/aidd-telemetry/skills/01-usage/references/envelope.md index 0945d89fa..926c434c4 100644 --- a/plugins/aidd-telemetry/skills/01-usage/references/envelope.md +++ b/plugins/aidd-telemetry/skills/01-usage/references/envelope.md @@ -70,4 +70,4 @@ A field may be added to version `2`: a figure outside `run`, a fact of the read | `run.not_stored.unreadable-consent` above `0` | A clone's git config or git directory cannot be read: nothing is stored for it until it can. For a clone that had opted in, its calls are then stored, read again by the next run; for any other directory, they are not | | `run.consent_log_damaged` is `true` | A line of the consent log `ledger/consents.jsonl` is not an event, so nothing is stored for any clone. Recovery is `aidd telemetry forget --yes`, then `aidd telemetry on` in each clone: nothing from before is stored. Do not edit the log by hand | | `totals.records` of `0` and `coverage.records` above `0`, with a filter given | No call of the period matches the filters: a value no call has, or a start of a session or repository id that names none, printed as typed in `filters` | -| `coverage.records` of `0`, `run.files_read` of `0` and every `run.not_stored` at `0` | Nothing new was read: no clone on this machine has opted in, so no session file is read (`aidd telemetry status` says so in the clone); no Claude Code session file was written since the first `aidd telemetry on`; or an earlier report already read the calls and left them out, and only its `run` said why | +| `coverage.records` of `0`, `run.files_read` of `0` and every `run.not_stored` at `0` | Nothing new was read: no clone on this machine has opted in, so no session file is read (`aidd telemetry status` says so in the clone); no Claude Code or Codex session file was written since the first `aidd telemetry on`; or an earlier report already read the calls and left them out, and only its `run` said why | diff --git a/scripts/check-claude-tokens-end-to-end.cjs b/scripts/check-claude-tokens-end-to-end.cjs index 4233d5719..91157e5d5 100644 --- a/scripts/check-claude-tokens-end-to-end.cjs +++ b/scripts/check-claude-tokens-end-to-end.cjs @@ -338,6 +338,7 @@ async function checkIn({ root, cli, api, log, attended }) { AIDD_USER_CONFIG_DIR: path.join(home, ".config", "aidd"), AIDD_TELEMETRY_DIR: telemetryDir, CLAUDE_CONFIG_DIR: configDir, + CODEX_HOME: path.join(home, ".codex"), GIT_CONFIG_NOSYSTEM: "1", ANTHROPIC_BASE_URL: api.url, ANTHROPIC_API_KEY: API_KEY, diff --git a/scripts/probe-identifier-join.cjs b/scripts/probe-identifier-join.cjs index e6be1382c..c6ea6f033 100644 --- a/scripts/probe-identifier-join.cjs +++ b/scripts/probe-identifier-join.cjs @@ -140,6 +140,7 @@ function probeIn(root, host, log) { ...env, HOME: home, CLAUDE_CONFIG_DIR: configDir, + CODEX_HOME: path.join(home, ".codex"), ANTHROPIC_BASE_URL: "http://127.0.0.1:9", ANTHROPIC_API_KEY: "sk-ant-probe-not-a-real-key", },