Skip to content

codexwire: the Codex app-server client crate - #1

Merged
macalinao merged 10 commits into
mainfrom
crate
Nov 6, 2026
Merged

macalinao merged 10 commits into
mainfrom
crate

Conversation

@macalinao

@macalinao macalinao commented Nov 6, 2026 •

Copy link
Copy Markdown
Contributor

The codexwire crate, moved out of sortie's sortied (talaagent/sortie#4, crates/sortied/src/codexwire and scripts/gen-codex-protocol; originally macalinao/tala-agent#744) into its own repo: a Rust client that drives codex app-server over JSON-RPC, the Codex counterpart to ccwire.

  • protocol: serde types generated from the app-server JSON Schema of Codex 0.158.0 (schema/VERSION, CODEX_VERSION)
  • rpc: line framing
  • client: spawn_app_server (allowlisted env), Client (id matching, ordered notifications and server requests), stderr redact
  • auth: a session's private auth.json, copied back only when it's safe
  • scripts/gen-codex-protocol regenerates offline. --refresh [CODEX] re-dumps the schema and moves the pin. --check [CODEX] fails if a binary isn't the pinned version. scripts/check-codegen catches drift.
  • Standalone: its own [workspace], edition 2024, MSRV 1.85 (as in ccwire). Apache-2.0 only, LICENSE plus a NOTICE that credits OpenAI Codex (Apache-2.0) for the schema. deny.toml, and a clippy.toml like ccwire's. No tala dependency or strings. Example in examples/initialize.rs.

Changes from the sortied copy: crate:: imports and a crate root; impl Error for ClientError; neutral test fixtures (no user paths); the auth temp-file prefix is codexwire-; tests allow block_on, which #[tokio::test] expands to.

Why we don't use an upstream crate

openai/codex doesn't publish its app-server protocol types, so we keep the generator.

  • On crates.io, codex-app-server-protocol and codex-protocol are owned by namastex888, a third party. They come from a fork (namastexlabs/codex) and stop at 0.63.0 (2025-12-11), about 95 releases behind 0.158.0. They aren't OpenAI's.
  • codex-codes (meawoppl/rust-code-agent-sdks, Apache-2.0) is a separate, hand-written community SDK. It's a peer, not a source of upstream types.
  • A git dependency on openai/codex codex-rs/app-server-protocol would be Apache-2.0, pinnable by the rust-v0.158.0 tag, and serde-ready, but too heavy:
    • It pulls about a dozen internal workspace crates: codex-protocol, codex-rollout, codex-secrets, codex-shell-command, codex-history, macros…
    • codex-protocol alone brings tokio, ICU, gix-url, quick-xml, schemars, ts-rs, http, a network proxy and an HTTP client, plus landlock and seccompiler on Linux.
    • Its deps include rmcp =3.2.0 (an exact pin) and zstd.
    • The git source is the whole ~150-member workspace, which pins Rust 1.95 and isn't semver'd for outside use.

Generating from the schema that codex app-server generate-json-schema dumps is the interface Codex supports for outside clients. It costs serde, serde_json and tokio.

Review fixes (074d368..08d0fed)

Must-fix:

  • auth: Copied's Debug no longer prints the tokens. orig is private (orig_len() instead), and a hand-written Debug shows it as [redacted; N bytes]. Test: debug_redacts_the_tokens.
  • client: no hang after close. A closed flag lives under the pending-map lock. The reader sets it on exit, and call returns ClientError::Closed at once when it is set. Test: a_call_after_the_server_output_ended_fails_at_once (stdout closed, stdin still open).
  • client: a non-UTF-8 line no longer ends the connection. The reader uses read_until(b'\n') and a line that isn't UTF-8 becomes ServerMessage::Malformed, then reading goes on. It stops only on EOF or an I/O error. Stderr gets the same fix through a new public StderrTail::read_from(reader), which keeps lines lossily (U+FFFD). spawn_app_server uses it, and a caller can use it for children it spawns itself. Tests: a_line_that_is_not_utf8_is_malformed_and_reading_goes_on, stderr_lines_that_are_not_utf8_are_kept_lossily, and the fake-codex spawn test.
  • CI: .github/workflows/ci.yml is ported from ccwire with the same private gate. It runs fmt, clippy -D warnings, tests and doctests (--locked), MSRV (from rust-version), docs (-D warnings) and cargo-deny, plus a new check-codegen job (python3 + rustfmt, offline). There is one clippy and one test job because the crate has no features. deny.toml is now ccwire's (unmaintained = "all", crates.io only, no git), with this crate's extra allowed dependency licenses kept. The crate itself stays Apache-2.0 only.

Should-fix:

  • A cancelled call cleans up. A drop guard removes its pending entry, so a timeout or select! leaks nothing, and a late answer is ignored. There is no timeout API: the call docs and the README say to wrap calls in tokio::time::timeout. Test: a_cancelled_call_forgets_its_request.
  • New test tests/corpus.rs. It decodes tests/corpus/codex-0.158.0-turn-unauthenticated.jsonl with the generated types. That file is the server's side of a real codex 0.158.0 session (initialize, thread/start, then a turn that fails without a login: error notifications, items, turn/completed). Paths, host name and ids are scrubbed. Every known item must decode as something other than Unknown. The test also covers the Unknown fallback, including a known tag with a broken body.
  • New test tests/spawn.rs. It runs spawn_app_server against tests/fake_codex, a python3 stand-in. It covers the args, the cwd, the env allowlist (env_clear), a typed initialize, both readers past a non-UTF-8 line, and kill on drop.

Nits taken:

  • The README is now the crate doc, so its example compiles as a no_run doctest.
  • --check is documented as a binary-version check, not a codegen check.
  • Client::new documents that it panics outside a runtime and that line length and the message channel are unbounded.
  • New ClientError::Encode for params that fail to serialise.
  • Docs on ENV_ALLOW's missing proxy variables, the graceful stop (shutdown, then child.wait() under a timeout), copy_back's race window, and mkdir_private on an existing directory. The README notes the crate is Unix only.
  • The generator docstring and the generated Unknown doc say that a known tag whose body doesn't decode also lands in Unknown, so protocol.rs was regenerated.

Not taken:

  • AppServer::close() (nit 14): it needs a timer in the library, so it is documented instead.
  • A bounded line length (nit 9): documented instead.
  • release-plz and dependabot: they wait until the crate is published.

Gate

  • At 08d0fed (run by the reviewer under tala-build-lock): fmt, the tests (13+2+1), the doctests and check-codegen passed. Clippy -D warnings and RUSTDOCFLAGS="-D warnings" cargo doc --no-deps each failed with one error in the lib.
  • 46587db fixes both:
    • rustdoc: the generator now escapes the schema's stray brackets ([AbsolutePathBufGuard::new], a broken intra-doc link; [UNSTABLE]; []) and wraps its bare URL in <...>, and src/protocol.rs is regenerated.
    • clippy: doc_markdown objected to "OpenAI" in the README, which is now the crate doc. clippy.toml adds it to doc-valid-idents.
  • At 46587db: scripts/check-codegen passes. Build, clippy, doc and tests were not run here: tala-build-lock exited 75 (all slots busy) every time, so this head is handed to the gate queue.
  • GitHub Actions: the jobs are skipped while the repo is private (the gate's design).
    GPG signing needs a passphrase in this headless run, so these commits are unsigned, like the branch's earlier ones.

Later follow-ups (not in this PR)

  • tala-agent's image_gen codex exec call (crates/tala-agent/src/image_gen.rs) moves onto codexwire's app-server client.
  • tala-agent#816's single-refresh token lock (one auth.json refresh at a time) moves into codexwire (auth), so every consumer shares it.
  • Tag v0.1.0 once this lands, and move sortie's pin (talaagent/sortie#6, at this PR's head 9da6cf3) from the rev to the tag.
  • CI is in (.github/workflows/ci.yml). It is gated off while the repo is private and turns on when the repo goes public.

…and the pinned schema

Moved from sortie's sortied crate (crates/sortied/src/codexwire, and
scripts/gen-codex-protocol): auth, client, protocol (generated from the
Codex 0.158.0 app-server schema), rpc. Standalone: its own workspace,
Apache-2.0 with a NOTICE for the schema, deny.toml, clippy.toml, an
example, and gen-codex-protocol --check for the pin.
orig (the whole auth.json, tokens included) is private now, with
orig_len(); Debug shows it as [redacted; N bytes]. Docs: unix only, the
copy-back's race window, mkdir_private on an existing directory.
- A closed flag under the pending-map lock: a call after the server's
  output ended returns Closed at once instead of waiting forever.
- A drop guard removes a cancelled call's pending entry; document
  tokio::time::timeout for timeouts.
- Both readers read bytes: a line that isn't UTF-8 is Malformed (stdout)
  or kept lossily (stderr, StderrTail::read_from), and reading goes on.
- ClientError::Encode for params that don't serialise.
- Docs: unbounded line length and channel, panics outside a runtime, no
  proxy variables in ENV_ALLOW, the graceful stop.
- Regression tests for each.
tests/corpus: the server's side of a codex 0.158.0 session (initialize,
thread/start, a turn that fails without a login), scrubbed, decoded with
the generated types. tests/spawn.rs runs spawn_app_server against
tests/fake_codex: args, cwd, the env allowlist, both readers past a
non-UTF-8 line, kill on drop.
… --check

Also: unix only, Malformed and Closed, timeouts.
fmt, clippy, tests and doctests, MSRV, docs, cargo-deny, and
scripts/check-codegen, each gated off while the repo is private.
… its URLs; OpenAI is a valid doc ident

The generated docs had [AbsolutePathBufGuard::new] (a broken intra-doc
link), [UNSTABLE] and a bare URL; the README, now the crate doc, has
OpenAI in its prose (clippy::doc_markdown).
@macalinao
macalinao merged commit 64e61d1 into main Nov 6, 2026
7 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant