Skip to content

Codex backend on a custom Responses endpoint: apply_patch_tool_type is not settable, namespace/web_search tools are sent by default, and the runner stream has no output_text.delta #202

Description

@master5d

Environment: HarnessRouter CE 0.17.3 (Docker Hub digest sha256:affaa83b048ed7875fc7e656dd6fab2a91dd0b98f95faad04d7426c1343a6b72, amd64, self-hosted via compose), backends HR_BACKENDS=codex,hermes (Codex CLI 0.154.0, Hermes Agent 0.19.0 installed at first start), models served by a self-hosted LiteLLM gateway (OpenAI Responses / Chat Completions, provider custom).

Three things we hit running the codex backend (Codex CLI 0.154.0) against a self-hosted OpenAI-Responses-compatible endpoint (a LiteLLM Responses→Chat bridge in front of open-weight models). They belong together because they all come from the generated per-turn config.toml (_CODEX_CONFIG_TMPL in runner/server.py) and the request shape it produces.

1. apply_patch_tool_type cannot be set, so Codex has no apply_patch tool at all

For models Codex does not know, it registers apply_patch only when apply_patch_tool_type is set in config.toml (core/src/tools/spec_plan.rs: the handler is added iff the option is_some()). The runner's template does not expose the knob, and there is no harness-level setting for it. Result: the request to the provider carries no apply_patch tool (verified on the wire), while Codex's own instructions still tell the model to "always use apply_patch" — an open-weight model then calls a tool that does not exist and loops on unsupported call: apply_patch (13 times in one turn in a controlled run). A per-harness (or per-integration) apply_patch_tool_type = "function" would fix file editing for every non-OpenAI model.

2. Tools of type namespace (multi-agent) and web_search are sent to a custom endpoint by default

On the wire the Responses request carries tools of type namespace (multi_agent_v1) and web_search next to the function tools. Endpoints that only understand function tools reject the request (an OpenAI-compatible backend answered 400; a llama.cpp-style backend answered 404 'dict object' has no attribute 'function' from its chat template). #150 added a way to turn off web_search; the namespace tool has no switch. Suggestion: for provider: custom integrations send only function tools unless the integration opts in.

3. Streaming a Codex task yields no output_text.delta events

POST /v1/responses with stream: true on a Codex harness completes with the final text, but the stream carries no output_text.delta (Hermes on the same instance streams deltas). The conformance suite's S-09 passes because it measures event spread, not text deltas.

Happy to provide the raw request captures (tool arrays, no secrets) if useful.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions