Environment: HarnessRouter CE 0.17.3 (Docker Hub digest sha256:affaa83b048ed7875fc7e656dd6fab2a91dd0b98f95faad04d7426c1343a6b72, amd64, self-hosted via compose), backends HR_BACKENDS=codex,hermes (Codex CLI 0.154.0, Hermes Agent 0.19.0 installed at first start), models served by a self-hosted LiteLLM gateway (OpenAI Responses / Chat Completions, provider custom).
Three things we hit running the codex backend (Codex CLI 0.154.0) against a self-hosted OpenAI-Responses-compatible endpoint (a LiteLLM Responses→Chat bridge in front of open-weight models). They belong together because they all come from the generated per-turn config.toml (_CODEX_CONFIG_TMPL in runner/server.py) and the request shape it produces.
1. apply_patch_tool_type cannot be set, so Codex has no apply_patch tool at all
For models Codex does not know, it registers apply_patch only when apply_patch_tool_type is set in config.toml (core/src/tools/spec_plan.rs: the handler is added iff the option is_some()). The runner's template does not expose the knob, and there is no harness-level setting for it. Result: the request to the provider carries no apply_patch tool (verified on the wire), while Codex's own instructions still tell the model to "always use apply_patch" — an open-weight model then calls a tool that does not exist and loops on unsupported call: apply_patch (13 times in one turn in a controlled run). A per-harness (or per-integration) apply_patch_tool_type = "function" would fix file editing for every non-OpenAI model.
2. Tools of type namespace (multi-agent) and web_search are sent to a custom endpoint by default
On the wire the Responses request carries tools of type namespace (multi_agent_v1) and web_search next to the function tools. Endpoints that only understand function tools reject the request (an OpenAI-compatible backend answered 400; a llama.cpp-style backend answered 404 'dict object' has no attribute 'function' from its chat template). #150 added a way to turn off web_search; the namespace tool has no switch. Suggestion: for provider: custom integrations send only function tools unless the integration opts in.
3. Streaming a Codex task yields no output_text.delta events
POST /v1/responses with stream: true on a Codex harness completes with the final text, but the stream carries no output_text.delta (Hermes on the same instance streams deltas). The conformance suite's S-09 passes because it measures event spread, not text deltas.
Happy to provide the raw request captures (tool arrays, no secrets) if useful.
Environment: HarnessRouter CE
0.17.3(Docker Hub digestsha256:affaa83b048ed7875fc7e656dd6fab2a91dd0b98f95faad04d7426c1343a6b72, amd64, self-hosted via compose), backendsHR_BACKENDS=codex,hermes(Codex CLI 0.154.0, Hermes Agent 0.19.0 installed at first start), models served by a self-hosted LiteLLM gateway (OpenAI Responses / Chat Completions, providercustom).Three things we hit running the
codexbackend (Codex CLI 0.154.0) against a self-hosted OpenAI-Responses-compatible endpoint (a LiteLLM Responses→Chat bridge in front of open-weight models). They belong together because they all come from the generated per-turnconfig.toml(_CODEX_CONFIG_TMPLinrunner/server.py) and the request shape it produces.1.
apply_patch_tool_typecannot be set, so Codex has noapply_patchtool at allFor models Codex does not know, it registers
apply_patchonly whenapply_patch_tool_typeis set inconfig.toml(core/src/tools/spec_plan.rs: the handler is added iff the optionis_some()). The runner's template does not expose the knob, and there is no harness-level setting for it. Result: the request to the provider carries noapply_patchtool (verified on the wire), while Codex's own instructions still tell the model to "always use apply_patch" — an open-weight model then calls a tool that does not exist and loops onunsupported call: apply_patch(13 times in one turn in a controlled run). A per-harness (or per-integration)apply_patch_tool_type = "function"would fix file editing for every non-OpenAI model.2. Tools of type
namespace(multi-agent) andweb_searchare sent to a custom endpoint by defaultOn the wire the Responses request carries tools of type
namespace(multi_agent_v1) andweb_searchnext to thefunctiontools. Endpoints that only understandfunctiontools reject the request (an OpenAI-compatible backend answered400; a llama.cpp-style backend answered404 'dict object' has no attribute 'function'from its chat template). #150 added a way to turn offweb_search; thenamespacetool has no switch. Suggestion: forprovider: customintegrations send onlyfunctiontools unless the integration opts in.3. Streaming a Codex task yields no
output_text.deltaeventsPOST /v1/responseswithstream: trueon a Codex harness completes with the final text, but the stream carries nooutput_text.delta(Hermes on the same instance streams deltas). The conformance suite's S-09 passes because it measures event spread, not text deltas.Happy to provide the raw request captures (tool arrays, no secrets) if useful.