Skip to content

Add host-configuration doctor to adaptive-delivery #161

Description

@BjRo

Motivation

Adaptive-delivery needs a doctor function that can check whether the host configuration supports its orchestration and required verification/review delegation.

In the recent Codex 0.154.0 N=3 run, the goal-preflight-high-risk-routine and goal-verification-existing-review cases each failed all three trials when the independent Spec reader could not launch. Retained assessments reported the agent thread limit/shared-agent capacity. The observed default was three concurrent subagents plus the primary agent.

Configuration diagnosis needs to distinguish concurrency from nesting depth. Codex exposes agents.max_concurrent_threads_per_session; its current schema describes agents.max_depth as V1-only and ignored by V2. The eval runner also uses isolated Codex homes, so a setting in the source checkout is not automatically the setting used by an eval.

Acceptance criteria

  • Adaptive-delivery offers doctor functionality that checks whether Codex configuration supports its orchestration and required assessment delegation.
  • The diagnostic checks relevant agent configuration, distinguishes concurrency from nesting depth and backend applicability, and reports configuration problems with actionable setup guidance.
  • The diagnostic identifies the configuration being checked so source-checkout settings are not mistaken for the effective settings of an isolated eval.
  • Include comparable Claude Code configuration checks where the host provides relevant controls.

Open questions

  • Which comparable Claude Code configuration controls can be checked reliably?

References

Activity

  1. BjRo commented on Sep 16, 2026

    @BjRo
    OwnerAuthor

    Claude Code configuration research

    The open Claude Code question is now answerable from current official documentation.

    Relevant controls

    • CLAUDE_CODE_MAX_CONCURRENT_SUBAGENTS caps concurrent Agent-tool subagents. The documented default is 20, the value must be a positive whole number, and the setting requires Claude Code 2.1.217 or newer. There is no cumulative per-session spawn cap. Concurrent subagent limit
    • CLAUDE_CODE_MAX_SUBAGENT_SPAWN_DEPTH controls the number of subagent layers below the main conversation. The current default is 3; setting it to 1 disables nested delegation. Defaults changed in older releases, so the diagnostic should inspect claude --version before applying a default. Nested subagents
    • A bare Agent rule in permissions.deny removes the tool. Agent(name) rules can deny required named agents, while --disallowedTools, --tools, and --bare can impose session-only restrictions. Agent permissions, CLI flags
    • An agent definition can prevent further delegation by omitting Agent from tools or adding it to disallowedTools. Subagent tool restrictions
    • CLAUDE_CODE_DISABLE_BACKGROUND_TASKS=1 forces foreground execution but does not disable subagents. Foreground/background behavior

    For Adaptive Delivery, the repository's documented topology implies a minimum concurrent capacity of 5 and a nesting depth of 3: owner, verification coordinator, review coordinator, and two review readers. This is a Darrow requirement, not an Anthropic default.

    Effective configuration identity

    • Normal locations are ~/.claude/settings.json, .claude/settings.json, and .claude/settings.local.json.
    • Precedence is managed, command line, local project, shared project, then user. Permission lists merge across scopes. Settings precedence
    • CLAUDE_CONFIG_DIR relocates the user configuration directory only. It does not relocate repository .claude files or system-managed settings. An isolated eval must therefore identify $CLAUDE_CONFIG_DIR/settings.json plus any project and managed sources it considered. Environment variables, .claude directory
    • An empty CLAUDE_CONFIG_DIR alone is not a clean configuration because project and managed settings can still load. Clean configuration testing

    Doctor behavior

    A portable Bash doctor can reliably inspect the Claude Code version, inherited environment values, visible settings files, Agent deny rules, agent definitions, and the absolute paths inspected. It can also run claude doctor to report invalid or rejected settings.

    It cannot fully reconstruct effective configuration from files when server/MDM policy, embedding-host policy, or invocation-only flags such as --settings, --setting-sources, --tools, --disallowedTools, --agents, or --bare may apply. Those cases should produce unknown or manual-check, directing the user to /status, /permissions, /context, and claude doctor. Configuration diagnostics

    For Claude Code 2.1.219 or newer, the documented defaults satisfy Darrow's inferred minimums. Explicit configuration can pin them:

    {
      "env": {
        "CLAUDE_CODE_MAX_CONCURRENT_SUBAGENTS": "5",
        "CLAUDE_CODE_MAX_SUBAGENT_SPAWN_DEPTH": "3"
      }
    }

    The Claude-side acceptance criterion is therefore implementable with a tri-state result: supported, unsupported, or manual-check.

  2. BjRo commented on Sep 22, 2026

    @BjRo
    OwnerAuthor

    Fixed in #167

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions