Skip to content

[feat] Enforce workflow requirements and execution limits #1082

Closed

Description

@hsuchifeng

Parent: #818

Goal

Check workflow requirements against member policy and driver capability before execution while keeping secrets out of leases and state.

Scope

  • Add requires, defaults, setup, workflow budget, step max_turns, and workflow/step timeout.
  • Add member workflow allow and concurrency settings.
  • Compare requirements, member ceiling, and driver capability before leasing.
  • Support v1 network, env, isolation, human, and resume capabilities.
  • Resolve required env through existing TeamAI facilities and process environment in unattended CI.
  • Put only key names in leases and state.
  • Fail before execution when a required key is missing.
  • Scan every string carried by a report and mask every known secret value before the report reaches state or records, including the local driver’s message and later-added fields; keep key names visible.
  • Prevent command inputs from shadowing required secret keys.
  • Run setup before the agent in each new workspace with separate requirements.
  • Enforce timeout and token budget deterministically.
  • Pass max_turns only when the adapter can enforce it.

Out of scope

requires.platform, remote secret transport, named remote capabilities, minimum sandbox policy, provider approvals, and OpenTelemetry.

Dependencies

Acceptance criteria

  • A lease is issued only when requirements fit policy and capability.
  • Secret values never appear in lease JSON, state, errors, snapshots, or records.
  • Every string in a report is scanned for known secret values, including message and fields added later, while key names remain visible.
  • CI may satisfy env requirements from process environment.
  • Setup credentials do not reach the agent unless separately required.
  • Unsupported enforcement fails explicitly.
  • Timeout and budget exhaustion settle predictably.

Verification

Focused negotiation, every-string masking, env, setup, timeout, and budget tests; build/typecheck/lint; one real-CLI safety E2E. Update bilingual workflow and environment documentation.

Activity

  1. SaulMoro commented on Oct 13, 2026

    @SaulMoro
    Collaborator

    For masking, maybe it could cover every string a report carries rather than a list of fields? The local driver already stores the first 400 characters of stderr in message, and #1084 publishes snapshots from that state. If everything is masked by default, a new field is covered without anyone having to remember it.

  2. hsuchifeng commented on Oct 14, 2026

    @hsuchifeng
    CollaboratorAuthor

    Mask every known secret value in every string carried by a report, not a fixed field list. That includes the local driver’s message, which contains the first 400 characters of stderr, and any field added later. Key names stay visible.

  3. SaulMoro commented on Oct 14, 2026

    @SaulMoro
    Collaborator

    local-driver.ts passes the whole process.env to every step, and the tool adapter in #1120 does the same. In CI, that gives the job's GH_TOKEN to every agent step, while #818 meant it to reach only the steps that list it. The driver could withhold every key declared in env.yaml or env/secrets.yaml that the step does not list in requires.env. That reuses the declarations from #875 and leaves PATH, HOME and the other variables alone.

    A driver that cannot report tokens fails every workflow with a budget, because unsupported enforcement fails explicitly. As far as I can see, T3 Code does not expose token usage today, so a T3 Code driver would hit this.

  4. SaulMoro commented on Oct 14, 2026

    @SaulMoro
    Collaborator

    A related gap on the tool side: if #1120 lands as it is, Codex and OpenCode steps report no token counts, so a workflow with a budget fails as soon as it runs a step on either tool. Both CLIs print structured output (codex exec --json, opencode run --format json). If that output carries usage, this issue could read the counts there, unless #1120 does it first.

    Update: resolved in #1120 (39a7dc1). Codex and OpenCode steps now report token counts from their JSON events.

  5. SaulMoro commented on Oct 14, 2026

    @SaulMoro
    Collaborator

    Notes from #1078 (#1129) that affect this issue:

    • env: on command steps. The lease carries the rendered map and the local driver merges it over the inherited environment. Keys only have to match ^[A-Za-z_][A-Za-z0-9_]*$, so today an env key can shadow a declared secret or PATH. Rejecting a key that collides with a declared secret, as Discussion: Workflow Runtime Integration #818 says, belongs here.
    • Inputs are stored in the run file (start --inputs), and rendered env values travel in the lease. A secret passed as an input would land in both. Masking needs to cover run state, not only report.
    • timeout as a stopped state. Expressions compare outcome with any string, so steps.x.outcome == 'timeout' already parses, but nothing produces that value and the engine's stopped() only knows failed. A step timeout should become a stopped state, so recovery steps and the run outcome see it.

    The engine rules that span files are in docs/dev/workflow-engine.md, added by #1129.

  6. SaulMoro commented on Oct 15, 2026

    @SaulMoro
    Collaborator

    Notes on network and isolation, from checking this against #818 and the drivers that will have to meet them (#1080, #1086, #1087):

    • Make network and isolation best effort in v1. Both factory examples in Discussion: Workflow Runtime Integration #818 set defaults: { requires: { network: false } }. Discussion: Workflow Runtime Integration #818 and this issue check requirements, the member's policy and the driver's capability before a lease, and "Unsupported enforcement fails explicitly". The local adapter passes no sandbox or network setting to its tools (local-tool-adapter.ts:68-89). A tool's own config or the host may still restrict them, but without that, docs and retro in ship-issue, which keep the network: false default, could not run on the local driver. Any driver without these controls would be blocked from the same workflow. I'd change the capability check and that criterion for these two requirements. A driver honors what it can and still takes the step when it cannot enforce one. The report and the run record state what was asked for and what was not enforced. [feat] Add the session skill driver and invocable workflows #1086's "decline unsupported isolation" would change with it. requires.env and human stay strict, because the step cannot run without them. The cost is real. A step that asks for no network may still reach it. A read-only step that asks for worktree may still run in the member's checkout, and a later read_only_dirty does not undo what it changed. A member who needs a guarantee picks a driver and host that enforce it. Minimum sandbox policy stays out of scope. One conformance case would hold this: an unsupported network or isolation still gets a lease and is recorded as not enforced, while a missing env key or human still blocks it, through both the engine and the CLI.
    • Three isolation levels are enough. Discussion: Workflow Runtime Integration #818 uses isolation: worktree in both the member's allow and the driver's can. The levels could be none, worktree and sandbox. none adds nothing beyond the workspace mode, so a new step still gets its own copy. worktree runs the step in a separate checkout on the same machine. It keeps changes out of the member's checkout, but does not stop writes elsewhere or network access. sandbox asks for containment beyond that. A container or a remote machine can provide it, though where a process runs does not prove it, since a container can mount the member's home. Container and remote can share one value, because nothing in a workflow treats them differently. A local driver that honors worktree would also run read-only steps outside the member's checkout, which workspace: new in [feat] Add workflow workspaces, branches, and session continuation #1080 does not cover. When a driver cannot detect the sandbox it runs in, such as in a CI job, I'd let the member's config declare it and the driver include it in can. TeamAI already trusts what a driver declares ("Choosing a driver is a trust decision").
  7. SaulMoro commented on Oct 16, 2026

    @SaulMoro
    Collaborator

    Three things from #1148 for budget and timeout:

    • Count the tokens of every pass. When a loop runs a step again, repend moves what the step reported, tokens included, into its passes. When it runs a workflow or for_each step again, the old children move to run.archive. A budget that sums tokens over run.steps misses every earlier pass. buildEndRecord reads both.
    • Stop the leases already out. With parallel leases, "next hands out no more leases" (Discussion: Workflow Runtime Integration #818) leaves the open ones running and spending. An end already stops them through stopUnder. It cancels each open lease, a late report gets lease_cancelled, and the local driver sends the command SIGTERM, then SIGKILL after 5 s. A spent budget and a lease past its timeout can take the same path. The engine runs only inside next and report, so a timeout takes effect on the first call after it passes.
    • New failure codes need the record allowlist. The run record publishes failure.code only for codes in RECORDED_FAILURES (records.ts). A timeout or budget code left out of it is dropped from the record without a warning.
  8. SaulMoro commented on Oct 16, 2026

    @SaulMoro
    Collaborator

    #1177 adds model to the workflow's defaults, next to the requires this issue adds. Nothing else here changes.

    • Precedence for a prompt step's model, first match wins: the step's model, the model its definition sets for the tool, defaults.model, the tool's default. defaults.model fills a gap, as Warp's agentDefaults does, and never replaces a model a definition chose for its role.
    • Decided point 3 still holds: a step's model replaces defaults.model. The difference is that a definition's model sits between the two, so defaults.model is the one key that does not act as if written on every step. requires keeps acting that way, including the merge of requires.env.
    • Whichever of the two issues merges first creates defaults. If [feat] Let a workflow choose the model of its prompt steps #1177 does, it holds only model until this issue adds requires.
  9. SaulMoro commented on Oct 20, 2026

    @SaulMoro
    Collaborator

    #1086 is now blocked by this issue. Notes for the session skill driver:

    • Check requirements in leaseNext, the way feat(workflow): add workspace branches #1175 checks can.workspace. A lease the driver's can does not cover should wait for another driver. A driver that releases a lease gets it back on its next next, so a requirement the driver has to decline after leasing loops. feat(workflow): add workspace branches #1175 also drops unknown can keys as ignored (key !== 'hitl' && key !== 'workspace'), and this issue adds its capabilities there.
    • A session host can provide some isolation. A host such as Claude Code can create a worktree and run a subagent in it, which matches worktree in the three-level proposal above. The session's own agent runs in the member's checkout, which is none. Whether network and isolation end up strict or best effort, [feat] Add the session skill driver and invocable workflows #1086 follows the same rule. It declares only what the host provides, and with best effort it reports what it did not enforce.
    • Secret values must not reach the agent. A session agent sees everything it runs and reads. So a command step with requires.env needs a way to run with those values without the agent handling them. teamai env exec keeps values out of output, but it applies the whole team env, not only the step's requires.env, so it would break the withholding rule from my earlier comment. A prompt step cannot get secrets into the session agent's own process at all. Either env exec takes a lease's key list, which extends an existing command, or a driver that cannot scope env leaves env out of can and such steps wait.
    • Budget needs a pre-lease answer too. "Unsupported enforcement fails explicitly" fails a budgeted workflow on any driver that cannot measure a step's tokens, as noted above for T3 Code. If a session driver cannot measure the tokens a step used, a can entry for token reporting would let those leases wait for a driver that can, instead of failing the run.
  10. SaulMoro commented on Oct 20, 2026

    @SaulMoro
    Collaborator

    Notes from #1175 for this issue:

    • Setup must not leave its writes or its secrets behind. The local driver commits every path that is not ignored with git add -A at report (worktree.ts:287) and pushes it, so a setup that runs before the agent would have its writes committed as the step's change. Ignored files are not committed, but the agent can still read them. I'd run setup after the base overlay and team delivery and fail with setup_dirty before the agent starts if setup changed any tracked or non-ignored path, following read_only_dirty. Separately, setup removes any credential file it writes, ignored files included, and a known setup secret left in a file setup created or changed blocks the launch with setup_secret. Both codes need RECORDED_FAILURES. Criterion: setup writes its token to a gitignored .npmrc; the agent never starts, the value appears in no state or error, and nothing is pushed.
    • Scorer steps stay out of the work budget, but keep a timeout. The workflow token budget and the workflow timeout should not count internal scorer steps ([feat] Add workflow scorers and recorded quality labels #1085), and a spent budget must not turn a pending scorer into a run failure, since "a failed scorer does not change the judged step or the run". A scorer still needs a timeout of its own: a pending step keeps the run from settling (engine.ts:717), so a judge that hangs would hold the run open. A scorer that times out settles without a label.
    • A held-back step must not block other steps. Requirement checks in leaseNext would reuse the path where feat(workflow): add workspace branches #1175 holds back a workspace step, and that path stops at the first held-back step (engine.ts:925). The fix is F3 in [feat] Close the #1080 gaps that #1175 leaves open #1246, which this issue now waits for.
  11. added a commit that references this issue on Oct 24, 2026
  12. added a commit that references this issue on Oct 24, 2026
    90283a4
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions