feat(scorers): judge workflow steps with team scorers

docs/dev/workflow-engine.md

@@ -20,7 +20,7 @@ Each rule below spans more than one file. The rule names the code that owns it,
 - **A step waits for every step it reads.** `readsOf` collects the steps that the step's expressions (`stepExpressions`), templates, and `with` read. For a step that runs an inline workflow, it also collects the outer steps that the inline steps read. `skipUnreachable` uses that set to decide readiness. If a new field reads steps, add it to `stepExpressions` or `templatedTexts`. Otherwise the step can read a value before it exists, or an inline workflow can wait forever. A step outside a loop that needs or reads a step inside it also waits until the loop owner is settled (`waitsFor` in `loop.ts`), and is neither skipped nor ready before.
 - **A read of a later step in the same loop is a previous-pass read, and never waits.** `previous` in `loop.ts` classifies it: the target is in the reader's loop range and needs the reader, directly or not. For an inline step, the reader is the step that runs its workflow. Pull allows the read, the engine leaves it out of the reads a step waits for, and `expressionSteps` gives the reader the target's latest archived pass (`previousPass`), which holds no pass fields on pass 1. Classify a new kind of read through `previous` too, or pull and the engine disagree on it.
 - **A step loop is decided once per run of its owner.** After checking for an end, `settleScope` calls `decideStepLoop` for each succeeded owner without `loopFinal`. A false `while` sets `loopFinal`. Otherwise the range runs again through `repend`, unless a `while` owner has run `max_iterations` times, which fails it `loop_exhausted`. A loop without `while` is bounded where the owner's action would start, after the skip rule and `if`, for a leaf and a workflow step alike (`exhaustedAtStart`). The owner counts its runs in `iteration`. Pull rejects ranges that share a step and loops that wait for each other (`loopProblems`), so the engine does not check them again.
-- **A pass id is never reused.** `repend` moves a step's current pass into its `passes` and bumps `pass`. `archiveUnder` moves what a repended workflow or `for_each` step ran into `run.archive`, and `firstPass` starts a new instance after every archived pass of its path. The full record of a run is `run.steps`, with each step's `passes`, plus `run.archive`; a run record, scorer, or budget reads both. `steps.<id>.passes` is a step's `passes` plus its current pass once it settled (`passesOf`). The expression projection keeps `results` for a `for_each` pass, including in `previousPass`. Pull checks the same outputs-or-results shape in step reads and workflow output aliases.
+- **A pass id is never reused.** `repend` moves a step's current pass into its `passes` and bumps `pass`. `archiveUnder` moves what a repended workflow or `for_each` step ran into `run.archive`, each pass stamped `within` with the step path and pass that archived it, and `firstPass` starts a new instance after every archived pass of its path. The full record of a run is `run.steps`, with each step's `passes`, plus `run.archive`; a run record, scorer, or budget reads both. `steps.<id>.passes` is a step's `passes` plus its current pass once it settled (`passesOf`). The expression projection keeps `results` for a `for_each` pass, including in `previousPass`. Pull checks the same outputs-or-results shape in step reads and workflow output aliases.
 - **An expression reads its absolute paths, wherever they are.** `expressionPaths` returns every `inputs` and `steps` path, including those inside function arguments and predicates, and never a relative path such as `.size`, which reads the current element of a predicate. `readsOf`, `skipUnreachable` and the pull read checks use it, so a new kind of expression must return its paths there.
 - **Expression types are checked in two places, through `typeProblems`.** `parseExpression` rejects what an expression shows by itself, such as `x < 'a'` or `any(...) < 1`. `readDiagnostics` checks paths against the types the workflow declares. It checks each path only in the workflow that owns it, so a predicate an inline step runs over an enclosing step is reported once.
 - **Pull and the engine share one list of templated fields, one list of step expressions, and one list of actions.** The templated fields are in `templatedTexts` and the expressions, such as `if` and `break`, in `stepExpressions`, both in `template.ts`. HITL declarations normalize in `parse-hitl.ts`. The actions are in `STEP_ACTIONS` in `types.ts`. Add a new field or action there, not to a copy in `parse.ts` or `engine.ts`. Pull, the cache's `prompt_file` check, the lease render and `readsOf` all split a template with `parseTemplate`, which decides which `{{ … }}` is a template, and the render evaluates it with `evaluateExpression`, as for an `if`. `parseReads` in `engine.ts` and the step parse in `parse.ts` key a table by each expression, so a new one does not compile until both say how to read it.
@@ -35,9 +35,10 @@ Each rule below spans more than one file. The rule names the code that owns it,
 
 - **A HITL request waits before any action or end.** `settleScope` checks loop exhaustion first, renders a HITL request message, and moves an unanswered step to `waiting`. Waiting is neither stopped nor settled. An answer moves it to pending without a revision bump. Approve no skips, choose succeeds, approve yes continues. An end-only step with HITL reaches its end only after yes. The shared `templatedTexts` makes HITL reads participate in readiness. Only `driver.can.hitl` declares HITL. `parseNextPayload` reads it into `NextOptions.hitl`; stored `DriverIdentity` and leases contain only kind, machine and workspace. Busy polls never answer waiting HITL requests through the no-HITL rules. Only a driver declaring HITL receives a waiting request; a busy poll without HITL receives wait. Both new leases and hand-backs enforce HITL for interactive steps. A refused hand-back fails at its lease revision and invalidates the attempt. The process-only autoApprove option is never read by the JSON payload parser. `respond --json` uses the same identity schema as next, reports unread fields through ignored, and records a driver UI's declared person answer with channel driver and agent false. The dashboard answers through the same `respondToHitl` with channel dashboard and agent false, always sending the pass it showed (`answerFromDashboard` in `src/dashboard/workflow-runs.ts`). Answer driver identity stays local; records publish only choice, channel and agent.
 - **Retry validates every container before resetting any.** `retryContainers` in `run-controls.ts` climbs from the target through scopes and fan-out items until a running runner or the top. Each dependency cone includes needs, reads and loop waits, or item after dependencies. An independent standing end or break rejects the retry. The apply pass archives replaced passes and descendants, reopens settled runners with `start` to keep inputs and round, then bumps revision and reconciles. Only the targeted loop owner's iteration resets; the top-level round does not. An answered refused approve or succeeded choose is retryable too, so a changed answer resets its dependent branches. A HITL request in the cone that is waiting, or answered and not yet run, starts a new pass too, so its answer is asked again in the new context and an answer to the old pass is stale. Unanswered skips in every affected container reset so skips caused by cancellation can run again. Retry's prelude reconciles without finishing a provisional outcome. Reopening clears the run's end, failure and endRecordQueued flag and removes its queued end file through records.ts. These three pieces stay consistent: run state, the pending end record, and published stats. The next end upserts the same record path; stats replace the origin end's outcome and duration contribution with the new end's, without adding another run. Replaying the same endedAt contributes nothing.
-- **The local driver keeps tools and questions interruptible.** A single question joins command and prompt exits in the driver's race. An interactive tool owns the terminal alone: no question opens while it runs, and starting one aborts and closes an open question. A poll closes the question when its own step is leased or a different step or pass waits; an unrelated lease leaves it open. It polls next every second while asking, every two seconds while tools run, and aborts a replaced HITL request or cancelled prompt. `process-stop.ts` shares process stopping between commands and adapters. Headless tools detach and stop with SIGTERM; interactive tools stay attached and stop with their captured descendant tree. Foreground interactive signals reuse env-exec's terminal detection and signal policy: when the terminal reaches the child, teamai ignores SIGINT and SIGQUIT, while SIGTERM and SIGHUP are forwarded and stop the driver. Interactive adapters inherit stdio and accept an optional result file. Repository edits are kept for interactive steps; only headless prompts run the read-only check. A nonzero interactive exit without a result reports its exit status. `leaseNext` records the identity of the latest driver that called next in `RunState.driver`, without capabilities. Resume uses this machine's open local lease workspace, then run.driver.workspace for a local driver on this machine, then cwd. Controls pass run.driver to finishRun so end records retain the driver kind.
+- **The local driver keeps tools and questions interruptible.** A single question joins command and prompt exits in the driver's race. An interactive tool owns the terminal alone: no question opens while it runs, and starting one aborts and closes an open question. A poll closes the question when its own step is leased or a different step or pass waits; an unrelated lease leaves it open. It polls next every second while asking, every two seconds while tools run, and aborts a replaced HITL request or cancelled prompt. `src/tool-launch/process-stop.ts` shares process stopping between commands and adapters. Headless tools detach and stop with SIGTERM; interactive tools stay attached and stop with their captured descendant tree. Foreground interactive signals reuse env-exec's terminal detection and signal policy: when the terminal reaches the child, teamai ignores SIGINT and SIGQUIT, while SIGTERM and SIGHUP are forwarded and stop the driver. Interactive adapters inherit stdio and accept an optional result file. Repository edits are kept for interactive steps; only headless prompts run the read-only check. A nonzero interactive exit without a result reports its exit status. `leaseNext` records the identity of the latest driver that called next in `RunState.driver`, without capabilities. Resume uses this machine's open local lease workspace, then run.driver.workspace for a local driver on this machine, then cwd. Controls pass run.driver to finishRun so end records retain the driver kind.
 
 - **One local driver owns a run on this machine.** `withDriverLock` in `local-driver.ts` takes `driverLockPath` once with `acquireLock` and releases it in finally. `runLocalDriver` drives inside it, and `workflow resume` also claims inside it, so a claim never lands under a driver already running the run. A live holder returns `driver_running`, including its pid, before anything is claimed or driven; the existing lock machinery reclaims dead owners. The driver lock spans tool execution and questions, while `withRun` holds a different run lock only during one engine operation. Neither lock's scope is extended by the other. Every HITL decision without a HITL driver and interactive hitl_required failure logs once when the engine changes the step. HITL replies carry one literal `respond` command template, which drivers display unchanged.
+- **Judge work never comes from `workflow next`.** `scoring.ts` computes open judge work on every `teamai score --next` from facts already on disk: the run files, the scorers `startRun` pinned in `run.scorers`, the `installId` that `reportAttempt` stamps on each pass it settles, the `within` stamp on archived passes, and the labels, attempts and on-demand requests (`teamai score --run`) under `<dataHome>/scores/`. Nothing else is queued. The engine has no scoring code path, and scoring never writes a run file, so a run's counts, tokens, failure and `endedAt` cannot move and it settles without waiting. A judge lease settles only through `teamai score`: `--lease <ref> --submit`, or `--lease <ref>` and `--run`, which run the local judge and settle through the same submit path. `workflow report` answers `judge_lease`. A driver never lists a judge ref in `workflow next`'s `running`, which holds step attempts only: judge refs go in `score --next`'s `running`, which renews them.
 
 ## Tests
 

docs/guide/reference.md

@@ -56,6 +56,7 @@ sharing:
         secret: my-signing-key
         timeout: 5000
         retries: 3
+
 ```
 
 ### config.yaml (local config)
@@ -74,6 +75,8 @@ gitExcludeEnabled: true
 contributeHintEnabled: false
 usageReportEnabled: false
 codexTrustEnabled: false
+scoring:
+  isolation: verified          # this scope's judges must be verified; the strictest of team, scorer and this wins
 toolRoots:
   claude: ~/.claude-work
   codex: ~/.codex-alt