Add bounded parallel reconstruction, Grok Build, and streaming agent monitoring

.gitignore

@@ -13,3 +13,10 @@ build/
 
 .DS_Store
 .coverage
+
+# Local reconstruction state and generated reports
+/reports/re-agent/
+/re-agent-progress.json*
+*.execution.json
+*.execution.stop
+*.coordinator.lock

README.md

@@ -1,5 +1,7 @@
 # ReAgent
 
+For a decoupled local progress window with optional worker controls, see the [live monitor guide](docs/live-monitor.md).
+
 [![PyPI](https://img.shields.io/pypi/v/auto-re-agent)](https://pypi.org/project/auto-re-agent/)
 [![Python](https://img.shields.io/pypi/pyversions/auto-re-agent)](https://pypi.org/project/auto-re-agent/)
 [![CI](https://github.com/Dryxio/reagent/actions/workflows/ci.yml/badge.svg)](https://github.com/Dryxio/reagent/actions/workflows/ci.yml)
@@ -243,6 +245,23 @@ Claude CLI supports real session resume and reports usage/cost metadata. A
 stale CLI login can still require re-authentication even when its auth-status
 command reports a session.
 
+### Grok Build CLI
+
+Authenticate with `grok login`, then configure:
+
+```yaml
+llm:
+  provider: grok-cli
+  model: "" # Use Grok Build's configured model, or specify a model ID.
+  cli_path: grok
+  timeout_s: 600
+```
+
+Supports native session resume, JSON response validation, usage metadata,
+and tool-free requests from an isolated working directory. Large evidence
+prompts use temporary files. See [Grok Build configuration](docs/grok-build.md)
+for supported settings and limits.
+
 ### OpenAI-compatible APIs
 
 ```yaml
@@ -524,3 +543,9 @@ mypy src
 ## License
 
 MIT
+
+### Bounded parallel function processing
+
+Class and manifest runs support provider-independent concurrency with isolated
+workers, durable recovery, and live progress. Sequential execution remains the
+default. See [configuration and behavior](docs/parallel-functions.md).

docs/grok-build.md

@@ -0,0 +1,50 @@
+# Grok Build CLI provider
+
+Install Grok Build and authenticate using its own `grok login` workflow. ReAgent uses the local CLI login; it does not read or store Grok credentials.
+
+```yaml
+llm:
+  provider: grok-cli
+  model: "" # Use Grok's isolated default, or supply a model from grok models.
+  cli_path: grok # An absolute path to grok.exe also works on Windows.
+  timeout_s: 600
+```
+
+Set `model` explicitly, including an empty string to use Grok's built-in default. Omitting it inherits ReAgent's general default model, which is not a Grok model.
+
+The same configuration can be used under `agents.reverser` or `agents.checker` for mixed-provider runs. Optional `effort` is forwarded to Grok's `--effort` argument.
+
+The adapter uses `--prompt-file` for large evidence inputs, native session IDs for repair conversations, and native JSON output (`text`, `sessionId`, `stopReason`). It requires a completed `end_turn` response and rejects empty, malformed, errored, or incomplete results. Usage and cost fields are retained in call metadata when supplied by Grok.
+
+Requests run in a temporary working directory and a separate `GROK_HOME` per provider.
+`GROK_AUTH_PATH` points Grok at the existing login store (respecting an explicit override);
+ReAgent does not read, copy, or log credentials. Global user plugins, MCP configuration,
+and compatibility discovery are not inherited. Local `requirements.toml` and
+`managed_config.toml` are preserved, and system policies still apply. Custom user
+model/default/endpoint and authentication-helper configuration is not copied; select
+an explicit supported model and use supported environment-based endpoint/auth settings
+when necessary. An enforced managed integration may still initialize.
+
+The adapter selects `read_file` with `--tools`, then removes it together with the
+always-on MCP meta-tools using `--disallowed-tools read_file,search_tool,use_tool`.
+This deliberately avoids the empty allowlist behavior observed in Grok Build 1.0.25,
+which left built-ins available. A deny rule, disabled web search, and disabled
+subagents remain additional restrictions. `--max-turns 1` is unchanged. In a native
+reconstruction trace, the old filter produced tool calls and hit that limit; the
+explicit filter completed with `end_turn` and one model turn.
+
+Grok SDK retries are disabled in the isolated configuration. Native error JSON retains
+stop reason, turn count, and reported usage in call metadata, even on nonzero exit.
+Provider cleanup removes its temporary home and sessions; ReAgent's configured call
+logs and reports persist separately. Original Grok sessions and user settings are unchanged.
+
+References: [headless tool filtering](https://github.com/xai-org/grok-build/blob/main/crates/codegen/xai-grok-pager/docs/user-guide/14-headless-mode.md),
+[configuration](https://github.com/xai-org/grok-build/blob/main/crates/codegen/xai-grok-pager/docs/user-guide/05-configuration.md),
+and [native auth path resolution](https://github.com/xai-org/grok-build/blob/main/crates/codegen/xai-grok-login/src/storage.rs).
+
+
+The shared subprocess runner terminates the child process tree on timeout. ReAgent's `max_tokens`, `temperature`, and API-key settings are not forwarded to this CLI. `max_budget_usd` is rejected because the adapter cannot enforce it through the supported CLI flags. Configure account/model limits in Grok Build.
+
+Validated with Grok Build 1.0.25 on Windows, including a live two-turn session. Unit tests run without a Grok installation or network access.
+
+Official CLI documentation: https://docs.x.ai/build/cli/headless-scripting

docs/live-monitor.md

@@ -0,0 +1,94 @@
+# Live progress monitor
+
+`re-agent monitor` serves a local dashboard independently of the reversal worker. It reads existing session JSON files and logs; it does not import model providers or start work automatically.
+
+```sh
+re-agent monitor --work-dir /path/to/project --session-glob re-agent-progress.json --total 500
+```
+
+Open the printed `http://127.0.0.1:8765` address. The page refreshes every two seconds and shows saved function results, review passes, failures, rounds, and recent log output. Omit `--total` when the planned count is unknown. A review pass is not a proof of compilation or binary equivalence.
+
+Session patterns are relative to `--work-dir` and can be repeated. For a batch runner:
+
+```sh
+re-agent monitor --work-dir /path/to/project --session-glob 'reports/batch-*/progress.json' --log-glob 'reports/batch-*/stderr.log' --total 500
+```
+
+The monitor deduplicates function addresses across matching files using their latest result timestamps. Select sessions for the same binary/version; unrelated projects can reuse addresses. Unchanged files are cached, partial JSON writes retain the last readable snapshot, and log output is bounded to the last 7,000 bytes. Counters describe saved final results, not in-flight rounds.
+
+## Optional worker controls
+
+Install the optional process-management dependency:
+
+```sh
+pip install 'auto-re-agent[monitor]'
+```
+
+Supply an explicit argument array after `--worker`, which must be the last monitor option:
+
+```sh
+re-agent monitor --work-dir /path/to/project --session-glob re-agent-progress.json --total 500 --worker re-agent --config re-agent.yaml reverse --manifest targets.json --max-functions 500
+```
+
+On Windows, quote paths containing spaces and use an executable such as `python.exe` or `re-agent.exe`, rather than a shell script. Worker arguments are passed directly, without shell interpretation.
+
+- **Start / resume** launches the configured command. Resuming completed work depends on that command's checkpoint behavior. ReAgent uses its configured session and attempt limits; the monitor does not reset them.
+- **Stop run** terminates the owned worker process tree. Saved session files remain intact. An interrupted model call may need to be repeated.
+- Closing the browser or monitor host leaves the worker running. Restart the monitor with the same work directory, state directory and worker command to reconnect.
+- Duplicate starts sharing a state directory are protected by a process lock. Adoption checks the command, working directory record, PID and process creation time, so stale PID records do not attach to unrelated processes.
+
+Control records and worker stdout/stderr live in `reports/monitor` by default; override with `--state-dir`. Treat these as local run artifacts, not source files. `--port` changes the port; zero selects an available port.
+
+## Local access
+
+The server binds only to IPv4 loopback. Host-header checks reject DNS-rebinding requests. Control endpoints require a random per-host token and reject cross-origin requests. HTTP clients cannot supply a different worker command or arbitrary file path. No external scripts, fonts, telemetry or services are used by the dashboard. This is a local tool, not a multi-user network service.
+# External batch progress
+
+Use the existing dashboard for a separately launched batch runner:
+
+```console
+re-agent monitor --work-dir /path/to/run --progress-file status.json --stop-file STOP --log-glob "batch-*/stderr.log"
+```
+
+Both file paths must stay within the working directory. The monitor does not adopt
+or restart the external process. Stop creates the configured cooperative signal;
+the runner must watch it and cancel its own children. Omit `--stop-file` for
+read-only reporting. To enable Start / resume, also configure `--worker` (last option) and `--stop-file`.
+The monitor clears the stop signal before launching its managed worker and refuses
+a duplicate launch while the recorded process exists. The runner must implement
+checkpoint recovery; the monitor does not reinterpret its saved work.
+
+The progress JSON uses `phase`, Unix-second `started`, `updated` and
+`batch_started` timestamps, and integer `total`, `completed`, `compiled`, `failed`,
+`verified`, `batch`, `batches`, `child_started`, `child_returned` and
+`active_children` counters. `recent` contains rows with `address`, `compiled` and
+optional `diagnostic`; optional `error` describes a run-level failure.
+Active phases are `opening-analysis`, `exporting-evidence`, `native-subagents` and
+`validating-candidates`. Updates older than 60 seconds are marked stale; this is
+a freshness indication, not proof that a process has exited.
+
+Compiled drafts are explicitly distinguished from accepted reconstructions.
+Child counts describe starts and collected results, not measured model-request
+concurrency. The existing layout displays elapsed time, throughput, batch
+progress, diagnostics and source-data age. The browser receives an initial snapshot over `/api/stream`, followed by WebSocket
+updates. Reconnects load a fresh snapshot; no periodic browser status requests are used.
+The host checks local progress files every 500 ms. Controls remain authenticated HTTP POSTs.
+
+Use `--event-glob "batch-*/native.jsonl"` to enable the agent workspace for native
+Grok event logs. Select the live batch or an earlier batch, then an agent to inspect
+its code, full response, and tool activity. Unattributed token events stay in a
+shared stream; child results are assigned only by provider task IDs. Thought events
+are excluded. Each agent text/log tail is bounded to 64 KiB, with at most 128 recent
+agents retained. Historical reads are limited to 32 MiB per source. The source must
+match the configured pattern inside the run directory.
+
+## Native Windows session storage
+
+Keep native CLI session storage separate from deeply nested report directories.
+Grok embeds the encoded working directory and two session IDs in child output
+paths. On Windows, native callers can pass an explicitly allocated short `home`
+to `GrokCLIProvider._isolated_environment(..., native_subagents=True)`. The helper
+checks a conservative 240-character output-path budget before creating the home.
+It preserves the original auth path and managed requirements without copying
+credentials. The caller owns the separate home and its retention/cleanup. Use a
+new isolated directory for a clean run; existing homes are never overwritten.