Complete Rust orchestration migration - #7
Draft
areshand wants to merge 16 commits into
Draft
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What changed
multiagentbinary.launch.shis the only retained source-checkout bootstrap and immediately executesmultiagent launch.Why
The prior split across Rust, Python, and shell duplicated command routing and state transitions. The SWE adapter also behaved like a second verifier: a potentially scoreable patch could be discarded because an agent omitted adapter-specific evidence markers. This migration creates one locked production writer and one command surface while preserving the v1 durable-state format and tmux interactive behavior.
The adapter now organizes the submission only: it launches the production workflow, waits for the orchestrator lifecycle, materializes worker commits, marks all untracked files intent-to-add, and returns the workspace to EvalScope. It neither validates nor pre-accepts the patch.
Validation
cargo test --locked— 14 passedbash tests/run.sh— full Rust, shell, lifecycle, evaluation, provenance, and migration suite passedgit diff --checkpassedRegression tests verify that orchestrator exit prepares the workspace for the official scorer, committed worker changes are materialized, and every untracked source or test file is included in the EvalScope diff. Contract assertions also prevent reintroduction of adapter evidence parsing, rejection diagnostics, validation wrappers, and status files.
First-10 SWE-bench Pro regression
Exact-revision rerun: solver
f15f225e05163bacf99296c83d64a75e5e99cd96, EvalScope 1.8.1 (fce1d21), SWE-bench Proca10a60,gpt-5.4, two shards of five.0.5), unchanged from the preceding first-10 migration run.0.6; indices 5–9 scored0.4.The rerun confirms no aggregate score regression and preserves the pass-through boundary. It also exposed a packaging concern rather than an evaluation concern: under the intentional no-filter policy, repository-local runtime artifacts such as Redis AOF files and
patch.txtare included in the submitted diff. Correctness remains exclusively the official verifier's responsibility; framework artifacts should preferably be written outside/apprather than introducing semantic validation into the adapter.