Repository navigation
Conversation
Four things kept an opencode agent from producing a single training step: - middleware: after replaying the rewritten request body, receive() kept fabricating empty http.request messages; Starlette's streaming response polls receive() for the disconnect, so every streamed completion hung. - watcher: LiteLLM strips reef's receipt frame from streamed responses and drops the record-id header, so reports carried zero references. Resolve references from the tags reef stores with each record as well. - processor: a coding agent's call sequence is not one linear prompt extension, so the multi-turn assembler returned None and the attempt was dropped as a tensor contract violation. Fall back to the terminal call and report the actual violation text. - group_by=release: single-agent evolution is a chain with a new parent every attempt, so parent-hash groups never reach group_size. run.py gains --task/--scenario so any CORAL task package runs.
…outing serve-coder30b.yaml (LoRA, tensor/expert parallel 8, qwen3_coder tool-call parser, 128k sequence length, group_by=release, per-run checkpoint storage cap) and a frozen-policy control whose group size never releases. litellm_config.yaml routes through hosted_vllm so opencode's Responses API calls become chat completions, with the model aliases opencode asks for. Megatron LoRA export name mapping, LoRA name fallback and the slime weight_version kwarg are needed for the 30B-A3B MoE stack to boot and update weights. The config test allowlist gains the Megatron MoE flags.
…3-Coder-30B-A3B Frozen-policy and test-time-trained arms, one and four agents, next to the paper's Table 1 SOTA and Table 2 MiniMax M2.5 columns. Every graded attempt, every report posted to reef and the final reef status are retained per run; collect.py and compare.py rebuild the table. Single seed per arm.
…up_by=agent Analysis of the paper-task runs: 92% of consecutive calls extend the previous prompt exactly, but 68% of attempts contain at least one fork (history compaction, a re-rendered system prompt, or an interleaved helper call), so nearly every attempt fell back to its terminal call and only 3-4% of an attempt's calls were trained. Whether a suffix assembles is monotone in its length; binary-search the longest one. On the recorded runs this keeps 25-34% of calls instead of 3-4%, a median of 54-68% of each forked attempt. status() reports kept/total. A default scaffold_tolerance of 8 realigns the few-token re-render of Qwen3-style assistant turns instead of treating it as a fork. group_by=agent groups one agent's attempts under one policy release, so multi-agent runs compare each agent's own lineage rather than mixing agents that started from different commits.
…options Since #424 the launcher records hf-checkpoint as ${reef.model_path} and the runtime config is written un-interpolated, so the driver handed the literal placeholder to Slime and every managed training deployment failed at model load. Interpolate the native options against the runtime config.
A restart used to start a new CORAL run from the seed, discarding the worktree, notes and attempt history. --resume hands the latest run's paths to AgentManager.resume_all; --fresh-sessions drops the saved opencode sessions so an agent that ended on its own 'task complete' summary starts a new session in the same worktree instead of re-reading it. The reef run id and the watcher's reported-set persist across the restart.
…date suffix On main a ValueError from make_sample is a training data error that stops the scenario's training, so the 'spans releases' rejection took the whole TTT run down the first time an agent's attempt straddled a LoRA release, which is routine. Keep the calls made under the release that produced the graded submission (a suffix, since calls are time-ordered) and count the truncation in status().
|
Merge Oncall: @BobbyZhouZijian The merge oncall will route reviews and keep this pull request moving toward a clear decision. |
github-actions
Bot
added
area: docs
Xuan-1998
force-pushed
the
coral/ttt-improvements
branch
from
September 14, 2026 15:10
1a7cb83 to
cda38d3
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Follow-up to #444; this branch includes its commits, so the diff against main shows both until #444 lands. The paper-task runs showed the recipe was training on almost nothing: nearly every attempt fell back to its last call, so of an attempt's 20-30 model calls only the final response reached the gradient. This branch fixes that and the two things that broke a run outright, and adds the pieces I needed to keep a single agent going.
Looking at the recorded runs, 92% of consecutive calls extend the previous prompt exactly; forks come from three places, opencode compacting history (prompt drops from 99k to 45k tokens), a re-rendered system prompt, and its small title-agent calls mixed into the sequence. Whether a suffix of the call sequence assembles is monotone in its length, so the processor now binary-searches the longest one instead of taking the last call. On the recorded traces this keeps 25-34% of calls (median 54-68% per forked attempt) versus 3-4% before, and a
scaffold_toleranceof 8 stops the few-token re-render of Qwen3 assistant turns from counting as a fork.status()reports kept/total.group_by=agentgroups one agent's attempts under one release, so multi-agent runs stop mixing agents that started from different commits.Two fatal paths. Since #424 the launcher records
hf-checkpointas${reef.model_path}and the runtime config is written un-interpolated, so the Slime driver handed the placeholder totransformersand every managed training deployment died at model load; the driver now resolves references in its native options. And on current main aValueErrorfrommake_samplestops the scenario's training, so the recipe's "attempt spans releases" rejection killed the whole TTT run the first time an attempt straddled a LoRA release, which is routine; the processor now keeps the calls made under the release that produced the graded submission instead.run.py --resume [--fresh-sessions]continues the latest CORAL run in place (worktrees, notes, attempt history) instead of starting from the seed; fresh sessions matter because a resumed opencode session ends with the agent's own "task complete" summary and just repeats it.One check on the paper's Polyominoes task (Frontier-CS algorithmic #0), single agent,
group_size: 8, 40 attempts, Qwen3-Coder-30B-A3B: frozen policy best 1.08/100 over 43 evals; trained policy best 56.84/100 over 39 evals (paper: SOTA 87.0, CORAL 1-agent 80.2), reef status at the end: 4 LoRA steps, suffix fallback kept 208 of 537 calls, 2 release truncations. One run each, so treat it as a smoke signal, not a result; the recorded events are underresults/2026-09-14-polyomino-single-agent/.Tests: new cases for the suffix fallback, the fork-at-last-call case,
group_by=agent, the release truncation and the driver interpolation; the coral, sao-config, config-layout and training-startup suites pass.AI assistance: driven with an AI coding assistant (analysis of the recorded traces, code, runs, this text); I reviewed everything.