English | 简体中文

AI that accelerates the research that advances AI.

ResearchBoost-Agent turns AI capabilities into faster AI experimentation, starting with the training code researchers run every day. It autonomously profiles bottlenecks, implements optimizations, and validates speedups against a frozen evaluation protocol, turning the same compute budget into more room to explore.

🚀 Project Blog & Wishlist Read the story behind ResearchBoost and our training speedups. Have a project you want accelerated? Submit it to our Wishlist.

Quick start

Give the agent access to a model endpoint:

cp goal/codex.env.example goal/codex.env
# edit goal/codex.env: set OPENAI_API_KEY and OPENAI_BASE_URL (optionally CODEX_MODEL)

Then point your agent at docs/run-a-task.md.

How a run works

1. Frame the task. The owner fixes what may not change: the upstream workload and per-GPU load, the baseline precision, the training semantics. A formulating agent builds the judge for this task (bench/ plus a frozen copy of numerics/), records the baseline, freezes δ, proves with known-bad candidates that every gate fires, and writes GOAL.md.
2. Run goal mode. One Codex /goal session gets the workspace, the GPUs and the skill cards inside a bubblewrap sandbox. Every round it profiles the current best version, changes one hotspot behind a switch, measures with bench.py time, and logs the round in WORK_LOG.md. A supervisor watches from outside and intervenes only on rules and facts, never on optimization direction. See docs/goal-mode.md.
3. Accept only what the judge accepts. bench.py check is the only verdict. A full-check PASS is then confirmed with a real end-to-end training or evaluation run.

Results

Every number below was printed by that task's frozen judge, or comes from a full training or evaluation run of the accepted candidate.

Task	Upstream	Hardware during optimization	Judge verdict	Speedup	Code
RAE Stage-1 training (decoder + GAN)	bytetriper/RAE @a4d18c4	2 × H200	PASS, statistically equivalent	5.32× step time; full 16-epoch run on 8 × H20: rFID 0.541 vs 0.569 for the public checkpoint	Faster-RAE
Self-Forcing video DMD training	guandeh17/Self-Forcing @33593df	2 × H200	PASS, statistically equivalent	2.08×	Faster-SelfForcing
LLaVA-1.5-7B stage-2 fine-tuning	haotian-liu/LLaVA @c121f04	4 × H20	PASS, statistically equivalent	2.39×; real 1-epoch run: four benchmarks within −0.50 to +0.44 of the official weights	Faster-LLaVA
DMD2 one-step SD1.5 distillation + GAN	tianweiy/DMD2 @8d8fa55	4 × H20	PASS, statistically equivalent	1.72×; FID-30k 8.70 vs 8.65 on 8 × H20	Faster-DMD

The judge

A candidate counts only if the task's frozen judge accepts it. The judge runs these gates in order and stops at the first one that fails:

0. Structure: the training entry points the judge drives are unchanged (AST compare), data loading is bitwise identical, and exact items match: phase, call counts, RNG fingerprints, learning rate, cross-rank sync.
1. Benefit: baseline → candidate → baseline timing in independent processes. A speedup that does not exceed max(1.02, 1 + 2 × baseline spread) is rejected as no benefit; precision is not evaluated.
2. Bitwise: every tensor recorded in short deterministic cases matches the baseline bit for bit → accept.
3. Statistical: otherwise, paired-seed training trajectories; every endpoint must stay within δ, twice the baseline's own seed-to-seed standard deviation, frozen before any candidate is seen.

The judge is frozen and hashed before optimization starts and is never relaxed to let a candidate through. Details: docs/judge.md.

Starting a new task

Read AGENTS.md and docs/methodology.md, give the formulating agent goal/templates/FORMULATOR_BRIEF.md, and use tasks/dmd2 as the template: GOAL.md, a bench/ with check / time / profile, a task adapter for numerics/, and the prep scripts that rebuild the environment and freeze the baseline.

Repository layout

Path	Contents
docs/	Methodology, the judge, goal mode, running a task, lessons from four tasks
goal/	Running one Codex /goal session: bubblewrap sandbox, Codex config template and model-access file, launch / resume / watchdog scripts, the brief for the formulating and supervising agent
numerics/	The equivalence component every task judge copies and freezes: recording, bitwise and tiered comparison, paired-seed trajectories, the task adapter base class, a self-test
skills/	Eight skill cards read before choosing what to try: train-opt-* (profiling, measurement, action ladder, numerics validation, fusion patterns, FSDP multi-forward) and sglang-diffusion-* (diffusion benchmarking and performance)
tasks/	Per task: the GOAL.md the agent saw, the frozen judge code, archived judge records, supervision notes and the prep scripts

Acknowledgements

We thank the SGLang project. Part of the kernel-optimization skills in this repository are ported from SGLang: skills/sglang-diffusion-benchmark-profile and skills/sglang-diffusion-performance are adapted from the skills in python/sglang/multimodal_gen/.agents/skills, released under the Apache-2.0 license.