Skip to content

feat(leaderboard): add Qwen results and time/token costs - #64

Draft
ye1457 wants to merge 2 commits into
RLinf:mainfrom
ye1457:feat/leaderboard-qwen-time-token-costs
Draft

ye1457 wants to merge 2 commits into
RLinf:mainfrom
ye1457:feat/leaderboard-qwen-time-token-costs

Conversation

@ye1457

@ye1457 ye1457 commented Sep 18, 2026

Copy link
Copy Markdown
Contributor

Summary

  • Add RPent / Qwen3.6 27B / no-reasoning to LIBERO-PRO: 565/800 (70.63%), with all eight independent suite results. Leave the unreported planner unset.
  • Add a same-page Time & Token Costs section after the four success leaderboards. It contains 11 contributor-confirmed rows grouped by LIBERO-PRO, RoboCasa365, and RoboTwin, with configuration, mean episode time, and total output tokens only.
  • Keep costs separate from success records. Do not restore previously removed GPT-5.6 or RoboTwin Astra success results; omit Opus cost rows and preserve Flash Mode's real zero tokens.
  • Sort costs ascending by time or tokens and export the current order. Success tables remain descending. Update bilingual light/dark overview images and the standalone no-JavaScript fallback.
  • Use Method / 方法 headers and the same two-line method/details layout in success and cost tables. GPT-5.6 is GPT-5.6 Sol, with Codex and either xhigh · reasoning or no-reasoning. Qwen uses no-reasoning in both languages. All numeric results and costs remain unchanged.

Accounting: time averages include successful, failed, and timed-out evaluation episodes; tokens are evaluation-stage output tokens only. Data comes from contributor-confirmed results, not new experiments in this PR. All 171 previous success records are unchanged; the dataset now contains 180 records across the same 19 views.

Validation

  • Exact Qwen counts and all 11 cost tuples checked; existing records compared field-for-field against misc main.
  • 1,928 focused browser checks: bilingual desktop/mobile, light/dark and live switching, all nine Qwen views, method/planner labels, keyboard sorting, current-order CSV, zero values, offline and no-JavaScript/blocked-resource fallbacks.
  • 629 existing leaderboard browser checks; 290 theme/fallback checks; 65 candidate-at-main resource checks.
  • Four overview images regenerated using the actual page themes; no method logos added.
  • Both full RPent Sphinx builds pass with -E -a -W --keep-going; RPent CPU regression: 581 passed, 3 skipped. No robot experiments rerun.
  • Known existing Sphinx bootstrap.js error (missing .bd-header during back-to-top initialization) is recorded separately; not claimed fixed by this presentation change.

Review / Rollout

Companion documentation: RLinf/RPent#205.

Chinese overview with Qwen | English overview.

Keep Draft until the contributor approves the Chinese local preview. This PR contains all presentation code, data, and images. The paired RPent PR only updates bilingual native documentation tables and retains existing misc@main URLs. Merge this resource PR first, verify the public resources, then merge RPent. No automatic merge or Ready transition.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant