Skip to content

docs(leaderboard): add Qwen scores and cost summaries - #205

Draft
ye1457 wants to merge 3 commits into
RLinf:mainfrom
ye1457:docs/leaderboard-qwen-time-token-costs
Draft

ye1457 wants to merge 3 commits into
RLinf:mainfrom
ye1457:docs/leaderboard-qwen-time-token-costs

Conversation

@ye1457

@ye1457 ye1457 commented Sep 18, 2026

Copy link
Copy Markdown
Contributor

Summary

  • Add Qwen3.6 27B / no-reasoning to the English and Chinese LIBERO-PRO summary and eight-suite fallback tables: 565/800, 70.63%; Spatial 82/78, Object 83/84, Goal 68/68, Long 61/41 (Task/Swap).
  • Group the four success-rate environments under Performance and place Time & Token Costs after it, with 11 rows and only method, mean seconds per episode, and evaluation-stage output tokens. No Opus cost row or new success-rate records for cost-only configurations.
  • Fix cost table/CSV method order to GPT-6 Astra, GPT-5.6 Sol reasoning, GPT-5.6 Sol no-reasoning, RPent Flash Mode, Qwen, skipping absent methods. Performance ranking remains success-rate descending. Align bilingual native fallback grouping with the interactive page.
  • Preserve all existing success results and memory notes. Keep presentation code, JSON, and images exclusively in misc; no new runtime code, source-directory README, or media in RPent.
  • Keep the formal resource references on misc@main, as requested. Local candidate previews load the proposed misc resources; public main is not claimed to contain them before merge.
  • Use Method / 方法 and two-line method/planner details in cost tables, matching the success tables. Label the model GPT-5.6 Sol with Codex xhigh · reasoning or no-reasoning; keep Qwen's no-reasoning spelling in both languages. No numerical changes.

Validation

  • Clean-output English and Chinese Sphinx builds: sphinx-build -E -a -W --keep-going.
  • pre-commit run --all-files: passed.
  • Preceding PR validation: python -m pytest tests/unit_tests -q: 581 passed, 3 skipped (existing Starlette/AnyIO deprecation warning). This follow-up only changes native documentation grouping and method order.
  • Field-level data check: 171 old success records unchanged; 9 Qwen additions; 11 exact cost rows including Flash Mode zero tokens.
  • Candidate integration: 1,932 Qwen/cost browser checks; 629 existing leaderboard checks; 296 theme/fallback checks; 74 method-layout checks; 65 routed-main resource checks. Two-section navigation, fixed-order cost CSV, bilingual method/planner labels, desktop/mobile, dark/light, offline and blocked-resource/no-JS fallback covered.
  • 112 pointer/keyboard back-to-top checks pass across both renderers and offline pages.
  • Ordinary pages do not request leaderboard JS/CSS/JSON. Known upstream Sphinx bootstrap.js initialization errors are recorded separately, not counted as fixes here.
  • Legacy URL redirects verified against the actual docs: preserve legacy Leaderboard links and fragments #204 RTD preview (4 checks). docs: preserve legacy Leaderboard links and fragments #204 is still separate and must merge for these redirects to exist on main; this PR does not duplicate its implementation.

Rollout

Depends on RLinf/misc#64 for the interactive renderer, shared data, and updated images.

Candidate Chinese overview. The RTD interactive page uses published misc@main; it will reflect these additions after the misc PR merges. The new native tables are already present in this branch.

Keep Draft pending approval of the Chinese preview. Merge the paired misc Qwen/cost PR first, verify public resources, then merge this PR. Do not turn Ready or merge automatically. This update changes published presentation only, not robot evaluation code, memory, or experiments.

@read-the-docs-community

read-the-docs-community Bot commented Sep 18, 2026

Copy link
Copy Markdown

Documentation build overview

📚 RPent | 🛠️ Build #34649295 | 📁 Comparing 5a10279 against latest (886b3b2)

  🔍 Preview build  

1 file changed
± rst_source/leaderboard.html

@read-the-docs-community

read-the-docs-community Bot commented Sep 18, 2026

Copy link
Copy Markdown

Documentation build overview

📚 RPent-cn | 🛠️ Build #34649296 | 📁 Comparing 5a10279 against latest (886b3b2)

  🔍 Preview build  

1 file changed
± rst_source/leaderboard.html

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant