Skip to content

feat(memory): select versioned LIBERO corpora by model - #202

Draft
ye1457 wants to merge 1 commit into
RLinf:mainfrom
ye1457:feat/libero-memory-versions
Draft

ye1457 wants to merge 1 commit into
RLinf:mainfrom
ye1457:feat/libero-memory-versions

Conversation

@ye1457

@ye1457 ye1457 commented Sep 17, 2026

Copy link
Copy Markdown
Contributor

LIBERO HF evaluation currently downloads one shared memory tree regardless of the planner model. Add --memory-version auto|GPT_5.5_xhigh|GPT_6_astra_low: automatic selection follows the resolved model (including Codex CODEX_MODEL and API provider prefixes), explicit versions override it, and unknown models visibly fall back to GPT-5.5. Memory selection never changes the running model or reasoning effort.

The downloader fetches only the selected corpus, validates the published file manifest, and isolates complete caches by repository, commit and version. CLI, Dashboard prompts/tools and Flash share the resolved root. Dashboard model changes take effect at the next task boundary; rpent-memory sync supports revision pinning and export to a local directory. Local/exploration modes retain local behavior and reject explicit remote versions. Legacy unversioned HF layouts map only to GPT-5.5; Flash also accepts its original task_card/ assets and rejects Astra's missing replay assets.

Paired English/Chinese guides document the two corpora, legacy-client migration and provenance: historical 741/800 used two original frozen Astra snapshots separately by suite, generated before the scene-seed fix. The merged release has not been reevaluated. Historical Flash evaluation coverage is distinguished from the 20 published Object Task/Swap plans actually present in the migrated corpus.

Validation:

  • pre-commit run --all-files: passed.
  • pytest tests/unit_tests -v: 601 passed, 1 skipped, using Python 3.11 and CPU Flywheel dependencies; the full Python-version CI matrix was not run locally.
  • English and Chinese sphinx-build -W --keep-going: passed.
  • Offline tests cover model/env/prefix selection, manual override, concurrent version isolation, pinned refs, corrupt/missing caches, interrupted downloads, manifest mismatch, local-mode conflicts, legacy layout/replay and Dashboard model switching.
  • Real download of legacy revision 21a62795fe3b7e500c8381ac47938f6d713ebe18: all 327 file hashes match the staged GPT-5.5 corpus. Current Flash loads all 20 preserved plan/anchor pairs.
  • Astra staging audit: 318 files, 79 byte-identical audit/recipe pairs, no duplicate experience IDs or broken experience links; both original snapshots match their freeze reports.
  • English/Chinese browser smoke tests: the next-task model control saves correctly with no JavaScript errors. No GPU evaluation or 800-episode rerun was performed.

Dataset changes are now submitted as Hugging Face data PR #12. The proposed tree contains both model directories, with original Astra snapshots preserved as separate commits. The HF PR lists exact revisions and the three archive tags maintainers should create before merging. Contributor PR upload succeeded; repository-tag creation and merging dataset main require a maintainer.

The code remains draft pending the coordinated dataset main migration, so default Astra selection will have a published directory to download. Reviewers can already exercise the uploaded data using rpent-memory sync --robot libero --memory-version GPT_6_astra_low --revision refs/pr/12. The dataset migration preserves GPT-5.5 bytes and retains both conflicting Astra experience variants with source suffixes. This PR is independent of leaderboard PR #165.

@read-the-docs-community

Copy link
Copy Markdown

Documentation build overview

📚 RPent-cn | 🛠️ Build #34612802 | 📁 Comparing ab32856 against latest (d6876ca)

  🔍 Preview build  

3 files changed
± rst_source/development/memory.html
± rst_source/usage/flash.html
± rst_source/usage/libero.html

@read-the-docs-community

Copy link
Copy Markdown

Documentation build overview

📚 RPent | 🛠️ Build #34612801 | 📁 Comparing ab32856 against latest (d6876ca)

  🔍 Preview build  

3 files changed
± rst_source/development/memory.html
± rst_source/usage/flash.html
± rst_source/usage/libero.html

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant