Fix to ros2 action list - #193
Closed
joaoluiz-godoy wants to merge 45 commits into
Closed
Conversation
Register finetuning and local inference as GPU courses so the workshop notebooks can run through AUP Learning Cloud. Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Run the RAI demo as ipywidgets in a notebook cell instead of a separate Streamlit page, so it no longer needs a second browser tab. The agent, the O3DE bridge and the chat run in the kernel; camera frames are fetched kernel-side and pushed into an Image widget. Also: add --infra-only to the demo script, SIGKILL web_video_server by name on cleanup (VIDEO_PID was the "ros2 run" wrapper, so the server was orphaned holding port 8080), add a watchdog for when it wedges, and give user pods a 2Gi /dev/shm for Fast-DDS.
Add a Behind the scenes section after the demo: an architecture diagram, the agent's LangGraph, the ROS 2 topic list, and the system prompt RAI generates from the embodiment file. Fix the chat log overlapping the prompt box for manipulation demo.
Lemonade and rai notebooks
molmoact2 finetuning and inference notebook
Bring in 195 commits from AMDResearch develop (installer auth modes, GPU-access via host udev + fsGid, plugin/skills tooling, spawner and overlay updates, etc.) while keeping the roscon26 course additions (Finetuning + LocalInference images, notebooks, scripts, /dev/shm volume). Conflicts resolved to develop: - runtime/hub/core/spawner/kubernetes.py: develop sets restartPolicy=Never more robustly later in start(); dropped the redundant roscon26 one-liner. - runtime/values.yaml: adopted develop's fsGid:100 GPU-access model (host udev controls device modes) over the supplemental render-group tweak. Co-authored-by: Cursor <cursoragent@cursor.com>
Fastwam notebook inference and demonstration added on top of existing molmoact2 fine-tuning and inference notebooks. Same build style, same source folder, builds automatically.
Notebook 04 runs CaP-X against a locally served model: it configures LaunchArgs and ModelQueryArgs directly, generates one program, executes it in Robosuite, and benchmarks the success rate over five layouts. capx_demo.py holds the setup and reporting around it. The LocalInference image gains the CaP-X stack in its own venv and Jupyter kernel, with the gated SAM3 weights pre-fetched at build time so nothing needs a HuggingFace token at runtime. lemonade_env.sh gains a --serve-only flag, so the notebook can reuse its model-serving half without the RAI config rewrite and ROS overlay that follow it, and its wait loop is now bounded.
Bring in the CaP-X notebook and Lemonade demo port-note fix.
Stage hands-on.ipynb, headless rendering helpers, the Brax checkpoint archive, and a Dockerfile that pre-installs the inference stack and extracts the checkpoint for the course image.
Drop pixi and notebook install cells. Install Python and headless GL libraries in the Dockerfile via pip and apt, and teach headless_gl.py to use system Mesa.
Add PandaPickCube inference demo to RLLearning
fix: install RL Learning deps as root with python3
Demonstrate repository-as-policy evolution locally with HELIX, OpenCode, and CaP-X while preserving a workshop-safe runtime and explicit fallback. Co-authored-by: Cursor <cursoragent@cursor.com>
Pre-load PandaPickCube (and MuJoCo Menagerie) in the Docker build so jovyan can run registry.load() without writing under site-packages. Add demo-quiet helpers to suppress optional Warp probe prints and benign JAX RuntimeWarnings during the hands-on notebook rollout.
fix(rl-learning): bake menagerie and quiet demo warnings
Bake both notebook models into the image, pinning gpt-oss to the MXFP4 file so the build stops pulling the EAGLE3 drafts that made llama-server segfault on load. Route the notebook's noisy setup and trial output through a log file, slow the episode videos to a watchable frame rate, and broaden the benchmark to six Robosuite tasks.
Restructure hands-on.ipynb into a three-stage weak→mid→strong rollout demo with audience-friendly labels, add maintainer docs and a one-off render helper, and ignore generated trajectory/video artifacts.
Cache ungated perception and LLM assets, add reproducible CaP-X/RHO/RAI workflows, and validate fast ROCm model startup so the workshop runs locally without gated downloads. Co-authored-by: Cursor <cursoragent@cursor.com>
Use the validated Gemma E2B path to remove the 17.3 GB Qwen cache, embed concise timing and rollout evidence, and support bounded multi-file task evolution without noisy service logs. Co-authored-by: Cursor <cursoragent@cursor.com>
Keep cube stacking as the workshop example because it was the only tested task to produce a completed policy. Co-authored-by: Cursor <cursoragent@cursor.com>
Add robust repeated hidden evaluation and a navigable notebook sequence so the workshop demonstrates repository evolution without relying on single-rollout noise. Co-authored-by: Cursor <cursoragent@cursor.com>
Bring the expanded multi-task RHO workshop and renamed notebook sequence into the ROSCon branch. Co-authored-by: Cursor <cursoragent@cursor.com>
RLLearning: three-checkpoint progression demo for ROSCon
Move the workshop helpers, study CLIs, and RAI launch scripts into projects/LocalInference/scripts/ so the course root holds only notebooks, tests, and fixtures. Every consumer now resolves them under /ryzers/notebooks/scripts. Set file ownership in the Dockerfile layers that create the files instead of sweeping recursively at the end. That trailing chown made overlayfs copy up the model caches and /ryzers, storing tens of gigabytes twice. The image drops from 186 GB to 118 GB by the docker images measure, which is 69 GB of actual content across 73.4 GB of layers. Register a rai kernelspec naming /opt/rai-venv/bin/python explicitly. The stock python3 kernelspec cannot run the RAI notebooks: its argv is a bare python, and jupyter_client rewrites that to its own sys.executable, an interpreter that has neither langchain_core nor cv2. Repair five malformed stored outputs in 1_local_inference.ipynb that made nbconvert fail validation before executing a single cell, and add a .dockerignore so local test artifacts stay out of /ryzers/notebooks. Co-authored-by: Cursor <cursoragent@cursor.com>
Notebook 4 now evolves two equal targets instead of three, so drop the cube_lift policy and bump the fixture manifest to rho-multitask-fixtures/v2. test_rho_multitask.sh already asserts that schema version and the two-policy set. Stop the manifest pointing at per-trial CaP-X sources that no longer ship and record the seed policy in prose instead. The sha256 digests stay, so each policy remains verifiable against its historical trial. Remove the study drivers and trial fixtures the rewritten README no longer documents. Nothing in the tree references them. The ignore rule follows the study output directory rename to sweep_results. Co-authored-by: Cursor <cursoragent@cursor.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.