4 investigate and likely switch to openvla oft - #5
Conversation
First commit for paper: "Fine-Tuning Vision-Language-Action Models: Optimizing Speed and Success" - Moo Jin Kim, Chelsea Finn, Percy Liang Website: https://openvla-oft.github.io/
Prevents race conditions in multi-GPU training runs.
Also update ALOHA VRAM usage in README.md
(Previous commit was supposed to do this)
(Python 3.10 actually works for the client conda environment)
…s://github.com/nomagiclab/openvla into 4-investigate-and-likely-switch-to-openvla-oft
…s://github.com/nomagiclab/openvla into 4-investigate-and-likely-switch-to-openvla-oft
There was a problem hiding this comment.
Copilot reviewed 44 out of 52 changed files in this pull request and generated no comments.
Files not reviewed (8)
- .devcontainer/devcontainer.json: Language not supported
- .devcontainer/setup_host.sh: Language not supported
- .dockerignore: Language not supported
- .env.template: Language not supported
- .gitmodules: Language not supported
- Dockerfile: Language not supported
- LICENSE: Language not supported
- experiments/robot/aloha/requirements_aloha.txt: Language not supported
Comments suppressed due to low confidence (1)
experiments/robot/aloha/preprocess_split_aloha_data.py:97
- The variable 'args' is referenced directly in the function without being passed as an argument; consider passing 'img_resize_size' explicitly or otherwise defining 'args' in this scope to avoid a NameError.
Image.fromarray(image_dict[k][i]).resize((args.img_resize_size, args.img_resize_size), resample=Image.BICUBIC)
Revert to state before direct integration of robot-interface into openvla environment. We will use a different approach: serve openvla in one Python environment (using `vla-scripts/finetune.py`), and query server from robot-interface client in another Python environment. modified: .devcontainer/devcontainer.json modified: .devcontainer/docker-compose.yml deleted: .devcontainer/setup_host.sh modified: Dockerfile modified: prismatic/util/extern/hf/lerobot_utils.py modified: pyproject.toml modified: vla-scripts/finetune.py
* fix to val set split * read action chunks from prepared lerobotdatasets temporary fix to a pressing issue, implement action chunking properly by reading actions from 'action_chunk' col of step instead of 'action.pose' and 'action.gripper'. 'action_chunk' col must be prepared offline * updated slurm jobconf for action chunked lerbdset * fixes to manual create env * create data/ dir when manually creating env * fix double-saving weights when finetuning
There was a problem hiding this comment.
Copilot reviewed 43 out of 51 changed files in this pull request and generated no comments.
Files not reviewed (8)
- .devcontainer/devcontainer.json: Language not supported
- .dockerignore: Language not supported
- .env.template: Language not supported
- .gitmodules: Language not supported
- Dockerfile: Language not supported
- LICENSE: Language not supported
- experiments/robot/aloha/requirements_aloha.txt: Language not supported
- finetune_lerobot.sh: Language not supported
Comments suppressed due to low confidence (1)
experiments/robot/aloha/preprocess_split_aloha_data.py:96
- The variable 'args' is referenced in the image resize call without being defined in this scope. Consider passing the resize size as a parameter to the function instead.
Image.fromarray(image_dict[k][i]).resize((args.img_resize_size, args.img_resize_size), resample=Image.BICUBIC)
|
In case anyone wants to take a detailed look at the ommit history, there are roughly speaking these blocks of changes here:
|
No description provided.