Long Term Goal

• Take in a natural language input from a human.
• Utilize LLMs in order interpet the command and translate the input into robotic movements.
• Follow the movements.

Current Docker Container / ROS2 + Gazebo Setup

This repository will consist of two main directories: local and remote. The local directory holds files that are meant to be used on the machine you are using to program, which might be a laptop or a desktop computer. The remote directory holds files that are meant to be used on the machine you want to use to supply the main computing power for the project (this might be the same as your local machine, but it could also be a cloud hosted computer). The local directory mainly contains a description of our remote machine (the Dockerfile and docker-compose.yml files) and scripts you might use to interact with that machine. The remote is a standard ROS2 workspace.

Steps to Use

The following steps may differ if you want to use your local machine or a cloud provider like Runpod as your remote machine.

1. Build and Push the Docker Image (optional, only used if you made changes to the image)

• cd local
  source scripts/build.sh
  scripts/push.sh

2. Build and start your container.

• Your local machine as your remote machine:

  cd local
  source scripts/build.sh
  scripts/up.sh

• Runpod as your remote machine: Go to the runpod website, create a new pod using kawgit/gprp:amd64 as the image. Set the PASSWORD environment variable to whatever you want (this will become the password for the computer). Expose TCP ports 22 and 80. Select whatever GPU you want, start the pod, and wait until the pod has started.

3. Interact with your container.

• Your local machine as your remote machine:

  • View the screen at http://localhost:6080/vnc.html
  • Access the terminal with scripts/exec.sh

• Runpod as your remote machine:

  • Wait until the pod completes its setup. Press connect and look for the public IP and port mapping of the container. It should be something like "Public IP: A, Port 22 maps to B, Port 80 maps to C" where A is an IP address, and B and C are port numbers.
  • View the screen at http://A:C
  • SSH onto the remote container with ssh ubuntu@A -p B

4. Start the simulation

• TBD

Acknoledgements

This project incorporates a Dockerfile from the Tiryoh/docker-ros2-desktop-vnc, licensed under the Apache License 2.0.