Remote grocery shopping through a teleoperated robot. Put on a Meta Quest, drive a tracked robot down a real supermarket aisle with your own body movement and the controllers, and buy what you see — products on the shelf are recognized in view, labelled in 3D, and selectable straight into a cart.
Built so that someone with limited mobility can shop a physical store independently, from anywhere.
Made at the Stanford d.school for Design for Extended Realities, by Lexi Lee, Abhishek Venkataraman, and Ein Jun (engineering).
📺 Demo videos, expo booth, and pitch
The robot is a tracked chassis built from scratch, carrying a camera. It moves with the user's own motion plus Quest controller input, so steering an aisle feels like walking it rather than operating a vehicle.
Products are recognized in view. Vuforia runs image-target recognition over the robot's camera feed; a recognized item gets a 3D label anchored to it in the user's view, which they can select to add to the cart. Shopping happens inside the scene, not in a separate menu.
Two cameras, one frame. The hard part is that the Quest and Vuforia track independently — the headset tracks the user's real room, while Vuforia's pose comes from what the robot's camera sees. Left alone they disagree, and the product labels land in the wrong place.
SyncOVRToVuforia.cs captures the
position and rotation offset between the two rigs at startup and reasserts it
every LateUpdate, so the OVR rig stays locked to the Vuforia camera:
ovrCameraRig.position = vuforiaCamera.position + vuforiaCamera.rotation * initialPositionOffset;
ovrCameraRig.rotation = vuforiaCamera.rotation * initialRotationOffset;TrackingManager.cs then calls
XRDevice.DisableAutoXRCameraTracking so the headset's own positional tracking
stops fighting that pose — otherwise the user's real-room head motion drags the
camera out of the frame the labels are anchored in.
AnchorToUser.cs keeps interface elements
pinned in front of the user's gaze.
Embodied presence carries the experience. Users reported feeling present in the store while teleoperating, with no autonomy anywhere in the loop. Presence mattered more than robot capability.
Designing for accessibility improved it for everyone. Building motion control for wheelchair users is what produced the smoother movement model and the clearer interaction affordances that made the whole thing better to use.
Human-robot collaboration has room well past this demo — retail, remote care, disaster response.
Unity (URP) · Meta XR SDK, Quest 3 · Meta Interaction SDK · Vuforia (QCAR/)
for product recognition · Unity Input System · custom shaders and meshes.
Assets/
Scripts/ tracking sync, XR camera control, UI anchoring
3d labels/ in-world product labels
Custom Meshes/ robot and environment geometry
Custom Shaders/ ShaderLab materials
InteractionSDK/ Meta Interaction SDK integration
MetaXR/ Quest rig and camera setup
Cart.png cart UI
QCAR/ Vuforia image targets
Unity with the Meta XR and Interaction SDK packages plus Vuforia. Open the project and build to a Quest, or run over Link. The scenes load without the robot, but teleoperation needs the physical hardware on the other end.
Thanks to instructors Payam Tabrizian and Yicheng Sun, and to Chaitanya Shah for Unity support.
On repository size. This is a full Unity project with assets, so a clone is large.
mono_crash.mem.*and.plastic/are editor artifacts committed by accident and carry no meaning.