Jay Vakil
I am a Computer Science PhD student at the University of Colorado Boulder, researching Robot Learning, Mobile Manipulation, and Embodied AI. My work focuses on scaling robotic learning, contributing to foundational projects like RT-X and RoboAgent.
Previously, I was a researcher at Facebook AI Research (FAIR). I am fortunate to be advised by Alessandro Roncone, Nikolaus Correll.
jay.vakil [at] colorado.edu BS Electrical Engineering, University of Washington Boulderer, Photographer, and Car Enthusiast
Refresh to see different me
Education & Experience
Working on Multimodal agent LLM based planning, Diffusion-based human-motion synthesis, and generative modeling on human-motion manifolds.
Researching Robot Learning, Mobile Manipulation, and Embodied AI. Advised by Alessandro Roncone, Nikolaus Correll, and Christoffer Heckman.
Designed a distributed robotic arm cluster for large-scale experimentation. Developed universal agents for complex manipulation tasks (RoboAgent).
Minor in Mathematics and Computer Science. Dean's List recipient. Focus on signal processing, robotics, and embedded systems.
Research
Open X-Embodiment
Scaling robotic learning via the RT-X model and a massive cross-platform dataset of 60+ robots. The 'ImageNet' moment for robotics.
RoboAgent
Universal robot learning with semantic augmentations. Teaching robots broadly generalizable skills through action chunking across 38+ tasks.
OK-Robot
A zero-shot system for open-vocabulary mobile manipulation. Integrating Vision-Language Models with classical navigation primitives to handle messy environments.
Dynamics-Compliant Trajectory Diffusion
Payload-conditioned diffusion model for generating dynamic motions for robots handling up to 3x nominal payloads.
Sim-to-Real Visual Representations
A large-scale study of pre-trained visual representations in sim and real environments to understand transfer efficacy.
Visual Cortex for Embodied Intelligence
Evaluating mid-level visual representations for robotic control tasks. Are we closer to an artificial visual cortex?
Spatial-Language Attention Policies
Efficient robot learning using spatial-language attention mechanisms for manipulation tasks.
Towards Open‑World Mobile Manipulation in Homes: Lessons from the Neurips 2023HomeRobot Open Vocabulary Mobile Manipulation Challenge
Open-vocabulary mobile manipulation challenge. Benchmarking robot performance in novel home environments.
RoboHive
A unified framework for Robot Learning and Embodied AI research, offering diverse simulation environments.
Impact & Press
Teaching
- University of Colorado Teaching Assistant (2024 — Present) CSCI 3202: Introduction to AI CSCI 2400: Computer Systems
Service
- Reviewer:
Affiliations
