Core AI · Advanced · 15-25 hours
Vision-Language-Action Models
Models that map visual observations and language instructions to robot actions.
Direct answer
What is Vision-Language-Action Models?
Models that map visual observations and language instructions to robot actions.
Definition and scope
Models that map visual observations and language instructions to robot actions.
A VLA encodes images and text, fuses their representations, and predicts discrete or continuous action sequences.
Why it matters
VLAs bring semantic knowledge and instruction following into general-purpose robot policies.
How it works
A VLA encodes images and text, fuses their representations, and predicts discrete or continuous action sequences.
Beginner learning path
First understand vision-language models, imitation learning, action spaces, and transformer policies.
Recommended next topics
Primary sources
Key papers
RT-1: Robotics Transformer for Real-World Control at Scale
RT-1 trains one transformer policy on a large multi-task dataset of real robot demonstrations.
RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Control
RT-2 co-trains vision-language models on web and robot data so semantic knowledge can influence actions.
Open X-Embodiment and RT-X
Open X-Embodiment combines robot datasets across institutions and trains policies that transfer across embodiments.
OpenVLA: An Open-Source Vision-Language-Action Model
OpenVLA is an open 7B-parameter VLA trained on the Open X-Embodiment dataset.
Gemini Robotics 1.5
Gemini Robotics 1.5 turns visual observations and instructions into motor commands while supporting multi-step physical tasks.
pi0: A Vision-Language-Action Flow Model for General Robot Control
pi0 is a generalist robot policy trained on broad robot data to follow language instructions across dexterous tasks.
NVIDIA Isaac GR00T N1.7
GR00T N1.7 is an open vision-language-action model for generalized humanoid manipulation skills.
Figure Helix
Helix is Figure's generalist humanoid VLA model for onboard perception, reasoning, and full-upper-body control.
Research ecosystem
Organizations working in this area
Organization
Google DeepMind
World models, robot learning, VLA systems, embodied reasoning
View profile →Organization
Physical Intelligence
Generalist robot foundation models
View profile →Organization
NVIDIA
Robot foundation models, simulation, synthetic data, edge deployment, and functional safety
View profile →Organization
Figure AI
General-purpose humanoid robots and onboard VLA systems
View profile →Organization
1X Technologies
Home humanoid robots, Redwood AI, real-world supervision and learning
View profile →Organization
Skild AI
General-purpose robotic brain trained across tasks, embodiments, and human videos
View profile →Common questions
Frequently asked questions
What is Vision-Language-Action Models?
Models that map visual observations and language instructions to robot actions.
Why does Vision-Language-Action Models matter for Physical AI?
VLAs bring semantic knowledge and instruction following into general-purpose robot policies.
How should a beginner learn Vision-Language-Action Models?
First understand vision-language models, imitation learning, action spaces, and transformer policies.