2022 · Google Robotics · Intermediate
RT-1: Robotics Transformer for Real-World Control at Scale
RT-1 trains one transformer policy on a large multi-task dataset of real robot demonstrations.
Direct answer
What does RT-1: Robotics Transformer for Real-World Control at Scale contribute?
RT-1 trains one transformer policy on a large multi-task dataset of real robot demonstrations.
Background
RT-1 tokenizes robot actions and processes image histories plus language instructions with a transformer. It was trained on approximately 130,000 episodes covering hundreds of tasks collected using a fleet of robots.
Problem
The work addresses a central constraint in VLA: building systems that learn useful representations or actions while remaining general enough to transfer beyond a single demonstration or environment.
Core idea
RT-1 trains one transformer policy on a large multi-task dataset of real robot demonstrations.
Architecture and method
RT-1 tokenizes robot actions and processes image histories plus language instructions with a transformer. It was trained on approximately 130,000 episodes covering hundreds of tasks collected using a fleet of robots.
- Action tokenization
- Large real-robot dataset
- Multi-task transformer policy
Results and impact
It established a practical scaling recipe for multi-task real-world robot control.
Prerequisites
- Transformers
- Imitation learning
- Robot manipulation
Recommended reading order
Read the explanation above, review the related topic pages, then use the primary-source links below to inspect the abstract, figures, experiments, and released implementation.
Primary sources
External links are provided after the context needed to evaluate the work.
Follow-up research
Related papers and concepts
RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Control
RT-2 co-trains vision-language models on web and robot data so semantic knowledge can influence actions.
Open X-Embodiment and RT-X
Open X-Embodiment combines robot datasets across institutions and trains policies that transfer across embodiments.
OpenVLA: An Open-Source Vision-Language-Action Model
OpenVLA is an open 7B-parameter VLA trained on the Open X-Embodiment dataset.
Gemini Robotics 1.5
Gemini Robotics 1.5 turns visual observations and instructions into motor commands while supporting multi-step physical tasks.
Common questions
Frequently asked questions
What is the main idea of RT-1: Robotics Transformer for Real-World Control at Scale?
RT-1 trains one transformer policy on a large multi-task dataset of real robot demonstrations.
Why is RT-1: Robotics Transformer for Real-World Control at Scale important?
It established a practical scaling recipe for multi-task real-world robot control.
What should I learn before reading RT-1: Robotics Transformer for Real-World Control at Scale?
Recommended prerequisites are Transformers, Imitation learning, Robot manipulation.