Resources · Beginner · 4-8 hours
Datasets
Collections of robot trajectories, observations, actions, language instructions, and environment interactions.
Direct answer
What is Datasets?
Collections of robot trajectories, observations, actions, language instructions, and environment interactions.
Definition and scope
Collections of robot trajectories, observations, actions, language instructions, and environment interactions.
Datasets record demonstrations or autonomous experience and standardize observations, actions, tasks, and annotations.
Why it matters
Data diversity and quality strongly shape a policy's capabilities and generalization.
How it works
Datasets record demonstrations or autonomous experience and standardize observations, actions, tasks, and annotations.
Beginner learning path
Learn the difference between teleoperation, autonomous collection, synthetic data, and cross-embodiment datasets.
Recommended next topics
Primary sources
Key papers
Open X-Embodiment and RT-X
Open X-Embodiment combines robot datasets across institutions and trains policies that transfer across embodiments.
OpenVLA: An Open-Source Vision-Language-Action Model
OpenVLA is an open 7B-parameter VLA trained on the Open X-Embodiment dataset.
DROID: Distributed Robot Interaction Dataset
DROID provides diverse in-the-wild robot manipulation demonstrations across many scenes, tasks, and collectors.
Mobile ALOHA: Low-Cost Whole-Body Teleoperation
Mobile ALOHA collects whole-body, bimanual mobile manipulation demonstrations with a low-cost teleoperation system.
Ego4D: Unscripted First-Person Video Dataset
Ego4D is a large first-person video dataset for studying what people see, remember, manipulate, and anticipate.
Ego-Exo4D: First- and Third-Person Skilled Activity Dataset
Ego-Exo4D pairs synchronized first-person and third-person video with language, gaze, audio, pose, and 3D signals.
Aria Gen 2 Pilot Dataset
Aria Gen 2 Pilot Dataset captures daily activities with research glasses and synchronized multimodal sensor data.
Research ecosystem
Organizations working in this area
Organization
Google DeepMind
World models, robot learning, VLA systems, embodied reasoning
View profile →Organization
Physical Intelligence
Generalist robot foundation models
View profile →Organization
Meta AI
Embodied perception, video prediction, egocentric AI
View profile →Organization
Encord
Multimodal data infrastructure for physical AI, VLA data, video, LiDAR, and robotics datasets
View profile →Organization
Labelbox
Sensor and robotics data labeling for video, images, and computer vision pipelines
View profile →Common questions
Frequently asked questions
What is Datasets?
Collections of robot trajectories, observations, actions, language instructions, and environment interactions.
Why does Datasets matter for Physical AI?
Data diversity and quality strongly shape a policy's capabilities and generalization.
How should a beginner learn Datasets?
Learn the difference between teleoperation, autonomous collection, synthetic data, and cross-embodiment datasets.