Practical index

Datasets, benchmarks, simulators, and frameworks

A curated set of primary resources for studying and building Physical AI systems.

Dataset

Open X-Embodiment

A large cross-embodiment collection of robot trajectories from many institutions.

Open resource
Dataset

DROID

A diverse real-world robot manipulation dataset collected across many scenes and operators.

Open resource
Benchmark

LIBERO

A benchmark for lifelong robot learning with manipulation task suites.

Open resource
Benchmark

CALVIN

A language-conditioned long-horizon manipulation benchmark.

Open resource
Simulation

NVIDIA Isaac Sim

A robotics simulation platform for synthetic data, training, and testing.

Open resource
Safety platform

NVIDIA Halos for Robotics

A full-stack functional-safety platform spanning IGX Thor, Halos OS, safety application blueprints, and inspection support.

Open resource
Open-source blueprint

NVIDIA Halos Outside-In Safety

Infrastructure-camera safety agents for robot zoning, perception monitoring, event integration, and safety decisions.

Open resource
Simulation

MuJoCo

An open-source physics engine widely used for robotics and control research.

Open resource
Framework

LeRobot

Open models, datasets, and tools for real-world robotics in PyTorch.

Open resource
Framework

OpenPI

Open-source models and tooling from Physical Intelligence.

Open resource
Model platform

NVIDIA Cosmos

World foundation models, guardrails, tokenizers, and tooling for physical AI data generation.

Open resource
Model platform

NVIDIA Isaac GR00T

Robot foundation model and data pipeline platform for humanoid and general-purpose robot learning.

Open resource
Framework

Isaac Lab

A unified framework for robot learning research built around Isaac Sim.

Open resource
Model platform

Gemini Robotics

Google DeepMind's embodied AI model family for visual reasoning and robot action.

Open resource
Dataset

Ego4D

Large-scale first-person video dataset for hand-object interaction, memory, forecasting, and social understanding.

Open resource
Dataset

Ego-Exo4D

Synchronized first-person and third-person skilled activity dataset with language, audio, gaze, pose, and 3D signals.

Open resource
Dataset

Project Aria Gen 2 Pilot Dataset

Open egocentric wearable-sensor dataset for contextual AI, robotics, and smart-glasses research.

Open resource
Spatial AI

World Labs World API

API for generating, rendering, and exporting navigable 3D worlds from images, text, panoramas, and video.

Open resource
Data infrastructure

Encord Physical AI Data Layer

Multimodal data operations for video, robotics, world models, and VLA training datasets.

Open resource
Data infrastructure

Labelbox Sensors and Robotics

Labeling and curation workflows for sensor, robotics, video, and computer-vision datasets.

Open resource
Data infrastructure

Scale AI Data Labeling Guide

Reference for image, video, sensor, and temporal annotation workflows used in AI data engines.

Open resource
Data infrastructure

Label Studio

Open-source multimodal data labeling and evaluation tool for images, video, text, audio, and time series.

Open resource