Resources · Intermediate · 8-16 hours
Egocentric Data
First-person video, audio, gaze, hand-object interaction, and wearable sensor data used to teach AI what humans see and do.
Direct answer
What is Egocentric Data?
First-person video, audio, gaze, hand-object interaction, and wearable sensor data used to teach AI what humans see and do.
Definition and scope
First-person video, audio, gaze, hand-object interaction, and wearable sensor data used to teach AI what humans see and do.
Wearable cameras and research glasses capture synchronized video, audio, pose, gaze, and language annotations from the actor's viewpoint.
Why it matters
Egocentric data is central for smart glasses, contextual assistants, robot imitation, and models that learn from human demonstrations.
How it works
Wearable cameras and research glasses capture synchronized video, audio, pose, gaze, and language annotations from the actor's viewpoint.
Beginner learning path
Compare third-person video with first-person video. Notice how hands, gaze, object affordances, and intent become more visible.
Recommended next topics
Primary sources
Key papers
Ego4D: Unscripted First-Person Video Dataset
Ego4D is a large first-person video dataset for studying what people see, remember, manipulate, and anticipate.
Ego-Exo4D: First- and Third-Person Skilled Activity Dataset
Ego-Exo4D pairs synchronized first-person and third-person video with language, gaze, audio, pose, and 3D signals.
Aria Gen 2 Pilot Dataset
Aria Gen 2 Pilot Dataset captures daily activities with research glasses and synchronized multimodal sensor data.
V-JEPA 2: Self-Supervised Video Models for Physical Planning
V-JEPA 2 learns predictive video representations that support visual understanding and zero-shot robot control.
Research ecosystem
Organizations working in this area
Organization
Meta AI
Embodied perception, video prediction, egocentric AI
View profile →Organization
Labelbox
Sensor and robotics data labeling for video, images, and computer vision pipelines
View profile →Organization
Encord
Multimodal data infrastructure for physical AI, VLA data, video, LiDAR, and robotics datasets
View profile →Organization
Sama
Human-in-the-loop data annotation for video, computer vision, and multimodal AI
View profile →Organization
Appen
Multimodal training data for VLMs, video understanding, audio-visual alignment, and physical AI
View profile →Common questions
Frequently asked questions
What is Egocentric Data?
First-person video, audio, gaze, hand-object interaction, and wearable sensor data used to teach AI what humans see and do.
Why does Egocentric Data matter for Physical AI?
Egocentric data is central for smart glasses, contextual assistants, robot imitation, and models that learn from human demonstrations.
How should a beginner learn Egocentric Data?
Compare third-person video with first-person video. Notice how hands, gaze, object affordances, and intent become more visible.