2024 · Google DeepMind · Advanced
Genie: Generative Interactive Environments
Genie learns controllable interactive environments from unlabeled internet video.
Direct answer
What does Genie: Generative Interactive Environments contribute?
Genie learns controllable interactive environments from unlabeled internet video.
Background
Genie discovers latent actions and learns video dynamics without requiring action labels. Given an image or prompt, it can generate an interactive sequence controlled through the learned action space.
Problem
The work addresses a central constraint in World Models: building systems that learn useful representations or actions while remaining general enough to transfer beyond a single demonstration or environment.
Core idea
Genie learns controllable interactive environments from unlabeled internet video.
Architecture and method
Genie discovers latent actions and learns video dynamics without requiring action labels. Given an image or prompt, it can generate an interactive sequence controlled through the learned action space.
- Latent action model
- Unlabeled video training
- Interactive generated environments
Results and impact
It connected large-scale video generation with action-controllable world models.
Prerequisites
- Video models
- World models
- Discrete latent representations
Recommended reading order
Read the explanation above, review the related topic pages, then use the primary-source links below to inspect the abstract, figures, experiments, and released implementation.
Primary sources
External links are provided after the context needed to evaluate the work.
Follow-up research
Related papers and concepts
World Models
A compact latent model can let an agent learn behavior inside its own predicted environment.
Dreamer: Reinforcement Learning with Latent Imagination
Dreamer learns long-horizon behavior by propagating value gradients through imagined latent trajectories.
DreamerV3: Mastering Diverse Domains through World Models
DreamerV3 uses robust normalization and objectives to learn across more than 150 tasks with one configuration.
Genie 2: A Large-Scale Foundation World Model
Genie 2 generates action-controllable 3D environments from a single prompt image.
Common questions
Frequently asked questions
What is the main idea of Genie: Generative Interactive Environments?
Genie learns controllable interactive environments from unlabeled internet video.
Why is Genie: Generative Interactive Environments important?
It connected large-scale video generation with action-controllable world models.
What should I learn before reading Genie: Generative Interactive Environments?
Recommended prerequisites are Video models, World models, Discrete latent representations.