2018 · Google Brain / IDSIA · Intermediate
World Models
A compact latent model can let an agent learn behavior inside its own predicted environment.
Direct answer
What does World Models contribute?
A compact latent model can let an agent learn behavior inside its own predicted environment.
Background
World Models separates perception, dynamics, and control into a visual encoder, a recurrent predictive model, and a controller. The paper showed that useful policies can be trained using imagined rollouts rather than only direct environment interaction.
Problem
The work addresses a central constraint in World Models: building systems that learn useful representations or actions while remaining general enough to transfer beyond a single demonstration or environment.
Core idea
A compact latent model can let an agent learn behavior inside its own predicted environment.
Architecture and method
World Models separates perception, dynamics, and control into a visual encoder, a recurrent predictive model, and a controller. The paper showed that useful policies can be trained using imagined rollouts rather than only direct environment interaction.
- Latent visual representation
- Recurrent dynamics model
- Policy learning in imagined environments
Results and impact
It made the idea of learning a predictive internal model concrete and accessible, influencing later model-based reinforcement learning systems.
Prerequisites
- Reinforcement learning
- Autoencoders
- Recurrent neural networks
Recommended reading order
Read the explanation above, review the related topic pages, then use the primary-source links below to inspect the abstract, figures, experiments, and released implementation.
Primary sources
External links are provided after the context needed to evaluate the work.
Follow-up research
Related papers and concepts
Dreamer: Reinforcement Learning with Latent Imagination
Dreamer learns long-horizon behavior by propagating value gradients through imagined latent trajectories.
DreamerV3: Mastering Diverse Domains through World Models
DreamerV3 uses robust normalization and objectives to learn across more than 150 tasks with one configuration.
Genie: Generative Interactive Environments
Genie learns controllable interactive environments from unlabeled internet video.
Genie 2: A Large-Scale Foundation World Model
Genie 2 generates action-controllable 3D environments from a single prompt image.
Common questions
Frequently asked questions
What is the main idea of World Models?
A compact latent model can let an agent learn behavior inside its own predicted environment.
Why is World Models important?
It made the idea of learning a predictive internal model concrete and accessible, influencing later model-based reinforcement learning systems.
What should I learn before reading World Models?
Recommended prerequisites are Reinforcement learning, Autoencoders, Recurrent neural networks.