DIAMOND (DIffusion As a Model Of eNvironment Dreams) is a reinforcement learning agent trained in a diffusion world model. NeurIPS 2024 Spotlight.

Mastering Diverse Domains through World Models

Mastering Atari with Discrete World Models

Transformers are Sample-Efficient World Models. ICLR 2023, notable top 5%.

[CVPR 2024 Highlight] GenAD: Generalized Predictive Model for Autonomous Driving & Foundation Models in Autonomous System

Dream to Control: Learning Behaviors by Latent Imagination

DayDreamer: World Models for Physical Robot Learning

A comprehensive survey of forging vision foundation models for autonomous driving, including challenges, methodologies, and opportunities.

A structured implementation of MuZero

A curated list of world models for autonomous driving. Keep updated.

World Model based Autonomous Driving Platform in CARLA 🚗

《多模态大模型：新一代人工智能技术范式》作者：刘阳，林倞

Deep Hierarchical Planning from Pixels

Efficient World Models with Context-Aware Tokenization. ICML 2024

World Models applied to the Open AI Sonic Retro Contest

Code for the ICLR 2024 spotlight paper: "Learning to Act without Actions" (introducing Latent Action Policies)

Code for "Planning Goals for Exploration", ICLR2023 Spotlight. An unsupervised RL agent for hard exploration tasks.

Transformer-based World Models

[NeurIPS 2022] SGAM: Building a Virtual 3D World through Simultaneous Generation and Mapping

[NeurIPS 2024] GenRL: Multimodal foundation world models allow grounding language and video prompts into embodied domains, by turning them into sequences of latent world model states. Latent state sequences can be decoded using the decoder of the model, allowing visualization of the expected behavior, before training the agent to execute it.