README.md

metadata

license: mit
tags:
  - diffusion
  - world-models
  - video-generation
  - carracing
  - reinforcement-learning
  - pytorch

Interactive Diffusion: A Playable World Model

This repository hosts the trained model weights and sample data for the project Demystifying World Models: A Playable Diffusion Model.

• Blog Post: A Playable Diffusion Model
• GitHub Repository: nlml/interactive-diffusion

Contents

• model.pt: 145 MB checkpoint (optimizer stripped) of the CausalSpatiotemporalDiT trained with context noise augmentation (50,000 steps).
• sample_data/: Sample shard 0 (shard_00000_*) containing gameplay frames, actions, and valid episode masks collected from CarRacing-v3.

Quickstart

Clone the GitHub repository and run the live driving server:

git clone https://github.com/nlml/interactive-diffusion.git
cd interactive-diffusion
pip install -e .

# Run the live interactive driving server:
python webapp/server.py --checkpoint nlml/interactive-diffusion --data_dir ./carracing_data