OctWorld-1.5

Code | Project Page & Demos | Paper

OctWorld: Long-Range World-Consistent Video Generation with Octree-based 3D Mapping

OctWorld-1.5 is the Wan 14B-based world-generation checkpoint for OctWorld. Starting from a single image, the framework generates video along user-specified camera trajectories, using text prompts, camera poses, and anchor depth as inputs.

OctWorld maintains persistent 3D spatial memory through OctMap, a spatially adaptive octree map. OctMap accumulates geometry and appearance from generated RGB-D observations and renders spatial conditions for subsequent video chunks, helping preserve scene consistency over long camera paths and revisits.

Release Contents

This repository contains the OctWorld-1.5 checkpoint and its pipeline assets:

OctWorld-1.5/
  model/
    .metadata
    __0_0.distcp
    ...
    __15_0.distcp
  pipe_config.json
  neg_text_embed.pt

Asset	Description
model/	PyTorch distributed checkpoint: 16 weight shards and the required .metadata file.
pipe_config.json	Wan 14B backbone, camera adapter, and FramePack configuration.
neg_text_embed.pt	Precomputed negative-text embedding used by the inference pipeline.

The model package occupies approximately 101.84 GB (94.85 GiB) on disk, excluding auxiliary models and generated outputs. Keep all checkpoint shards and model/.metadata together.

Use the official OctWorld inference code to load this checkpoint. This is not a standalone Transformers or Diffusers pipeline.

The code release includes Wan-based inference, OctMap, depth-prediction integration, bundled examples, and custom-input preparation. The training-data pipeline and end-to-end training/fine-tuning code are not released yet.

Requirements

The tested inference environment uses:

• Linux x86_64 and Python 3.10 (tested with 3.10.21).
• An NVIDIA Ampere-or-newer GPU with at least 70 GB of GPU memory; tested on an A100 80 GB.
• CUDA Toolkit 12.6, a compatible NVIDIA driver, a C++ compiler, Git, and FFmpeg.
• PyTorch 2.8.0+cu126, torchvision 0.23.0+cu126, xformers 0.0.32.post2, and flash-attn 2.8.3.

The CUDA toolkit is needed to build native rendering extensions; the CUDA runtime bundled with PyTorch alone is not sufficient. Follow the installation guide and the repository's pinned dependency setup.

Quick Start

Run the following commands from the code repository root after cloning it. Use a new or empty output directory for each generation run.

1. Install the Code

git clone --recurse-submodules https://github.com/microsoft/OctWorld.git
cd OctWorld

conda create -n octworld python=3.10.21 -y
conda activate octworld

export CUDA_HOME=/usr/local/cuda-12.6
bash setup.sh

For an existing checkout, initialize the pinned depth-model submodules with git submodule update --init --recursive. Skip environment creation if you already have a suitable Python 3.10 environment. setup.sh installs dependencies into the selected environment; it does not install system packages or download model weights.

2. Download OctWorld-1.5

hf download Maxtir/OctWorld-1.5 --local-dir model_weights/octworld-1.5

The hf CLI is included in the repository's huggingface-hub dependency. This downloads the checkpoint, configuration, and negative-text embedding into the layout expected by the examples.

3. Download Auxiliary Models

OctWorld-1.5 alone is not sufficient to run the full framework:

python scripts/download_models.py

Component	Purpose
Wan2.1 VAE, T5, CLIP, and tokenizers	Image, text, and video encoding/decoding. The original Wan DiT is not downloaded.
Depth Anything 3	Per-chunk depth aligned to user cameras.
RAFT	Optical-flow-based keyframe selection.
Sky segmentation	Sky masking before map integration.
MoGe v1	Anchor depth and intrinsics for custom inputs.
Qwen3-VL-8B-Instruct	Prompt planning for custom inputs.

The default downloader prepares all six auxiliary components, approximately 42.44 GB (39.53 GiB) in total. For bundled-case inference only, use python scripts/download_models.py --models wan da3 raft skyseg. Custom-input preparation additionally needs MoGe and Qwen3-VL.

Every downloaded file and revision is pinned in the model manifest. The auxiliary downloader does not download OctWorld-1.5.

4. Generate a Video

python example/worldgen/wan/world_gen.py \
    --dit_ckpt model_weights/octworld-1.5/model \
    --model_weights_dir model_weights \
    --save_dir outputs/dl3dv_gt

The default example uses the bundled DL3DV scene and its original camera trajectory. It generates 4 chunks / 321 frames at 480 x 832 (height x width), 15 fps, or approximately 21.4 seconds.

Successful single-case runs produce:

Output	Description
output.mp4	Generated RGB video.
depth.mp4	Colorized inverse-depth visualization, not metric depth.
warp.mp4	Rendered spatial conditions used during generation.
sha256.json	Seed information and sampling/conditioning tensor hashes.

The default uses online OctMap integration, DA3 depth aligned to user cameras, and seed 425. The final chunk of a multi-chunk run skips depth prediction and map integration because there is no subsequent chunk to condition; its depth-video segment is black.

See the inference guide for input formats, generation options, output semantics, and reproducibility limits.

Batch Generation and Custom Inputs

Generate all three bundled cases with one model load:

python example/worldgen/wan/batch_world_gen.py \
    --instances assets/standard/instances.json \
    --dit_ckpt model_weights/octworld-1.5/model \
    --model_weights_dir model_weights \
    --base_folder outputs/batch

For your own images and camera paths, follow the custom-input guide. The tools prepare inference inputs from an image and camera-movement instructions, with optional smooth camera interpolation, MoGe anchor depth, and Qwen3-VL prompt planning. They are not a training-data pipeline.

Intended Use and Limitations

OctWorld supports image-conditioned scene exploration, camera-controlled video generation, and research on long-range world-consistent generation.

• Spatial memory helps preserve consistency but does not guarantee physically accurate geometry or artifact-free videos. Generated content and inferred depth should not be treated as ground-truth observations.
• Anchor depth defines the metric scale of the user trajectory. The exported depth.mp4 is only a visualization and must not be interpreted as metric depth measurements.
• Results depend on inputs, camera paths, prompts, seeds, weights, and runtime environment. A matching seed or initial-noise hash alone does not guarantee pixel-identical outputs.
• This checkpoint requires the official pipeline and auxiliary models; this repository does not include the full software environment or training code.

For the method and experimental evaluation, see the paper. For visual examples, visit the project page.

License

The OctWorld-1.5 model weights are released under the Apache License 2.0.

The OctWorld code is released under the MIT License. Third-party code, auxiliary model weights, and datasets remain subject to their respective licenses and usage terms.

Citation

If you find OctWorld useful for your research, please cite:

@article{lv2026octworld,
    title   = {OctWorld: Long-Range World-Consistent Video Generation
                         with Octree-based 3D Mapping},
    author  = {Lv, Zelong and Xu, Sicheng and Xiang, Jianfeng and
                         Wang, Ruicheng and Dong, Yue and Deng, Yu and
                         Sun, Guangzhong and Yang, Jiaolong},
    journal = {arXiv preprint arXiv:2609.03919},
    year    = {2026}
}