--- language: en tags: - pytorch - autonomous-driving - occupancy-prediction - lidar --- # LO-World Pretrained PyTorch checkpoints for LO-World, a BEV latent world model for joint future occupancy and LiDAR prediction. ## Download ```bash hf download JimmyLiaoo/LO-World \ bev_world_6f.pth \ bev_world_12f_autoregressive.pth bev_world_12f_layout_autoregressive.pth \ bev_world_6f_stc.pth bev_world_6f_waymo.pth occ_lidar_vae.pth SHA256SUMS --local-dir ckpts ``` ## Checkpoints The release contains six checkpoints for inference, fine-tuning and result reproduction. | File | Model | | --- | --- | | bev_world_6f.pth | 6-frame nuScenes World Model; source for STCOcc and Waymo fine-tuning | | bev_world_12f_autoregressive.pth | 12-frame autoregressive World Model | | bev_world_12f_layout_autoregressive.pth | 12-frame autoregressive World Model with physical v1 BEVLayout conditioning | | bev_world_6f_stc.pth | 6-frame nuScenes World Model fine-tuned on STCOcc-conditioned latents | | bev_world_6f_waymo.pth | 6-frame Waymo World Model | | occ_lidar_vae.pth | Joint occupancy-LiDAR tokenizer | Use the corresponding configuration from the LO-World code repository. The Layout checkpoint is the physical v1 fine-tuned epoch 8 EMA and must be paired with the corrected layouts extracted to `data/bev_layout/`. World Model checkpoints include the tokenizer parameters. Training and evaluation require separately prepared dataset files and tokenizer latents. Download the [STCOcc prediction results](https://drive.google.com/file/d/1dXB9mtROLWChycBZlhYIf_JBLshXogBs/view?usp=drive_link), generated with the `r50_704_256_16f_occ3d` checkpoint from [STCOcc](https://github.com/lzzzzzm/STCOcc#-model-zoo), and arrange them as `data/nuscenes/stc-results///labels.npz`. Use `configs/tokenizer/occ_lidar_vae_stc_latent.py` to prepare the matching input latents. `bev_world_6f.pth` is the shared source checkpoint for STCOcc and Waymo fine-tuning. Both adaptations predict six future frames at 2 Hz (0.5–3 seconds). Their configurations load this complete checkpoint through `load_from`, keeping the tokenizer frozen. The STCOcc fine-tuning configuration is `configs/world_model/bev_world_6f_stc_occ.py` (8 epochs, AdamW learning rate 1e-5); the Waymo configuration is `configs/world_model/bev_world_6f_waymo.py`. For `bev_world_6f_stc.pth`, use `configs/world_model/bev_world_6f_stc_occ.py` and matching STCOcc-conditioned validation latents at `data/stc_occ_lidar_latent/.npz`. This checkpoint's embedded tokenizer parameters are identical to the released `occ_lidar_vae.pth`. Encode the STCOcc predictions with that tokenizer: ```bash bash tools/dist_test.sh configs/tokenizer/occ_lidar_vae_stc_latent.py \ ckpts/occ_lidar_vae.pth 8 ``` Evaluation targets remain the original Occ3D labels and LiDAR range images. ## Verify downloads ```bash cd ckpts sha256sum -c SHA256SUMS ``` ## nuScenes infos Prepared train/validation annotation infos are available under `nus_infos/`. They contain 28,130 training samples and 6,019 validation samples, respectively. Download them into the code repository's `data/` directory: ```bash hf download JimmyLiaoo/LO-World --include "nus_infos/*.pkl" --local-dir data ``` The resulting paths are `data/nus_infos/nuscenes_infos_train.pkl` and `data/nus_infos/nuscenes_infos_val.pkl`. Original nuScenes sensor data and Occ3D occupancy labels must be downloaded separately. ## BEVLayout conditioning data The complete 2 Hz train/validation BEVLayout arrays used by the Layout-conditioned models are available under [`data/bev_layout/`](data/bev_layout/). The package contains 34,149 binary `uint8` arrays of shape `(18, 200, 200)`: 28,130 train and 6,019 validation samples. It is approximately 22.9 GiB uncompressed and 77 MiB as split Zstandard archives. Download the package into the LO-World code repository and extract the arrays to `data/bev_layout/`: ```bash hf download JimmyLiaoo/LO-World --include "data/bev_layout/**" --local-dir . mkdir -p data/bev_layout tar --zstd -xf data/bev_layout/2hz/train.tar.zst -C data/bev_layout tar --zstd -xf data/bev_layout/2hz/val.tar.zst -C data/bev_layout (cd data/bev_layout && sha256sum -c SHA256SUMS) ``` These are 2 Hz keyframe layouts, not dense 12 Hz frames. The arrays are derived from nuScenes and are for non-commercial research use subject to the [nuScenes terms](https://www.nuscenes.org/terms-of-use). See the [package card](data/bev_layout/README.md) for format, split manifests, checksums, source terms, and citation. ## Selected 12 Hz conditions: uniform physical timeline Updated 2026-10-22 at the existing download paths. The train700 and val150 packages preserve all 850 original target tokens, clip order and seeds, with one 81-frame window per scene (68,850 total Layouts). They use the corrected physical clock `t0 + round(k * 1e6 / 12)` for ego poses, instance tracks and map rendering, replacing the old sweep-pose clock. The checkpoint and 2 Hz archives are unchanged. ```bash hf download JimmyLiaoo/LO-World --include "data/bev_layout/12hz_train700_81f/**" --local-dir . hf download JimmyLiaoo/LO-World --include "data/bev_layout/12hz_val150_81f/**" --local-dir . (cd data/bev_layout/12hz_train700_81f && sha256sum -c SHA256SUMS) (cd data/bev_layout/12hz_val150_81f && sha256sum -c SHA256SUMS) ``` Each condition NPZ contains 80 future Layouts, binary uint8 with shape `(1,80,18,200,200)`, ego transforms, uniform timestamps and provenance. Each package now also includes `current_layouts/` for frame 0 and `selected_12hz_frames.pkl` for the same frozen rendering timeline. The manifest, release validation summary and SHA256 checksums travel with each package. These are selected windows, not full-scene 12 Hz coverage or high-frequency ground truth. Synthetic future tokens differ from the old sweep tokens; regenerate old predictions and videos rather than pairing them with the new annotations. Original RGB acquisition timestamps are retained, without RGB interpolation. Use the updated LO-World code for timeline/source validation. Use `data/bev_layout/12hz_val150_81f` as `--dataset-dir` with `--dataset-split val` in the batch generator; use the train package with `--dataset-split train`. The updated preparation shell pipeline also stages these installed packages into its existing work_dirs generation paths. See the [training package](data/bev_layout/12hz_train700_81f/README.md) and [validation package](data/bev_layout/12hz_val150_81f/README.md).