| --- |
| language: en |
| tags: |
| - pytorch |
| - autonomous-driving |
| - occupancy-prediction |
| - lidar |
| --- |
| |
| # LO-World |
|
|
| Pretrained PyTorch checkpoints for LO-World, a BEV latent world model for joint future occupancy and LiDAR prediction. |
|
|
| ## Download |
|
|
| ```bash |
| hf download JimmyLiaoo/LO-World \ |
| bev_world_6f.pth \ |
| bev_world_12f_autoregressive.pth bev_world_12f_layout_autoregressive.pth \ |
| bev_world_6f_stc.pth bev_world_6f_waymo.pth occ_lidar_vae.pth SHA256SUMS --local-dir ckpts |
| ``` |
|
|
| ## Checkpoints |
|
|
| The release contains six checkpoints for inference, fine-tuning and result reproduction. |
|
|
| | File | Model | |
| | --- | --- | |
| | bev_world_6f.pth | 6-frame nuScenes World Model; source for STCOcc and Waymo fine-tuning | |
| | bev_world_12f_autoregressive.pth | 12-frame autoregressive World Model | |
| | bev_world_12f_layout_autoregressive.pth | 12-frame autoregressive World Model with physical v1 BEVLayout conditioning | |
| | bev_world_6f_stc.pth | 6-frame nuScenes World Model fine-tuned on STCOcc-conditioned latents | |
| | bev_world_6f_waymo.pth | 6-frame Waymo World Model | |
| | occ_lidar_vae.pth | Joint occupancy-LiDAR tokenizer | |
| |
| Use the corresponding configuration from the LO-World code repository. The Layout checkpoint is the physical v1 fine-tuned epoch 8 EMA and must be paired with the corrected layouts extracted to `data/bev_layout/`. World Model checkpoints include the tokenizer parameters. Training and evaluation require separately prepared dataset files and tokenizer latents. |
|
|
| Download the [STCOcc prediction results](https://drive.google.com/file/d/1dXB9mtROLWChycBZlhYIf_JBLshXogBs/view?usp=drive_link), |
| generated with the `r50_704_256_16f_occ3d` checkpoint from |
| [STCOcc](https://github.com/lzzzzzm/STCOcc#-model-zoo), and arrange them as |
| `data/nuscenes/stc-results/<scene_name>/<sample_token>/labels.npz`. |
| Use `configs/tokenizer/occ_lidar_vae_stc_latent.py` to prepare the matching |
| input latents. |
|
|
| `bev_world_6f.pth` is the shared source checkpoint for STCOcc and Waymo |
| fine-tuning. Both adaptations predict six future frames at 2 Hz (0.5–3 |
| seconds). Their configurations load this complete checkpoint through |
| `load_from`, keeping the tokenizer frozen. The STCOcc fine-tuning |
| configuration is `configs/world_model/bev_world_6f_stc_occ.py` (8 epochs, |
| AdamW learning rate 1e-5); the Waymo configuration is |
| `configs/world_model/bev_world_6f_waymo.py`. |
|
|
| For `bev_world_6f_stc.pth`, use |
| `configs/world_model/bev_world_6f_stc_occ.py` and matching STCOcc-conditioned |
| validation latents at `data/stc_occ_lidar_latent/<sample_token>.npz`. |
| This checkpoint's embedded tokenizer parameters are identical to the released |
| `occ_lidar_vae.pth`. Encode the STCOcc predictions with that tokenizer: |
|
|
| ```bash |
| bash tools/dist_test.sh configs/tokenizer/occ_lidar_vae_stc_latent.py \ |
| ckpts/occ_lidar_vae.pth 8 |
| ``` |
|
|
| Evaluation targets remain the original Occ3D labels and LiDAR range images. |
|
|
| ## Verify downloads |
|
|
| ```bash |
| cd ckpts |
| sha256sum -c SHA256SUMS |
| ``` |
|
|
| ## nuScenes infos |
|
|
| Prepared train/validation annotation infos are available under `nus_infos/`. |
| They contain 28,130 training samples and 6,019 validation samples, respectively. |
| Download them into the code repository's `data/` directory: |
|
|
| ```bash |
| hf download JimmyLiaoo/LO-World --include "nus_infos/*.pkl" --local-dir data |
| ``` |
|
|
| The resulting paths are `data/nus_infos/nuscenes_infos_train.pkl` and |
| `data/nus_infos/nuscenes_infos_val.pkl`. Original nuScenes sensor data and |
| Occ3D occupancy labels must be downloaded separately. |
|
|
| ## BEVLayout conditioning data |
|
|
| The complete 2 Hz train/validation BEVLayout arrays used by the Layout-conditioned models are available under [`data/bev_layout/`](data/bev_layout/). The package contains 34,149 binary `uint8` arrays of shape `(18, 200, 200)`: 28,130 train and 6,019 validation samples. It is approximately 22.9 GiB uncompressed and 77 MiB as split Zstandard archives. |
|
|
| Download the package into the LO-World code repository and extract the arrays to `data/bev_layout/`: |
|
|
| ```bash |
| hf download JimmyLiaoo/LO-World --include "data/bev_layout/**" --local-dir . |
| mkdir -p data/bev_layout |
| tar --zstd -xf data/bev_layout/2hz/train.tar.zst -C data/bev_layout |
| tar --zstd -xf data/bev_layout/2hz/val.tar.zst -C data/bev_layout |
| (cd data/bev_layout && sha256sum -c SHA256SUMS) |
| ``` |
|
|
| These are 2 Hz keyframe layouts, not dense 12 Hz frames. The arrays are derived from nuScenes and are for non-commercial research use subject to the [nuScenes terms](https://www.nuscenes.org/terms-of-use). See the [package card](data/bev_layout/README.md) for format, split manifests, checksums, source terms, and citation. |
|
|
| ## Selected 12 Hz conditions: uniform physical timeline |
|
|
| Updated 2026-10-22 at the existing download paths. The train700 and val150 |
| packages preserve all 850 original target tokens, clip order and seeds, with |
| one 81-frame window per scene (68,850 total Layouts). They use the corrected |
| physical clock `t0 + round(k * 1e6 / 12)` for ego poses, instance tracks and map |
| rendering, replacing the old sweep-pose clock. The checkpoint and 2 Hz archives |
| are unchanged. |
|
|
| ```bash |
| hf download JimmyLiaoo/LO-World --include "data/bev_layout/12hz_train700_81f/**" --local-dir . |
| hf download JimmyLiaoo/LO-World --include "data/bev_layout/12hz_val150_81f/**" --local-dir . |
| (cd data/bev_layout/12hz_train700_81f && sha256sum -c SHA256SUMS) |
| (cd data/bev_layout/12hz_val150_81f && sha256sum -c SHA256SUMS) |
| ``` |
|
|
| Each condition NPZ contains 80 future Layouts, binary uint8 with shape |
| `(1,80,18,200,200)`, ego transforms, uniform timestamps and provenance. Each |
| package now also includes `current_layouts/` for frame 0 and |
| `selected_12hz_frames.pkl` for the same frozen rendering timeline. The manifest, |
| release validation summary and SHA256 checksums travel with each package. |
|
|
| These are selected windows, not full-scene 12 Hz coverage or high-frequency |
| ground truth. Synthetic future tokens differ from the old sweep tokens; |
| regenerate old predictions and videos rather than pairing them with the new |
| annotations. Original RGB acquisition timestamps are retained, without RGB |
| interpolation. Use the updated LO-World code for timeline/source validation. |
|
|
| Use `data/bev_layout/12hz_val150_81f` as `--dataset-dir` with |
| `--dataset-split val` in the batch generator; use the train package with |
| `--dataset-split train`. The updated preparation shell pipeline also stages |
| these installed packages into its existing work_dirs generation paths. |
| |
| See the [training package](data/bev_layout/12hz_train700_81f/README.md) and |
| [validation package](data/bev_layout/12hz_val150_81f/README.md). |
| |
