Release video generation transformer and document downloads
Browse files
README.md
CHANGED
|
@@ -9,7 +9,7 @@ tags:
|
|
| 9 |
|
| 10 |
# LO-World
|
| 11 |
|
| 12 |
-
Pretrained checkpoints for joint future 3D occupancy and LiDAR prediction in a shared BEV latent space.
|
| 13 |
|
| 14 |
## Checkpoints
|
| 15 |
|
|
@@ -34,6 +34,29 @@ hf download JimmyLiaoo/LO-World \
|
|
| 34 |
|
| 35 |
Weights are saved to `ckpts/`. Use the matching configuration and data preparation instructions in the code repository. World Model checkpoints include tokenizer parameters.
|
| 36 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 37 |
## Data downloads
|
| 38 |
|
| 39 |
Download nuScenes train/validation annotation infos to `data/nus_infos/`:
|
|
|
|
| 9 |
|
| 10 |
# LO-World
|
| 11 |
|
| 12 |
+
Pretrained checkpoints for joint future 3D occupancy and LiDAR prediction in a shared BEV latent space, with a GenieDrive transformer for six-camera video generation.
|
| 13 |
|
| 14 |
## Checkpoints
|
| 15 |
|
|
|
|
| 34 |
|
| 35 |
Weights are saved to `ckpts/`. Use the matching configuration and data preparation instructions in the code repository. World Model checkpoints include tokenizer parameters.
|
| 36 |
|
| 37 |
+
## Video generation
|
| 38 |
+
|
| 39 |
+
The released GenieDrive transformer generates six-camera videos at 448 × 800. Install the `loworld-genie` environment and prepare scene RGB/semantic conditions using the LO-World code repository.
|
| 40 |
+
|
| 41 |
+
Download the transformer and its required Wan base model from the repository root:
|
| 42 |
+
|
| 43 |
+
```bash
|
| 44 |
+
hf download JimmyLiaoo/LO-World --include "ckpts/geniedrive/loworld_448x800/**" --local-dir .
|
| 45 |
+
hf download alibaba-pai/Wan2.1-Fun-V1.1-1.3B-Control \
|
| 46 |
+
--local-dir ckpts/geniedrive/Wan2.1-Fun-V1.1-1.3B-Control
|
| 47 |
+
(cd ckpts/geniedrive/loworld_448x800 && sha256sum -c SHA256SUMS)
|
| 48 |
+
```
|
| 49 |
+
|
| 50 |
+
The transformer is saved to `ckpts/geniedrive/loworld_448x800/checkpoint-1000/transformer/`, matching the default inference path. Generate videos for a prepared scene:
|
| 51 |
+
|
| 52 |
+
```bash
|
| 53 |
+
conda run --no-capture-output -n loworld-genie python \
|
| 54 |
+
tools/genie/inference/generate_scene.py \
|
| 55 |
+
--scene-dir work_dirs/visualization/scene_0001
|
| 56 |
+
```
|
| 57 |
+
|
| 58 |
+
Output: six camera MP4s and a surround-view GIF under `work_dirs/visualization/scene_0001/video/`.
|
| 59 |
+
|
| 60 |
## Data downloads
|
| 61 |
|
| 62 |
Download nuScenes train/validation annotation infos to `data/nus_infos/`:
|
ckpts/geniedrive/loworld_448x800/SHA256SUMS
ADDED
|
@@ -0,0 +1 @@
|
|
|
|
|
|
|
| 1 |
+
39991dbcacc030e55e5349a9900d60b3763063a61d656523e6bd01792f9907e1 checkpoint-1000/transformer/diffusion_pytorch_model.safetensors
|
ckpts/geniedrive/loworld_448x800/checkpoint-1000/transformer/diffusion_pytorch_model.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:39991dbcacc030e55e5349a9900d60b3763063a61d656523e6bd01792f9907e1
|
| 3 |
+
size 3337138152
|