| --- |
| license: cc-by-nc-4.0 |
| language: |
| - en |
| base_model: Wan-AI/Wan2.2-TI2V-5B |
| base_model_relation: finetune |
| tags: |
| - arxiv:2610.25690 |
| - robotics |
| - navigation |
| - video-diffusion |
| - image-text-to-action |
| - safetensors |
| library_name: scalenav |
| pipeline_tag: robotics |
| widget: |
| - example_title: 'ScaleNav: full navigation demo' |
| output: |
| url: replay.mp4 |
| datasets: |
| - Yangyihui/ScaleNav-Data |
| --- |
| |
| # ScaleNav |
|
|
| **Scalable Internet Video Pretraining for Language-Conditioned Local Navigation** |
|
|
| [Paper](https://arxiv.org/abs/2610.25690) 路 [Code](https://github.com/MachEmbodied/ScaleNav) 路 [Data](https://huggingface.co/datasets/Yangyihui/ScaleNav-Data) 路 [Project page](https://zju3dv.github.io/ScaleNav/) 路 [Collection](https://huggingface.co/collections/Yangyihui/scalenav-6ad591e8ff62bb75ec75f645) |
|
|
| ScaleNav learns navigation priors through action-free pretraining on Internet |
| videos, then turns an RGB observation and a language instruction into robot actions. |
| It navigates to a target visible in the initial view and stops at the instructed location. |
|
|
| **5B parameters 路 10.02 GB checkpoint 路 [CC BY-NC 4.0](LICENSE)** |
|
|
| Watch the full demo in the video preview, or visit the |
| [project page](https://zju3dv.github.io/ScaleNav/#demos). |
|
|
| ## Download |
|
|
| Complete the [installation](https://github.com/MachEmbodied/ScaleNav/blob/main/docs/INSTALL.md#environment), |
| then run from the ScaleNav code repository root: |
|
|
| ```bash |
| python scripts/download_checkpoint.py --output checkpoints/scalenav |
| ``` |
|
|
| The downloader resumes interrupted transfers and verifies the pinned |
| [checkpoint checksums](https://github.com/MachEmbodied/ScaleNav/blob/main/docs/checkpoint-download-manifest.json). |
| No HF login is required for this checkpoint. |
|
|
| Also download the **Wan VAE, text encoder and tokenizer** (14.20 GB) following |
| the [checkpoint setup](https://github.com/MachEmbodied/ScaleNav/blob/main/docs/INSTALL.md#checkpoints). |
| For inference, use a BF16-capable NVIDIA GPU; **24 GB GPU memory and 64 GB system RAM** |
| are recommended. |
|
|
| <details> |
| <summary>Download with the Hugging Face CLI</summary> |
|
|
| ```bash |
| hf download Yangyihui/ScaleNav --revision 8808b3589142b52e3bf0719aa0ecff8f86715a4a --local-dir checkpoints/scalenav \ |
| --include 'transformer/*' ar_model.pt train_meta.pt SHA256SUMS.json |
| ``` |
|
|
| </details> |
|
|
| ## Inference |
|
|
| Use ScaleNav's `awm` CLI with your own image and an instruction describing a target |
| visible in it: |
|
|
| ```bash |
| awm check --checkpoint checkpoints/scalenav --sha256 |
| awm infer --checkpoint checkpoints/scalenav \ |
| --base-model checkpoints/Wan2.2-TI2V-5B \ |
| --image /path/to/observation.png \ |
| --prompt "Walk to the chair visible in the initial view." \ |
| --output outputs/example |
| ``` |
|
|
| Replace the image path and instruction with a matching pair. This saves nine |
| per-frame action deltas in `observation_trajectory.npy` and a visualization in |
| `observation_trajectory.png`. Actions are forward/right/clockwise-yaw displacements |
| in metres/metres/radians. Add `--decode-video` to save the nine predicted RGB frames. |
|
|
| For continuous navigation, use [`awm serve`](https://github.com/MachEmbodied/ScaleNav/blob/main/docs/INFERENCE.md#http-api). |
| See the [inference guide](https://github.com/MachEmbodied/ScaleNav/blob/main/docs/INFERENCE.md) |
| for paired examples, outputs and the robot adapter. |
|
|
| ## Model settings |
|
|
| | Setting | Value | |
| | --- | --- | |
| | Observation size | 288 脳 512 | |
| | Observation/action rate | 4 Hz | |
| | Chunk length | 9 frames | |
| | Sampling steps | 4 | |
| | Video / action guidance | 5 / 1 | |
| | Video / action scheduler shifts | 5 / 1 | |
| | History | Initial-frame anchor and two recent chunks | |
| | Action normalization | Symlog, c = 0.01 | |
|
|
| ## Checkpoint contents |
|
|
| | File | Contents | |
| | --- | --- | |
| | `transformer/diffusion_pytorch_model.safetensors` | ScaleNav model weights | |
| | `transformer/config.json` | Model architecture | |
| | `train_meta.pt`, `ar_model.pt` | Deployment metadata | |
| | `SHA256SUMS.json` | File checksums | |
|
|
| Total checkpoint size: **10.02 GB**. The separate Wan components occupy about |
| 14.2 GB. The code repository includes [training](https://github.com/MachEmbodied/ScaleNav/blob/main/docs/TRAINING.md), |
| [evaluation](https://github.com/MachEmbodied/ScaleNav/blob/main/docs/EVALUATION.md) |
| and [trajectory visualization](https://github.com/MachEmbodied/ScaleNav/blob/main/docs/INFERENCE.md#outputs). |
|
|
| The released checkpoint is the final joint-fine-tuning checkpoint at step 6500. |
| It contains model weights and deployment metadata. Optimizer, scheduler and RNG |
| state for exact training resumption are not included. The Wan VAE, text encoder |
| and tokenizer are downloaded separately. |
|
|
| ## License |
|
|
| The ScaleNav fine-tuned checkpoint is released under [CC BY-NC 4.0](LICENSE) |
| for noncommercial use. When sharing it or adaptations, provide attribution, |
| link to the license, and indicate any changes. |
|
|
| ScaleNav modifies [Wan2.2-TI2V-5B](https://huggingface.co/Wan-AI/Wan2.2-TI2V-5B) |
| through navigation pretraining, action alignment and joint fine-tuning. The |
| CC BY-NC 4.0 terms apply to the ScaleNav contributions; upstream Wan materials |
| retain their [Apache-2.0 license](LICENSE-WAN). The |
| [code](https://github.com/MachEmbodied/ScaleNav/blob/main/LICENSE) is separately |
| licensed under Apache-2.0, and [dataset components](https://huggingface.co/datasets/Yangyihui/ScaleNav-Data#sources-and-license) |
| retain their respective terms. |
|
|
