Yangyihui commited on
Commit
79e5656
路
verified 路
1 Parent(s): bfffc52

Align model card with the public ScaleNav code and project page

Browse files
Files changed (1) hide show
  1. README.md +30 -19
README.md CHANGED
@@ -15,52 +15,58 @@ tags:
15
 
16
  **Scalable Internet Video Pretraining for Language-Conditioned Local Navigation**
17
 
18
- [Code](https://github.com/yangyihui0502/ScaleNav) 路 [Data](https://huggingface.co/datasets/Yangyihui/ScaleNav-Data) 路 [Project page](https://yangyihui0502.github.io/scalenav-review/)
19
 
20
- ScaleNav learns navigation priors from Internet videos through action-free
21
- pretraining, then predicts local actions and future observations conditioned on
22
- RGB observations and a navigation instruction.
 
23
 
24
  ## Download
25
 
26
- From the ScaleNav code repository:
 
 
27
 
28
  ```bash
29
- hf auth login
30
  python scripts/download_checkpoint.py --output checkpoints/scalenav
31
  ```
32
 
33
- The downloader uses a fixed Hugging Face revision and verifies each file's
34
- SHA-256. A [GitHub mirror](https://github.com/yangyihui0502/ScaleNav/releases/tag/scalenav-v1)
35
- is available with `--source github`.
36
 
37
  To download directly with the Hugging Face CLI:
38
 
39
  ```bash
40
- hf download Yangyihui/ScaleNav --revision v1.0 --local-dir checkpoints/scalenav \
41
  --include 'transformer/*' ar_model.pt train_meta.pt SHA256SUMS.json
42
  ```
43
 
44
  Download the Wan VAE, text encoder and tokenizer using the
45
- [installation and checkpoint guide](https://github.com/yangyihui0502/ScaleNav#checkpoints).
46
 
47
  ## Inference
48
 
49
  Prepare the paired observation and its original instruction using the
50
- [data instructions](https://github.com/yangyihui0502/ScaleNav#data), then run:
51
 
52
  ```bash
53
  awm check --checkpoint checkpoints/scalenav --sha256
54
  awm infer --checkpoint checkpoints/scalenav \
55
  --base-model checkpoints/Wan2.2-TI2V-5B \
56
  --image data/example/observation.png \
57
- --prompt "$(cat data/example/instruction.txt)" --output outputs/example
 
58
  ```
59
 
60
- This produces a nine-step action trajectory and a generated video. Actions are
61
- per-frame forward/right/clockwise-yaw displacements in metres/metres/radians.
 
 
 
62
  Continuous navigation uses incoming camera frames through
63
- [`awm serve`](https://github.com/yangyihui0502/ScaleNav/blob/main/docs/INFERENCE.md).
64
 
65
  ## Model settings
66
 
@@ -85,6 +91,11 @@ Continuous navigation uses incoming camera frames through
85
  | `SHA256SUMS.json` | File checksums |
86
 
87
  Total checkpoint size: **10.02 GB**. The separate Wan components occupy about
88
- 14.2 GB. The code repository includes [training](https://github.com/yangyihui0502/ScaleNav/blob/main/docs/TRAINING.md),
89
- [evaluation](https://github.com/yangyihui0502/ScaleNav/blob/main/docs/EVALUATION.md)
90
- and [trajectory visualization](https://github.com/yangyihui0502/ScaleNav#inference).
 
 
 
 
 
 
15
 
16
  **Scalable Internet Video Pretraining for Language-Conditioned Local Navigation**
17
 
18
+ [Code](https://github.com/MachEmbodied/ScaleNav) 路 [Data](https://huggingface.co/datasets/Yangyihui/ScaleNav-Data) 路 [Project page](https://zju3dv.github.io/ScaleNav/)
19
 
20
+ ScaleNav learns navigation priors from Internet egocentric videos through
21
+ action-free pretraining and transfers them to action generation for
22
+ language-conditioned local navigation. The goal is to reach a language-specified
23
+ target visible in the initial observation and stop at the instructed location.
24
 
25
  ## Download
26
 
27
+ Install ScaleNav using the [installation guide](https://github.com/MachEmbodied/ScaleNav#installation),
28
+ then run from the code repository root. This model repository is public;
29
+ authentication is optional:
30
 
31
  ```bash
 
32
  python scripts/download_checkpoint.py --output checkpoints/scalenav
33
  ```
34
 
35
+ The downloader uses the revision and file checksums in the
36
+ [download manifest](https://github.com/MachEmbodied/ScaleNav/blob/main/docs/checkpoint-download-manifest.json),
37
+ resumes interrupted downloads and verifies SHA-256.
38
 
39
  To download directly with the Hugging Face CLI:
40
 
41
  ```bash
42
+ hf download Yangyihui/ScaleNav --revision 8808b3589142b52e3bf0719aa0ecff8f86715a4a --local-dir checkpoints/scalenav \
43
  --include 'transformer/*' ar_model.pt train_meta.pt SHA256SUMS.json
44
  ```
45
 
46
  Download the Wan VAE, text encoder and tokenizer using the
47
+ [installation and checkpoint guide](https://github.com/MachEmbodied/ScaleNav#checkpoints).
48
 
49
  ## Inference
50
 
51
  Prepare the paired observation and its original instruction using the
52
+ [single-image inference guide](https://github.com/MachEmbodied/ScaleNav/blob/main/docs/INFERENCE.md#single-image-inference), then run:
53
 
54
  ```bash
55
  awm check --checkpoint checkpoints/scalenav --sha256
56
  awm infer --checkpoint checkpoints/scalenav \
57
  --base-model checkpoints/Wan2.2-TI2V-5B \
58
  --image data/example/observation.png \
59
+ --prompt "$(cat data/example/instruction.txt)" --seed 1065612570 \
60
+ --output outputs/example
61
  ```
62
 
63
+ This saves the nine-step action trajectory as `observation_trajectory.npy` and
64
+ a trajectory visualization as `observation_trajectory.png`. Add `--decode-video`
65
+ to also save nine predicted RGB frames as `observation_generated_raw.mp4`.
66
+ Actions are per-frame forward/right/clockwise-yaw displacements in
67
+ metres/metres/radians.
68
  Continuous navigation uses incoming camera frames through
69
+ [`awm serve`](https://github.com/MachEmbodied/ScaleNav/blob/main/docs/INFERENCE.md).
70
 
71
  ## Model settings
72
 
 
91
  | `SHA256SUMS.json` | File checksums |
92
 
93
  Total checkpoint size: **10.02 GB**. The separate Wan components occupy about
94
+ 14.2 GB. The code repository includes [training](https://github.com/MachEmbodied/ScaleNav/blob/main/docs/TRAINING.md),
95
+ [evaluation](https://github.com/MachEmbodied/ScaleNav/blob/main/docs/EVALUATION.md)
96
+ and [trajectory visualization](https://github.com/MachEmbodied/ScaleNav/blob/main/docs/INFERENCE.md#outputs).
97
+
98
+ The released checkpoint is the final joint-fine-tuning checkpoint at step 6500.
99
+ It contains model weights and deployment metadata. Optimizer, scheduler and RNG
100
+ state for exact training resumption are not included. The Wan VAE, text encoder
101
+ and tokenizer are downloaded separately.