Upload README.md with huggingface_hub
Browse files
README.md
CHANGED
|
@@ -1,3 +1,76 @@
|
|
| 1 |
---
|
| 2 |
license: apache-2.0
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 3 |
---
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
---
|
| 2 |
license: apache-2.0
|
| 3 |
+
library_name: pytorch
|
| 4 |
+
tags:
|
| 5 |
+
- robotics
|
| 6 |
+
- uav
|
| 7 |
+
- drone
|
| 8 |
+
- flow-matching
|
| 9 |
+
- autonomous-navigation
|
| 10 |
+
- flightmare
|
| 11 |
---
|
| 12 |
+
|
| 13 |
+
# FlowPilot Checkpoints
|
| 14 |
+
|
| 15 |
+
Model checkpoints for **FlowPilot**, a Flow Matching based world-action model for real-time onboard UAV navigation.
|
| 16 |
+
|
| 17 |
+
The model uses a dual-stream MoT to jointly model video latents and action tokens. Actions are parameterized by a 7th-order Bernstein polynomial with 8 control points (3 fixed by the initial state and 5 predicted by the network), producing 48 waypoints at 30 Hz.
|
| 18 |
+
|
| 19 |
+
- Code: [flyingbitac/FlowPilot](https://github.com/flyingbitac/FlowPilot)
|
| 20 |
+
|
| 21 |
+
## Files
|
| 22 |
+
|
| 23 |
+
| File | Size | Description |
|
| 24 |
+
| --- | --- | --- |
|
| 25 |
+
| `ckpt/phase2/ckpt_final.pt` | 651 MB | Phase-2 checkpoint. The inference state dict is stored under the `model` key; `step`, `optimizer`, and `lr_scheduler` are also included. |
|
| 26 |
+
| `ckpt/action_stats.pt` | 3.1 KB | Action and state normalization statistics. |
|
| 27 |
+
| `ckpt/coeff_stats.pt` | 2.7 KB | Bernstein coefficient normalization statistics. |
|
| 28 |
+
|
| 29 |
+
The Wan2.2 VAE weights are not stored in this repository; download the official file from [Wan-AI/Wan2.2-TI2V-5B](https://huggingface.co/Wan-AI/Wan2.2-TI2V-5B) as shown below.
|
| 30 |
+
|
| 31 |
+
## Usage
|
| 32 |
+
|
| 33 |
+
Clone the code repository first, then download the checkpoints into its root:
|
| 34 |
+
|
| 35 |
+
```bash
|
| 36 |
+
git clone https://github.com/flyingbitac/FlowPilot.git
|
| 37 |
+
cd FlowPilot
|
| 38 |
+
|
| 39 |
+
hf download YuDinging/FlowPilot --include "ckpt/*" --local-dir .
|
| 40 |
+
hf download Wan-AI/Wan2.2-TI2V-5B Wan2.2_VAE.pth --local-dir ckpt
|
| 41 |
+
```
|
| 42 |
+
|
| 43 |
+
If `hf` is not available, install it with `pip install -U huggingface_hub`.
|
| 44 |
+
|
| 45 |
+
The `--include "ckpt/*"` filter matters: without it, `hf download` would also write this model card as `README.md` and overwrite the README of the code repository.
|
| 46 |
+
|
| 47 |
+
The files land under `ckpt/`, the default location expected by `configs/runtime/rviz.py`:
|
| 48 |
+
|
| 49 |
+
```
|
| 50 |
+
ckpt/
|
| 51 |
+
βββ phase2/ckpt_final.pt
|
| 52 |
+
βββ Wan2.2_VAE.pth
|
| 53 |
+
βββ action_stats.pt
|
| 54 |
+
βββ coeff_stats.pt
|
| 55 |
+
```
|
| 56 |
+
|
| 57 |
+
`hf download` also creates a `.cache/huggingface/` directory in the current directory. It only holds download metadata and can be removed afterwards. If you prefer to keep the weights outside the repository, download to another directory with `--local-dir` and symlink its `ckpt/` into the repository instead.
|
| 58 |
+
|
| 59 |
+
|
| 60 |
+
|
| 61 |
+
Then start the simulation from the code repository:
|
| 62 |
+
|
| 63 |
+
```bash
|
| 64 |
+
python docker/docker.py build
|
| 65 |
+
python docker/docker.py start
|
| 66 |
+
```
|
| 67 |
+
|
| 68 |
+
## Intended Use
|
| 69 |
+
|
| 70 |
+
These weights are released for research on learning-based UAV navigation, mainly for simulation and onboard inference experiments. They are not certified for safety-critical flight.
|
| 71 |
+
|
| 72 |
+
## License
|
| 73 |
+
|
| 74 |
+
The checkpoints are released under the Apache License 2.0.
|
| 75 |
+
|
| 76 |
+
`ckpt/Wan2.2_VAE.pth` is downloaded from the official [Wan2.2 model repository](https://huggingface.co/Wan-AI/Wan2.2-TI2V-5B) (Alibaba Wan Team) and is also licensed under Apache-2.0.
|