| --- |
| library_name: pytorch |
| tags: |
| - motion-generation |
| - text-to-motion |
| - diffusion |
| - streaming |
| - humanml3d |
| language: |
| - en |
| --- |
| |
| # FloodDiffusion 2 |
|
|
| **Efficient and Path Controllable Streaming Motion Generation** |
|
|
| [Code and setup instructions](https://github.com/AlayaLab/FloodDiffusion2) |
|
|
| FloodDiffusion 2 supports continuous text-conditioned motion generation with |
| Partial Attention, diffusion-compatible geometric supervision, and root-path |
| control. This release includes six FD2 checkpoints and the paired 263D LDF/VAE |
| models for first-generation compatibility. |
|
|
| | Checkpoint folder | Configuration | Training step | CFG | |
| |---|---|---:|---:| |
| | `humanml3d_fk_60k` | `df_humanml3d_263.yaml` | 60,000 | 4 | |
| | `humanml3d_path_fk_55k` | `df_humanml3d_263_path.yaml` | 55,000 | 3 | |
| | `humanml3d_babel_fk_200k` | `df_humanml3d_babel_263.yaml` | 200,000 | 4 | |
| | `humanml3d_babel_path_200k` | `df_humanml3d_babel_263_path.yaml` | 200,000 | 4 | |
| | `seed_fk_300k` | `df_seed_138.yaml` | 300,000 | 2 | |
| | `seed_path_fk_300k` | `df_seed_138_path.yaml` | 300,000 | 2 | |
| | `ldf_263` | `ldf_263.yaml` | 180,000 | 6 | |
| | `vae_263` | `vae_263.yaml` | 2,250,000 | — | |
|
|
| Each `checkpoints/<folder>/` contains `model.ckpt`, a matching `config.yaml`, |
| and normalization statistics under `assets/`. FD2 folders also include the |
| quadratic FK matrix `assets/W.npy`. Checkpoints retain model, EMA, optimizer and |
| scheduler state. |
|
|
| ## Setup |
|
|
| Clone the code repository and run the following in a Python 3.10+ environment: |
|
|
| ```bash |
| python setup_project.py |
| ``` |
|
|
| The command installs Python requirements and downloads the four paper checkpoints |
| (HumanML3D text/path and SEED text/path), their assets, and the |
| [shared dependency archive](https://huggingface.co/AlayaLab/FloodDiffusion2/blob/main/deps.zip) |
| for UMT5, T2M and GloVe. Add `--with-data` to also install the |
| [prepared datasets](https://huggingface.co/datasets/AlayaLab/FloodDiffusion2-Data) |
| and paired VAE tokens. |
|
|
| For mesh rendering, obtain SMPL-H from its |
| [official site](https://mano.is.tue.mpg.de/) and import the licensed neutral model: |
|
|
| ```bash |
| python setup_project.py --smplh /path/to/neutral/model.npz |
| ``` |
|
|
| ## Evaluation |
|
|
| ```bash |
| python evaluate.py --config configs/df_humanml3d_263.yaml |
| ``` |
|
|
| For additional checkpoints, use their bundled `checkpoints/<folder>/config.yaml`. |
|
|
| The selected YAML defines the checkpoint, dataset and evaluation settings. |
| Use the corresponding configuration for other variants. LDF uses the paired |
| VAE from this release. |
|
|
| ## Dependencies |
|
|
| Third-party weights and datasets retain their respective licenses. SMPL-H is |
| obtained separately from its official distributor. |
|
|
