| --- |
| pretty_name: DEGAS Pretrained Avatars |
| license: cc-by-nc-sa-4.0 |
| language: |
| - en |
| tags: |
| - 3d |
| - avatar |
| - gaussian-splatting |
| - smplx |
| - degas |
| library_name: pytorch |
| --- |
| |
| # DEGAS: pretrained full-body Gaussian avatars |
|
|
| Trained avatars for [**DEGAS: Detailed Expressions on Full-Body Gaussian |
| Avatars**](https://initialneil.github.io/DEGAS) (3DV 2025), built on the |
| [DREAMS-AVATAR](https://huggingface.co/datasets/initialneil/DREAMS-AVATAR) captures. |
|
|
| - **Code:** [github.com/initialneil/DEGAS](https://github.com/initialneil/DEGAS) |
| - **Dataset:** [initialneil/DREAMS-AVATAR](https://huggingface.co/datasets/initialneil/DREAMS-AVATAR) |
| - **Paper:** [arXiv:2408.10588](https://arxiv.org/abs/2408.10588) |
| - **Registration:** our multiview SMPL-X tracker (the fitted result ships with the dataset) |
|
|
| ## Available avatars |
|
|
| | Avatar | Trained on | Iterations | Face driven by | |
| |---|---|---|---| |
| | `P1_smplx` | P1C1 | 800k | the fitted SMPL-X expression + jaw | |
| | `P1_dpe` | P1C1 | 800k | a per-frame 512-d DPE code, mesh face neutralised | |
| | `P2_smplx` | P2C1 | 800k | SMPL-X expression + jaw *(training, uploads when it finishes)* | |
| | `P3_smplx` | P3C1 | 800k | SMPL-X expression + jaw *(training, uploads when it finishes)* | |
| | `P4_smplx` | P4C1 | 800k | SMPL-X expression + jaw *(training, uploads when it finishes)* | |
|
|
| `P1_smplx` and `P1_dpe` are the same subject, the same data, the same architecture and the |
| same schedule. They differ **only** in how the face is animated, so they are a clean |
| side-by-side of the two options the code supports. |
|
|
| Each capture's `C1` session is the training session and `C2` is held out, so evaluating an |
| avatar on `PxC2` is a genuine cross-session drive. |
|
|
| ## Layout |
|
|
| Each avatar is stored exactly as `degas_eval.py --model_path` expects: |
|
|
| ``` |
| avatars/<NAME>/ |
| config.yaml the run's own config, paths made portable |
| avatar.json provenance: subject, capture, face path |
| point_cloud/iteration_800000/checkpoint.pt the model (~1.3 GB) |
| point_cloud/iteration_800000/point_cloud.ply the Gaussians |
| point_cloud/iteration_800000/smplx_refined.pt the refined SMPL-X parameters |
| ``` |
|
|
| ## Usage |
|
|
| ```bash |
| git clone --recursive https://github.com/initialneil/DEGAS && cd DEGAS |
| # install per the repo's Setup section, and place SMPLX_NEUTRAL.npz under model/data/ |
| |
| pip install -U "huggingface_hub[cli]" |
| hf download initialneil/DEGAS --include "avatars/P1_smplx/*" --local-dir weights |
| hf download initialneil/DREAMS-AVATAR --repo-type dataset \ |
| --include "data/P1C2/*" --local-dir DREAMS-AVATAR |
| |
| python degas_eval.py \ |
| --dat_dir "$PWD/DREAMS-AVATAR/data/P1C2" \ |
| --ip none \ |
| --model_path "$PWD/weights/avatars/P1_smplx" \ |
| --configs "$PWD/weights/avatars/P1_smplx/config.yaml" \ |
| dataset.cache_dir="$PWD/cache/P1C2_eval_cam3" \ |
| dataset.test.cam_select=[3] \ |
| "dataset.test.frm_list=np.arange(0, 293, 8).tolist()" |
| ``` |
|
|
| **You pass the avatar's config yourself.** `degas_eval.py` does not read the run's |
| `config.yaml` behind your back; it used to append it last, which let a training-time value |
| silently outrank the command line. Each avatar here ships a portable, fully resolved |
| `config.yaml` (`dat_dir: ???`, no machine-specific paths), so naming it alone is enough. |
| Anything required but missing is reported by name up front, not deep inside the model. |
|
|
| **`--model_path` must be absolute.** A relative one is resolved *inside* `--dat_dir` |
| (`os.path.join(dat_dir, model_path)`), because that is where a training run writes by |
| default, so `weights/avatars/P1_smplx` fails with a `FileNotFoundError` on a path you never |
| typed. Hence `$PWD`. |
| |
| Three more things fail quietly rather than loudly, so they are worth stating plainly: |
| |
| 1. **Pass the avatar's own `config.yaml`**, or it is not evaluated under the settings it was |
| trained with. Command-line `key=value` overrides still outrank every config file, which |
| is why the test split is set that way above. |
| 2. **Give every capture its own `dataset.cache_dir`.** Decoded frames are named |
| `cam%02d/%08d.jpg` with no capture in the path, so a shared cache would serve P1C1's |
| frame 110 for P1C2's frame 110. |
| 3. **For `P1_dpe`, `dataset.with_face_dpe` must point at the session you are driving |
| with**, not the one it was trained on. These avatars store it as the *relative* value |
| `dpe`, and the loader resolves relative paths against `dat_dir`, so they follow |
| `--dat_dir` on their own. A config saved by your own training run holds an absolute path |
| to the training session's codes instead, and there you must override it. |
| |
| For `P1_dpe`, point `--model_path` and `--configs` at that avatar instead, and add |
| `dataset.with_face_dpe=` for the session you are driving with. |
| |
| ## Training setup |
| |
| 800k iterations per avatar, roughly 41 h on a single RTX 3090. Trained on all frames of the |
| `C1` session across 29 of 32 cameras at `2x` (1024x750). **cam03**, the frontal tele face |
| closeup, is held out as an unseen view, and the whole `C2` session is held out as an unseen |
| session. The DPE codes were extracted from cam07 and cam30, both frontal tele views inside |
| the training split, so no evaluation view leaks into the face conditioning. |
| |
| ## Which face path won |
| |
| On P1, with identical data, schedule and architecture, `P1_smplx` **beat** `P1_dpe`. That is |
| why the other released avatars use the SMPL-X path. `P1_dpe` is the paper-faithful |
| formulation, and the right choice when you have no trustworthy face fit, but it is not the |
| stronger one on this data. |
|
|
| Read that carefully, though: whole-image and even head-crop metrics could not separate the |
| two at all, differing only in the fourth decimal. A face ablation moves roughly 1% of the |
| pixels, so whole-image PSNR is dominated by torso and clothing. Only a mouth region defined |
| from the jaw-driven SMPL-X vertices distinguished them (PSNR +0.58, SSIM +0.015, LPIPS |
| -10%). It is one subject, so treat it as a direction, not a settled result. |
|
|
| ## Limitations |
|
|
| - **`P1_dpe` evaluation is not deterministic.** The per-frame face code is a random convex |
| combination of the two camera codes, redrawn on every sample, and that path runs at eval |
| time as well as during training. Restrict the evaluation to one camera per frame if you |
| need reproducible numbers. |
| - One subject per avatar. These are personalised avatars, not a generalisable model. |
| - Driving signal must be a registered SMPL-X sequence in the DREAMS-AVATAR convention. |
| - Densification is disabled during training, and there is no oral-cavity geometry, so teeth |
| render as a specular smear when the mouth opens wide. |
| - The SMPL-X body model is **not** included here. Register at |
| [smpl-x.is.tue.mpg.de](https://smpl-x.is.tue.mpg.de/) and install it yourself. |
| |
| ## Licence |
| |
| **[CC BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/)**, non-commercial |
| use only, matching the DEGAS code. The DREAMS-AVATAR captures these were trained on are |
| [CC BY-NC 4.0](https://creativecommons.org/licenses/by-nc/4.0/). Any commercial use needs |
| formal permission first. |
| |
| ## Citation |
| |
| ```bibtex |
| @inproceedings{shao2025degas, |
| title = {{DEGAS: Detailed Expressions on Full-Body Gaussian Avatars}}, |
| author = {Zhijing Shao and Duotun Wang and Qing-Yao Tian and Yao-Dong Yang and Hengyu Meng and Zeyu Cai and Bo Dong and Yu Zhang and Kang Zhang and Zeyu Wang}, |
| booktitle = {Proceedings of the International Conference on 3D Vision (3DV)}, |
| year = {2025} |
| } |
| ``` |
| |
