--- pretty_name: DEGAS Pretrained Avatars license: cc-by-nc-sa-4.0 language: - en tags: - 3d - avatar - gaussian-splatting - smplx - degas library_name: pytorch --- # DEGAS: pretrained full-body Gaussian avatars Trained avatars for [**DEGAS: Detailed Expressions on Full-Body Gaussian Avatars**](https://initialneil.github.io/DEGAS) (3DV 2025), built on the [DREAMS-AVATAR](https://huggingface.co/datasets/initialneil/DREAMS-AVATAR) captures. - **Code:** [github.com/initialneil/DEGAS](https://github.com/initialneil/DEGAS) - **Dataset:** [initialneil/DREAMS-AVATAR](https://huggingface.co/datasets/initialneil/DREAMS-AVATAR) - **Paper:** [arXiv:2408.10588](https://arxiv.org/abs/2408.10588) - **Registration:** our multiview SMPL-X tracker (the fitted result ships with the dataset) ## Available avatars | Avatar | Trained on | Iterations | Face driven by | |---|---|---|---| | `P1_smplx` | P1C1 | 800k | the fitted SMPL-X expression + jaw | | `P1_dpe` | P1C1 | 800k | a per-frame 512-d DPE code, mesh face neutralised | | `P2_smplx` | P2C1 | 800k | SMPL-X expression + jaw *(training, uploads when it finishes)* | | `P3_smplx` | P3C1 | 800k | SMPL-X expression + jaw *(training, uploads when it finishes)* | | `P4_smplx` | P4C1 | 800k | SMPL-X expression + jaw *(training, uploads when it finishes)* | `P1_smplx` and `P1_dpe` are the same subject, the same data, the same architecture and the same schedule. They differ **only** in how the face is animated, so they are a clean side-by-side of the two options the code supports. Each capture's `C1` session is the training session and `C2` is held out, so evaluating an avatar on `PxC2` is a genuine cross-session drive. ## Layout Each avatar is stored exactly as `degas_eval.py --model_path` expects: ``` avatars// config.yaml the run's own config, paths made portable avatar.json provenance: subject, capture, face path point_cloud/iteration_800000/checkpoint.pt the model (~1.3 GB) point_cloud/iteration_800000/point_cloud.ply the Gaussians point_cloud/iteration_800000/smplx_refined.pt the refined SMPL-X parameters ``` ## Usage ```bash git clone --recursive https://github.com/initialneil/DEGAS && cd DEGAS # install per the repo's Setup section, and place SMPLX_NEUTRAL.npz under model/data/ pip install -U "huggingface_hub[cli]" hf download initialneil/DEGAS --include "avatars/P1_smplx/*" --local-dir weights hf download initialneil/DREAMS-AVATAR --repo-type dataset \ --include "data/P1C2/*" --local-dir DREAMS-AVATAR python degas_eval.py \ --dat_dir "$PWD/DREAMS-AVATAR/data/P1C2" \ --ip none \ --model_path "$PWD/weights/avatars/P1_smplx" \ --configs "$PWD/weights/avatars/P1_smplx/config.yaml" \ dataset.cache_dir="$PWD/cache/P1C2_eval_cam3" \ dataset.test.cam_select=[3] \ "dataset.test.frm_list=np.arange(0, 293, 8).tolist()" ``` **You pass the avatar's config yourself.** `degas_eval.py` does not read the run's `config.yaml` behind your back; it used to append it last, which let a training-time value silently outrank the command line. Each avatar here ships a portable, fully resolved `config.yaml` (`dat_dir: ???`, no machine-specific paths), so naming it alone is enough. Anything required but missing is reported by name up front, not deep inside the model. **`--model_path` must be absolute.** A relative one is resolved *inside* `--dat_dir` (`os.path.join(dat_dir, model_path)`), because that is where a training run writes by default, so `weights/avatars/P1_smplx` fails with a `FileNotFoundError` on a path you never typed. Hence `$PWD`. Three more things fail quietly rather than loudly, so they are worth stating plainly: 1. **Pass the avatar's own `config.yaml`**, or it is not evaluated under the settings it was trained with. Command-line `key=value` overrides still outrank every config file, which is why the test split is set that way above. 2. **Give every capture its own `dataset.cache_dir`.** Decoded frames are named `cam%02d/%08d.jpg` with no capture in the path, so a shared cache would serve P1C1's frame 110 for P1C2's frame 110. 3. **For `P1_dpe`, `dataset.with_face_dpe` must point at the session you are driving with**, not the one it was trained on. These avatars store it as the *relative* value `dpe`, and the loader resolves relative paths against `dat_dir`, so they follow `--dat_dir` on their own. A config saved by your own training run holds an absolute path to the training session's codes instead, and there you must override it. For `P1_dpe`, point `--model_path` and `--configs` at that avatar instead, and add `dataset.with_face_dpe=` for the session you are driving with. ## Training setup 800k iterations per avatar, roughly 41 h on a single RTX 3090. Trained on all frames of the `C1` session across 29 of 32 cameras at `2x` (1024x750). **cam03**, the frontal tele face closeup, is held out as an unseen view, and the whole `C2` session is held out as an unseen session. The DPE codes were extracted from cam07 and cam30, both frontal tele views inside the training split, so no evaluation view leaks into the face conditioning. ## Which face path won On P1, with identical data, schedule and architecture, `P1_smplx` **beat** `P1_dpe`. That is why the other released avatars use the SMPL-X path. `P1_dpe` is the paper-faithful formulation, and the right choice when you have no trustworthy face fit, but it is not the stronger one on this data. Read that carefully, though: whole-image and even head-crop metrics could not separate the two at all, differing only in the fourth decimal. A face ablation moves roughly 1% of the pixels, so whole-image PSNR is dominated by torso and clothing. Only a mouth region defined from the jaw-driven SMPL-X vertices distinguished them (PSNR +0.58, SSIM +0.015, LPIPS -10%). It is one subject, so treat it as a direction, not a settled result. ## Limitations - **`P1_dpe` evaluation is not deterministic.** The per-frame face code is a random convex combination of the two camera codes, redrawn on every sample, and that path runs at eval time as well as during training. Restrict the evaluation to one camera per frame if you need reproducible numbers. - One subject per avatar. These are personalised avatars, not a generalisable model. - Driving signal must be a registered SMPL-X sequence in the DREAMS-AVATAR convention. - Densification is disabled during training, and there is no oral-cavity geometry, so teeth render as a specular smear when the mouth opens wide. - The SMPL-X body model is **not** included here. Register at [smpl-x.is.tue.mpg.de](https://smpl-x.is.tue.mpg.de/) and install it yourself. ## Licence **[CC BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/)**, non-commercial use only, matching the DEGAS code. The DREAMS-AVATAR captures these were trained on are [CC BY-NC 4.0](https://creativecommons.org/licenses/by-nc/4.0/). Any commercial use needs formal permission first. ## Citation ```bibtex @inproceedings{shao2025degas, title = {{DEGAS: Detailed Expressions on Full-Body Gaussian Avatars}}, author = {Zhijing Shao and Duotun Wang and Qing-Yao Tian and Yao-Dong Yang and Hengyu Meng and Zeyu Cai and Bo Dong and Yu Zhang and Kang Zhang and Zeyu Wang}, booktitle = {Proceedings of the International Conference on 3D Vision (3DV)}, year = {2025} } ```