Add model card

README.md

@@ -0,0 +1,128 @@
+---
+pretty_name: DEGAS Pretrained Avatars
+license: cc-by-nc-sa-4.0
+language:
+- en
+tags:
+- 3d
+- avatar
+- gaussian-splatting
+- smplx
+- degas
+library_name: pytorch
+---
+
+# DEGAS: pretrained full-body Gaussian avatars
+
+Trained avatars for [**DEGAS: Detailed Expressions on Full-Body Gaussian
+Avatars**](https://initialneil.github.io/DEGAS) (3DV 2025), built on the
+[DREAMS-AVATAR](https://huggingface.co/datasets/initialneil/DREAMS-AVATAR) captures.
+
+- **Code:** [github.com/initialneil/DEGAS](https://github.com/initialneil/DEGAS)
+- **Dataset:** [initialneil/DREAMS-AVATAR](https://huggingface.co/datasets/initialneil/DREAMS-AVATAR)
+- **Paper:** [arXiv:2408.10588](https://arxiv.org/abs/2408.10588)
+- **Registration:** [Holistic-Multiview-Tracker](https://github.com/initialneil/Holistic-Multiview-Tracker)
+
+## Available avatars
+
+| Avatar | Trained on | Iterations | Face driven by |
+|---|---|---|---|
+| `P1_smplx` | P1C1 | 800k | the fitted SMPL-X expression + jaw |
+| `P1_dpe` | P1C1 | 800k | a per-frame 512-d DPE code, mesh face neutralised |
+| `P2_smplx` | P2C1 | 800k | SMPL-X expression + jaw *(training, uploads when it finishes)* |
+| `P3_smplx` | P3C1 | 800k | SMPL-X expression + jaw *(training, uploads when it finishes)* |
+| `P4_smplx` | P4C1 | 800k | SMPL-X expression + jaw *(training, uploads when it finishes)* |
+
+`P1_smplx` and `P1_dpe` are the same subject, the same data, the same architecture and the
+same schedule. They differ **only** in how the face is animated, so they are a clean
+side-by-side of the two options the code supports.
+
+Each capture's `C1` session is the training session and `C2` is held out, so evaluating an
+avatar on `PxC2` is a genuine cross-session drive.
+
+## Layout
+
+Each avatar is stored exactly as `degas_eval.py --model_path` expects:
+
+```
+avatars/<NAME>/
+  config.yaml                                    the run's own config, paths made portable
+  avatar.json                                    provenance: subject, capture, face path
+  point_cloud/iteration_800000/checkpoint.pt     the model (~1.3 GB)
+  point_cloud/iteration_800000/point_cloud.ply   the Gaussians
+  point_cloud/iteration_800000/smplx_refined.pt  the refined SMPL-X parameters
+```
+
+## Usage
+
+```bash
+git clone --recursive https://github.com/initialneil/DEGAS && cd DEGAS
+# install per the repo's Setup section, and place SMPLX_NEUTRAL.npz under model/data/
+
+pip install -U "huggingface_hub[cli]"
+hf download initialneil/DEGAS --include "avatars/P1_smplx/*" --local-dir weights
+hf download initialneil/DREAMS-AVATAR --repo-type dataset \
+    --include "data/P1C2/*" --local-dir DREAMS-AVATAR
+
+python degas_eval.py \
+    --dat_dir DREAMS-AVATAR/data/P1C2 \
+    --ip none \
+    --model_path weights/avatars/P1_smplx \
+    --configs configs/degas_config.yaml,configs/degas_vae_driver.yaml,configs/dreams/p1_train_base.yaml,configs/dreams/p1_face_B.yaml \
+    dataset.cache_dir=cache/P1C2_eval_cam3 \
+    dataset.test.cam_select=[3] \
+    "dataset.test.frm_list=np.arange(0, 293, 8).tolist()"
+```
+
+Three things fail quietly rather than loudly, so they are worth stating plainly:
+
+1. **The run's saved `config.yaml` is applied last**, so an avatar is always evaluated
+   under its own face setting. Only CLI overrides outrank it, which is why the test split
+   is set on the command line above.
+2. **Give every capture its own `dataset.cache_dir`.** Decoded frames are named
+   `cam%02d/%08d.jpg` with no capture in the path, so a shared cache would serve P1C1's
+   frame 110 for P1C2's frame 110.
+3. **For `P1_dpe`, override `dataset.with_face_dpe`** to the session you are driving with
+   (`DREAMS-AVATAR/data/P1C2/dpe/dpe-multi-faces.zip`). The stored config points at the
+   *training* session's codes.
+
+For `P1_dpe`, swap the face config to `configs/dreams/p1_face_A_dpe.yaml` and add that
+override.
+
+## Training setup
+
+800k iterations per avatar, roughly 41 h on a single RTX 3090. Trained on all frames of the
+`C1` session across 29 of 32 cameras at `2x` (1024x750). **cam03**, the frontal tele face
+closeup, is held out as an unseen view, and the whole `C2` session is held out as an unseen
+session. The DPE codes were extracted from cam07 and cam30, both frontal tele views inside
+the training split, so no evaluation view leaks into the face conditioning.
+
+## Limitations
+
+- One subject per avatar. These are personalised avatars, not a generalisable model.
+- Driving signal must be a registered SMPL-X sequence in the DREAMS-AVATAR convention.
+- Densification is disabled during training, and there is no oral-cavity geometry, so teeth
+  render as a specular smear when the mouth opens wide.
+- The SMPL-X body model is **not** included here. Register at
+  [smpl-x.is.tue.mpg.de](https://smpl-x.is.tue.mpg.de/) and install it yourself.
+
+## Licence
+
+**[CC BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/)**, non-commercial
+use only, matching the DEGAS code. The DREAMS-AVATAR captures these were trained on are
+[CC BY-NC 4.0](https://creativecommons.org/licenses/by-nc/4.0/). Any commercial use needs
+formal permission first.
+
+## Citation
+
+```bibtex
+@misc{shao2024degas,
+  title={DEGAS: Detailed Expressions on Full-Body Gaussian Avatars},
+  author={Zhijing Shao and Duotun Wang and Qing-Yao Tian and Yao-Dong Yang and Hengyu Meng and Zeyu Cai and Bo Dong and Yu Zhang and Kang Zhang and Zeyu Wang},
+  year={2024},
+  eprint={2408.10588},
+  archivePrefix={arXiv},
+  primaryClass={cs.CV},
+  url={https://arxiv.org/abs/2408.10588}
+}
+```