Improve paper and artifact metadata
Browse files
README.md
CHANGED
|
@@ -3,14 +3,23 @@ license: mit
|
|
| 3 |
library_name: pytorch
|
| 4 |
tags:
|
| 5 |
- co-speech-gesture-generation
|
|
|
|
|
|
|
| 6 |
- motion-generation
|
| 7 |
- smpl-x
|
| 8 |
- streamtalk
|
|
|
|
| 9 |
datasets:
|
| 10 |
- H-Liu1997/BEAT2
|
| 11 |
---
|
| 12 |
|
| 13 |
-
# StreamTalk
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 14 |
|
| 15 |
These are hardware-agnostic FP32 generator state dictionaries for
|
| 16 |
`DiffusionDITNetPartsFixedExpressions2PostNormInteraction2`. They were retrained
|
|
@@ -54,3 +63,18 @@ linked StreamTalk release and matching the published SHA256 values.
|
|
| 54 |
End-to-end inference also requires WavLM Large, the bundled
|
| 55 |
`SimpleSpeechModel`, SMPL-X neutral, and the BEAT2 retrieval database. Those
|
| 56 |
runtime assets are independent of the generator checkpoint format.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 3 |
library_name: pytorch
|
| 4 |
tags:
|
| 5 |
- co-speech-gesture-generation
|
| 6 |
+
- streaming-generation
|
| 7 |
+
- real-time-generation
|
| 8 |
- motion-generation
|
| 9 |
- smpl-x
|
| 10 |
- streamtalk
|
| 11 |
+
- arxiv:2608.01643
|
| 12 |
datasets:
|
| 13 |
- H-Liu1997/BEAT2
|
| 14 |
---
|
| 15 |
|
| 16 |
+
# StreamTalk official CFG checkpoints
|
| 17 |
+
|
| 18 |
+
Official checkpoints for **StreamTalk: Streaming Co-Speech Gesture Generation with Key-Pose Anchoring** (ECCV 2026), a closed-loop method for real-time, long-horizon **co-speech gesture generation**.
|
| 19 |
+
|
| 20 |
+
- [Paper](https://arxiv.org/abs/2608.01643) 路 [Hugging Face Paper page](https://huggingface.co/papers/2608.01643)
|
| 21 |
+
- [Project page](https://xiangyuezhang.com/StreamTalk/) 路 [Code](https://github.com/Xiangyue-Zhang/StreamTalk)
|
| 22 |
+
- [Generated inference data](https://huggingface.co/datasets/X-Zhang/StreamTalk-Inference-Data)
|
| 23 |
|
| 24 |
These are hardware-agnostic FP32 generator state dictionaries for
|
| 25 |
`DiffusionDITNetPartsFixedExpressions2PostNormInteraction2`. They were retrained
|
|
|
|
| 63 |
End-to-end inference also requires WavLM Large, the bundled
|
| 64 |
`SimpleSpeechModel`, SMPL-X neutral, and the BEAT2 retrieval database. Those
|
| 65 |
runtime assets are independent of the generator checkpoint format.
|
| 66 |
+
|
| 67 |
+
## Citation
|
| 68 |
+
|
| 69 |
+
```bibtex
|
| 70 |
+
@inproceedings{zhang2026streamtalk,
|
| 71 |
+
title={StreamTalk: Streaming Co-Speech Gesture Generation with Key-Pose Anchoring},
|
| 72 |
+
author={Zhang, Xiangyue and Li, Jianfang and Zhang, Jiaxu and Yang, Kaixing and Hoi, Steven},
|
| 73 |
+
booktitle={European Conference on Computer Vision},
|
| 74 |
+
year={2026},
|
| 75 |
+
eprint={2608.01643},
|
| 76 |
+
archivePrefix={arXiv},
|
| 77 |
+
primaryClass={cs.CV},
|
| 78 |
+
url={https://arxiv.org/abs/2608.01643}
|
| 79 |
+
}
|
| 80 |
+
```
|