Translation
Transformers
Safetensors
German
Chinese
sign-language
sign-language-production
text-to-gloss
retrieval
Instructions to use LionelLow/SignSparK_Retrieval with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use LionelLow/SignSparK_Retrieval with Transformers:
# Use a pipeline as a high-level helper # Warning: Pipeline type "translation" is no longer supported in transformers v5. # You must load the model directly (see below) or downgrade to v4.x with: # pip install "transformers<5.0.0" from transformers import pipeline pipe = pipeline("translation", model="LionelLow/SignSparK_Retrieval")# pip install -U transformers accelerate # Load model directly from transformers import AutoModel model = AutoModel.from_pretrained("LionelLow/SignSparK_Retrieval", device_map="auto") - Notebooks
- Google Colab
- Kaggle
Upload README.md with huggingface_hub
Browse files
README.md
ADDED
|
@@ -0,0 +1,79 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
license: other
|
| 3 |
+
license_name: non-commercial-research
|
| 4 |
+
library_name: transformers
|
| 5 |
+
base_model: facebook/mbart-large-cc25
|
| 6 |
+
pipeline_tag: translation
|
| 7 |
+
tags:
|
| 8 |
+
- sign-language
|
| 9 |
+
- sign-language-production
|
| 10 |
+
- text-to-gloss
|
| 11 |
+
- retrieval
|
| 12 |
+
language:
|
| 13 |
+
- de
|
| 14 |
+
- zh
|
| 15 |
+
datasets:
|
| 16 |
+
- LionelLow/SignSparK_data
|
| 17 |
+
---
|
| 18 |
+
|
| 19 |
+
# SignSparK-Retrieval
|
| 20 |
+
|
| 21 |
+
Text-to-gloss models and isolated-sign annotations for the retrieval pipeline of
|
| 22 |
+
[SignSparK](https://github.com/JianHe0628/SignSparK): text → glosses → retrieved signs →
|
| 23 |
+
stitched SignSparK LMDB → SignSparK fills the transitions.
|
| 24 |
+
|
| 25 |
+
Code: [JianHe0628/SignSparK_Retrieval](https://github.com/JianHe0628/SignSparK_Retrieval).
|
| 26 |
+
Pose data: [LionelLow/SignSparK_data](https://huggingface.co/datasets/LionelLow/SignSparK_data).
|
| 27 |
+
|
| 28 |
+
## Contents
|
| 29 |
+
|
| 30 |
+
| Path | What |
|
| 31 |
+
|---|---|
|
| 32 |
+
| `t2g/<dataset>/` | fine-tuned text-to-gloss model (Hugging Face MBart export) |
|
| 33 |
+
| `annotations/<prefix>.{train,dev,test}` | isolated-sign spans, used to build the sign dictionaries |
|
| 34 |
+
| `annotations/phoenix_iso_signers.tsv` | PHOENIX-2014T video → signer, for signer-aware retrieval |
|
| 35 |
+
| `predictions/<dataset>/{dev,test}_pres.txt` | the paper's original T2G outputs |
|
| 36 |
+
| `base/<dataset>/` | vocabulary-trimmed mbart-large-cc25, only needed to train T2G |
|
| 37 |
+
|
| 38 |
+
`<dataset>` is `CSL-Daily` or `PHOENIX-2014T`; `<prefix>` is `csl_iso` or `phoenix_iso`.
|
| 39 |
+
|
| 40 |
+
|
| 41 |
+
|
| 42 |
+
## Text-to-gloss
|
| 43 |
+
|
| 44 |
+
Gloss-level sacreBLEU and WER (%) against the raw LMDB glosses.
|
| 45 |
+
|
| 46 |
+
| Model | dev BLEU-4 | dev WER | test BLEU-4 | test WER |
|
| 47 |
+
|---|---|---|---|---|
|
| 48 |
+
| `t2g/PHOENIX-2014T` | 24.53 | 55.02 | 21.01 | 58.92 |
|
| 49 |
+
| `t2g/CSL-Daily` | 30.05 | 49.18 | 29.48 | 49.49 |
|
| 50 |
+
|
| 51 |
+
|
| 52 |
+
Our released T2G models have been retrained, but score similarly to the original paper's values. Note that our reproduced retrieval and SignSparK results are in the
|
| 53 |
+
[code README](https://github.com/JianHe0628/SignSparK_Retrieval#results).
|
| 54 |
+
|
| 55 |
+
## Annotations
|
| 56 |
+
|
| 57 |
+
`annotations/<prefix>.<split>` is a pickled list with one dict per sign occurrence:
|
| 58 |
+
`{"video_file", "label", "start", "end", "seq_len"}`. `start`/`end` are frame indices into
|
| 59 |
+
the SignSparK LMDB clip of `video_file` (`end` exclusive).
|
| 60 |
+
|
| 61 |
+
## License
|
| 62 |
+
|
| 63 |
+
The models are fine-tuned from [mbart-large-cc25](https://huggingface.co/facebook/mbart-large-cc25)
|
| 64 |
+
(vocabulary-trimmed as in [GFSLT-VLP](https://github.com/zhoubenjia/GFSLT-VLP)). The models,
|
| 65 |
+
annotations and predictions are derived from PHOENIX-2014T and CSL-Daily and are provided
|
| 66 |
+
for **non-commercial research use only**, under the terms of those datasets.
|
| 67 |
+
|
| 68 |
+
## Citation
|
| 69 |
+
|
| 70 |
+
```bibtex
|
| 71 |
+
@inproceedings{low2026signspark,
|
| 72 |
+
title={SignSparK: Efficient Multilingual Sign Language Production via Sparse Keyframe Learning},
|
| 73 |
+
author={Low, Jianhe and Symeonidis-Herzig, Alexandre and Ivashechkin, Maksym and Sincan, Ozge Mercanoglu and Bowden, Richard},
|
| 74 |
+
booktitle={European Conference on Computer Vision},
|
| 75 |
+
pages={648--670},
|
| 76 |
+
year={2026},
|
| 77 |
+
organization={Springer}
|
| 78 |
+
}
|
| 79 |
+
```
|