LionelLow commited on
Commit
a883bc2
·
verified ·
1 Parent(s): eac8cef

Upload README.md with huggingface_hub

Browse files
Files changed (1) hide show
  1. README.md +79 -0
README.md ADDED
@@ -0,0 +1,79 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: other
3
+ license_name: non-commercial-research
4
+ library_name: transformers
5
+ base_model: facebook/mbart-large-cc25
6
+ pipeline_tag: translation
7
+ tags:
8
+ - sign-language
9
+ - sign-language-production
10
+ - text-to-gloss
11
+ - retrieval
12
+ language:
13
+ - de
14
+ - zh
15
+ datasets:
16
+ - LionelLow/SignSparK_data
17
+ ---
18
+
19
+ # SignSparK-Retrieval
20
+
21
+ Text-to-gloss models and isolated-sign annotations for the retrieval pipeline of
22
+ [SignSparK](https://github.com/JianHe0628/SignSparK): text → glosses → retrieved signs →
23
+ stitched SignSparK LMDB → SignSparK fills the transitions.
24
+
25
+ Code: [JianHe0628/SignSparK_Retrieval](https://github.com/JianHe0628/SignSparK_Retrieval).
26
+ Pose data: [LionelLow/SignSparK_data](https://huggingface.co/datasets/LionelLow/SignSparK_data).
27
+
28
+ ## Contents
29
+
30
+ | Path | What |
31
+ |---|---|
32
+ | `t2g/<dataset>/` | fine-tuned text-to-gloss model (Hugging Face MBart export) |
33
+ | `annotations/<prefix>.{train,dev,test}` | isolated-sign spans, used to build the sign dictionaries |
34
+ | `annotations/phoenix_iso_signers.tsv` | PHOENIX-2014T video → signer, for signer-aware retrieval |
35
+ | `predictions/<dataset>/{dev,test}_pres.txt` | the paper's original T2G outputs |
36
+ | `base/<dataset>/` | vocabulary-trimmed mbart-large-cc25, only needed to train T2G |
37
+
38
+ `<dataset>` is `CSL-Daily` or `PHOENIX-2014T`; `<prefix>` is `csl_iso` or `phoenix_iso`.
39
+
40
+
41
+
42
+ ## Text-to-gloss
43
+
44
+ Gloss-level sacreBLEU and WER (%) against the raw LMDB glosses.
45
+
46
+ | Model | dev BLEU-4 | dev WER | test BLEU-4 | test WER |
47
+ |---|---|---|---|---|
48
+ | `t2g/PHOENIX-2014T` | 24.53 | 55.02 | 21.01 | 58.92 |
49
+ | `t2g/CSL-Daily` | 30.05 | 49.18 | 29.48 | 49.49 |
50
+
51
+
52
+ Our released T2G models have been retrained, but score similarly to the original paper's values. Note that our reproduced retrieval and SignSparK results are in the
53
+ [code README](https://github.com/JianHe0628/SignSparK_Retrieval#results).
54
+
55
+ ## Annotations
56
+
57
+ `annotations/<prefix>.<split>` is a pickled list with one dict per sign occurrence:
58
+ `{"video_file", "label", "start", "end", "seq_len"}`. `start`/`end` are frame indices into
59
+ the SignSparK LMDB clip of `video_file` (`end` exclusive).
60
+
61
+ ## License
62
+
63
+ The models are fine-tuned from [mbart-large-cc25](https://huggingface.co/facebook/mbart-large-cc25)
64
+ (vocabulary-trimmed as in [GFSLT-VLP](https://github.com/zhoubenjia/GFSLT-VLP)). The models,
65
+ annotations and predictions are derived from PHOENIX-2014T and CSL-Daily and are provided
66
+ for **non-commercial research use only**, under the terms of those datasets.
67
+
68
+ ## Citation
69
+
70
+ ```bibtex
71
+ @inproceedings{low2026signspark,
72
+ title={SignSparK: Efficient Multilingual Sign Language Production via Sparse Keyframe Learning},
73
+ author={Low, Jianhe and Symeonidis-Herzig, Alexandre and Ivashechkin, Maksym and Sincan, Ozge Mercanoglu and Bowden, Richard},
74
+ booktitle={European Conference on Computer Vision},
75
+ pages={648--670},
76
+ year={2026},
77
+ organization={Springer}
78
+ }
79
+ ```