|
Download results/README.md from CrocoSign/pjm-models: direct link, hf CLI and curl.
- Browser
- Download file 3.12 kB
-
https://huggingface.co/CrocoSign/pjm-models/resolve/main/results/README.md
- Command line
-
hf download hf://CrocoSign/pjm-models/results/README.md
-
curl -L -o README.md https://huggingface.co/CrocoSign/pjm-models/resolve/main/results/README.md
3.12 kB
| # Test outputs and logs | |
| Model outputs, references, metric logs and training logs behind the CrocoSign paper. No weights; those are in | |
| `checkpoints/` and as ONNX in `unisign/` and `spamo/`. | |
| ## Uni-Sign (`unisign/eval_<run>/`) | |
| `test_tmp_pres.txt` and `test_tmp_refs.txt` hold one line per test clip (`sample: <key>, prediction: ...` and | |
| `sample: <key>, ground-truth: ...`), `log.txt` the evaluation log with BLEU, ROUGE-L and chrF, `bleurt_BLEURT-20.json` | |
| the per-clip BLEURT-20 scores, `train.log.gz` the training log. Table 3 rows: | |
| | Row | Directory | | |
| |---|---| | |
| | CSL-News MS / SI | `eval_pjm_phase3_full_ms`, `eval_pjm_phase3_full_si` | | |
| | How2Sign MS / SI | `eval_pjm_phase3_full_ms_how2sign`, `eval_pjm_phase3_full_si_how2sign` | | |
| | OpenASL MS / SI | `eval_pjm_phase3_full_ms_openasl`, `eval_pjm_phase3_full_si_openasl` | | |
| | scratch MS / SI | `eval_pjm_phase3_full_ms_scratch`, `eval_pjm_phase3_full_si_scratch` | | |
| The other directories are the second seeds (`_s43`), the chained and distilled experiments, and the SI | |
| regularisation experiments (`_aug`, `_drop03*`). `greedy_ens.log` is the greedy ensemble selection. | |
| ## SpaMo and Mod-SpaMo (`spamo/<run>/`) | |
| `test_outputs.txt` holds reference and generated text per clip (lowercased, in test-set order without keys), plus | |
| the wandb summary, config and console log of the test run where they exist. Table 3 rows: | |
| | Row | Directory | | |
| |---|---| | |
| | SpaMo PHOENIX-2014T MS | `2026-05-05T23-00-55_test_upstream_pjm_ms_dual` | | |
| | SpaMo PHOENIX-2014T SI | `2026-10-31T07-09-05_test_upstream_pjm_si_dual` | | |
| | SpaMo scratch MS / SI | `2026-05-05T21-58-47_test_v30_dual_ms_scratch`, `2026-05-06T15-27-35_test_v32_dual_si_scratch` | | |
| | Mod-SpaMo PHOENIX-2014T MS | `2026-05-04T09-09-26_test_v27_quad_xattn` | | |
| | Mod-SpaMo PHOENIX-2014T SI | `2026-10-31T15-20-57_test_v27_quad_si_filtered` | | |
| | Mod-SpaMo scratch MS / SI | `2026-05-05T22-10-23_test_v31_quad_ms_scratch`, `2026-05-07T04-18-55_test_v33_quad_si_scratch` | | |
| `run_phoenix_si.log.gz` is the log of the SpaMo and Mod-SpaMo PHOENIX-2014T SI runs, `finetune_pjm_ms.log.gz` | |
| the training log of the SpaMo PHOENIX-2014T MS run. BLEURT for the older SpaMo rows was computed separately and | |
| is not stored here. | |
| ## Polish targets (`polish/`) | |
| `pjm_pl_{ms,si}_how2sign`: Uni-Sign How2Sign trained on Polish targets (`pl_fixed` of | |
| `CrocoSign/pjm-data/texts/texts_fixed.csv`), weights in `checkpoints/unisign/pjm_pl_{ms,si}_how2sign/`. | |
| `en2pl_{ms,si}_how2sign`: the English How2Sign outputs translated to Polish by a language model (Claude Sonnet). | |
| Both are scored against `pl_fixed`; see the "Polish targets" paragraph of the paper. | |
| ## Training-size control (`unisign/eval_pjm_ms_sub_openasl/`) | |
| Uni-Sign OpenASL trained on the MS training set cut to the SI size (10,062 pairs, every MS signer kept with about | |
| 42% of their clips, `split_train_ms_sub.csv` here) with the SI schedule, and tested on the full MS test set: 7.19 | |
| BLEU-4 and 39.75 BLEURT, against 8.53 and 42.43 with the full MS training set. Weights in | |
| `checkpoints/unisign/pjm_ms_sub_openasl/`; scripts `make_ms_subset.py` and `queue_ms_subset.sh` in Uni-Sign-PJM. | |