|
Download README.md from sahoosk/anlp-assignment3: direct link, hf CLI and curl.
- Browser
- Download file 2.01 kB
-
https://huggingface.co/sahoosk/anlp-assignment3/resolve/main/README.md
- Command line
-
hf download hf://sahoosk/anlp-assignment3/README.md
-
curl -L -o README.md https://huggingface.co/sahoosk/anlp-assignment3/resolve/main/README.md
2.01 kB
| tags: | |
| - cipher-decryption | |
| - mamba | |
| - deltanet | |
| - graph-neural-network | |
| - fake-news-detection | |
| library_name: pytorch | |
| ## Layout | |
| | Folder | Contents | | |
| | --- | --- | | |
| | `part1/<model>/` | Part A cipher decoders: `best.pt`, training history, protocol, summary, test and validation metrics | | |
| | `part2/checkpoints/<model>_<feature>/seed<k>/` | Part B checkpoints (best validation macro F1), per-run history and result | | |
| | `part2/` | `results.json` (means), `summary.json` (mean ± std), `runs.json` (every run) | | |
| | `results/` | metric files, cipher analysis, summary tables and figures | | |
| | `outputs/part1/` | decrypted validation and test sets for every model | | |
| | `logs/` | training, evaluation and analysis logs | | |
| | `report/` | the report PDF and its LaTeX source | | |
| ## Part A: cipher decryption (test set, 1,318 messages) | |
| | Model | Weights | Char acc | Word acc | F1 | Mean Levenshtein | Params | | |
| | --- | --- | ---: | ---: | ---: | ---: | ---: | | |
| | transformer | `part1/transformer/best.pt` | 0.0904 | 0.0313 | 0.3589 | 663.5 | 8,165,888 | | |
| | mamba | `part1/mamba/best.pt` | 0.0923 | 0.0291 | 0.3914 | 644.1 | 6,858,752 | | |
| | deltanet | `part1/deltanet/best.pt` | 0.0915 | 0.0318 | 0.3528 | 665.3 | 8,195,480 | | |
| | hybrid | `part1/hybrid/best.pt` | 0.0913 | 0.0376 | 0.3764 | 650.0 | 7,527,116 | | |
| Run inference with the assignment code (downloads from this repository by default): | |
| ```bash | |
| bash scripts/part1/test_assn.sh --model mamba | |
| ``` | |
| ## Part B: UPFD GossipCop (test set, mean ± std over 5 seeds) | |
| | Configuration | Accuracy | Macro F1 | | |
| | --- | ---: | ---: | | |
| | root_mlp_bert | 0.7383 ± 0.0030 | 0.7369 ± 0.0042 | | |
| | gcn_bert | 0.9479 ± 0.0027 | 0.9479 ± 0.0027 | | |
| | graphsage_bert | 0.9274 ± 0.0027 | 0.9274 ± 0.0027 | | |
| | gat_bert | 0.9658 ± 0.0021 | 0.9658 ± 0.0021 | | |
| | bigcn_bert | 0.9327 ± 0.0033 | 0.9327 ± 0.0033 | | |
| | graphsage_profile | 0.9035 ± 0.0049 | 0.9034 ± 0.0050 | | |
| | graphsage_spacy | 0.9237 ± 0.0071 | 0.9237 ± 0.0072 | | |
| | graphsage_content | 0.9486 ± 0.0071 | 0.9485 ± 0.0071 | | |