| license: mit | |
| library_name: pytorch | |
| tags: | |
| - transformer | |
| - cipher-decoding | |
| - byte-latent-transformer | |
| # ANLP Assignment 1: Cipher-to-Text Models | |
| Best inference checkpoints for five encoder-decoder configurations trained to reconstruct plaintext from binary ciphertext: | |
| - C1: Base Transformer | |
| - C2: Rotary positional embeddings (RoPE) | |
| - C3: Grouped-query attention (GQA) | |
| - C4: RMSNorm | |
| - C5: Tokenizer-free byte latent transformer (BLT) | |
| C1--C4 use the included byte-level BPE tokenizer. C5 consumes cipher bytes directly and uses entropy-based patches. Each checkpoint contains the model state and the exact architecture configuration used during training. | |
| Training curves and evaluation results are available in the [Weights & Biases report](https://api.wandb.ai/links/vanshnawander-ocr/mgggijo5). Try the models in the [Gradio demo](https://huggingface.co/spaces/vanshnawander/anlp-assignment1-cipher-decoder-demo). | |