|
Download README.md from haoyanl78/persistent-sparse-autoencoders: direct link, hf CLI and curl.
- Browser
- Download file 1.71 kB
-
https://huggingface.co/haoyanl78/persistent-sparse-autoencoders/resolve/main/README.md
- Command line
-
hf download hf://haoyanl78/persistent-sparse-autoencoders/README.md
-
curl -L -o README.md https://huggingface.co/haoyanl78/persistent-sparse-autoencoders/resolve/main/README.md
1.71 kB
| license: mit | |
| library_name: pytorch | |
| tags: | |
| - sparse-autoencoder | |
| - interpretability | |
| - mechanistic-interpretability | |
| # Persistent Sparse Autoencoders | |
| Trained checkpoints from the paper [*Persistent Sparse Autoencoders: Learning Feature-Specific Timescales in Language Model Representations*](https://arxiv.org/abs/2607.17117). | |
| ## Checkpoints | |
| | Folder | Model | Layer | d_model | Training steps | | |
| |---|---|---|---|---| | |
| | `pythia-160m-deduped/layer_6` | `EleutherAI/pythia-160m-deduped` | 6 | 768 | 244,140 | | |
| | `pythia-410m-deduped/layer_12` | `EleutherAI/pythia-410m-deduped` | 12 | 1024 | 244,140 | | |
| | `pythia-1b-deduped/layer_8` | `EleutherAI/pythia-1b-deduped` | 8 | 2048 | 244,140 | | |
| | `pythia-1.4b-deduped/layer_12` | `EleutherAI/pythia-1.4b-deduped` | 12 | 2048 | 244,140 | | |
| | `gemma-2-2b/layer_12` | `google/gemma-2-2b` | 12 | 2304 | 244,140 | | |
| | `pythia-2.8b-deduped/layer_16` | `EleutherAI/pythia-2.8b-deduped` | 16 | 2560 | 367,648 | | |
| | `pythia-6.9b-deduped/layer_16` | `EleutherAI/pythia-6.9b-deduped` | 16 | 4096 | 367,648 | | |
| | `llama-3.1-8b/layer_19` | `meta-llama/Llama-3.1-8B` | 19 | 4096 | 367,648 | | |
| | `gemma-2-9b/layer_20` | `google/gemma-2-9b` | 20 | 3584 | 367,648 | | |
| | `pythia-12b-deduped/layer_18` | `EleutherAI/pythia-12b-deduped` | 18 | 5120 | 367,648 | | |
| ## Citation | |
| ```bibtex | |
| @misc{luo2026persistentsparseautoencoderslearning, | |
| title={Persistent Sparse Autoencoders: Learning Feature-Specific Timescales in Language Model Representations}, | |
| author={Haoyan Luo and Mateo Espinosa Zarlenga and Mateja Jamnik}, | |
| year={2026}, | |
| eprint={2607.17117}, | |
| archivePrefix={arXiv}, | |
| primaryClass={cs.LG}, | |
| url={https://arxiv.org/abs/2607.17117}, | |
| } | |
| ``` | |