|
Download README.md from haoyanl78/persistent-sparse-autoencoders: direct link, hf CLI and curl.
- Browser
- Download file 1.71 kB
-
https://huggingface.co/haoyanl78/persistent-sparse-autoencoders/resolve/main/README.md
- Command line
-
hf download hf://haoyanl78/persistent-sparse-autoencoders/README.md
-
curl -L -o README.md https://huggingface.co/haoyanl78/persistent-sparse-autoencoders/resolve/main/README.md
1.71 kB
metadata
license: mit
library_name: pytorch
tags:
- sparse-autoencoder
- interpretability
- mechanistic-interpretability
Persistent Sparse Autoencoders
Trained checkpoints from the paper Persistent Sparse Autoencoders: Learning Feature-Specific Timescales in Language Model Representations.
Checkpoints
| Folder | Model | Layer | d_model | Training steps |
|---|---|---|---|---|
pythia-160m-deduped/layer_6 |
EleutherAI/pythia-160m-deduped |
6 | 768 | 244,140 |
pythia-410m-deduped/layer_12 |
EleutherAI/pythia-410m-deduped |
12 | 1024 | 244,140 |
pythia-1b-deduped/layer_8 |
EleutherAI/pythia-1b-deduped |
8 | 2048 | 244,140 |
pythia-1.4b-deduped/layer_12 |
EleutherAI/pythia-1.4b-deduped |
12 | 2048 | 244,140 |
gemma-2-2b/layer_12 |
google/gemma-2-2b |
12 | 2304 | 244,140 |
pythia-2.8b-deduped/layer_16 |
EleutherAI/pythia-2.8b-deduped |
16 | 2560 | 367,648 |
pythia-6.9b-deduped/layer_16 |
EleutherAI/pythia-6.9b-deduped |
16 | 4096 | 367,648 |
llama-3.1-8b/layer_19 |
meta-llama/Llama-3.1-8B |
19 | 4096 | 367,648 |
gemma-2-9b/layer_20 |
google/gemma-2-9b |
20 | 3584 | 367,648 |
pythia-12b-deduped/layer_18 |
EleutherAI/pythia-12b-deduped |
18 | 5120 | 367,648 |
Citation
@misc{luo2026persistentsparseautoencoderslearning,
title={Persistent Sparse Autoencoders: Learning Feature-Specific Timescales in Language Model Representations},
author={Haoyan Luo and Mateo Espinosa Zarlenga and Mateja Jamnik},
year={2026},
eprint={2607.17117},
archivePrefix={arXiv},
primaryClass={cs.LG},
url={https://arxiv.org/abs/2607.17117},
}