MoGround vision-language SAEs
Top-$K$ sparse autoencoders ($K{=}32$) trained on the last-token residual of a frozen VLM as it processes an image-caption pair, which is the position its answer is read from. Trained unsupervised on reconstruction alone, over 1,242,328 image-caption pairs drawn from CC3M (natural photos), PlotQA (charts), WikiArt (paintings) and OpenI (radiology).
The encoder and decoder are separate matrices rather than transposes of each other, both biases are learned, and the decoder rows are kept at unit norm. Width is 8x the model dimension. Training used batch size 4096, learning rate 3e-4 and no weight decay.
| backbone | layer | d_model | d_sae | FVU | alive frac. | used in the paper | file |
|---|---|---|---|---|---|---|---|
| Qwen2.5-VL-3B | 13 | 2048 | 16384 | 0.011 | 0.64 | qwen/layer13/sae.pt |
|
| Qwen2.5-VL-3B | 20 | 2048 | 16384 | 0.016 | 0.75 | qwen/layer20/sae.pt |
|
| Qwen2.5-VL-3B | 28 | 2048 | 16384 | 0.009 | 0.61 | yes | qwen/layer28/sae.pt |
| Qwen2.5-VL-3B | 31 | 2048 | 16384 | 0.005 | 0.4 | qwen/layer31/sae.pt |
|
| LLaVA-NeXT-8B | 12 | 4096 | 32768 | 0.081 | 0.43 | llavanext/layer12/sae.pt |
|
| LLaVA-NeXT-8B | 18 | 4096 | 32768 | 0.026 | 0.4 | yes | llavanext/layer18/sae.pt |
| LLaVA-NeXT-8B | 25 | 4096 | 32768 | 0.01 | 0.45 | llavanext/layer25/sae.pt |
FVU is the share of variance the reconstruction fails to explain, so lower is better. Alive fraction is the share of features that fire at least once on an evaluation batch. Both are measured on held-out activations from the pretraining distribution.
Expect a much higher FVU off that distribution. On MoGround's multiple-choice prompts the layer-28 checkpoint reconstructs to FVU 0.78 about its own centre, and the centre itself has moved: the mean MoGround residual sits 107 away from the decoder bias, against a mean activation norm of 147. Measured against MoGround's own mean the apparent FVU is above 1. That is distribution shift between natural-caption pretraining and answer-time MCQ prompts, not a damaged checkpoint.
Use
from load_sae import load
sae = load("qwen/layer28/sae.pt")
out = sae(residual) # dict with x_hat, z, indices, pre
MANIFEST.json carries a sha256 and the size of every checkpoint.
What these are for, and what they are not
They were trained to ask whether modality distraction is readable, and removable, in a sparse basis. It is readable: a probe on the SAE code separates distracted from robust items above a label-permutation null. It is not removable this way: ablating the most distraction-predictive features does no better than ablating the same number of random ones. Treat these as an interpretability tool, not as a mitigation.