Vision backbones for SAE training
Use these pretrained vision backbones to collect activations and train an SAE. Each card lists the model checkpoint and output layer; the training guide covers activation collection, dictionary fitting and evaluation.
SigLIP ViT-B/16
Model ID: backbone-siglip-vit-b16
| Property | Value |
|---|---|
| Artifact | Vision backbone; no SAE dictionary |
| Base model | timm/vit_base_patch16_siglip_224.webli |
| Method | not_applicable |
| Source hook / output | forward_features |
| Layer | Not specified |
| Activation width | 768 |
| Dictionary features | Not specified |
| Checkpoint license | apache-2.0 |
| Checkpoint revision | 8db107e594ed7552e44e108f16650d81ac6de9a6 |
| Tensor format | timm |
Input / activation conventions:
{"image_size": 224, "resize": "square 224, bicubic, antialias", "mean": [0.5, 0.5, 0.5], "std": [0.5, 0.5, 0.5], "scope": "Overcomplete wrapper input transform; preserve checkpoint-specific input conventions when training a new SAE"}
Weight SHA256: b07f8a4bad267232a87888b2527d054d130b1fba18f75e6e26c17766dc26bdc2
ViT-B/16, explicit AugReg2 variant
Model ID: backbone-vit-b16
| Property | Value |
|---|---|
| Artifact | Vision backbone; no SAE dictionary |
| Base model | timm/vit_base_patch16_224.augreg2_in21k_ft_in1k |
| Method | not_applicable |
| Source hook / output | forward_features |
| Layer | Not specified |
| Activation width | 768 |
| Dictionary features | Not specified |
| Checkpoint license | apache-2.0 |
| Checkpoint revision | 063c6c38a5d8510b2e57df480445e94b231dad2c |
| Tensor format | timm |
Input / activation conventions:
{"image_size": 224, "resize": "square 224, bicubic, antialias", "mean": [0.5, 0.5, 0.5], "std": [0.5, 0.5, 0.5], "scope": "Overcomplete wrapper input transform; preserve checkpoint-specific input conventions when training a new SAE"}
Weight SHA256: 32aa17d6e17b43500f531d5f6dc9bc93e56ed8841b8a75682e1bb295d722405b
The upstream wrapper uses untagged vit_base_patch16_224. This card pins one explicit published variant; it does not claim the alias pins these weights.
ResNet-V2-BiT 50x1
Model ID: backbone-resnet-v2-bit
| Property | Value |
|---|---|
| Artifact | Vision backbone; no SAE dictionary |
| Base model | timm/resnetv2_50x1_bit.goog_distilled_in1k |
| Method | not_applicable |
| Source hook / output | forward_features |
| Layer | Not specified |
| Activation width | 2048 |
| Dictionary features | Not specified |
| Checkpoint license | apache-2.0 |
| Checkpoint revision | 39aef79105c877c0d9f9410a761fab8ea28542c3 |
| Tensor format | timm |
Input / activation conventions:
{"image_size": 224, "resize": "square 224, bicubic, antialias", "mean": [0.5, 0.5, 0.5], "std": [0.5, 0.5, 0.5], "scope": "Overcomplete wrapper input transform; preserve checkpoint-specific input conventions when training a new SAE"}
Weight SHA256: d8a84ddc4683b81fb691141c571960a6e64be1c50a3e543a62bd959bf43c98f3
ConvNeXt Small
Model ID: backbone-convnext-small
| Property | Value |
|---|---|
| Artifact | Vision backbone; no SAE dictionary |
| Base model | timm/convnext_small.fb_in1k |
| Method | not_applicable |
| Source hook / output | forward_features |
| Layer | Not specified |
| Activation width | 768 |
| Dictionary features | Not specified |
| Checkpoint license | apache-2.0 |
| Checkpoint revision | 7941c0c00f3ca64f5048de06464b90ad12da0ccb |
| Tensor format | timm |
Input / activation conventions:
{"image_size": 224, "resize": "square 224, bicubic, antialias", "mean": [0.485, 0.456, 0.406], "std": [0.229, 0.224, 0.225], "scope": "Overcomplete wrapper input transform; preserve checkpoint-specific input conventions when training a new SAE"}
Weight SHA256: 5ec5ed9e97fb835093aaf3e6c3fce406b160b367117b6a2b8c7b703386218d1b
DINOv2 ViT-S/14
Model ID: backbone-dinov2-s14
| Property | Value |
|---|---|
| Artifact | Vision backbone; no SAE dictionary |
| Base model | facebookresearch/dinov2:dinov2_vits14 |
| Method | not_applicable |
| Source hook / output | x_norm_patchtokens |
| Layer | Not specified |
| Activation width | 384 |
| Dictionary features | Not specified |
| Checkpoint license | apache-2.0 |
| Checkpoint revision | Not specified |
| Tensor format | torch_hub |
Input / activation conventions:
{"image_size": 224, "resize": "square 224, bicubic, antialias", "mean": [0.485, 0.456, 0.406], "std": [0.229, 0.224, 0.225]}