+
\ No newline at end of file
diff --git a/shrink/DeepSeek-Coder-V2-Lite-Instruct-mic.html b/shrink/DeepSeek-Coder-V2-Lite-Instruct-mic.html
new file mode 100644
index 0000000000000000000000000000000000000000..b1d36db1515329f0f631d12d5af9a942508dadef
--- /dev/null
+++ b/shrink/DeepSeek-Coder-V2-Lite-Instruct-mic.html
@@ -0,0 +1,34 @@
+
+
+Shrunk architecture
+
+
Shrunk architecture
huggingface-mic/DeepSeek-Coder-V2-Lite-Instruct-mic@90601ffd1320fd0db167a3ade1c6cd0ed35c2229 ← deepseek-ai/DeepSeek-Coder-V2-Lite-Instruct@e434a23f91ba5b4923cf6c9d9a238eb4a08e3a11 · DeepseekV2ForCausalLM · random weights
+
Original15.7Bparameters
Fixture13.2Mparameters
Layer stacks1compared
Findings0to review
✓ Same architecture, smaller
Every module class, layer kind and checked property of the original is in the fixture.
Layer stacks
One square per layer, colored by kind: the layer's module classes and type attributes. Hover a square for its index and kind.
model.layers
Original · 27
Fixture · 2
Kind
Original
Fixture
no Experts
1
1
✓ kept
Experts, Moe, TopkRouter
26
1
✓ kept
Where the parameters are
Share of each model's parameters per component. Vocabulary and patch sizes are kept, so embeddings usually take a larger share of the fixture.
Original
Fixture
Component
Original
Share
Fixture
Share
model.layers
15.3B
97.3%
89.6K
0.7%
lm_head
209.7M
1.3%
6.6M
49.7%
model.embed_tokens
209.7M
1.3%
6.6M
49.7%
model.norm
2.0K
0.0%
64
0.0%
Preserved properties
Config
Property
Original
Fixture
model
attention style
MHA (16 heads, 16 KV)
MHA (4 heads, 4 KV)
✓ kept
model
MoE routing
top-6 of 64 experts
top-2 of 4 experts
✓ kept
model
RoPE type
yarn
yarn
✓ kept
model
model_type
deepseek_v2
deepseek_v2
✓ kept
model
vocab_size
102400
102400
✓ kept
model
tie_word_embeddings
False
False
✓ kept
model
hidden_act
silu
silu
✓ kept
model
max_position_embeddings
163840
163840
✓ kept
Module classes
Instances of each class in the built model (prefix DeepseekV2 omitted).
Class
Original
Fixture
Linear
190
15
RMSNorm
82
7
SiLUActivation
53
3
Attention
27
2
DecoderLayer
27
2
MLP
27
2
Experts
26
1
Moe
26
1
TopkRouter
26
1
ForCausalLM
1
1
Model
1
1
RotaryEmbedding
1
1
Embedding
1
1
ModuleList
1
1
Config changes
13 fields changed, 39 unchanged.
Field
Original
Fixture
head_dim
64
8
hidden_size
2048
64
intermediate_size
10944
128
kv_lora_rank
512
16
moe_intermediate_size
1408
32
n_routed_experts
64
4
num_attention_heads
16
4
num_experts_per_tok
6
2
num_hidden_layers
27
2
num_key_value_heads
16
4
qk_nope_head_dim
128
16
qk_rope_head_dim
64
8
v_head_dim
128
16
+
\ No newline at end of file
diff --git a/shrink/DeepSeek-R1-mic.html b/shrink/DeepSeek-R1-mic.html
new file mode 100644
index 0000000000000000000000000000000000000000..a28e1ba5e0c5dcf28a13a1e4cbb65b6efb121f92
--- /dev/null
+++ b/shrink/DeepSeek-R1-mic.html
@@ -0,0 +1,34 @@
+
+
+Shrunk architecture
+
+
Shrunk architecture
huggingface-mic/DeepSeek-R1-mic@89eddb3d8f9419e63b667eb7d5244fec8c768ed9 ← deepseek-ai/DeepSeek-R1@56d4cbbb4d29f4355bab4b9a39ccb717a14ad5ad · DeepseekV3ForCausalLM · random weights · built before the shrink rule (no recorded recipe)
+
Original671.0Bparameters
Fixture16.6Mparameters
Layer stacks1compared
Findings0to review
✓ Same architecture, smaller
Every module class, layer kind and checked property of the original is in the fixture.
Layer stacks
One square per layer, colored by kind: the layer's module classes and type attributes. Hover a square for its index and kind.
model.layers
Original · 61
Fixture · 2
Kind
Original
Fixture
no Experts
3
1
✓ kept
Experts, MoE, TopkRouter
58
1
✓ kept
Where the parameters are
Share of each model's parameters per component. Vocabulary and patch sizes are kept, so embeddings usually take a larger share of the fixture.
Original
Fixture
Component
Original
Share
Fixture
Share
model.layers
669.2B
99.7%
76.4K
0.5%
model.embed_tokens
926.7M
0.1%
8.3M
49.8%
lm_head
926.7M
0.1%
8.3M
49.8%
model.norm
7.2K
0.0%
64
0.0%
Preserved properties
Config
Property
Original
Fixture
model
attention style
MHA (128 heads, 128 KV)
MHA (4 heads, 4 KV)
✓ kept
model
MoE routing
top-8 of 256 experts
top-2 of 4 experts
✓ kept
model
RoPE type
yarn
yarn
✓ kept
model
model_type
deepseek_v3
deepseek_v3
✓ kept
model
vocab_size
129280
129280
✓ kept
model
tie_word_embeddings
False
False
✓ kept
model
hidden_act
silu
silu
✓ kept
model
max_position_embeddings
163840
163840
✓ kept
Module classes
Instances of each class in the built model (prefix DeepseekV3 omitted).
+
\ No newline at end of file
diff --git a/shrink/DeepSeek-V3.2-mic.html b/shrink/DeepSeek-V3.2-mic.html
new file mode 100644
index 0000000000000000000000000000000000000000..6440a2c04744163aabcaf8f9176593ed5082c171
--- /dev/null
+++ b/shrink/DeepSeek-V3.2-mic.html
@@ -0,0 +1,34 @@
+
+
+Shrunk architecture
+
+
Shrunk architecture
huggingface-mic/DeepSeek-V3.2-mic@2156444e97b761624604df50de0ac677d402f289 ← deepseek-ai/DeepSeek-V3.2@a7e62ac04ecb2c0a54d736dc46601c5606cf10a6 · DeepseekV32ForCausalLM · random weights · built before the shrink rule (no recorded recipe)
+
Original671.9Bparameters
Fixture16.9Mparameters
Layer stacks1compared
Findings0to review
✓ Same architecture, smaller
Every module class, layer kind and checked property of the original is in the fixture.
Layer stacks
One square per layer, colored by kind: the layer's module classes and type attributes. Hover a square for its index and kind.
model.layers
Original · 61
Fixture · 2
Kind
Original
Fixture
no Experts
3
1
✓ kept
Experts, MoE, TopkRouter
58
1
✓ kept
Where the parameters are
Share of each model's parameters per component. Vocabulary and patch sizes are kept, so embeddings usually take a larger share of the fixture.
Original
Fixture
Component
Original
Share
Fixture
Share
model.layers
670.0B
99.7%
363.6K
2.1%
model.embed_tokens
926.7M
0.1%
8.3M
48.9%
lm_head
926.7M
0.1%
8.3M
48.9%
model.norm
7.2K
0.0%
64
0.0%
Preserved properties
Config
Property
Original
Fixture
model
attention style
MHA (128 heads, 128 KV)
MHA (4 heads, 4 KV)
✓ kept
model
MoE routing
top-8 of 256 experts
top-2 of 4 experts
✓ kept
model
RoPE type
yarn
yarn
✓ kept
model
model_type
deepseek_v32
deepseek_v32
✓ kept
model
vocab_size
129280
129280
✓ kept
model
tie_word_embeddings
False
False
✓ kept
model
hidden_act
silu
silu
✓ kept
model
max_position_embeddings
163840
163840
✓ kept
Module classes
Instances of each class in the built model (prefix DeepseekV32 omitted).
+
\ No newline at end of file
diff --git a/shrink/ESMC-300M-hf-mic.html b/shrink/ESMC-300M-hf-mic.html
new file mode 100644
index 0000000000000000000000000000000000000000..721bfab75bcdfa8e823a16f6f3406e06f631b264
--- /dev/null
+++ b/shrink/ESMC-300M-hf-mic.html
@@ -0,0 +1,34 @@
+
+
+Shrunk architecture
+
+
Shrunk architecture
huggingface-mic/ESMC-300M-hf-mic@7c11c22ceffcbddb7fb1bb3216255d34ff38a43c ← biohub/ESMC-300M-hf@5554e4b3ab19eb66f04d7ff8e8c2d36c69b52fa0 · EsmcForMaskedLM · random weights
+
Original333.0Mparameters
Fixture95.3Kparameters
Layer stacks1compared
Findings0to review
✓ Same architecture, smaller
Every module class, layer kind and checked property of the original is in the fixture.
Layer stacks
One square per layer, colored by kind: the layer's module classes and type attributes. Hover a square for its index and kind.
esmc.layers
Original · 30
Fixture · 2
Kind
Original
Fixture
Layer
30
2
✓ kept
Where the parameters are
Share of each model's parameters per component. Vocabulary and patch sizes are kept, so embeddings usually take a larger share of the fixture.
Original
Fixture
Component
Original
Share
Fixture
Share
esmc.layers
331.9M
99.7%
82.7K
86.8%
lm_head.dense
922.6K
0.3%
4.2K
4.4%
lm_head.decoder
61.5K
0.0%
4.2K
4.4%
esmc.embed_tokens
61.4K
0.0%
4.1K
4.3%
lm_head.layer_norm
1.9K
0.0%
128
0.1%
esmc.norm
960
0.0%
64
0.1%
Preserved properties
Config
Property
Original
Fixture
model
attention style
MHA (15 heads, 15 KV)
MHA (4 heads, 4 KV)
✓ kept
model
RoPE type
default
default
✓ kept
model
model_type
esmc
esmc
✓ kept
model
vocab_size
64
64
✓ kept
model
tie_word_embeddings
False
False
✓ kept
model
hidden_act
silu
silu
✓ kept
model
max_position_embeddings
2048
2048
✓ kept
Module classes
Instances of each class in the built model (prefix Esmc omitted).
Class
Original
Fixture
Linear
212
16
LayerNorm
121
9
Attention
30
2
Layer
30
2
MLP
30
2
SiLUActivation
30
2
Embedding
1
1
ForMaskedLM
1
1
MaskedLMHead
1
1
Model
1
1
RotaryEmbedding
1
1
LayerNorm
1
1
ModuleList
1
1
Config changes
6 fields changed, 23 unchanged.
Field
Original
Fixture
head_dim
64
16
hidden_size
960
64
intermediate_size
2560
128
num_attention_heads
15
4
num_hidden_layers
30
2
num_key_value_heads
15
4
+
\ No newline at end of file
diff --git a/shrink/EuroBERT-210m-mic.html b/shrink/EuroBERT-210m-mic.html
new file mode 100644
index 0000000000000000000000000000000000000000..b433e9c2f65392126f9f3cdf3b63be778144f9f1
--- /dev/null
+++ b/shrink/EuroBERT-210m-mic.html
@@ -0,0 +1,34 @@
+
+
+Shrunk architecture
+
+
Shrunk architecture
huggingface-mic/EuroBERT-210m-mic@514cea1b679f118f051124ef88272fb809f25f16 ← EuroBERT/EuroBERT-210m@39b51e15dd1f1a06f58b5cbf6a8a188cec60bd0e · EuroBertForMaskedLM · random weights
+
Original310.3Mparameters
Fixture16.5Mparameters
Layer stacks1compared
Findings0to review
✓ Same architecture, smaller
Every module class, layer kind and checked property of the original is in the fixture.
Layer stacks
One square per layer, colored by kind: the layer's module classes and type attributes. Hover a square for its index and kind.
model.layers
Original · 12
Fixture · 2
Kind
Original
Fixture
DecoderLayer
12
2
✓ kept
Where the parameters are
Share of each model's parameters per component. Vocabulary and patch sizes are kept, so embeddings usually take a larger share of the fixture.
Original
Fixture
Component
Original
Share
Fixture
Share
model.layers
113.3M
36.5%
82.2K
0.5%
model.embed_tokens
98.5M
31.7%
8.2M
49.8%
lm_head
98.5M
31.7%
8.2M
49.8%
model.norm
768
0.0%
64
0.0%
Preserved properties
Config
Property
Original
Fixture
model
attention style
MHA (12 heads, 12 KV)
MHA (4 heads, 4 KV)
✓ kept
model
RoPE type
default
default
✓ kept
model
model_type
eurobert
eurobert
✓ kept
model
vocab_size
128256
128256
✓ kept
model
tie_word_embeddings
False
False
✓ kept
model
hidden_act
silu
silu
✓ kept
model
max_position_embeddings
8192
8192
✓ kept
Module classes
Instances of each class in the built model (prefix EuroBert omitted).
Class
Original
Fixture
Linear
85
15
RMSNorm
25
5
Attention
12
2
DecoderLayer
12
2
MLP
12
2
SiLUActivation
12
2
Embedding
1
1
ForMaskedLM
1
1
Model
1
1
RotaryEmbedding
1
1
ModuleList
1
1
Config changes
6 fields changed, 33 unchanged.
Field
Original
Fixture
head_dim
64
16
hidden_size
768
64
intermediate_size
3072
128
num_attention_heads
12
4
num_hidden_layers
12
2
num_key_value_heads
12
4
+
\ No newline at end of file
diff --git a/shrink/Falcon-H1-Tiny-90M-Instruct-mic.html b/shrink/Falcon-H1-Tiny-90M-Instruct-mic.html
new file mode 100644
index 0000000000000000000000000000000000000000..21ad67604569a33dc7996ea08a76fd55052ed4cb
--- /dev/null
+++ b/shrink/Falcon-H1-Tiny-90M-Instruct-mic.html
@@ -0,0 +1,34 @@
+
+
+Shrunk architecture
+
+
Shrunk architecture
huggingface-mic/Falcon-H1-Tiny-90M-Instruct-mic@0db75b996c3e47fea9cc1a9c213983988b24495a ← tiiuae/Falcon-H1-Tiny-90M-Instruct@e6389502a0b12cd8da894b395ba5bf7436873b16 · FalconH1ForCausalLM · random weights
+
Original91.1Mparameters
Fixture2.5Mparameters
Layer stacks1compared
Findings0to review
✓ Same architecture, smaller
Every module class, layer kind and checked property of the original is in the fixture.
Layer stacks
One square per layer, colored by kind: the layer's module classes and type attributes. Hover a square for its index and kind.
model.layers
Original · 24
Fixture · 2
Kind
Original
Fixture
DecoderLayer
24
2
✓ kept
Where the parameters are
Share of each model's parameters per component. Vocabulary and patch sizes are kept, so embeddings usually take a larger share of the fixture.
Original
Fixture
Component
Original
Share
Fixture
Share
model.layers
74.4M
81.6%
382.1K
15.4%
model.embed_tokens
16.8M
18.4%
2.1M
84.6%
model.final_layernorm
512
0.0%
64
0.0%
Preserved properties
Config
Property
Original
Fixture
model
attention style
GQA (8 heads, 2 KV)
GQA (4 heads, 2 KV)
✓ kept
model
RoPE type
default
default
✓ kept
model
model_type
falcon_h1
falcon_h1
✓ kept
model
vocab_size
32768
32768
✓ kept
model
tie_word_embeddings
True
True
✓ kept
model
hidden_act
silu
silu
✓ kept
model
max_position_embeddings
262144
262144
✓ kept
Module classes
Instances of each class in the built model (prefix FalconH1 omitted).
Class
Original
Fixture
Linear
217
19
RMSNorm
49
5
SiLUActivation
48
4
Conv1d
24
2
Attention
24
2
DecoderLayer
24
2
MLP
24
2
Mixer
24
2
Identity
24
2
Embedding
1
1
ForCausalLM
1
1
Model
1
1
RotaryEmbedding
1
1
ModuleList
1
1
Config changes
9 fields changed, 53 unchanged.
Field
Original
Fixture
head_dim
64
16
hidden_size
512
64
intermediate_size
768
128
mamba_chunk_size
128
8
mamba_d_head
32
96
mamba_d_state
64
16
mamba_n_heads
24
8
num_attention_heads
8
4
num_hidden_layers
24
2
+
\ No newline at end of file
diff --git a/shrink/Florence-2-base-mic.html b/shrink/Florence-2-base-mic.html
new file mode 100644
index 0000000000000000000000000000000000000000..641104dd1814b250c090b8dc9688793f10681607
--- /dev/null
+++ b/shrink/Florence-2-base-mic.html
@@ -0,0 +1,34 @@
+
+
+Shrunk architecture
+
+
Shrunk architecture
huggingface-mic/Florence-2-base-mic@85c1b12e1d9565763f6f2b9466669c58bc7aac33 ← florence-community/Florence-2-base@00921df66db728a9ceb750f5eca43e5c203a2051 · Florence2ForConditionalGeneration · overfit weights · built before the shrink rule (no recorded recipe)
+
Original231.4Mparameters
Fixture6.4Mparameters
Layer stacks4compared
Findings0to review
✓ Same architecture, smaller
Every module class, layer kind and checked property of the original is in the fixture.
Layer stacks
One square per layer, colored by kind: the layer's module classes and type attributes. Hover a square for its index and kind.
model.vision_tower.convs
Original · 4
Fixture · 4
Kind
Original
Fixture
Conv2d
4
4
✓ kept
model.vision_tower.blocks
Original · 4
Fixture · 4
Kind
Original
Fixture
VisionBlock
4
4
✓ kept
model.language_model.encoder.layers
Original · 6
Fixture · 1
Kind
Original
Fixture
BartEncoderLayer
6
1
✓ kept
model.language_model.decoder.layers
Original · 6
Fixture · 1
Kind
Original
Fixture
BartDecoderLayer
6
1
✓ kept
Where the parameters are
Share of each model's parameters per component. Vocabulary and patch sizes are kept, so embeddings usually take a larger share of the fixture.
Original
Fixture
Component
Original
Share
Fixture
Share
model.vision_tower.blocks
84.2M
36.4%
2.5M
39.1%
model.language_model.decoder.layers
56.7M
24.5%
50.2K
0.8%
model.language_model.encoder.layers
42.5M
18.4%
33.5K
0.5%
model.language_model
41.0M
17.7%
3.4M
53.0%
model.vision_tower.convs
6.2M
2.7%
392.8K
6.1%
model.multi_modal_projector
839.2K
0.4%
29.3K
0.5%
Preserved properties
Config
Property
Original
Fixture
model
model_type
florence2
florence2
✓ kept
model
tie_word_embeddings
True
True
✓ kept
text_config
model_type
bart
bart
✓ kept
text_config
vocab_size
51328
51328
✓ kept
text_config
tie_word_embeddings
True
True
✓ kept
text_config
max_position_embeddings
1024
1024
✓ kept
vision_config
model_type
florence_vision
florence_vision
✓ kept
vision_config
max_position_embeddings
50
50
✓ kept
vision_config
patch_size
[7, 3, 3, 3]
[7, 3, 3, 3]
✓ kept
Module classes
Instances of each class in the built model (prefix Florence2 omitted).
Class
Original
Fixture
Linear
194
58
LayerNorm
85
32
Conv2d
52
24
DropPath
46
18
GELUActivation
36
12
VisionMLP
24
10
BartAttention
18
3
VisionBlock
12
5
VisionChannelAttention
12
5
VisionChannelBlock
12
5
VisionSpatialBlock
12
5
VisionWindowAttention
12
5
ModuleList
8
8
BartDecoderLayer
6
1
BartEncoderLayer
6
1
VisionConvEmbed
4
4
BartScaledWordEmbedding
3
3
BartLearnedPositionalEmbedding
2
2
Embedding
2
2
Identity
2
2
BartDecoder
1
1
BartEncoder
1
1
BartModel
1
1
ForConditionalGeneration
1
1
Model
1
1
MultiModalProjector
1
1
VisionBackbone
1
1
VisionLearnedAbsolutePositionEmbedding2D
1
1
VisionPositionalEmbeddingCosine1D
1
1
Config changes
12 fields changed, 72 unchanged.
Field
Original
Fixture
text_config.d_model
768
64
text_config.decoder_attention_heads
12
2
text_config.decoder_ffn_dim
3072
128
text_config.decoder_layers
6
1
text_config.encoder_attention_heads
12
2
text_config.encoder_ffn_dim
3072
128
text_config.encoder_layers
6
1
vision_config.depths
[1, 1, 9, 1]
[1, 1, 2, 1]
vision_config.embed_dim
[128, 256, 512, 1024]
[32, 64, 128, 256]
vision_config.num_groups
[4, 8, 16, 32]
[2, 4, 8, 16]
vision_config.num_heads
[4, 8, 16, 32]
[2, 4, 8, 16]
vision_config.projection_dim
768
64
+
\ No newline at end of file
diff --git a/shrink/GLM-4-9B-0414-mic.html b/shrink/GLM-4-9B-0414-mic.html
new file mode 100644
index 0000000000000000000000000000000000000000..1b1fc2dfe2de73cb2964dffacb00a96e16a4b991
--- /dev/null
+++ b/shrink/GLM-4-9B-0414-mic.html
@@ -0,0 +1,34 @@
+
+
+Shrunk architecture
+
+
Shrunk architecture
huggingface-mic/GLM-4-9B-0414-mic@140a76e1cd7001bc3d0aed3b715a2a902214ece6 ← zai-org/GLM-4-9B-0414@645b8482494e31b6b752272bf7f7f273ef0f3caf · Glm4ForCausalLM · random weights
+
Original9.4Bparameters
Fixture19.5Mparameters
Layer stacks1compared
Findings0to review
✓ Same architecture, smaller
Every module class, layer kind and checked property of the original is in the fixture.
Layer stacks
One square per layer, colored by kind: the layer's module classes and type attributes. Hover a square for its index and kind.
model.layers
Original · 40
Fixture · 2
Kind
Original
Fixture
DecoderLayer
40
2
✓ kept
Where the parameters are
Share of each model's parameters per component. Vocabulary and patch sizes are kept, so embeddings usually take a larger share of the fixture.
Original
Fixture
Component
Original
Share
Fixture
Share
model.layers
8.2B
86.8%
74.5K
0.4%
model.embed_tokens
620.8M
6.6%
9.7M
49.8%
lm_head
620.8M
6.6%
9.7M
49.8%
model.norm
4.1K
0.0%
64
0.0%
Preserved properties
Config
Property
Original
Fixture
model
attention style
GQA (32 heads, 2 KV)
GQA (4 heads, 2 KV)
✓ kept
model
RoPE type
default
default
✓ kept
model
model_type
glm4
glm4
✓ kept
model
vocab_size
151552
151552
✓ kept
model
tie_word_embeddings
False
False
✓ kept
model
hidden_act
silu
silu
✓ kept
model
max_position_embeddings
32768
32768
✓ kept
Module classes
Instances of each class in the built model (prefix Glm4 omitted).
Class
Original
Fixture
Linear
241
13
RMSNorm
161
9
Attention
40
2
DecoderLayer
40
2
MLP
40
2
SiLUActivation
40
2
Embedding
1
1
ForCausalLM
1
1
Model
1
1
RotaryEmbedding
1
1
ModuleList
1
1
Config changes
5 fields changed, 23 unchanged.
Field
Original
Fixture
head_dim
128
16
hidden_size
4096
64
intermediate_size
13696
128
num_attention_heads
32
4
num_hidden_layers
40
2
+
\ No newline at end of file
diff --git a/shrink/GLM-4.5-Air-mic.html b/shrink/GLM-4.5-Air-mic.html
new file mode 100644
index 0000000000000000000000000000000000000000..2a87324fcfbf8e6088e9e59bbbd08ff249123167
--- /dev/null
+++ b/shrink/GLM-4.5-Air-mic.html
@@ -0,0 +1,34 @@
+
+
+Shrunk architecture
+
+
Shrunk architecture
huggingface-mic/GLM-4.5-Air-mic@abc7ea97d6d633936cfd54900edce3b610755f86 ← zai-org/GLM-4.5-Air@a24ceef6ce4f3536971efe9b778bdaa1bab18daa · Glm4MoeForCausalLM · random weights
+
Original106.9Bparameters
Fixture19.5Mparameters
Layer stacks1compared
Findings0to review
✓ Same architecture, smaller
Every module class, layer kind and checked property of the original is in the fixture.
Layer stacks
One square per layer, colored by kind: the layer's module classes and type attributes. Hover a square for its index and kind.
model.layers
Original · 46
Fixture · 2
Kind
Original
Fixture
no Experts
1
1
✓ kept
Experts, MoE, TopkRouter
45
1
✓ kept
Where the parameters are
Share of each model's parameters per component. Vocabulary and patch sizes are kept, so embeddings usually take a larger share of the fixture.
Original
Fixture
Component
Original
Share
Fixture
Share
model.layers
105.6B
98.8%
80.6K
0.4%
model.embed_tokens
620.8M
0.6%
9.7M
49.8%
lm_head
620.8M
0.6%
9.7M
49.8%
model.norm
4.1K
0.0%
64
0.0%
Preserved properties
Config
Property
Original
Fixture
model
attention style
GQA (96 heads, 8 KV)
GQA (4 heads, 2 KV)
✓ kept
model
MoE routing
top-8 of 128 experts
top-2 of 4 experts
✓ kept
model
RoPE type
default
default
✓ kept
model
model_type
glm4_moe
glm4_moe
✓ kept
model
vocab_size
151552
151552
✓ kept
model
tie_word_embeddings
False
False
✓ kept
model
hidden_act
silu
silu
✓ kept
model
max_position_embeddings
131072
131072
✓ kept
Module classes
Instances of each class in the built model (prefix Glm4Moe omitted).
Class
Original
Fixture
Linear
323
15
RMSNorm
93
5
SiLUActivation
91
3
Attention
46
2
DecoderLayer
46
2
MLP
46
2
Experts
45
1
MoE
45
1
TopkRouter
45
1
Embedding
1
1
ForCausalLM
1
1
Model
1
1
RotaryEmbedding
1
1
ModuleList
1
1
Config changes
9 fields changed, 31 unchanged.
Field
Original
Fixture
head_dim
128
16
hidden_size
4096
64
intermediate_size
10944
128
moe_intermediate_size
1408
32
n_routed_experts
128
4
num_attention_heads
96
4
num_experts_per_tok
8
2
num_hidden_layers
46
2
num_key_value_heads
8
2
+
\ No newline at end of file
diff --git a/shrink/GLM-4.7-Flash-mic.html b/shrink/GLM-4.7-Flash-mic.html
new file mode 100644
index 0000000000000000000000000000000000000000..1e5d95f5bb7d82316a71c509a78503c1f3f5c1fc
--- /dev/null
+++ b/shrink/GLM-4.7-Flash-mic.html
@@ -0,0 +1,34 @@
+
+
+Shrunk architecture
+
+
Shrunk architecture
huggingface-mic/GLM-4.7-Flash-mic@8b4c9af3fc9e6a8325e3dade3558514edd2cd694 ← zai-org/GLM-4.7-Flash@7dd20894a642a0aa287e9827cb1a1f7f91386b67 · Glm4MoeLiteForCausalLM · random weights
+
Original29.9Bparameters
Fixture19.9Mparameters
Layer stacks1compared
Findings0to review
✓ Same architecture, smaller
Every module class, layer kind and checked property of the original is in the fixture.
Layer stacks
One square per layer, colored by kind: the layer's module classes and type attributes. Hover a square for its index and kind.
model.layers
Original · 47
Fixture · 2
Kind
Original
Fixture
no Experts
1
1
✓ kept
Experts, MoE, TopkRouter
46
1
✓ kept
Where the parameters are
Share of each model's parameters per component. Vocabulary and patch sizes are kept, so embeddings usually take a larger share of the fixture.
Original
Fixture
Component
Original
Share
Fixture
Share
model.layers
29.3B
97.9%
76.4K
0.4%
model.embed_tokens
317.2M
1.1%
9.9M
49.8%
lm_head
317.2M
1.1%
9.9M
49.8%
model.norm
2.0K
0.0%
64
0.0%
Preserved properties
Config
Property
Original
Fixture
model
attention style
MHA (20 heads, 20 KV)
MHA (4 heads, 4 KV)
✓ kept
model
MoE routing
top-4 of 64 experts
top-2 of 4 experts
✓ kept
model
RoPE type
default
default
✓ kept
model
model_type
glm4_moe_lite
glm4_moe_lite
✓ kept
model
vocab_size
154880
154880
✓ kept
model
tie_word_embeddings
False
False
✓ kept
model
hidden_act
silu
silu
✓ kept
model
max_position_embeddings
202752
202752
✓ kept
Module classes
Instances of each class in the built model (prefix Glm4MoeLite omitted).
Class
Original
Fixture
Linear
377
17
RMSNorm
189
9
SiLUActivation
93
3
Attention
47
2
DecoderLayer
47
2
MLP
47
2
Experts
46
1
MoE
46
1
TopkRouter
46
1
Embedding
1
1
ForCausalLM
1
1
Model
1
1
RotaryEmbedding
1
1
ModuleList
1
1
Config changes
14 fields changed, 34 unchanged.
Field
Original
Fixture
hidden_size
2048
64
intermediate_size
10240
128
kv_lora_rank
512
16
mlp_layer_types
47 entries: 46× sparse, 1× dense
["dense", "sparse"]
moe_intermediate_size
1536
32
n_routed_experts
64
4
num_attention_heads
20
4
num_experts_per_tok
4
2
num_hidden_layers
47
2
num_key_value_heads
20
4
q_lora_rank
768
16
qk_nope_head_dim
192
16
qk_rope_head_dim
64
8
v_head_dim
256
16
+
\ No newline at end of file
diff --git a/shrink/GLM-5.3-Flash-mic.html b/shrink/GLM-5.3-Flash-mic.html
new file mode 100644
index 0000000000000000000000000000000000000000..195d98ddc0e2053fc602b6a54d8ca81e1787b4d5
--- /dev/null
+++ b/shrink/GLM-5.3-Flash-mic.html
@@ -0,0 +1,34 @@
+
+
+Shrunk architecture
+
+
Shrunk architecture
huggingface-mic/GLM-5.3-Flash-mic@53f3e5bd10a67d2c75a170a3c2a28009e338c256 ← zai-org/GLM-5.3-Flash@46a050bec13002ad29ee795c16e61cf4fa3c4a9c · Glm5NextForConditionalGeneration · random weights · built before the shrink rule (no recorded recipe)
+
Original313.9Bparameters
Fixture40.7Mparameters
Layer stacks2compared
Findings1to review
⚠ To review
vision_config: image_size 448 became 56
Layer stacks
One square per layer, colored by kind: the layer's module classes and type attributes. Hover a square for its index and kind.
model.visual.blocks
Original · 24
Fixture · 2
Kind
Original
Fixture
VisionBlock
24
2
✓ kept
model.language_model.layers
Original · 45
Fixture · 3
Kind
Original
Fixture
linear_attention, Conv1d, TextForgetGate …
3
1
✓ kept
is_causal=True, TextAttention, TextExperts …
11
1
✓ kept
linear_attention, Conv1d, TextExperts …
31
1
✓ kept
Where the parameters are
Share of each model's parameters per component. Vocabulary and patch sizes are kept, so embeddings usually take a larger share of the fixture.
Original
Fixture
Component
Original
Share
Fixture
Share
model.language_model.layers
312.1B
99.4%
811.0K
2.0%
model.language_model
634.4M
0.2%
19.8M
48.7%
lm_head
634.4M
0.2%
19.8M
48.7%
model.visual.blocks
403.0M
0.1%
83.4K
0.2%
model.visual
160.6M
0.1%
174.1K
0.4%
Preserved properties
Config
Property
Original
Fixture
model
model_type
glm5_next
glm5_next
✓ kept
model
tie_word_embeddings
False
False
✓ kept
text_config
attention style
MHA (64 heads, 64 KV)
MHA (4 heads, 4 KV)
✓ kept
text_config
MoE routing
top-8 of 288 experts
top-2 of 8 experts
✓ kept
text_config
model_type
glm5_next_text
glm5_next_text
✓ kept
text_config
vocab_size
154880
154880
✓ kept
text_config
tie_word_embeddings
False
False
✓ kept
text_config
hidden_act
silu
silu
✓ kept
text_config
max_position_embeddings
1048576
1048576
✓ kept
vision_config
RoPE type
axial
axial
✓ kept
vision_config
model_type
glm5_next_vision
glm5_next_vision
✓ kept
vision_config
hidden_act
silu
silu
✓ kept
vision_config
image_size
448
56
⚠ changed
vision_config
patch_size
14
14
✓ kept
Module classes
Instances of each class in the built model (prefix Glm5Next omitted).
Share of each model's parameters per component. Vocabulary and patch sizes are kept, so embeddings usually take a larger share of the fixture.
Original
Fixture
Component
Original
Share
Fixture
Share
model.layers
251.6M
78.9%
443.3K
5.0%
lm_head
33.6M
10.5%
4.2M
47.5%
model.embed_tokens
33.6M
10.5%
4.2M
47.5%
model.final_layernorm
512
0.0%
64
0.0%
Preserved properties
Config
Property
Original
Fixture
model
attention style
GQA (8 heads, 2 KV)
GQA (4 heads, 2 KV)
✓ kept
model
MoE routing
top-2 of 8 experts
top-2 of 4 experts
✓ kept
model
model_type
jamba
jamba
✓ kept
model
vocab_size
65536
65536
✓ kept
model
tie_word_embeddings
False
False
✓ kept
model
hidden_act
silu
silu
✓ kept
model
max_position_embeddings
262144
262144
✓ kept
Module classes
Instances of each class in the built model (prefix Jamba omitted).
Class
Original
Fixture
Linear
89
30
RMSNorm
75
23
SiLUActivation
30
9
Conv1d
14
4
MambaDecoderLayer
14
4
MambaMixer
14
4
Experts
8
2
MLP
8
3
SparseMoeBlock
8
2
TopKRouter
8
2
Attention
2
1
AttentionDecoderLayer
2
1
Embedding
1
1
ForCausalLM
1
1
Model
1
1
ModuleList
1
1
Config changes
5 fields changed, 39 unchanged.
Field
Original
Fixture
hidden_size
512
64
intermediate_size
2048
128
num_attention_heads
8
4
num_experts
8
4
num_hidden_layers
16
5
+
\ No newline at end of file
diff --git a/shrink/Janus-Pro-1B-mic.html b/shrink/Janus-Pro-1B-mic.html
new file mode 100644
index 0000000000000000000000000000000000000000..fdb32f404e33a7823f04dd1acc41cce989693d96
--- /dev/null
+++ b/shrink/Janus-Pro-1B-mic.html
@@ -0,0 +1,34 @@
+
+
+Shrunk architecture
+
+
Shrunk architecture
huggingface-mic/Janus-Pro-1B-mic@5d0bfc05499ea83be331a434ea79583ca970581a ← deepseek-community/Janus-Pro-1B@1655280bb75959cc1cb85529a2a8b26e7016072e · JanusForConditionalGeneration · random weights · built before the shrink rule (no recorded recipe)
+
Original1.9Bparameters
Fixture81.9Mparameters
Layer stacks4compared
Findings0to review
✓ Same architecture, smaller
Every module class, layer kind and checked property of the original is in the fixture.
Layer stacks
One square per layer, colored by kind: the layer's module classes and type attributes. Hover a square for its index and kind.
model.vision_model.encoder.layers
Original · 24
Fixture · 2
Kind
Original
Fixture
VisionEncoderLayer
24
2
✓ kept
model.vqmodel.encoder.down
Original · 5
Fixture · 5
Kind
Original
Fixture
VQVAEConvDownsample
4
4
✓ kept
VQVAEAttnBlock
1
1
✓ kept
model.vqmodel.decoder.up
Original · 5
Fixture · 5
Kind
Original
Fixture
VQVAEAttnBlock, VQVAEConvUpsample
1
1
✓ kept
VQVAEConvUpsample
3
3
✓ kept
no VQVAEAttnBlock
1
1
✓ kept
model.language_model.layers
Original · 24
Fixture · 2
Kind
Original
Fixture
LlamaDecoderLayer
24
2
✓ kept
Where the parameters are
Share of each model's parameters per component. Vocabulary and patch sizes are kept, so embeddings usually take a larger share of the fixture.
Original
Fixture
Component
Original
Share
Fixture
Share
model.language_model.layers
1.2B
66.1%
82.2K
0.1%
model.vision_model.encoder.layers
302.3M
16.2%
100.0K
0.1%
model.language_model
209.7M
11.2%
6.6M
8.0%
model.generation_head
37.8M
2.0%
1.2M
1.5%
model.vqmodel.decoder.up
30.8M
1.6%
30.8M
37.6%
model.vqmodel
23.5M
1.3%
24.4M
29.8%
model.vqmodel.encoder.down
17.6M
0.9%
17.6M
21.5%
4 others
12.0M
0.6%
1.2M
1.4%
Preserved properties
Config
Property
Original
Fixture
model
model_type
janus
janus
✓ kept
model
tie_word_embeddings
True
True
✓ kept
text_config
attention style
MHA (16 heads, 16 KV)
MHA (4 heads, 4 KV)
✓ kept
text_config
RoPE type
default
default
✓ kept
text_config
model_type
llama
llama
✓ kept
text_config
vocab_size
102400
102400
✓ kept
text_config
tie_word_embeddings
False
False
✓ kept
text_config
hidden_act
silu
silu
✓ kept
text_config
max_position_embeddings
16384
16384
✓ kept
vision_config
model_type
janus_vision_model
janus_vision_model
✓ kept
vision_config
hidden_act
gelu
gelu
✓ kept
vision_config
image_size
384
384
✓ kept
vision_config
patch_size
16
16
✓ kept
vision_config
num_channels
3
3
✓ kept
vq_config
model_type
janus_vqgan
janus_vqgan
✓ kept
vq_config
hidden_act
gelu
gelu
✓ kept
Module classes
Instances of each class in the built model (prefix Janus omitted).
Class
Original
Fixture
Linear
319
33
Conv2d
105
105
Dropout
77
33
Identity
72
6
GroupNorm
67
67
LayerNorm
49
5
LlamaRMSNorm
49
5
VQVAEResnetBlock
29
29
GELUActivation
27
5
ModuleList
26
26
VisionAttention
24
2
VisionEncoderLayer
24
2
VisionMLP
24
2
LlamaAttention
24
2
LlamaDecoderLayer
24
2
LlamaMLP
24
2
SiLUActivation
24
2
Module
10
10
VQVAEAttnBlock
7
7
Embedding
4
4
VQVAEConvDownsample
4
4
VQVAEConvUpsample
4
4
VQVAEMidBlock
2
2
ForConditionalGeneration
1
1
Model
1
1
VQVAE
1
1
VQVAEAlignerMLP
1
1
VQVAEDecoder
1
1
VQVAEEncoder
1
1
VQVAEHead
1
1
VQVAEVectorQuantizer
1
1
VisionAlignerMLP
1
1
VisionEmbeddings
1
1
VisionEncoder
1
1
VisionModel
1
1
LlamaModel
1
1
LlamaRotaryEmbedding
1
1
Config changes
12 fields changed, 82 unchanged.
Field
Original
Fixture
text_config.head_dim
128
16
text_config.hidden_size
2048
64
text_config.intermediate_size
5632
128
text_config.num_attention_heads
16
4
text_config.num_hidden_layers
24
2
text_config.num_key_value_heads
16
4
vision_config.hidden_size
1024
64
vision_config.num_attention_heads
16
4
vision_config.num_hidden_layers
24
2
vision_config.projection_dim
2048
64
vq_config.embed_dim
8
64
vq_config.projection_dim
2048
64
+
\ No newline at end of file
diff --git a/shrink/LCO-Embedding-Omni-3B-mic.html b/shrink/LCO-Embedding-Omni-3B-mic.html
new file mode 100644
index 0000000000000000000000000000000000000000..2fbe0f857e38f829102a4bda829c8fea63d372a4
--- /dev/null
+++ b/shrink/LCO-Embedding-Omni-3B-mic.html
@@ -0,0 +1,34 @@
+
+
+Shrunk architecture
+
+
Shrunk architecture
huggingface-mic/LCO-Embedding-Omni-3B-mic@1e770f7015e573f2c4287893af7c6eb0c1d45c33 ← LCO-Embedding/LCO-Embedding-Omni-3B@73434565c33d032551639362f7fc91fa3c6f065f · Qwen2_5OmniThinkerForConditionalGeneration · random weights
+
Original4.7Bparameters
Fixture21.1Mparameters
Layer stacks3compared
Findings0to review
✓ Same architecture, smaller
Every module class, layer kind and checked property of the original is in the fixture.
Layer stacks
One square per layer, colored by kind: the layer's module classes and type attributes. Hover a square for its index and kind.
model.audio_tower.layers
Original · 32
Fixture · 2
Kind
Original
Fixture
Qwen2_5OmniAudioEncoderLayer
32
2
✓ kept
model.visual.blocks
Original · 32
Fixture · 32
Kind
Original
Fixture
Qwen2_5OmniVisionBlock
32
32
✓ kept
model.language_model.layers
Original · 36
Fixture · 2
Kind
Original
Fixture
Qwen2_5OmniDecoderLayer
36
2
✓ kept
Where the parameters are
Share of each model's parameters per component. Vocabulary and patch sizes are kept, so embeddings usually take a larger share of the fixture.
Original
Fixture
Component
Original
Share
Fixture
Share
model.language_model.layers
2.8B
59.0%
74.2K
0.4%
model.visual.blocks
630.5M
13.4%
1.3M
6.3%
model.audio_tower.layers
629.6M
13.4%
66.8K
0.3%
model.language_model
311.2M
6.6%
9.7M
46.0%
lm_head
311.2M
6.6%
9.7M
46.0%
model.visual
38.2M
0.8%
157.6K
0.7%
model.audio_tower
8.0M
0.2%
41.4K
0.2%
Preserved properties
Config
Property
Original
Fixture
model
model_type
qwen2_5_omni_thinker
qwen2_5_omni_thinker
✓ kept
model
tie_word_embeddings
False
False
✓ kept
audio_config
model_type
qwen2_5_omni_audio_encoder
qwen2_5_omni_audio_encoder
✓ kept
audio_config
num_mel_bins
128
128
✓ kept
vision_config
RoPE type
axial
axial
✓ kept
vision_config
model_type
qwen2_5_omni_vision_encoder
qwen2_5_omni_vision_encoder
✓ kept
vision_config
hidden_act
silu
silu
✓ kept
vision_config
patch_size
14
14
✓ kept
text_config
attention style
GQA (16 heads, 2 KV)
GQA (4 heads, 2 KV)
✓ kept
text_config
RoPE type
default
default
✓ kept
text_config
model_type
qwen2_5_omni_text
qwen2_5_omni_text
✓ kept
text_config
vocab_size
151936
151936
✓ kept
text_config
tie_word_embeddings
True
True
✓ kept
text_config
hidden_act
silu
silu
✓ kept
text_config
max_position_embeddings
32768
32768
✓ kept
Module classes
Instances of each class in the built model (prefix Qwen2_5OmniThinker omitted).
+
\ No newline at end of file
diff --git a/shrink/LFM2-8B-A1B-mic.html b/shrink/LFM2-8B-A1B-mic.html
new file mode 100644
index 0000000000000000000000000000000000000000..2374bbb352d035c84326b8a69b3cc008eea8f545
--- /dev/null
+++ b/shrink/LFM2-8B-A1B-mic.html
@@ -0,0 +1,34 @@
+
+
+Shrunk architecture
+
+
Shrunk architecture
huggingface-mic/LFM2-8B-A1B-mic@02493cd540778260b98790f375fb2070e8acd824 ← LiquidAI/LFM2-8B-A1B@c1c44ff9fc00db3ebf4516970563f5f383d23670 · Lfm2MoeForCausalLM · random weights
+
Original8.3Bparameters
Fixture4.4Mparameters
Layer stacks1compared
Findings0to review
✓ Same architecture, smaller
Every module class, layer kind and checked property of the original is in the fixture.
Layer stacks
One square per layer, colored by kind: the layer's module classes and type attributes. Hover a square for its index and kind.
model.layers
Original · 24
Fixture · 4
Kind
Original
Fixture
conv, Conv1d, MLP …
2
2
✓ kept
is_causal=True, Attention, Experts …
6
1
✓ kept
conv, Conv1d, Experts …
16
1
✓ kept
Where the parameters are
Share of each model's parameters per component. Vocabulary and patch sizes are kept, so embeddings usually take a larger share of the fixture.
Original
Fixture
Component
Original
Share
Fixture
Share
model.layers
8.2B
98.4%
161.4K
3.7%
model.embed_tokens
134.2M
1.6%
4.2M
96.3%
model.embedding_norm
2.0K
0.0%
64
0.0%
Preserved properties
Config
Property
Original
Fixture
model
attention style
GQA (32 heads, 8 KV)
GQA (4 heads, 2 KV)
✓ kept
model
MoE routing
top-4 of 32 experts
top-2 of 4 experts
✓ kept
model
RoPE type
default
default
✓ kept
model
model_type
lfm2_moe
lfm2_moe
✓ kept
model
vocab_size
65536
65536
✓ kept
model
tie_word_embeddings
True
True
✓ kept
model
max_position_embeddings
128000
128000
✓ kept
Module classes
Instances of each class in the built model (prefix Lfm2Moe omitted).
Class
Original
Fixture
Linear
67
17
RMSNorm
61
11
DecoderLayer
24
4
Experts
22
2
SparseMoeBlock
22
2
TopKRouter
22
2
Conv1d
18
3
ShortConv
18
3
Attention
6
1
MLP
2
2
Embedding
1
1
ForCausalLM
1
1
Model
1
1
RotaryEmbedding
1
1
ModuleList
1
1
Config changes
9 fields changed, 26 unchanged.
Field
Original
Fixture
hidden_size
2048
64
intermediate_size
7168
128
layer_types
24 entries: 18× conv, 6× full_attention
["conv", "conv", "full_attention", "conv"]
moe_intermediate_size
1792
32
num_attention_heads
32
4
num_experts
32
4
num_experts_per_tok
4
2
num_hidden_layers
24
4
num_key_value_heads
8
2
+
\ No newline at end of file
diff --git a/shrink/LFM2.5-230M-Base-mic.html b/shrink/LFM2.5-230M-Base-mic.html
new file mode 100644
index 0000000000000000000000000000000000000000..ff18db627fa25b7f9da28d9edaaf1c96f40c5bb4
--- /dev/null
+++ b/shrink/LFM2.5-230M-Base-mic.html
@@ -0,0 +1,34 @@
+
+
+Shrunk architecture
+
+
Shrunk architecture
huggingface-mic/LFM2.5-230M-Base-mic@d6697c673a52ed87ae4e8ed607e5b6a1ea6f7a8d ← LiquidAI/LFM2.5-230M-Base@9d2be5519834990d30996f878b6771cccbd24f2c · Lfm2ForCausalLM · random weights
+
Original229.7Mparameters
Fixture4.3Mparameters
Layer stacks1compared
Findings0to review
✓ Same architecture, smaller
Every module class, layer kind and checked property of the original is in the fixture.
Layer stacks
One square per layer, colored by kind: the layer's module classes and type attributes. Hover a square for its index and kind.
model.layers
Original · 14
Fixture · 2
Kind
Original
Fixture
conv, Conv1d, ShortConv
8
1
✓ kept
is_causal=True, Attention
6
1
✓ kept
Where the parameters are
Share of each model's parameters per component. Vocabulary and patch sizes are kept, so embeddings usually take a larger share of the fixture.
Original
Fixture
Component
Original
Share
Fixture
Share
model.layers
162.6M
70.8%
78.3K
1.8%
model.embed_tokens
67.1M
29.2%
4.2M
98.2%
model.embedding_norm
1.0K
0.0%
64
0.0%
Preserved properties
Config
Property
Original
Fixture
model
attention style
GQA (16 heads, 8 KV)
GQA (4 heads, 2 KV)
✓ kept
model
RoPE type
default
default
✓ kept
model
model_type
lfm2
lfm2
✓ kept
model
vocab_size
65536
65536
✓ kept
model
tie_word_embeddings
True
True
✓ kept
model
max_position_embeddings
128000
128000
✓ kept
Module classes
Instances of each class in the built model (prefix Lfm2 omitted).
Class
Original
Fixture
Linear
83
13
RMSNorm
41
7
DecoderLayer
14
2
MLP
14
2
Conv1d
8
1
ShortConv
8
1
Attention
6
1
Embedding
1
1
ForCausalLM
1
1
Model
1
1
RotaryEmbedding
1
1
ModuleList
1
1
Config changes
7 fields changed, 41 unchanged.
Field
Original
Fixture
hidden_size
1024
64
intermediate_size
2560
128
layer_types
14 entries: 8× conv, 6× full_attention
["conv", "full_attention"]
num_attention_heads
16
4
num_heads
16
4
num_hidden_layers
14
2
num_key_value_heads
8
2
+
\ No newline at end of file
diff --git a/shrink/LFM2.5-VL-450M-mic.html b/shrink/LFM2.5-VL-450M-mic.html
new file mode 100644
index 0000000000000000000000000000000000000000..2391e81249a633171a7f2fb31f3fed94cece319a
--- /dev/null
+++ b/shrink/LFM2.5-VL-450M-mic.html
@@ -0,0 +1,34 @@
+
+
+Shrunk architecture
+
+
Shrunk architecture
huggingface-mic/LFM2.5-VL-450M-mic@27b8f6b75e00ddd9758961d5abac08622f2b6d73 ← LiquidAI/LFM2.5-VL-450M@fc6221ca597f3315e4f82fc2df606783267b34ba · Lfm2VlForConditionalGeneration · random weights
+
Original448.7Mparameters
Fixture5.1Mparameters
Layer stacks2compared
Findings0to review
✓ Same architecture, smaller
Every module class, layer kind and checked property of the original is in the fixture.
Layer stacks
One square per layer, colored by kind: the layer's module classes and type attributes. Hover a square for its index and kind.
model.vision_tower.encoder.layers
Original · 12
Fixture · 2
Kind
Original
Fixture
Siglip2EncoderLayer
12
2
✓ kept
model.language_model.layers
Original · 16
Fixture · 2
Kind
Original
Fixture
conv, Conv1d, Lfm2ShortConv
10
1
✓ kept
is_causal=True, Lfm2Attention
6
1
✓ kept
Where the parameters are
Share of each model's parameters per component. Vocabulary and patch sizes are kept, so embeddings usually take a larger share of the fixture.
Original
Fixture
Component
Original
Share
Fixture
Share
model.language_model.layers
287.4M
64.0%
127.5K
2.5%
model.vision_tower.encoder.layers
85.1M
19.0%
66.9K
1.3%
model.language_model
67.1M
15.0%
4.2M
82.0%
model.multi_modal_projector
8.4M
1.9%
657.5K
12.9%
model.vision_tower
788.7K
0.2%
65.7K
1.3%
Preserved properties
Config
Property
Original
Fixture
model
model_type
lfm2_vl
lfm2_vl
✓ kept
model
tie_word_embeddings
True
True
✓ kept
vision_config
model_type
siglip2_vision_model
siglip2_vision_model
✓ kept
vision_config
hidden_act
gelu_pytorch_tanh
gelu_pytorch_tanh
✓ kept
vision_config
patch_size
16
16
✓ kept
vision_config
num_channels
3
3
✓ kept
text_config
attention style
GQA (16 heads, 8 KV)
GQA (4 heads, 2 KV)
✓ kept
text_config
RoPE type
default
default
✓ kept
text_config
model_type
lfm2
lfm2
✓ kept
text_config
vocab_size
65536
65536
✓ kept
text_config
tie_word_embeddings
True
True
✓ kept
text_config
max_position_embeddings
128000
128000
✓ kept
Module classes
Instances of each class in the built model (prefix Lfm2Vl omitted).
Class
Original
Fixture
Linear
168
28
Lfm2RMSNorm
45
7
LayerNorm
25
5
Lfm2DecoderLayer
16
2
Lfm2MLP
16
2
GELUTanh
12
2
Siglip2Attention
12
2
Siglip2EncoderLayer
12
2
Siglip2MLP
12
2
Conv1d
10
1
Lfm2ShortConv
10
1
Lfm2Attention
6
1
Embedding
2
2
ModuleList
2
2
GELUActivation
1
1
Lfm2Model
1
1
Lfm2RotaryEmbedding
1
1
ForConditionalGeneration
1
1
Model
1
1
MultiModalProjector
1
1
Siglip2Encoder
1
1
Siglip2VisionEmbeddings
1
1
Siglip2VisionModel
1
1
Config changes
11 fields changed, 77 unchanged.
Field
Original
Fixture
text_config.hidden_size
1024
64
text_config.intermediate_size
6656
128
text_config.layer_types
16 entries: 10× conv, 6× full_attention
["conv", "full_attention"]
text_config.num_attention_heads
16
4
text_config.num_heads
16
4
text_config.num_hidden_layers
16
2
text_config.num_key_value_heads
8
2
vision_config.hidden_size
768
64
vision_config.intermediate_size
3072
128
vision_config.num_attention_heads
12
4
vision_config.num_hidden_layers
12
2
+
\ No newline at end of file
diff --git a/shrink/LLaVA-NeXT-Video-7B-hf-mic.html b/shrink/LLaVA-NeXT-Video-7B-hf-mic.html
new file mode 100644
index 0000000000000000000000000000000000000000..8712bf054fc039a3f36c4c156a63eacd9637c5dc
--- /dev/null
+++ b/shrink/LLaVA-NeXT-Video-7B-hf-mic.html
@@ -0,0 +1,34 @@
+
+
+Shrunk architecture
+
+
Shrunk architecture
huggingface-mic/LLaVA-NeXT-Video-7B-hf-mic@e025a3d15d2cc35a893c1362a237f82b09b90410 ← llava-hf/LLaVA-NeXT-Video-7B-hf@f417d9edeb69a9eaa88283780332d00ce3c66593 · LlavaNextVideoForConditionalGeneration · random weights · built before the shrink rule (no recorded recipe)
+
Original7.1Bparameters
Fixture4.3Mparameters
Layer stacks2compared
Findings0to review
✓ Same architecture, smaller
Every module class, layer kind and checked property of the original is in the fixture.
Layer stacks
One square per layer, colored by kind: the layer's module classes and type attributes. Hover a square for its index and kind.
model.vision_tower.encoder.layers
Original · 24
Fixture · 2
Kind
Original
Fixture
CLIPEncoderLayer
24
2
✓ kept
model.language_model.layers
Original · 32
Fixture · 2
Kind
Original
Fixture
LlamaDecoderLayer
32
2
✓ kept
Where the parameters are
Share of each model's parameters per component. Vocabulary and patch sizes are kept, so embeddings usually take a larger share of the fixture.
Original
Fixture
Component
Original
Share
Fixture
Share
model.language_model.layers
6.5B
91.7%
82.2K
1.9%
model.vision_tower.encoder.layers
302.3M
4.3%
66.9K
1.5%
model.language_model
131.3M
1.9%
2.1M
47.3%
lm_head
131.3M
1.9%
2.1M
47.3%
model.multi_modal_projector
21.0M
0.3%
8.3K
0.2%
model.vision_tower
1.2M
0.0%
74.9K
1.7%
model
4.1K
0.0%
64
0.0%
Preserved properties
Config
Property
Original
Fixture
model
model_type
llava_next_video
llava_next_video
✓ kept
model
tie_word_embeddings
False
False
✓ kept
vision_config
model_type
clip_vision_model
clip_vision_model
✓ kept
vision_config
vocab_size
32000
32000
✓ kept
vision_config
hidden_act
quick_gelu
quick_gelu
✓ kept
vision_config
image_size
336
336
✓ kept
vision_config
patch_size
14
14
✓ kept
vision_config
num_channels
3
3
✓ kept
text_config
attention style
MHA (32 heads, 32 KV)
MHA (4 heads, 4 KV)
✓ kept
text_config
RoPE type
linear
linear
✓ kept
text_config
model_type
llama
llama
✓ kept
text_config
vocab_size
32064
32064
✓ kept
text_config
tie_word_embeddings
False
False
✓ kept
text_config
hidden_act
silu
silu
✓ kept
text_config
max_position_embeddings
4096
4096
✓ kept
Module classes
Instances of each class in the built model (prefix LlavaNextVideo omitted).
Class
Original
Fixture
Linear
371
29
LlamaRMSNorm
65
5
LayerNorm
50
6
LlamaAttention
32
2
LlamaDecoderLayer
32
2
LlamaMLP
32
2
SiLUActivation
32
2
CLIPAttention
24
2
CLIPEncoderLayer
24
2
CLIPMLP
24
2
QuickGELUActivation
24
2
Embedding
2
2
ModuleList
2
2
AvgPool2d
1
1
CLIPEncoder
1
1
CLIPVisionEmbeddings
1
1
CLIPVisionModel
1
1
Conv2d
1
1
GELUActivation
1
1
LlamaModel
1
1
LlamaRotaryEmbedding
1
1
ForConditionalGeneration
1
1
Model
1
1
MultiModalProjector
1
1
Pooler
1
1
Config changes
11 fields changed, 67 unchanged.
Field
Original
Fixture
text_config.head_dim
128
16
text_config.hidden_size
4096
64
text_config.intermediate_size
11008
128
text_config.num_attention_heads
32
4
text_config.num_hidden_layers
32
2
text_config.num_key_value_heads
32
4
vision_config.hidden_size
1024
64
vision_config.intermediate_size
4096
128
vision_config.num_attention_heads
16
4
vision_config.num_hidden_layers
24
2
vision_config.projection_dim
768
32
+
\ No newline at end of file
diff --git a/shrink/Laguna-XS.2-mic.html b/shrink/Laguna-XS.2-mic.html
new file mode 100644
index 0000000000000000000000000000000000000000..c23042c055589605a4078249c313dc1c3547892b
--- /dev/null
+++ b/shrink/Laguna-XS.2-mic.html
@@ -0,0 +1,34 @@
+
+
+Shrunk architecture
+
+
Shrunk architecture
huggingface-mic/Laguna-XS.2-mic@21bbfb128fc8f6764e660376a3482ec8b6e8c93b ← poolside/Laguna-XS.2@a397bde04501591d0ac3d74a5d9d0a4ec298ffb8 · LagunaForCausalLM · random weights · built before the shrink rule (no recorded recipe)
+
Original33.4Bparameters
Fixture13.2Mparameters
Layer stacks1compared
Findings0to review
✓ Same architecture, smaller
Every module class, layer kind and checked property of the original is in the fixture.
Layer stacks
One square per layer, colored by kind: the layer's module classes and type attributes. Hover a square for its index and kind.
model.layers
Original · 40
Fixture · 2
Kind
Original
Fixture
no Experts
1
1
✓ kept
Experts, SparseMoeBlock, TopKRouter
39
1
✓ kept
Where the parameters are
Share of each model's parameters per component. Vocabulary and patch sizes are kept, so embeddings usually take a larger share of the fixture.
Original
Fixture
Component
Original
Share
Fixture
Share
model.layers
33.0B
98.8%
319.0K
2.4%
lm_head
205.5M
0.6%
6.4M
48.8%
model.embed_tokens
205.5M
0.6%
6.4M
48.8%
model.norm
2.0K
0.0%
64
0.0%
Preserved properties
Config
Property
Original
Fixture
model
attention style
GQA (48 heads, 8 KV)
GQA (4 heads, 2 KV)
✓ kept
model
MoE routing
top-8 of 256 experts
top-2 of 4 experts
✓ kept
model
model_type
laguna
laguna
✓ kept
model
vocab_size
100352
100352
✓ kept
model
tie_word_embeddings
False
False
✓ kept
model
hidden_act
silu
silu
✓ kept
model
max_position_embeddings
262144
262144
✓ kept
Module classes
Instances of each class in the built model (prefix Laguna omitted).
+
\ No newline at end of file
diff --git a/shrink/MiniCPM3-4B-mic.html b/shrink/MiniCPM3-4B-mic.html
new file mode 100644
index 0000000000000000000000000000000000000000..816529da0456dc0394f117c07d50395341ab89bf
--- /dev/null
+++ b/shrink/MiniCPM3-4B-mic.html
@@ -0,0 +1,34 @@
+
+
+Shrunk architecture
+
+
Shrunk architecture
huggingface-mic/MiniCPM3-4B-mic@874f11d622cb165e995468f15d2f43f242b52b4a ← openbmb/MiniCPM3-4B@d6b14ddaefdb11c624dd75c3c779549bc90b08cb · MiniCPM3ForCausalLM · random weights · built before the shrink rule (no recorded recipe)
+
Original4.1Bparameters
Fixture4.8Mparameters
Layer stacks1compared
Findings0to review
✓ Same architecture, smaller
Every module class, layer kind and checked property of the original is in the fixture.
Layer stacks
One square per layer, colored by kind: the layer's module classes and type attributes. Hover a square for its index and kind.
model.layers
Original · 62
Fixture · 2
Kind
Original
Fixture
DecoderLayer
62
2
✓ kept
Where the parameters are
Share of each model's parameters per component. Vocabulary and patch sizes are kept, so embeddings usually take a larger share of the fixture.
Original
Fixture
Component
Original
Share
Fixture
Share
model.layers
3.9B
95.4%
70.0K
1.5%
model.embed_tokens
188.0M
4.6%
4.7M
98.5%
model.norm
2.6K
0.0%
64
0.0%
Preserved properties
Config
Property
Original
Fixture
model
attention style
MHA (40 heads, 40 KV)
MHA (4 heads, 4 KV)
✓ kept
model
RoPE type
longrope
longrope
✓ kept
model
model_type
minicpm3
minicpm3
✓ kept
model
vocab_size
73448
73448
✓ kept
model
tie_word_embeddings
True
True
✓ kept
model
hidden_act
silu
silu
✓ kept
model
max_position_embeddings
32768
32768
✓ kept
Module classes
Instances of each class in the built model (prefix MiniCPM3 omitted).
+
\ No newline at end of file
diff --git a/shrink/Ministral-3-3B-Instruct-2512-mic.html b/shrink/Ministral-3-3B-Instruct-2512-mic.html
new file mode 100644
index 0000000000000000000000000000000000000000..e31c7a9607dc3696edda5e7f3873b250e23e6895
--- /dev/null
+++ b/shrink/Ministral-3-3B-Instruct-2512-mic.html
@@ -0,0 +1,34 @@
+
+
+Shrunk architecture
+
+
Shrunk architecture
huggingface-mic/Ministral-3-3B-Instruct-2512-mic@444a971de7a3a92574df14dcdd5dde549e0d660f ← mistralai/Ministral-3-3B-Instruct-2512@b35d4dfe56c142746f54dbd64f579faab2744308 · Mistral3ForConditionalGeneration · random weights · built before the shrink rule (no recorded recipe)
+
Original3.8Bparameters
Fixture8.6Mparameters
Layer stacks2compared
Findings0to review
✓ Same architecture, smaller
Every module class, layer kind and checked property of the original is in the fixture.
Layer stacks
One square per layer, colored by kind: the layer's module classes and type attributes. Hover a square for its index and kind.
model.vision_tower.transformer.layers
Original · 24
Fixture · 2
Kind
Original
Fixture
PixtralAttentionLayer
24
2
✓ kept
model.language_model.layers
Original · 26
Fixture · 2
Kind
Original
Fixture
Ministral3DecoderLayer
26
2
✓ kept
Where the parameters are
Share of each model's parameters per component. Vocabulary and patch sizes are kept, so embeddings usually take a larger share of the fixture.
Original
Fixture
Component
Original
Share
Fixture
Share
model.language_model.layers
3.0B
78.6%
74.0K
0.9%
model.vision_tower.transformer.layers
402.7M
10.5%
82.2K
1.0%
model.language_model
402.7M
10.5%
8.4M
97.5%
model.multi_modal_projector
16.8M
0.4%
24.6K
0.3%
model.vision_tower
603.1K
0.0%
37.7K
0.4%
Preserved properties
Config
Property
Original
Fixture
model
model_type
mistral3
mistral3
✓ kept
model
tie_word_embeddings
True
True
✓ kept
vision_config
RoPE type
axial
axial
✓ kept
vision_config
model_type
pixtral
pixtral
✓ kept
vision_config
hidden_act
silu
silu
✓ kept
vision_config
image_size
1540
1540
✓ kept
vision_config
patch_size
14
14
✓ kept
vision_config
num_channels
3
3
✓ kept
text_config
attention style
GQA (32 heads, 8 KV)
GQA (4 heads, 2 KV)
✓ kept
text_config
RoPE type
yarn
yarn
✓ kept
text_config
model_type
ministral3
ministral3
✓ kept
text_config
vocab_size
131072
131072
✓ kept
text_config
tie_word_embeddings
True
True
✓ kept
text_config
hidden_act
silu
silu
✓ kept
text_config
max_position_embeddings
262144
262144
✓ kept
Module classes
Instances of each class in the built model (prefix Mistral3 omitted).
+
\ No newline at end of file
diff --git a/shrink/Mistral-7B-Instruct-v0.2-mic.html b/shrink/Mistral-7B-Instruct-v0.2-mic.html
new file mode 100644
index 0000000000000000000000000000000000000000..1233543a12b96facaed9566e580bbeb4ef95d505
--- /dev/null
+++ b/shrink/Mistral-7B-Instruct-v0.2-mic.html
@@ -0,0 +1,34 @@
+
+
+Shrunk architecture
+
+
Shrunk architecture
huggingface-mic/Mistral-7B-Instruct-v0.2-mic@90938461b4cb2497a78ea7f817f89df453996848 ← mistralai/Mistral-7B-Instruct-v0.2@63a8b081895390a26e140280378bc85ec8bce07a · MistralForCausalLM · random weights · built before the shrink rule (no recorded recipe)
+
Original7.2Bparameters
Fixture8.4Mparameters
Layer stacks1compared
Findings1to review
⚠ To review
model: max_position_embeddings 32768 became 512
Layer stacks
One square per layer, colored by kind: the layer's module classes and type attributes. Hover a square for its index and kind.
model.layers
Original · 32
Fixture · 2
Kind
Original
Fixture
DecoderLayer
32
2
✓ kept
Where the parameters are
Share of each model's parameters per component. Vocabulary and patch sizes are kept, so embeddings usually take a larger share of the fixture.
Original
Fixture
Component
Original
Share
Fixture
Share
model.layers
7.0B
96.4%
246.3K
2.9%
model.embed_tokens
131.1M
1.8%
4.1M
48.5%
lm_head
131.1M
1.8%
4.1M
48.5%
model.norm
4.1K
0.0%
128
0.0%
Preserved properties
Config
Property
Original
Fixture
model
attention style
GQA (32 heads, 8 KV)
GQA (4 heads, 2 KV)
✓ kept
model
RoPE type
default
default
✓ kept
model
model_type
mistral
mistral
✓ kept
model
vocab_size
32000
32000
✓ kept
model
tie_word_embeddings
False
False
✓ kept
model
hidden_act
silu
silu
✓ kept
model
max_position_embeddings
32768
512
⚠ changed
Module classes
Instances of each class in the built model (prefix Mistral omitted).
Class
Original
Fixture
Linear
225
15
RMSNorm
65
5
Attention
32
2
DecoderLayer
32
2
MLP
32
2
SiLUActivation
32
2
Embedding
1
1
ForCausalLM
1
1
Model
1
1
RotaryEmbedding
1
1
ModuleList
1
1
Config changes
7 fields changed, 21 unchanged.
Field
Original
Fixture
head_dim
128
16
hidden_size
4096
128
intermediate_size
14336
256
max_position_embeddings
32768
512
num_attention_heads
32
4
num_hidden_layers
32
2
num_key_value_heads
8
2
+
\ No newline at end of file
diff --git a/shrink/Mixtral-8x7B-Instruct-v0.1-mic.html b/shrink/Mixtral-8x7B-Instruct-v0.1-mic.html
new file mode 100644
index 0000000000000000000000000000000000000000..7dd8050beb1d5fd1f0caeaebaf5d93df23c146dd
--- /dev/null
+++ b/shrink/Mixtral-8x7B-Instruct-v0.1-mic.html
@@ -0,0 +1,34 @@
+
+
+Shrunk architecture
+
+
Shrunk architecture
huggingface-mic/Mixtral-8x7B-Instruct-v0.1-mic@9c86dd7f51c866a4511616932943f95f6d163e66 ← mistralai/Mixtral-8x7B-Instruct-v0.1@eba92302a2861cdc0098cc54bc9f17cb2c47eb61 · MixtralForCausalLM · random weights
+
Original46.7Bparameters
Fixture4.3Mparameters
Layer stacks1compared
Findings0to review
✓ Same architecture, smaller
Every module class, layer kind and checked property of the original is in the fixture.
Layer stacks
One square per layer, colored by kind: the layer's module classes and type attributes. Hover a square for its index and kind.
model.layers
Original · 32
Fixture · 2
Kind
Original
Fixture
DecoderLayer
32
2
✓ kept
Where the parameters are
Share of each model's parameters per component. Vocabulary and patch sizes are kept, so embeddings usually take a larger share of the fixture.
Original
Fixture
Component
Original
Share
Fixture
Share
model.layers
46.4B
99.4%
222.0K
5.1%
lm_head
131.1M
0.3%
2.0M
47.4%
model.embed_tokens
131.1M
0.3%
2.0M
47.4%
model.norm
4.1K
0.0%
64
0.0%
Preserved properties
Config
Property
Original
Fixture
model
attention style
GQA (32 heads, 8 KV)
GQA (4 heads, 2 KV)
✓ kept
model
MoE routing
top-2 of 8 experts
top-2 of 4 experts
✓ kept
model
RoPE type
default
default
✓ kept
model
model_type
mixtral
mixtral
✓ kept
model
vocab_size
32000
32000
✓ kept
model
tie_word_embeddings
False
False
✓ kept
model
hidden_act
silu
silu
✓ kept
model
max_position_embeddings
32768
32768
✓ kept
Module classes
Instances of each class in the built model (prefix Mixtral omitted).
Class
Original
Fixture
Linear
129
9
RMSNorm
65
5
Attention
32
2
DecoderLayer
32
2
Experts
32
2
SparseMoeBlock
32
2
TopKRouter
32
2
SiLUActivation
32
2
Embedding
1
1
ForCausalLM
1
1
Model
1
1
RotaryEmbedding
1
1
ModuleList
1
1
Config changes
6 fields changed, 27 unchanged.
Field
Original
Fixture
hidden_size
4096
64
intermediate_size
14336
128
num_attention_heads
32
4
num_hidden_layers
32
2
num_key_value_heads
8
2
num_local_experts
8
4
+
\ No newline at end of file
diff --git a/shrink/NVIDIA-Nemotron-3-Nano-4B-BF16-mic.html b/shrink/NVIDIA-Nemotron-3-Nano-4B-BF16-mic.html
new file mode 100644
index 0000000000000000000000000000000000000000..60f8b259be959c1d26f30b277721e97c627c9000
--- /dev/null
+++ b/shrink/NVIDIA-Nemotron-3-Nano-4B-BF16-mic.html
@@ -0,0 +1,34 @@
+
+
+Shrunk architecture
+
+
Shrunk architecture
huggingface-mic/NVIDIA-Nemotron-3-Nano-4B-BF16-mic@9e2cf8579314aef2d07fb651a0e47989c49ce98d ← nvidia/NVIDIA-Nemotron-3-Nano-4B-BF16@dfaf35de3e30f1867dd8dbc38a7fc9fb52d3914f · NemotronHForCausalLM · random weights · built before the shrink rule (no recorded recipe)
+
Original4.0Bparameters
Fixture49.5Mparameters
Layer stacks1compared
Findings0to review
✓ Same architecture, smaller
Every module class, layer kind and checked property of the original is in the fixture.
Layer stacks
One square per layer, colored by kind: the layer's module classes and type attributes. Hover a square for its index and kind.
model.layers
Original · 42
Fixture · 42
Kind
Original
Fixture
linear_attention, Conv1d, Mamba2Mixer …
21
21
✓ kept
MLP, ReLUSquaredActivation
17
17
✓ kept
is_causal=True, Attention
4
4
✓ kept
Where the parameters are
Share of each model's parameters per component. Vocabulary and patch sizes are kept, so embeddings usually take a larger share of the fixture.
Original
Fixture
Component
Original
Share
Fixture
Share
model.layers
3.2B
79.3%
32.8M
66.1%
lm_head
411.0M
10.3%
8.4M
16.9%
model.embeddings
411.0M
10.3%
8.4M
16.9%
model.norm_f
3.1K
0.0%
64
0.0%
Preserved properties
Config
Property
Original
Fixture
model
attention style
GQA (40 heads, 8 KV)
GQA (4 heads, 2 KV)
✓ kept
model
MoE routing
top-2 of 8 experts
top-2 of 4 experts
✓ kept
model
model_type
nemotron_h
nemotron_h
✓ kept
model
vocab_size
131072
131072
✓ kept
model
tie_word_embeddings
False
False
✓ kept
model
max_position_embeddings
262144
262144
✓ kept
Module classes
Instances of each class in the built model (prefix NemotronH omitted).
Class
Original
Fixture
Linear
93
93
RMSNorm
43
43
Block
42
42
Conv1d
21
21
Mamba2Mixer
21
21
RMSNormGated
21
21
SiLUActivation
21
21
MLP
17
17
ReLUSquaredActivation
17
17
Attention
4
4
Embedding
1
1
ModuleList
1
1
ForCausalLM
1
1
Model
1
1
Config changes
9 fields changed, 58 unchanged.
Field
Original
Fixture
chunk_size
256
8
head_dim
128
16
hidden_size
3136
64
intermediate_size
12544
128
moe_intermediate_size
7688
32
n_routed_experts
8
4
num_attention_heads
40
4
num_key_value_heads
8
2
ssm_state_size
128
16
+
\ No newline at end of file
diff --git a/shrink/Nemotron-3-Embed-1B-BF16-mic.html b/shrink/Nemotron-3-Embed-1B-BF16-mic.html
new file mode 100644
index 0000000000000000000000000000000000000000..6f6adaadb9b384ef5d36477d5c7fbd666980c7d8
--- /dev/null
+++ b/shrink/Nemotron-3-Embed-1B-BF16-mic.html
@@ -0,0 +1,34 @@
+
+
+Shrunk architecture
+
+
Shrunk architecture
huggingface-mic/Nemotron-3-Embed-1B-BF16-mic@645864f9759d320c610786a2270ecb18e982534a ← nvidia/Nemotron-3-Embed-1B-BF16@c0c9fea93ea424587517f2c59e20db9f1d6bf615 · Ministral3Model · random weights
+
Original1.1Bparameters
Fixture8.5Mparameters
Layer stacks1compared
Findings0to review
✓ Same architecture, smaller
Every module class, layer kind and checked property of the original is in the fixture.
Layer stacks
One square per layer, colored by kind: the layer's module classes and type attributes. Hover a square for its index and kind.
layers
Original · 16
Fixture · 2
Kind
Original
Fixture
DecoderLayer
16
2
✓ kept
Where the parameters are
Share of each model's parameters per component. Vocabulary and patch sizes are kept, so embeddings usually take a larger share of the fixture.
Original
Fixture
Component
Original
Share
Fixture
Share
layers
872.5M
76.5%
74.0K
0.9%
embed_tokens
268.4M
23.5%
8.4M
99.1%
norm
2.0K
0.0%
64
0.0%
Preserved properties
Config
Property
Original
Fixture
model
attention style
GQA (24 heads, 8 KV)
GQA (4 heads, 2 KV)
✓ kept
model
RoPE type
yarn
yarn
✓ kept
model
model_type
ministral3
ministral3
✓ kept
model
vocab_size
131072
131072
✓ kept
model
tie_word_embeddings
True
True
✓ kept
model
hidden_act
silu
silu
✓ kept
model
max_position_embeddings
262144
262144
✓ kept
Module classes
Instances of each class in the built model (prefix Ministral3 omitted).
Class
Original
Fixture
Linear
112
14
RMSNorm
33
5
Attention
16
2
DecoderLayer
16
2
MLP
16
2
SiLUActivation
16
2
Embedding
1
1
Model
1
1
RotaryEmbedding
1
1
ModuleList
1
1
Config changes
6 fields changed, 26 unchanged.
Field
Original
Fixture
head_dim
128
16
hidden_size
2048
64
intermediate_size
6144
128
num_attention_heads
24
4
num_hidden_layers
16
2
num_key_value_heads
8
2
+
\ No newline at end of file
diff --git a/shrink/OLMo-1B-hf-mic.html b/shrink/OLMo-1B-hf-mic.html
new file mode 100644
index 0000000000000000000000000000000000000000..629d5d1a54013f4a5fdd744195b2e53bbedc9f61
--- /dev/null
+++ b/shrink/OLMo-1B-hf-mic.html
@@ -0,0 +1,34 @@
+
+
+Shrunk architecture
+
+
Shrunk architecture
huggingface-mic/OLMo-1B-hf-mic@87c81bd7d9f93663d9906bbde0737125e4d85c54 ← allenai/OLMo-1B-hf@aee7752d9c08ee4775e9b0091426d8410e8f6a89 · OlmoForCausalLM · random weights
+
Original1.2Bparameters
Fixture3.3Mparameters
Layer stacks1compared
Findings0to review
✓ Same architecture, smaller
Every module class, layer kind and checked property of the original is in the fixture.
Layer stacks
One square per layer, colored by kind: the layer's module classes and type attributes. Hover a square for its index and kind.
model.layers
Original · 16
Fixture · 2
Kind
Original
Fixture
DecoderLayer
16
2
✓ kept
Where the parameters are
Share of each model's parameters per component. Vocabulary and patch sizes are kept, so embeddings usually take a larger share of the fixture.
Original
Fixture
Component
Original
Share
Fixture
Share
model.layers
1.1B
91.2%
81.9K
2.5%
model.embed_tokens
103.0M
8.8%
3.2M
97.5%
Preserved properties
Config
Property
Original
Fixture
model
attention style
MHA (16 heads, 16 KV)
MHA (4 heads, 4 KV)
✓ kept
model
RoPE type
default
default
✓ kept
model
model_type
olmo
olmo
✓ kept
model
vocab_size
50304
50304
✓ kept
model
tie_word_embeddings
True
True
✓ kept
model
hidden_act
silu
silu
✓ kept
model
max_position_embeddings
2048
2048
✓ kept
Module classes
Instances of each class in the built model (prefix Olmo omitted).
Class
Original
Fixture
Linear
113
15
LayerNorm
33
5
Attention
16
2
DecoderLayer
16
2
MLP
16
2
SiLUActivation
16
2
Embedding
1
1
ModuleList
1
1
ForCausalLM
1
1
Model
1
1
RotaryEmbedding
1
1
Config changes
5 fields changed, 23 unchanged.
Field
Original
Fixture
hidden_size
2048
64
intermediate_size
8192
128
num_attention_heads
16
4
num_hidden_layers
16
2
num_key_value_heads
16
4
+
\ No newline at end of file
diff --git a/shrink/OLMo-2-0425-1B-mic.html b/shrink/OLMo-2-0425-1B-mic.html
new file mode 100644
index 0000000000000000000000000000000000000000..30f3e4376c6fb1219c1719d9d8feca661d30cc59
--- /dev/null
+++ b/shrink/OLMo-2-0425-1B-mic.html
@@ -0,0 +1,34 @@
+
+
+Shrunk architecture
+
+
Shrunk architecture
huggingface-mic/OLMo-2-0425-1B-mic@7a6eea1e741066708957df80693b23000613b907 ← allenai/OLMo-2-0425-1B@a1847dff35000b4271fa70afc5db10fd29fedbdf · Olmo2ForCausalLM · random weights
+
Original1.5Bparameters
Fixture12.9Mparameters
Layer stacks1compared
Findings0to review
✓ Same architecture, smaller
Every module class, layer kind and checked property of the original is in the fixture.
Layer stacks
One square per layer, colored by kind: the layer's module classes and type attributes. Hover a square for its index and kind.
model.layers
Original · 16
Fixture · 2
Kind
Original
Fixture
DecoderLayer
16
2
✓ kept
Where the parameters are
Share of each model's parameters per component. Vocabulary and patch sizes are kept, so embeddings usually take a larger share of the fixture.
Original
Fixture
Component
Original
Share
Fixture
Share
model.layers
1.1B
72.3%
82.4K
0.6%
model.embed_tokens
205.5M
13.8%
6.4M
49.7%
lm_head
205.5M
13.8%
6.4M
49.7%
model.norm
2.0K
0.0%
64
0.0%
Preserved properties
Config
Property
Original
Fixture
model
attention style
MHA (16 heads, 16 KV)
MHA (4 heads, 4 KV)
✓ kept
model
RoPE type
default
default
✓ kept
model
model_type
olmo2
olmo2
✓ kept
model
vocab_size
100352
100352
✓ kept
model
tie_word_embeddings
False
False
✓ kept
model
hidden_act
silu
silu
✓ kept
model
max_position_embeddings
4096
4096
✓ kept
Module classes
Instances of each class in the built model (prefix Olmo2 omitted).
Class
Original
Fixture
Linear
113
15
RMSNorm
65
9
Attention
16
2
DecoderLayer
16
2
MLP
16
2
SiLUActivation
16
2
Embedding
1
1
ModuleList
1
1
ForCausalLM
1
1
Model
1
1
RotaryEmbedding
1
1
Config changes
5 fields changed, 22 unchanged.
Field
Original
Fixture
hidden_size
2048
64
intermediate_size
8192
128
num_attention_heads
16
4
num_hidden_layers
16
2
num_key_value_heads
16
4
+
\ No newline at end of file
diff --git a/shrink/OLMoE-1B-7B-0924-mic.html b/shrink/OLMoE-1B-7B-0924-mic.html
new file mode 100644
index 0000000000000000000000000000000000000000..b7697681bb69d4e281ce6de6e45207a3d6973324
--- /dev/null
+++ b/shrink/OLMoE-1B-7B-0924-mic.html
@@ -0,0 +1,34 @@
+
+
+Shrunk architecture
+
+
Shrunk architecture
huggingface-mic/OLMoE-1B-7B-0924-mic@abd7fb026a08bdb27dfa8ff4d6a1680721f1e85a ← allenai/OLMoE-1B-7B-0924@6d84c48581ece794365f2b8e9cfb043c68ade9c5 · OlmoeForCausalLM · random weights
+
Original6.9Bparameters
Fixture6.7Mparameters
Layer stacks1compared
Findings0to review
✓ Same architecture, smaller
Every module class, layer kind and checked property of the original is in the fixture.
Layer stacks
One square per layer, colored by kind: the layer's module classes and type attributes. Hover a square for its index and kind.
model.layers
Original · 16
Fixture · 2
Kind
Original
Fixture
DecoderLayer
16
2
✓ kept
Where the parameters are
Share of each model's parameters per component. Vocabulary and patch sizes are kept, so embeddings usually take a larger share of the fixture.
Original
Fixture
Component
Original
Share
Fixture
Share
model.layers
6.7B
97.0%
230.4K
3.5%
model.embed_tokens
103.0M
1.5%
3.2M
48.3%
lm_head
103.0M
1.5%
3.2M
48.3%
model.norm
2.0K
0.0%
64
0.0%
Preserved properties
Config
Property
Original
Fixture
model
attention style
MHA (16 heads, 16 KV)
MHA (4 heads, 4 KV)
✓ kept
model
MoE routing
top-8 of 64 experts
top-2 of 4 experts
✓ kept
model
RoPE type
default
default
✓ kept
model
model_type
olmoe
olmoe
✓ kept
model
vocab_size
50304
50304
✓ kept
model
tie_word_embeddings
False
False
✓ kept
model
hidden_act
silu
silu
✓ kept
model
max_position_embeddings
4096
4096
✓ kept
Module classes
Instances of each class in the built model (prefix Olmoe omitted).
Class
Original
Fixture
Linear
65
9
RMSNorm
65
9
Attention
16
2
DecoderLayer
16
2
Experts
16
2
SparseMoeBlock
16
2
TopKRouter
16
2
SiLUActivation
16
2
Embedding
1
1
ModuleList
1
1
ForCausalLM
1
1
Model
1
1
RotaryEmbedding
1
1
Config changes
7 fields changed, 26 unchanged.
Field
Original
Fixture
hidden_size
2048
64
intermediate_size
1024
128
num_attention_heads
16
4
num_experts
64
4
num_experts_per_tok
8
2
num_hidden_layers
16
2
num_key_value_heads
16
4
+
\ No newline at end of file
diff --git a/shrink/Olmo-3-7B-Instruct-mic.html b/shrink/Olmo-3-7B-Instruct-mic.html
new file mode 100644
index 0000000000000000000000000000000000000000..5867734cfec34930f34907058694e15ea6e0e110
--- /dev/null
+++ b/shrink/Olmo-3-7B-Instruct-mic.html
@@ -0,0 +1,34 @@
+
+
+Shrunk architecture
+
+
Shrunk architecture
huggingface-mic/Olmo-3-7B-Instruct-mic@3d2269c8c387b9dec2f5a987d2739464ad990224 ← allenai/Olmo-3-7B-Instruct@6e5971d9eba42665f5bd5a0fcf047f299ce1dccc · Olmo3ForCausalLM · random weights
+
Original7.3Bparameters
Fixture12.9Mparameters
Layer stacks1compared
Findings0to review
✓ Same architecture, smaller
Every module class, layer kind and checked property of the original is in the fixture.
Layer stacks
One square per layer, colored by kind: the layer's module classes and type attributes. Hover a square for its index and kind.
model.layers
Original · 32
Fixture · 2
Kind
Original
Fixture
sliding_attention
24
1
✓ kept
full_attention
8
1
✓ kept
Where the parameters are
Share of each model's parameters per component. Vocabulary and patch sizes are kept, so embeddings usually take a larger share of the fixture.
Original
Fixture
Component
Original
Share
Fixture
Share
model.layers
6.5B
88.7%
82.4K
0.6%
model.embed_tokens
410.7M
5.6%
6.4M
49.7%
lm_head
410.7M
5.6%
6.4M
49.7%
model.norm
4.1K
0.0%
64
0.0%
Preserved properties
Config
Property
Original
Fixture
model
attention style
MHA (32 heads, 32 KV)
MHA (4 heads, 4 KV)
✓ kept
model
model_type
olmo3
olmo3
✓ kept
model
vocab_size
100278
100278
✓ kept
model
tie_word_embeddings
False
False
✓ kept
model
hidden_act
silu
silu
✓ kept
model
max_position_embeddings
65536
65536
✓ kept
Module classes
Instances of each class in the built model (prefix Olmo3 omitted).
+
\ No newline at end of file
diff --git a/shrink/Phi-3.5-mini-instruct-mic.html b/shrink/Phi-3.5-mini-instruct-mic.html
new file mode 100644
index 0000000000000000000000000000000000000000..097a62a59c1293d59b9008a2e4351a1bb461971d
--- /dev/null
+++ b/shrink/Phi-3.5-mini-instruct-mic.html
@@ -0,0 +1,34 @@
+
+
+Shrunk architecture
+
+
Shrunk architecture
huggingface-mic/Phi-3.5-mini-instruct-mic@0c7964e6c77b965e4cccc04ccb4f4d4a20a9ff81 ← microsoft/Phi-3.5-mini-instruct@2fe192450127e6a83f7441aef6e3ca586c338b77 · Phi3ForCausalLM · random weights · built before the shrink rule (no recorded recipe)
+
Original3.8Bparameters
Fixture4.2Mparameters
Layer stacks1compared
Findings0to review
✓ Same architecture, smaller
Every module class, layer kind and checked property of the original is in the fixture.
Layer stacks
One square per layer, colored by kind: the layer's module classes and type attributes. Hover a square for its index and kind.
model.layers
Original · 32
Fixture · 2
Kind
Original
Fixture
DecoderLayer
32
2
✓ kept
Where the parameters are
Share of each model's parameters per component. Vocabulary and patch sizes are kept, so embeddings usually take a larger share of the fixture.
Original
Fixture
Component
Original
Share
Fixture
Share
model.layers
3.6B
94.8%
82.2K
2.0%
lm_head
98.5M
2.6%
2.1M
49.0%
model.embed_tokens
98.5M
2.6%
2.1M
49.0%
model.norm
3.1K
0.0%
64
0.0%
Preserved properties
Config
Property
Original
Fixture
model
attention style
MHA (32 heads, 32 KV)
MHA (4 heads, 4 KV)
✓ kept
model
RoPE type
longrope
longrope
✓ kept
model
model_type
phi3
phi3
✓ kept
model
vocab_size
32064
32064
✓ kept
model
tie_word_embeddings
False
False
✓ kept
model
hidden_act
silu
silu
✓ kept
model
max_position_embeddings
131072
131072
✓ kept
Module classes
Instances of each class in the built model (prefix Phi3 omitted).
+
\ No newline at end of file
diff --git a/shrink/Phi-4-multimodal-instruct-mic.html b/shrink/Phi-4-multimodal-instruct-mic.html
new file mode 100644
index 0000000000000000000000000000000000000000..a797629e9b7e17d67e6bff65cd85c596301b627e
--- /dev/null
+++ b/shrink/Phi-4-multimodal-instruct-mic.html
@@ -0,0 +1,34 @@
+
+
+Shrunk architecture
+
+
Shrunk architecture
huggingface-mic/Phi-4-multimodal-instruct-mic@721c204bb9a6c7113710f11eb53f6798c3b87039 ← microsoft/Phi-4-multimodal-instruct@b5b5a5f60533515297d0e14ece86ed80d4b13b19 · Phi4MultimodalForCausalLM · random weights · built before the shrink rule (no recorded recipe)
+
Original4.7Bparameters
Fixture16.6Mparameters
Layer stacks3compared
Findings0to review
✓ Same architecture, smaller
Every module class, layer kind and checked property of the original is in the fixture.
Layer stacks
One square per layer, colored by kind: the layer's module classes and type attributes. Hover a square for its index and kind.
+
\ No newline at end of file
diff --git a/shrink/Qwen1.5-MoE-A2.7B-mic.html b/shrink/Qwen1.5-MoE-A2.7B-mic.html
new file mode 100644
index 0000000000000000000000000000000000000000..feed41be511ef8ba6219f9d10e3106efbec6abad
--- /dev/null
+++ b/shrink/Qwen1.5-MoE-A2.7B-mic.html
@@ -0,0 +1,34 @@
+
+
+Shrunk architecture
+
+
Shrunk architecture
huggingface-mic/Qwen1.5-MoE-A2.7B-mic@2fd4813071f2c23b77666d8c796ff80235d42b55 ← Qwen/Qwen1.5-MoE-A2.7B@1a758c50ecb6350748b9ce0a99d2352fd9fc11c9 · Qwen2MoeForCausalLM · random weights
+
Original14.3Bparameters
Fixture19.6Mparameters
Layer stacks1compared
Findings0to review
✓ Same architecture, smaller
Every module class, layer kind and checked property of the original is in the fixture.
Layer stacks
One square per layer, colored by kind: the layer's module classes and type attributes. Hover a square for its index and kind.
model.layers
Original · 24
Fixture · 2
Kind
Original
Fixture
DecoderLayer
24
2
✓ kept
Where the parameters are
Share of each model's parameters per component. Vocabulary and patch sizes are kept, so embeddings usually take a larger share of the fixture.
Original
Fixture
Component
Original
Share
Fixture
Share
model.layers
13.7B
95.7%
132.4K
0.7%
model.embed_tokens
311.2M
2.2%
9.7M
49.7%
lm_head
311.2M
2.2%
9.7M
49.7%
model.norm
2.0K
0.0%
64
0.0%
Preserved properties
Config
Property
Original
Fixture
model
attention style
MHA (16 heads, 16 KV)
MHA (4 heads, 4 KV)
✓ kept
model
MoE routing
top-4 of 60 experts
top-2 of 4 experts
✓ kept
model
RoPE type
default
default
✓ kept
model
model_type
qwen2_moe
qwen2_moe
✓ kept
model
vocab_size
151936
151936
✓ kept
model
tie_word_embeddings
False
False
✓ kept
model
hidden_act
silu
silu
✓ kept
model
max_position_embeddings
8192
8192
✓ kept
Module classes
Instances of each class in the built model (prefix Qwen2Moe omitted).
Class
Original
Fixture
Linear
193
17
RMSNorm
49
5
SiLUActivation
48
4
Attention
24
2
DecoderLayer
24
2
Experts
24
2
MLP
24
2
SparseMoeBlock
24
2
TopKRouter
24
2
Embedding
1
1
ModuleList
1
1
ForCausalLM
1
1
Model
1
1
RotaryEmbedding
1
1
Config changes
10 fields changed, 30 unchanged.
Field
Original
Fixture
hidden_size
2048
64
intermediate_size
5632
128
layer_types
24 entries: 24× full_attention
["full_attention", "full_attention"]
moe_intermediate_size
1408
32
num_attention_heads
16
4
num_experts
60
4
num_experts_per_tok
4
2
num_hidden_layers
24
2
num_key_value_heads
16
4
shared_expert_intermediate_size
5632
128
+
\ No newline at end of file
diff --git a/shrink/Qwen2-VL-2B-Instruct-mic.html b/shrink/Qwen2-VL-2B-Instruct-mic.html
new file mode 100644
index 0000000000000000000000000000000000000000..690097fa88e9a6eb7c992fb9033f92ff306a77a6
--- /dev/null
+++ b/shrink/Qwen2-VL-2B-Instruct-mic.html
@@ -0,0 +1,34 @@
+
+
+Shrunk architecture
+
+
Shrunk architecture
huggingface-mic/Qwen2-VL-2B-Instruct-mic@743a11f21777bbaacd3519d20014517268130159 ← Qwen/Qwen2-VL-2B-Instruct@895c3a49bc3fa70a340399125c650a463535e71c · Qwen2VLForConditionalGeneration · random weights · built before the shrink rule (no recorded recipe)
+
Original2.2Bparameters
Fixture10.1Mparameters
Layer stacks2compared
Findings0to review
✓ Same architecture, smaller
Every module class, layer kind and checked property of the original is in the fixture.
Layer stacks
One square per layer, colored by kind: the layer's module classes and type attributes. Hover a square for its index and kind.
model.visual.blocks
Original · 32
Fixture · 2
Kind
Original
Fixture
VisionBlock
32
2
✓ kept
model.language_model.layers
Original · 28
Fixture · 2
Kind
Original
Fixture
DecoderLayer
28
2
✓ kept
Where the parameters are
Share of each model's parameters per component. Vocabulary and patch sizes are kept, so embeddings usually take a larger share of the fixture.
Original
Fixture
Component
Original
Share
Fixture
Share
model.language_model.layers
1.3B
59.3%
74.2K
0.7%
model.visual.blocks
629.7M
28.5%
100.0K
1.0%
model.language_model
233.4M
10.6%
9.7M
96.7%
model.visual
35.6M
1.6%
157.6K
1.6%
Preserved properties
Config
Property
Original
Fixture
model
model_type
qwen2_vl
qwen2_vl
✓ kept
model
tie_word_embeddings
True
True
✓ kept
text_config
attention style
GQA (12 heads, 2 KV)
GQA (4 heads, 2 KV)
✓ kept
text_config
RoPE type
default
default
✓ kept
text_config
model_type
qwen2_vl_text
qwen2_vl_text
✓ kept
text_config
vocab_size
151936
151936
✓ kept
text_config
hidden_act
silu
silu
✓ kept
text_config
max_position_embeddings
32768
32768
✓ kept
vision_config
RoPE type
axial
axial
✓ kept
vision_config
model_type
qwen2_vl_vision
qwen2_vl_vision
✓ kept
vision_config
hidden_act
quick_gelu
quick_gelu
✓ kept
vision_config
patch_size
14
14
✓ kept
Module classes
Instances of each class in the built model (prefix Qwen2VL omitted).
+
\ No newline at end of file
diff --git a/shrink/Qwen3-0.6B-mic.html b/shrink/Qwen3-0.6B-mic.html
new file mode 100644
index 0000000000000000000000000000000000000000..c922ea44176f803a343dac6d8f0cbf105377c3c6
--- /dev/null
+++ b/shrink/Qwen3-0.6B-mic.html
@@ -0,0 +1,34 @@
+
+
+Shrunk architecture
+
+
Shrunk architecture
huggingface-mic/Qwen3-0.6B-mic@f3c69d34835d4cf1d9aba9a08cd6af8282aae6a7 ← Qwen/Qwen3-0.6B@c1899de289a04d12100db370d81485cdf75e47ca · Qwen3ForCausalLM · random weights · built before the shrink rule (no recorded recipe)
+
Original596.0Mparameters
Fixture9.8Mparameters
Layer stacks1compared
Findings0to review
✓ Same architecture, smaller
Every module class, layer kind and checked property of the original is in the fixture.
Layer stacks
One square per layer, colored by kind: the layer's module classes and type attributes. Hover a square for its index and kind.
model.layers
Original · 28
Fixture · 2
Kind
Original
Fixture
DecoderLayer
28
2
✓ kept
Where the parameters are
Share of each model's parameters per component. Vocabulary and patch sizes are kept, so embeddings usually take a larger share of the fixture.
Original
Fixture
Component
Original
Share
Fixture
Share
model.layers
440.5M
73.9%
74.0K
0.8%
model.embed_tokens
155.6M
26.1%
9.7M
99.2%
model.norm
1.0K
0.0%
64
0.0%
Preserved properties
Config
Property
Original
Fixture
model
attention style
GQA (16 heads, 8 KV)
GQA (4 heads, 2 KV)
✓ kept
model
RoPE type
default
default
✓ kept
model
model_type
qwen3
qwen3
✓ kept
model
vocab_size
151936
151936
✓ kept
model
tie_word_embeddings
True
True
✓ kept
model
hidden_act
silu
silu
✓ kept
model
max_position_embeddings
40960
40960
✓ kept
Module classes
Instances of each class in the built model (prefix Qwen3 omitted).
Class
Original
Fixture
Linear
197
15
RMSNorm
113
9
Attention
28
2
DecoderLayer
28
2
MLP
28
2
SiLUActivation
28
2
Embedding
1
1
ModuleList
1
1
ForCausalLM
1
1
Model
1
1
RotaryEmbedding
1
1
Config changes
7 fields changed, 25 unchanged.
Field
Original
Fixture
head_dim
128
16
hidden_size
1024
64
intermediate_size
3072
128
layer_types
28 entries: 28× full_attention
["full_attention", "full_attention"]
num_attention_heads
16
4
num_hidden_layers
28
2
num_key_value_heads
8
2
+
\ No newline at end of file
diff --git a/shrink/Qwen3-30B-A3B-mic.html b/shrink/Qwen3-30B-A3B-mic.html
new file mode 100644
index 0000000000000000000000000000000000000000..aa65446848e3a1bac86aa50a17ac13094f70a271
--- /dev/null
+++ b/shrink/Qwen3-30B-A3B-mic.html
@@ -0,0 +1,34 @@
+
+
+Shrunk architecture
+
+
Shrunk architecture
huggingface-mic/Qwen3-30B-A3B-mic@02a209fca46248ff0b3da72e3b190428138b2449 ← Qwen/Qwen3-30B-A3B@ad44e777bcd18fa416d9da3bd8f70d33ebb85d39 · Qwen3MoeForCausalLM · random weights · built before the shrink rule (no recorded recipe)
+
Original30.5Bparameters
Fixture19.5Mparameters
Layer stacks1compared
Findings0to review
✓ Same architecture, smaller
Every module class, layer kind and checked property of the original is in the fixture.
Layer stacks
One square per layer, colored by kind: the layer's module classes and type attributes. Hover a square for its index and kind.
model.layers
Original · 48
Fixture · 2
Kind
Original
Fixture
DecoderLayer
48
2
✓ kept
Where the parameters are
Share of each model's parameters per component. Vocabulary and patch sizes are kept, so embeddings usually take a larger share of the fixture.
Original
Fixture
Component
Original
Share
Fixture
Share
model.layers
29.9B
98.0%
74.6K
0.4%
model.embed_tokens
311.2M
1.0%
9.7M
49.8%
lm_head
311.2M
1.0%
9.7M
49.8%
model.norm
2.0K
0.0%
64
0.0%
Preserved properties
Config
Property
Original
Fixture
model
attention style
GQA (32 heads, 4 KV)
GQA (4 heads, 2 KV)
✓ kept
model
MoE routing
top-8 of 128 experts
top-2 of 4 experts
✓ kept
model
RoPE type
default
default
✓ kept
model
model_type
qwen3_moe
qwen3_moe
✓ kept
model
vocab_size
151936
151936
✓ kept
model
tie_word_embeddings
False
False
✓ kept
model
hidden_act
silu
silu
✓ kept
model
max_position_embeddings
40960
40960
✓ kept
Module classes
Instances of each class in the built model (prefix Qwen3Moe omitted).
Class
Original
Fixture
Linear
193
9
RMSNorm
193
9
Attention
48
2
DecoderLayer
48
2
Experts
48
2
SparseMoeBlock
48
2
TopKRouter
48
2
SiLUActivation
48
2
Embedding
1
1
ModuleList
1
1
ForCausalLM
1
1
Model
1
1
RotaryEmbedding
1
1
Config changes
9 fields changed, 30 unchanged.
Field
Original
Fixture
head_dim
128
16
hidden_size
2048
64
intermediate_size
6144
128
moe_intermediate_size
768
32
num_attention_heads
32
4
num_experts_per_tok
8
2
num_hidden_layers
48
2
num_key_value_heads
4
2
num_local_experts
128
4
+
\ No newline at end of file
diff --git a/shrink/Qwen3-ASR-0.6B-hf-mic.html b/shrink/Qwen3-ASR-0.6B-hf-mic.html
new file mode 100644
index 0000000000000000000000000000000000000000..c86e23146a081607cbf2341597151941b7d23610
--- /dev/null
+++ b/shrink/Qwen3-ASR-0.6B-hf-mic.html
@@ -0,0 +1,34 @@
+
+
+Shrunk architecture
+
+
Shrunk architecture
huggingface-mic/Qwen3-ASR-0.6B-hf-mic@6b969c14b8cf776fa7152fa143a3c0dcc6a30b40 ← Qwen/Qwen3-ASR-0.6B-hf@7f1569a48a89f3e3f4dc3a5c9d28bddd903bc76c · Qwen3ASRForConditionalGeneration · random weights · built before the shrink rule (no recorded recipe)
+
Original782.4Mparameters
Fixture14.5Mparameters
Layer stacks2compared
Findings0to review
✓ Same architecture, smaller
Every module class, layer kind and checked property of the original is in the fixture.
Layer stacks
One square per layer, colored by kind: the layer's module classes and type attributes. Hover a square for its index and kind.
model.audio_tower.layers
Original · 18
Fixture · 2
Kind
Original
Fixture
AudioEncoderLayer
18
2
✓ kept
model.language_model.layers
Original · 28
Fixture · 2
Kind
Original
Fixture
Qwen3DecoderLayer
28
2
✓ kept
Where the parameters are
Share of each model's parameters per component. Vocabulary and patch sizes are kept, so embeddings usually take a larger share of the fixture.
Original
Fixture
Component
Original
Share
Fixture
Share
model.language_model.layers
440.5M
56.3%
74.0K
0.5%
model.audio_tower.layers
173.6M
22.2%
66.9K
0.5%
model.language_model
155.6M
19.9%
9.7M
67.0%
model.audio_tower
11.0M
1.4%
4.6M
32.0%
model.multi_modal_projector
1.7M
0.2%
8.3K
0.1%
Preserved properties
Config
Property
Original
Fixture
model
model_type
qwen3_asr
qwen3_asr
✓ kept
model
tie_word_embeddings
True
True
✓ kept
audio_config
model_type
qwen3_asr_encoder
qwen3_asr_encoder
✓ kept
audio_config
max_position_embeddings
13
13
✓ kept
audio_config
num_mel_bins
128
128
✓ kept
text_config
attention style
GQA (16 heads, 8 KV)
GQA (4 heads, 2 KV)
✓ kept
text_config
RoPE type
default
default
✓ kept
text_config
model_type
qwen3
qwen3
✓ kept
text_config
vocab_size
151936
151936
✓ kept
text_config
tie_word_embeddings
True
True
✓ kept
text_config
hidden_act
silu
silu
✓ kept
text_config
max_position_embeddings
65536
65536
✓ kept
Module classes
Instances of each class in the built model (prefix Qwen3ASR omitted).
Class
Original
Fixture
Linear
308
30
Qwen3RMSNorm
113
9
LayerNorm
37
5
Qwen3Attention
28
2
Qwen3DecoderLayer
28
2
Qwen3MLP
28
2
SiLUActivation
28
2
GELUActivation
19
3
AudioAttention
18
2
AudioEncoderLayer
18
2
Conv2d
3
3
ModuleList
2
2
Embedding
1
1
Encoder
1
1
ForConditionalGeneration
1
1
Model
1
1
MultiModalProjector
1
1
Qwen3Model
1
1
Qwen3RotaryEmbedding
1
1
SinusoidsPositionEmbedding
1
1
Config changes
13 fields changed, 62 unchanged.
Field
Original
Fixture
audio_config.d_model
896
64
audio_config.encoder_attention_heads
14
4
audio_config.encoder_ffn_dim
3584
128
audio_config.encoder_layers
18
2
audio_config.num_key_value_heads
14
4
audio_config.output_dim
1024
64
text_config.head_dim
128
16
text_config.hidden_size
1024
64
text_config.intermediate_size
3072
128
text_config.layer_types
28 entries: 28× full_attention
["full_attention", "full_attention"]
text_config.num_attention_heads
16
4
text_config.num_hidden_layers
28
2
text_config.num_key_value_heads
8
2
+
\ No newline at end of file
diff --git a/shrink/Qwen3-Next-80B-A3B-Instruct-mic.html b/shrink/Qwen3-Next-80B-A3B-Instruct-mic.html
new file mode 100644
index 0000000000000000000000000000000000000000..9f73633c9e94019abd58e630c7b78afe76724c2c
--- /dev/null
+++ b/shrink/Qwen3-Next-80B-A3B-Instruct-mic.html
@@ -0,0 +1,34 @@
+
+
+Shrunk architecture
+
+
Shrunk architecture
huggingface-mic/Qwen3-Next-80B-A3B-Instruct-mic@69466af2a651a5263712f1430d3baf98b0e16463 ← Qwen/Qwen3-Next-80B-A3B-Instruct@9c7f2fbe84465e40164a94cc16cd30b6999b0cc7 · Qwen3NextForCausalLM · random weights · built before the shrink rule (no recorded recipe)
+
Original79.7Bparameters
Fixture20.6Mparameters
Layer stacks1compared
Findings0to review
✓ Same architecture, smaller
Every module class, layer kind and checked property of the original is in the fixture.
Layer stacks
One square per layer, colored by kind: the layer's module classes and type attributes. Hover a square for its index and kind.
model.layers
Original · 48
Fixture · 2
Kind
Original
Fixture
linear_attention, Conv1d, GatedDeltaNet …
36
1
✓ kept
is_causal=True, Attention
12
1
✓ kept
Where the parameters are
Share of each model's parameters per component. Vocabulary and patch sizes are kept, so embeddings usually take a larger share of the fixture.
Original
Fixture
Component
Original
Share
Fixture
Share
model.layers
79.1B
99.2%
1.2M
5.8%
lm_head
311.2M
0.4%
9.7M
47.1%
model.embed_tokens
311.2M
0.4%
9.7M
47.1%
model.norm
2.0K
0.0%
64
0.0%
Preserved properties
Config
Property
Original
Fixture
model
attention style
GQA (16 heads, 2 KV)
GQA (4 heads, 2 KV)
✓ kept
model
MoE routing
top-10 of 512 experts
top-2 of 4 experts
✓ kept
model
RoPE type
default
default
✓ kept
model
model_type
qwen3_next
qwen3_next
✓ kept
model
vocab_size
151936
151936
✓ kept
model
tie_word_embeddings
False
False
✓ kept
model
hidden_act
silu
silu
✓ kept
model
max_position_embeddings
262144
262144
✓ kept
Module classes
Instances of each class in the built model (prefix Qwen3Next omitted).
+
\ No newline at end of file
diff --git a/shrink/Qwen3-VL-2B-Instruct-mic.html b/shrink/Qwen3-VL-2B-Instruct-mic.html
new file mode 100644
index 0000000000000000000000000000000000000000..6b8c942f827dc941454bc337b615089f30c05ea0
--- /dev/null
+++ b/shrink/Qwen3-VL-2B-Instruct-mic.html
@@ -0,0 +1,34 @@
+
+
+Shrunk architecture
+
+
Shrunk architecture
huggingface-mic/Qwen3-VL-2B-Instruct-mic@7e045dfc87cda2240c98f5ae78e8bb377a08dc38 ← Qwen/Qwen3-VL-2B-Instruct@89644892e4d85e24eaac8bacfd4f463576704203 · Qwen3VLForConditionalGeneration · random weights · built before the shrink rule (no recorded recipe)
+
Original2.1Bparameters
Fixture11.2Mparameters
Layer stacks3compared
Findings0to review
✓ Same architecture, smaller
Every module class, layer kind and checked property of the original is in the fixture.
Layer stacks
One square per layer, colored by kind: the layer's module classes and type attributes. Hover a square for its index and kind.
model.visual.blocks
Original · 24
Fixture · 24
Kind
Original
Fixture
VisionBlock
24
24
✓ kept
model.visual.deepstack_merger_list
Original · 3
Fixture · 3
Kind
Original
Fixture
GELU
3
3
✓ kept
model.language_model.layers
Original · 28
Fixture · 2
Kind
Original
Fixture
TextDecoderLayer
28
2
✓ kept
Where the parameters are
Share of each model's parameters per component. Vocabulary and patch sizes are kept, so embeddings usually take a larger share of the fixture.
Original
Fixture
Component
Original
Share
Fixture
Share
model.language_model.layers
1.4B
66.2%
74.0K
0.7%
model.language_model
311.2M
14.6%
9.7M
87.0%
model.visual.blocks
302.3M
14.2%
803.3K
7.2%
model.visual.deepstack_merger_list
75.5M
3.6%
248.3K
2.2%
model.visual
29.1M
1.4%
328.2K
2.9%
Preserved properties
Config
Property
Original
Fixture
model
model_type
qwen3_vl
qwen3_vl
✓ kept
model
tie_word_embeddings
True
True
✓ kept
text_config
attention style
GQA (16 heads, 8 KV)
GQA (4 heads, 2 KV)
✓ kept
text_config
RoPE type
default
default
✓ kept
text_config
model_type
qwen3_vl_text
qwen3_vl_text
✓ kept
text_config
vocab_size
151936
151936
✓ kept
text_config
tie_word_embeddings
True
True
✓ kept
text_config
hidden_act
silu
silu
✓ kept
text_config
max_position_embeddings
262144
262144
✓ kept
vision_config
RoPE type
axial
axial
✓ kept
vision_config
model_type
qwen3_vl_vision
qwen3_vl_vision
✓ kept
vision_config
hidden_act
gelu_pytorch_tanh
gelu_pytorch_tanh
✓ kept
vision_config
patch_size
16
16
✓ kept
Module classes
Instances of each class in the built model (prefix Qwen3VL omitted).
+
\ No newline at end of file
diff --git a/shrink/Qwen3-VL-30B-A3B-Instruct-mic.html b/shrink/Qwen3-VL-30B-A3B-Instruct-mic.html
new file mode 100644
index 0000000000000000000000000000000000000000..eac4522f115731506fdb57e3bd45f21423c4665b
--- /dev/null
+++ b/shrink/Qwen3-VL-30B-A3B-Instruct-mic.html
@@ -0,0 +1,34 @@
+
+
+Shrunk architecture
+
+
Shrunk architecture
huggingface-mic/Qwen3-VL-30B-A3B-Instruct-mic@fce9290a8553fe0d943eb0e9c8e577bc5e7e3631 ← Qwen/Qwen3-VL-30B-A3B-Instruct@9c4b90e1e4ba969fd3b5378b57d966d725f1b86c · Qwen3VLMoeForConditionalGeneration · random weights · built before the shrink rule (no recorded recipe)
+
Original31.1Bparameters
Fixture21.0Mparameters
Layer stacks3compared
Findings0to review
✓ Same architecture, smaller
Every module class, layer kind and checked property of the original is in the fixture.
Layer stacks
One square per layer, colored by kind: the layer's module classes and type attributes. Hover a square for its index and kind.
model.visual.blocks
Original · 27
Fixture · 27
Kind
Original
Fixture
VisionBlock
27
27
✓ kept
model.visual.deepstack_merger_list
Original · 3
Fixture · 3
Kind
Original
Fixture
GELU
3
3
✓ kept
model.language_model.layers
Original · 48
Fixture · 2
Kind
Original
Fixture
TextDecoderLayer
48
2
✓ kept
Where the parameters are
Share of each model's parameters per component. Vocabulary and patch sizes are kept, so embeddings usually take a larger share of the fixture.
Original
Fixture
Component
Original
Share
Fixture
Share
model.language_model.layers
29.9B
96.3%
74.6K
0.4%
model.visual.blocks
411.5M
1.3%
903.7K
4.3%
model.language_model
311.2M
1.0%
9.7M
46.3%
lm_head
311.2M
1.0%
9.7M
46.3%
model.visual.deepstack_merger_list
92.1M
0.3%
248.3K
1.2%
model.visual
35.1M
0.1%
328.2K
1.6%
Preserved properties
Config
Property
Original
Fixture
model
model_type
qwen3_vl_moe
qwen3_vl_moe
✓ kept
model
tie_word_embeddings
False
False
✓ kept
text_config
attention style
GQA (32 heads, 4 KV)
GQA (4 heads, 2 KV)
✓ kept
text_config
MoE routing
top-8 of 128 experts
top-2 of 4 experts
✓ kept
text_config
RoPE type
default
default
✓ kept
text_config
model_type
qwen3_vl_moe_text
qwen3_vl_moe_text
✓ kept
text_config
vocab_size
151936
151936
✓ kept
text_config
tie_word_embeddings
True
True
✓ kept
text_config
hidden_act
silu
silu
✓ kept
text_config
max_position_embeddings
262144
262144
✓ kept
vision_config
RoPE type
axial
axial
✓ kept
vision_config
model_type
qwen3_vl_moe_vision
qwen3_vl_moe_vision
✓ kept
vision_config
hidden_act
gelu_pytorch_tanh
gelu_pytorch_tanh
✓ kept
vision_config
patch_size
16
16
✓ kept
Module classes
Instances of each class in the built model (prefix Qwen3VLMoe omitted).
+
\ No newline at end of file
diff --git a/shrink/Qwen3.5-35B-A3B-mic.html b/shrink/Qwen3.5-35B-A3B-mic.html
new file mode 100644
index 0000000000000000000000000000000000000000..a8cc190e6437096bce5ba05f628a0120c12ec6a3
--- /dev/null
+++ b/shrink/Qwen3.5-35B-A3B-mic.html
@@ -0,0 +1,34 @@
+
+
+Shrunk architecture
+
+
Shrunk architecture
huggingface-mic/Qwen3.5-35B-A3B-mic@8197a27e48d360625a51c9ddf03148b0e186f558 ← Qwen/Qwen3.5-35B-A3B@59d61f3ce65a6d9863b86d2e96597125219dc754 · Qwen3_5MoeForConditionalGeneration · random weights · built before the shrink rule (no recorded recipe)
+
Original35.1Bparameters
Fixture34.2Mparameters
Layer stacks2compared
Findings0to review
✓ Same architecture, smaller
Every module class, layer kind and checked property of the original is in the fixture.
Layer stacks
One square per layer, colored by kind: the layer's module classes and type attributes. Hover a square for its index and kind.
model.visual.blocks
Original · 27
Fixture · 27
Kind
Original
Fixture
VisionBlock
27
27
✓ kept
model.language_model.layers
Original · 40
Fixture · 2
Kind
Original
Fixture
linear_attention, Conv1d, GatedDeltaNet …
30
1
✓ kept
is_causal=True, Attention
10
1
✓ kept
Where the parameters are
Share of each model's parameters per component. Vocabulary and patch sizes are kept, so embeddings usually take a larger share of the fixture.
Original
Fixture
Component
Original
Share
Fixture
Share
model.language_model.layers
33.6B
95.8%
1.2M
3.5%
model.language_model
508.6M
1.4%
15.9M
46.4%
lm_head
508.6M
1.4%
15.9M
46.4%
model.visual.blocks
411.5M
1.2%
903.7K
2.6%
model.visual
35.1M
0.1%
328.2K
1.0%
Preserved properties
Config
Property
Original
Fixture
model
model_type
qwen3_5_moe
qwen3_5_moe
✓ kept
model
tie_word_embeddings
False
False
✓ kept
text_config
attention style
GQA (16 heads, 2 KV)
GQA (4 heads, 2 KV)
✓ kept
text_config
MoE routing
top-8 of 256 experts
top-2 of 4 experts
✓ kept
text_config
RoPE type
default
default
✓ kept
text_config
model_type
qwen3_5_moe_text
qwen3_5_moe_text
✓ kept
text_config
vocab_size
248320
248320
✓ kept
text_config
tie_word_embeddings
False
False
✓ kept
text_config
hidden_act
silu
silu
✓ kept
text_config
max_position_embeddings
262144
262144
✓ kept
vision_config
RoPE type
axial
axial
✓ kept
vision_config
model_type
qwen3_5_moe_vision
qwen3_5_moe_vision
✓ kept
vision_config
hidden_act
gelu_pytorch_tanh
gelu_pytorch_tanh
✓ kept
vision_config
patch_size
16
16
✓ kept
Module classes
Instances of each class in the built model (prefix Qwen3_5Moe omitted).
+
\ No newline at end of file
diff --git a/shrink/Seed-OSS-36B-Instruct-mic.html b/shrink/Seed-OSS-36B-Instruct-mic.html
new file mode 100644
index 0000000000000000000000000000000000000000..91deea27e15834e150d01acba6953c054c6c2b6e
--- /dev/null
+++ b/shrink/Seed-OSS-36B-Instruct-mic.html
@@ -0,0 +1,34 @@
+
+
+Shrunk architecture
+
+
Shrunk architecture
huggingface-mic/Seed-OSS-36B-Instruct-mic@5a7c613134e50c88bce20805ea4b02458007b844 ← ByteDance-Seed/Seed-OSS-36B-Instruct@497f1dca95ebdec98e41d517b9f060ee753c902f · SeedOssForCausalLM · random weights
+
Original36.2Bparameters
Fixture19.9Mparameters
Layer stacks1compared
Findings0to review
✓ Same architecture, smaller
Every module class, layer kind and checked property of the original is in the fixture.
Layer stacks
One square per layer, colored by kind: the layer's module classes and type attributes. Hover a square for its index and kind.
model.layers
Original · 64
Fixture · 2
Kind
Original
Fixture
DecoderLayer
64
2
✓ kept
Where the parameters are
Share of each model's parameters per component. Vocabulary and patch sizes are kept, so embeddings usually take a larger share of the fixture.
Original
Fixture
Component
Original
Share
Fixture
Share
model.layers
34.6B
95.6%
74.2K
0.4%
lm_head
794.3M
2.2%
9.9M
49.8%
model.embed_tokens
794.3M
2.2%
9.9M
49.8%
model.norm
5.1K
0.0%
64
0.0%
Preserved properties
Config
Property
Original
Fixture
model
attention style
GQA (80 heads, 8 KV)
GQA (4 heads, 2 KV)
✓ kept
model
RoPE type
default
default
✓ kept
model
model_type
seed_oss
seed_oss
✓ kept
model
vocab_size
155136
155136
✓ kept
model
tie_word_embeddings
False
False
✓ kept
model
hidden_act
silu
silu
✓ kept
model
max_position_embeddings
524288
524288
✓ kept
Module classes
Instances of each class in the built model (prefix SeedOss omitted).
Class
Original
Fixture
Linear
449
15
RMSNorm
129
5
Attention
64
2
DecoderLayer
64
2
MLP
64
2
SiLUActivation
64
2
Embedding
1
1
ModuleList
1
1
ForCausalLM
1
1
Model
1
1
RotaryEmbedding
1
1
Config changes
6 fields changed, 26 unchanged.
Field
Original
Fixture
head_dim
128
16
hidden_size
5120
64
intermediate_size
27648
128
num_attention_heads
80
4
num_hidden_layers
64
2
num_key_value_heads
8
2
+
\ No newline at end of file
diff --git a/shrink/SmolLM3-3B-mic.html b/shrink/SmolLM3-3B-mic.html
new file mode 100644
index 0000000000000000000000000000000000000000..c63f06f8f36abb976c3cef22ba5239aea3c44edf
--- /dev/null
+++ b/shrink/SmolLM3-3B-mic.html
@@ -0,0 +1,34 @@
+
+
+Shrunk architecture
+
+
Shrunk architecture
huggingface-mic/SmolLM3-3B-mic@717fbb55a2a1e454d1de0cd85e1c944183b0cbd7 ← HuggingFaceTB/SmolLM3-3B@a07cc9a04f16550a088caea529712d1d335b0ac1 · SmolLM3ForCausalLM · random weights · built before the shrink rule (no recorded recipe)
+
Original3.1Bparameters
Fixture35.2Mparameters
Layer stacks1compared
Findings1to review
⚠ To review
model: max_position_embeddings 65536 became 512
Layer stacks
One square per layer, colored by kind: the layer's module classes and type attributes. Hover a square for its index and kind.
model.layers
Original · 36
Fixture · 4
Kind
Original
Fixture
DecoderLayer
36
4
✓ kept
Where the parameters are
Share of each model's parameters per component. Vocabulary and patch sizes are kept, so embeddings usually take a larger share of the fixture.
Original
Fixture
Component
Original
Share
Fixture
Share
model.layers
2.8B
91.5%
2.4M
6.7%
model.embed_tokens
262.7M
8.5%
32.8M
93.3%
model.norm
2.0K
0.0%
256
0.0%
Preserved properties
Config
Property
Original
Fixture
model
attention style
GQA (16 heads, 4 KV)
GQA (4 heads, 2 KV)
✓ kept
model
RoPE type
default
default
✓ kept
model
model_type
smollm3
smollm3
✓ kept
model
vocab_size
128256
128256
✓ kept
model
tie_word_embeddings
True
True
✓ kept
model
hidden_act
silu
silu
✓ kept
model
max_position_embeddings
65536
512
⚠ changed
Module classes
Instances of each class in the built model (prefix SmolLM3 omitted).
+
\ No newline at end of file
diff --git a/shrink/VibeVoice-1.5B-hf-mic.html b/shrink/VibeVoice-1.5B-hf-mic.html
new file mode 100644
index 0000000000000000000000000000000000000000..fdf306855c6ed028a3044a9c867c32380a089a2d
--- /dev/null
+++ b/shrink/VibeVoice-1.5B-hf-mic.html
@@ -0,0 +1,34 @@
+
+
+Shrunk architecture
+
+
Shrunk architecture
huggingface-mic/VibeVoice-1.5B-hf-mic@a651225b62eeb6b3aa47c4f0600111b0055ab257 ← vibevoice/VibeVoice-1.5B-hf@d4cd82eb4371bf3e3d088e1ea7fa4eefb60449bf · VibeVoiceForConditionalGeneration · random weights
+
Original2.7Bparameters
Fixture268.3Mparameters
Layer stacks8compared
Findings0to review
✓ Same architecture, smaller
Every module class, layer kind and checked property of the original is in the fixture.
Layer stacks
One square per layer, colored by kind: the layer's module classes and type attributes. Hover a square for its index and kind.
model.audio_tower.encoder.stem.stage
Original · 3
Fixture · 1
Kind
Original
Fixture
AcousticTokenizerConvNext1dLayer
3
1
✓ kept
model.audio_tower.encoder.conv_layers
Original · 6
Fixture · 6
Kind
Original
Fixture
AcousticTokenizerEncoderLayer
6
6
✓ kept
model.audio_tower.decoder.stem.stage
Original · 8
Fixture · 1
Kind
Original
Fixture
AcousticTokenizerConvNext1dLayer
8
1
✓ kept
model.audio_tower.decoder.conv_layers
Original · 6
Fixture · 6
Kind
Original
Fixture
AcousticTokenizerDecoderLayer
6
6
✓ kept
model.language_model.layers
Original · 28
Fixture · 2
Kind
Original
Fixture
Qwen2DecoderLayer
28
2
✓ kept
model.semantic_tokenizer_encoder.stem.stage
Original · 3
Fixture · 1
Kind
Original
Fixture
AcousticTokenizerConvNext1dLayer
3
1
✓ kept
model.semantic_tokenizer_encoder.conv_layers
Original · 6
Fixture · 6
Kind
Original
Fixture
AcousticTokenizerEncoderLayer
6
6
✓ kept
model.diffusion_head.layers
Original · 4
Fixture · 2
Kind
Original
Fixture
Linear
4
2
✓ kept
Where the parameters are
Share of each model's parameters per component. Vocabulary and patch sizes are kept, so embeddings usually take a larger share of the fixture.
Original
Fixture
Component
Original
Share
Fixture
Share
model.language_model.layers
1.3B
48.5%
74.2K
0.0%
model.audio_tower.encoder.conv_layers
342.8M
12.7%
85.2M
31.8%
model.semantic_tokenizer_encoder.conv_layers
342.8M
12.7%
85.2M
31.8%
model.audio_tower.decoder.stem.stage
268.7M
9.9%
33.6M
12.5%
model.language_model
233.4M
8.6%
9.7M
3.6%
model.diffusion_head.layers
113.3M
4.2%
73.9K
0.0%
model.audio_tower.decoder.conv_layers
74.1M
2.7%
51.6M
19.2%
8 others
18.8M
0.7%
2.8M
1.1%
Preserved properties
Config
Property
Original
Fixture
model
model_type
vibevoice
vibevoice
✓ kept
model
vocab_size
151936
151936
✓ kept
model
tie_word_embeddings
True
True
✓ kept
audio_config
model_type
vibevoice_acoustic_tokenizer
vibevoice_acoustic_tokenizer
✓ kept
audio_config
hidden_act
gelu
gelu
✓ kept
semantic_model_config
model_type
vibevoice_acoustic_tokenizer_encoder
vibevoice_acoustic_tokenizer_encoder
✓ kept
semantic_model_config
hidden_act
gelu
gelu
✓ kept
text_config
attention style
GQA (12 heads, 2 KV)
GQA (4 heads, 2 KV)
✓ kept
text_config
RoPE type
default
default
✓ kept
text_config
model_type
qwen2
qwen2
✓ kept
text_config
vocab_size
151936
151936
✓ kept
text_config
tie_word_embeddings
True
True
✓ kept
text_config
hidden_act
silu
silu
✓ kept
text_config
max_position_embeddings
65536
65536
✓ kept
diffusion_head_config
model_type
vibevoice_diffusion
vibevoice_diffusion
✓ kept
diffusion_head_config
hidden_act
silu
silu
✓ kept
Module classes
Instances of each class in the built model (prefix VibeVoice omitted).
Class
Original
Fixture
Linear
379
75
AcousticTokenizerRMSNorm
156
42
Conv1d
96
39
AcousticTokenizerCausalConv1d
96
39
GELUActivation
78
21
AcousticTokenizerConvNext1dLayer
78
21
AcousticTokenizerFeedForward
78
21
Qwen2RMSNorm
57
5
SiLUActivation
38
8
Qwen2Attention
28
2
Qwen2DecoderLayer
28
2
Qwen2MLP
28
2
ModuleList
26
26
AcousticTokenizerEncoderLayer
12
12
ConvTranspose1d
6
6
AcousticTokenizerCausalConvTranspose1d
6
6
AcousticTokenizerDecoderLayer
6
6
RMSNorm
6
4
DiffusionHeadAdaLayerNorm
4
2
MLP
4
2
AcousticTokenizerEncoderModel
2
2
AcousticTokenizerEncoderStem
2
2
MultiModalProjector
2
2
Embedding
1
1
Qwen2Model
1
1
Qwen2RotaryEmbedding
1
1
AcousticTokenizerDecoderModel
1
1
AcousticTokenizerDecoderStem
1
1
AcousticTokenizerModel
1
1
DiffusionHead
1
1
DiffusionHeadFinalLayer
1
1
DiffusionHeadMLP
1
1
DiffusionHeadSinusoidalEmbedding
1
1
ForConditionalGeneration
1
1
Model
1
1
Config changes
11 fields changed, 98 unchanged.
Field
Original
Fixture
audio_config.depths
7 entries: 6× 3, 1× 8
7 entries: 7× 1
diffusion_head_config.hidden_size
1536
64
diffusion_head_config.intermediate_size
4608
128
diffusion_head_config.num_hidden_layers
4
2
semantic_model_config.depths
7 entries: 6× 3, 1× 8
7 entries: 7× 1
semantic_model_config.hidden_size
128
64
text_config.hidden_size
1536
64
text_config.intermediate_size
8960
128
text_config.layer_types
28 entries: 28× full_attention
["full_attention", "full_attention"]
text_config.num_attention_heads
12
4
text_config.num_hidden_layers
28
2
+
\ No newline at end of file
diff --git a/shrink/Voxtral-Mini-3B-2507-mic.html b/shrink/Voxtral-Mini-3B-2507-mic.html
new file mode 100644
index 0000000000000000000000000000000000000000..6bedf6367706cb32ddc4ac87c066572cfaf2c034
--- /dev/null
+++ b/shrink/Voxtral-Mini-3B-2507-mic.html
@@ -0,0 +1,34 @@
+
+
+Shrunk architecture
+
+
Shrunk architecture
huggingface-mic/Voxtral-Mini-3B-2507-mic@88954b6754a54ae7b39c94a53b59d3d54313fae9 ← mistralai/Voxtral-Mini-3B-2507@3060fe34b35ba5d44202ce9ff3c097642914f8f3 · VoxtralForConditionalGeneration · random weights
+
Original4.7Bparameters
Fixture17.1Mparameters
Layer stacks2compared
Findings0to review
✓ Same architecture, smaller
Every module class, layer kind and checked property of the original is in the fixture.
Layer stacks
One square per layer, colored by kind: the layer's module classes and type attributes. Hover a square for its index and kind.
model.audio_tower.layers
Original · 32
Fixture · 2
Kind
Original
Fixture
EncoderLayer
32
2
✓ kept
model.language_model.layers
Original · 30
Fixture · 2
Kind
Original
Fixture
LlamaDecoderLayer
30
2
✓ kept
Where the parameters are
Share of each model's parameters per component. Vocabulary and patch sizes are kept, so embeddings usually take a larger share of the fixture.
Original
Fixture
Component
Original
Share
Fixture
Share
model.language_model.layers
3.2B
68.6%
74.0K
0.4%
model.audio_tower.layers
629.6M
13.5%
99.8K
0.6%
model.language_model
402.7M
8.6%
8.4M
49.0%
lm_head
402.7M
8.6%
8.4M
49.0%
model.multi_modal_projector
25.2M
0.5%
20.5K
0.1%
model.audio_tower
7.3M
0.2%
133.1K
0.8%
Preserved properties
Config
Property
Original
Fixture
model
model_type
voxtral
voxtral
✓ kept
model
vocab_size
131072
131072
✓ kept
audio_config
attention style
MHA (20 heads, 20 KV)
MHA (4 heads, 4 KV)
✓ kept
audio_config
model_type
voxtral_encoder
voxtral_encoder
✓ kept
audio_config
vocab_size
51866
51866
✓ kept
audio_config
num_mel_bins
128
128
✓ kept
text_config
attention style
GQA (32 heads, 8 KV)
GQA (4 heads, 2 KV)
✓ kept
text_config
RoPE type
default
default
✓ kept
text_config
model_type
llama
llama
✓ kept
text_config
vocab_size
131072
131072
✓ kept
text_config
tie_word_embeddings
False
False
✓ kept
text_config
hidden_act
silu
silu
✓ kept
text_config
max_position_embeddings
131072
131072
✓ kept
Module classes
Instances of each class in the built model (prefix Voxtral omitted).
Class
Original
Fixture
Linear
405
29
LayerNorm
65
5
LlamaRMSNorm
61
5
GELUActivation
33
3
Attention
32
2
EncoderLayer
32
2
LlamaAttention
30
2
LlamaDecoderLayer
30
2
LlamaMLP
30
2
SiLUActivation
30
2
Conv1d
2
2
Embedding
2
2
ModuleList
2
2
AvgPool1d
1
1
LlamaModel
1
1
LlamaRotaryEmbedding
1
1
Encoder
1
1
ForConditionalGeneration
1
1
Model
1
1
MultiModalProjector
1
1
Config changes
13 fields changed, 56 unchanged.
Field
Original
Fixture
audio_config.head_dim
64
16
audio_config.hidden_size
1280
64
audio_config.intermediate_size
5120
256
audio_config.num_attention_heads
20
4
audio_config.num_hidden_layers
32
2
audio_config.num_key_value_heads
20
4
hidden_size
3072
64
text_config.head_dim
128
16
text_config.hidden_size
3072
64
text_config.intermediate_size
8192
128
text_config.num_attention_heads
32
4
text_config.num_hidden_layers
30
2
text_config.num_key_value_heads
8
2
+
\ No newline at end of file
diff --git a/shrink/albert-base-v2-mic.html b/shrink/albert-base-v2-mic.html
new file mode 100644
index 0000000000000000000000000000000000000000..d49bb5e130b7d36a9f9b9f6d9dcef8a3351d8832
--- /dev/null
+++ b/shrink/albert-base-v2-mic.html
@@ -0,0 +1,34 @@
+
+
+Shrunk architecture
+
+
Shrunk architecture
huggingface-mic/albert-base-v2-mic@258e9f08fe272ff4b05d6823ad031e0ba86d9be6 ← albert/albert-base-v2@8e2f239c5f8a2c0f253781ca60135db913e5c80c · AlbertForMaskedLM · random weights
+
Original11.2Mparameters
Fixture4.0Mparameters
Layer stacks1compared
Findings0to review
✓ Same architecture, smaller
Every module class, layer kind and checked property of the original is in the fixture.
Layer stacks
One square per layer, colored by kind: the layer's module classes and type attributes. Hover a square for its index and kind.
albert.encoder.albert_layer_groups
Original · 1
Fixture · 1
Kind
Original
Fixture
Layer
1
1
✓ kept
Where the parameters are
Share of each model's parameters per component. Vocabulary and patch sizes are kept, so embeddings usually take a larger share of the fixture.
Original
Fixture
Component
Original
Share
Fixture
Share
albert.encoder.albert_layer_groups
7.1M
63.2%
33.5K
0.8%
albert.embeddings
3.9M
34.8%
3.9M
98.0%
albert.encoder
99.1K
0.9%
8.3K
0.2%
predictions.dense
98.4K
0.9%
8.3K
0.2%
predictions
30.0K
0.3%
30.0K
0.8%
predictions.LayerNorm
256
0.0%
256
0.0%
Preserved properties
Config
Property
Original
Fixture
model
model_type
albert
albert
✓ kept
model
vocab_size
30000
30000
✓ kept
model
tie_word_embeddings
True
True
✓ kept
model
hidden_act
gelu_new
gelu_new
✓ kept
model
max_position_embeddings
512
512
✓ kept
Module classes
Instances of each class in the built model (prefix Albert omitted).
Class
Original
Fixture
Linear
9
9
Dropout
4
4
LayerNorm
4
4
Embedding
3
3
ModuleList
2
2
NewGELUActivation
2
2
Attention
1
1
Embeddings
1
1
ForMaskedLM
1
1
Layer
1
1
LayerGroup
1
1
MLMHead
1
1
Model
1
1
Transformer
1
1
Config changes
4 fields changed, 29 unchanged.
Field
Original
Fixture
hidden_size
768
64
intermediate_size
3072
128
num_attention_heads
12
4
num_hidden_layers
12
2
+
\ No newline at end of file
diff --git a/shrink/audio-flamingo-3-hf-mic.html b/shrink/audio-flamingo-3-hf-mic.html
new file mode 100644
index 0000000000000000000000000000000000000000..e768ebcb5bab6d114b2a3b36dde26de304b1dd8b
--- /dev/null
+++ b/shrink/audio-flamingo-3-hf-mic.html
@@ -0,0 +1,34 @@
+
+
+Shrunk architecture
+
+
Shrunk architecture
huggingface-mic/audio-flamingo-3-hf-mic@5e785e0943d8a895cad2acdfc3d7a4366f961920 ← nvidia/audio-flamingo-3-hf@7d4bae64ee29878af6504ae6f6bb3e40492838ad · AudioFlamingo3ForConditionalGeneration · random weights
+
Original8.3Bparameters
Fixture19.7Mparameters
Layer stacks2compared
Findings0to review
✓ Same architecture, smaller
Every module class, layer kind and checked property of the original is in the fixture.
Layer stacks
One square per layer, colored by kind: the layer's module classes and type attributes. Hover a square for its index and kind.
model.audio_tower.layers
Original · 32
Fixture · 2
Kind
Original
Fixture
EncoderLayer
32
2
✓ kept
model.language_model.layers
Original · 28
Fixture · 2
Kind
Original
Fixture
Qwen2DecoderLayer
28
2
✓ kept
Where the parameters are
Share of each model's parameters per component. Vocabulary and patch sizes are kept, so embeddings usually take a larger share of the fixture.
Original
Fixture
Component
Original
Share
Fixture
Share
model.language_model.layers
6.5B
78.9%
74.2K
0.4%
model.audio_tower.layers
629.6M
7.6%
66.8K
0.3%
model.language_model
543.6M
6.6%
9.7M
49.3%
lm_head
543.6M
6.6%
9.7M
49.3%
model.multi_modal_projector
17.4M
0.2%
8.3K
0.0%
model.audio_tower
7.3M
0.1%
133.1K
0.7%
Preserved properties
Config
Property
Original
Fixture
model
model_type
audioflamingo3
audioflamingo3
✓ kept
audio_config
model_type
audioflamingo3_encoder
audioflamingo3_encoder
✓ kept
audio_config
num_mel_bins
128
128
✓ kept
text_config
attention style
GQA (28 heads, 4 KV)
GQA (4 heads, 2 KV)
✓ kept
text_config
RoPE type
default
default
✓ kept
text_config
model_type
qwen2
qwen2
✓ kept
text_config
vocab_size
151672
151672
✓ kept
text_config
tie_word_embeddings
False
False
✓ kept
text_config
hidden_act
silu
silu
✓ kept
text_config
max_position_embeddings
32768
32768
✓ kept
Module classes
Instances of each class in the built model (prefix AudioFlamingo3 omitted).
Class
Original
Fixture
Linear
391
29
LayerNorm
65
5
Qwen2RMSNorm
57
5
GELUActivation
33
3
Attention
32
2
EncoderLayer
32
2
Qwen2Attention
28
2
Qwen2DecoderLayer
28
2
Qwen2MLP
28
2
SiLUActivation
28
2
Conv1d
2
2
Embedding
2
2
ModuleList
2
2
Encoder
1
1
ForConditionalGeneration
1
1
Model
1
1
MultiModalProjector
1
1
AvgPool1d
1
1
Qwen2Model
1
1
Qwen2RotaryEmbedding
1
1
Config changes
10 fields changed, 55 unchanged.
Field
Original
Fixture
audio_config.hidden_size
1280
64
audio_config.intermediate_size
5120
128
audio_config.num_attention_heads
20
4
audio_config.num_hidden_layers
32
2
text_config.hidden_size
3584
64
text_config.intermediate_size
18944
128
text_config.layer_types
28 entries: 28× full_attention
["full_attention", "full_attention"]
text_config.num_attention_heads
28
4
text_config.num_hidden_layers
28
2
text_config.num_key_value_heads
4
2
+
\ No newline at end of file
diff --git a/shrink/aya-expanse-8b-ungated-mic.html b/shrink/aya-expanse-8b-ungated-mic.html
new file mode 100644
index 0000000000000000000000000000000000000000..c3bb972f76e54b54b79db8eaff1c840aae963d94
--- /dev/null
+++ b/shrink/aya-expanse-8b-ungated-mic.html
@@ -0,0 +1,34 @@
+
+
+Shrunk architecture
+
+
Shrunk architecture
huggingface-mic/aya-expanse-8b-ungated-mic@ed87e2b6202ba7f881df3e5b8a9859126103bcec ← adamo1139/aya-expanse-8b-ungated@871465ab4030a036a24433b5d100d640f5142999 · CohereForCausalLM · random weights
+
Original8.0Bparameters
Fixture16.5Mparameters
Layer stacks1compared
Findings0to review
✓ Same architecture, smaller
Every module class, layer kind and checked property of the original is in the fixture.
Layer stacks
One square per layer, colored by kind: the layer's module classes and type attributes. Hover a square for its index and kind.
model.layers
Original · 32
Fixture · 2
Kind
Original
Fixture
DecoderLayer
32
2
✓ kept
Where the parameters are
Share of each model's parameters per component. Vocabulary and patch sizes are kept, so embeddings usually take a larger share of the fixture.
Original
Fixture
Component
Original
Share
Fixture
Share
model.layers
7.0B
86.9%
73.9K
0.4%
model.embed_tokens
1.0B
13.1%
16.4M
99.6%
model.norm
4.1K
0.0%
64
0.0%
Preserved properties
Config
Property
Original
Fixture
model
attention style
GQA (32 heads, 8 KV)
GQA (4 heads, 2 KV)
✓ kept
model
RoPE type
default
default
✓ kept
model
model_type
cohere
cohere
✓ kept
model
vocab_size
256000
256000
✓ kept
model
tie_word_embeddings
True
True
✓ kept
model
hidden_act
silu
silu
✓ kept
model
max_position_embeddings
8192
8192
✓ kept
Module classes
Instances of each class in the built model (prefix Cohere omitted).
Class
Original
Fixture
Linear
225
15
LayerNorm
33
3
Attention
32
2
DecoderLayer
32
2
MLP
32
2
SiLUActivation
32
2
ForCausalLM
1
1
Model
1
1
RotaryEmbedding
1
1
Embedding
1
1
ModuleList
1
1
Config changes
5 fields changed, 24 unchanged.
Field
Original
Fixture
hidden_size
4096
64
intermediate_size
14336
128
num_attention_heads
32
4
num_hidden_layers
32
2
num_key_value_heads
8
2
+
\ No newline at end of file
diff --git a/shrink/bart-base-mic.html b/shrink/bart-base-mic.html
new file mode 100644
index 0000000000000000000000000000000000000000..acb846b251acf6642693a1b1d6b51102123a05e7
--- /dev/null
+++ b/shrink/bart-base-mic.html
@@ -0,0 +1,34 @@
+
+
+Shrunk architecture
+
+
Shrunk architecture
huggingface-mic/bart-base-mic@1d300e47ad358f5882f422d8c0cfdbc62d975c7b ← facebook/bart-base@aadd2ab0ae0c8268c7c9693540e9904811f36177 · BartForConditionalGeneration · overfit weights · built before the shrink rule (no recorded recipe)
+
Original139.4Mparameters
Fixture3.5Mparameters
Layer stacks2compared
Findings0to review
✓ Same architecture, smaller
Every module class, layer kind and checked property of the original is in the fixture.
Layer stacks
One square per layer, colored by kind: the layer's module classes and type attributes. Hover a square for its index and kind.
model.encoder.layers
Original · 6
Fixture · 2
Kind
Original
Fixture
EncoderLayer
6
2
✓ kept
model.decoder.layers
Original · 6
Fixture · 2
Kind
Original
Fixture
DecoderLayer
6
2
✓ kept
Where the parameters are
Share of each model's parameters per component. Vocabulary and patch sizes are kept, so embeddings usually take a larger share of the fixture.
Original
Fixture
Component
Original
Share
Fixture
Share
model.decoder.layers
56.7M
40.7%
100.5K
2.9%
model.encoder.layers
42.5M
30.5%
66.9K
1.9%
model.shared
38.6M
27.7%
3.2M
91.5%
model.encoder
789.5K
0.6%
65.8K
1.9%
model.decoder
789.5K
0.6%
65.8K
1.9%
Preserved properties
Config
Property
Original
Fixture
model
model_type
bart
bart
✓ kept
model
vocab_size
50265
50265
✓ kept
model
tie_word_embeddings
True
True
✓ kept
model
max_position_embeddings
1024
1024
✓ kept
Module classes
Instances of each class in the built model (prefix Bart omitted).
Class
Original
Fixture
Linear
97
33
LayerNorm
32
12
Attention
18
6
GELUActivation
12
4
DecoderLayer
6
2
EncoderLayer
6
2
ScaledWordEmbedding
3
3
LearnedPositionalEmbedding
2
2
ModuleList
2
2
Decoder
1
1
Encoder
1
1
ForConditionalGeneration
1
1
Model
1
1
Config changes
7 fields changed, 35 unchanged.
Field
Original
Fixture
d_model
768
64
decoder_attention_heads
12
4
decoder_ffn_dim
3072
128
decoder_layers
6
2
encoder_attention_heads
12
4
encoder_ffn_dim
3072
128
encoder_layers
6
2
+
\ No newline at end of file
diff --git a/shrink/beit-base-patch16-224-mic.html b/shrink/beit-base-patch16-224-mic.html
new file mode 100644
index 0000000000000000000000000000000000000000..f193860c8172173566e2a351907a1ff34639c50b
--- /dev/null
+++ b/shrink/beit-base-patch16-224-mic.html
@@ -0,0 +1,34 @@
+
+
+Shrunk architecture
+
+
Shrunk architecture
huggingface-mic/beit-base-patch16-224-mic@1a760472d6b5a9ada187942e594ec0220135fd38 ← microsoft/beit-base-patch16-224@f02e8f77db4703e3fbd3766e3375a4619c5a4863 · BeitForImageClassification · random weights
+
Original86.5Mparameters
Fixture187.3Kparameters
Layer stacks1compared
Findings0to review
✓ Same architecture, smaller
Every module class, layer kind and checked property of the original is in the fixture.
Layer stacks
One square per layer, colored by kind: the layer's module classes and type attributes. Hover a square for its index and kind.
beit.layers
Original · 12
Fixture · 2
Kind
Original
Fixture
Layer
12
2
✓ kept
Where the parameters are
Share of each model's parameters per component. Vocabulary and patch sizes are kept, so embeddings usually take a larger share of the fixture.
Original
Fixture
Component
Original
Share
Fixture
Share
beit.layers
85.2M
98.4%
72.9K
38.9%
classifier
769.0K
0.9%
65.0K
34.7%
beit.embeddings
591.4K
0.7%
49.3K
26.3%
beit.pooler
1.5K
0.0%
128
0.1%
Preserved properties
Config
Property
Original
Fixture
model
model_type
beit
beit
✓ kept
model
vocab_size
8192
8192
✓ kept
model
hidden_act
gelu
gelu
✓ kept
model
image_size
224
224
✓ kept
model
patch_size
16
16
✓ kept
model
num_channels
3
3
✓ kept
Module classes
Instances of each class in the built model (prefix Beit omitted).
Class
Original
Fixture
Linear
73
13
LayerNorm
25
5
Dropout
13
3
Attention
12
2
Layer
12
2
MLP
12
2
RelativePositionBias
12
2
GELUActivation
12
2
DropPath
11
1
Identity
2
2
Embeddings
1
1
ForImageClassification
1
1
Model
1
1
PatchEmbeddings
1
1
Pooler
1
1
Conv2d
1
1
ModuleList
1
1
Config changes
6 fields changed, 35 unchanged.
Field
Original
Fixture
hidden_size
768
64
intermediate_size
3072
128
num_attention_heads
12
4
num_hidden_layers
12
2
out_features
["stage12"]
["stage2"]
out_indices
[12]
[2]
+
\ No newline at end of file
diff --git a/shrink/bert-base-uncased-mic.html b/shrink/bert-base-uncased-mic.html
new file mode 100644
index 0000000000000000000000000000000000000000..a12e79d2fe1ce99fa5f1604df098a9626f2003a8
--- /dev/null
+++ b/shrink/bert-base-uncased-mic.html
@@ -0,0 +1,34 @@
+
+
+Shrunk architecture
+
+
Shrunk architecture
huggingface-mic/bert-base-uncased-mic@b22d1695360ee0c7764b138cc9f00107ed1c3b8a ← google-bert/bert-base-uncased@86b5e0934494bd15c9632b12f734a8a67f723594 · BertForMaskedLM · random weights · built before the shrink rule (no recorded recipe)
+
Original109.5Mparameters
Fixture2.1Mparameters
Layer stacks1compared
Findings0to review
✓ Same architecture, smaller
Every module class, layer kind and checked property of the original is in the fixture.
Layer stacks
One square per layer, colored by kind: the layer's module classes and type attributes. Hover a square for its index and kind.
bert.encoder.layer
Original · 12
Fixture · 2
Kind
Original
Fixture
Layer
12
2
✓ kept
Where the parameters are
Share of each model's parameters per component. Vocabulary and patch sizes are kept, so embeddings usually take a larger share of the fixture.
Original
Fixture
Component
Original
Share
Fixture
Share
bert.encoder.layer
85.1M
77.7%
66.9K
3.2%
bert.embeddings
23.8M
21.8%
2.0M
95.1%
cls.predictions
622.6K
0.6%
34.8K
1.7%
Preserved properties
Config
Property
Original
Fixture
model
model_type
bert
bert
✓ kept
model
vocab_size
30522
30522
✓ kept
model
tie_word_embeddings
True
True
✓ kept
model
hidden_act
gelu
gelu
✓ kept
model
max_position_embeddings
512
512
✓ kept
Module classes
Instances of each class in the built model (prefix Bert omitted).
Class
Original
Fixture
Linear
74
14
Dropout
37
7
LayerNorm
26
6
GELUActivation
13
3
Attention
12
2
Intermediate
12
2
Layer
12
2
Output
12
2
SelfAttention
12
2
SelfOutput
12
2
Embedding
3
3
Embeddings
1
1
Encoder
1
1
ForMaskedLM
1
1
LMPredictionHead
1
1
Model
1
1
OnlyMLMHead
1
1
PredictionHeadTransform
1
1
ModuleList
1
1
Config changes
4 fields changed, 27 unchanged.
Field
Original
Fixture
hidden_size
768
64
intermediate_size
3072
128
num_attention_heads
12
4
num_hidden_layers
12
2
+
\ No newline at end of file
diff --git a/shrink/bert-mic.html b/shrink/bert-mic.html
new file mode 100644
index 0000000000000000000000000000000000000000..4ff4eb2b7ae482fd18e90c558a2d08237f664ba3
--- /dev/null
+++ b/shrink/bert-mic.html
@@ -0,0 +1,34 @@
+
+
+Shrunk architecture
+
+
Shrunk architecture
huggingface-mic/bert-mic@55802f5bb3e27f6c6effb21b38bbcf867d1a0cdd ← google-bert/bert-base-uncased@86b5e0934494bd15c9632b12f734a8a67f723594 · BertForMaskedLM · random weights · built before the shrink rule (no recorded recipe)
+
Original109.5Mparameters
Fixture2.1Mparameters
Layer stacks1compared
Findings0to review
✓ Same architecture, smaller
Every module class, layer kind and checked property of the original is in the fixture.
Layer stacks
One square per layer, colored by kind: the layer's module classes and type attributes. Hover a square for its index and kind.
bert.encoder.layer
Original · 12
Fixture · 2
Kind
Original
Fixture
Layer
12
2
✓ kept
Where the parameters are
Share of each model's parameters per component. Vocabulary and patch sizes are kept, so embeddings usually take a larger share of the fixture.
Original
Fixture
Component
Original
Share
Fixture
Share
bert.encoder.layer
85.1M
77.7%
66.9K
3.2%
bert.embeddings
23.8M
21.8%
2.0M
95.1%
cls.predictions
622.6K
0.6%
34.8K
1.7%
Preserved properties
Config
Property
Original
Fixture
model
model_type
bert
bert
✓ kept
model
vocab_size
30522
30522
✓ kept
model
tie_word_embeddings
True
True
✓ kept
model
hidden_act
gelu
gelu
✓ kept
model
max_position_embeddings
512
512
✓ kept
Module classes
Instances of each class in the built model (prefix Bert omitted).
Class
Original
Fixture
Linear
74
14
Dropout
37
7
LayerNorm
26
6
GELUActivation
13
3
Attention
12
2
Intermediate
12
2
Layer
12
2
Output
12
2
SelfAttention
12
2
SelfOutput
12
2
Embedding
3
3
Embeddings
1
1
Encoder
1
1
ForMaskedLM
1
1
LMPredictionHead
1
1
Model
1
1
OnlyMLMHead
1
1
PredictionHeadTransform
1
1
ModuleList
1
1
Config changes
4 fields changed, 27 unchanged.
Field
Original
Fixture
hidden_size
768
64
intermediate_size
3072
128
num_attention_heads
12
4
num_hidden_layers
12
2
+
\ No newline at end of file
diff --git a/shrink/bert2bert-cnn_dailymail-fp16-mic.html b/shrink/bert2bert-cnn_dailymail-fp16-mic.html
new file mode 100644
index 0000000000000000000000000000000000000000..93bae9ffe536e81fdfb42ab89e68bdae4bcbd8a7
--- /dev/null
+++ b/shrink/bert2bert-cnn_dailymail-fp16-mic.html
@@ -0,0 +1,34 @@
+
+
+Shrunk architecture
+
+
Shrunk architecture
huggingface-mic/bert2bert-cnn_dailymail-fp16-mic@4dc2839f1cfebb971828d84d932fed961be06b92 ← patrickvonplaten/bert2bert-cnn_dailymail-fp16@51b5d5cac0fa0ed09ed505df5800579996a2fe12 · EncoderDecoderModel · random weights · built before the shrink rule (no recorded recipe)
+
Original247.4Mparameters
Fixture4.2Mparameters
Layer stacks2compared
Findings0to review
✓ Same architecture, smaller
Every module class, layer kind and checked property of the original is in the fixture.
Layer stacks
One square per layer, colored by kind: the layer's module classes and type attributes. Hover a square for its index and kind.
encoder.encoder.layer
Original · 12
Fixture · 2
Kind
Original
Fixture
BertLayer
12
2
✓ kept
decoder.bert.encoder.layer
Original · 12
Fixture · 2
Kind
Original
Fixture
BertLayer
12
2
✓ kept
Where the parameters are
Share of each model's parameters per component. Vocabulary and patch sizes are kept, so embeddings usually take a larger share of the fixture.
Original
Fixture
Component
Original
Share
Fixture
Share
decoder.bert.encoder.layer
113.4M
45.9%
100.5K
2.4%
encoder.encoder.layer
85.1M
34.4%
66.9K
1.6%
decoder.bert
23.8M
9.6%
2.0M
47.5%
encoder.embeddings
23.8M
9.6%
2.0M
47.5%
decoder.cls
622.6K
0.3%
34.8K
0.8%
encoder.pooler
590.6K
0.2%
4.2K
0.1%
Preserved properties
Config
Property
Original
Fixture
model
model_type
encoder-decoder
encoder-decoder
✓ kept
encoder
model_type
bert
bert
✓ kept
encoder
vocab_size
30522
30522
✓ kept
encoder
tie_word_embeddings
True
True
✓ kept
encoder
hidden_act
gelu
gelu
✓ kept
encoder
max_position_embeddings
512
512
✓ kept
decoder
model_type
bert
bert
✓ kept
decoder
vocab_size
30522
30522
✓ kept
decoder
tie_word_embeddings
True
True
✓ kept
decoder
hidden_act
gelu
gelu
✓ kept
decoder
max_position_embeddings
512
512
✓ kept
Module classes
Instances of each class in the built model (prefix EncoderDecoder omitted).
Class
Original
Fixture
Linear
195
35
Dropout
98
18
LayerNorm
63
13
BertAttention
36
6
BertSelfOutput
36
6
GELUActivation
25
5
BertIntermediate
24
4
BertLayer
24
4
BertOutput
24
4
BertSelfAttention
24
4
BertCrossAttention
12
2
Embedding
6
6
BertEmbeddings
2
2
BertEncoder
2
2
BertModel
2
2
ModuleList
2
2
BertLMHeadModel
1
1
BertLMPredictionHead
1
1
BertOnlyMLMHead
1
1
BertPooler
1
1
BertPredictionHeadTransform
1
1
Model
1
1
Tanh
1
1
Config changes
9 fields changed, 79 unchanged.
Field
Original
Fixture
decoder.hidden_size
768
64
decoder.intermediate_size
3072
128
decoder.num_attention_heads
12
4
decoder.num_hidden_layers
12
2
encoder.hidden_size
768
64
encoder.intermediate_size
3072
128
encoder.num_attention_heads
12
4
encoder.num_hidden_layers
12
2
pad_token_id
null
0
+
\ No newline at end of file
diff --git a/shrink/bigbird-roberta-base-mic.html b/shrink/bigbird-roberta-base-mic.html
new file mode 100644
index 0000000000000000000000000000000000000000..c9ee6fcd19e38b51cffb5fb762b75a916eec1ed4
--- /dev/null
+++ b/shrink/bigbird-roberta-base-mic.html
@@ -0,0 +1,34 @@
+
+
+Shrunk architecture
+
+
Shrunk architecture
huggingface-mic/bigbird-roberta-base-mic@5c8d7c34dc917be7de46595c2fc895c9d25f8d2b ← google/bigbird-roberta-base@5a145f7852cba9bd431386a58137bf8a29903b90 · BigBirdForPreTraining · random weights
+
Original128.1Mparameters
Fixture3.6Mparameters
Layer stacks1compared
Findings0to review
✓ Same architecture, smaller
Every module class, layer kind and checked property of the original is in the fixture.
Layer stacks
One square per layer, colored by kind: the layer's module classes and type attributes. Hover a square for its index and kind.
bert.encoder.layer
Original · 12
Fixture · 2
Kind
Original
Fixture
Layer
12
2
✓ kept
Where the parameters are
Share of each model's parameters per component. Vocabulary and patch sizes are kept, so embeddings usually take a larger share of the fixture.
Original
Fixture
Component
Original
Share
Fixture
Share
bert.encoder.layer
85.1M
66.4%
66.9K
1.9%
bert.embeddings
41.8M
32.6%
3.5M
96.5%
cls.predictions
642.5K
0.5%
54.6K
1.5%
bert.pooler
590.6K
0.5%
4.2K
0.1%
cls.seq_relationship
1.5K
0.0%
130
0.0%
Preserved properties
Config
Property
Original
Fixture
model
model_type
big_bird
big_bird
✓ kept
model
vocab_size
50358
50358
✓ kept
model
tie_word_embeddings
True
True
✓ kept
model
hidden_act
gelu_new
gelu_new
✓ kept
model
max_position_embeddings
4096
4096
✓ kept
Module classes
Instances of each class in the built model (prefix BigBird omitted).
Class
Original
Fixture
Linear
76
16
LayerNorm
26
6
Dropout
25
5
NewGELUActivation
13
3
Attention
12
2
BlockSparseAttention
12
2
Intermediate
12
2
Layer
12
2
Output
12
2
SelfOutput
12
2
Embedding
3
3
Embeddings
1
1
Encoder
1
1
ForPreTraining
1
1
LMPredictionHead
1
1
Model
1
1
PreTrainingHeads
1
1
PredictionHeadTransform
1
1
ModuleList
1
1
Tanh
1
1
Config changes
4 fields changed, 33 unchanged.
Field
Original
Fixture
hidden_size
768
64
intermediate_size
3072
128
num_attention_heads
12
4
num_hidden_layers
12
2
+
\ No newline at end of file
diff --git a/shrink/blenderbot-400M-distill-mic.html b/shrink/blenderbot-400M-distill-mic.html
new file mode 100644
index 0000000000000000000000000000000000000000..a72a13c65dee217f0a40514b880694dba8f8bf5b
--- /dev/null
+++ b/shrink/blenderbot-400M-distill-mic.html
@@ -0,0 +1,34 @@
+
+
+Shrunk architecture
+
+
Shrunk architecture
huggingface-mic/blenderbot-400M-distill-mic@3971683a079bd80db6eb44a0aa2d427b8d53d030 ← facebook/blenderbot-400M-distill@eaaf64e3be20ad1f1fb0bdf689565ba52c97eafe · BlenderbotForConditionalGeneration · random weights
+
Original364.8Mparameters
Fixture696.6Kparameters
Layer stacks2compared
Findings0to review
✓ Same architecture, smaller
Every module class, layer kind and checked property of the original is in the fixture.
Layer stacks
One square per layer, colored by kind: the layer's module classes and type attributes. Hover a square for its index and kind.
model.encoder.layers
Original · 2
Fixture · 2
Kind
Original
Fixture
EncoderLayer
2
2
✓ kept
model.decoder.layers
Original · 12
Fixture · 2
Kind
Original
Fixture
DecoderLayer
12
2
✓ kept
Where the parameters are
Share of each model's parameters per component. Vocabulary and patch sizes are kept, so embeddings usually take a larger share of the fixture.
Original
Fixture
Component
Original
Share
Fixture
Share
model.decoder.layers
314.9M
86.3%
100.5K
14.4%
model.encoder.layers
39.4M
10.8%
66.9K
9.6%
model.shared
10.3M
2.8%
512.5K
73.6%
model.decoder
166.4K
0.0%
8.3K
1.2%
model.encoder
166.4K
0.0%
8.3K
1.2%
Preserved properties
Config
Property
Original
Fixture
model
model_type
blenderbot
blenderbot
✓ kept
model
vocab_size
8008
8008
✓ kept
model
tie_word_embeddings
True
True
✓ kept
model
max_position_embeddings
128
128
✓ kept
Module classes
Instances of each class in the built model (prefix Blenderbot omitted).
Class
Original
Fixture
Linear
133
33
LayerNorm
42
12
Attention
26
6
GELUActivation
14
4
DecoderLayer
12
2
ScaledWordEmbedding
3
3
EncoderLayer
2
2
LearnedPositionalEmbedding
2
2
ModuleList
2
2
Decoder
1
1
Encoder
1
1
ForConditionalGeneration
1
1
Model
1
1
Config changes
6 fields changed, 43 unchanged.
Field
Original
Fixture
d_model
1280
64
decoder_attention_heads
32
4
decoder_ffn_dim
5120
128
decoder_layers
12
2
encoder_attention_heads
32
4
encoder_ffn_dim
5120
128
+
\ No newline at end of file
diff --git a/shrink/bloomz-560m-mic.html b/shrink/bloomz-560m-mic.html
new file mode 100644
index 0000000000000000000000000000000000000000..9e07c2b98ee90da0211289ecb007942100a90011
--- /dev/null
+++ b/shrink/bloomz-560m-mic.html
@@ -0,0 +1,34 @@
+
+
+Shrunk architecture
+
+
Shrunk architecture
huggingface-mic/bloomz-560m-mic@1fa586c833eadf22565c7300a184de964d067039 ← bigscience/bloomz-560m@a2845d7e13dd12efae154a9f1c63fcc2e0cc4b05 · BloomForCausalLM · random weights
+
Original559.2Mparameters
Fixture16.2Mparameters
Layer stacks1compared
Findings0to review
✓ Same architecture, smaller
Every module class, layer kind and checked property of the original is in the fixture.
Layer stacks
One square per layer, colored by kind: the layer's module classes and type attributes. Hover a square for its index and kind.
transformer.h
Original · 24
Fixture · 2
Kind
Original
Fixture
Block
24
2
✓ kept
Where the parameters are
Share of each model's parameters per component. Vocabulary and patch sizes are kept, so embeddings usually take a larger share of the fixture.
Original
Fixture
Component
Original
Share
Fixture
Share
transformer.h
302.3M
54.1%
100.0K
0.6%
transformer.word_embeddings
256.9M
45.9%
16.1M
99.4%
transformer.word_embeddings_layernorm
2.0K
0.0%
128
0.0%
transformer.ln_f
2.0K
0.0%
128
0.0%
Preserved properties
Config
Property
Original
Fixture
model
model_type
bloom
bloom
✓ kept
model
vocab_size
250880
250880
✓ kept
model
tie_word_embeddings
True
True
✓ kept
Module classes
Instances of each class in the built model (prefix Bloom omitted).
Class
Original
Fixture
Linear
97
9
LayerNorm
50
6
Attention
24
2
Block
24
2
Gelu
24
2
MLP
24
2
Dropout
24
2
ForCausalLM
1
1
Model
1
1
Embedding
1
1
ModuleList
1
1
Config changes
3 fields changed, 31 unchanged.
Field
Original
Fixture
hidden_size
1024
64
n_head
16
4
n_layer
24
2
+
\ No newline at end of file
diff --git a/shrink/camembert-base-mic.html b/shrink/camembert-base-mic.html
new file mode 100644
index 0000000000000000000000000000000000000000..4659603d4b2a5da3f1e0f0d2e613494d63a56a6e
--- /dev/null
+++ b/shrink/camembert-base-mic.html
@@ -0,0 +1,34 @@
+
+
+Shrunk architecture
+
+
Shrunk architecture
huggingface-mic/camembert-base-mic@20f8ae1a74c39ea94985475b75af352499fe19ff ← almanach/camembert-base@a75967561c78f2aa81cc41045378d3b4ee25af9e · CamembertForMaskedLM · random weights · built before the shrink rule (no recorded recipe)
+
Original110.7Mparameters
Fixture2.2Mparameters
Layer stacks1compared
Findings0to review
✓ Same architecture, smaller
Every module class, layer kind and checked property of the original is in the fixture.
Layer stacks
One square per layer, colored by kind: the layer's module classes and type attributes. Hover a square for its index and kind.
roberta.encoder.layer
Original · 12
Fixture · 2
Kind
Original
Fixture
Layer
12
2
✓ kept
Where the parameters are
Share of each model's parameters per component. Vocabulary and patch sizes are kept, so embeddings usually take a larger share of the fixture.
Original
Fixture
Component
Original
Share
Fixture
Share
roberta.encoder.layer
85.1M
76.9%
66.9K
3.1%
lm_head.decoder
24.6M
22.2%
2.0M
93.8%
lm_head.dense
590.6K
0.5%
4.2K
0.2%
roberta.embeddings
397.1K
0.4%
33.1K
1.5%
lm_head
32.0K
0.0%
32.0K
1.5%
lm_head.layer_norm
1.5K
0.0%
128
0.0%
Preserved properties
Config
Property
Original
Fixture
model
model_type
camembert
camembert
✓ kept
model
vocab_size
32005
32005
✓ kept
model
tie_word_embeddings
True
True
✓ kept
model
hidden_act
gelu
gelu
✓ kept
model
max_position_embeddings
514
514
✓ kept
Module classes
Instances of each class in the built model (prefix Camembert omitted).
Class
Original
Fixture
Linear
74
14
Dropout
37
7
LayerNorm
26
6
Attention
12
2
Intermediate
12
2
Layer
12
2
Output
12
2
SelfAttention
12
2
SelfOutput
12
2
GELUActivation
12
2
Embedding
3
3
Embeddings
1
1
Encoder
1
1
ForMaskedLM
1
1
LMHead
1
1
Model
1
1
ModuleList
1
1
Config changes
4 fields changed, 26 unchanged.
Field
Original
Fixture
hidden_size
768
64
intermediate_size
3072
128
num_attention_heads
12
4
num_hidden_layers
12
2
+
\ No newline at end of file
diff --git a/shrink/canine-c-mic.html b/shrink/canine-c-mic.html
new file mode 100644
index 0000000000000000000000000000000000000000..e7692c3e8d857fc364e9371a80805af1385f3d90
--- /dev/null
+++ b/shrink/canine-c-mic.html
@@ -0,0 +1,34 @@
+
+
+Shrunk architecture
+
+
Shrunk architecture
huggingface-mic/canine-c-mic@74effe6821673648251eccbcce73a671448d42f9 ← google/canine-c@dc0eaffdff3fa9161613311c7096eeb3e133ee19 · CanineModel · random weights
+
Original132.1Mparameters
Fixture2.3Mparameters
Layer stacks3compared
Findings0to review
✓ Same architecture, smaller
Every module class, layer kind and checked property of the original is in the fixture.
Layer stacks
One square per layer, colored by kind: the layer's module classes and type attributes. Hover a square for its index and kind.
initial_char_encoder.layer
Original · 1
Fixture · 1
Kind
Original
Fixture
Layer
1
1
✓ kept
encoder.layer
Original · 12
Fixture · 2
Kind
Original
Fixture
Layer
12
2
✓ kept
final_char_encoder.layer
Original · 1
Fixture · 1
Kind
Original
Fixture
Layer
1
1
✓ kept
Where the parameters are
Share of each model's parameters per component. Vocabulary and patch sizes are kept, so embeddings usually take a larger share of the fixture.
Original
Fixture
Component
Original
Share
Fixture
Share
encoder.layer
85.1M
64.4%
66.9K
2.9%
char_embeddings.char_position_embeddings
12.6M
9.5%
1.0M
45.9%
initial_char_encoder.layer
7.1M
5.4%
33.5K
1.5%
final_char_encoder.layer
7.1M
5.4%
33.5K
1.5%
projection.conv
4.7M
3.6%
32.8K
1.4%
chars_to_molecules.conv
2.4M
1.8%
16.4K
0.7%
char_embeddings.HashBucketCodepointEmbedder_1
1.6M
1.2%
131.1K
5.7%
12 others
11.6M
8.8%
923.1K
40.4%
Preserved properties
Config
Property
Original
Fixture
model
model_type
canine
canine
✓ kept
model
hidden_act
gelu
gelu
✓ kept
model
max_position_embeddings
16384
16384
✓ kept
Module classes
Instances of each class in the built model (prefix Canine omitted).
Class
Original
Fixture
Linear
85
25
Dropout
44
14
LayerNorm
31
11
GELUActivation
16
6
Attention
14
4
Intermediate
14
4
Layer
14
4
Output
14
4
SelfAttention
14
4
SelfOutput
14
4
Embedding
10
10
Encoder
3
3
ModuleList
3
3
Conv1d
2
2
Embeddings
1
1
Model
1
1
Pooler
1
1
CharactersToMolecules
1
1
ConvProjection
1
1
Tanh
1
1
Config changes
4 fields changed, 24 unchanged.
Field
Original
Fixture
hidden_size
768
64
intermediate_size
3072
128
num_attention_heads
12
4
num_hidden_layers
12
2
+
\ No newline at end of file
diff --git a/shrink/chinese-clip-vit-base-patch16-mic.html b/shrink/chinese-clip-vit-base-patch16-mic.html
new file mode 100644
index 0000000000000000000000000000000000000000..caa66ff032180fac2d4ee19f25a96cec823b6ab2
--- /dev/null
+++ b/shrink/chinese-clip-vit-base-patch16-mic.html
@@ -0,0 +1,34 @@
+
+
+Shrunk architecture
+
+
Shrunk architecture
huggingface-mic/chinese-clip-vit-base-patch16-mic@e4f19250757a82c065b189e77bbffde2acdfe746 ← OFA-Sys/chinese-clip-vit-base-patch16@36e679e65c2a2fead755ae21162091293ad37834 · ChineseCLIPModel · random weights
+
Original188.3Mparameters
Fixture1.6Mparameters
Layer stacks2compared
Findings0to review
✓ Same architecture, smaller
Every module class, layer kind and checked property of the original is in the fixture.
Layer stacks
One square per layer, colored by kind: the layer's module classes and type attributes. Hover a square for its index and kind.
text_model.encoder.layer
Original · 12
Fixture · 2
Kind
Original
Fixture
TextLayer
12
2
✓ kept
vision_model.encoder.layers
Original · 12
Fixture · 2
Kind
Original
Fixture
VisionLayer
12
2
✓ kept
Where the parameters are
Share of each model's parameters per component. Vocabulary and patch sizes are kept, so embeddings usually take a larger share of the fixture.
Original
Fixture
Component
Original
Share
Fixture
Share
text_model.encoder.layer
85.1M
45.2%
66.9K
4.2%
vision_model.encoder.layers
85.1M
45.2%
66.9K
4.2%
text_model.embeddings
16.6M
8.8%
1.4M
87.4%
vision_model.embeddings
741.9K
0.4%
61.8K
3.9%
text_projection
393.2K
0.2%
2.0K
0.1%
visual_projection
393.2K
0.2%
2.0K
0.1%
vision_model.pre_layrnorm
1.5K
0.0%
128
0.0%
2 others
1.5K
0.0%
129
0.0%
Preserved properties
Config
Property
Original
Fixture
model
model_type
chinese_clip
chinese_clip
✓ kept
text_config
model_type
chinese_clip_text_model
chinese_clip_text_model
✓ kept
text_config
vocab_size
21128
21128
✓ kept
text_config
hidden_act
gelu
gelu
✓ kept
text_config
max_position_embeddings
512
512
✓ kept
vision_config
model_type
chinese_clip_vision_model
chinese_clip_vision_model
✓ kept
vision_config
tie_word_embeddings
True
True
✓ kept
vision_config
hidden_act
quick_gelu
quick_gelu
✓ kept
vision_config
image_size
224
224
✓ kept
vision_config
patch_size
16
16
✓ kept
vision_config
num_channels
3
3
✓ kept
Module classes
Instances of each class in the built model (prefix ChineseCLIP omitted).
Class
Original
Fixture
Linear
146
26
LayerNorm
51
11
Dropout
37
7
TextAttention
12
2
TextIntermediate
12
2
TextLayer
12
2
TextOutput
12
2
TextSelfAttention
12
2
TextSelfOutput
12
2
VisionAttention
12
2
VisionLayer
12
2
VisionMLP
12
2
GELUActivation
12
2
QuickGELUActivation
12
2
Embedding
4
4
ModuleList
2
2
Model
1
1
TextEmbeddings
1
1
TextEncoder
1
1
TextModel
1
1
VisionEmbeddings
1
1
VisionEncoder
1
1
VisionModel
1
1
Conv2d
1
1
Config changes
10 fields changed, 74 unchanged.
Field
Original
Fixture
projection_dim
512
32
text_config.hidden_size
768
64
text_config.intermediate_size
3072
128
text_config.num_attention_heads
12
4
text_config.num_hidden_layers
12
2
vision_config.hidden_size
768
64
vision_config.intermediate_size
3072
128
vision_config.num_attention_heads
12
4
vision_config.num_hidden_layers
12
2
vision_config.projection_dim
512
32
+
\ No newline at end of file
diff --git a/shrink/clap-htsat-fused-mic.html b/shrink/clap-htsat-fused-mic.html
new file mode 100644
index 0000000000000000000000000000000000000000..9ee12b4cf7066e99c91e06b5c76e67da655401b6
--- /dev/null
+++ b/shrink/clap-htsat-fused-mic.html
@@ -0,0 +1,34 @@
+
+
+Shrunk architecture
+
+
Shrunk architecture
huggingface-mic/clap-htsat-fused-mic@fb8962e8a171e9824f1f2b920fe20ea46a28895c ← laion/clap-htsat-fused@365dea6ef167def6676140ed93bbc43f84dabb28 · ClapModel · random weights · built before the shrink rule (no recorded recipe)
+
Original153.5Mparameters
Fixture3.7Mparameters
Layer stacks2compared
Findings0to review
✓ Same architecture, smaller
Every module class, layer kind and checked property of the original is in the fixture.
Layer stacks
One square per layer, colored by kind: the layer's module classes and type attributes. Hover a square for its index and kind.
text_model.encoder.layer
Original · 12
Fixture · 2
Kind
Original
Fixture
TextLayer
12
2
✓ kept
audio_model.audio_encoder.layers
Original · 4
Fixture · 4
Kind
Original
Fixture
AudioPatchMerging
3
3
✓ kept
no AudioPatchMerging
1
1
✓ kept
Where the parameters are
Share of each model's parameters per component. Vocabulary and patch sizes are kept, so embeddings usually take a larger share of the fixture.
Original
Fixture
Component
Original
Share
Fixture
Share
text_model.encoder.layer
85.1M
55.4%
66.9K
1.8%
text_model.embeddings
39.0M
25.4%
3.3M
89.0%
audio_model.audio_encoder.layers
27.5M
17.9%
321.6K
8.8%
text_model.pooler
590.6K
0.4%
4.2K
0.1%
audio_projection.linear1
393.7K
0.3%
4.1K
0.1%
text_projection.linear1
393.7K
0.3%
2.1K
0.1%
text_projection.linear2
262.7K
0.2%
1.1K
0.0%
4 others
280.8K
0.2%
2.9K
0.1%
Preserved properties
Config
Property
Original
Fixture
model
model_type
clap
clap
✓ kept
text_config
model_type
clap_text_model
clap_text_model
✓ kept
text_config
vocab_size
50265
50265
✓ kept
text_config
tie_word_embeddings
True
True
✓ kept
text_config
hidden_act
gelu
gelu
✓ kept
text_config
max_position_embeddings
514
514
✓ kept
audio_config
model_type
clap_audio_model
clap_audio_model
✓ kept
audio_config
tie_word_embeddings
True
True
✓ kept
audio_config
hidden_act
gelu
gelu
✓ kept
audio_config
patch_size
4
4
✓ kept
audio_config
num_mel_bins
64
64
✓ kept
Module classes
Instances of each class in the built model (prefix Clap omitted).
Class
Original
Fixture
Linear
152
44
Dropout
73
19
LayerNorm
54
18
GELUActivation
24
6
AudioAttention
12
4
AudioIntermediate
12
4
AudioLayer
12
4
AudioOutput
12
4
AudioSelfAttention
12
4
AudioSelfOutput
12
4
TextAttention
12
2
TextIntermediate
12
2
TextLayer
12
2
TextOutput
12
2
TextSelfAttention
12
2
TextSelfOutput
12
2
Identity
12
4
Conv2d
6
6
ModuleList
6
6
BatchNorm2d
5
5
AudioStage
4
4
ReLU
4
4
AudioPatchMerging
3
3
Embedding
3
3
ProjectionLayer
2
2
Sequential
2
2
AdaptiveAvgPool1d
1
1
AdaptiveAvgPool2d
1
1
AudioAFFBlock
1
1
AudioEncoder
1
1
AudioModel
1
1
AudioPatchEmbed
1
1
Model
1
1
TextEmbeddings
1
1
TextEncoder
1
1
TextModel
1
1
TextPooler
1
1
Sigmoid
1
1
Tanh
1
1
Config changes
13 fields changed, 106 unchanged.
Field
Original
Fixture
audio_config.depths
[2, 2, 6, 2]
[1, 1, 1, 1]
audio_config.hidden_size
768
128
audio_config.patch_embeds_hidden_size
96
16
audio_config.projection_dim
512
32
audio_config.projection_hidden_size
768
128
hidden_size
768
64
num_hidden_layers
16
6
projection_dim
512
32
text_config.hidden_size
768
64
text_config.intermediate_size
3072
128
text_config.num_attention_heads
12
4
text_config.num_hidden_layers
12
2
text_config.projection_dim
512
32
+
\ No newline at end of file
diff --git a/shrink/clip-mic.html b/shrink/clip-mic.html
new file mode 100644
index 0000000000000000000000000000000000000000..890c00fc01b609f6fc104b1128ba2188c8aa680c
--- /dev/null
+++ b/shrink/clip-mic.html
@@ -0,0 +1,34 @@
+
+
+Shrunk architecture
+
+
Shrunk architecture
huggingface-mic/clip-mic@44c8d8042c6d0355f856d0b8549f5f8270ab9640 ← openai/clip-vit-base-patch32@3d74acf9a28c67741b2f4f2ea7635f0aaf6f0268 · CLIPModel · random weights · built before the shrink rule (no recorded recipe)
+
Original151.3Mparameters
Fixture3.5Mparameters
Layer stacks2compared
Findings0to review
✓ Same architecture, smaller
Every module class, layer kind and checked property of the original is in the fixture.
Layer stacks
One square per layer, colored by kind: the layer's module classes and type attributes. Hover a square for its index and kind.
text_model.encoder.layers
Original · 12
Fixture · 2
Kind
Original
Fixture
EncoderLayer
12
2
✓ kept
vision_model.encoder.layers
Original · 12
Fixture · 2
Kind
Original
Fixture
EncoderLayer
12
2
✓ kept
Where the parameters are
Share of each model's parameters per component. Vocabulary and patch sizes are kept, so embeddings usually take a larger share of the fixture.
Original
Fixture
Component
Original
Share
Fixture
Share
vision_model.encoder.layers
85.1M
56.2%
66.9K
1.9%
text_model.encoder.layers
37.8M
25.0%
66.9K
1.9%
text_model.embeddings
25.3M
16.7%
3.2M
90.4%
vision_model.embeddings
2.4M
1.6%
199.9K
5.7%
visual_projection
393.2K
0.3%
2.0K
0.1%
text_projection
262.1K
0.2%
2.0K
0.1%
vision_model.pre_layrnorm
1.5K
0.0%
128
0.0%
3 others
2.6K
0.0%
257
0.0%
Preserved properties
Config
Property
Original
Fixture
model
model_type
clip
clip
✓ kept
text_config
model_type
clip_text_model
clip_text_model
✓ kept
text_config
vocab_size
49408
49408
✓ kept
text_config
tie_word_embeddings
True
True
✓ kept
text_config
hidden_act
quick_gelu
quick_gelu
✓ kept
text_config
max_position_embeddings
77
77
✓ kept
vision_config
model_type
clip_vision_model
clip_vision_model
✓ kept
vision_config
tie_word_embeddings
True
True
✓ kept
vision_config
hidden_act
quick_gelu
quick_gelu
✓ kept
vision_config
image_size
224
224
✓ kept
vision_config
patch_size
32
32
✓ kept
vision_config
num_channels
3
3
✓ kept
Module classes
Instances of each class in the built model (prefix CLIP omitted).
Class
Original
Fixture
Linear
146
26
LayerNorm
51
11
Attention
24
4
EncoderLayer
24
4
MLP
24
4
QuickGELUActivation
24
4
Embedding
3
3
Encoder
2
2
ModuleList
2
2
Model
1
1
TextEmbeddings
1
1
TextModel
1
1
VisionEmbeddings
1
1
VisionModel
1
1
Conv2d
1
1
Config changes
11 fields changed, 80 unchanged.
Field
Original
Fixture
projection_dim
512
32
text_config.hidden_size
512
64
text_config.intermediate_size
2048
128
text_config.num_attention_heads
8
4
text_config.num_hidden_layers
12
2
text_config.projection_dim
512
32
vision_config.hidden_size
768
64
vision_config.intermediate_size
3072
128
vision_config.num_attention_heads
12
4
vision_config.num_hidden_layers
12
2
vision_config.projection_dim
512
32
+
\ No newline at end of file
diff --git a/shrink/clip-vit-base-patch32-mic.html b/shrink/clip-vit-base-patch32-mic.html
new file mode 100644
index 0000000000000000000000000000000000000000..69945ba23e38fb02d26e87da91828784ccfebc4b
--- /dev/null
+++ b/shrink/clip-vit-base-patch32-mic.html
@@ -0,0 +1,34 @@
+
+
+Shrunk architecture
+
+
Shrunk architecture
huggingface-mic/clip-vit-base-patch32-mic@51ae90d9c34bf49fafe02004f9b4b1c2062cd278 ← openai/clip-vit-base-patch32@3d74acf9a28c67741b2f4f2ea7635f0aaf6f0268 · CLIPModel · random weights · built before the shrink rule (no recorded recipe)
+
Original151.3Mparameters
Fixture3.5Mparameters
Layer stacks2compared
Findings0to review
✓ Same architecture, smaller
Every module class, layer kind and checked property of the original is in the fixture.
Layer stacks
One square per layer, colored by kind: the layer's module classes and type attributes. Hover a square for its index and kind.
text_model.encoder.layers
Original · 12
Fixture · 2
Kind
Original
Fixture
EncoderLayer
12
2
✓ kept
vision_model.encoder.layers
Original · 12
Fixture · 2
Kind
Original
Fixture
EncoderLayer
12
2
✓ kept
Where the parameters are
Share of each model's parameters per component. Vocabulary and patch sizes are kept, so embeddings usually take a larger share of the fixture.
Original
Fixture
Component
Original
Share
Fixture
Share
vision_model.encoder.layers
85.1M
56.2%
66.9K
1.9%
text_model.encoder.layers
37.8M
25.0%
66.9K
1.9%
text_model.embeddings
25.3M
16.7%
3.2M
90.4%
vision_model.embeddings
2.4M
1.6%
199.9K
5.7%
visual_projection
393.2K
0.3%
2.0K
0.1%
text_projection
262.1K
0.2%
2.0K
0.1%
vision_model.pre_layrnorm
1.5K
0.0%
128
0.0%
3 others
2.6K
0.0%
257
0.0%
Preserved properties
Config
Property
Original
Fixture
model
model_type
clip
clip
✓ kept
text_config
model_type
clip_text_model
clip_text_model
✓ kept
text_config
vocab_size
49408
49408
✓ kept
text_config
tie_word_embeddings
True
True
✓ kept
text_config
hidden_act
quick_gelu
quick_gelu
✓ kept
text_config
max_position_embeddings
77
77
✓ kept
vision_config
model_type
clip_vision_model
clip_vision_model
✓ kept
vision_config
tie_word_embeddings
True
True
✓ kept
vision_config
hidden_act
quick_gelu
quick_gelu
✓ kept
vision_config
image_size
224
224
✓ kept
vision_config
patch_size
32
32
✓ kept
vision_config
num_channels
3
3
✓ kept
Module classes
Instances of each class in the built model (prefix CLIP omitted).
Class
Original
Fixture
Linear
146
26
LayerNorm
51
11
Attention
24
4
EncoderLayer
24
4
MLP
24
4
QuickGELUActivation
24
4
Embedding
3
3
Encoder
2
2
ModuleList
2
2
Model
1
1
TextEmbeddings
1
1
TextModel
1
1
VisionEmbeddings
1
1
VisionModel
1
1
Conv2d
1
1
Config changes
11 fields changed, 80 unchanged.
Field
Original
Fixture
projection_dim
512
32
text_config.hidden_size
512
64
text_config.intermediate_size
2048
128
text_config.num_attention_heads
8
4
text_config.num_hidden_layers
12
2
text_config.projection_dim
512
32
vision_config.hidden_size
768
64
vision_config.intermediate_size
3072
128
vision_config.num_attention_heads
12
4
vision_config.num_hidden_layers
12
2
vision_config.projection_dim
512
32
+
\ No newline at end of file
diff --git a/shrink/clip-vit-base-patch32-vision-mic.html b/shrink/clip-vit-base-patch32-vision-mic.html
new file mode 100644
index 0000000000000000000000000000000000000000..a7f21c695960ce9066a7d9a0e99c20decf316c74
--- /dev/null
+++ b/shrink/clip-vit-base-patch32-vision-mic.html
@@ -0,0 +1,34 @@
+
+
+Shrunk architecture
+
+
Shrunk architecture
huggingface-mic/clip-vit-base-patch32-vision-mic@b1084cb8a6e683e107ed57c9657c25ddbf107137 ← kunalkanaujiya/clip-vit-base-patch32-vision@4c3ca43dcae1f63f1cda22eeabd99674c894d2b7 · CLIPVisionModelWithProjection · random weights
+
Original87.8Mparameters
Fixture269.1Kparameters
Layer stacks1compared
Findings0to review
✓ Same architecture, smaller
Every module class, layer kind and checked property of the original is in the fixture.
Layer stacks
One square per layer, colored by kind: the layer's module classes and type attributes. Hover a square for its index and kind.
vision_model.encoder.layers
Original · 12
Fixture · 2
Kind
Original
Fixture
CLIPEncoderLayer
12
2
✓ kept
Where the parameters are
Share of each model's parameters per component. Vocabulary and patch sizes are kept, so embeddings usually take a larger share of the fixture.
Original
Fixture
Component
Original
Share
Fixture
Share
vision_model.encoder.layers
85.1M
96.8%
66.9K
24.9%
vision_model.embeddings
2.4M
2.7%
199.9K
74.3%
visual_projection
393.2K
0.4%
2.0K
0.8%
vision_model.post_layernorm
1.5K
0.0%
128
0.0%
vision_model.pre_layrnorm
1.5K
0.0%
128
0.0%
Preserved properties
Config
Property
Original
Fixture
model
model_type
clip_vision_model
clip_vision_model
✓ kept
model
tie_word_embeddings
True
True
✓ kept
model
hidden_act
quick_gelu
quick_gelu
✓ kept
model
image_size
224
224
✓ kept
model
patch_size
32
32
✓ kept
model
num_channels
3
3
✓ kept
Module classes
Instances of each class in the built model (prefix CLIPVision omitted).
Class
Original
Fixture
Linear
73
13
LayerNorm
26
6
CLIPAttention
12
2
CLIPEncoderLayer
12
2
CLIPMLP
12
2
QuickGELUActivation
12
2
CLIPEncoder
1
1
Embeddings
1
1
Model
1
1
ModelWithProjection
1
1
Conv2d
1
1
Embedding
1
1
ModuleList
1
1
Config changes
5 fields changed, 35 unchanged.
Field
Original
Fixture
hidden_size
768
64
intermediate_size
3072
128
num_attention_heads
12
4
num_hidden_layers
12
2
projection_dim
512
32
+
\ No newline at end of file
diff --git a/shrink/clipseg-rd64-refined-mic.html b/shrink/clipseg-rd64-refined-mic.html
new file mode 100644
index 0000000000000000000000000000000000000000..450a8ba0290472569f27d5df117a90e9233eaf2e
--- /dev/null
+++ b/shrink/clipseg-rd64-refined-mic.html
@@ -0,0 +1,34 @@
+
+
+Shrunk architecture
+
+
Shrunk architecture
huggingface-mic/clipseg-rd64-refined-mic@5be0dc861834231292b5d365fffc3d3e1d268cd0 ← CIDAS/clipseg-rd64-refined@999e0328d9e10b484360c477313983f9afdd7050 · CLIPSegForImageSegmentation · random weights
+
Original150.7Mparameters
Fixture4.3Mparameters
Layer stacks3compared
Findings0to review
✓ Same architecture, smaller
Every module class, layer kind and checked property of the original is in the fixture.
Layer stacks
One square per layer, colored by kind: the layer's module classes and type attributes. Hover a square for its index and kind.
clip.text_model.encoder.layers
Original · 12
Fixture · 2
Kind
Original
Fixture
EncoderLayer
12
2
✓ kept
clip.vision_model.encoder.layers
Original · 12
Fixture · 2
Kind
Original
Fixture
EncoderLayer
12
2
✓ kept
decoder.layers
Original · 3
Fixture · 3
Kind
Original
Fixture
DecoderLayer
3
3
✓ kept
Where the parameters are
Share of each model's parameters per component. Vocabulary and patch sizes are kept, so embeddings usually take a larger share of the fixture.
Original
Fixture
Component
Original
Share
Fixture
Share
clip.vision_model.encoder.layers
85.1M
56.4%
66.9K
1.6%
clip.text_model.encoder.layers
37.8M
25.1%
66.9K
1.6%
clip.text_model
25.3M
16.8%
3.2M
73.7%
decoder.layers
843.5K
0.6%
843.5K
19.6%
clip.vision_model
745.0K
0.5%
62.1K
1.4%
clip.visual_projection
393.2K
0.3%
2.0K
0.0%
clip.text_projection
262.1K
0.2%
2.0K
0.0%
5 others
283.6K
0.2%
86.9K
2.0%
Preserved properties
Config
Property
Original
Fixture
model
model_type
clipseg
clipseg
✓ kept
text_config
model_type
clipseg_text_model
clipseg_text_model
✓ kept
text_config
vocab_size
49408
49408
✓ kept
text_config
tie_word_embeddings
True
True
✓ kept
text_config
hidden_act
quick_gelu
quick_gelu
✓ kept
text_config
max_position_embeddings
77
77
✓ kept
vision_config
model_type
clipseg_vision_model
clipseg_vision_model
✓ kept
vision_config
tie_word_embeddings
True
True
✓ kept
vision_config
hidden_act
quick_gelu
quick_gelu
✓ kept
vision_config
image_size
224
224
✓ kept
vision_config
patch_size
16
16
✓ kept
vision_config
num_channels
3
3
✓ kept
Module classes
Instances of each class in the built model (prefix CLIPSeg omitted).
Class
Original
Fixture
Linear
169
49
LayerNorm
57
17
Attention
27
7
MLP
27
7
EncoderLayer
24
4
QuickGELUActivation
24
4
ReLU
5
5
ModuleList
4
4
DecoderLayer
3
3
Embedding
3
3
Encoder
2
2
Conv2d
2
2
ConvTranspose2d
2
2
Decoder
1
1
ForImageSegmentation
1
1
Model
1
1
TextEmbeddings
1
1
TextModel
1
1
VisionEmbeddings
1
1
VisionModel
1
1
Sequential
1
1
Config changes
10 fields changed, 89 unchanged.
Field
Original
Fixture
extract_layers
[3, 6, 9]
[0, 1, 1]
projection_dim
512
32
text_config.hidden_size
512
64
text_config.intermediate_size
2048
128
text_config.num_attention_heads
8
4
text_config.num_hidden_layers
12
2
vision_config.hidden_size
768
64
vision_config.intermediate_size
3072
128
vision_config.num_attention_heads
12
4
vision_config.num_hidden_layers
12
2
+
\ No newline at end of file
diff --git a/shrink/conv-bert-base-mic.html b/shrink/conv-bert-base-mic.html
new file mode 100644
index 0000000000000000000000000000000000000000..f187c8b42013eb8df3f3c72ae4b143cfa71e5fb6
--- /dev/null
+++ b/shrink/conv-bert-base-mic.html
@@ -0,0 +1,34 @@
+
+
+Shrunk architecture
+
+
Shrunk architecture
huggingface-mic/conv-bert-base-mic@939ec1a4a34866f8fe836950538ad5ba620139ef ← YituTech/conv-bert-base@5cb451936b5c4a96562d8b146de85f64f9cf2c22 · ConvBertModel · random weights
+
Original105.7Mparameters
Fixture24.0Mparameters
Layer stacks1compared
Findings0to review
✓ Same architecture, smaller
Every module class, layer kind and checked property of the original is in the fixture.
Layer stacks
One square per layer, colored by kind: the layer's module classes and type attributes. Hover a square for its index and kind.
encoder.layer
Original · 12
Fixture · 2
Kind
Original
Fixture
Layer
12
2
✓ kept
Where the parameters are
Share of each model's parameters per component. Vocabulary and patch sizes are kept, so embeddings usually take a larger share of the fixture.
Original
Fixture
Component
Original
Share
Fixture
Share
encoder.layer
81.8M
77.4%
65.1K
0.3%
embeddings.word_embeddings
23.4M
22.2%
23.4M
97.9%
embeddings.position_embeddings
393.2K
0.4%
393.2K
1.6%
embeddings.token_type_embeddings
1.5K
0.0%
1.5K
0.0%
embeddings.LayerNorm
1.5K
0.0%
1.5K
0.0%
embeddings_project
0
0.0%
49.2K
0.2%
Preserved properties
Config
Property
Original
Fixture
model
model_type
convbert
convbert
✓ kept
model
vocab_size
30522
30522
✓ kept
model
tie_word_embeddings
True
True
✓ kept
model
hidden_act
gelu
gelu
✓ kept
model
max_position_embeddings
512
512
✓ kept
Module classes
Instances of each class in the built model (prefix ConvBert omitted).
Class
Original
Fixture
Linear
96
17
Dropout
37
7
LayerNorm
25
5
Conv1d
24
4
Attention
12
2
Intermediate
12
2
Layer
12
2
Output
12
2
SelfAttention
12
2
SelfOutput
12
2
GELUActivation
12
2
SeparableConv1D
12
2
Unfold
12
2
Embedding
3
3
Embeddings
1
1
Encoder
1
1
Model
1
1
ModuleList
1
1
Config changes
4 fields changed, 28 unchanged.
Field
Original
Fixture
hidden_size
768
64
intermediate_size
3072
128
num_attention_heads
12
4
num_hidden_layers
12
2
+
\ No newline at end of file
diff --git a/shrink/convnext-tiny-224-mic.html b/shrink/convnext-tiny-224-mic.html
new file mode 100644
index 0000000000000000000000000000000000000000..96c413cea742ccd44d8dce19f40b083b1eb2a4fa
--- /dev/null
+++ b/shrink/convnext-tiny-224-mic.html
@@ -0,0 +1,34 @@
+
+
+Shrunk architecture
+
+
Shrunk architecture
huggingface-mic/convnext-tiny-224-mic@f5dae79dc4935e97ae29a9414541fa8d790454ee ← facebook/convnext-tiny-224@6166b7613034066690a621d8bf25ffdf181a34f0 · ConvNextForImageClassification · random weights
+
Original28.6Mparameters
Fixture361.5Kparameters
Layer stacks1compared
Findings0to review
✓ Same architecture, smaller
Every module class, layer kind and checked property of the original is in the fixture.
Layer stacks
One square per layer, colored by kind: the layer's module classes and type attributes. Hover a square for its index and kind.
convnext.encoder.stages
Original · 4
Fixture · 4
Kind
Original
Fixture
Layer
4
4
✓ kept
Where the parameters are
Share of each model's parameters per component. Vocabulary and patch sizes are kept, so embeddings usually take a larger share of the fixture.
Original
Fixture
Component
Original
Share
Fixture
Share
convnext.encoder.stages
27.8M
97.3%
231.5K
64.0%
classifier
769.0K
2.7%
129.0K
35.7%
convnext.embeddings
4.9K
0.0%
816
0.2%
convnext.layernorm
1.5K
0.0%
256
0.1%
Preserved properties
Config
Property
Original
Fixture
model
model_type
convnext
convnext
✓ kept
model
hidden_act
gelu
gelu
✓ kept
model
image_size
224
224
✓ kept
model
patch_size
4
4
✓ kept
model
num_channels
3
3
✓ kept
Module classes
Instances of each class in the built model (prefix ConvNext omitted).
Class
Original
Fixture
Linear
37
9
Conv2d
22
8
LayerNorm
22
8
Layer
18
4
GELUActivation
18
4
Identity
18
4
ModuleList
9
9
Stage
4
4
Embeddings
1
1
Encoder
1
1
ForImageClassification
1
1
Model
1
1
LayerNorm
1
1
Config changes
2 fields changed, 21 unchanged.
Field
Original
Fixture
depths
[3, 3, 9, 3]
[1, 1, 1, 1]
hidden_sizes
[96, 192, 384, 768]
[16, 32, 64, 128]
+
\ No newline at end of file
diff --git a/shrink/convnextv2-nano-1k-224-mic.html b/shrink/convnextv2-nano-1k-224-mic.html
new file mode 100644
index 0000000000000000000000000000000000000000..8a82dda42f7e0f5f2666be925c3bb5fa47679545
--- /dev/null
+++ b/shrink/convnextv2-nano-1k-224-mic.html
@@ -0,0 +1,34 @@
+
+
+Shrunk architecture
+
+
Shrunk architecture
huggingface-mic/convnextv2-nano-1k-224-mic@df320ec01c275577e0266bbffdecd6eec693d41b ← facebook/convnextv2-nano-1k-224@5fd38629f57580835f733e36a7da810d5c8df9ec · ConvNextV2ForImageClassification · random weights
+
Original15.6Mparameters
Fixture363.2Kparameters
Layer stacks1compared
Findings0to review
✓ Same architecture, smaller
Every module class, layer kind and checked property of the original is in the fixture.
Layer stacks
One square per layer, colored by kind: the layer's module classes and type attributes. Hover a square for its index and kind.
convnextv2.encoder.stages
Original · 4
Fixture · 4
Kind
Original
Fixture
Layer
4
4
✓ kept
Where the parameters are
Share of each model's parameters per component. Vocabulary and patch sizes are kept, so embeddings usually take a larger share of the fixture.
Original
Fixture
Component
Original
Share
Fixture
Share
convnextv2.encoder.stages
15.0M
95.9%
233.1K
64.2%
classifier
641.0K
4.1%
129.0K
35.5%
convnextv2.embeddings
4.1K
0.0%
816
0.2%
convnextv2.layernorm
1.3K
0.0%
256
0.1%
Preserved properties
Config
Property
Original
Fixture
model
model_type
convnextv2
convnextv2
✓ kept
model
hidden_act
gelu
gelu
✓ kept
model
image_size
224
224
✓ kept
model
patch_size
4
4
✓ kept
model
num_channels
3
3
✓ kept
Module classes
Instances of each class in the built model (prefix ConvNextV2 omitted).
Class
Original
Fixture
Linear
29
9
Conv2d
18
8
LayerNorm
18
8
GRN
14
4
Layer
14
4
GELUActivation
14
4
Identity
14
4
ModuleList
9
9
Stage
4
4
Embeddings
1
1
Encoder
1
1
ForImageClassification
1
1
Model
1
1
LayerNorm
1
1
Config changes
2 fields changed, 20 unchanged.
Field
Original
Fixture
depths
[2, 2, 8, 2]
[1, 1, 1, 1]
hidden_sizes
[80, 160, 320, 640]
[16, 32, 64, 128]
+
\ No newline at end of file
diff --git a/shrink/csm-1b-mic.html b/shrink/csm-1b-mic.html
new file mode 100644
index 0000000000000000000000000000000000000000..14ac9a1a9fa957b7d2c7dac67239d6729b92f961
--- /dev/null
+++ b/shrink/csm-1b-mic.html
@@ -0,0 +1,34 @@
+
+
+Shrunk architecture
+
+
Shrunk architecture
huggingface-mic/csm-1b-mic@ceb9f9aebdb8fe8d8aca3a3c51091ae2527893ca ← unsloth/csm-1b@39e8c756bec133ad7eb9ea1097c84a2bd891c949 · CsmForConditionalGeneration · random weights
+
Original1.6Bparameters
Fixture39.7Mparameters
Layer stacks6compared
Findings0to review
✓ Same architecture, smaller
Every module class, layer kind and checked property of the original is in the fixture.
Layer stacks
One square per layer, colored by kind: the layer's module classes and type attributes. Hover a square for its index and kind.
Every module class, layer kind and checked property of the original is in the fixture.
Layer stacks
One square per layer, colored by kind: the layer's module classes and type attributes. Hover a square for its index and kind.
deberta.encoder.layer
Original · 24
Fixture · 2
Kind
Original
Fixture
Layer
24
2
✓ kept
Where the parameters are
Share of each model's parameters per component. Vocabulary and patch sizes are kept, so embeddings usually take a larger share of the fixture.
Original
Fixture
Component
Original
Share
Fixture
Share
deberta.encoder.layer
352.6M
86.8%
83.3K
2.5%
deberta.embeddings
51.5M
12.7%
3.2M
95.5%
pooler.dense
1.0M
0.3%
4.2K
0.1%
deberta.encoder
1.0M
0.3%
65.5K
1.9%
classifier
3.1K
0.0%
195
0.0%
Preserved properties
Config
Property
Original
Fixture
model
model_type
deberta
deberta
✓ kept
model
vocab_size
50265
50265
✓ kept
model
tie_word_embeddings
True
True
✓ kept
model
hidden_act
gelu
gelu
✓ kept
model
max_position_embeddings
512
512
✓ kept
Module classes
Instances of each class in the built model (prefix Deberta omitted).
Class
Original
Fixture
Linear
146
14
Dropout
99
11
LayerNorm
49
5
Attention
24
2
Intermediate
24
2
Layer
24
2
Output
24
2
SelfOutput
24
2
DisentangledSelfAttention
24
2
GELUActivation
24
2
Embedding
2
2
ContextPooler
1
1
Embeddings
1
1
Encoder
1
1
ForSequenceClassification
1
1
Model
1
1
ModuleList
1
1
Config changes
5 fields changed, 28 unchanged.
Field
Original
Fixture
hidden_size
1024
64
intermediate_size
4096
128
num_attention_heads
16
4
num_hidden_layers
24
2
pooler_hidden_size
1024
64
+
\ No newline at end of file
diff --git a/shrink/detr-mic.html b/shrink/detr-mic.html
new file mode 100644
index 0000000000000000000000000000000000000000..2886151593bd808df0539005f80a16ebbbb6d01b
--- /dev/null
+++ b/shrink/detr-mic.html
@@ -0,0 +1,34 @@
+
+
+Shrunk architecture
+
+
Shrunk architecture
huggingface-mic/detr-mic@47543c8af7646760a74482fedf00ec00fd8735e1 ← facebook/detr-resnet-50@1d5f47bd3bdd2c4bbfa585418ffe6da5028b4c0b · DetrForObjectDetection · random weights · built before the shrink rule (no recorded recipe)
+
Original41.5Mparameters
Fixture23.8Mparameters
Layer stacks2compared
Findings0to review
✓ Same architecture, smaller
Every module class, layer kind and checked property of the original is in the fixture.
Layer stacks
One square per layer, colored by kind: the layer's module classes and type attributes. Hover a square for its index and kind.
model.encoder.layers
Original · 6
Fixture · 2
Kind
Original
Fixture
EncoderLayer
6
2
✓ kept
model.decoder.layers
Original · 6
Fixture · 2
Kind
Original
Fixture
DecoderLayer
6
2
✓ kept
Where the parameters are
Share of each model's parameters per component. Vocabulary and patch sizes are kept, so embeddings usually take a larger share of the fixture.
Original
Fixture
Component
Original
Share
Fixture
Share
model.backbone
23.5M
56.5%
23.5M
98.7%
model.decoder.layers
9.5M
22.8%
100.5K
0.4%
model.encoder.layers
7.9M
19.0%
66.9K
0.3%
model.input_projection
524.5K
1.3%
131.1K
0.6%
bbox_predictor.layers
132.6K
0.3%
8.6K
0.0%
model.query_position_embeddings
25.6K
0.1%
6.4K
0.0%
class_labels_classifier
23.6K
0.1%
6.0K
0.0%
1 others
512
0.0%
128
0.0%
Preserved properties
Config
Property
Original
Fixture
model
model_type
detr
detr
✓ kept
model
max_position_embeddings
1024
1024
✓ kept
model
num_channels
3
3
✓ kept
backbone_config
model_type
timm_backbone
timm_backbone
✓ kept
backbone_config
num_channels
3
3
✓ kept
Module classes
Instances of each class in the built model (prefix Detr omitted).
Class
Original
Fixture
Linear
100
36
ReLU
61
53
Conv2d
54
54
FrozenBatchNorm2d
53
53
Identity
32
32
LayerNorm
31
11
Bottleneck
16
16
MLP
12
4
SelfAttention
12
4
Sequential
8
8
CrossAttention
6
2
DecoderLayer
6
2
EncoderLayer
6
2
ModuleList
3
3
ConvEncoder
1
1
Decoder
1
1
Encoder
1
1
ForObjectDetection
1
1
MLPPredictionHead
1
1
Model
1
1
SinePositionEmbedding
1
1
Embedding
1
1
FeatureListNet
1
1
MaxPool2d
1
1
Config changes
10 fields changed, 48 unchanged.
Field
Original
Fixture
backbone_config.out_features
null
["layer4"]
backbone_config.out_indices
[1, 2, 3, 4]
[-1]
backbone_config.stage_names
—
["act1", "layer1", "layer2", "layer3", "layer4"]
d_model
256
64
decoder_attention_heads
8
4
decoder_ffn_dim
2048
128
decoder_layers
6
2
encoder_attention_heads
8
4
encoder_ffn_dim
2048
128
encoder_layers
6
2
+
\ No newline at end of file
diff --git a/shrink/dinov2-small-mic.html b/shrink/dinov2-small-mic.html
new file mode 100644
index 0000000000000000000000000000000000000000..0a197f34b46ad830643925490d21fa4376bb35dd
--- /dev/null
+++ b/shrink/dinov2-small-mic.html
@@ -0,0 +1,34 @@
+
+
+Shrunk architecture
+
+
Shrunk architecture
huggingface-mic/dinov2-small-mic@770cc8e9a4eccc42eb2f4ea67371b9997d0395d5 ← facebook/dinov2-small@ed25f3a31f01632728cabb09d1542f84ab7b0056 · Dinov2Model · random weights · built before the shrink rule (no recorded recipe)
+
Original22.1Mparameters
Fixture225.9Kparameters
Layer stacks1compared
Findings0to review
✓ Same architecture, smaller
Every module class, layer kind and checked property of the original is in the fixture.
Layer stacks
One square per layer, colored by kind: the layer's module classes and type attributes. Hover a square for its index and kind.
encoder.layer
Original · 12
Fixture · 2
Kind
Original
Fixture
Layer
12
2
✓ kept
Where the parameters are
Share of each model's parameters per component. Vocabulary and patch sizes are kept, so embeddings usually take a larger share of the fixture.
Original
Fixture
Component
Original
Share
Fixture
Share
encoder.layer
21.3M
96.6%
100.2K
44.4%
embeddings
526.8K
2.4%
87.8K
38.9%
embeddings.patch_embeddings
226.2K
1.0%
37.7K
16.7%
layernorm
768
0.0%
128
0.1%
Preserved properties
Config
Property
Original
Fixture
model
model_type
dinov2
dinov2
✓ kept
model
hidden_act
gelu
gelu
✓ kept
model
image_size
518
518
✓ kept
model
patch_size
14
14
✓ kept
model
num_channels
3
3
✓ kept
Module classes
Instances of each class in the built model (prefix Dinov2 omitted).
Class
Original
Fixture
Linear
72
12
LayerNorm
25
5
LayerScale
24
4
Attention
12
2
Layer
12
2
MLP
12
2
GELUActivation
12
2
Identity
12
2
Conv2d
1
1
Embeddings
1
1
Encoder
1
1
Model
1
1
PatchEmbeddings
1
1
Dropout
1
1
ModuleList
1
1
Config changes
5 fields changed, 26 unchanged.
Field
Original
Fixture
hidden_size
384
64
num_attention_heads
6
4
num_hidden_layers
12
2
out_features
["stage12"]
["stage2"]
out_indices
[12]
[2]
+
\ No newline at end of file
diff --git a/shrink/dinov2-with-registers-base-mic.html b/shrink/dinov2-with-registers-base-mic.html
new file mode 100644
index 0000000000000000000000000000000000000000..19736b64d13838fb5b2121b1a461664a2a168c45
--- /dev/null
+++ b/shrink/dinov2-with-registers-base-mic.html
@@ -0,0 +1,34 @@
+
+
+Shrunk architecture
+
+
Shrunk architecture
huggingface-mic/dinov2-with-registers-base-mic@186e99a19d6165f6a42c2794175a5d87d8781984 ← facebook/dinov2-with-registers-base@a1d738ccfa7ae170945f210395d99dde8adb1805 · Dinov2WithRegistersModel · random weights
+
Original86.6Mparameters
Fixture226.1Kparameters
Layer stacks1compared
Findings0to review
✓ Same architecture, smaller
Every module class, layer kind and checked property of the original is in the fixture.
Layer stacks
One square per layer, colored by kind: the layer's module classes and type attributes. Hover a square for its index and kind.
encoder.layer
Original · 12
Fixture · 2
Kind
Original
Fixture
Layer
12
2
✓ kept
Where the parameters are
Share of each model's parameters per component. Vocabulary and patch sizes are kept, so embeddings usually take a larger share of the fixture.
Original
Fixture
Component
Original
Share
Fixture
Share
encoder.layer
85.1M
98.3%
100.2K
44.3%
embeddings
1.1M
1.2%
88.1K
38.9%
embeddings.patch_embeddings
452.4K
0.5%
37.7K
16.7%
layernorm
1.5K
0.0%
128
0.1%
Preserved properties
Config
Property
Original
Fixture
model
model_type
dinov2_with_registers
dinov2_with_registers
✓ kept
model
hidden_act
gelu
gelu
✓ kept
model
image_size
518
518
✓ kept
model
patch_size
14
14
✓ kept
model
num_channels
3
3
✓ kept
Module classes
Instances of each class in the built model (prefix Dinov2WithRegisters omitted).
Class
Original
Fixture
Linear
72
12
LayerNorm
25
5
LayerScale
24
4
Attention
12
2
Layer
12
2
MLP
12
2
GELUActivation
12
2
Identity
12
2
Conv2d
1
1
Embeddings
1
1
Encoder
1
1
Model
1
1
PatchEmbeddings
1
1
Dropout
1
1
ModuleList
1
1
Config changes
5 fields changed, 28 unchanged.
Field
Original
Fixture
hidden_size
768
64
num_attention_heads
12
4
num_hidden_layers
12
2
out_features
["stage12"]
["stage2"]
out_indices
[12]
[2]
+
\ No newline at end of file
diff --git a/shrink/dinov3-vitl16-pretrain-lvd1689m-mic.html b/shrink/dinov3-vitl16-pretrain-lvd1689m-mic.html
new file mode 100644
index 0000000000000000000000000000000000000000..19e75966d1501e6818d3914e10df586c4ddf0398
--- /dev/null
+++ b/shrink/dinov3-vitl16-pretrain-lvd1689m-mic.html
@@ -0,0 +1,34 @@
+
+
+Shrunk architecture
+
+
Shrunk architecture
huggingface-mic/dinov3-vitl16-pretrain-lvd1689m-mic@ba70f2bd5556b75212ed9bdd17efebacf33b37ae ← camenduru/dinov3-vitl16-pretrain-lvd1689m@3c276edd87d6f6e569ff0c4400e086807d0f3881 · DINOv3ViTModel · random weights
+
Original303.1Mparameters
Fixture116.8Kparameters
Layer stacks1compared
Findings0to review
✓ Same architecture, smaller
Every module class, layer kind and checked property of the original is in the fixture.
Layer stacks
One square per layer, colored by kind: the layer's module classes and type attributes. Hover a square for its index and kind.
model.layer
Original · 24
Fixture · 2
Kind
Original
Fixture
Layer
24
2
✓ kept
Where the parameters are
Share of each model's parameters per component. Vocabulary and patch sizes are kept, so embeddings usually take a larger share of the fixture.
Original
Fixture
Component
Original
Share
Fixture
Share
model.layer
302.3M
99.7%
67.1K
57.4%
embeddings.patch_embeddings
787.5K
0.3%
49.2K
42.1%
embeddings
6.1K
0.0%
384
0.3%
norm
2.0K
0.0%
128
0.1%
Preserved properties
Config
Property
Original
Fixture
model
model_type
dinov3_vit
dinov3_vit
✓ kept
model
hidden_act
gelu
gelu
✓ kept
model
rope_theta
100.0
100.0
✓ kept
model
image_size
224
224
✓ kept
model
patch_size
16
16
✓ kept
model
num_channels
3
3
✓ kept
Module classes
Instances of each class in the built model (prefix DINOv3ViT omitted).
Class
Original
Fixture
Linear
144
12
LayerNorm
49
5
LayerScale
48
4
Attention
24
2
Layer
24
2
MLP
24
2
GELUActivation
24
2
Identity
24
2
Conv2d
1
1
Embeddings
1
1
Encoder
1
1
Model
1
1
RopePositionEmbedding
1
1
ModuleList
1
1
Config changes
6 fields changed, 32 unchanged.
Field
Original
Fixture
hidden_size
1024
64
intermediate_size
4096
128
num_attention_heads
16
4
num_hidden_layers
24
2
out_features
["stage24"]
["stage2"]
out_indices
[24]
[2]
+
\ No newline at end of file
diff --git a/shrink/distilbert-base-uncased-mic.html b/shrink/distilbert-base-uncased-mic.html
new file mode 100644
index 0000000000000000000000000000000000000000..c28b7989061d081ff6f006362deeaf41024c20ff
--- /dev/null
+++ b/shrink/distilbert-base-uncased-mic.html
@@ -0,0 +1,34 @@
+
+
+Shrunk architecture
+
+
Shrunk architecture
huggingface-mic/distilbert-base-uncased-mic@c65e2b7864ac5e15a46a61bdb7c1c89da10ab09c ← distilbert/distilbert-base-uncased@12040accade4e8a0f71eabdb258fecc2e7e948be · DistilBertForMaskedLM · random weights · built before the shrink rule (no recorded recipe)
+
Original67.0Mparameters
Fixture2.1Mparameters
Layer stacks1compared
Findings0to review
✓ Same architecture, smaller
Every module class, layer kind and checked property of the original is in the fixture.
Layer stacks
One square per layer, colored by kind: the layer's module classes and type attributes. Hover a square for its index and kind.
distilbert.transformer.layer
Original · 6
Fixture · 2
Kind
Original
Fixture
TransformerBlock
6
2
✓ kept
Where the parameters are
Share of each model's parameters per component. Vocabulary and patch sizes are kept, so embeddings usually take a larger share of the fixture.
Original
Fixture
Component
Original
Share
Fixture
Share
distilbert.transformer.layer
42.5M
63.5%
66.9K
3.2%
distilbert.embeddings
23.8M
35.6%
2.0M
95.1%
vocab_transform
590.6K
0.9%
4.2K
0.2%
vocab_projector
30.5K
0.0%
30.5K
1.5%
vocab_layer_norm
1.5K
0.0%
128
0.0%
Preserved properties
Config
Property
Original
Fixture
model
model_type
distilbert
distilbert
✓ kept
model
vocab_size
30522
30522
✓ kept
model
tie_word_embeddings
True
True
✓ kept
model
max_position_embeddings
512
512
✓ kept
Module classes
Instances of each class in the built model (prefix DistilBert omitted).
Class
Original
Fixture
Linear
38
14
LayerNorm
14
6
Dropout
13
5
GELUActivation
7
3
SelfAttention
6
2
FFN
6
2
TransformerBlock
6
2
Embedding
2
2
CrossEntropyLoss
1
1
ForMaskedLM
1
1
Model
1
1
Embeddings
1
1
ModuleList
1
1
Transformer
1
1
Config changes
4 fields changed, 23 unchanged.
Field
Original
Fixture
dim
768
64
hidden_dim
3072
128
n_heads
12
4
n_layers
6
2
+
\ No newline at end of file
diff --git a/shrink/docling-layout-heron-mic.html b/shrink/docling-layout-heron-mic.html
new file mode 100644
index 0000000000000000000000000000000000000000..9edd384edb7bf99920a032106bde8482c37ee562
--- /dev/null
+++ b/shrink/docling-layout-heron-mic.html
@@ -0,0 +1,34 @@
+
+
+Shrunk architecture
+
+
Shrunk architecture
huggingface-mic/docling-layout-heron-mic@a4c39589d12b25a98e2c975388b2629748602771 ← docling-project/docling-layout-heron@8f39ad3c0b4c58e9c2d2c84a38465abf757272d8 · RTDetrV2ForObjectDetection · random weights
+
Original42.8Mparameters
Fixture10.9Mparameters
Layer stacks11compared
Findings0to review
✓ Same architecture, smaller
Every module class, layer kind and checked property of the original is in the fixture.
Layer stacks
One square per layer, colored by kind: the layer's module classes and type attributes. Hover a square for its index and kind.
model.backbone.model.encoder.stages
Original · 4
Fixture · 4
Kind
Original
Fixture
no AvgPool2d
1
1
✓ kept
AvgPool2d
3
3
✓ kept
model.encoder_input_proj
Original · 3
Fixture · 3
Kind
Original
Fixture
Conv2d
3
3
✓ kept
model.encoder.aifi
Original · 1
Fixture · 1
Kind
Original
Fixture
AIFILayer
1
1
✓ kept
model.encoder.lateral_convs
Original · 2
Fixture · 2
Kind
Original
Fixture
ConvNormLayer
2
2
✓ kept
model.encoder.fpn_blocks
Original · 2
Fixture · 2
Kind
Original
Fixture
RepVggBlock
2
2
✓ kept
model.encoder.downsample_convs
Original · 2
Fixture · 2
Kind
Original
Fixture
ConvNormLayer
2
2
✓ kept
model.encoder.pan_blocks
Original · 2
Fixture · 2
Kind
Original
Fixture
RepVggBlock
2
2
✓ kept
model.decoder_input_proj
Original · 3
Fixture · 3
Kind
Original
Fixture
Conv2d
3
3
✓ kept
model.decoder.layers
Original · 6
Fixture · 2
Kind
Original
Fixture
DecoderLayer
6
2
✓ kept
model.decoder.bbox_embed
Original · 6
Fixture · 2
Kind
Original
Fixture
Linear
6
2
✓ kept
bbox_embed
Original · 6
Fixture · 2
Kind
Original
Fixture
Linear
6
2
✓ kept
Where the parameters are
Share of each model's parameters per component. Vocabulary and patch sizes are kept, so embeddings usually take a larger share of the fixture.
Original
Fixture
Component
Original
Share
Fixture
Share
model.backbone.model.encoder.stages
23.4M
54.8%
32.4K
0.3%
model.decoder.layers
6.0M
14.0%
102.6K
0.9%
model.encoder.fpn_blocks
4.5M
10.4%
4.5M
41.0%
model.encoder.pan_blocks
4.5M
10.4%
4.5M
41.0%
model.encoder.downsample_convs
1.2M
2.8%
1.2M
10.8%
model.encoder_input_proj
919.0K
2.1%
58.9K
0.5%
model.decoder.bbox_embed
795.7K
1.9%
17.2K
0.2%
9 others
1.5M
3.5%
566.5K
5.2%
Preserved properties
Config
Property
Original
Fixture
model
model_type
rt_detr_v2
rt_detr_v2
✓ kept
model
tie_word_embeddings
True
True
✓ kept
backbone_config
model_type
rt_detr_resnet
rt_detr_resnet
✓ kept
backbone_config
hidden_act
relu
relu
✓ kept
backbone_config
num_channels
3
3
✓ kept
Module classes
Instances of each class in the built model (prefix RTDetrV2 omitted).
Class
Original
Fixture
Linear
121
49
Conv2d
97
61
ReLU
57
17
Identity
56
32
FrozenBatchNorm2d
55
19
RTDetrResNetConvLayer
51
15
BatchNorm2d
42
42
ConvNormLayer
36
36
Sequential
35
23
ModuleList
28
20
SiLUActivation
24
24
LayerNorm
21
9
RTDetrResNetBottleNeckLayer
16
4
MLPPredictionHead
14
6
RepVggBlock
12
12
MLP
7
3
SelfAttention
7
3
DecoderLayer
6
2
MultiscaleDeformableAttention
6
2
RTDetrResNetShortCut
4
4
RTDetrResNetStage
4
4
CSPRepLayer
4
4
AvgPool2d
3
3
Embedding
1
1
GELUActivation
1
1
MaxPool2d
1
1
RTDetrResNetBackbone
1
1
RTDetrResNetEmbeddings
1
1
RTDetrResNetEncoder
1
1
AIFILayer
1
1
ConvEncoder
1
1
Decoder
1
1
EncoderLayer
1
1
ForObjectDetection
1
1
HybridEncoder
1
1
Model
1
1
SinePositionEmbedding
1
1
Config changes
9 fields changed, 74 unchanged.
Field
Original
Fixture
backbone_config.depths
[3, 4, 6, 3]
[1, 1, 1, 1]
backbone_config.hidden_sizes
[256, 512, 1024, 2048]
[16, 32, 64, 128]
d_model
256
64
decoder_attention_heads
8
4
decoder_ffn_dim
1024
128
decoder_layers
6
2
encoder_attention_heads
8
4
encoder_ffn_dim
1024
128
encoder_in_channels
[512, 1024, 2048]
[32, 64, 128]
+
\ No newline at end of file
diff --git a/shrink/dpt-large-mic.html b/shrink/dpt-large-mic.html
new file mode 100644
index 0000000000000000000000000000000000000000..29b3939f17a9357532f83137f01cdb30389f6bd3
--- /dev/null
+++ b/shrink/dpt-large-mic.html
@@ -0,0 +1,34 @@
+
+
+Shrunk architecture
+
+
Every module class, layer kind and checked property of the original is in the fixture.
Layer stacks
One square per layer, colored by kind: the layer's module classes and type attributes. Hover a square for its index and kind.
dpt.encoder.layer
Original · 24
Fixture · 4
Kind
Original
Fixture
ViTLayer
24
4
✓ kept
neck.reassemble_stage.layers
Original · 4
Fixture · 4
Kind
Original
Fixture
ConvTranspose2d
2
2
✓ kept
no ConvTranspose2d
2
2
✓ kept
neck.reassemble_stage.readout_projects
Original · 4
Fixture · 4
Kind
Original
Fixture
Linear
4
4
✓ kept
neck.fusion_stage.layers
Original · 4
Fixture · 4
Kind
Original
Fixture
FeatureFusionLayer
4
4
✓ kept
Where the parameters are
Share of each model's parameters per component. Vocabulary and patch sizes are kept, so embeddings usually take a larger share of the fixture.
Original
Fixture
Component
Original
Share
Fixture
Share
dpt.encoder.layer
302.3M
88.1%
133.9K
12.8%
neck.reassemble_stage.layers
14.4M
4.2%
56.6K
5.4%
neck.fusion_stage.layers
9.7M
2.8%
607.5K
58.1%
neck.reassemble_stage.readout_projects
8.4M
2.4%
33.0K
3.2%
neck.convs
6.5M
1.9%
101.4K
9.7%
dpt.embeddings
1.4M
0.4%
86.2K
8.2%
head.head
332.0K
0.1%
27.7K
2.7%
1 others
2.0K
0.0%
128
0.0%
Preserved properties
Config
Property
Original
Fixture
model
model_type
dpt
dpt
✓ kept
model
hidden_act
gelu
gelu
✓ kept
model
image_size
384
384
✓ kept
model
patch_size
16
16
✓ kept
model
num_channels
3
3
✓ kept
Module classes
Instances of each class in the built model (prefix DPT omitted).
Class
Original
Fixture
Linear
148
28
Dropout
49
9
LayerNorm
49
9
Conv2d
33
33
GELUActivation
28
8
SelfAttention
24
4
ViTAttention
24
4
ViTIntermediate
24
4
ViTLayer
24
4
ViTOutput
24
4
ViTSelfOutput
24
4
ReLU
18
18
PreActResidualLayer
8
8
ModuleList
5
5
Sequential
5
5
FeatureFusionLayer
4
4
ReassembleLayer
4
4
ConvTranspose2d
2
2
DepthEstimationHead
1
1
FeatureFusionStage
1
1
ForDepthEstimation
1
1
Model
1
1
Neck
1
1
ReassembleStage
1
1
ViTEmbeddings
1
1
ViTEncoder
1
1
ViTPatchEmbeddings
1
1
Identity
1
1
Upsample
1
1
Config changes
7 fields changed, 34 unchanged.
Field
Original
Fixture
backbone_out_indices
[5, 11, 17, 23]
[0, 1, 2, 3]
fusion_hidden_size
256
64
hidden_size
1024
64
intermediate_size
4096
128
neck_hidden_sizes
[256, 512, 1024, 1024]
[16, 32, 64, 64]
num_attention_heads
16
4
num_hidden_layers
24
4
+
\ No newline at end of file
diff --git a/shrink/electra-small-discriminator-mic.html b/shrink/electra-small-discriminator-mic.html
new file mode 100644
index 0000000000000000000000000000000000000000..1d1848adc3a0cc204a4e2fc8e7a3f6e3b858c729
--- /dev/null
+++ b/shrink/electra-small-discriminator-mic.html
@@ -0,0 +1,34 @@
+
+
+Shrunk architecture
+
+
Shrunk architecture
huggingface-mic/electra-small-discriminator-mic@ac7792ec804c827b79e2d41cc27764deb121a907 ← google/electra-small-discriminator@fa8239aadc095e9164941d05878b98afe9b953c3 · ElectraForPreTraining · random weights · built before the shrink rule (no recorded recipe)
+
Original13.5Mparameters
Fixture4.1Mparameters
Layer stacks1compared
Findings0to review
✓ Same architecture, smaller
Every module class, layer kind and checked property of the original is in the fixture.
Layer stacks
One square per layer, colored by kind: the layer's module classes and type attributes. Hover a square for its index and kind.
electra.encoder.layer
Original · 12
Fixture · 2
Kind
Original
Fixture
Layer
12
2
✓ kept
Where the parameters are
Share of each model's parameters per component. Vocabulary and patch sizes are kept, so embeddings usually take a larger share of the fixture.
Original
Fixture
Component
Original
Share
Fixture
Share
electra.encoder.layer
9.5M
69.9%
66.9K
1.7%
electra.embeddings
4.0M
29.3%
4.0M
98.0%
discriminator_predictions.dense
65.8K
0.5%
4.2K
0.1%
electra.embeddings_project
33.0K
0.2%
8.3K
0.2%
discriminator_predictions.dense_prediction
257
0.0%
65
0.0%
Preserved properties
Config
Property
Original
Fixture
model
model_type
electra
electra
✓ kept
model
vocab_size
30522
30522
✓ kept
model
tie_word_embeddings
True
True
✓ kept
model
hidden_act
gelu
gelu
✓ kept
model
max_position_embeddings
512
512
✓ kept
Module classes
Instances of each class in the built model (prefix Electra omitted).
Class
Original
Fixture
Linear
75
15
Dropout
37
7
LayerNorm
25
5
GELUActivation
13
3
Attention
12
2
Intermediate
12
2
Layer
12
2
Output
12
2
SelfAttention
12
2
SelfOutput
12
2
Embedding
3
3
DiscriminatorPredictions
1
1
Embeddings
1
1
Encoder
1
1
ForPreTraining
1
1
Model
1
1
ModuleList
1
1
Config changes
3 fields changed, 32 unchanged.
Field
Original
Fixture
hidden_size
256
64
intermediate_size
1024
128
num_hidden_layers
12
2
+
\ No newline at end of file
diff --git a/shrink/embeddinggemma-2-mic.html b/shrink/embeddinggemma-2-mic.html
new file mode 100644
index 0000000000000000000000000000000000000000..3c6421de6fa2d5ce2706e849f217426dbbb07bef
--- /dev/null
+++ b/shrink/embeddinggemma-2-mic.html
@@ -0,0 +1,34 @@
+
+
+Shrunk architecture
+
+
Shrunk architecture
huggingface-mic/embeddinggemma-2-mic@154c2d84cda18ee708d45273b3e509a34dc9d5cb ← google/embeddinggemma-2@f9972ac22e696a105c482202646979e33e4b5a1e · EmbeddingGemma2Model · random weights
+
Original744.4Mparameters
Fixture19.4Mparameters
Layer stacks3compared
Findings0to review
✓ Same architecture, smaller
Every module class, layer kind and checked property of the original is in the fixture.
Layer stacks
One square per layer, colored by kind: the layer's module classes and type attributes. Hover a square for its index and kind.
vision_tower.encoder.layers
Original · 16
Fixture · 2
Kind
Original
Fixture
Gemma4VisionEncoderLayer
16
2
✓ kept
language_model.layers
Original · 24
Fixture · 2
Kind
Original
Fixture
EncoderLayer
24
2
✓ kept
audio_tower.layers
Original · 12
Fixture · 2
Kind
Original
Fixture
Gemma4AudioLayer
12
2
✓ kept
Where the parameters are
Share of each model's parameters per component. Vocabulary and patch sizes are kept, so embeddings usually take a larger share of the fixture.
Original
Fixture
Component
Original
Share
Fixture
Share
audio_tower.layers
302.2M
40.6%
198.4K
1.0%
vision_tower.encoder.layers
151.0M
20.3%
82.5K
0.4%
language_model.embed_tokens
134.2M
18.0%
16.8M
86.7%
language_model.layers
130.1M
17.5%
521.9K
2.7%
vision_tower.patch_embedder
16.3M
2.2%
1.4M
7.0%
language_model.ple
6.3M
0.8%
66.0K
0.3%
audio_tower.output_proj
1.6M
0.2%
99.8K
0.5%
5 others
2.7M
0.4%
255.3K
1.3%
Preserved properties
Config
Property
Original
Fixture
model
model_type
embedding_gemma2
embedding_gemma2
✓ kept
text_config
attention style
GQA (4 heads, 2 KV)
GQA (4 heads, 2 KV)
✓ kept
text_config
model_type
embedding_gemma2_text
embedding_gemma2_text
✓ kept
text_config
vocab_size
262144
262144
✓ kept
text_config
max_position_embeddings
262144
262144
✓ kept
vision_config
attention style
MHA (12 heads, 12 KV)
MHA (4 heads, 4 KV)
✓ kept
vision_config
RoPE type
axial
axial
✓ kept
vision_config
model_type
gemma4_vision
gemma4_vision
✓ kept
vision_config
max_position_embeddings
131072
131072
✓ kept
vision_config
patch_size
16
16
✓ kept
audio_config
model_type
gemma4_audio
gemma4_audio
✓ kept
audio_config
hidden_act
silu
silu
✓ kept
Module classes
Instances of each class in the built model (prefix EmbeddingGemma2 omitted).
+
\ No newline at end of file
diff --git a/shrink/esm2_t6_8M_UR50D-mic.html b/shrink/esm2_t6_8M_UR50D-mic.html
new file mode 100644
index 0000000000000000000000000000000000000000..ff7449cfd49e2d875c9e6d623d2040eaef4077b6
--- /dev/null
+++ b/shrink/esm2_t6_8M_UR50D-mic.html
@@ -0,0 +1,34 @@
+
+
+Shrunk architecture
+
+
Shrunk architecture
huggingface-mic/esm2_t6_8M_UR50D-mic@4b96e1d6e983ef0096ac5ca7257ccb03fdd256ca ← facebook/esm2_t6_8M_UR50D@c731040fcd8d73dceaa04b0a8e6329b345b0f5df · EsmForMaskedLM · random weights · built before the shrink rule (no recorded recipe)
+
Original7.5Mparameters
Fixture73.5Kparameters
Layer stacks1compared
Findings0to review
✓ Same architecture, smaller
Every module class, layer kind and checked property of the original is in the fixture.
Layer stacks
One square per layer, colored by kind: the layer's module classes and type attributes. Hover a square for its index and kind.
esm.encoder.layer
Original · 6
Fixture · 2
Kind
Original
Fixture
Layer
6
2
✓ kept
Where the parameters are
Share of each model's parameters per component. Vocabulary and patch sizes are kept, so embeddings usually take a larger share of the fixture.
Original
Fixture
Component
Original
Share
Fixture
Share
esm.encoder.layer
7.4M
98.5%
66.9K
91.1%
lm_head.dense
102.7K
1.4%
4.2K
5.7%
esm.embeddings
10.6K
0.1%
2.1K
2.9%
lm_head.layer_norm
640
0.0%
128
0.2%
esm.encoder
640
0.0%
128
0.2%
esm.contact_head
121
0.0%
9
0.0%
lm_head
33
0.0%
33
0.0%
Preserved properties
Config
Property
Original
Fixture
model
model_type
esm
esm
✓ kept
model
vocab_size
33
33
✓ kept
model
tie_word_embeddings
True
True
✓ kept
model
hidden_act
gelu
gelu
✓ kept
model
max_position_embeddings
1026
1026
✓ kept
model
rope_theta
10000.0
10000.0
✓ kept
Module classes
Instances of each class in the built model (prefix Esm omitted).
Class
Original
Fixture
Linear
39
15
LayerNorm
14
6
Dropout
13
5
Attention
6
2
Intermediate
6
2
Layer
6
2
Output
6
2
SelfAttention
6
2
SelfOutput
6
2
Embedding
1
1
ContactPredictionHead
1
1
Embeddings
1
1
Encoder
1
1
ForMaskedLM
1
1
LMHead
1
1
Model
1
1
RotaryEmbedding
1
1
ModuleList
1
1
Sigmoid
1
1
Config changes
4 fields changed, 32 unchanged.
Field
Original
Fixture
hidden_size
320
64
intermediate_size
1280
128
num_attention_heads
20
4
num_hidden_layers
6
2
+
\ No newline at end of file
diff --git a/shrink/falcon-7b-mic.html b/shrink/falcon-7b-mic.html
new file mode 100644
index 0000000000000000000000000000000000000000..63d30c5c1cfa99307e9408d346871377ecee1e71
--- /dev/null
+++ b/shrink/falcon-7b-mic.html
@@ -0,0 +1,34 @@
+
+
+Shrunk architecture
+
+
Shrunk architecture
huggingface-mic/falcon-7b-mic@c2ffcf79ca756a8157331b274fea56a5485a34d0 ← tiiuae/falcon-7b@ec89142b67d748a1865ea4451372db8313ada0d8 · FalconForCausalLM · random weights
+
Original6.9Bparameters
Fixture8.8Mparameters
Layer stacks1compared
Findings0to review
✓ Same architecture, smaller
Every module class, layer kind and checked property of the original is in the fixture.
Layer stacks
One square per layer, colored by kind: the layer's module classes and type attributes. Hover a square for its index and kind.
transformer.h
Original · 32
Fixture · 2
Kind
Original
Fixture
DecoderLayer
32
2
✓ kept
Where the parameters are
Share of each model's parameters per component. Vocabulary and patch sizes are kept, so embeddings usually take a larger share of the fixture.
Original
Fixture
Component
Original
Share
Fixture
Share
transformer.h
6.6B
95.7%
4.7M
52.9%
transformer.word_embeddings
295.5M
4.3%
4.2M
47.1%
transformer.ln_f
9.1K
0.0%
128
0.0%
Preserved properties
Config
Property
Original
Fixture
model
RoPE type
default
default
✓ kept
model
model_type
falcon
falcon
✓ kept
model
vocab_size
65024
65024
✓ kept
model
tie_word_embeddings
True
True
✓ kept
model
max_position_embeddings
2048
2048
✓ kept
Module classes
Instances of each class in the built model (prefix Falcon omitted).
Class
Original
Fixture
Linear
128
8
LayerNorm
33
3
Dropout
32
2
Attention
32
2
DecoderLayer
32
2
MLP
32
2
GELUActivation
32
2
Embedding
1
1
ForCausalLM
1
1
Model
1
1
RotaryEmbedding
1
1
Linear
1
1
ModuleList
1
1
Config changes
3 fields changed, 32 unchanged.
Field
Original
Fixture
hidden_size
4544
64
num_attention_heads
71
4
num_hidden_layers
32
2
+
\ No newline at end of file
diff --git a/shrink/fnet-base-mic.html b/shrink/fnet-base-mic.html
new file mode 100644
index 0000000000000000000000000000000000000000..f33490964dd6ac942fba63d30f4c662f915e939b
--- /dev/null
+++ b/shrink/fnet-base-mic.html
@@ -0,0 +1,34 @@
+
+
+Shrunk architecture
+
+
Shrunk architecture
huggingface-mic/fnet-base-mic@0b11088ebc19ded28553b5f9b49c0c66d5d4c346 ← google/fnet-base@d89b6fad3cf5384848b783dc480f9685f49d008c · FNetForPreTraining · random weights
+
Original83.5Mparameters
Fixture2.2Mparameters
Layer stacks1compared
Findings0to review
✓ Same architecture, smaller
Every module class, layer kind and checked property of the original is in the fixture.
Layer stacks
One square per layer, colored by kind: the layer's module classes and type attributes. Hover a square for its index and kind.
fnet.encoder.layer
Original · 12
Fixture · 2
Kind
Original
Fixture
Layer
12
2
✓ kept
Where the parameters are
Share of each model's parameters per component. Vocabulary and patch sizes are kept, so embeddings usually take a larger share of the fixture.
Original
Fixture
Component
Original
Share
Fixture
Share
fnet.encoder.layer
56.7M
67.9%
33.7K
1.6%
fnet.embeddings
25.6M
30.6%
2.1M
96.6%
cls.predictions
624.1K
0.7%
36.3K
1.7%
fnet.pooler
590.6K
0.7%
4.2K
0.2%
cls.seq_relationship
1.5K
0.0%
130
0.0%
Preserved properties
Config
Property
Original
Fixture
model
model_type
fnet
fnet
✓ kept
model
vocab_size
32000
32000
✓ kept
model
tie_word_embeddings
True
True
✓ kept
model
hidden_act
gelu_new
gelu_new
✓ kept
model
max_position_embeddings
512
512
✓ kept
Module classes
Instances of each class in the built model (prefix FNet omitted).
Class
Original
Fixture
Linear
29
9
LayerNorm
26
6
Dropout
13
3
NewGELUActivation
13
3
BasicFourierTransform
12
2
BasicOutput
12
2
FourierTransform
12
2
Intermediate
12
2
Layer
12
2
Output
12
2
Embedding
3
3
Embeddings
1
1
Encoder
1
1
ForPreTraining
1
1
LMPredictionHead
1
1
Model
1
1
Pooler
1
1
PreTrainingHeads
1
1
PredictionHeadTransform
1
1
ModuleList
1
1
Tanh
1
1
Config changes
3 fields changed, 25 unchanged.
Field
Original
Fixture
hidden_size
768
64
intermediate_size
3072
128
num_hidden_layers
12
2
+
\ No newline at end of file
diff --git a/shrink/gemma-2-2b-it-mic.html b/shrink/gemma-2-2b-it-mic.html
new file mode 100644
index 0000000000000000000000000000000000000000..b1eb4b85463874281129a3e83eda961e67674f9d
--- /dev/null
+++ b/shrink/gemma-2-2b-it-mic.html
@@ -0,0 +1,34 @@
+
+
+Shrunk architecture
+
+
Shrunk architecture
huggingface-mic/gemma-2-2b-it-mic@e3af84449295759dc428e54735fc829837f32d81 ← google/gemma-2-2b-it@299a8560bedf22ed1c72a8a11e7dce4a7f9f51f8 · Gemma2ForCausalLM · random weights · built before the shrink rule (no recorded recipe)
+
Original2.6Bparameters
Fixture16.5Mparameters
Layer stacks1compared
Findings0to review
✓ Same architecture, smaller
Every module class, layer kind and checked property of the original is in the fixture.
Layer stacks
One square per layer, colored by kind: the layer's module classes and type attributes. Hover a square for its index and kind.
model.layers
Original · 26
Fixture · 2
Kind
Original
Fixture
sliding_attention
13
1
✓ kept
full_attention
13
1
✓ kept
Where the parameters are
Share of each model's parameters per component. Vocabulary and patch sizes are kept, so embeddings usually take a larger share of the fixture.
Original
Fixture
Component
Original
Share
Fixture
Share
model.layers
2.0B
77.4%
74.2K
0.5%
model.embed_tokens
589.8M
22.6%
16.4M
99.5%
model.norm
2.3K
0.0%
64
0.0%
Preserved properties
Config
Property
Original
Fixture
model
attention style
GQA (8 heads, 4 KV)
GQA (4 heads, 2 KV)
✓ kept
model
RoPE type
default
default
✓ kept
model
model_type
gemma2
gemma2
✓ kept
model
vocab_size
256000
256000
✓ kept
model
tie_word_embeddings
True
True
✓ kept
model
hidden_act
gelu_pytorch_tanh
gelu_pytorch_tanh
✓ kept
model
max_position_embeddings
8192
8192
✓ kept
Module classes
Instances of each class in the built model (prefix Gemma2 omitted).
+
\ No newline at end of file
diff --git a/shrink/gemma-2b-mic.html b/shrink/gemma-2b-mic.html
new file mode 100644
index 0000000000000000000000000000000000000000..1e377fe171b87006277293aa2e277bf5d5e969d2
--- /dev/null
+++ b/shrink/gemma-2b-mic.html
@@ -0,0 +1,34 @@
+
+
+Shrunk architecture
+
+
Shrunk architecture
huggingface-mic/gemma-2b-mic@7ce6a0c1274da42dd2eb5c07e703e6198b444783 ← google/gemma-2b@9cf48e52b224239de00d483ec8eb84fb8d0f3a3a · GemmaForCausalLM · random weights
+
Original2.5Bparameters
Fixture16.5Mparameters
Layer stacks1compared
Findings0to review
✓ Same architecture, smaller
Every module class, layer kind and checked property of the original is in the fixture.
Layer stacks
One square per layer, colored by kind: the layer's module classes and type attributes. Hover a square for its index and kind.
model.layers
Original · 18
Fixture · 2
Kind
Original
Fixture
DecoderLayer
18
2
✓ kept
Where the parameters are
Share of each model's parameters per component. Vocabulary and patch sizes are kept, so embeddings usually take a larger share of the fixture.
Original
Fixture
Component
Original
Share
Fixture
Share
model.layers
2.0B
79.1%
69.9K
0.4%
model.embed_tokens
524.3M
20.9%
16.4M
99.6%
model.norm
2.0K
0.0%
64
0.0%
Preserved properties
Config
Property
Original
Fixture
model
attention style
MQA (8 heads, 1 KV)
MQA (4 heads, 1 KV)
✓ kept
model
RoPE type
default
default
✓ kept
model
model_type
gemma
gemma
✓ kept
model
vocab_size
256000
256000
✓ kept
model
tie_word_embeddings
True
True
✓ kept
model
hidden_act
gelu_pytorch_tanh
gelu_pytorch_tanh
✓ kept
model
max_position_embeddings
8192
8192
✓ kept
Module classes
Instances of each class in the built model (prefix Gemma omitted).
Class
Original
Fixture
Linear
127
15
RMSNorm
37
5
GELUTanh
18
2
Attention
18
2
DecoderLayer
18
2
MLP
18
2
ForCausalLM
1
1
Model
1
1
RotaryEmbedding
1
1
TextScaledWordEmbedding
1
1
ModuleList
1
1
Config changes
5 fields changed, 24 unchanged.
Field
Original
Fixture
head_dim
256
16
hidden_size
2048
64
intermediate_size
16384
128
num_attention_heads
8
4
num_hidden_layers
18
2
+
\ No newline at end of file
diff --git a/shrink/gemma-3n-E2B-it-mic.html b/shrink/gemma-3n-E2B-it-mic.html
new file mode 100644
index 0000000000000000000000000000000000000000..2f17c10c11a8b5893413abeea5c0434a2eb56e39
--- /dev/null
+++ b/shrink/gemma-3n-E2B-it-mic.html
@@ -0,0 +1,34 @@
+
+
+Shrunk architecture
+
+
Shrunk architecture
huggingface-mic/gemma-3n-E2B-it-mic@bd78bb3f74709093f27ded6bdf0d329ffcf01493 ← google/gemma-3n-E2B-it@5e092ebca197cdcd8d8b195040accf22693501bc · Gemma3nForConditionalGeneration · random weights · built before the shrink rule (no recorded recipe)
+
Original5.4Bparameters
Fixture581.1Mparameters
Layer stacks2compared
Findings0to review
✓ Same architecture, smaller
Every module class, layer kind and checked property of the original is in the fixture.
Layer stacks
One square per layer, colored by kind: the layer's module classes and type attributes. Hover a square for its index and kind.
model.language_model.layers
Original · 30
Fixture · 4
Kind
Original
Fixture
sliding_attention
24
2
✓ kept
full_attention
6
2
✓ kept
model.audio_tower.conformer
Original · 12
Fixture · 12
Kind
Original
Fixture
AudioConformerBlock
12
12
✓ kept
Where the parameters are
Share of each model's parameters per component. Vocabulary and patch sizes are kept, so embeddings usually take a larger share of the fixture.
Original
Fixture
Component
Original
Share
Fixture
Share
model.language_model
2.6B
47.8%
285.3M
49.1%
model.language_model.layers
1.8B
34.0%
306.6K
0.1%
model.audio_tower.conformer
679.7M
12.5%
1.2M
0.2%
model.vision_tower
294.1M
5.4%
294.1M
50.6%
model.embed_vision
4.5M
0.1%
12.4K
0.0%
model.embed_audio
3.3M
0.1%
12.4K
0.0%
model.audio_tower
1.6M
0.0%
103.7K
0.0%
Preserved properties
Config
Property
Original
Fixture
model
model_type
gemma3n
gemma3n
✓ kept
model
tie_word_embeddings
True
True
✓ kept
text_config
attention style
GQA (8 heads, 2 KV)
GQA (4 heads, 2 KV)
✓ kept
text_config
model_type
gemma3n_text
gemma3n_text
✓ kept
text_config
vocab_size
262400
262400
✓ kept
text_config
tie_word_embeddings
True
True
✓ kept
text_config
max_position_embeddings
32768
32768
✓ kept
vision_config
model_type
gemma3n_vision
gemma3n_vision
✓ kept
vision_config
vocab_size
128
128
✓ kept
audio_config
model_type
gemma3n_audio
gemma3n_audio
✓ kept
audio_config
vocab_size
128
128
✓ kept
Module classes
Instances of each class in the built model (prefix Gemma3n omitted).
+
\ No newline at end of file
diff --git a/shrink/gemma-4-12B-it-mic.html b/shrink/gemma-4-12B-it-mic.html
new file mode 100644
index 0000000000000000000000000000000000000000..eef052cef266a8836dd2abda5facc49bad800fa3
--- /dev/null
+++ b/shrink/gemma-4-12B-it-mic.html
@@ -0,0 +1,34 @@
+
+
+Shrunk architecture
+
+
Shrunk architecture
huggingface-mic/gemma-4-12B-it-mic@a190e22989d3fddef2c51348c8b282c409c20dc3 ← google/gemma-4-12B-it@707f0a3b8a3c7ad586ed01e27eafbad8a27dd0f7 · Gemma4UnifiedForConditionalGeneration · random weights · built before the shrink rule (no recorded recipe)
+
Original12.0Bparameters
Fixture52.6Mparameters
Layer stacks1compared
Findings0to review
✓ Same architecture, smaller
Every module class, layer kind and checked property of the original is in the fixture.
Layer stacks
One square per layer, colored by kind: the layer's module classes and type attributes. Hover a square for its index and kind.
model.language_model.layers
Original · 48
Fixture · 2
Kind
Original
Fixture
sliding_attention
40
1
✓ kept
full_attention
8
1
✓ kept
Where the parameters are
Share of each model's parameters per component. Vocabulary and patch sizes are kept, so embeddings usually take a larger share of the fixture.
Original
Fixture
Component
Original
Share
Fixture
Share
model.language_model.layers
10.9B
91.1%
357.9K
0.7%
model.language_model
1.0B
8.4%
16.8M
31.9%
model.embed_vision
49.9M
0.4%
35.4M
67.3%
model.embed_audio
2.5M
0.0%
41.0K
0.1%
Preserved properties
Config
Property
Original
Fixture
model
model_type
gemma4_unified
gemma4_unified
✓ kept
model
tie_word_embeddings
True
True
✓ kept
text_config
attention style
GQA (16 heads, 8 KV)
GQA (4 heads, 2 KV)
✓ kept
text_config
model_type
gemma4_unified_text
gemma4_unified_text
✓ kept
text_config
vocab_size
262144
262144
✓ kept
text_config
tie_word_embeddings
True
True
✓ kept
text_config
max_position_embeddings
262144
262144
✓ kept
vision_config
model_type
gemma4_unified_vision
gemma4_unified_vision
✓ kept
vision_config
patch_size
16
16
✓ kept
audio_config
model_type
gemma4_unified_audio
gemma4_unified_audio
✓ kept
Module classes
Instances of each class in the built model (prefix Gemma4Unified omitted).
+
\ No newline at end of file
diff --git a/shrink/gemma3-mic.html b/shrink/gemma3-mic.html
new file mode 100644
index 0000000000000000000000000000000000000000..fb6d4c038ab2baa1bac305c4196fb5c178e236aa
--- /dev/null
+++ b/shrink/gemma3-mic.html
@@ -0,0 +1,34 @@
+
+
+Shrunk architecture
+
+
Shrunk architecture
huggingface-mic/gemma3-mic@97b8077b83d60421590191ff1d19535224e77dce ← google/gemma-3-4b-it@093f9f388b31de276ce2de164bdc2081324b9767 · Gemma3ForConditionalGeneration · random weights · built before the shrink rule (no recorded recipe)
+
Original4.3Bparameters
Fixture17.2Mparameters
Layer stacks2compared
Findings0to review
✓ Same architecture, smaller
Every module class, layer kind and checked property of the original is in the fixture.
Layer stacks
One square per layer, colored by kind: the layer's module classes and type attributes. Hover a square for its index and kind.
model.vision_tower.encoder.layers
Original · 27
Fixture · 2
Kind
Original
Fixture
SiglipEncoderLayer
27
2
✓ kept
model.language_model.layers
Original · 34
Fixture · 2
Kind
Original
Fixture
sliding_attention
29
1
✓ kept
full_attention
5
1
✓ kept
Where the parameters are
Share of each model's parameters per component. Vocabulary and patch sizes are kept, so embeddings usually take a larger share of the fixture.
Original
Fixture
Component
Original
Share
Fixture
Share
model.language_model.layers
3.2B
74.6%
74.3K
0.4%
model.language_model
671.3M
15.6%
16.8M
97.4%
model.vision_tower.encoder.layers
411.5M
9.6%
66.9K
0.4%
model.vision_tower
5.4M
0.1%
300.0K
1.7%
model.multi_modal_projector
3.0M
0.1%
4.2K
0.0%
Preserved properties
Config
Property
Original
Fixture
model
model_type
gemma3
gemma3
✓ kept
model
tie_word_embeddings
True
True
✓ kept
text_config
attention style
GQA (8 heads, 4 KV)
GQA (4 heads, 2 KV)
✓ kept
text_config
model_type
gemma3_text
gemma3_text
✓ kept
text_config
vocab_size
262208
262208
✓ kept
text_config
tie_word_embeddings
True
True
✓ kept
text_config
max_position_embeddings
131072
131072
✓ kept
vision_config
model_type
siglip_vision_model
siglip_vision_model
✓ kept
vision_config
hidden_act
gelu_pytorch_tanh
gelu_pytorch_tanh
✓ kept
vision_config
image_size
896
896
✓ kept
vision_config
patch_size
14
14
✓ kept
vision_config
num_channels
3
3
✓ kept
Module classes
Instances of each class in the built model (prefix Gemma3 omitted).
+
\ No newline at end of file
diff --git a/shrink/got_ocr2-mic.html b/shrink/got_ocr2-mic.html
new file mode 100644
index 0000000000000000000000000000000000000000..b6330eaf5661fc0ca6294c75e588a05eca4496d5
--- /dev/null
+++ b/shrink/got_ocr2-mic.html
@@ -0,0 +1,34 @@
+
+
+Shrunk architecture
+
+
Shrunk architecture
huggingface-mic/got_ocr2-mic@5bfaf853e15a6fdf1c253e1368b81df3ed528ab5 ← stepfun-ai/GOT-OCR-2.0-hf@d3017ef2c2c1395888c8d635c5e0508bcb0ac78d · GotOcr2ForConditionalGeneration · overfit weights · built before the shrink rule (no recorded recipe)
+
Original560.5Mparameters
Fixture13.0Mparameters
Layer stacks2compared
Findings0to review
✓ Same architecture, smaller
Every module class, layer kind and checked property of the original is in the fixture.
Layer stacks
One square per layer, colored by kind: the layer's module classes and type attributes. Hover a square for its index and kind.
model.vision_tower.layers
Original · 12
Fixture · 2
Kind
Original
Fixture
MLPBlock
12
2
✓ kept
model.language_model.layers
Original · 24
Fixture · 2
Kind
Original
Fixture
Qwen2DecoderLayer
24
2
✓ kept
Where the parameters are
Share of each model's parameters per component. Vocabulary and patch sizes are kept, so embeddings usually take a larger share of the fixture.
Original
Fixture
Component
Original
Share
Fixture
Share
model.language_model.layers
308.4M
55.0%
82.6K
0.6%
model.language_model
155.5M
27.7%
9.7M
74.6%
model.vision_tower.layers
85.1M
15.2%
828.2K
6.4%
model.multi_modal_projector
6.9M
1.2%
1.5M
11.4%
model.vision_tower
4.5M
0.8%
918.6K
7.1%
Preserved properties
Config
Property
Original
Fixture
model
model_type
got_ocr2
got_ocr2
✓ kept
model
tie_word_embeddings
True
True
✓ kept
vision_config
model_type
got_ocr2_vision
got_ocr2_vision
✓ kept
vision_config
hidden_act
gelu
gelu
✓ kept
vision_config
image_size
1024
1024
✓ kept
vision_config
patch_size
16
16
✓ kept
vision_config
num_channels
3
3
✓ kept
text_config
attention style
MHA (16 heads, 16 KV)
MHA (4 heads, 4 KV)
✓ kept
text_config
RoPE type
default
default
✓ kept
text_config
model_type
qwen2
qwen2
✓ kept
text_config
vocab_size
151860
151860
✓ kept
text_config
tie_word_embeddings
True
True
✓ kept
text_config
hidden_act
silu
silu
✓ kept
text_config
max_position_embeddings
32768
32768
✓ kept
Module classes
Instances of each class in the built model (prefix GotOcr2 omitted).
Class
Original
Fixture
Linear
218
24
Qwen2RMSNorm
49
5
LayerNorm
24
4
Qwen2Attention
24
2
Qwen2DecoderLayer
24
2
Qwen2MLP
24
2
SiLUActivation
24
2
GELUActivation
12
2
MLPBlock
12
2
VisionAttention
12
2
VisionLayer
12
2
Conv2d
5
5
LayerNorm
2
2
ModuleList
2
2
Embedding
1
1
ForConditionalGeneration
1
1
Model
1
1
MultiModalProjector
1
1
PatchEmbeddings
1
1
VisionEncoder
1
1
VisionNeck
1
1
Qwen2Model
1
1
Qwen2RotaryEmbedding
1
1
Config changes
9 fields changed, 60 unchanged.
Field
Original
Fixture
text_config.hidden_size
1024
64
text_config.intermediate_size
2816
128
text_config.layer_types
24 entries: 24× full_attention
["full_attention", "full_attention"]
text_config.num_attention_heads
16
4
text_config.num_hidden_layers
24
2
text_config.num_key_value_heads
16
4
vision_config.hidden_size
768
64
vision_config.num_attention_heads
12
4
vision_config.num_hidden_layers
12
2
+
\ No newline at end of file
diff --git a/shrink/gpt2-mic.html b/shrink/gpt2-mic.html
new file mode 100644
index 0000000000000000000000000000000000000000..f224fe762bb8393eb12e210e5d9d47ec3299587c
--- /dev/null
+++ b/shrink/gpt2-mic.html
@@ -0,0 +1,34 @@
+
+
+Shrunk architecture
+
+
Shrunk architecture
huggingface-mic/gpt2-mic@0168ed11cf4c9161437d47f144bdf52cdfeddd20 ← openai-community/gpt2@607a30d783dfa663caf39e06633721c8d4cfcd7e · GPT2LMHeadModel · random weights · built before the shrink rule (no recorded recipe)
+
Original124.4Mparameters
Fixture3.4Mparameters
Layer stacks1compared
Findings0to review
✓ Same architecture, smaller
Every module class, layer kind and checked property of the original is in the fixture.
Layer stacks
One square per layer, colored by kind: the layer's module classes and type attributes. Hover a square for its index and kind.
transformer.h
Original · 12
Fixture · 2
Kind
Original
Fixture
GPT2Block
12
2
✓ kept
Where the parameters are
Share of each model's parameters per component. Vocabulary and patch sizes are kept, so embeddings usually take a larger share of the fixture.
Original
Fixture
Component
Original
Share
Fixture
Share
transformer.h
85.1M
68.3%
100.0K
3.0%
transformer.wte
38.6M
31.0%
3.2M
95.1%
transformer.wpe
786.4K
0.6%
65.5K
1.9%
transformer.ln_f
1.5K
0.0%
128
0.0%
Preserved properties
Config
Property
Original
Fixture
model
model_type
gpt2
gpt2
✓ kept
model
vocab_size
50257
50257
✓ kept
model
tie_word_embeddings
True
True
✓ kept
Module classes
Instances of each class in the built model (prefix GPT2LMHead omitted).
Class
Original
Fixture
Conv1D
48
8
Dropout
37
7
LayerNorm
25
5
GPT2Attention
12
2
GPT2Block
12
2
GPT2MLP
12
2
NewGELUActivation
12
2
Embedding
2
2
Model
1
1
GPT2Model
1
1
Linear
1
1
ModuleList
1
1
Config changes
3 fields changed, 34 unchanged.
Field
Original
Fixture
n_embd
768
64
n_head
12
4
n_layer
12
2
+
\ No newline at end of file
diff --git a/shrink/gpt_oss-mic.html b/shrink/gpt_oss-mic.html
new file mode 100644
index 0000000000000000000000000000000000000000..d6c621519e8ecb3e9ca0ad152e11fcf0972f3847
--- /dev/null
+++ b/shrink/gpt_oss-mic.html
@@ -0,0 +1,34 @@
+
+
+Shrunk architecture
+
+
Shrunk architecture
huggingface-mic/gpt_oss-mic@8fea014b7848db31a7032015195dc9b6947e63e2 ← openai/gpt-oss-20b@6cee5e81ee83917806bbde320786a8fb61efebee · GptOssForCausalLM · random weights · built before the shrink rule (no recorded recipe)
+
Original20.9Bparameters
Fixture26.0Mparameters
Layer stacks1compared
Findings0to review
✓ Same architecture, smaller
Every module class, layer kind and checked property of the original is in the fixture.
Layer stacks
One square per layer, colored by kind: the layer's module classes and type attributes. Hover a square for its index and kind.
model.layers
Original · 24
Fixture · 2
Kind
Original
Fixture
sliding_attention
12
1
✓ kept
full_attention
12
1
✓ kept
Where the parameters are
Share of each model's parameters per component. Vocabulary and patch sizes are kept, so embeddings usually take a larger share of the fixture.
Original
Fixture
Component
Original
Share
Fixture
Share
model.layers
19.8B
94.5%
224.9K
0.9%
model.embed_tokens
579.1M
2.8%
12.9M
49.6%
lm_head
579.1M
2.8%
12.9M
49.6%
model.norm
2.9K
0.0%
64
0.0%
Preserved properties
Config
Property
Original
Fixture
model
attention style
GQA (64 heads, 8 KV)
GQA (4 heads, 2 KV)
✓ kept
model
MoE routing
top-4 of 32 experts
top-2 of 4 experts
✓ kept
model
RoPE type
yarn
yarn
✓ kept
model
model_type
gpt_oss
gpt_oss
✓ kept
model
vocab_size
201088
201088
✓ kept
model
tie_word_embeddings
False
False
✓ kept
model
hidden_act
silu
silu
✓ kept
model
max_position_embeddings
131072
131072
✓ kept
Module classes
Instances of each class in the built model (prefix GptOss omitted).
+
\ No newline at end of file
diff --git a/shrink/granite-docling-258M-mic.html b/shrink/granite-docling-258M-mic.html
new file mode 100644
index 0000000000000000000000000000000000000000..580e5b01d8744daf3e50a4a982c28f717f09f151
--- /dev/null
+++ b/shrink/granite-docling-258M-mic.html
@@ -0,0 +1,34 @@
+
+
+Shrunk architecture
+
+
Shrunk architecture
huggingface-mic/granite-docling-258M-mic@100175103449d7ed8e24b4d0cbd79e15671b781c ← ibm-granite/granite-docling-258M@982fe3b40f2fa73c365bdb1bcacf6c81b7184bfe · Idefics3ForConditionalGeneration · random weights
+
Original257.5Mparameters
Fixture6.7Mparameters
Layer stacks2compared
Findings0to review
✓ Same architecture, smaller
Every module class, layer kind and checked property of the original is in the fixture.
Layer stacks
One square per layer, colored by kind: the layer's module classes and type attributes. Hover a square for its index and kind.
model.vision_model.encoder.layers
Original · 12
Fixture · 2
Kind
Original
Fixture
EncoderLayer
12
2
✓ kept
model.text_model.layers
Original · 30
Fixture · 2
Kind
Original
Fixture
LlamaDecoderLayer
30
2
✓ kept
Where the parameters are
Share of each model's parameters per component. Vocabulary and patch sizes are kept, so embeddings usually take a larger share of the fixture.
Original
Fixture
Component
Original
Share
Fixture
Share
model.text_model.layers
106.2M
41.2%
74.0K
1.1%
model.vision_model.encoder.layers
85.1M
33.0%
66.9K
1.0%
model.text_model
57.8M
22.4%
6.4M
95.2%
model.connector
7.1M
2.7%
65.5K
1.0%
model.vision_model
1.4M
0.5%
114.9K
1.7%
Preserved properties
Config
Property
Original
Fixture
model
model_type
idefics3
idefics3
✓ kept
model
vocab_size
100352
100352
✓ kept
model
tie_word_embeddings
True
True
✓ kept
vision_config
model_type
idefics3_vision
idefics3_vision
✓ kept
vision_config
hidden_act
gelu_pytorch_tanh
gelu_pytorch_tanh
✓ kept
vision_config
image_size
512
512
✓ kept
vision_config
patch_size
16
16
✓ kept
vision_config
num_channels
3
3
✓ kept
text_config
attention style
GQA (9 heads, 3 KV)
GQA (4 heads, 2 KV)
✓ kept
text_config
RoPE type
default
default
✓ kept
text_config
model_type
llama
llama
✓ kept
text_config
vocab_size
100352
100352
✓ kept
text_config
tie_word_embeddings
True
True
✓ kept
text_config
hidden_act
silu
silu
✓ kept
text_config
max_position_embeddings
8192
8192
✓ kept
Module classes
Instances of each class in the built model (prefix Idefics3 omitted).
Class
Original
Fixture
Linear
284
28
LlamaRMSNorm
61
5
LlamaAttention
30
2
LlamaDecoderLayer
30
2
LlamaMLP
30
2
SiLUActivation
30
2
LayerNorm
25
5
GELUTanh
12
2
EncoderLayer
12
2
VisionAttention
12
2
VisionMLP
12
2
Embedding
2
2
ModuleList
2
2
Conv2d
1
1
Connector
1
1
Encoder
1
1
ForConditionalGeneration
1
1
Model
1
1
SimpleMLP
1
1
VisionEmbeddings
1
1
VisionTransformer
1
1
LlamaModel
1
1
LlamaRotaryEmbedding
1
1
Config changes
10 fields changed, 59 unchanged.
Field
Original
Fixture
text_config.head_dim
64
16
text_config.hidden_size
576
64
text_config.intermediate_size
1536
128
text_config.num_attention_heads
9
4
text_config.num_hidden_layers
30
2
text_config.num_key_value_heads
3
2
vision_config.hidden_size
768
64
vision_config.intermediate_size
3072
128
vision_config.num_attention_heads
12
4
vision_config.num_hidden_layers
12
2
+
\ No newline at end of file
diff --git a/shrink/granite-vision-4.1-4b-mic.html b/shrink/granite-vision-4.1-4b-mic.html
new file mode 100644
index 0000000000000000000000000000000000000000..b8c0dabce39859153d47819d3c29ca64957ac993
--- /dev/null
+++ b/shrink/granite-vision-4.1-4b-mic.html
@@ -0,0 +1,34 @@
+
+
+Shrunk architecture
+
+
Shrunk architecture
huggingface-mic/granite-vision-4.1-4b-mic@a6d32301c1b54aa5ef889d46d64998c1fb01c7df ← ibm-granite/granite-vision-4.1-4b@37d591f06319e8f1638b5adcf58bdf50e0f84f7a · Granite4VisionForConditionalGeneration · random weights
+
Original4.0Bparameters
Fixture7.2Mparameters
Layer stacks4compared
Findings0to review
✓ Same architecture, smaller
Every module class, layer kind and checked property of the original is in the fixture.
Layer stacks
One square per layer, colored by kind: the layer's module classes and type attributes. Hover a square for its index and kind.
model.vision_tower.encoder.layers
Original · 27
Fixture · 2
Kind
Original
Fixture
SiglipEncoderLayer
27
2
✓ kept
model.language_model.layers
Original · 40
Fixture · 2
Kind
Original
Fixture
TextDecoderLayer
40
2
✓ kept
model.layerwise_projectors
Original · 4
Fixture · 4
Kind
Original
Fixture
Blip2QFormerLayer
4
4
✓ kept
model.spatial_projectors
Original · 4
Fixture · 4
Kind
Original
Fixture
Blip2QFormerLayer
4
4
✓ kept
Where the parameters are
Share of each model's parameters per component. Vocabulary and patch sizes are kept, so embeddings usually take a larger share of the fixture.
Original
Fixture
Component
Original
Share
Fixture
Share
model.language_model.layers
3.1B
78.7%
74.0K
1.0%
model.vision_tower.encoder.layers
411.5M
10.3%
66.9K
0.9%
model.language_model
256.9M
6.4%
6.4M
89.7%
model.spatial_projectors
83.1M
2.1%
239.1K
3.3%
model.layerwise_projectors
83.1M
2.1%
239.1K
3.3%
model.vision_tower
16.8M
0.4%
119.6K
1.7%
model
2.6K
0.0%
64
0.0%
Preserved properties
Config
Property
Original
Fixture
model
model_type
granite4_vision
granite4_vision
✓ kept
model
tie_word_embeddings
True
True
✓ kept
vision_config
model_type
siglip_vision_model
siglip_vision_model
✓ kept
vision_config
hidden_act
gelu_pytorch_tanh
gelu_pytorch_tanh
✓ kept
vision_config
image_size
384
384
✓ kept
vision_config
patch_size
16
16
✓ kept
vision_config
num_channels
3
3
✓ kept
text_config
attention style
GQA (40 heads, 8 KV)
GQA (4 heads, 2 KV)
✓ kept
text_config
RoPE type
default
default
✓ kept
text_config
model_type
granite
granite
✓ kept
text_config
vocab_size
100352
100352
✓ kept
text_config
tie_word_embeddings
True
True
✓ kept
text_config
hidden_act
silu
silu
✓ kept
text_config
max_position_embeddings
131072
131072
✓ kept
qformer_config
model_type
blip_2_qformer
blip_2_qformer
✓ kept
qformer_config
vocab_size
30522
30522
✓ kept
qformer_config
hidden_act
gelu
gelu
✓ kept
qformer_config
max_position_embeddings
2048
2048
✓ kept
Module classes
Instances of each class in the built model (prefix Granite4Vision omitted).
Class
Original
Fixture
Linear
533
117
LayerNorm
96
46
TextRMSNorm
81
5
Dropout
40
40
TextAttention
40
2
TextDecoderLayer
40
2
TextMLP
40
2
SiLUActivation
40
2
GELUTanh
28
3
SiglipMLP
28
3
SiglipAttention
27
2
SiglipEncoderLayer
27
2
Blip2QFormerAttention
16
16
Blip2QFormerMultiHeadAttention
16
16
Blip2QFormerSelfOutput
16
16
ModuleList
12
12
Blip2QFormerEncoder
8
8
Blip2QFormerIntermediate
8
8
Blip2QFormerLayer
8
8
Blip2QFormerModel
8
8
Blip2QFormerOutput
8
8
GELUActivation
8
8
WindowQFormerDownsampler
8
8
Embedding
2
2
Conv2d
1
1
ForConditionalGeneration
1
1
Model
1
1
TextModel
1
1
TextRotaryEmbedding
1
1
MultiheadAttention
1
1
NonDynamicallyQuantizableLinear
1
1
SiglipEncoder
1
1
SiglipMultiheadAttentionPoolingHead
1
1
SiglipVisionEmbeddings
1
1
SiglipVisionModel
1
1
Config changes
15 fields changed, 86 unchanged.
Field
Original
Fixture
deepstack_layer_map
[[-19, 9], [-13, 6], [-7, 3], [-1, 0]]
[[-3, 0], [-2, 0], [-2, 0], [-1, 0]]
qformer_config.encoder_hidden_size
1152
64
qformer_config.hidden_size
1152
64
qformer_config.intermediate_size
3072
128
qformer_config.num_attention_heads
18
1
spatial_target_layers
[12, 15, 18, 21]
[0, 1]
text_config.hidden_size
2560
64
text_config.intermediate_size
8192
128
text_config.num_attention_heads
40
4
text_config.num_hidden_layers
40
2
text_config.num_key_value_heads
8
2
vision_config.hidden_size
1152
64
vision_config.intermediate_size
4304
128
vision_config.num_attention_heads
16
4
vision_config.num_hidden_layers
27
2
+
\ No newline at end of file
diff --git a/shrink/grounding-dino-tiny-mic.html b/shrink/grounding-dino-tiny-mic.html
new file mode 100644
index 0000000000000000000000000000000000000000..ec55da07e3f7ceaecfc57368f75bdd7863e314f6
--- /dev/null
+++ b/shrink/grounding-dino-tiny-mic.html
@@ -0,0 +1,34 @@
+
+
+Shrunk architecture
+
+
Shrunk architecture
huggingface-mic/grounding-dino-tiny-mic@fb35353454540cd677db5cdcf3df93a1b60cca4a ← IDEA-Research/grounding-dino-tiny@a2bb814dd30d776dcf7e30523b00659f4f141c71 · GroundingDinoForObjectDetection · random weights · built before the shrink rule (no recorded recipe)
+
Original172.3Mparameters
Fixture7.7Mparameters
Layer stacks7compared
Findings0to review
✓ Same architecture, smaller
Every module class, layer kind and checked property of the original is in the fixture.
Layer stacks
One square per layer, colored by kind: the layer's module classes and type attributes. Hover a square for its index and kind.
Instances of each class in the built model (prefix GroundingDino omitted).
Class
Original
Fixture
Linear
382
122
LayerNorm
119
43
Dropout
68
18
ModuleList
26
18
GELUActivation
24
6
MultiheadAttention
18
6
ReLU
18
6
MLPPredictionHead
14
6
BertAttention
12
2
BertIntermediate
12
2
BertLayer
12
2
BertOutput
12
2
BertSelfAttention
12
2
BertSelfOutput
12
2
MultiscaleDeformableAttention
12
4
MultiScaleDeformableAttention
12
4
SwinAttention
12
4
SwinLayer
12
4
SwinMLP
12
4
SwinRelativePositionBias
12
4
SwinDropPath
11
3
ContrastiveEmbedding
7
3
BiMultiHeadAttention
6
2
DecoderLayer
6
2
DeformableLayer
6
2
DropPath
6
2
EncoderLayer
6
2
FusionLayer
6
2
TextEnhancerLayer
6
2
Conv2d
5
5
Embedding
4
4
GroupNorm
4
4
Sequential
4
4
SwinStage
4
4
SwinPatchMerging
3
3
BertEmbeddings
1
1
BertEncoder
1
1
BertModel
1
1
ConvEncoder
1
1
ConvModel
1
1
Decoder
1
1
Encoder
1
1
ForObjectDetection
1
1
Model
1
1
SinePositionEmbedding
1
1
Identity
1
1
ModuleDict
1
1
SwinBackbone
1
1
SwinEmbeddings
1
1
SwinEncoder
1
1
SwinModel
1
1
SwinPatchEmbeddings
1
1
Config changes
16 fields changed, 91 unchanged.
Field
Original
Fixture
backbone_config.depths
[2, 2, 6, 2]
[1, 1, 1, 1]
backbone_config.embed_dim
96
64
backbone_config.hidden_size
768
64
backbone_config.num_heads
[3, 6, 12, 24]
[2, 4, 8, 16]
backbone_config.num_layers
4
2
d_model
256
64
decoder_attention_heads
8
4
decoder_ffn_dim
2048
128
decoder_layers
6
2
encoder_attention_heads
8
4
encoder_ffn_dim
2048
128
encoder_layers
6
2
text_config.hidden_size
768
64
text_config.intermediate_size
3072
128
text_config.num_attention_heads
12
4
text_config.num_hidden_layers
12
2
+
\ No newline at end of file
diff --git a/shrink/hubert-large-ls960-ft-mic.html b/shrink/hubert-large-ls960-ft-mic.html
new file mode 100644
index 0000000000000000000000000000000000000000..f4934d24388da3bfbab107a08beefd7c43f5e80b
--- /dev/null
+++ b/shrink/hubert-large-ls960-ft-mic.html
@@ -0,0 +1,34 @@
+
+
+Shrunk architecture
+
+
Shrunk architecture
huggingface-mic/hubert-large-ls960-ft-mic@5f4ea3f40f370b2a93c05c17a1ea550648a1d8b7 ← facebook/hubert-large-ls960-ft@ece5fabbf034c1073acae96d5401b25be96709d8 · HubertForCTC · overfit weights · built before the shrink rule (no recorded recipe)
+
Original315.5Mparameters
Fixture45.0Kparameters
Layer stacks2compared
Findings0to review
✓ Same architecture, smaller
Every module class, layer kind and checked property of the original is in the fixture.
Layer stacks
One square per layer, colored by kind: the layer's module classes and type attributes. Hover a square for its index and kind.
hubert.feature_extractor.conv_layers
Original · 7
Fixture · 7
Kind
Original
Fixture
LayerNormConvLayer
7
7
✓ kept
hubert.encoder.layers
Original · 24
Fixture · 2
Kind
Original
Fixture
Linear
24
2
✓ kept
Where the parameters are
Share of each model's parameters per component. Vocabulary and patch sizes are kept, so embeddings usually take a larger share of the fixture.
Original
Fixture
Component
Original
Share
Fixture
Share
hubert.encoder.layers
302.3M
95.8%
17.1K
38.0%
hubert.encoder
8.4M
2.7%
8.3K
18.5%
hubert.feature_extractor.conv_layers
4.2M
1.3%
17.4K
38.6%
hubert.feature_projection
526.3K
0.2%
1.1K
2.5%
lm_head
32.8K
0.0%
1.1K
2.3%
hubert
1.0K
0.0%
32
0.1%
Preserved properties
Config
Property
Original
Fixture
model
model_type
hubert
hubert
✓ kept
model
vocab_size
32
32
✓ kept
model
hidden_act
gelu
gelu
✓ kept
Module classes
Instances of each class in the built model (prefix Hubert omitted).
Class
Original
Fixture
Linear
146
14
Dropout
75
9
LayerNorm
57
13
GELUActivation
32
10
Attention
24
2
EncoderLayerStableLayerNorm
24
2
FeedForward
24
2
Conv1d
7
7
LayerNormConvLayer
7
7
ModuleList
2
2
EncoderStableLayerNorm
1
1
FeatureEncoder
1
1
FeatureProjection
1
1
ForCTC
1
1
Model
1
1
PositionalConvEmbedding
1
1
SamePadLayer
1
1
ModuleDict
1
1
ParametrizationList
1
1
ParametrizedConv1d
1
1
_WeightNorm
1
1
Config changes
7 fields changed, 46 unchanged.
Field
Original
Fixture
conv_dim
7 entries: 7× 512
7 entries: 7× 32
hidden_size
1024
32
intermediate_size
4096
64
num_attention_heads
16
4
num_conv_pos_embedding_groups
16
4
num_conv_pos_embeddings
128
32
num_hidden_layers
24
2
+
\ No newline at end of file
diff --git a/shrink/internvl-mic.html b/shrink/internvl-mic.html
new file mode 100644
index 0000000000000000000000000000000000000000..d866be01dd0aeed19a19b4dc773611206e5747f1
--- /dev/null
+++ b/shrink/internvl-mic.html
@@ -0,0 +1,34 @@
+
+
+Shrunk architecture
+
+
Shrunk architecture
huggingface-mic/internvl-mic@9bf394da9450b552cff318878bcdee131e669a91 ← OpenGVLab/InternVL3-1B-hf@014c0583a0d4bedf29fbe2dbff4f865eb998e171 · InternVLForConditionalGeneration · random weights · built before the shrink rule (no recorded recipe)
+
Original802.3Mparameters
Fixture10.0Mparameters
Layer stacks2compared
Findings0to review
✓ Same architecture, smaller
Every module class, layer kind and checked property of the original is in the fixture.
Layer stacks
One square per layer, colored by kind: the layer's module classes and type attributes. Hover a square for its index and kind.
model.vision_tower.encoder.layer
Original · 24
Fixture · 2
Kind
Original
Fixture
VisionLayer
24
2
✓ kept
model.language_model.layers
Original · 24
Fixture · 2
Kind
Original
Fixture
Qwen2DecoderLayer
24
2
✓ kept
Where the parameters are
Share of each model's parameters per component. Vocabulary and patch sizes are kept, so embeddings usually take a larger share of the fixture.
Original
Fixture
Component
Original
Share
Fixture
Share
model.language_model.layers
357.9M
44.6%
74.2K
0.7%
model.vision_tower.encoder.layer
302.4M
37.7%
67.2K
0.7%
model.language_model
135.9M
16.9%
9.7M
97.3%
model.multi_modal_projector
4.5M
0.6%
21.1K
0.2%
model.vision_tower
1.7M
0.2%
103.4K
1.0%
Preserved properties
Config
Property
Original
Fixture
model
model_type
internvl
internvl
✓ kept
model
tie_word_embeddings
True
True
✓ kept
vision_config
model_type
internvl_vision
internvl_vision
✓ kept
vision_config
hidden_act
gelu
gelu
✓ kept
vision_config
image_size
[448, 448]
[448, 448]
✓ kept
vision_config
patch_size
[14, 14]
[14, 14]
✓ kept
vision_config
num_channels
3
3
✓ kept
text_config
attention style
GQA (14 heads, 2 KV)
GQA (4 heads, 2 KV)
✓ kept
text_config
RoPE type
dynamic
dynamic
✓ kept
text_config
model_type
qwen2
qwen2
✓ kept
text_config
vocab_size
151674
151674
✓ kept
text_config
tie_word_embeddings
False
False
✓ kept
text_config
hidden_act
silu
silu
✓ kept
text_config
max_position_embeddings
32768
32768
✓ kept
Module classes
Instances of each class in the built model (prefix InternVL omitted).
Class
Original
Fixture
Linear
315
29
Identity
73
7
LayerNorm
49
5
Qwen2RMSNorm
49
5
Dropout
25
3
GELUActivation
25
3
VisionAttention
24
2
VisionLayer
24
2
VisionMLP
24
2
Qwen2Attention
24
2
Qwen2DecoderLayer
24
2
Qwen2MLP
24
2
SiLUActivation
24
2
ModuleList
2
2
Conv2d
1
1
Embedding
1
1
ForConditionalGeneration
1
1
Model
1
1
MultiModalProjector
1
1
VisionEmbeddings
1
1
VisionEncoder
1
1
VisionModel
1
1
VisionPatchEmbeddings
1
1
Qwen2Model
1
1
Qwen2RotaryEmbedding
1
1
Config changes
9 fields changed, 68 unchanged.
Field
Original
Fixture
text_config.hidden_size
896
64
text_config.intermediate_size
4864
128
text_config.layer_types
24 entries: 24× full_attention
["full_attention", "full_attention"]
text_config.num_attention_heads
14
4
text_config.num_hidden_layers
24
2
vision_config.hidden_size
1024
64
vision_config.intermediate_size
4096
128
vision_config.num_attention_heads
16
4
vision_config.num_hidden_layers
24
2
+
\ No newline at end of file
diff --git a/shrink/jina-embeddings-v3-hf-mic.html b/shrink/jina-embeddings-v3-hf-mic.html
new file mode 100644
index 0000000000000000000000000000000000000000..8915da9eb00119a472135c93e0e9e87be198ef9e
--- /dev/null
+++ b/shrink/jina-embeddings-v3-hf-mic.html
@@ -0,0 +1,34 @@
+
+
+Shrunk architecture
+
+
Shrunk architecture
huggingface-mic/jina-embeddings-v3-hf-mic@7288a7adb5335fb7805f7c28ffa3efd588b8be67 ← jinaai/jina-embeddings-v3-hf@d18862d9a48706220815554fac3ebb4dfa46fc28 · JinaEmbeddingsV3Model · random weights
+
Original559.4Mparameters
Fixture16.1Mparameters
Layer stacks1compared
Findings0to review
✓ Same architecture, smaller
Every module class, layer kind and checked property of the original is in the fixture.
Layer stacks
One square per layer, colored by kind: the layer's module classes and type attributes. Hover a square for its index and kind.
layers
Original · 24
Fixture · 2
Kind
Original
Fixture
Layer
24
2
✓ kept
Where the parameters are
Share of each model's parameters per component. Vocabulary and patch sizes are kept, so embeddings usually take a larger share of the fixture.
Original
Fixture
Component
Original
Share
Fixture
Share
layers
302.3M
54.0%
66.9K
0.4%
embeddings.word_embeddings
256.0M
45.8%
16.0M
99.6%
pooler.dense
1.0M
0.2%
4.2K
0.0%
embeddings.LayerNorm
2.0K
0.0%
128
0.0%
embeddings.token_type_embeddings
1.0K
0.0%
64
0.0%
Preserved properties
Config
Property
Original
Fixture
model
RoPE type
default
default
✓ kept
model
model_type
jina_embeddings_v3
jina_embeddings_v3
✓ kept
model
vocab_size
250002
250002
✓ kept
model
tie_word_embeddings
True
True
✓ kept
model
hidden_act
gelu
gelu
✓ kept
model
max_position_embeddings
8194
8194
✓ kept
Module classes
Instances of each class in the built model (prefix JinaEmbeddingsV3 omitted).
Class
Original
Fixture
Linear
145
13
Dropout
49
5
LayerNorm
49
5
GELUActivation
24
2
Attention
24
2
Layer
24
2
MLP
24
2
Embedding
2
2
Embeddings
1
1
Model
1
1
Pooler
1
1
RotaryEmbedding
1
1
ModuleList
1
1
Tanh
1
1
Config changes
4 fields changed, 38 unchanged.
Field
Original
Fixture
hidden_size
1024
64
intermediate_size
4096
128
num_attention_heads
16
4
num_hidden_layers
24
2
+
\ No newline at end of file
diff --git a/shrink/language-perceiver-mic.html b/shrink/language-perceiver-mic.html
new file mode 100644
index 0000000000000000000000000000000000000000..4c15f4e047e34c75869f55529ecdf898579c4be5
--- /dev/null
+++ b/shrink/language-perceiver-mic.html
@@ -0,0 +1,34 @@
+
+
+Shrunk architecture
+
+
Shrunk architecture
huggingface-mic/language-perceiver-mic@6c70b7e085e8e6fd4544e720e75ea0e0cbdd438a ← deepmind/language-perceiver@47a1ba43d8198ea10cf9a158fce114953e9441a1 · PerceiverForMaskedLM · random weights · built before the shrink rule (no recorded recipe)
+
Original201.1Mparameters
Fixture194.1Mparameters
Layer stacks1compared
Findings0to review
✓ Same architecture, smaller
Every module class, layer kind and checked property of the original is in the fixture.
Layer stacks
One square per layer, colored by kind: the layer's module classes and type attributes. Hover a square for its index and kind.
perceiver.encoder.self_attends
Original · 26
Fixture · 26
Kind
Original
Fixture
Layer
26
26
✓ kept
Where the parameters are
Share of each model's parameters per component. Vocabulary and patch sizes are kept, so embeddings usually take a larger share of the fixture.
Original
Fixture
Component
Original
Share
Fixture
Share
perceiver.encoder.self_attends
187.7M
93.3%
187.7M
96.7%
perceiver.encoder
6.4M
3.2%
5.4M
2.8%
perceiver.decoder
4.9M
2.4%
572.9K
0.3%
perceiver.input_preprocessor
1.8M
0.9%
147.8K
0.1%
perceiver.embeddings
327.7K
0.2%
327.7K
0.2%
embedding_decoder
262
0.0%
262
0.0%
Preserved properties
Config
Property
Original
Fixture
model
model_type
perceiver
perceiver
✓ kept
model
vocab_size
262
262
✓ kept
model
hidden_act
gelu
gelu
✓ kept
model
max_position_embeddings
2048
2048
✓ kept
model
image_size
56
56
✓ kept
Module classes
Instances of each class in the built model (prefix Perceiver omitted).
Class
Original
Fixture
Linear
168
168
LayerNorm
58
58
Dropout
28
28
GELUActivation
28
28
Identity
28
28
Attention
28
28
Layer
28
28
MLP
28
28
SelfAttention
28
28
SelfOutput
28
28
Embedding
2
2
ModuleList
1
1
BasicDecoder
1
1
EmbeddingDecoder
1
1
Embeddings
1
1
Encoder
1
1
ForMaskedLM
1
1
Model
1
1
TextPreprocessor
1
1
TrainablePositionEncoding
1
1
Config changes
1 fields changed, 35 unchanged.
Field
Original
Fixture
d_model
768
64
+
\ No newline at end of file
diff --git a/shrink/layoutlm-base-uncased-mic.html b/shrink/layoutlm-base-uncased-mic.html
new file mode 100644
index 0000000000000000000000000000000000000000..518865d49dfd45901791914153268d30b67f0b31
--- /dev/null
+++ b/shrink/layoutlm-base-uncased-mic.html
@@ -0,0 +1,34 @@
+
+
+Shrunk architecture
+
+
Shrunk architecture
huggingface-mic/layoutlm-base-uncased-mic@7aa363dd7f97db6248afc704386db78edea79ae6 ← microsoft/layoutlm-base-uncased@30e3cdd39c11f09757b0fcf7598533d05052acef · LayoutLMForMaskedLM · random weights
+
Original113.3Mparameters
Fixture2.4Mparameters
Layer stacks1compared
Findings0to review
✓ Same architecture, smaller
Every module class, layer kind and checked property of the original is in the fixture.
Layer stacks
One square per layer, colored by kind: the layer's module classes and type attributes. Hover a square for its index and kind.
layoutlm.encoder.layer
Original · 12
Fixture · 2
Kind
Original
Fixture
Layer
12
2
✓ kept
Where the parameters are
Share of each model's parameters per component. Vocabulary and patch sizes are kept, so embeddings usually take a larger share of the fixture.
Original
Fixture
Component
Original
Share
Fixture
Share
layoutlm.encoder.layer
85.1M
75.1%
66.9K
2.8%
layoutlm.embeddings
27.0M
23.8%
2.2M
95.5%
cls.predictions
622.6K
0.5%
34.8K
1.5%
layoutlm.pooler
590.6K
0.5%
4.2K
0.2%
Preserved properties
Config
Property
Original
Fixture
model
model_type
layoutlm
layoutlm
✓ kept
model
vocab_size
30522
30522
✓ kept
model
tie_word_embeddings
True
True
✓ kept
model
hidden_act
gelu
gelu
✓ kept
model
max_position_embeddings
512
512
✓ kept
Module classes
Instances of each class in the built model (prefix LayoutLM omitted).
Class
Original
Fixture
Linear
75
15
Dropout
37
7
LayerNorm
26
6
GELUActivation
13
3
Attention
12
2
Intermediate
12
2
Layer
12
2
Output
12
2
SelfAttention
12
2
SelfOutput
12
2
Embedding
7
7
Embeddings
1
1
Encoder
1
1
ForMaskedLM
1
1
LMPredictionHead
1
1
Model
1
1
OnlyMLMHead
1
1
Pooler
1
1
PredictionHeadTransform
1
1
ModuleList
1
1
Tanh
1
1
Config changes
4 fields changed, 25 unchanged.
Field
Original
Fixture
hidden_size
768
64
intermediate_size
3072
128
num_attention_heads
12
4
num_hidden_layers
12
2
+
\ No newline at end of file
diff --git a/shrink/llama-mic.html b/shrink/llama-mic.html
new file mode 100644
index 0000000000000000000000000000000000000000..58c7cd961f5f9fd4cc6dec3a59f3351ddf316ff7
--- /dev/null
+++ b/shrink/llama-mic.html
@@ -0,0 +1,34 @@
+
+
+Shrunk architecture
+
+
Shrunk architecture
huggingface-mic/llama-mic@8231ab98d1dd670f702753a80097b854fb5e91b5 ← HuggingFaceTB/SmolLM2-135M@93efa2f097d58c2a74874c7e644dbc9b0cee75a2 · LlamaForCausalLM · random weights · built before the shrink rule (no recorded recipe)
+
Original134.5Mparameters
Fixture3.2Mparameters
Layer stacks1compared
Findings0to review
✓ Same architecture, smaller
Every module class, layer kind and checked property of the original is in the fixture.
Layer stacks
One square per layer, colored by kind: the layer's module classes and type attributes. Hover a square for its index and kind.
model.layers
Original · 30
Fixture · 2
Kind
Original
Fixture
DecoderLayer
30
2
✓ kept
Where the parameters are
Share of each model's parameters per component. Vocabulary and patch sizes are kept, so embeddings usually take a larger share of the fixture.
Original
Fixture
Component
Original
Share
Fixture
Share
model.layers
106.2M
79.0%
74.0K
2.3%
model.embed_tokens
28.3M
21.0%
3.1M
97.7%
model.norm
576
0.0%
64
0.0%
Preserved properties
Config
Property
Original
Fixture
model
attention style
GQA (9 heads, 3 KV)
GQA (4 heads, 2 KV)
✓ kept
model
RoPE type
default
default
✓ kept
model
model_type
llama
llama
✓ kept
model
vocab_size
49152
49152
✓ kept
model
tie_word_embeddings
True
True
✓ kept
model
hidden_act
silu
silu
✓ kept
model
max_position_embeddings
8192
8192
✓ kept
Module classes
Instances of each class in the built model (prefix Llama omitted).
Class
Original
Fixture
Linear
211
15
RMSNorm
61
5
Attention
30
2
DecoderLayer
30
2
MLP
30
2
SiLUActivation
30
2
Embedding
1
1
ForCausalLM
1
1
Model
1
1
RotaryEmbedding
1
1
ModuleList
1
1
Config changes
6 fields changed, 26 unchanged.
Field
Original
Fixture
head_dim
64
16
hidden_size
576
64
intermediate_size
1536
128
num_attention_heads
9
4
num_hidden_layers
30
2
num_key_value_heads
3
2
+
\ No newline at end of file
diff --git a/shrink/llava-mic.html b/shrink/llava-mic.html
new file mode 100644
index 0000000000000000000000000000000000000000..0040252a3a4a2726e009eb78841d19093e2ce5bb
--- /dev/null
+++ b/shrink/llava-mic.html
@@ -0,0 +1,34 @@
+
+
+Shrunk architecture
+
+
Shrunk architecture
huggingface-mic/llava-mic@dab442b1a1f407ad1083d0a1b5cf11dc99c02294 ← llava-hf/llava-1.5-7b-hf@b234b804b114d9e37bb655e11cbbb5f5e971b7a9 · LlavaForConditionalGeneration · random weights · built before the shrink rule (no recorded recipe)
+
Original7.1Bparameters
Fixture4.3Mparameters
Layer stacks2compared
Findings0to review
✓ Same architecture, smaller
Every module class, layer kind and checked property of the original is in the fixture.
Layer stacks
One square per layer, colored by kind: the layer's module classes and type attributes. Hover a square for its index and kind.
model.vision_tower.encoder.layers
Original · 24
Fixture · 2
Kind
Original
Fixture
CLIPEncoderLayer
24
2
✓ kept
model.language_model.layers
Original · 32
Fixture · 2
Kind
Original
Fixture
LlamaDecoderLayer
32
2
✓ kept
Where the parameters are
Share of each model's parameters per component. Vocabulary and patch sizes are kept, so embeddings usually take a larger share of the fixture.
Original
Fixture
Component
Original
Share
Fixture
Share
model.language_model.layers
6.5B
91.7%
82.2K
1.9%
model.vision_tower.encoder.layers
302.3M
4.3%
66.9K
1.5%
model.language_model
131.3M
1.9%
2.1M
47.3%
lm_head
131.3M
1.9%
2.1M
47.3%
model.multi_modal_projector
21.0M
0.3%
8.3K
0.2%
model.vision_tower
1.2M
0.0%
74.9K
1.7%
Preserved properties
Config
Property
Original
Fixture
model
model_type
llava
llava
✓ kept
model
vocab_size
32064
32064
✓ kept
model
tie_word_embeddings
False
False
✓ kept
vision_config
model_type
clip_vision_model
clip_vision_model
✓ kept
vision_config
vocab_size
32000
32000
✓ kept
vision_config
hidden_act
quick_gelu
quick_gelu
✓ kept
vision_config
image_size
336
336
✓ kept
vision_config
patch_size
14
14
✓ kept
vision_config
num_channels
3
3
✓ kept
text_config
attention style
MHA (32 heads, 32 KV)
MHA (4 heads, 4 KV)
✓ kept
text_config
RoPE type
default
default
✓ kept
text_config
model_type
llama
llama
✓ kept
text_config
vocab_size
32064
32064
✓ kept
text_config
tie_word_embeddings
False
False
✓ kept
text_config
hidden_act
silu
silu
✓ kept
text_config
max_position_embeddings
4096
4096
✓ kept
Module classes
Instances of each class in the built model (prefix Llava omitted).
Class
Original
Fixture
Linear
371
29
LlamaRMSNorm
65
5
LayerNorm
50
6
LlamaAttention
32
2
LlamaDecoderLayer
32
2
LlamaMLP
32
2
SiLUActivation
32
2
CLIPAttention
24
2
CLIPEncoderLayer
24
2
CLIPMLP
24
2
QuickGELUActivation
24
2
Embedding
2
2
ModuleList
2
2
CLIPEncoder
1
1
CLIPVisionEmbeddings
1
1
CLIPVisionModel
1
1
Conv2d
1
1
GELUActivation
1
1
LlamaModel
1
1
LlamaRotaryEmbedding
1
1
ForConditionalGeneration
1
1
Model
1
1
MultiModalProjector
1
1
Config changes
11 fields changed, 61 unchanged.
Field
Original
Fixture
text_config.head_dim
128
16
text_config.hidden_size
4096
64
text_config.intermediate_size
11008
128
text_config.num_attention_heads
32
4
text_config.num_hidden_layers
32
2
text_config.num_key_value_heads
32
4
vision_config.hidden_size
1024
64
vision_config.intermediate_size
4096
128
vision_config.num_attention_heads
16
4
vision_config.num_hidden_layers
24
2
vision_config.projection_dim
768
32
+
\ No newline at end of file
diff --git a/shrink/llava-onevision-qwen2-0.5b-ov-hf-mic.html b/shrink/llava-onevision-qwen2-0.5b-ov-hf-mic.html
new file mode 100644
index 0000000000000000000000000000000000000000..711d257983e23ba0cd7525dc8085c66a7144e069
--- /dev/null
+++ b/shrink/llava-onevision-qwen2-0.5b-ov-hf-mic.html
@@ -0,0 +1,34 @@
+
+
+Shrunk architecture
+
+
Shrunk architecture
huggingface-mic/llava-onevision-qwen2-0.5b-ov-hf-mic@c8d8d3c952d7ba613a1c76b3eb2d1641b8fd053f ← llava-hf/llava-onevision-qwen2-0.5b-ov-hf@74dd0bf867a4cda7950c17663794267c60cf4b40 · LlavaOnevisionForConditionalGeneration · random weights
+
Original893.7Mparameters
Fixture10.0Mparameters
Layer stacks2compared
Findings0to review
✓ Same architecture, smaller
Every module class, layer kind and checked property of the original is in the fixture.
Layer stacks
One square per layer, colored by kind: the layer's module classes and type attributes. Hover a square for its index and kind.
model.vision_tower.encoder.layers
Original · 26
Fixture · 2
Kind
Original
Fixture
SiglipEncoderLayer
26
2
✓ kept
model.language_model.layers
Original · 24
Fixture · 2
Kind
Original
Fixture
Qwen2DecoderLayer
24
2
✓ kept
Where the parameters are
Share of each model's parameters per component. Vocabulary and patch sizes are kept, so embeddings usually take a larger share of the fixture.
Original
Fixture
Component
Original
Share
Fixture
Share
model.vision_tower.encoder.layers
396.2M
44.3%
66.9K
0.7%
model.language_model.layers
357.9M
40.0%
74.2K
0.7%
model.language_model
136.2M
15.2%
9.7M
97.7%
model.multi_modal_projector
1.8M
0.2%
8.3K
0.1%
model.vision_tower
1.5M
0.2%
84.5K
0.8%
model
896
0.0%
64
0.0%
Preserved properties
Config
Property
Original
Fixture
model
model_type
llava_onevision
llava_onevision
✓ kept
model
tie_word_embeddings
True
True
✓ kept
vision_config
model_type
siglip_vision_model
siglip_vision_model
✓ kept
vision_config
hidden_act
gelu_pytorch_tanh
gelu_pytorch_tanh
✓ kept
vision_config
image_size
384
384
✓ kept
vision_config
patch_size
14
14
✓ kept
vision_config
num_channels
3
3
✓ kept
text_config
attention style
GQA (14 heads, 2 KV)
GQA (4 heads, 2 KV)
✓ kept
text_config
RoPE type
default
default
✓ kept
text_config
model_type
qwen2
qwen2
✓ kept
text_config
vocab_size
152000
152000
✓ kept
text_config
tie_word_embeddings
True
True
✓ kept
text_config
hidden_act
silu
silu
✓ kept
text_config
max_position_embeddings
32768
32768
✓ kept
Module classes
Instances of each class in the built model (prefix LlavaOnevision omitted).
Class
Original
Fixture
Linear
327
29
LayerNorm
53
5
Qwen2RMSNorm
49
5
GELUTanh
26
2
SiglipAttention
26
2
SiglipEncoderLayer
26
2
SiglipMLP
26
2
Qwen2Attention
24
2
Qwen2DecoderLayer
24
2
Qwen2MLP
24
2
SiLUActivation
24
2
Embedding
2
2
ModuleList
2
2
Conv2d
1
1
GELUActivation
1
1
ForConditionalGeneration
1
1
Model
1
1
MultiModalProjector
1
1
Qwen2Model
1
1
Qwen2RotaryEmbedding
1
1
SiglipEncoder
1
1
SiglipVisionEmbeddings
1
1
SiglipVisionModel
1
1
Config changes
9 fields changed, 61 unchanged.
Field
Original
Fixture
text_config.hidden_size
896
64
text_config.intermediate_size
4864
128
text_config.layer_types
24 entries: 24× full_attention
["full_attention", "full_attention"]
text_config.num_attention_heads
14
4
text_config.num_hidden_layers
24
2
vision_config.hidden_size
1152
64
vision_config.intermediate_size
4304
128
vision_config.num_attention_heads
16
4
vision_config.num_hidden_layers
26
2
+
\ No newline at end of file
diff --git a/shrink/llava-v1.6-mistral-7b-hf-mic.html b/shrink/llava-v1.6-mistral-7b-hf-mic.html
new file mode 100644
index 0000000000000000000000000000000000000000..7d67417ef6db69f92e4c14147d497db15a3b7d2a
--- /dev/null
+++ b/shrink/llava-v1.6-mistral-7b-hf-mic.html
@@ -0,0 +1,34 @@
+
+
+Shrunk architecture
+
+
Shrunk architecture
huggingface-mic/llava-v1.6-mistral-7b-hf-mic@2575b1c20c7676e7707d37ef750132fd64d1fc03 ← llava-hf/llava-v1.6-mistral-7b-hf@2424fdd47412fccc66d91719126b420e9fbd7065 · LlavaNextForConditionalGeneration · random weights · built before the shrink rule (no recorded recipe)
+
Original7.6Bparameters
Fixture4.3Mparameters
Layer stacks2compared
Findings0to review
✓ Same architecture, smaller
Every module class, layer kind and checked property of the original is in the fixture.
Layer stacks
One square per layer, colored by kind: the layer's module classes and type attributes. Hover a square for its index and kind.
model.vision_tower.encoder.layers
Original · 24
Fixture · 2
Kind
Original
Fixture
CLIPEncoderLayer
24
2
✓ kept
model.language_model.layers
Original · 32
Fixture · 2
Kind
Original
Fixture
MistralDecoderLayer
32
2
✓ kept
Where the parameters are
Share of each model's parameters per component. Vocabulary and patch sizes are kept, so embeddings usually take a larger share of the fixture.
Original
Fixture
Component
Original
Share
Fixture
Share
model.language_model.layers
7.0B
92.2%
74.0K
1.7%
model.vision_tower.encoder.layers
302.3M
4.0%
66.9K
1.5%
model.language_model
131.3M
1.7%
2.1M
47.4%
lm_head
131.3M
1.7%
2.1M
47.4%
model.multi_modal_projector
21.0M
0.3%
8.3K
0.2%
model.vision_tower
1.2M
0.0%
74.9K
1.7%
model
4.1K
0.0%
64
0.0%
Preserved properties
Config
Property
Original
Fixture
model
model_type
llava_next
llava_next
✓ kept
model
vocab_size
32064
32064
✓ kept
model
tie_word_embeddings
False
False
✓ kept
vision_config
model_type
clip_vision_model
clip_vision_model
✓ kept
vision_config
vocab_size
32000
32000
✓ kept
vision_config
hidden_act
quick_gelu
quick_gelu
✓ kept
vision_config
image_size
336
336
✓ kept
vision_config
patch_size
14
14
✓ kept
vision_config
num_channels
3
3
✓ kept
text_config
attention style
GQA (32 heads, 8 KV)
GQA (4 heads, 2 KV)
✓ kept
text_config
RoPE type
default
default
✓ kept
text_config
model_type
mistral
mistral
✓ kept
text_config
vocab_size
32064
32064
✓ kept
text_config
tie_word_embeddings
False
False
✓ kept
text_config
hidden_act
silu
silu
✓ kept
text_config
max_position_embeddings
32768
32768
✓ kept
Module classes
Instances of each class in the built model (prefix LlavaNext omitted).
Class
Original
Fixture
Linear
371
29
MistralRMSNorm
65
5
LayerNorm
50
6
MistralAttention
32
2
MistralDecoderLayer
32
2
MistralMLP
32
2
SiLUActivation
32
2
CLIPAttention
24
2
CLIPEncoderLayer
24
2
CLIPMLP
24
2
QuickGELUActivation
24
2
Embedding
2
2
ModuleList
2
2
CLIPEncoder
1
1
CLIPVisionEmbeddings
1
1
CLIPVisionModel
1
1
Conv2d
1
1
GELUActivation
1
1
ForConditionalGeneration
1
1
Model
1
1
MultiModalProjector
1
1
MistralModel
1
1
MistralRotaryEmbedding
1
1
Config changes
11 fields changed, 60 unchanged.
Field
Original
Fixture
text_config.head_dim
128
16
text_config.hidden_size
4096
64
text_config.intermediate_size
14336
128
text_config.num_attention_heads
32
4
text_config.num_hidden_layers
32
2
text_config.num_key_value_heads
8
2
vision_config.hidden_size
1024
64
vision_config.intermediate_size
4096
128
vision_config.num_attention_heads
16
4
vision_config.num_hidden_layers
24
2
vision_config.projection_dim
768
64
+
\ No newline at end of file
diff --git a/shrink/long-t5-tglobal-base-mic.html b/shrink/long-t5-tglobal-base-mic.html
new file mode 100644
index 0000000000000000000000000000000000000000..47259b441c2c222f3b9511d84c5cc7b40783571f
--- /dev/null
+++ b/shrink/long-t5-tglobal-base-mic.html
@@ -0,0 +1,34 @@
+
+
+Shrunk architecture
+
+
Shrunk architecture
huggingface-mic/long-t5-tglobal-base-mic@04d69ed32e31d0ce673622fa6832568a878efb1d ← google/long-t5-tglobal-base@aecb1376e5bd78db32ebc5c9deb257449b9e2b21 · LongT5ForConditionalGeneration · random weights
+
Original222.9Mparameters
Fixture2.3Mparameters
Layer stacks2compared
Findings0to review
✓ Same architecture, smaller
Every module class, layer kind and checked property of the original is in the fixture.
Layer stacks
One square per layer, colored by kind: the layer's module classes and type attributes. Hover a square for its index and kind.
encoder.block
Original · 12
Fixture · 2
Kind
Original
Fixture
Embedding
1
1
✓ kept
no Embedding
11
1
✓ kept
decoder.block
Original · 12
Fixture · 2
Kind
Original
Fixture
Embedding
1
1
✓ kept
no Embedding
11
1
✓ kept
Where the parameters are
Share of each model's parameters per component. Vocabulary and patch sizes are kept, so embeddings usually take a larger share of the fixture.
Original
Fixture
Component
Original
Share
Fixture
Share
decoder.block
113.3M
50.8%
115.2K
5.1%
encoder.block
85.0M
38.1%
82.6K
3.7%
shared
24.7M
11.1%
2.1M
91.2%
encoder.final_layer_norm
768
0.0%
64
0.0%
decoder.final_layer_norm
768
0.0%
64
0.0%
Preserved properties
Config
Property
Original
Fixture
model
model_type
longt5
longt5
✓ kept
model
vocab_size
32128
32128
✓ kept
model
tie_word_embeddings
True
True
✓ kept
Module classes
Instances of each class in the built model (prefix LongT5 omitted).
Class
Original
Fixture
Linear
217
37
Dropout
86
16
LayerNorm
74
14
ModuleList
26
6
Attention
24
4
Block
24
4
DenseGatedActDense
24
4
LayerFF
24
4
NewGELUActivation
24
4
LayerCrossAttention
12
2
LayerSelfAttention
12
2
LayerTransientGlobalSelfAttention
12
2
TransientGlobalAttention
12
2
Embedding
6
6
Stack
2
2
ForConditionalGeneration
1
1
Config changes
6 fields changed, 31 unchanged.
Field
Original
Fixture
d_ff
2048
128
d_kv
64
16
d_model
768
64
num_decoder_layers
12
2
num_heads
12
4
num_layers
12
2
+
\ No newline at end of file
diff --git a/shrink/longformer-base-4096-mic.html b/shrink/longformer-base-4096-mic.html
new file mode 100644
index 0000000000000000000000000000000000000000..78f48dcedb382268dc3c738f47850e39e109cda5
--- /dev/null
+++ b/shrink/longformer-base-4096-mic.html
@@ -0,0 +1,34 @@
+
+
+Shrunk architecture
+
+
Shrunk architecture
huggingface-mic/longformer-base-4096-mic@5e8ae916a995900686949962df7210f7676e90fd ← allenai/longformer-base-4096@301e6a42cb0d9976a6d6a26a079fef81c18aa895 · LongformerForMaskedLM · random weights
+
Original148.7Mparameters
Fixture3.6Mparameters
Layer stacks1compared
Findings0to review
✓ Same architecture, smaller
Every module class, layer kind and checked property of the original is in the fixture.
Layer stacks
One square per layer, colored by kind: the layer's module classes and type attributes. Hover a square for its index and kind.
longformer.encoder.layer
Original · 12
Fixture · 2
Kind
Original
Fixture
Layer
12
2
✓ kept
Where the parameters are
Share of each model's parameters per component. Vocabulary and patch sizes are kept, so embeddings usually take a larger share of the fixture.
Original
Fixture
Component
Original
Share
Fixture
Share
longformer.encoder.layer
106.3M
71.5%
91.9K
2.5%
longformer.embeddings
41.8M
28.1%
3.5M
96.0%
lm_head.dense
590.6K
0.4%
4.2K
0.1%
lm_head
50.3K
0.0%
50.3K
1.4%
lm_head.layer_norm
1.5K
0.0%
128
0.0%
Preserved properties
Config
Property
Original
Fixture
model
model_type
longformer
longformer
✓ kept
model
vocab_size
50265
50265
✓ kept
model
tie_word_embeddings
True
True
✓ kept
model
hidden_act
gelu
gelu
✓ kept
model
max_position_embeddings
4098
4098
✓ kept
Module classes
Instances of each class in the built model (prefix Longformer omitted).
Class
Original
Fixture
Linear
110
20
LayerNorm
26
6
Dropout
25
5
GELUActivation
12
2
Attention
12
2
Intermediate
12
2
Layer
12
2
Output
12
2
SelfAttention
12
2
SelfOutput
12
2
Embedding
3
3
Embeddings
1
1
Encoder
1
1
ForMaskedLM
1
1
LMHead
1
1
Model
1
1
ModuleList
1
1
Config changes
5 fields changed, 26 unchanged.
Field
Original
Fixture
attention_window
12 entries: 12× 512
[512, 512]
hidden_size
768
64
intermediate_size
3072
128
num_attention_heads
12
4
num_hidden_layers
12
2
+
\ No newline at end of file
diff --git a/shrink/luke-base-mic.html b/shrink/luke-base-mic.html
new file mode 100644
index 0000000000000000000000000000000000000000..27dec0fefcd754da93c8646ee4c0b3a98fa03f38
--- /dev/null
+++ b/shrink/luke-base-mic.html
@@ -0,0 +1,34 @@
+
+
+Shrunk architecture
+
+
Shrunk architecture
huggingface-mic/luke-base-mic@39d0a41c62c207695ca9fda87542f953dc1d0b56 ← studio-ousia/luke-base@7438924defd9f3c2018d63c16073bf4bcb6a70aa · LukeForMaskedLM · random weights
+
Original314.4Mparameters
Fixture135.2Mparameters
Layer stacks1compared
Findings0to review
✓ Same architecture, smaller
Every module class, layer kind and checked property of the original is in the fixture.
Layer stacks
One square per layer, colored by kind: the layer's module classes and type attributes. Hover a square for its index and kind.
luke.encoder.layer
Original · 12
Fixture · 2
Kind
Original
Fixture
Layer
12
2
✓ kept
Where the parameters are
Share of each model's parameters per component. Vocabulary and patch sizes are kept, so embeddings usually take a larger share of the fixture.
Original
Fixture
Component
Original
Share
Fixture
Share
luke.entity_embeddings
128.6M
40.9%
128.0M
94.7%
luke.encoder.layer
106.3M
33.8%
91.9K
0.1%
luke.embeddings
39.0M
12.4%
3.3M
2.4%
lm_head.decoder
38.6M
12.3%
3.2M
2.4%
luke.pooler
590.6K
0.2%
4.2K
0.0%
lm_head.dense
590.6K
0.2%
4.2K
0.0%
entity_predictions
500.0K
0.2%
500.0K
0.4%
3 others
249.2K
0.1%
67.5K
0.0%
Preserved properties
Config
Property
Original
Fixture
model
model_type
luke
luke
✓ kept
model
vocab_size
50267
50267
✓ kept
model
tie_word_embeddings
True
True
✓ kept
model
hidden_act
gelu
gelu
✓ kept
model
max_position_embeddings
514
514
✓ kept
Module classes
Instances of each class in the built model (prefix Luke omitted).
Class
Original
Fixture
Linear
114
24
Dropout
38
8
LayerNorm
28
8
GELUActivation
13
3
Attention
12
2
Intermediate
12
2
Layer
12
2
Output
12
2
SelfAttention
12
2
SelfOutput
12
2
Embedding
6
6
CrossEntropyLoss
1
1
EntityPredictionHead
1
1
EntityPredictionHeadTransform
1
1
Embeddings
1
1
Encoder
1
1
EntityEmbeddings
1
1
ForMaskedLM
1
1
LMHead
1
1
Model
1
1
Pooler
1
1
ModuleList
1
1
Tanh
1
1
Config changes
4 fields changed, 29 unchanged.
Field
Original
Fixture
hidden_size
768
64
intermediate_size
3072
128
num_attention_heads
12
4
num_hidden_layers
12
2
+
\ No newline at end of file
diff --git a/shrink/mDeBERTa-v3-base-xnli-multilingual-nli-2mil7-mic.html b/shrink/mDeBERTa-v3-base-xnli-multilingual-nli-2mil7-mic.html
new file mode 100644
index 0000000000000000000000000000000000000000..d3f1bf15da14a499799c07086e4f4267da127180
--- /dev/null
+++ b/shrink/mDeBERTa-v3-base-xnli-multilingual-nli-2mil7-mic.html
@@ -0,0 +1,34 @@
+
+
+Shrunk architecture
+
+
Shrunk architecture
huggingface-mic/mDeBERTa-v3-base-xnli-multilingual-nli-2mil7-mic@b088660049cbac11aedecd214c7612c0717f466c ← MoritzLaurer/mDeBERTa-v3-base-xnli-multilingual-nli-2mil7@b5113eb38ab63efdd7f280f8c144ea8b13f978ce · DebertaV2ForSequenceClassification · overfit weights · built before the shrink rule (no recorded recipe)
+
Original278.8Mparameters
Fixture16.2Mparameters
Layer stacks1compared
Findings0to review
✓ Same architecture, smaller
Every module class, layer kind and checked property of the original is in the fixture.
Layer stacks
One square per layer, colored by kind: the layer's module classes and type attributes. Hover a square for its index and kind.
deberta.encoder.layer
Original · 12
Fixture · 2
Kind
Original
Fixture
Layer
12
2
✓ kept
Where the parameters are
Share of each model's parameters per component. Vocabulary and patch sizes are kept, so embeddings usually take a larger share of the fixture.
Original
Fixture
Component
Original
Share
Fixture
Share
deberta.embeddings
192.8M
69.1%
16.1M
99.4%
deberta.encoder.layer
85.1M
30.5%
66.9K
0.4%
pooler.dense
590.6K
0.2%
4.2K
0.0%
deberta.encoder
394.8K
0.1%
32.9K
0.2%
classifier
2.3K
0.0%
195
0.0%
Preserved properties
Config
Property
Original
Fixture
model
model_type
deberta-v2
deberta-v2
✓ kept
model
vocab_size
251000
251000
✓ kept
model
tie_word_embeddings
True
True
✓ kept
model
hidden_act
gelu
gelu
✓ kept
model
max_position_embeddings
512
512
✓ kept
Module classes
Instances of each class in the built model (prefix DebertaV2 omitted).
Class
Original
Fixture
Linear
74
14
Dropout
51
11
LayerNorm
26
6
Attention
12
2
Intermediate
12
2
Layer
12
2
Output
12
2
SelfOutput
12
2
DisentangledSelfAttention
12
2
GELUActivation
12
2
Embedding
2
2
ContextPooler
1
1
Embeddings
1
1
Encoder
1
1
ForSequenceClassification
1
1
Model
1
1
ModuleList
1
1
Config changes
5 fields changed, 31 unchanged.
Field
Original
Fixture
hidden_size
768
64
intermediate_size
3072
128
num_attention_heads
12
4
num_hidden_layers
12
2
pooler_hidden_size
768
64
+
\ No newline at end of file
diff --git a/shrink/mamba-130m-hf-mic.html b/shrink/mamba-130m-hf-mic.html
new file mode 100644
index 0000000000000000000000000000000000000000..be92ba4d4f582ce6690e1735958321b72eb02ce9
--- /dev/null
+++ b/shrink/mamba-130m-hf-mic.html
@@ -0,0 +1,34 @@
+
+
+Shrunk architecture
+
+
Shrunk architecture
huggingface-mic/mamba-130m-hf-mic@049c3bff1eb71679fcc5bf5ffcd2c99dc505f11b ← state-spaces/mamba-130m-hf@1e76775f628fbf1350fbe4dbb3d971ba64af25a1 · MambaForCausalLM · random weights
+
Original129.1Mparameters
Fixture3.3Mparameters
Layer stacks1compared
Findings0to review
✓ Same architecture, smaller
Every module class, layer kind and checked property of the original is in the fixture.
Layer stacks
One square per layer, colored by kind: the layer's module classes and type attributes. Hover a square for its index and kind.
backbone.layers
Original · 24
Fixture · 2
Kind
Original
Fixture
Block
24
2
✓ kept
Where the parameters are
Share of each model's parameters per component. Vocabulary and patch sizes are kept, so embeddings usually take a larger share of the fixture.
Original
Fixture
Component
Original
Share
Fixture
Share
backbone.layers
90.5M
70.1%
87.9K
2.7%
backbone.embeddings
38.6M
29.9%
3.2M
97.3%
backbone.norm_f
768
0.0%
64
0.0%
Preserved properties
Config
Property
Original
Fixture
model
model_type
mamba
mamba
✓ kept
model
vocab_size
50280
50280
✓ kept
model
tie_word_embeddings
True
True
✓ kept
model
hidden_act
silu
silu
✓ kept
Module classes
Instances of each class in the built model (prefix Mamba omitted).
Class
Original
Fixture
Linear
97
9
RMSNorm
25
3
Conv1d
24
2
Block
24
2
Mixer
24
2
SiLUActivation
24
2
Embedding
1
1
ForCausalLM
1
1
Model
1
1
ModuleList
1
1
Config changes
5 fields changed, 38 unchanged.
Field
Original
Fixture
d_model
768
64
hidden_size
768
64
intermediate_size
1536
128
n_layer
24
2
num_hidden_layers
24
2
+
\ No newline at end of file
diff --git a/shrink/mask2former-swin-tiny-coco-instance-mic.html b/shrink/mask2former-swin-tiny-coco-instance-mic.html
new file mode 100644
index 0000000000000000000000000000000000000000..f9bb51daebcc014e3cb45044902fc61626f6eb51
--- /dev/null
+++ b/shrink/mask2former-swin-tiny-coco-instance-mic.html
@@ -0,0 +1,34 @@
+
+
+Shrunk architecture
+
+
Shrunk architecture
huggingface-mic/mask2former-swin-tiny-coco-instance-mic@851c4b7fb35cc3ef537b77c5ad1164f3f1747f3a ← facebook/mask2former-swin-tiny-coco-instance@22c4a2f15dc88149b8b8d9f4d42c54431fbd66f6 · Mask2FormerForUniversalSegmentation · random weights
+
Original47.4Mparameters
Fixture8.0Mparameters
Layer stacks4compared
Findings0to review
✓ Same architecture, smaller
Every module class, layer kind and checked property of the original is in the fixture.
Layer stacks
One square per layer, colored by kind: the layer's module classes and type attributes. Hover a square for its index and kind.
+
\ No newline at end of file
diff --git a/shrink/mbart-large-50-mic.html b/shrink/mbart-large-50-mic.html
new file mode 100644
index 0000000000000000000000000000000000000000..b8488ce8874038d045b13df332dad58d9d79ab5f
--- /dev/null
+++ b/shrink/mbart-large-50-mic.html
@@ -0,0 +1,34 @@
+
+
+Shrunk architecture
+
+
Shrunk architecture
huggingface-mic/mbart-large-50-mic@ef21bcc00bb0aac630f0218f55b5c88b59a01fb5 ← facebook/mbart-large-50@4ef55a20b36c6903b832e38f0604ab4bdf22c7d6 · MBartForConditionalGeneration · random weights
+
Original610.9Mparameters
Fixture16.3Mparameters
Layer stacks2compared
Findings0to review
✓ Same architecture, smaller
Every module class, layer kind and checked property of the original is in the fixture.
Layer stacks
One square per layer, colored by kind: the layer's module classes and type attributes. Hover a square for its index and kind.
model.encoder.layers
Original · 12
Fixture · 2
Kind
Original
Fixture
EncoderLayer
12
2
✓ kept
model.decoder.layers
Original · 12
Fixture · 2
Kind
Original
Fixture
DecoderLayer
12
2
✓ kept
Where the parameters are
Share of each model's parameters per component. Vocabulary and patch sizes are kept, so embeddings usually take a larger share of the fixture.
Original
Fixture
Component
Original
Share
Fixture
Share
model.shared
256.1M
41.9%
16.0M
98.2%
model.decoder.layers
201.6M
33.0%
100.5K
0.6%
model.encoder.layers
151.2M
24.7%
66.9K
0.4%
model.decoder
1.1M
0.2%
65.9K
0.4%
model.encoder
1.1M
0.2%
65.9K
0.4%
Preserved properties
Config
Property
Original
Fixture
model
model_type
mbart
mbart
✓ kept
model
vocab_size
250054
250054
✓ kept
model
tie_word_embeddings
True
True
✓ kept
model
max_position_embeddings
1024
1024
✓ kept
Module classes
Instances of each class in the built model (prefix MBart omitted).
Class
Original
Fixture
Linear
193
33
LayerNorm
64
14
Attention
36
6
GELUActivation
24
4
DecoderLayer
12
2
EncoderLayer
12
2
ScaledWordEmbedding
3
3
LearnedPositionalEmbedding
2
2
ModuleList
2
2
Decoder
1
1
Encoder
1
1
ForConditionalGeneration
1
1
Model
1
1
Config changes
7 fields changed, 38 unchanged.
Field
Original
Fixture
d_model
1024
64
decoder_attention_heads
16
4
decoder_ffn_dim
4096
128
decoder_layers
12
2
encoder_attention_heads
16
4
encoder_ffn_dim
4096
128
encoder_layers
12
2
+
\ No newline at end of file
diff --git a/shrink/mistral3-mic.html b/shrink/mistral3-mic.html
new file mode 100644
index 0000000000000000000000000000000000000000..80113077c29cb2a5f0db7e33e08e50933a585032
--- /dev/null
+++ b/shrink/mistral3-mic.html
@@ -0,0 +1,34 @@
+
+
+Shrunk architecture
+
+
Shrunk architecture
huggingface-mic/mistral3-mic@ad30ba063d8bf5a68e71a54668e8eb4a8ebd2a94 ← mistralai/Mistral-Small-3.1-24B-Instruct-2503@68faf511d618ef198fef186659617cfd2eb8e33a · Mistral3ForConditionalGeneration · random weights · built before the shrink rule (no recorded recipe)
+
Original23.3Bparameters
Fixture8.6Mparameters
Layer stacks2compared
Findings0to review
✓ Same architecture, smaller
Every module class, layer kind and checked property of the original is in the fixture.
Layer stacks
One square per layer, colored by kind: the layer's module classes and type attributes. Hover a square for its index and kind.
model.vision_tower.transformer.layers
Original · 24
Fixture · 2
Kind
Original
Fixture
PixtralAttentionLayer
24
2
✓ kept
model.language_model.layers
Original · 40
Fixture · 2
Kind
Original
Fixture
MistralDecoderLayer
40
2
✓ kept
Where the parameters are
Share of each model's parameters per component. Vocabulary and patch sizes are kept, so embeddings usually take a larger share of the fixture.
Original
Fixture
Component
Original
Share
Fixture
Share
model.language_model.layers
22.2B
95.2%
74.0K
0.9%
model.language_model
671.1M
2.9%
8.4M
97.5%
model.vision_tower.transformer.layers
402.7M
1.7%
82.2K
1.0%
model.multi_modal_projector
35.7M
0.2%
24.6K
0.3%
model.vision_tower
603.1K
0.0%
37.7K
0.4%
Preserved properties
Config
Property
Original
Fixture
model
model_type
mistral3
mistral3
✓ kept
model
tie_word_embeddings
True
True
✓ kept
vision_config
RoPE type
axial
axial
✓ kept
vision_config
model_type
pixtral
pixtral
✓ kept
vision_config
hidden_act
silu
silu
✓ kept
vision_config
image_size
1540
1540
✓ kept
vision_config
patch_size
14
14
✓ kept
vision_config
num_channels
3
3
✓ kept
text_config
attention style
GQA (32 heads, 8 KV)
GQA (4 heads, 2 KV)
✓ kept
text_config
RoPE type
default
default
✓ kept
text_config
model_type
mistral
mistral
✓ kept
text_config
vocab_size
131072
131072
✓ kept
text_config
tie_word_embeddings
False
False
✓ kept
text_config
hidden_act
silu
silu
✓ kept
text_config
max_position_embeddings
131072
131072
✓ kept
Module classes
Instances of each class in the built model (prefix Mistral3 omitted).
Class
Original
Fixture
Linear
452
32
MistralRMSNorm
81
5
SiLUActivation
64
4
PixtralRMSNorm
49
5
MistralAttention
40
2
MistralDecoderLayer
40
2
MistralMLP
40
2
PixtralAttention
24
2
PixtralAttentionLayer
24
2
PixtralMLP
24
2
ModuleList
2
2
Conv2d
1
1
Embedding
1
1
GELUActivation
1
1
ForConditionalGeneration
1
1
Model
1
1
MultiModalProjector
1
1
PatchMerger
1
1
RMSNorm
1
1
MistralModel
1
1
MistralRotaryEmbedding
1
1
PixtralTransformer
1
1
PixtralVisionModel
1
1
PixtralVisionRotaryEmbedding
1
1
Config changes
11 fields changed, 53 unchanged.
Field
Original
Fixture
text_config.head_dim
128
16
text_config.hidden_size
5120
64
text_config.intermediate_size
32768
128
text_config.num_attention_heads
32
4
text_config.num_hidden_layers
40
2
text_config.num_key_value_heads
8
2
vision_config.head_dim
64
16
vision_config.hidden_size
1024
64
vision_config.intermediate_size
4096
128
vision_config.num_attention_heads
16
4
vision_config.num_hidden_layers
24
2
+
\ No newline at end of file
diff --git a/shrink/mm_grounding_dino_tiny_o365v1_goldg_v3det-mic.html b/shrink/mm_grounding_dino_tiny_o365v1_goldg_v3det-mic.html
new file mode 100644
index 0000000000000000000000000000000000000000..861b0e2d0074e58017616f9cba85aba458673894
--- /dev/null
+++ b/shrink/mm_grounding_dino_tiny_o365v1_goldg_v3det-mic.html
@@ -0,0 +1,34 @@
+
+
+Shrunk architecture
+
+
Shrunk architecture
huggingface-mic/mm_grounding_dino_tiny_o365v1_goldg_v3det-mic@3b6c70187a273832016dad164af1af9a796a0bbd ← openmmlab-community/mm_grounding_dino_tiny_o365v1_goldg_v3det@ef2053aa1e0cc9d9cee1df31ca919ee9aae4fc8e · MMGroundingDinoForObjectDetection · random weights · built before the shrink rule (no recorded recipe)
+
Original172.9Mparameters
Fixture7.7Mparameters
Layer stacks7compared
Findings0to review
✓ Same architecture, smaller
Every module class, layer kind and checked property of the original is in the fixture.
Layer stacks
One square per layer, colored by kind: the layer's module classes and type attributes. Hover a square for its index and kind.
Every module class, layer kind and checked property of the original is in the fixture.
Layer stacks
One square per layer, colored by kind: the layer's module classes and type attributes. Hover a square for its index and kind.
mobilenet_v2.layer
Original · 16
Fixture · 16
Kind
Original
Fixture
ConvLayer
16
16
✓ kept
Where the parameters are
Share of each model's parameters per component. Vocabulary and patch sizes are kept, so embeddings usually take a larger share of the fixture.
Original
Fixture
Component
Original
Share
Fixture
Share
mobilenet_v2.layer
1.8M
51.6%
1.8M
51.6%
classifier
1.3M
36.6%
1.3M
36.6%
mobilenet_v2.conv_1x1
412.2K
11.8%
412.2K
11.8%
mobilenet_v2.conv_stem
1.8K
0.1%
1.8K
0.1%
Preserved properties
Config
Property
Original
Fixture
model
model_type
mobilenet_v2
mobilenet_v2
✓ kept
model
hidden_act
relu6
relu6
✓ kept
model
image_size
224
224
✓ kept
model
num_channels
3
3
✓ kept
Module classes
Instances of each class in the built model (prefix MobileNetV2 omitted).
Class
Original
Fixture
BatchNorm2d
52
52
Conv2d
52
52
ConvLayer
52
52
ReLU6
35
35
InvertedResidual
16
16
AdaptiveAvgPool2d
1
1
Dropout
1
1
Linear
1
1
ForImageClassification
1
1
Model
1
1
Stem
1
1
ModuleList
1
1
Config changes
1 fields changed, 23 unchanged.
Field
Original
Fixture
problem_type
null
single_label_classification
+
\ No newline at end of file
diff --git a/shrink/mobilevit-xx-small-mic.html b/shrink/mobilevit-xx-small-mic.html
new file mode 100644
index 0000000000000000000000000000000000000000..b3c8b101e5ac998c8a9f65797e3b4ea438fdeaaa
--- /dev/null
+++ b/shrink/mobilevit-xx-small-mic.html
@@ -0,0 +1,34 @@
+
+
+Shrunk architecture
+
+
Shrunk architecture
huggingface-mic/mobilevit-xx-small-mic@1d4aa4d88fb85d89dc12a8e52ffcd0c66cf855ec ← apple/mobilevit-xx-small@6703997f9e940e312a04b8b7c26463d631c0d1f8 · MobileViTForImageClassification · overfit weights · built before the shrink rule (no recorded recipe)
+
Original1.3Mparameters
Fixture895.3Kparameters
Layer stacks1compared
Findings0to review
✓ Same architecture, smaller
Every module class, layer kind and checked property of the original is in the fixture.
Layer stacks
One square per layer, colored by kind: the layer's module classes and type attributes. Hover a square for its index and kind.
mobilevit.encoder.layer
Original · 5
Fixture · 5
Kind
Original
Fixture
MobileNetLayer
2
2
✓ kept
LayerNorm, Linear, Attention …
3
3
✓ kept
Where the parameters are
Share of each model's parameters per component. Vocabulary and patch sizes are kept, so embeddings usually take a larger share of the fixture.
Original
Fixture
Component
Original
Share
Fixture
Share
mobilevit.encoder.layer
924.3K
72.7%
547.6K
61.2%
classifier
321.0K
25.2%
321.0K
35.9%
mobilevit.conv_1x1_exp
26.2K
2.1%
26.2K
2.9%
mobilevit.conv_stem
464
0.0%
464
0.1%
Preserved properties
Config
Property
Original
Fixture
model
model_type
mobilevit
mobilevit
✓ kept
model
hidden_act
silu
silu
✓ kept
model
image_size
256
256
✓ kept
model
patch_size
2
2
✓ kept
model
num_channels
3
3
✓ kept
Module classes
Instances of each class in the built model (prefix MobileViT omitted).
Class
Original
Fixture
Linear
55
55
Conv2d
35
35
ConvLayer
35
35
SiLUActivation
34
34
BatchNorm2d
32
32
Dropout
28
28
LayerNorm
21
21
Attention
9
9
Intermediate
9
9
Output
9
9
SelfAttention
9
9
SelfOutput
9
9
TransformerLayer
9
9
InvertedResidual
7
7
ModuleList
6
6
Layer
3
3
Transformer
3
3
MobileNetLayer
2
2
Encoder
1
1
ForImageClassification
1
1
Model
1
1
Config changes
2 fields changed, 28 unchanged.
Field
Original
Fixture
hidden_sizes
[64, 80, 96]
[16, 32, 64]
problem_type
null
single_label_classification
+
\ No newline at end of file
diff --git a/shrink/modernbert-mic.html b/shrink/modernbert-mic.html
new file mode 100644
index 0000000000000000000000000000000000000000..13e6a1a22a1da2c74bc10c3035fa4c5b359c0cbb
--- /dev/null
+++ b/shrink/modernbert-mic.html
@@ -0,0 +1,34 @@
+
+
+Shrunk architecture
+
+
Shrunk architecture
huggingface-mic/modernbert-mic@59688bef85e5803f4ebc8a02227907e465355a30 ← answerdotai/ModernBERT-base@8949b909ec900327062f0ebf497f51aef5e6f0c8 · ModernBertForMaskedLM · random weights · built before the shrink rule (no recorded recipe)
+
Original149.7Mparameters
Fixture3.4Mparameters
Layer stacks1compared
Findings0to review
✓ Same architecture, smaller
Every module class, layer kind and checked property of the original is in the fixture.
Layer stacks
One square per layer, colored by kind: the layer's module classes and type attributes. Hover a square for its index and kind.
model.layers
Original · 22
Fixture · 2
Kind
Original
Fixture
full_attention
8
1
✓ kept
sliding_attention
14
1
✓ kept
Where the parameters are
Share of each model's parameters per component. Vocabulary and patch sizes are kept, so embeddings usually take a larger share of the fixture.
Original
Fixture
Component
Original
Share
Fixture
Share
model.layers
110.3M
73.7%
82.1K
2.4%
model.embeddings
38.7M
25.8%
3.2M
95.9%
head.dense
589.8K
0.4%
4.1K
0.1%
decoder
50.4K
0.0%
50.4K
1.5%
head.norm
768
0.0%
64
0.0%
model.final_norm
768
0.0%
64
0.0%
Preserved properties
Config
Property
Original
Fixture
model
model_type
modernbert
modernbert
✓ kept
model
vocab_size
50368
50368
✓ kept
model
tie_word_embeddings
True
True
✓ kept
model
max_position_embeddings
8192
8192
✓ kept
Module classes
Instances of each class in the built model (prefix ModernBert omitted).
+
\ No newline at end of file
diff --git a/shrink/mpnet-base-mic.html b/shrink/mpnet-base-mic.html
new file mode 100644
index 0000000000000000000000000000000000000000..1467b58ce16e82074eba17af415b1319479ba436
--- /dev/null
+++ b/shrink/mpnet-base-mic.html
@@ -0,0 +1,34 @@
+
+
+Shrunk architecture
+
+
Shrunk architecture
huggingface-mic/mpnet-base-mic@9d98c3d72a6cae57a9f3ed7f52cf9144628a5f6e ← microsoft/mpnet-base@6996ce1e91bd2a9c7d7f61daec37463394f73f09 · MPNetForMaskedLM · random weights · built before the shrink rule (no recorded recipe)
+
Original109.5Mparameters
Fixture2.1Mparameters
Layer stacks1compared
Findings0to review
✓ Same architecture, smaller
Every module class, layer kind and checked property of the original is in the fixture.
Layer stacks
One square per layer, colored by kind: the layer's module classes and type attributes. Hover a square for its index and kind.
mpnet.encoder.layer
Original · 12
Fixture · 2
Kind
Original
Fixture
Layer
12
2
✓ kept
Where the parameters are
Share of each model's parameters per component. Vocabulary and patch sizes are kept, so embeddings usually take a larger share of the fixture.
Original
Fixture
Component
Original
Share
Fixture
Share
mpnet.encoder.layer
85.1M
77.7%
66.9K
3.2%
mpnet.embeddings
23.8M
21.8%
2.0M
95.1%
lm_head.dense
590.6K
0.5%
4.2K
0.2%
lm_head
30.5K
0.0%
30.5K
1.5%
lm_head.layer_norm
1.5K
0.0%
128
0.0%
mpnet.encoder
384
0.0%
128
0.0%
Preserved properties
Config
Property
Original
Fixture
model
model_type
mpnet
mpnet
✓ kept
model
vocab_size
30527
30527
✓ kept
model
tie_word_embeddings
True
True
✓ kept
model
hidden_act
gelu
gelu
✓ kept
model
max_position_embeddings
514
514
✓ kept
Module classes
Instances of each class in the built model (prefix MPNet omitted).
Class
Original
Fixture
Linear
74
14
Dropout
37
7
LayerNorm
26
6
GELUActivation
12
2
Attention
12
2
Intermediate
12
2
Layer
12
2
Output
12
2
SelfAttention
12
2
Embedding
3
3
Embeddings
1
1
Encoder
1
1
ForMaskedLM
1
1
LMHead
1
1
Model
1
1
ModuleList
1
1
Config changes
4 fields changed, 21 unchanged.
Field
Original
Fixture
hidden_size
768
64
intermediate_size
3072
128
num_attention_heads
12
4
num_hidden_layers
12
2
+
\ No newline at end of file
diff --git a/shrink/mt5-small-mic.html b/shrink/mt5-small-mic.html
new file mode 100644
index 0000000000000000000000000000000000000000..412612abd83efc7b559519552d309ef3589fbe72
--- /dev/null
+++ b/shrink/mt5-small-mic.html
@@ -0,0 +1,34 @@
+
+
+Shrunk architecture
+
+
Shrunk architecture
huggingface-mic/mt5-small-mic@13bc3e6cb33e75d32f533822c690e2115f739038 ← google/mt5-small@73fb5dbe4756edadc8fbe8c769b0a109493acf7a · MT5ForConditionalGeneration · random weights
+
Original172.1Mparameters
Fixture16.2Mparameters
Layer stacks2compared
Findings0to review
✓ Same architecture, smaller
Every module class, layer kind and checked property of the original is in the fixture.
Layer stacks
One square per layer, colored by kind: the layer's module classes and type attributes. Hover a square for its index and kind.
encoder.block
Original · 8
Fixture · 2
Kind
Original
Fixture
Embedding
1
1
✓ kept
no Embedding
7
1
✓ kept
decoder.block
Original · 8
Fixture · 2
Kind
Original
Fixture
Embedding
1
1
✓ kept
no Embedding
7
1
✓ kept
Where the parameters are
Share of each model's parameters per component. Vocabulary and patch sizes are kept, so embeddings usually take a larger share of the fixture.
Original
Fixture
Component
Original
Share
Fixture
Share
shared
128.1M
74.4%
16.0M
98.8%
decoder.block
25.2M
14.6%
115.2K
0.7%
encoder.block
18.9M
11.0%
82.3K
0.5%
encoder.final_layer_norm
512
0.0%
64
0.0%
decoder.final_layer_norm
512
0.0%
64
0.0%
Preserved properties
Config
Property
Original
Fixture
model
model_type
mt5
mt5
✓ kept
model
vocab_size
250112
250112
✓ kept
model
tie_word_embeddings
True
True
✓ kept
Module classes
Instances of each class in the built model (prefix MT5 omitted).
Class
Original
Fixture
Linear
145
37
Dropout
58
16
LayerNorm
42
12
Attention
24
6
ModuleList
18
6
Block
16
4
DenseGatedActDense
16
4
LayerFF
16
4
LayerSelfAttention
16
4
NewGELUActivation
16
4
LayerCrossAttention
8
2
Embedding
5
5
Stack
2
2
ForConditionalGeneration
1
1
Config changes
6 fields changed, 27 unchanged.
Field
Original
Fixture
d_ff
1024
128
d_kv
64
16
d_model
512
64
num_decoder_layers
8
2
num_heads
6
4
num_layers
8
2
+
\ No newline at end of file
diff --git a/shrink/nllb-200-distilled-600M-mic.html b/shrink/nllb-200-distilled-600M-mic.html
new file mode 100644
index 0000000000000000000000000000000000000000..0400cce4f9c438e636af5f1a2cd431dfa7bef402
--- /dev/null
+++ b/shrink/nllb-200-distilled-600M-mic.html
@@ -0,0 +1,34 @@
+
+
+Shrunk architecture
+
+
Shrunk architecture
huggingface-mic/nllb-200-distilled-600M-mic@fcaaa8d89cb3bc751afd9c7511e8f2975319ea14 ← facebook/nllb-200-distilled-600M@f8d333a098d19b4fd9a8b18f94170487ad3f821d · M2M100ForConditionalGeneration · random weights · built before the shrink rule (no recorded recipe)
+
Original615.1Mparameters
Fixture16.6Mparameters
Layer stacks2compared
Findings0to review
✓ Same architecture, smaller
Every module class, layer kind and checked property of the original is in the fixture.
Layer stacks
One square per layer, colored by kind: the layer's module classes and type attributes. Hover a square for its index and kind.
model.encoder.layers
Original · 12
Fixture · 2
Kind
Original
Fixture
EncoderLayer
12
2
✓ kept
model.decoder.layers
Original · 12
Fixture · 2
Kind
Original
Fixture
DecoderLayer
12
2
✓ kept
Where the parameters are
Share of each model's parameters per component. Vocabulary and patch sizes are kept, so embeddings usually take a larger share of the fixture.
Original
Fixture
Component
Original
Share
Fixture
Share
model.shared
262.4M
42.7%
16.4M
99.0%
model.decoder.layers
201.6M
32.8%
100.5K
0.6%
model.encoder.layers
151.2M
24.6%
66.9K
0.4%
model.decoder
2.0K
0.0%
128
0.0%
model.encoder
2.0K
0.0%
128
0.0%
Preserved properties
Config
Property
Original
Fixture
model
model_type
m2m_100
m2m_100
✓ kept
model
vocab_size
256206
256206
✓ kept
model
tie_word_embeddings
True
True
✓ kept
model
max_position_embeddings
1024
1024
✓ kept
Module classes
Instances of each class in the built model (prefix M2M100 omitted).
Class
Original
Fixture
Linear
193
33
LayerNorm
62
12
Attention
36
6
ReLU
24
4
DecoderLayer
12
2
EncoderLayer
12
2
ScaledWordEmbedding
3
3
SinusoidalPositionalEmbedding
2
2
ModuleList
2
2
Decoder
1
1
Encoder
1
1
ForConditionalGeneration
1
1
Model
1
1
Config changes
7 fields changed, 26 unchanged.
Field
Original
Fixture
d_model
1024
64
decoder_attention_heads
16
4
decoder_ffn_dim
4096
128
decoder_layers
12
2
encoder_attention_heads
16
4
encoder_ffn_dim
4096
128
encoder_layers
12
2
+
\ No newline at end of file
diff --git a/shrink/nomic-embed-text-v1.5-mic.html b/shrink/nomic-embed-text-v1.5-mic.html
new file mode 100644
index 0000000000000000000000000000000000000000..7406f935e0a1a829a7de204fabab9b6cc4b93b3d
--- /dev/null
+++ b/shrink/nomic-embed-text-v1.5-mic.html
@@ -0,0 +1,34 @@
+
+
+Shrunk architecture
+
+
Shrunk architecture
huggingface-mic/nomic-embed-text-v1.5-mic@2cbaf2971908b9ad612ba5d9b74e819440dc5344 ← nomic-ai/nomic-embed-text-v1.5@e9b6763023c676ca8431644204f50c2b100d9aab · NomicBertModel · random weights · built before the shrink rule (no recorded recipe)
+
Original136.7Mparameters
Fixture2.0Mparameters
Layer stacks1compared
Findings0to review
✓ Same architecture, smaller
Every module class, layer kind and checked property of the original is in the fixture.
Layer stacks
One square per layer, colored by kind: the layer's module classes and type attributes. Hover a square for its index and kind.
layers
Original · 12
Fixture · 2
Kind
Original
Fixture
Layer
12
2
✓ kept
Where the parameters are
Share of each model's parameters per component. Vocabulary and patch sizes are kept, so embeddings usually take a larger share of the fixture.
Original
Fixture
Component
Original
Share
Fixture
Share
layers
113.3M
82.9%
82.4K
4.0%
embeddings.word_embeddings
23.4M
17.1%
2.0M
95.9%
embeddings.token_type_embeddings
1.5K
0.0%
128
0.0%
embeddings.LayerNorm
1.5K
0.0%
128
0.0%
Preserved properties
Config
Property
Original
Fixture
model
RoPE type
default
default
✓ kept
model
model_type
nomic_bert
nomic_bert
✓ kept
model
vocab_size
30528
30528
✓ kept
model
hidden_act
silu
silu
✓ kept
model
max_position_embeddings
2048
2048
✓ kept
Module classes
Instances of each class in the built model (prefix NomicBert omitted).
Class
Original
Fixture
Linear
84
14
Dropout
25
5
LayerNorm
25
5
Attention
12
2
Layer
12
2
MLP
12
2
SiLUActivation
12
2
Embedding
2
2
ModuleList
1
1
Embeddings
1
1
Model
1
1
RotaryEmbedding
1
1
Config changes
9 fields changed, 57 unchanged.
Field
Original
Fixture
head_dim
64
16
hidden_size
768
64
intermediate_size
3072
128
n_embd
768
64
n_head
12
4
n_inner
3072
128
n_layer
12
2
num_attention_heads
12
4
num_hidden_layers
12
2
+
\ No newline at end of file
diff --git a/shrink/opt-125m-mic.html b/shrink/opt-125m-mic.html
new file mode 100644
index 0000000000000000000000000000000000000000..6915e01cb6127ae4afd9519120512f3843e334f5
--- /dev/null
+++ b/shrink/opt-125m-mic.html
@@ -0,0 +1,34 @@
+
+
+Shrunk architecture
+
+
Shrunk architecture
huggingface-mic/opt-125m-mic@30dda77b685d9eb9cbf9efebdfdb2a5f14028349 ← facebook/opt-125m@27dcfa74d334bc871f3234de431e71c6eeba5dd6 · OPTForCausalLM · random weights · built before the shrink rule (no recorded recipe)
+
Original125.2Mparameters
Fixture38.9Mparameters
Layer stacks1compared
Findings0to review
✓ Same architecture, smaller
Every module class, layer kind and checked property of the original is in the fixture.
Layer stacks
One square per layer, colored by kind: the layer's module classes and type attributes. Hover a square for its index and kind.
model.decoder.layers
Original · 12
Fixture · 2
Kind
Original
Fixture
DecoderLayer
12
2
✓ kept
Where the parameters are
Share of each model's parameters per component. Vocabulary and patch sizes are kept, so embeddings usually take a larger share of the fixture.
Original
Fixture
Component
Original
Share
Fixture
Share
model.decoder.layers
85.1M
67.9%
66.9K
0.2%
model.decoder
40.2M
32.1%
38.8M
99.8%
Preserved properties
Config
Property
Original
Fixture
model
model_type
opt
opt
✓ kept
model
vocab_size
50272
50272
✓ kept
model
tie_word_embeddings
True
True
✓ kept
model
max_position_embeddings
2048
2048
✓ kept
Module classes
Instances of each class in the built model (prefix OPT omitted).
Class
Original
Fixture
Linear
73
15
LayerNorm
25
5
Attention
12
2
DecoderLayer
12
2
ReLU
12
2
Embedding
1
1
ModuleList
1
1
Decoder
1
1
ForCausalLM
1
1
LearnedPositionalEmbedding
1
1
Model
1
1
Config changes
4 fields changed, 28 unchanged.
Field
Original
Fixture
ffn_dim
3072
128
hidden_size
768
64
num_attention_heads
12
4
num_hidden_layers
12
2
+
\ No newline at end of file
diff --git a/shrink/opus-mt-fr-en-mic.html b/shrink/opus-mt-fr-en-mic.html
new file mode 100644
index 0000000000000000000000000000000000000000..a828bcf2b9c1b16fd7d9279aaabc39ecff260e63
--- /dev/null
+++ b/shrink/opus-mt-fr-en-mic.html
@@ -0,0 +1,34 @@
+
+
+Shrunk architecture
+
+
Shrunk architecture
huggingface-mic/opus-mt-fr-en-mic@d09105cda74d27a8ab952466f1810a83cc412893 ← Helsinki-NLP/opus-mt-fr-en@c4aed37b318c763fd177aa449b44e3b783cc6c02 · MarianMTModel · overfit weights · built before the shrink rule (no recorded recipe)
+
Original75.1Mparameters
Fixture4.0Mparameters
Layer stacks2compared
Findings0to review
✓ Same architecture, smaller
Every module class, layer kind and checked property of the original is in the fixture.
Layer stacks
One square per layer, colored by kind: the layer's module classes and type attributes. Hover a square for its index and kind.
model.encoder.layers
Original · 6
Fixture · 2
Kind
Original
Fixture
MarianEncoderLayer
6
2
✓ kept
model.decoder.layers
Original · 6
Fixture · 2
Kind
Original
Fixture
MarianDecoderLayer
6
2
✓ kept
Where the parameters are
Share of each model's parameters per component. Vocabulary and patch sizes are kept, so embeddings usually take a larger share of the fixture.
Original
Fixture
Component
Original
Share
Fixture
Share
model.shared
30.5M
40.6%
3.8M
94.2%
model.decoder.layers
25.2M
33.6%
100.5K
2.5%
model.encoder.layers
18.9M
25.2%
66.9K
1.7%
model.decoder
262.1K
0.3%
32.8K
0.8%
model.encoder
262.1K
0.3%
32.8K
0.8%
Preserved properties
Config
Property
Original
Fixture
model
model_type
marian
marian
✓ kept
model
vocab_size
59514
59514
✓ kept
model
tie_word_embeddings
True
True
✓ kept
model
max_position_embeddings
512
512
✓ kept
Module classes
Instances of each class in the built model (prefix MarianMT omitted).
Class
Original
Fixture
Linear
97
33
LayerNorm
30
10
MarianAttention
18
6
SiLU
12
4
MarianDecoderLayer
6
2
MarianEncoderLayer
6
2
Embedding
3
3
MarianSinusoidalPositionalEmbedding
2
2
ModuleList
2
2
MarianDecoder
1
1
MarianEncoder
1
1
Model
1
1
MarianModel
1
1
Config changes
7 fields changed, 38 unchanged.
Field
Original
Fixture
d_model
512
64
decoder_attention_heads
8
4
decoder_ffn_dim
2048
128
decoder_layers
6
2
encoder_attention_heads
8
4
encoder_ffn_dim
2048
128
encoder_layers
6
2
+
\ No newline at end of file
diff --git a/shrink/owlv2-base-patch16-ensemble-mic.html b/shrink/owlv2-base-patch16-ensemble-mic.html
new file mode 100644
index 0000000000000000000000000000000000000000..678cd41688ae11348678352cf9266223dbbad308
--- /dev/null
+++ b/shrink/owlv2-base-patch16-ensemble-mic.html
@@ -0,0 +1,34 @@
+
+
+Shrunk architecture
+
+
Shrunk architecture
huggingface-mic/owlv2-base-patch16-ensemble-mic@de5f6eb0e6f68273b883bf046cd0b1dc76bce3ad ← google/owlv2-base-patch16-ensemble@cfd3195ba4ea9592eec887ded089f4c08eff231d · Owlv2ForObjectDetection · random weights
+
Original155.0Mparameters
Fixture3.6Mparameters
Layer stacks2compared
Findings0to review
✓ Same architecture, smaller
Every module class, layer kind and checked property of the original is in the fixture.
Layer stacks
One square per layer, colored by kind: the layer's module classes and type attributes. Hover a square for its index and kind.
owlv2.text_model.encoder.layers
Original · 12
Fixture · 2
Kind
Original
Fixture
EncoderLayer
12
2
✓ kept
owlv2.vision_model.encoder.layers
Original · 12
Fixture · 2
Kind
Original
Fixture
EncoderLayer
12
2
✓ kept
Where the parameters are
Share of each model's parameters per component. Vocabulary and patch sizes are kept, so embeddings usually take a larger share of the fixture.
Original
Fixture
Component
Original
Share
Fixture
Share
owlv2.vision_model.encoder.layers
85.1M
54.9%
66.9K
1.9%
owlv2.text_model.encoder.layers
37.8M
24.4%
66.9K
1.9%
owlv2.text_model
25.3M
16.3%
3.2M
87.7%
owlv2.vision_model
3.4M
2.2%
279.9K
7.8%
box_head.dense0
590.6K
0.4%
4.2K
0.1%
objectness_head.dense1
590.6K
0.4%
4.2K
0.1%
box_head.dense1
590.6K
0.4%
4.2K
0.1%
10 others
1.6M
1.1%
17.1K
0.5%
Preserved properties
Config
Property
Original
Fixture
model
model_type
owlv2
owlv2
✓ kept
text_config
model_type
owlv2_text_model
owlv2_text_model
✓ kept
text_config
vocab_size
49408
49408
✓ kept
text_config
hidden_act
quick_gelu
quick_gelu
✓ kept
text_config
max_position_embeddings
16
16
✓ kept
vision_config
model_type
owlv2_vision_model
owlv2_vision_model
✓ kept
vision_config
hidden_act
quick_gelu
quick_gelu
✓ kept
vision_config
image_size
960
960
✓ kept
vision_config
patch_size
16
16
✓ kept
vision_config
num_channels
3
3
✓ kept
Module classes
Instances of each class in the built model (prefix Owlv2 omitted).
Class
Original
Fixture
Linear
155
35
LayerNorm
52
12
Attention
24
4
EncoderLayer
24
4
MLP
24
4
QuickGELUActivation
24
4
Embedding
3
3
GELU
2
2
ModuleList
2
2
BoxPredictionHead
2
2
Encoder
2
2
Conv2d
1
1
ELU
1
1
ClassPredictionHead
1
1
ForObjectDetection
1
1
Model
1
1
TextEmbeddings
1
1
TextTransformer
1
1
VisionEmbeddings
1
1
VisionTransformer
1
1
Sigmoid
1
1
Config changes
9 fields changed, 47 unchanged.
Field
Original
Fixture
projection_dim
512
64
text_config.hidden_size
512
64
text_config.intermediate_size
2048
128
text_config.num_attention_heads
8
4
text_config.num_hidden_layers
12
2
vision_config.hidden_size
768
64
vision_config.intermediate_size
3072
128
vision_config.num_attention_heads
12
4
vision_config.num_hidden_layers
12
2
+
\ No newline at end of file
diff --git a/shrink/owlvit-base-patch32-mic.html b/shrink/owlvit-base-patch32-mic.html
new file mode 100644
index 0000000000000000000000000000000000000000..0f7aa3b44631582b0125220905439b3c29c16665
--- /dev/null
+++ b/shrink/owlvit-base-patch32-mic.html
@@ -0,0 +1,34 @@
+
+
+Shrunk architecture
+
+
Shrunk architecture
huggingface-mic/owlvit-base-patch32-mic@09d714a0dd1f21dcf214cb144777eb72042a08d0 ← google/owlvit-base-patch32@cbc355fb364588351c5d51c7f74465e8e7ec6f72 · OwlViTForObjectDetection · random weights
+
Original153.2Mparameters
Fixture3.6Mparameters
Layer stacks2compared
Findings0to review
✓ Same architecture, smaller
Every module class, layer kind and checked property of the original is in the fixture.
Layer stacks
One square per layer, colored by kind: the layer's module classes and type attributes. Hover a square for its index and kind.
owlvit.text_model.encoder.layers
Original · 12
Fixture · 2
Kind
Original
Fixture
EncoderLayer
12
2
✓ kept
owlvit.vision_model.encoder.layers
Original · 12
Fixture · 2
Kind
Original
Fixture
EncoderLayer
12
2
✓ kept
Where the parameters are
Share of each model's parameters per component. Vocabulary and patch sizes are kept, so embeddings usually take a larger share of the fixture.
Original
Fixture
Component
Original
Share
Fixture
Share
owlvit.vision_model.encoder.layers
85.1M
55.5%
66.9K
1.9%
owlvit.text_model.encoder.layers
37.8M
24.7%
66.9K
1.9%
owlvit.text_model
25.3M
16.5%
3.2M
89.1%
owlvit.vision_model
2.8M
1.8%
233.9K
6.6%
box_head.dense1
590.6K
0.4%
4.2K
0.1%
box_head.dense0
590.6K
0.4%
4.2K
0.1%
class_head.dense0
393.7K
0.3%
4.2K
0.1%
7 others
661.5K
0.4%
8.7K
0.2%
Preserved properties
Config
Property
Original
Fixture
model
model_type
owlvit
owlvit
✓ kept
text_config
model_type
owlvit_text_model
owlvit_text_model
✓ kept
text_config
vocab_size
49408
49408
✓ kept
text_config
tie_word_embeddings
True
True
✓ kept
text_config
hidden_act
quick_gelu
quick_gelu
✓ kept
text_config
max_position_embeddings
16
16
✓ kept
vision_config
model_type
owlvit_vision_model
owlvit_vision_model
✓ kept
vision_config
tie_word_embeddings
True
True
✓ kept
vision_config
hidden_act
quick_gelu
quick_gelu
✓ kept
vision_config
image_size
768
768
✓ kept
vision_config
patch_size
32
32
✓ kept
vision_config
num_channels
3
3
✓ kept
Module classes
Instances of each class in the built model (prefix OwlViT omitted).
Class
Original
Fixture
Linear
152
32
LayerNorm
52
12
Attention
24
4
EncoderLayer
24
4
MLP
24
4
QuickGELUActivation
24
4
Embedding
3
3
ModuleList
2
2
Encoder
2
2
Conv2d
1
1
ELU
1
1
GELU
1
1
BoxPredictionHead
1
1
ClassPredictionHead
1
1
ForObjectDetection
1
1
Model
1
1
TextEmbeddings
1
1
TextTransformer
1
1
VisionEmbeddings
1
1
VisionTransformer
1
1
Sigmoid
1
1
Config changes
9 fields changed, 84 unchanged.
Field
Original
Fixture
projection_dim
512
64
text_config.hidden_size
512
64
text_config.intermediate_size
2048
128
text_config.num_attention_heads
8
4
text_config.num_hidden_layers
12
2
vision_config.hidden_size
768
64
vision_config.intermediate_size
3072
128
vision_config.num_attention_heads
12
4
vision_config.num_hidden_layers
12
2
+
\ No newline at end of file
diff --git a/shrink/paligemma-3b-pt-224-mic.html b/shrink/paligemma-3b-pt-224-mic.html
new file mode 100644
index 0000000000000000000000000000000000000000..b3fed4df6136af4ca480293bbe604262e1b9b16b
--- /dev/null
+++ b/shrink/paligemma-3b-pt-224-mic.html
@@ -0,0 +1,34 @@
+
+
+Shrunk architecture
+
+
Shrunk architecture
huggingface-mic/paligemma-3b-pt-224-mic@adfd7e09d295b39ddf4dbeb88ffdb645139b72d8 ← leo009/paligemma-3b-pt-224@39996beb6fb17c5d16a50d3ef8f7a96ad9d03986 · PaliGemmaForConditionalGeneration · random weights
+
Original2.9Bparameters
Fixture16.7Mparameters
Layer stacks2compared
Findings0to review
✓ Same architecture, smaller
Every module class, layer kind and checked property of the original is in the fixture.
Layer stacks
One square per layer, colored by kind: the layer's module classes and type attributes. Hover a square for its index and kind.
model.vision_tower.encoder.layers
Original · 27
Fixture · 2
Kind
Original
Fixture
SiglipEncoderLayer
27
2
✓ kept
model.language_model.layers
Original · 18
Fixture · 2
Kind
Original
Fixture
GemmaDecoderLayer
18
2
✓ kept
Where the parameters are
Share of each model's parameters per component. Vocabulary and patch sizes are kept, so embeddings usually take a larger share of the fixture.
Original
Fixture
Component
Original
Share
Fixture
Share
model.language_model.layers
2.0B
67.8%
69.9K
0.4%
model.language_model
526.8M
18.0%
16.5M
98.8%
model.vision_tower.encoder.layers
411.5M
14.1%
66.9K
0.4%
model.multi_modal_projector
2.4M
0.1%
4.2K
0.0%
model.vision_tower
975.7K
0.0%
54.2K
0.3%
Preserved properties
Config
Property
Original
Fixture
model
model_type
paligemma
paligemma
✓ kept
model
vocab_size
257216
257216
✓ kept
model
tie_word_embeddings
True
True
✓ kept
vision_config
model_type
siglip_vision_model
siglip_vision_model
✓ kept
vision_config
hidden_act
gelu_pytorch_tanh
gelu_pytorch_tanh
✓ kept
vision_config
image_size
224
224
✓ kept
vision_config
patch_size
14
14
✓ kept
vision_config
num_channels
3
3
✓ kept
text_config
attention style
MQA (8 heads, 1 KV)
MQA (4 heads, 1 KV)
✓ kept
text_config
RoPE type
default
default
✓ kept
text_config
model_type
gemma
gemma
✓ kept
text_config
vocab_size
257216
257216
✓ kept
text_config
tie_word_embeddings
True
True
✓ kept
text_config
hidden_act
gelu_pytorch_tanh
gelu_pytorch_tanh
✓ kept
text_config
max_position_embeddings
8192
8192
✓ kept
Module classes
Instances of each class in the built model (prefix PaliGemma omitted).
Class
Original
Fixture
Linear
290
28
LayerNorm
55
5
GELUTanh
45
4
GemmaRMSNorm
37
5
SiglipAttention
27
2
SiglipEncoderLayer
27
2
SiglipMLP
27
2
GemmaAttention
18
2
GemmaDecoderLayer
18
2
GemmaMLP
18
2
ModuleList
2
2
Conv2d
1
1
Embedding
1
1
GemmaModel
1
1
GemmaRotaryEmbedding
1
1
GemmaTextScaledWordEmbedding
1
1
ForConditionalGeneration
1
1
Model
1
1
MultiModalProjector
1
1
SiglipEncoder
1
1
SiglipVisionEmbeddings
1
1
SiglipVisionModel
1
1
Config changes
12 fields changed, 59 unchanged.
Field
Original
Fixture
hidden_size
2048
64
projection_dim
2048
64
text_config.head_dim
256
16
text_config.hidden_size
2048
64
text_config.intermediate_size
16384
128
text_config.num_attention_heads
8
4
text_config.num_hidden_layers
18
2
vision_config.hidden_size
1152
64
vision_config.intermediate_size
4304
128
vision_config.num_attention_heads
16
4
vision_config.num_hidden_layers
27
2
vision_config.projection_dim
2048
64
+
\ No newline at end of file
diff --git a/shrink/pegasus-xsum-mic.html b/shrink/pegasus-xsum-mic.html
new file mode 100644
index 0000000000000000000000000000000000000000..46377c2db9c7c75d6700be162b9ecfa834f46eca
--- /dev/null
+++ b/shrink/pegasus-xsum-mic.html
@@ -0,0 +1,34 @@
+
+
+Shrunk architecture
+
+
Shrunk architecture
huggingface-mic/pegasus-xsum-mic@3703172718d029fd3c19c93391ed0e84937d6eb6 ← google/pegasus-xsum@8d8ffc158a3bee9fbb03afacdfc347c823c5ec8b · PegasusForConditionalGeneration · random weights
+
Original569.7Mparameters
Fixture6.4Mparameters
Layer stacks2compared
Findings0to review
✓ Same architecture, smaller
Every module class, layer kind and checked property of the original is in the fixture.
Layer stacks
One square per layer, colored by kind: the layer's module classes and type attributes. Hover a square for its index and kind.
model.encoder.layers
Original · 16
Fixture · 2
Kind
Original
Fixture
EncoderLayer
16
2
✓ kept
model.decoder.layers
Original · 16
Fixture · 2
Kind
Original
Fixture
DecoderLayer
16
2
✓ kept
Where the parameters are
Share of each model's parameters per component. Vocabulary and patch sizes are kept, so embeddings usually take a larger share of the fixture.
Original
Fixture
Component
Original
Share
Fixture
Share
model.decoder.layers
268.7M
47.2%
100.5K
1.6%
model.encoder.layers
201.5M
35.4%
66.9K
1.0%
model.shared
98.4M
17.3%
6.2M
96.3%
model.decoder
526.3K
0.1%
32.9K
0.5%
model.encoder
526.3K
0.1%
32.9K
0.5%
Preserved properties
Config
Property
Original
Fixture
model
model_type
pegasus
pegasus
✓ kept
model
vocab_size
96103
96103
✓ kept
model
tie_word_embeddings
True
True
✓ kept
model
max_position_embeddings
512
512
✓ kept
Module classes
Instances of each class in the built model (prefix Pegasus omitted).
Class
Original
Fixture
Linear
257
33
LayerNorm
82
12
Attention
48
6
ReLU
32
4
DecoderLayer
16
2
EncoderLayer
16
2
Embedding
3
3
ModuleList
2
2
SinusoidalPositionalEmbedding
2
2
Decoder
1
1
Encoder
1
1
ForConditionalGeneration
1
1
Model
1
1
Config changes
7 fields changed, 38 unchanged.
Field
Original
Fixture
d_model
1024
64
decoder_attention_heads
16
4
decoder_ffn_dim
4096
128
decoder_layers
16
2
encoder_attention_heads
16
4
encoder_ffn_dim
4096
128
encoder_layers
16
2
+
\ No newline at end of file
diff --git a/shrink/phi-2-mic.html b/shrink/phi-2-mic.html
new file mode 100644
index 0000000000000000000000000000000000000000..c00a90aefe5e568348f0074aa69f899f1a887a27
--- /dev/null
+++ b/shrink/phi-2-mic.html
@@ -0,0 +1,34 @@
+
+
+Shrunk architecture
+
+
Shrunk architecture
huggingface-mic/phi-2-mic@af26f3c429a4d873dfe9c7fdc51fc78a845768fc ← microsoft/phi-2@810d367871c1d460086d9f82db8696f2e0a0fcd0 · PhiForCausalLM · random weights
+
Original2.8Bparameters
Fixture6.7Mparameters
Layer stacks1compared
Findings0to review
✓ Same architecture, smaller
Every module class, layer kind and checked property of the original is in the fixture.
Layer stacks
One square per layer, colored by kind: the layer's module classes and type attributes. Hover a square for its index and kind.
model.layers
Original · 32
Fixture · 2
Kind
Original
Fixture
DecoderLayer
32
2
✓ kept
Where the parameters are
Share of each model's parameters per component. Vocabulary and patch sizes are kept, so embeddings usually take a larger share of the fixture.
Original
Fixture
Component
Original
Share
Fixture
Share
model.layers
2.5B
90.6%
66.7K
1.0%
lm_head
131.1M
4.7%
3.3M
49.9%
model.embed_tokens
131.1M
4.7%
3.3M
49.1%
model.final_layernorm
5.1K
0.0%
128
0.0%
Preserved properties
Config
Property
Original
Fixture
model
attention style
MHA (32 heads, 32 KV)
MHA (4 heads, 4 KV)
✓ kept
model
RoPE type
default
default
✓ kept
model
model_type
phi
phi
✓ kept
model
vocab_size
51200
51200
✓ kept
model
tie_word_embeddings
False
False
✓ kept
model
hidden_act
gelu_new
gelu_new
✓ kept
model
max_position_embeddings
2048
2048
✓ kept
Module classes
Instances of each class in the built model (prefix Phi omitted).
Class
Original
Fixture
Linear
193
13
Dropout
33
3
LayerNorm
33
3
NewGELUActivation
32
2
Attention
32
2
DecoderLayer
32
2
MLP
32
2
Embedding
1
1
ModuleList
1
1
ForCausalLM
1
1
Model
1
1
RotaryEmbedding
1
1
Config changes
5 fields changed, 24 unchanged.
Field
Original
Fixture
hidden_size
2560
64
intermediate_size
10240
128
num_attention_heads
32
4
num_hidden_layers
32
2
num_key_value_heads
32
4
+
\ No newline at end of file
diff --git a/shrink/privacy-filter-mic.html b/shrink/privacy-filter-mic.html
new file mode 100644
index 0000000000000000000000000000000000000000..5ab27bdc56d6fccfaa924f67ed63b93b82364e28
--- /dev/null
+++ b/shrink/privacy-filter-mic.html
@@ -0,0 +1,34 @@
+
+
+Shrunk architecture
+
+
Shrunk architecture
huggingface-mic/privacy-filter-mic@ad3dba8d001d1fe12c033288bf38745f9772c1b7 ← openai/privacy-filter@7ffa9a043d54d1be65afb281eddf0ffbe629385b · OpenAIPrivacyFilterForTokenClassification · random weights
+
Original1.4Bparameters
Fixture13.0Mparameters
Layer stacks1compared
Findings0to review
✓ Same architecture, smaller
Every module class, layer kind and checked property of the original is in the fixture.
Layer stacks
One square per layer, colored by kind: the layer's module classes and type attributes. Hover a square for its index and kind.
model.layers
Original · 8
Fixture · 2
Kind
Original
Fixture
EncoderLayer
8
2
✓ kept
Where the parameters are
Share of each model's parameters per component. Vocabulary and patch sizes are kept, so embeddings usually take a larger share of the fixture.
Original
Fixture
Component
Original
Share
Fixture
Share
model.layers
1.3B
90.8%
224.9K
1.7%
model.embed_tokens
128.0M
9.1%
12.8M
98.3%
score
21.2K
0.0%
2.1K
0.0%
model.norm
640
0.0%
64
0.0%
Preserved properties
Config
Property
Original
Fixture
model
attention style
GQA (14 heads, 2 KV)
GQA (4 heads, 2 KV)
✓ kept
model
MoE routing
top-4 of 128 experts
top-2 of 4 experts
✓ kept
model
RoPE type
yarn
yarn
✓ kept
model
model_type
openai_privacy_filter
openai_privacy_filter
✓ kept
model
vocab_size
200064
200064
✓ kept
model
tie_word_embeddings
False
False
✓ kept
model
hidden_act
silu
silu
✓ kept
model
max_position_embeddings
131072
131072
✓ kept
Module classes
Instances of each class in the built model (prefix OpenAIPrivacyFilter omitted).
Class
Original
Fixture
Linear
33
9
RMSNorm
17
5
Attention
8
2
EncoderLayer
8
2
Experts
8
2
MLP
8
2
TopKRouter
8
2
Dropout
1
1
Embedding
1
1
ModuleList
1
1
ForTokenClassification
1
1
Model
1
1
RotaryEmbedding
1
1
Config changes
7 fields changed, 32 unchanged.
Field
Original
Fixture
head_dim
64
16
hidden_size
640
64
intermediate_size
640
128
num_attention_heads
14
4
num_experts_per_tok
4
2
num_hidden_layers
8
2
num_local_experts
128
4
+
\ No newline at end of file
diff --git a/shrink/pvt-tiny-224-mic.html b/shrink/pvt-tiny-224-mic.html
new file mode 100644
index 0000000000000000000000000000000000000000..4658fa68c7a9c124c741f71d5988c7bd778239d0
--- /dev/null
+++ b/shrink/pvt-tiny-224-mic.html
@@ -0,0 +1,34 @@
+
+
+Shrunk architecture
+
+
Shrunk architecture
huggingface-mic/pvt-tiny-224-mic@b1c239c03ea2b52e24e2e72cea4498c1932cc76b ← Zetatech/pvt-tiny-224@6a44ba96dda6a6f3e2b6913682acfa0071482e83 · PvtForImageClassification · overfit weights · built before the shrink rule (no recorded recipe)
+
Original13.2Mparameters
Fixture592.2Kparameters
Layer stacks2compared
Findings0to review
✓ Same architecture, smaller
Every module class, layer kind and checked property of the original is in the fixture.
Layer stacks
One square per layer, colored by kind: the layer's module classes and type attributes. Hover a square for its index and kind.
pvt.encoder.patch_embeddings
Original · 4
Fixture · 4
Kind
Original
Fixture
Conv2d
4
4
✓ kept
pvt.encoder.block
Original · 4
Fixture · 4
Kind
Original
Fixture
Conv2d
3
3
✓ kept
no Conv2d
1
1
✓ kept
Where the parameters are
Share of each model's parameters per component. Vocabulary and patch sizes are kept, so embeddings usually take a larger share of the fixture.
Original
Fixture
Component
Original
Share
Fixture
Share
pvt.encoder.block
11.5M
86.7%
324.2K
54.7%
pvt.encoder.patch_embeddings
1.2M
9.4%
138.8K
23.4%
classifier
513.0K
3.9%
129.0K
21.8%
pvt.encoder
1.0K
0.0%
256
0.0%
Preserved properties
Config
Property
Original
Fixture
model
model_type
pvt
pvt
✓ kept
model
hidden_act
gelu
gelu
✓ kept
model
image_size
224
224
✓ kept
model
num_channels
3
3
✓ kept
Module classes
Instances of each class in the built model (prefix Pvt omitted).
Class
Original
Fixture
Linear
49
25
Dropout
28
16
LayerNorm
27
16
Conv2d
10
7
GELUActivation
8
4
Identity
8
4
Attention
8
4
EfficientSelfAttention
8
4
FFN
8
4
Layer
8
4
SelfOutput
8
4
ModuleList
6
6
PatchEmbeddings
4
4
Encoder
1
1
ForImageClassification
1
1
Model
1
1
Config changes
4 fields changed, 24 unchanged.
Field
Original
Fixture
depths
[2, 2, 2, 2]
[1, 1, 1, 1]
hidden_sizes
[64, 128, 320, 512]
[16, 32, 64, 128]
num_attention_heads
[1, 2, 5, 8]
[1, 2, 4, 8]
problem_type
null
single_label_classification
+
\ No newline at end of file
diff --git a/shrink/pythia-160m-mic.html b/shrink/pythia-160m-mic.html
new file mode 100644
index 0000000000000000000000000000000000000000..c4718eccf59726190ce034dcf1653bd3042e231d
--- /dev/null
+++ b/shrink/pythia-160m-mic.html
@@ -0,0 +1,34 @@
+
+
+Shrunk architecture
+
+
Shrunk architecture
huggingface-mic/pythia-160m-mic@f21c9ac8c4e817ebcf9cb2963749d2ff55b756e3 ← EleutherAI/pythia-160m@50f5173d932e8e61f858120bcb800b97af589f46 · GPTNeoXForCausalLM · random weights · built before the shrink rule (no recorded recipe)
+
Original162.3Mparameters
Fixture6.5Mparameters
Layer stacks1compared
Findings0to review
✓ Same architecture, smaller
Every module class, layer kind and checked property of the original is in the fixture.
Layer stacks
One square per layer, colored by kind: the layer's module classes and type attributes. Hover a square for its index and kind.
gpt_neox.layers
Original · 12
Fixture · 2
Kind
Original
Fixture
Layer
12
2
✓ kept
Where the parameters are
Share of each model's parameters per component. Vocabulary and patch sizes are kept, so embeddings usually take a larger share of the fixture.
Original
Fixture
Component
Original
Share
Fixture
Share
gpt_neox.layers
85.1M
52.4%
66.9K
1.0%
lm_head
38.6M
23.8%
3.2M
49.5%
gpt_neox.embed_in
38.6M
23.8%
3.2M
49.5%
gpt_neox.final_layer_norm
1.5K
0.0%
128
0.0%
Preserved properties
Config
Property
Original
Fixture
model
RoPE type
default
default
✓ kept
model
model_type
gpt_neox
gpt_neox
✓ kept
model
vocab_size
50304
50304
✓ kept
model
tie_word_embeddings
False
False
✓ kept
model
hidden_act
gelu
gelu
✓ kept
model
max_position_embeddings
2048
2048
✓ kept
Module classes
Instances of each class in the built model (prefix GPTNeoX omitted).
Class
Original
Fixture
Linear
49
9
Dropout
25
5
LayerNorm
25
5
GELUActivation
12
2
Attention
12
2
Layer
12
2
MLP
12
2
Embedding
1
1
ForCausalLM
1
1
Model
1
1
RotaryEmbedding
1
1
ModuleList
1
1
Config changes
4 fields changed, 26 unchanged.
Field
Original
Fixture
hidden_size
768
64
intermediate_size
3072
128
num_attention_heads
12
4
num_hidden_layers
12
2
+
\ No newline at end of file
diff --git a/shrink/qwen2-mic.html b/shrink/qwen2-mic.html
new file mode 100644
index 0000000000000000000000000000000000000000..03b1c2da376658cd8f585a276d00cb7f007d414b
--- /dev/null
+++ b/shrink/qwen2-mic.html
@@ -0,0 +1,34 @@
+
+
+Shrunk architecture
+
+
Shrunk architecture
huggingface-mic/qwen2-mic@f9c74f62c62500474b51b9ae8b086be6bcff3621 ← Qwen/Qwen2-0.5B@91d2aff3f957f99e4c74c962f2f408dcc88a18d8 · Qwen2ForCausalLM · random weights · built before the shrink rule (no recorded recipe)
+
Original494.0Mparameters
Fixture9.8Mparameters
Layer stacks1compared
Findings0to review
✓ Same architecture, smaller
Every module class, layer kind and checked property of the original is in the fixture.
Layer stacks
One square per layer, colored by kind: the layer's module classes and type attributes. Hover a square for its index and kind.
model.layers
Original · 24
Fixture · 2
Kind
Original
Fixture
DecoderLayer
24
2
✓ kept
Where the parameters are
Share of each model's parameters per component. Vocabulary and patch sizes are kept, so embeddings usually take a larger share of the fixture.
Original
Fixture
Component
Original
Share
Fixture
Share
model.layers
357.9M
72.4%
74.2K
0.8%
model.embed_tokens
136.1M
27.6%
9.7M
99.2%
model.norm
896
0.0%
64
0.0%
Preserved properties
Config
Property
Original
Fixture
model
attention style
GQA (14 heads, 2 KV)
GQA (4 heads, 2 KV)
✓ kept
model
RoPE type
default
default
✓ kept
model
model_type
qwen2
qwen2
✓ kept
model
vocab_size
151936
151936
✓ kept
model
tie_word_embeddings
True
True
✓ kept
model
hidden_act
silu
silu
✓ kept
model
max_position_embeddings
131072
131072
✓ kept
Module classes
Instances of each class in the built model (prefix Qwen2 omitted).
Class
Original
Fixture
Linear
169
15
RMSNorm
49
5
Attention
24
2
DecoderLayer
24
2
MLP
24
2
SiLUActivation
24
2
Embedding
1
1
ModuleList
1
1
ForCausalLM
1
1
Model
1
1
RotaryEmbedding
1
1
Config changes
5 fields changed, 25 unchanged.
Field
Original
Fixture
hidden_size
896
64
intermediate_size
4864
128
layer_types
24 entries: 24× full_attention
["full_attention", "full_attention"]
num_attention_heads
14
4
num_hidden_layers
24
2
+
\ No newline at end of file
diff --git a/shrink/qwen2_5_omni-mic.html b/shrink/qwen2_5_omni-mic.html
new file mode 100644
index 0000000000000000000000000000000000000000..4e6dba6e01577a461d1f9180cc8591c8d565d13c
--- /dev/null
+++ b/shrink/qwen2_5_omni-mic.html
@@ -0,0 +1,34 @@
+
+
+Shrunk architecture
+
+
Shrunk architecture
huggingface-mic/qwen2_5_omni-mic@72e87592273c7e7662ffb5e249400c12a3488549 ← Qwen/Qwen2.5-Omni-7B@ae9e1690543ffd5c0221dc27f79834d0294cba00 · Qwen2_5OmniForConditionalGeneration · random weights · built before the shrink rule (no recorded recipe)
+
Original10.7Bparameters
Fixture170.4Mparameters
Layer stacks8compared
Findings0to review
✓ Same architecture, smaller
Every module class, layer kind and checked property of the original is in the fixture.
Layer stacks
One square per layer, colored by kind: the layer's module classes and type attributes. Hover a square for its index and kind.
+
\ No newline at end of file
diff --git a/shrink/qwen2_5_vl-mic.html b/shrink/qwen2_5_vl-mic.html
new file mode 100644
index 0000000000000000000000000000000000000000..e3baac95a36415522c4d28f45bffc5cba3ed9cf5
--- /dev/null
+++ b/shrink/qwen2_5_vl-mic.html
@@ -0,0 +1,34 @@
+
+
+Shrunk architecture
+
+
Shrunk architecture
huggingface-mic/qwen2_5_vl-mic@32d6627e4ea6fc3ba2fe84c2f3f06e7f5116245a ← Qwen/Qwen2.5-VL-3B-Instruct@66285546d2b821cf421d4f5eb2576359d3770cd3 · Qwen2_5_VLForConditionalGeneration · random weights · built before the shrink rule (no recorded recipe)
+
Original3.8Bparameters
Fixture11.3Mparameters
Layer stacks2compared
Findings0to review
✓ Same architecture, smaller
Every module class, layer kind and checked property of the original is in the fixture.
Layer stacks
One square per layer, colored by kind: the layer's module classes and type attributes. Hover a square for its index and kind.
model.visual.blocks
Original · 32
Fixture · 32
Kind
Original
Fixture
VisionBlock
32
32
✓ kept
model.language_model.layers
Original · 36
Fixture · 2
Kind
Original
Fixture
DecoderLayer
36
2
✓ kept
Where the parameters are
Share of each model's parameters per component. Vocabulary and patch sizes are kept, so embeddings usually take a larger share of the fixture.
Original
Fixture
Component
Original
Share
Fixture
Share
model.language_model.layers
2.8B
73.9%
74.2K
0.7%
model.visual.blocks
630.5M
16.8%
1.3M
11.8%
model.language_model
311.2M
8.3%
9.7M
86.1%
model.visual
38.2M
1.0%
157.6K
1.4%
Preserved properties
Config
Property
Original
Fixture
model
model_type
qwen2_5_vl
qwen2_5_vl
✓ kept
model
tie_word_embeddings
True
True
✓ kept
text_config
attention style
GQA (16 heads, 2 KV)
GQA (4 heads, 2 KV)
✓ kept
text_config
RoPE type
default
default
✓ kept
text_config
model_type
qwen2_5_vl_text
qwen2_5_vl_text
✓ kept
text_config
vocab_size
151936
151936
✓ kept
text_config
hidden_act
silu
silu
✓ kept
text_config
max_position_embeddings
128000
128000
✓ kept
vision_config
RoPE type
axial
axial
✓ kept
vision_config
model_type
qwen2_5_vl_vision
qwen2_5_vl_vision
✓ kept
vision_config
hidden_act
silu
silu
✓ kept
vision_config
patch_size
14
14
✓ kept
Module classes
Instances of each class in the built model (prefix Qwen2_5_VL omitted).
+
\ No newline at end of file
diff --git a/shrink/qwen2_audio-mic.html b/shrink/qwen2_audio-mic.html
new file mode 100644
index 0000000000000000000000000000000000000000..06bcd7037399c1dc29dc9f1cb49bec174bd81555
--- /dev/null
+++ b/shrink/qwen2_audio-mic.html
@@ -0,0 +1,34 @@
+
+
+Shrunk architecture
+
+
Shrunk architecture
huggingface-mic/qwen2_audio-mic@c426549756a18dc16495239883efd38d9a66002b ← Qwen/Qwen2-Audio-7B@dd84470756e6277a71d4d7188773a43cde92696e · Qwen2AudioForConditionalGeneration · random weights · built before the shrink rule (no recorded recipe)
+
Original8.4Bparameters
Fixture20.3Mparameters
Layer stacks2compared
Findings0to review
✓ Same architecture, smaller
Every module class, layer kind and checked property of the original is in the fixture.
Layer stacks
One square per layer, colored by kind: the layer's module classes and type attributes. Hover a square for its index and kind.
model.audio_tower.layers
Original · 32
Fixture · 2
Kind
Original
Fixture
EncoderLayer
32
2
✓ kept
model.language_model.layers
Original · 32
Fixture · 2
Kind
Original
Fixture
Qwen2DecoderLayer
32
2
✓ kept
Where the parameters are
Share of each model's parameters per component. Vocabulary and patch sizes are kept, so embeddings usually take a larger share of the fixture.
Original
Fixture
Component
Original
Share
Fixture
Share
model.language_model.layers
6.5B
77.1%
82.6K
0.4%
model.language_model
639.1M
7.6%
10.0M
49.3%
lm_head
639.1M
7.6%
10.0M
49.3%
model.audio_tower.layers
629.6M
7.5%
66.8K
0.3%
model.audio_tower
7.3M
0.1%
133.1K
0.7%
model.multi_modal_projector
5.2M
0.1%
4.2K
0.0%
Preserved properties
Config
Property
Original
Fixture
model
model_type
qwen2_audio
qwen2_audio
✓ kept
model
vocab_size
156032
156032
✓ kept
audio_config
model_type
qwen2_audio_encoder
qwen2_audio_encoder
✓ kept
audio_config
num_mel_bins
128
128
✓ kept
text_config
attention style
MHA (32 heads, 32 KV)
MHA (4 heads, 4 KV)
✓ kept
text_config
RoPE type
default
default
✓ kept
text_config
model_type
qwen2
qwen2
✓ kept
text_config
vocab_size
156032
156032
✓ kept
text_config
tie_word_embeddings
False
False
✓ kept
text_config
hidden_act
silu
silu
✓ kept
text_config
max_position_embeddings
8192
8192
✓ kept
Module classes
Instances of each class in the built model (prefix Qwen2Audio omitted).
Class
Original
Fixture
Linear
418
28
LayerNorm
65
5
Qwen2RMSNorm
65
5
GELUActivation
32
2
Qwen2Attention
32
2
Attention
32
2
EncoderLayer
32
2
Qwen2DecoderLayer
32
2
Qwen2MLP
32
2
SiLUActivation
32
2
Conv1d
2
2
Embedding
2
2
ModuleList
2
2
AvgPool1d
1
1
Encoder
1
1
ForConditionalGeneration
1
1
Model
1
1
MultiModalProjector
1
1
Qwen2Model
1
1
Qwen2RotaryEmbedding
1
1
Config changes
10 fields changed, 55 unchanged.
Field
Original
Fixture
audio_config.d_model
1280
64
audio_config.encoder_attention_heads
20
4
audio_config.encoder_ffn_dim
5120
128
audio_config.encoder_layers
32
2
text_config.hidden_size
4096
64
text_config.intermediate_size
11008
128
text_config.layer_types
32 entries: 32× full_attention
["full_attention", "full_attention"]
text_config.num_attention_heads
32
4
text_config.num_hidden_layers
32
2
text_config.num_key_value_heads
32
4
+
\ No newline at end of file
diff --git a/shrink/recurrentgemma-2b-mic.html b/shrink/recurrentgemma-2b-mic.html
new file mode 100644
index 0000000000000000000000000000000000000000..d7c40394b1071c02732fef4afea8930920e1b22b
--- /dev/null
+++ b/shrink/recurrentgemma-2b-mic.html
@@ -0,0 +1,34 @@
+
+
+Shrunk architecture
+
+
Shrunk architecture
huggingface-mic/recurrentgemma-2b-mic@732c0b9399dcbbdb7c806f7043968c4aafea55c8 ← google/recurrentgemma-2b@3620f4ca9c5d16ee56c00180474a3201ec7f734a · RecurrentGemmaForCausalLM · random weights
+
Original2.7Bparameters
Fixture24.0Mparameters
Layer stacks1compared
Findings0to review
✓ Same architecture, smaller
Every module class, layer kind and checked property of the original is in the fixture.
Layer stacks
One square per layer, colored by kind: the layer's module classes and type attributes. Hover a square for its index and kind.
model.layers
Original · 26
Fixture · 3
Kind
Original
Fixture
Conv1d, RecurrentBlock, Rglru
18
2
✓ kept
is_causal=True, Attention, RotaryEmbedding
8
1
✓ kept
Where the parameters are
Share of each model's parameters per component. Vocabulary and patch sizes are kept, so embeddings usually take a larger share of the fixture.
Original
Fixture
Component
Original
Share
Fixture
Share
model.layers
2.0B
75.6%
7.6M
31.8%
model.embed_tokens
655.4M
24.4%
16.4M
68.2%
model.final_norm
2.6K
0.0%
64
0.0%
Preserved properties
Config
Property
Original
Fixture
model
attention style
MQA (10 heads, 1 KV)
MQA (4 heads, 1 KV)
✓ kept
model
RoPE type
default
default
✓ kept
model
model_type
recurrent_gemma
recurrent_gemma
✓ kept
model
vocab_size
256000
256000
✓ kept
model
tie_word_embeddings
True
True
✓ kept
Module classes
Instances of each class in the built model (prefix RecurrentGemma omitted).
Class
Original
Fixture
Linear
165
20
RMSNorm
53
7
GELUTanh
44
5
DecoderLayer
26
3
Mlp
26
3
Conv1d
18
2
RecurrentBlock
18
2
Rglru
18
2
Attention
8
1
RotaryEmbedding
8
1
Embedding
1
1
ModuleList
1
1
ForCausalLM
1
1
Model
1
1
Config changes
5 fields changed, 30 unchanged.
Field
Original
Fixture
head_dim
256
16
hidden_size
2560
64
intermediate_size
15360
128
num_attention_heads
10
4
num_hidden_layers
26
3
+
\ No newline at end of file
diff --git a/shrink/resnet-18-mic.html b/shrink/resnet-18-mic.html
new file mode 100644
index 0000000000000000000000000000000000000000..3a01319e205706b5825c3a4029cdfaa861ee70a6
--- /dev/null
+++ b/shrink/resnet-18-mic.html
@@ -0,0 +1,34 @@
+
+
+Shrunk architecture
+
+
Every module class, layer kind and checked property of the original is in the fixture.
Layer stacks
One square per layer, colored by kind: the layer's module classes and type attributes. Hover a square for its index and kind.
resnet.encoder.stages
Original · 4
Fixture · 4
Kind
Original
Fixture
no ShortCut
1
1
✓ kept
ShortCut
3
3
✓ kept
Where the parameters are
Share of each model's parameters per component. Vocabulary and patch sizes are kept, so embeddings usually take a larger share of the fixture.
Original
Fixture
Component
Original
Share
Fixture
Share
resnet.encoder.stages
11.2M
95.5%
307.1K
70.0%
classifier.1
513.0K
4.4%
129.0K
29.4%
resnet.embedder
9.5K
0.1%
2.4K
0.5%
Preserved properties
Config
Property
Original
Fixture
model
model_type
resnet
resnet
✓ kept
model
hidden_act
relu
relu
✓ kept
model
num_channels
3
3
✓ kept
Module classes
Instances of each class in the built model (prefix ResNet omitted).
Class
Original
Fixture
BatchNorm2d
20
12
Conv2d
20
12
ReLU
17
9
ConvLayer
17
9
Identity
13
5
Sequential
13
9
BasicLayer
8
4
Stage
4
4
ShortCut
3
3
AdaptiveAvgPool2d
1
1
Flatten
1
1
Linear
1
1
MaxPool2d
1
1
ModuleList
1
1
Embeddings
1
1
Encoder
1
1
ForImageClassification
1
1
Model
1
1
Config changes
4 fields changed, 16 unchanged.
Field
Original
Fixture
depths
[2, 2, 2, 2]
[1, 1, 1, 1]
embedding_size
64
16
hidden_sizes
[64, 128, 256, 512]
[16, 32, 64, 128]
problem_type
null
single_label_classification
+
\ No newline at end of file
diff --git a/shrink/roberta-base-mic.html b/shrink/roberta-base-mic.html
new file mode 100644
index 0000000000000000000000000000000000000000..69b5c0ef8149550575e661f6a666dfcfd8195c91
--- /dev/null
+++ b/shrink/roberta-base-mic.html
@@ -0,0 +1,34 @@
+
+
+Shrunk architecture
+
+
Shrunk architecture
huggingface-mic/roberta-base-mic@ff3c8d1292ae8885553023bad9a04744ced2effe ← FacebookAI/roberta-base@e2da8e2f811d1448a5b465c236feacd80ffbac7b · RobertaForMaskedLM · random weights · built before the shrink rule (no recorded recipe)
+
Original124.7Mparameters
Fixture3.4Mparameters
Layer stacks1compared
Findings0to review
✓ Same architecture, smaller
Every module class, layer kind and checked property of the original is in the fixture.
Layer stacks
One square per layer, colored by kind: the layer's module classes and type attributes. Hover a square for its index and kind.
roberta.encoder.layer
Original · 12
Fixture · 2
Kind
Original
Fixture
Layer
12
2
✓ kept
Where the parameters are
Share of each model's parameters per component. Vocabulary and patch sizes are kept, so embeddings usually take a larger share of the fixture.
Original
Fixture
Component
Original
Share
Fixture
Share
roberta.encoder.layer
85.1M
68.2%
66.9K
2.0%
roberta.embeddings
39.0M
31.3%
3.3M
96.4%
lm_head.dense
590.6K
0.5%
4.2K
0.1%
lm_head
50.3K
0.0%
50.3K
1.5%
lm_head.layer_norm
1.5K
0.0%
128
0.0%
Preserved properties
Config
Property
Original
Fixture
model
model_type
roberta
roberta
✓ kept
model
vocab_size
50265
50265
✓ kept
model
tie_word_embeddings
True
True
✓ kept
model
hidden_act
gelu
gelu
✓ kept
model
max_position_embeddings
514
514
✓ kept
Module classes
Instances of each class in the built model (prefix Roberta omitted).
Class
Original
Fixture
Linear
74
14
Dropout
37
7
LayerNorm
26
6
GELUActivation
12
2
Attention
12
2
Intermediate
12
2
Layer
12
2
Output
12
2
SelfAttention
12
2
SelfOutput
12
2
Embedding
3
3
ModuleList
1
1
Embeddings
1
1
Encoder
1
1
ForMaskedLM
1
1
LMHead
1
1
Model
1
1
Config changes
4 fields changed, 25 unchanged.
Field
Original
Fixture
hidden_size
768
64
intermediate_size
3072
128
num_attention_heads
12
4
num_hidden_layers
12
2
+
\ No newline at end of file
diff --git a/shrink/rtdetr_r50vd-mic.html b/shrink/rtdetr_r50vd-mic.html
new file mode 100644
index 0000000000000000000000000000000000000000..ccf3a0b8f1ccc5a107158d265d44d03de3b8fa5c
--- /dev/null
+++ b/shrink/rtdetr_r50vd-mic.html
@@ -0,0 +1,34 @@
+
+
+Shrunk architecture
+
+
Shrunk architecture
huggingface-mic/rtdetr_r50vd-mic@5655fbe502f7ba0ce74c6037780272095348ca82 ← PekingU/rtdetr_r50vd@df939e661d8c52e80608d1ec566561aabd25a4e7 · RTDetrForObjectDetection · random weights · built before the shrink rule (no recorded recipe)
+
Original42.9Mparameters
Fixture10.9Mparameters
Layer stacks10compared
Findings0to review
✓ Same architecture, smaller
Every module class, layer kind and checked property of the original is in the fixture.
Layer stacks
One square per layer, colored by kind: the layer's module classes and type attributes. Hover a square for its index and kind.
model.backbone.model.encoder.stages
Original · 4
Fixture · 4
Kind
Original
Fixture
no AvgPool2d
1
1
✓ kept
AvgPool2d
3
3
✓ kept
model.encoder_input_proj
Original · 3
Fixture · 3
Kind
Original
Fixture
Conv2d
3
3
✓ kept
model.encoder.aifi
Original · 1
Fixture · 1
Kind
Original
Fixture
AIFILayer
1
1
✓ kept
model.encoder.lateral_convs
Original · 2
Fixture · 2
Kind
Original
Fixture
ConvNormLayer
2
2
✓ kept
model.encoder.fpn_blocks
Original · 2
Fixture · 2
Kind
Original
Fixture
CSPRepLayer
2
2
✓ kept
model.encoder.downsample_convs
Original · 2
Fixture · 2
Kind
Original
Fixture
ConvNormLayer
2
2
✓ kept
model.encoder.pan_blocks
Original · 2
Fixture · 2
Kind
Original
Fixture
CSPRepLayer
2
2
✓ kept
model.decoder_input_proj
Original · 3
Fixture · 3
Kind
Original
Fixture
Conv2d
3
3
✓ kept
model.decoder.layers
Original · 6
Fixture · 2
Kind
Original
Fixture
DecoderLayer
6
2
✓ kept
model.decoder.bbox_embed
Original · 6
Fixture · 2
Kind
Original
Fixture
Linear
6
2
✓ kept
Where the parameters are
Share of each model's parameters per component. Vocabulary and patch sizes are kept, so embeddings usually take a larger share of the fixture.
Original
Fixture
Component
Original
Share
Fixture
Share
model.backbone.model.encoder.stages
23.4M
54.7%
32.4K
0.3%
model.decoder.layers
6.0M
13.9%
102.6K
0.9%
model.encoder.pan_blocks
4.5M
10.4%
4.5M
40.9%
model.encoder.fpn_blocks
4.5M
10.4%
4.5M
40.9%
model.encoder.downsample_convs
1.2M
2.8%
1.2M
10.8%
model.encoder_input_proj
919.0K
2.1%
58.9K
0.5%
model.decoder.bbox_embed
795.7K
1.9%
17.2K
0.2%
9 others
1.6M
3.8%
582.8K
5.3%
Preserved properties
Config
Property
Original
Fixture
model
model_type
rt_detr
rt_detr
✓ kept
backbone_config
model_type
rt_detr_resnet
rt_detr_resnet
✓ kept
backbone_config
hidden_act
relu
relu
✓ kept
backbone_config
num_channels
3
3
✓ kept
Module classes
Instances of each class in the built model (prefix RTDetr omitted).
Class
Original
Fixture
Conv2d
97
61
Linear
97
41
ReLU
57
17
Identity
56
32
FrozenBatchNorm2d
55
19
ResNetConvLayer
51
15
BatchNorm2d
42
42
ConvNormLayer
36
36
Sequential
35
23
SiLUActivation
24
24
LayerNorm
21
9
ModuleList
20
16
ResNetBottleNeckLayer
16
4
RepVggBlock
12
12
MLPPredictionHead
8
4
MLP
7
3
SelfAttention
7
3
MultiScaleDeformableAttention
6
2
DecoderLayer
6
2
MultiscaleDeformableAttention
6
2
CSPRepLayer
4
4
ResNetShortCut
4
4
ResNetStage
4
4
AvgPool2d
3
3
Embedding
1
1
GELUActivation
1
1
MaxPool2d
1
1
AIFILayer
1
1
ConvEncoder
1
1
Decoder
1
1
EncoderLayer
1
1
ForObjectDetection
1
1
HybridEncoder
1
1
Model
1
1
ResNetBackbone
1
1
ResNetEmbeddings
1
1
ResNetEncoder
1
1
SinePositionEmbedding
1
1
Config changes
9 fields changed, 71 unchanged.
Field
Original
Fixture
backbone_config.depths
[3, 4, 6, 3]
[1, 1, 1, 1]
backbone_config.hidden_sizes
[256, 512, 1024, 2048]
[16, 32, 64, 128]
d_model
256
64
decoder_attention_heads
8
4
decoder_ffn_dim
1024
128
decoder_layers
6
2
encoder_attention_heads
8
4
encoder_ffn_dim
1024
128
encoder_in_channels
[512, 1024, 2048]
[32, 64, 128]
+
\ No newline at end of file
diff --git a/shrink/sam2.1-hiera-tiny-mic.html b/shrink/sam2.1-hiera-tiny-mic.html
new file mode 100644
index 0000000000000000000000000000000000000000..ed0826efb64b01c39f110a0bc470ea16ebebac68
--- /dev/null
+++ b/shrink/sam2.1-hiera-tiny-mic.html
@@ -0,0 +1,34 @@
+
+
+Shrunk architecture
+
+
Shrunk architecture
huggingface-mic/sam2.1-hiera-tiny-mic@e740a31c3d860e3d217f291ba1dc2a154a98090c ← facebook/sam2.1-hiera-tiny@de431c4043854a71d8101e17995dfe596bf101a5 · Sam2Model · overfit weights · built before the shrink rule (no recorded recipe)
+
Original31.4Mparameters
Fixture1.9Mparameters
Layer stacks3compared
Findings4to review
⚠ To review
model: RoPE type axial became None
model: model_type sam2_video became sam2
prompt_encoder_config: model_type sam2_video_prompt_encoder became sam2_prompt_encoder
mask_decoder_config: model_type sam2_video_mask_decoder became sam2_mask_decoder
Layer stacks
One square per layer, colored by kind: the layer's module classes and type attributes. Hover a square for its index and kind.
vision_encoder.backbone.blocks
Original · 12
Fixture · 5
Kind
Original
Fixture
MultiScaleBlock
12
5
✓ kept
mask_decoder.transformer.layers
Original · 2
Fixture · 2
Kind
Original
Fixture
TwoWayAttentionBlock
2
2
✓ kept
mask_decoder.output_hypernetworks_mlps
Original · 4
Fixture · 4
Kind
Original
Fixture
ReLU
4
4
✓ kept
Where the parameters are
Share of each model's parameters per component. Vocabulary and patch sizes are kept, so embeddings usually take a larger share of the fixture.
Original
Fixture
Component
Original
Share
Fixture
Share
vision_encoder.backbone.blocks
26.8M
85.3%
1.2M
61.2%
mask_decoder.transformer.layers
3.2M
10.0%
398.3K
21.0%
mask_decoder.output_hypernetworks_mlps
559.2K
1.8%
140.4K
7.4%
vision_encoder.neck
369.7K
1.2%
62.0K
3.3%
mask_decoder.iou_prediction_head
132.6K
0.4%
33.5K
1.8%
mask_decoder.transformer
132.2K
0.4%
33.3K
1.8%
mask_decoder.pred_obj_score_head
131.8K
0.4%
33.2K
1.7%
14 others
131.7K
0.4%
37.3K
2.0%
Preserved properties
Config
Property
Original
Fixture
model
RoPE type
axial
None
⚠ changed
model
model_type
sam2_video
sam2
⚠ changed
model
image_size
1024
1024
✓ kept
vision_config
model_type
sam2_vision_model
sam2_vision_model
✓ kept
vision_config
hidden_act
gelu
gelu
✓ kept
vision_config.backbone_config
model_type
sam2_hiera_det_model
sam2_hiera_det_model
✓ kept
vision_config.backbone_config
tie_word_embeddings
True
True
✓ kept
vision_config.backbone_config
hidden_act
gelu
gelu
✓ kept
vision_config.backbone_config
image_size
[1024, 1024]
[1024, 1024]
✓ kept
vision_config.backbone_config
num_channels
3
3
✓ kept
prompt_encoder_config
model_type
sam2_video_prompt_encoder
sam2_prompt_encoder
⚠ changed
prompt_encoder_config
hidden_act
gelu
gelu
✓ kept
prompt_encoder_config
image_size
1024
1024
✓ kept
prompt_encoder_config
patch_size
16
16
✓ kept
mask_decoder_config
model_type
sam2_video_mask_decoder
sam2_mask_decoder
⚠ changed
mask_decoder_config
hidden_act
gelu
gelu
✓ kept
Module classes
Instances of each class in the built model (prefix Sam2 omitted).
+
\ No newline at end of file
diff --git a/shrink/seamless-m4t-v2-large-mic.html b/shrink/seamless-m4t-v2-large-mic.html
new file mode 100644
index 0000000000000000000000000000000000000000..f8c4608b5aa97379539dc6bd126e7dfb6f782917
--- /dev/null
+++ b/shrink/seamless-m4t-v2-large-mic.html
@@ -0,0 +1,34 @@
+
+
+Shrunk architecture
+
+
Shrunk architecture
huggingface-mic/seamless-m4t-v2-large-mic@43e20937aec2b5592d27e162442c2db3532e010b ← facebook/seamless-m4t-v2-large@5f8cc790b19fc3f67a61c105133b20b34e3dcb76 · SeamlessM4Tv2Model · random weights
+
Original2.3Bparameters
Fixture95.0Mparameters
Layer stacks7compared
Findings0to review
✓ Same architecture, smaller
Every module class, layer kind and checked property of the original is in the fixture.
Layer stacks
One square per layer, colored by kind: the layer's module classes and type attributes. Hover a square for its index and kind.
text_encoder.layers
Original · 24
Fixture · 2
Kind
Original
Fixture
EncoderLayer
24
2
✓ kept
speech_encoder.encoder.layers
Original · 24
Fixture · 24
Kind
Original
Fixture
ConformerEncoderLayer
24
24
✓ kept
speech_encoder.adapter.layers
Original · 1
Fixture · 1
Kind
Original
Fixture
ConformerAdapterLayer
1
1
✓ kept
text_decoder.layers
Original · 24
Fixture · 2
Kind
Original
Fixture
DecoderLayer
24
2
✓ kept
t2u_model.model.encoder.layers
Original · 6
Fixture · 6
Kind
Original
Fixture
EncoderLayer
6
6
✓ kept
t2u_model.model.decoder.layers
Original · 6
Fixture · 6
Kind
Original
Fixture
TextToUnitDecoderLayer
6
6
✓ kept
vocoder.hifi_gan.resblocks
Original · 15
Fixture · 15
Kind
Original
Fixture
HifiGanResidualBlock
15
15
✓ kept
Where the parameters are
Share of each model's parameters per component. Vocabulary and patch sizes are kept, so embeddings usually take a larger share of the fixture.
Original
Fixture
Component
Original
Share
Fixture
Share
text_decoder.layers
604.5M
26.2%
100.5K
0.1%
speech_encoder.encoder.layers
580.3M
25.1%
26.1M
27.5%
text_encoder.layers
503.7M
21.8%
66.9K
0.1%
shared
262.2M
11.4%
16.4M
17.2%
t2u_model.model.encoder.layers
125.9M
5.5%
6.4M
6.8%
t2u_model.model.decoder.layers
113.3M
4.9%
446.2K
0.5%
speech_encoder.adapter.layers
46.2M
2.0%
676.8K
0.7%
13 others
73.0M
3.2%
44.8M
47.1%
Preserved properties
Config
Property
Original
Fixture
model
model_type
seamless_m4t_v2
seamless_m4t_v2
✓ kept
model
vocab_size
256102
256102
✓ kept
model
tie_word_embeddings
True
True
✓ kept
model
max_position_embeddings
4096
4096
✓ kept
model
sampling_rate
16000
16000
✓ kept
Module classes
Instances of each class in the built model (prefix SeamlessM4Tv2 omitted).
Class
Original
Fixture
Linear
649
297
Dropout
346
214
LayerNorm
302
192
Conv1d
182
182
ReLU
88
22
Attention
84
18
SiLU
72
72
FeedForwardNetwork
54
10
ConformerFeedForward
50
50
ModuleList
38
38
Embedding
30
30
EncoderLayer
30
8
GLU
25
25
ConformerSelfAttention
25
25
ConformerConvolutionModule
24
24
ConformerEncoderLayer
24
24
DecoderLayer
24
2
HifiGanResidualBlock
15
15
TextToUnitDecoderLayer
6
6
ConvTranspose1d
5
5
SinusoidalPositionalEmbedding
4
4
Encoder
2
2
ScaledWordEmbedding
2
2
VariancePredictor
2
2
CodeHifiGan
1
1
ConformerAdapter
1
1
ConformerAdapterLayer
1
1
ConformerEncoder
1
1
ConformerFeatureProjection
1
1
Decoder
1
1
HifiGan
1
1
Model
1
1
SpeechEncoder
1
1
TextToUnitDecoder
1
1
TextToUnitForConditionalGeneration
1
1
TextToUnitModel
1
1
Config changes
7 fields changed, 77 unchanged.
Field
Original
Fixture
decoder_attention_heads
16
4
decoder_ffn_dim
8192
128
decoder_layers
24
2
encoder_attention_heads
16
4
encoder_ffn_dim
8192
128
encoder_layers
24
2
hidden_size
1024
64
+
\ No newline at end of file
diff --git a/shrink/segformer-b0-finetuned-ade-512-512-mic.html b/shrink/segformer-b0-finetuned-ade-512-512-mic.html
new file mode 100644
index 0000000000000000000000000000000000000000..44be066cc157ce4167228b1b073c32afe0f71cea
--- /dev/null
+++ b/shrink/segformer-b0-finetuned-ade-512-512-mic.html
@@ -0,0 +1,34 @@
+
+
+Shrunk architecture
+
+
Shrunk architecture
huggingface-mic/segformer-b0-finetuned-ade-512-512-mic@b49b77daf8758e1d1c6d7f37b58b535586bcf0fb ← nvidia/segformer-b0-finetuned-ade-512-512@489d5cd81a0b59fab9b7ea758d3548ebe99677da · SegformerForSemanticSegmentation · overfit weights · built before the shrink rule (no recorded recipe)
+
Original3.8Mparameters
Fixture787.3Kparameters
Layer stacks2compared
Findings0to review
✓ Same architecture, smaller
Every module class, layer kind and checked property of the original is in the fixture.
Layer stacks
One square per layer, colored by kind: the layer's module classes and type attributes. Hover a square for its index and kind.
segformer.stages
Original · 4
Fixture · 4
Kind
Original
Fixture
SequenceReduction
3
3
✓ kept
no SequenceReduction
1
1
✓ kept
decode_head.linear_projections
Original · 4
Fixture · 4
Kind
Original
Fixture
Linear
4
4
✓ kept
Where the parameters are
Share of each model's parameters per component. Vocabulary and patch sizes are kept, so embeddings usually take a larger share of the fixture.
Original
Fixture
Component
Original
Share
Fixture
Share
segformer.stages
3.3M
88.5%
423.6K
53.8%
decode_head.linear_fuse
262.1K
7.0%
262.1K
33.3%
decode_head.linear_projections
132.1K
3.5%
62.5K
7.9%
decode_head.classifier
38.5K
1.0%
38.5K
4.9%
decode_head.batch_norm
512
0.0%
512
0.1%
Preserved properties
Config
Property
Original
Fixture
model
model_type
segformer
segformer
✓ kept
model
hidden_act
gelu
gelu
✓ kept
model
image_size
224
224
✓ kept
model
num_channels
3
3
✓ kept
Module classes
Instances of each class in the built model (prefix Segformer omitted).
Class
Original
Fixture
Linear
52
28
LayerNorm
30
19
Conv2d
20
13
Dropout
17
9
GELUActivation
8
4
Attention
8
4
DepthWiseConv
8
4
Layer
8
4
MixMLP
8
4
DropPath
7
3
ModuleList
6
6
SequenceReduction
6
3
MLP
4
4
OverlapPatchEmbeddings
4
4
Stage
4
4
BatchNorm2d
1
1
Identity
1
1
ReLU
1
1
DecodeHead
1
1
ForSemanticSegmentation
1
1
Model
1
1
Config changes
3 fields changed, 27 unchanged.
Field
Original
Fixture
depths
[2, 2, 2, 2]
[1, 1, 1, 1]
hidden_sizes
[32, 64, 160, 256]
[16, 32, 64, 128]
num_attention_heads
[1, 2, 5, 8]
[1, 2, 4, 8]
+
\ No newline at end of file
diff --git a/shrink/siglip-base-patch16-224-mic.html b/shrink/siglip-base-patch16-224-mic.html
new file mode 100644
index 0000000000000000000000000000000000000000..25f2d6db94b31567a36c62ae5c3352a341fd23a6
--- /dev/null
+++ b/shrink/siglip-base-patch16-224-mic.html
@@ -0,0 +1,34 @@
+
+
+Shrunk architecture
+
+
Shrunk architecture
huggingface-mic/siglip-base-patch16-224-mic@13fcabe2240b7fb71b1d3f2889e4f71778df91eb ← google/siglip-base-patch16-224@7fd15f0689c79d79e38b1c2e2e2370a7bf2761ed · SiglipModel · random weights · built before the shrink rule (no recorded recipe)
+
Original203.2Mparameters
Fixture2.3Mparameters
Layer stacks2compared
Findings0to review
✓ Same architecture, smaller
Every module class, layer kind and checked property of the original is in the fixture.
Layer stacks
One square per layer, colored by kind: the layer's module classes and type attributes. Hover a square for its index and kind.
text_model.encoder.layers
Original · 12
Fixture · 2
Kind
Original
Fixture
EncoderLayer
12
2
✓ kept
vision_model.encoder.layers
Original · 12
Fixture · 2
Kind
Original
Fixture
EncoderLayer
12
2
✓ kept
Where the parameters are
Share of each model's parameters per component. Vocabulary and patch sizes are kept, so embeddings usually take a larger share of the fixture.
Original
Fixture
Component
Original
Share
Fixture
Share
vision_model.encoder.layers
85.1M
41.9%
66.9K
2.9%
text_model.encoder.layers
85.1M
41.9%
66.9K
2.9%
text_model.embeddings
24.6M
12.1%
2.1M
89.8%
vision_model.head
7.1M
3.5%
33.4K
1.5%
vision_model.embeddings
741.1K
0.4%
61.8K
2.7%
text_model.head
590.6K
0.3%
4.2K
0.2%
vision_model.post_layernorm
1.5K
0.0%
128
0.0%
3 others
1.5K
0.0%
130
0.0%
Preserved properties
Config
Property
Original
Fixture
model
model_type
siglip
siglip
✓ kept
text_config
model_type
siglip_text_model
siglip_text_model
✓ kept
text_config
vocab_size
32000
32000
✓ kept
text_config
hidden_act
gelu_pytorch_tanh
gelu_pytorch_tanh
✓ kept
text_config
max_position_embeddings
64
64
✓ kept
vision_config
model_type
siglip_vision_model
siglip_vision_model
✓ kept
vision_config
hidden_act
gelu_pytorch_tanh
gelu_pytorch_tanh
✓ kept
vision_config
image_size
224
224
✓ kept
vision_config
patch_size
16
16
✓ kept
vision_config
num_channels
3
3
✓ kept
Module classes
Instances of each class in the built model (prefix Siglip omitted).
Class
Original
Fixture
Linear
147
27
LayerNorm
51
11
GELUTanh
25
5
MLP
25
5
Attention
24
4
EncoderLayer
24
4
Embedding
3
3
ModuleList
2
2
Encoder
2
2
Conv2d
1
1
MultiheadAttention
1
1
NonDynamicallyQuantizableLinear
1
1
Model
1
1
MultiheadAttentionPoolingHead
1
1
TextEmbeddings
1
1
TextModel
1
1
VisionEmbeddings
1
1
VisionModel
1
1
Config changes
9 fields changed, 42 unchanged.
Field
Original
Fixture
text_config.hidden_size
768
64
text_config.intermediate_size
3072
128
text_config.num_attention_heads
12
4
text_config.num_hidden_layers
12
2
text_config.projection_size
768
64
vision_config.hidden_size
768
64
vision_config.intermediate_size
3072
128
vision_config.num_attention_heads
12
4
vision_config.num_hidden_layers
12
2
+
\ No newline at end of file
diff --git a/shrink/smollm-1.7b-mic.html b/shrink/smollm-1.7b-mic.html
new file mode 100644
index 0000000000000000000000000000000000000000..05b3456fdca44ea7679e07f6d83af47bef9407b8
--- /dev/null
+++ b/shrink/smollm-1.7b-mic.html
@@ -0,0 +1,34 @@
+
+
+Shrunk architecture
+
+
Shrunk architecture
huggingface-mic/smollm-1.7b-mic@fc0e707f0977de7e4095792e1b56203e3af7b5c6 ← HuggingFaceTB/SmolLM-1.7B@d7449ff7241c863f3e8accc475155f0f97afa011 · LlamaForCausalLM · built before the shrink rule (no recorded recipe)
+
Original1.7Bparameters
Fixture3.2Mparameters
Layer stacks1compared
Findings0to review
✓ Same architecture, smaller
Every module class, layer kind and checked property of the original is in the fixture.
Layer stacks
One square per layer, colored by kind: the layer's module classes and type attributes. Hover a square for its index and kind.
model.layers
Original · 24
Fixture · 2
Kind
Original
Fixture
DecoderLayer
24
2
✓ kept
Where the parameters are
Share of each model's parameters per component. Vocabulary and patch sizes are kept, so embeddings usually take a larger share of the fixture.
Original
Fixture
Component
Original
Share
Fixture
Share
model.layers
1.6B
94.1%
82.2K
2.5%
model.embed_tokens
100.7M
5.9%
3.1M
97.5%
model.norm
2.0K
0.0%
64
0.0%
Preserved properties
Config
Property
Original
Fixture
model
attention style
MHA (32 heads, 32 KV)
MHA (4 heads, 4 KV)
✓ kept
model
RoPE type
default
default
✓ kept
model
model_type
llama
llama
✓ kept
model
vocab_size
49152
49152
✓ kept
model
tie_word_embeddings
True
True
✓ kept
model
hidden_act
silu
silu
✓ kept
model
max_position_embeddings
2048
2048
✓ kept
Module classes
Instances of each class in the built model (prefix Llama omitted).
Class
Original
Fixture
Linear
169
15
RMSNorm
49
5
Attention
24
2
DecoderLayer
24
2
MLP
24
2
SiLUActivation
24
2
Embedding
1
1
ForCausalLM
1
1
Model
1
1
RotaryEmbedding
1
1
ModuleList
1
1
Config changes
7 fields changed, 23 unchanged.
Field
Original
Fixture
head_dim
64
16
hidden_size
2048
64
intermediate_size
8192
128
num_attention_heads
32
4
num_hidden_layers
24
2
num_key_value_heads
32
4
pad_token_id
null
0
+
\ No newline at end of file
diff --git a/shrink/smollm-mic.html b/shrink/smollm-mic.html
new file mode 100644
index 0000000000000000000000000000000000000000..fa293815c2692cc796d58d1d27dd44b84c7f314c
--- /dev/null
+++ b/shrink/smollm-mic.html
@@ -0,0 +1,34 @@
+
+
+Shrunk architecture
+
+
Shrunk architecture
huggingface-mic/smollm-mic@d0da7b30e6c71b294e950e4abdbc7b858854bf28 ← HuggingFaceTB/SmolLM-135M@1d461723eec654e65efdc40cf49301c89c0c92f4 · LlamaForCausalLM · overfit weights · built before the shrink rule (no recorded recipe)
+
Original134.5Mparameters
Fixture3.2Mparameters
Layer stacks1compared
Findings0to review
✓ Same architecture, smaller
Every module class, layer kind and checked property of the original is in the fixture.
Layer stacks
One square per layer, colored by kind: the layer's module classes and type attributes. Hover a square for its index and kind.
model.layers
Original · 30
Fixture · 2
Kind
Original
Fixture
DecoderLayer
30
2
✓ kept
Where the parameters are
Share of each model's parameters per component. Vocabulary and patch sizes are kept, so embeddings usually take a larger share of the fixture.
Original
Fixture
Component
Original
Share
Fixture
Share
model.layers
106.2M
79.0%
74.0K
2.3%
model.embed_tokens
28.3M
21.0%
3.1M
97.7%
model.norm
576
0.0%
64
0.0%
Preserved properties
Config
Property
Original
Fixture
model
attention style
GQA (9 heads, 3 KV)
GQA (4 heads, 2 KV)
✓ kept
model
RoPE type
default
default
✓ kept
model
model_type
llama
llama
✓ kept
model
vocab_size
49152
49152
✓ kept
model
tie_word_embeddings
True
True
✓ kept
model
hidden_act
silu
silu
✓ kept
model
max_position_embeddings
2048
2048
✓ kept
Module classes
Instances of each class in the built model (prefix Llama omitted).
Class
Original
Fixture
Linear
211
15
RMSNorm
61
5
Attention
30
2
DecoderLayer
30
2
MLP
30
2
SiLUActivation
30
2
Embedding
1
1
ForCausalLM
1
1
Model
1
1
RotaryEmbedding
1
1
ModuleList
1
1
Config changes
6 fields changed, 24 unchanged.
Field
Original
Fixture
head_dim
64
16
hidden_size
576
64
intermediate_size
1536
128
num_attention_heads
9
4
num_hidden_layers
30
2
num_key_value_heads
3
2
+
\ No newline at end of file
diff --git a/shrink/smolvlm-mic.html b/shrink/smolvlm-mic.html
new file mode 100644
index 0000000000000000000000000000000000000000..61e5e69b36662c73bf9a6a1d57fd1ec550acb852
--- /dev/null
+++ b/shrink/smolvlm-mic.html
@@ -0,0 +1,34 @@
+
+
+Shrunk architecture
+
+
Shrunk architecture
huggingface-mic/smolvlm-mic@8cefc6b12559831ef4a11f203b84848058513ee4 ← HuggingFaceTB/SmolVLM2-2.2B-Instruct@482adb537c021c86670beed01cd58990d01e72e4 · SmolVLMForConditionalGeneration · random weights · built before the shrink rule (no recorded recipe)
+
Original2.2Bparameters
Fixture6.6Mparameters
Layer stacks2compared
Findings0to review
✓ Same architecture, smaller
Every module class, layer kind and checked property of the original is in the fixture.
Layer stacks
One square per layer, colored by kind: the layer's module classes and type attributes. Hover a square for its index and kind.
model.vision_model.encoder.layers
Original · 27
Fixture · 2
Kind
Original
Fixture
EncoderLayer
27
2
✓ kept
model.text_model.layers
Original · 24
Fixture · 2
Kind
Original
Fixture
LlamaDecoderLayer
24
2
✓ kept
Where the parameters are
Share of each model's parameters per component. Vocabulary and patch sizes are kept, so embeddings usually take a larger share of the fixture.
Original
Fixture
Component
Original
Share
Fixture
Share
model.text_model.layers
1.6B
71.7%
82.2K
1.2%
model.vision_model.encoder.layers
411.5M
18.3%
66.9K
1.0%
model.text_model
100.9M
4.5%
3.2M
47.9%
lm_head
100.9M
4.5%
3.2M
47.9%
model.connector
21.2M
0.9%
36.9K
0.6%
model.vision_model
1.5M
0.1%
84.5K
1.3%
Preserved properties
Config
Property
Original
Fixture
model
model_type
smolvlm
smolvlm
✓ kept
model
vocab_size
49280
49280
✓ kept
model
tie_word_embeddings
False
False
✓ kept
vision_config
model_type
smolvlm_vision
smolvlm_vision
✓ kept
vision_config
tie_word_embeddings
False
False
✓ kept
vision_config
hidden_act
gelu_pytorch_tanh
gelu_pytorch_tanh
✓ kept
vision_config
image_size
384
384
✓ kept
vision_config
patch_size
14
14
✓ kept
vision_config
num_channels
3
3
✓ kept
text_config
attention style
MHA (32 heads, 32 KV)
MHA (4 heads, 4 KV)
✓ kept
text_config
RoPE type
default
default
✓ kept
text_config
model_type
llama
llama
✓ kept
text_config
vocab_size
49280
49280
✓ kept
text_config
tie_word_embeddings
False
False
✓ kept
text_config
hidden_act
silu
silu
✓ kept
text_config
max_position_embeddings
8192
8192
✓ kept
text_config.perceiver_config
model_type
vllama3
vllama3
✓ kept
text_config.perceiver_config
tie_word_embeddings
True
True
✓ kept
text_config.perceiver_config
hidden_act
silu
silu
✓ kept
Module classes
Instances of each class in the built model (prefix SmolVLM omitted).
Class
Original
Fixture
Linear
332
28
LayerNorm
55
5
LlamaRMSNorm
49
5
GELUTanh
27
2
EncoderLayer
27
2
VisionAttention
27
2
VisionMLP
27
2
LlamaAttention
24
2
LlamaDecoderLayer
24
2
LlamaMLP
24
2
SiLUActivation
24
2
Embedding
2
2
ModuleList
2
2
Conv2d
1
1
LlamaModel
1
1
LlamaRotaryEmbedding
1
1
Connector
1
1
Encoder
1
1
ForConditionalGeneration
1
1
Model
1
1
SimpleMLP
1
1
VisionEmbeddings
1
1
VisionTransformer
1
1
Config changes
10 fields changed, 127 unchanged.
Field
Original
Fixture
text_config.head_dim
64
16
text_config.hidden_size
2048
64
text_config.intermediate_size
8192
128
text_config.num_attention_heads
32
4
text_config.num_hidden_layers
24
2
text_config.num_key_value_heads
32
4
vision_config.hidden_size
1152
64
vision_config.intermediate_size
4304
128
vision_config.num_attention_heads
16
4
vision_config.num_hidden_layers
27
2
+
\ No newline at end of file
diff --git a/shrink/speech_t5-mic.html b/shrink/speech_t5-mic.html
new file mode 100644
index 0000000000000000000000000000000000000000..21b97122e177722ec1fbb1b1f285161742a2fb40
--- /dev/null
+++ b/shrink/speech_t5-mic.html
@@ -0,0 +1,34 @@
+
+
+Shrunk architecture
+
+
Shrunk architecture
huggingface-mic/speech_t5-mic@889b4e918f22534926a9817ffa5aa22df47af8df ← microsoft/speecht5_asr@53615c10408485422e09a12cda191a747f4bbe34 · SpeechT5ForSpeechToText · random weights · built before the shrink rule (no recorded recipe)
+
Original151.2Mparameters
Fixture4.4Mparameters
Layer stacks3compared
Findings0to review
✓ Same architecture, smaller
Every module class, layer kind and checked property of the original is in the fixture.
Layer stacks
One square per layer, colored by kind: the layer's module classes and type attributes. Hover a square for its index and kind.
Instances of each class in the built model (prefix SpeechT5 omitted).
Class
Original
Fixture
Linear
134
34
Dropout
57
15
LayerNorm
44
12
GELUActivation
26
12
Attention
24
6
FeedForward
18
4
EncoderLayer
12
2
Conv1d
7
7
DecoderLayer
6
2
NoLayerNormConvLayer
6
6
ModuleList
3
3
Embedding
2
2
SinusoidalPositionalEmbedding
2
2
GroupNorm
1
1
ModuleDict
1
1
ParametrizationList
1
1
ParametrizedConv1d
1
1
Decoder
1
1
DecoderWithTextPrenet
1
1
Encoder
1
1
EncoderWithSpeechPrenet
1
1
FeatureEncoder
1
1
FeatureProjection
1
1
ForSpeechToText
1
1
GroupNormConvLayer
1
1
Model
1
1
PositionalConvEmbedding
1
1
RelativePositionalEncoding
1
1
SamePadLayer
1
1
SpeechEncoderPrenet
1
1
TextDecoderPostnet
1
1
TextDecoderPrenet
1
1
_WeightNorm
1
1
Config changes
7 fields changed, 60 unchanged.
Field
Original
Fixture
decoder_attention_heads
12
4
decoder_ffn_dim
3072
128
decoder_layers
6
2
encoder_attention_heads
12
4
encoder_ffn_dim
3072
128
encoder_layers
12
2
hidden_size
768
64
+
\ No newline at end of file
diff --git a/shrink/stablelm-2-1_6b-mic.html b/shrink/stablelm-2-1_6b-mic.html
new file mode 100644
index 0000000000000000000000000000000000000000..f526688d83f8f49094820bf3fd8d21c5e3c272e1
--- /dev/null
+++ b/shrink/stablelm-2-1_6b-mic.html
@@ -0,0 +1,34 @@
+
+
+Shrunk architecture
+
+
Shrunk architecture
huggingface-mic/stablelm-2-1_6b-mic@eec0b53366b61c85645d4c7ec7618033265b6dd3 ← stabilityai/stablelm-2-1_6b@f499ead74c53749bd93cebc6ce8bc0d7bdf1eaef · StableLmForCausalLM · random weights
+
Original1.6Bparameters
Fixture12.9Mparameters
Layer stacks1compared
Findings0to review
✓ Same architecture, smaller
Every module class, layer kind and checked property of the original is in the fixture.
Layer stacks
One square per layer, colored by kind: the layer's module classes and type attributes. Hover a square for its index and kind.
model.layers
Original · 24
Fixture · 2
Kind
Original
Fixture
DecoderLayer
24
2
✓ kept
Where the parameters are
Share of each model's parameters per component. Vocabulary and patch sizes are kept, so embeddings usually take a larger share of the fixture.
Original
Fixture
Component
Original
Share
Fixture
Share
model.layers
1.2B
75.0%
82.8K
0.6%
model.embed_tokens
205.5M
12.5%
6.4M
49.7%
lm_head
205.5M
12.5%
6.4M
49.7%
model.norm
4.1K
0.0%
128
0.0%
Preserved properties
Config
Property
Original
Fixture
model
attention style
MHA (32 heads, 32 KV)
MHA (4 heads, 4 KV)
✓ kept
model
RoPE type
default
default
✓ kept
model
model_type
stablelm
stablelm
✓ kept
model
vocab_size
100352
100352
✓ kept
model
tie_word_embeddings
False
False
✓ kept
model
hidden_act
silu
silu
✓ kept
model
max_position_embeddings
4096
4096
✓ kept
Module classes
Instances of each class in the built model (prefix StableLm omitted).
Class
Original
Fixture
Linear
169
15
LayerNorm
49
5
Dropout
24
2
SiLUActivation
24
2
Attention
24
2
DecoderLayer
24
2
MLP
24
2
Embedding
1
1
ModuleList
1
1
ForCausalLM
1
1
Model
1
1
RotaryEmbedding
1
1
Config changes
5 fields changed, 25 unchanged.
Field
Original
Fixture
hidden_size
2048
64
intermediate_size
5632
128
num_attention_heads
32
4
num_hidden_layers
24
2
num_key_value_heads
32
4
+
\ No newline at end of file
diff --git a/shrink/starcoder2-3b-mic.html b/shrink/starcoder2-3b-mic.html
new file mode 100644
index 0000000000000000000000000000000000000000..1245c04b98582bdc602b8134fe88cffe13517d0f
--- /dev/null
+++ b/shrink/starcoder2-3b-mic.html
@@ -0,0 +1,34 @@
+
+
+Shrunk architecture
+
+
Shrunk architecture
huggingface-mic/starcoder2-3b-mic@f158c5574331920c1b49c8447d5bf4a4864df079 ← bigcode/starcoder2-3b@733247c55e3f73af49ce8e9c7949bf14af205928 · Starcoder2ForCausalLM · random weights
+
Original3.0Bparameters
Fixture3.2Mparameters
Layer stacks1compared
Findings0to review
✓ Same architecture, smaller
Every module class, layer kind and checked property of the original is in the fixture.
Layer stacks
One square per layer, colored by kind: the layer's module classes and type attributes. Hover a square for its index and kind.
model.layers
Original · 30
Fixture · 2
Kind
Original
Fixture
DecoderLayer
30
2
✓ kept
Where the parameters are
Share of each model's parameters per component. Vocabulary and patch sizes are kept, so embeddings usually take a larger share of the fixture.
Original
Fixture
Component
Original
Share
Fixture
Share
model.layers
2.9B
95.0%
58.6K
1.8%
model.embed_tokens
151.0M
5.0%
3.1M
98.2%
model.norm
6.1K
0.0%
128
0.0%
Preserved properties
Config
Property
Original
Fixture
model
attention style
GQA (24 heads, 2 KV)
GQA (4 heads, 2 KV)
✓ kept
model
RoPE type
default
default
✓ kept
model
model_type
starcoder2
starcoder2
✓ kept
model
vocab_size
49152
49152
✓ kept
model
tie_word_embeddings
True
True
✓ kept
model
hidden_act
gelu_pytorch_tanh
gelu_pytorch_tanh
✓ kept
model
max_position_embeddings
16384
16384
✓ kept
Module classes
Instances of each class in the built model (prefix Starcoder2 omitted).
Class
Original
Fixture
Linear
181
13
LayerNorm
61
5
GELUTanh
30
2
Attention
30
2
DecoderLayer
30
2
MLP
30
2
Embedding
1
1
ModuleList
1
1
ForCausalLM
1
1
Model
1
1
RotaryEmbedding
1
1
Config changes
4 fields changed, 28 unchanged.
Field
Original
Fixture
hidden_size
3072
64
intermediate_size
12288
128
num_attention_heads
24
4
num_hidden_layers
30
2
+
\ No newline at end of file
diff --git a/shrink/swin-tiny-patch4-window7-224-mic.html b/shrink/swin-tiny-patch4-window7-224-mic.html
new file mode 100644
index 0000000000000000000000000000000000000000..8be256d14ccbb2058e3e3e8ae5868da737d6f69c
--- /dev/null
+++ b/shrink/swin-tiny-patch4-window7-224-mic.html
@@ -0,0 +1,34 @@
+
+
+Shrunk architecture
+
+
Shrunk architecture
huggingface-mic/swin-tiny-patch4-window7-224-mic@89ae45d56ca206c1ba47aee79182a5e98d039eb6 ← microsoft/swin-tiny-patch4-window7-224@d00d478bfaf1f417d34a1186673ad4b4be264c51 · SwinForImageClassification · random weights
+
Original28.3Mparameters
Fixture5.4Mparameters
Layer stacks1compared
Findings0to review
✓ Same architecture, smaller
Every module class, layer kind and checked property of the original is in the fixture.
Layer stacks
One square per layer, colored by kind: the layer's module classes and type attributes. Hover a square for its index and kind.
swin.encoder.layers
Original · 4
Fixture · 4
Kind
Original
Fixture
PatchMerging
3
3
✓ kept
no PatchMerging
1
1
✓ kept
Where the parameters are
Share of each model's parameters per component. Vocabulary and patch sizes are kept, so embeddings usually take a larger share of the fixture.
Original
Fixture
Component
Original
Share
Fixture
Share
swin.encoder.layers
27.5M
97.3%
4.9M
90.4%
classifier
769.0K
2.7%
513.0K
9.5%
swin.embeddings
4.9K
0.0%
3.3K
0.1%
swin.layernorm
1.5K
0.0%
1.0K
0.0%
Preserved properties
Config
Property
Original
Fixture
model
model_type
swin
swin
✓ kept
model
hidden_act
gelu
gelu
✓ kept
model
image_size
224
224
✓ kept
model
patch_size
4
4
✓ kept
model
num_channels
3
3
✓ kept
Module classes
Instances of each class in the built model (prefix Swin omitted).
Class
Original
Fixture
Linear
76
28
LayerNorm
29
13
Dropout
13
5
GELUActivation
12
4
Attention
12
4
Layer
12
4
MLP
12
4
RelativePositionBias
12
4
DropPath
11
3
ModuleList
5
5
Stage
4
4
PatchMerging
3
3
AdaptiveAvgPool1d
1
1
Conv2d
1
1
Identity
1
1
Embeddings
1
1
Encoder
1
1
ForImageClassification
1
1
Model
1
1
PatchEmbeddings
1
1
Config changes
5 fields changed, 27 unchanged.
Field
Original
Fixture
depths
[2, 2, 6, 2]
[1, 1, 1, 1]
embed_dim
96
64
hidden_size
768
64
num_heads
[3, 6, 12, 24]
[2, 4, 8, 16]
num_layers
4
2
+
\ No newline at end of file
diff --git a/shrink/swinv2-tiny-patch4-window16-256-mic.html b/shrink/swinv2-tiny-patch4-window16-256-mic.html
new file mode 100644
index 0000000000000000000000000000000000000000..da58802b06dd553b96d38969f43ca48bcf57f471
--- /dev/null
+++ b/shrink/swinv2-tiny-patch4-window16-256-mic.html
@@ -0,0 +1,34 @@
+
+
+Shrunk architecture
+
+
Shrunk architecture
huggingface-mic/swinv2-tiny-patch4-window16-256-mic@8c997579b76fa9f22e2eb5f9e186497da2294e6c ← microsoft/swinv2-tiny-patch4-window16-256@f4d3075206f2ad5eda586c30d6b4d0500f312421 · Swinv2ForImageClassification · random weights
+
Original28.3Mparameters
Fixture5.4Mparameters
Layer stacks1compared
Findings0to review
✓ Same architecture, smaller
Every module class, layer kind and checked property of the original is in the fixture.
Layer stacks
One square per layer, colored by kind: the layer's module classes and type attributes. Hover a square for its index and kind.
swinv2.encoder.layers
Original · 4
Fixture · 4
Kind
Original
Fixture
PatchMerging
3
3
✓ kept
no PatchMerging
1
1
✓ kept
Where the parameters are
Share of each model's parameters per component. Vocabulary and patch sizes are kept, so embeddings usually take a larger share of the fixture.
Original
Fixture
Component
Original
Share
Fixture
Share
swinv2.encoder.layers
27.6M
97.3%
4.9M
90.5%
classifier
769.0K
2.7%
513.0K
9.5%
swinv2.embeddings
4.9K
0.0%
3.3K
0.1%
swinv2.layernorm
1.5K
0.0%
1.0K
0.0%
Preserved properties
Config
Property
Original
Fixture
model
model_type
swinv2
swinv2
✓ kept
model
hidden_act
gelu
gelu
✓ kept
model
image_size
256
256
✓ kept
model
patch_size
4
4
✓ kept
model
num_channels
3
3
✓ kept
Module classes
Instances of each class in the built model (prefix Swinv2 omitted).
Class
Original
Fixture
Linear
100
36
Dropout
37
13
LayerNorm
29
13
GELUActivation
12
4
ReLU
12
4
Sequential
12
4
Attention
12
4
Intermediate
12
4
Layer
12
4
Output
12
4
SelfAttention
12
4
SelfOutput
12
4
DropPath
11
3
ModuleList
5
5
Stage
4
4
PatchMerging
3
3
AdaptiveAvgPool1d
1
1
Conv2d
1
1
Identity
1
1
Embeddings
1
1
Encoder
1
1
ForImageClassification
1
1
Model
1
1
PatchEmbeddings
1
1
Config changes
5 fields changed, 28 unchanged.
Field
Original
Fixture
depths
[2, 2, 6, 2]
[1, 1, 1, 1]
embed_dim
96
64
hidden_size
768
64
num_heads
[3, 6, 12, 24]
[2, 4, 8, 16]
num_layers
4
2
+
\ No newline at end of file
diff --git a/shrink/t5-mic.html b/shrink/t5-mic.html
new file mode 100644
index 0000000000000000000000000000000000000000..60faae3d94baa6794f55a19b52beaf04730d8d3d
--- /dev/null
+++ b/shrink/t5-mic.html
@@ -0,0 +1,34 @@
+
+
+Shrunk architecture
+
+
Shrunk architecture
huggingface-mic/t5-mic@17a8c0b6ac87899e79dc2ab8a6b5567dc3948162 ← google-t5/t5-small@df1b051c49625cf57a3d0d8d3863ed4d13564fe4 · T5ForConditionalGeneration · random weights · built before the shrink rule (no recorded recipe)
+
Original60.5Mparameters
Fixture2.2Mparameters
Layer stacks2compared
Findings0to review
✓ Same architecture, smaller
Every module class, layer kind and checked property of the original is in the fixture.
Layer stacks
One square per layer, colored by kind: the layer's module classes and type attributes. Hover a square for its index and kind.
encoder.block
Original · 6
Fixture · 2
Kind
Original
Fixture
Embedding
1
1
✓ kept
no Embedding
5
1
✓ kept
decoder.block
Original · 6
Fixture · 2
Kind
Original
Fixture
Embedding
1
1
✓ kept
no Embedding
5
1
✓ kept
Where the parameters are
Share of each model's parameters per component. Vocabulary and patch sizes are kept, so embeddings usually take a larger share of the fixture.
Original
Fixture
Component
Original
Share
Fixture
Share
decoder.block
25.2M
41.6%
98.8K
4.4%
encoder.block
18.9M
31.2%
65.9K
3.0%
shared
16.4M
27.2%
2.1M
92.6%
decoder.final_layer_norm
512
0.0%
64
0.0%
encoder.final_layer_norm
512
0.0%
64
0.0%
Preserved properties
Config
Property
Original
Fixture
model
model_type
t5
t5
✓ kept
model
vocab_size
32128
32128
✓ kept
model
tie_word_embeddings
True
True
✓ kept
Module classes
Instances of each class in the built model (prefix T5 omitted).
Class
Original
Fixture
Linear
97
33
Dropout
44
16
LayerNorm
32
12
Attention
18
6
ModuleList
14
6
ReLU
12
4
Block
12
4
DenseActDense
12
4
LayerFF
12
4
LayerSelfAttention
12
4
LayerCrossAttention
6
2
Embedding
5
5
Stack
2
2
ForConditionalGeneration
1
1
Config changes
6 fields changed, 29 unchanged.
Field
Original
Fixture
d_ff
2048
128
d_kv
64
16
d_model
512
64
num_decoder_layers
6
2
num_heads
8
4
num_layers
6
2
+
\ No newline at end of file
diff --git a/shrink/t5-small-mic.html b/shrink/t5-small-mic.html
new file mode 100644
index 0000000000000000000000000000000000000000..fd905b219f7d13ed807cc4609f7e5528a4f6f90e
--- /dev/null
+++ b/shrink/t5-small-mic.html
@@ -0,0 +1,34 @@
+
+
+Shrunk architecture
+
+
Shrunk architecture
huggingface-mic/t5-small-mic@4f5dcdc6040660035cd0f86a8867270fb7405f46 ← google-t5/t5-small@df1b051c49625cf57a3d0d8d3863ed4d13564fe4 · T5ForConditionalGeneration · random weights · built before the shrink rule (no recorded recipe)
+
Original60.5Mparameters
Fixture2.2Mparameters
Layer stacks2compared
Findings0to review
✓ Same architecture, smaller
Every module class, layer kind and checked property of the original is in the fixture.
Layer stacks
One square per layer, colored by kind: the layer's module classes and type attributes. Hover a square for its index and kind.
encoder.block
Original · 6
Fixture · 2
Kind
Original
Fixture
Embedding
1
1
✓ kept
no Embedding
5
1
✓ kept
decoder.block
Original · 6
Fixture · 2
Kind
Original
Fixture
Embedding
1
1
✓ kept
no Embedding
5
1
✓ kept
Where the parameters are
Share of each model's parameters per component. Vocabulary and patch sizes are kept, so embeddings usually take a larger share of the fixture.
Original
Fixture
Component
Original
Share
Fixture
Share
decoder.block
25.2M
41.6%
98.8K
4.4%
encoder.block
18.9M
31.2%
65.9K
3.0%
shared
16.4M
27.2%
2.1M
92.6%
decoder.final_layer_norm
512
0.0%
64
0.0%
encoder.final_layer_norm
512
0.0%
64
0.0%
Preserved properties
Config
Property
Original
Fixture
model
model_type
t5
t5
✓ kept
model
vocab_size
32128
32128
✓ kept
model
tie_word_embeddings
True
True
✓ kept
Module classes
Instances of each class in the built model (prefix T5 omitted).
Class
Original
Fixture
Linear
97
33
Dropout
44
16
LayerNorm
32
12
Attention
18
6
ModuleList
14
6
ReLU
12
4
Block
12
4
DenseActDense
12
4
LayerFF
12
4
LayerSelfAttention
12
4
LayerCrossAttention
6
2
Embedding
5
5
Stack
2
2
ForConditionalGeneration
1
1
Config changes
6 fields changed, 29 unchanged.
Field
Original
Fixture
d_ff
2048
128
d_kv
64
16
d_model
512
64
num_decoder_layers
6
2
num_heads
8
4
num_layers
6
2
+
\ No newline at end of file
diff --git a/shrink/table_transformer-mic.html b/shrink/table_transformer-mic.html
new file mode 100644
index 0000000000000000000000000000000000000000..588886198c793d0593ca42181d5de2cb5056f86f
--- /dev/null
+++ b/shrink/table_transformer-mic.html
@@ -0,0 +1,34 @@
+
+
+Shrunk architecture
+
+
Shrunk architecture
huggingface-mic/table_transformer-mic@5b726b4a1f1b6488656b90260731a596ac3955fa ← microsoft/table-transformer-detection@2357cbe2b5a5d1c03e54f32764f06058933b65ab · TableTransformerForObjectDetection · random weights · built before the shrink rule (no recorded recipe)
+
Original28.8Mparameters
Fixture11.4Mparameters
Layer stacks2compared
Findings0to review
✓ Same architecture, smaller
Every module class, layer kind and checked property of the original is in the fixture.
Layer stacks
One square per layer, colored by kind: the layer's module classes and type attributes. Hover a square for its index and kind.
model.encoder.layers
Original · 6
Fixture · 2
Kind
Original
Fixture
EncoderLayer
6
2
✓ kept
model.decoder.layers
Original · 6
Fixture · 2
Kind
Original
Fixture
DecoderLayer
6
2
✓ kept
Where the parameters are
Share of each model's parameters per component. Vocabulary and patch sizes are kept, so embeddings usually take a larger share of the fixture.
Original
Fixture
Component
Original
Share
Fixture
Share
model.backbone
11.2M
38.8%
11.2M
98.2%
model.decoder.layers
9.5M
32.9%
100.5K
0.9%
model.encoder.layers
7.9M
27.4%
66.9K
0.6%
bbox_predictor.layers
132.6K
0.5%
8.6K
0.1%
model.input_projection
131.3K
0.5%
32.8K
0.3%
model.query_position_embeddings
3.8K
0.0%
960
0.0%
class_labels_classifier
771
0.0%
195
0.0%
2 others
1.0K
0.0%
256
0.0%
Preserved properties
Config
Property
Original
Fixture
model
model_type
table-transformer
table-transformer
✓ kept
model
max_position_embeddings
1024
1024
✓ kept
model
num_channels
3
3
✓ kept
backbone_config
model_type
timm_backbone
timm_backbone
✓ kept
backbone_config
num_channels
3
3
✓ kept
Module classes
Instances of each class in the built model (prefix TableTransformer omitted).
Class
Original
Fixture
Linear
100
36
LayerNorm
32
12
ReLU
29
21
Conv2d
21
21
FrozenBatchNorm2d
20
20
Attention
18
6
Identity
16
16
BasicBlock
8
8
Sequential
7
7
DecoderLayer
6
2
EncoderLayer
6
2
ModuleList
3
3
Embedding
1
1
FeatureListNet
1
1
MaxPool2d
1
1
ConvEncoder
1
1
ConvModel
1
1
Decoder
1
1
Encoder
1
1
ForObjectDetection
1
1
MLPPredictionHead
1
1
Model
1
1
SinePositionEmbedding
1
1
Config changes
10 fields changed, 48 unchanged.
Field
Original
Fixture
backbone_config.out_features
null
["layer4"]
backbone_config.out_indices
[1, 2, 3, 4]
[-1]
backbone_config.stage_names
—
["act1", "layer1", "layer2", "layer3", "layer4"]
d_model
256
64
decoder_attention_heads
8
4
decoder_ffn_dim
2048
128
decoder_layers
6
2
encoder_attention_heads
8
4
encoder_ffn_dim
2048
128
encoder_layers
6
2
+
\ No newline at end of file
diff --git a/shrink/tibetan-ocr-mic.html b/shrink/tibetan-ocr-mic.html
new file mode 100644
index 0000000000000000000000000000000000000000..c1e04ad68f5134f9961b83bb69d0fee03aa21f29
--- /dev/null
+++ b/shrink/tibetan-ocr-mic.html
@@ -0,0 +1,34 @@
+
+
+Shrunk architecture
+
+
Shrunk architecture
huggingface-mic/tibetan-ocr-mic@78b84a91abf7da0acb2773e935fee059d6bbe227 ← BDRC/tibetan-ocr@50506eb6d8ed8738df86b448f6f6cdc688de29ea · PaddleOCRVLForConditionalGeneration · random weights
+
Original810.7Mparameters
Fixture535.7Kparameters
Layer stacks2compared
Findings0to review
✓ Same architecture, smaller
Every module class, layer kind and checked property of the original is in the fixture.
Layer stacks
One square per layer, colored by kind: the layer's module classes and type attributes. Hover a square for its index and kind.
model.visual.vision_model.encoder.layers
Original · 27
Fixture · 2
Kind
Original
Fixture
PaddleOCRVisionEncoderLayer
27
2
✓ kept
model.language_model.layers
Original · 26
Fixture · 2
Kind
Original
Fixture
PaddleOCRDecoderLayer
26
2
✓ kept
Where the parameters are
Share of each model's parameters per component. Vocabulary and patch sizes are kept, so embeddings usually take a larger share of the fixture.
Original
Fixture
Component
Original
Share
Fixture
Share
model.visual.vision_model.encoder.layers
411.5M
50.8%
66.9K
12.5%
model.language_model.layers
368.1M
45.4%
74.0K
13.8%
model.projector
26.0M
3.2%
82.4K
15.4%
model.language_model
3.6M
0.4%
227.9K
42.5%
model.visual
1.5M
0.2%
84.5K
15.8%
Preserved properties
Config
Property
Original
Fixture
model
model_type
paddleocr_vl
paddleocr_vl
✓ kept
model
tie_word_embeddings
True
True
✓ kept
text_config
attention style
GQA (16 heads, 2 KV)
GQA (4 heads, 2 KV)
✓ kept
text_config
RoPE type
default
default
✓ kept
text_config
model_type
paddleocr_vl_text
paddleocr_vl_text
✓ kept
text_config
vocab_size
3560
3560
✓ kept
text_config
tie_word_embeddings
True
True
✓ kept
text_config
hidden_act
silu
silu
✓ kept
text_config
max_position_embeddings
131072
131072
✓ kept
vision_config
RoPE type
axial
axial
✓ kept
vision_config
model_type
paddleocr_vl_vision
paddleocr_vl_vision
✓ kept
vision_config
hidden_act
gelu_pytorch_tanh
gelu_pytorch_tanh
✓ kept
vision_config
image_size
384
384
✓ kept
vision_config
patch_size
14
14
✓ kept
vision_config
num_channels
3
3
✓ kept
Module classes
Instances of each class in the built model (prefix PaddleOCRVL omitted).
+
\ No newline at end of file
diff --git a/shrink/timesfm-2.0-500m-pytorch-mic.html b/shrink/timesfm-2.0-500m-pytorch-mic.html
new file mode 100644
index 0000000000000000000000000000000000000000..dd140682780b8e7663f45473f26e8af07b6bc92a
--- /dev/null
+++ b/shrink/timesfm-2.0-500m-pytorch-mic.html
@@ -0,0 +1,34 @@
+
+
+Shrunk architecture
+
+
Shrunk architecture
huggingface-mic/timesfm-2.0-500m-pytorch-mic@d9e2f9e3ea410197d1b0f1ef346cd0f1f33f693b ← google/timesfm-2.0-500m-pytorch@dc2443792ce5516872b89b37cf1bc058c3bf0c10 · TimesFmModelForPrediction · random weights · built before the shrink rule (no recorded recipe)
+
Original498.8Mparameters
Fixture344.4Kparameters
Layer stacks1compared
Findings0to review
✓ Same architecture, smaller
Every module class, layer kind and checked property of the original is in the fixture.
Layer stacks
One square per layer, colored by kind: the layer's module classes and type attributes. Hover a square for its index and kind.
decoder.layers
Original · 50
Fixture · 2
Kind
Original
Fixture
DecoderLayer
50
2
✓ kept
Where the parameters are
Share of each model's parameters per component. Vocabulary and patch sizes are kept, so embeddings usually take a larger share of the fixture.
Original
Fixture
Component
Original
Share
Fixture
Share
decoder.layers
492.1M
98.7%
66.8K
19.4%
decoder.input_ff_layer
1.8M
0.4%
20.7K
6.0%
horizon_ff_layer.residual_layer
1.6M
0.3%
83.2K
24.2%
horizon_ff_layer.input_layer
1.6M
0.3%
8.3K
2.4%
horizon_ff_layer.output_layer
1.6M
0.3%
165.1K
47.9%
decoder.freq_emb
3.8K
0.0%
192
0.1%
Preserved properties
Config
Property
Original
Fixture
model
model_type
timesfm
timesfm
✓ kept
Module classes
Instances of each class in the built model (prefix TimesFm omitted).
Class
Original
Fixture
Linear
306
18
LayerNorm
50
2
Attention
50
2
DecoderLayer
50
2
MLP
50
2
RMSNorm
50
2
SiLU
2
2
ResidualBlock
2
2
Embedding
1
1
ModuleList
1
1
Model
1
1
ModelForPrediction
1
1
Config changes
5 fields changed, 22 unchanged.
Field
Original
Fixture
head_dim
80
16
hidden_size
1280
64
intermediate_size
1280
128
num_attention_heads
16
4
num_hidden_layers
50
2
+
\ No newline at end of file
diff --git a/shrink/tiny-clip-text-2-mic.html b/shrink/tiny-clip-text-2-mic.html
new file mode 100644
index 0000000000000000000000000000000000000000..4dd5c459b903984e31080bb93cf0ed640ce6664a
--- /dev/null
+++ b/shrink/tiny-clip-text-2-mic.html
@@ -0,0 +1,34 @@
+
+
+Shrunk architecture
+
+
Shrunk architecture
huggingface-mic/tiny-clip-text-2-mic@c9fce363d5426f2b2bad7bd285c55932c0055daa ← peft-internal-testing/tiny-clip-text-2@4fbb8d27f0fdda658bc80bbccbd3d749044ace78 · CLIPTextModelWithProjection · random weights
+
Original69.5Kparameters
Fixture138.0Kparameters
Layer stacks1compared
Findings0to review
✓ Same architecture, smaller
Every module class, layer kind and checked property of the original is in the fixture.
Layer stacks
One square per layer, colored by kind: the layer's module classes and type attributes. Hover a square for its index and kind.
text_model.encoder.layers
Original · 5
Fixture · 2
Kind
Original
Fixture
CLIPEncoderLayer
5
2
✓ kept
Where the parameters are
Share of each model's parameters per component. Vocabulary and patch sizes are kept, so embeddings usually take a larger share of the fixture.
Original
Fixture
Component
Original
Share
Fixture
Share
text_model.embeddings
34.5K
49.6%
68.9K
49.9%
text_model.encoder.layers
33.9K
48.8%
66.9K
48.5%
text_projection
1.0K
1.5%
2.0K
1.5%
text_model.final_layer_norm
64
0.1%
128
0.1%
Preserved properties
Config
Property
Original
Fixture
model
model_type
clip_text_model
clip_text_model
✓ kept
model
vocab_size
1000
1000
✓ kept
model
hidden_act
gelu
gelu
✓ kept
model
max_position_embeddings
77
77
✓ kept
Module classes
Instances of each class in the built model (prefix CLIPText omitted).
Class
Original
Fixture
Linear
31
13
LayerNorm
11
5
CLIPAttention
5
2
CLIPEncoderLayer
5
2
CLIPMLP
5
2
GELUActivation
5
2
Embedding
2
2
CLIPEncoder
1
1
Embeddings
1
1
Model
1
1
ModelWithProjection
1
1
ModuleList
1
1
Config changes
3 fields changed, 21 unchanged.
Field
Original
Fixture
hidden_size
32
64
intermediate_size
37
128
num_hidden_layers
5
2
+
\ No newline at end of file
diff --git a/shrink/tiny-random-aya-base-mic.html b/shrink/tiny-random-aya-base-mic.html
new file mode 100644
index 0000000000000000000000000000000000000000..c3a43d891cf5b6539e161f1b1fecb84f48a4a5f6
--- /dev/null
+++ b/shrink/tiny-random-aya-base-mic.html
@@ -0,0 +1,34 @@
+
+
+Shrunk architecture
+
+
Shrunk architecture
huggingface-mic/tiny-random-aya-base-mic@42e97ac577e1e82f7cac9918e3fc87f3c6585b98 ← optimum-intel-internal-testing/tiny-random-aya-base@e85478d37f912454b23ff2a9c7af16a960837644 · Cohere2ForCausalLM · random weights
+
Original1.1Mparameters
Fixture16.9Mparameters
Layer stacks1compared
Findings0to review
✓ Same architecture, smaller
Every module class, layer kind and checked property of the original is in the fixture.
Layer stacks
One square per layer, colored by kind: the layer's module classes and type attributes. Hover a square for its index and kind.
model.layers
Original · 2
Fixture · 2
Kind
Original
Fixture
DecoderLayer
2
2
✓ kept
Where the parameters are
Share of each model's parameters per component. Vocabulary and patch sizes are kept, so embeddings usually take a larger share of the fixture.
Original
Fixture
Component
Original
Share
Fixture
Share
model.embed_tokens
1.0M
98.4%
16.8M
99.5%
model.layers
16.5K
1.6%
82.0K
0.5%
model.norm
4
0.0%
64
0.0%
Preserved properties
Config
Property
Original
Fixture
model
attention style
MHA (4 heads, 4 KV)
MHA (4 heads, 4 KV)
✓ kept
model
RoPE type
default
default
✓ kept
model
model_type
cohere2
cohere2
✓ kept
model
vocab_size
262144
262144
✓ kept
model
tie_word_embeddings
True
True
✓ kept
model
hidden_act
silu
silu
✓ kept
model
max_position_embeddings
500000
500000
✓ kept
Module classes
Instances of each class in the built model (prefix Cohere2 omitted).
Class
Original
Fixture
Linear
15
15
LayerNorm
3
3
Attention
2
2
DecoderLayer
2
2
MLP
2
2
SiLUActivation
2
2
ForCausalLM
1
1
Model
1
1
RotaryEmbedding
1
1
Embedding
1
1
ModuleList
1
1
Config changes
3 fields changed, 40 unchanged.
Field
Original
Fixture
head_dim
128
16
hidden_size
4
64
intermediate_size
5
128
+
\ No newline at end of file
diff --git a/shrink/tiny-random-muse-glimmer-mic.html b/shrink/tiny-random-muse-glimmer-mic.html
new file mode 100644
index 0000000000000000000000000000000000000000..e02972a65c3090ba249918bdca94fcd089b28ca0
--- /dev/null
+++ b/shrink/tiny-random-muse-glimmer-mic.html
@@ -0,0 +1,34 @@
+
+
+Shrunk architecture
+
+
Shrunk architecture
huggingface-mic/tiny-random-muse-glimmer-mic@5d21eb327b8963e4a0e32c69882b7593c211b0e5 ← optimum-intel-internal-testing/tiny-random-muse-glimmer@cb70af191bbbf704a47b7d24dd5287d97dd60103 · MuseGlimmerForConditionalGeneration · random weights
+
Original13.5Mparameters
Fixture13.3Mparameters
Layer stacks2compared
Findings0to review
✓ Same architecture, smaller
Every module class, layer kind and checked property of the original is in the fixture.
Layer stacks
One square per layer, colored by kind: the layer's module classes and type attributes. Hover a square for its index and kind.
model.vision_tower.layers
Original · 4
Fixture · 2
Kind
Original
Fixture
VisionEncoderLayer
4
2
✓ kept
model.language_model.layers
Original · 4
Fixture · 2
Kind
Original
Fixture
TextDecoderLayer
4
2
✓ kept
Where the parameters are
Share of each model's parameters per component. Vocabulary and patch sizes are kept, so embeddings usually take a larger share of the fixture.
Original
Fixture
Component
Original
Share
Fixture
Share
model.language_model
12.9M
95.6%
12.9M
97.4%
model.language_model.layers
263.2K
1.9%
82.4K
0.6%
model.vision_tower
141.1K
1.0%
141.1K
1.1%
model.vision_tower.layers
133.9K
1.0%
66.9K
0.5%
model.vision_adapter
49.2K
0.4%
49.2K
0.4%
model.vision_projection
8.2K
0.1%
8.2K
0.1%
Preserved properties
Config
Property
Original
Fixture
model
model_type
muse_glimmer
muse_glimmer
✓ kept
model
tie_word_embeddings
True
True
✓ kept
text_config
attention style
GQA (4 heads, 2 KV)
GQA (4 heads, 2 KV)
✓ kept
text_config
RoPE type
default
default
✓ kept
text_config
model_type
muse_glimmer_text
muse_glimmer_text
✓ kept
text_config
vocab_size
202048
202048
✓ kept
text_config
tie_word_embeddings
True
True
✓ kept
text_config
max_position_embeddings
131072
131072
✓ kept
vision_config
RoPE type
axial
axial
✓ kept
vision_config
model_type
muse_glimmer_vision
muse_glimmer_vision
✓ kept
vision_config
hidden_act
gelu
gelu
✓ kept
vision_config
max_position_embeddings
1024
1024
✓ kept
vision_config
patch_size
14
14
✓ kept
Module classes
Instances of each class in the built model (prefix MuseGlimmer omitted).
+
\ No newline at end of file
diff --git a/shrink/tiny_starcoder_py-mic.html b/shrink/tiny_starcoder_py-mic.html
new file mode 100644
index 0000000000000000000000000000000000000000..6ab1c9bd6dadd8bf6520af566ab52cbb0d6e6fb6
--- /dev/null
+++ b/shrink/tiny_starcoder_py-mic.html
@@ -0,0 +1,34 @@
+
+
+Shrunk architecture
+
+
Shrunk architecture
huggingface-mic/tiny_starcoder_py-mic@0ad90af0e2e20c84f2462ee730f66722e2cec933 ← bigcode/tiny_starcoder_py@8547527bef0bc927268c1653cce6948c5c242dd1 · GPTBigCodeForCausalLM · random weights
+
Original164.1Mparameters
Fixture3.7Mparameters
Layer stacks1compared
Findings0to review
✓ Same architecture, smaller
Every module class, layer kind and checked property of the original is in the fixture.
Layer stacks
One square per layer, colored by kind: the layer's module classes and type attributes. Hover a square for its index and kind.
transformer.h
Original · 20
Fixture · 2
Kind
Original
Fixture
Block
20
2
✓ kept
Where the parameters are
Share of each model's parameters per component. Vocabulary and patch sizes are kept, so embeddings usually take a larger share of the fixture.
Original
Fixture
Component
Original
Share
Fixture
Share
transformer.h
120.1M
73.2%
54.5K
1.5%
transformer.wte
37.7M
23.0%
3.1M
84.5%
transformer.wpe
6.3M
3.8%
524.3K
14.1%
transformer.ln_f
1.5K
0.0%
128
0.0%
Preserved properties
Config
Property
Original
Fixture
model
model_type
gpt_bigcode
gpt_bigcode
✓ kept
model
vocab_size
49152
49152
✓ kept
model
tie_word_embeddings
True
True
✓ kept
Module classes
Instances of each class in the built model (prefix GPTBigCode omitted).
Class
Original
Fixture
Linear
81
9
Dropout
41
5
LayerNorm
41
5
GELUTanh
20
2
Attention
20
2
Block
20
2
MLP
20
2
Embedding
2
2
ForCausalLM
1
1
Model
1
1
ModuleList
1
1
Config changes
5 fields changed, 38 unchanged.
Field
Original
Fixture
n_embd
768
64
n_head
12
4
n_inner
3072
128
n_layer
20
2
num_key_value_heads
1
2
+
\ No newline at end of file
diff --git a/shrink/tipsv2-b14-mic.html b/shrink/tipsv2-b14-mic.html
new file mode 100644
index 0000000000000000000000000000000000000000..f3a3da22d0bbe4c9f9c713ac57b6414e3e32c085
--- /dev/null
+++ b/shrink/tipsv2-b14-mic.html
@@ -0,0 +1,34 @@
+
+
+Shrunk architecture
+
+
Shrunk architecture
huggingface-mic/tipsv2-b14-mic@1ba3c8e34558ed9711f0a5dcb5cfc965bca5f380 ← google/tipsv2-b14@ed1e4dc6b74bf3935ae099e9d5eb30fa96528454 · Tipsv2Model · random weights
+
Original195.9Mparameters
Fixture2.3Mparameters
Layer stacks2compared
Findings0to review
✓ Same architecture, smaller
Every module class, layer kind and checked property of the original is in the fixture.
Layer stacks
One square per layer, colored by kind: the layer's module classes and type attributes. Hover a square for its index and kind.
text_model.encoder.layers
Original · 12
Fixture · 2
Kind
Original
Fixture
TextEncoderLayer
12
2
✓ kept
vision_model.encoder.layer
Original · 12
Fixture · 2
Kind
Original
Fixture
VisionLayer
12
2
✓ kept
Where the parameters are
Share of each model's parameters per component. Vocabulary and patch sizes are kept, so embeddings usually take a larger share of the fixture.
Original
Fixture
Component
Original
Share
Fixture
Share
vision_model.encoder.layer
85.1M
43.4%
100.2K
4.3%
text_model.encoder.layers
85.1M
43.4%
66.9K
2.9%
text_model.embeddings
24.6M
12.5%
2.0M
88.3%
vision_model.embeddings
1.2M
0.6%
103.5K
4.5%
vision_model.layernorm
1.5K
0.0%
128
0.0%
text_model.final_layer_norm
1.5K
0.0%
128
0.0%
temperature
1
0.0%
1
0.0%
Preserved properties
Config
Property
Original
Fixture
model
model_type
tipsv2
tipsv2
✓ kept
text_config
model_type
tipsv2_text_model
tipsv2_text_model
✓ kept
text_config
vocab_size
32000
32000
✓ kept
text_config
hidden_act
relu
relu
✓ kept
text_config
max_position_embeddings
64
64
✓ kept
vision_config
model_type
tipsv2_vision_model
tipsv2_vision_model
✓ kept
vision_config
hidden_act
gelu
gelu
✓ kept
vision_config
image_size
448
448
✓ kept
vision_config
patch_size
14
14
✓ kept
vision_config
num_channels
3
3
✓ kept
Module classes
Instances of each class in the built model (prefix Tipsv2 omitted).
Class
Original
Fixture
Linear
144
24
LayerNorm
50
10
VisionLayerScale
24
4
GELUActivation
12
2
Identity
12
2
ReLU
12
2
MLP
12
2
TextAttention
12
2
TextEncoderLayer
12
2
VisionAttention
12
2
VisionLayer
12
2
VisionMLP
12
2
ModuleList
2
2
Conv2d
1
1
Dropout
1
1
Embedding
1
1
Model
1
1
SinusoidalPositionalEmbedding
1
1
TextEmbeddings
1
1
TextEncoder
1
1
TextModel
1
1
VisionEmbeddings
1
1
VisionEncoder
1
1
VisionModel
1
1
VisionPatchEmbeddings
1
1
Config changes
9 fields changed, 59 unchanged.
Field
Original
Fixture
text_config.hidden_size
768
64
text_config.intermediate_size
3072
128
text_config.num_attention_heads
12
4
text_config.num_hidden_layers
12
2
vision_config.hidden_size
768
64
vision_config.num_attention_heads
12
4
vision_config.num_hidden_layers
12
2
vision_config.out_features
["stage12"]
["stage2"]
vision_config.out_indices
[12]
[2]
+
\ No newline at end of file
diff --git a/shrink/umt5-small-mic.html b/shrink/umt5-small-mic.html
new file mode 100644
index 0000000000000000000000000000000000000000..d2275de33f680a5d9cea13bc2a8271252a850f9d
--- /dev/null
+++ b/shrink/umt5-small-mic.html
@@ -0,0 +1,34 @@
+
+
+Shrunk architecture
+
+
Shrunk architecture
huggingface-mic/umt5-small-mic@3ea88ecd4caca76da9aeec8609c47d076387ddbb ← google/umt5-small@8c63c2b77efbf8e41206a2c8d994846cc9392360 · UMT5ForConditionalGeneration · random weights · built before the shrink rule (no recorded recipe)
+
Original175.3Mparameters
Fixture16.6Mparameters
Layer stacks2compared
Findings0to review
✓ Same architecture, smaller
Every module class, layer kind and checked property of the original is in the fixture.
Layer stacks
One square per layer, colored by kind: the layer's module classes and type attributes. Hover a square for its index and kind.
encoder.block
Original · 8
Fixture · 2
Kind
Original
Fixture
Block
8
2
✓ kept
decoder.block
Original · 8
Fixture · 2
Kind
Original
Fixture
Block
8
2
✓ kept
Where the parameters are
Share of each model's parameters per component. Vocabulary and patch sizes are kept, so embeddings usually take a larger share of the fixture.
Original
Fixture
Component
Original
Share
Fixture
Share
shared
131.3M
74.9%
16.4M
98.8%
decoder.block
25.2M
14.4%
115.3K
0.7%
encoder.block
18.9M
10.8%
82.4K
0.5%
encoder.final_layer_norm
512
0.0%
64
0.0%
decoder.final_layer_norm
512
0.0%
64
0.0%
Preserved properties
Config
Property
Original
Fixture
model
model_type
umt5
umt5
✓ kept
model
vocab_size
256384
256384
✓ kept
model
tie_word_embeddings
True
True
✓ kept
Module classes
Instances of each class in the built model (prefix UMT5 omitted).
Class
Original
Fixture
Linear
145
37
Dropout
58
16
LayerNorm
42
12
Attention
24
6
Embedding
19
7
ModuleList
18
6
NewGELUActivation
16
4
Block
16
4
DenseGatedActDense
16
4
LayerFF
16
4
LayerSelfAttention
16
4
LayerCrossAttention
8
2
Stack
2
2
ForConditionalGeneration
1
1
Config changes
6 fields changed, 28 unchanged.
Field
Original
Fixture
d_ff
1024
128
d_kv
64
16
d_model
512
64
num_decoder_layers
8
2
num_heads
6
4
num_layers
8
2
+
\ No newline at end of file
diff --git a/shrink/vit-mic.html b/shrink/vit-mic.html
new file mode 100644
index 0000000000000000000000000000000000000000..3b8ad0f24ea49763c1800374cee54b3d3ee90e11
--- /dev/null
+++ b/shrink/vit-mic.html
@@ -0,0 +1,34 @@
+
+
+Shrunk architecture
+
+
Shrunk architecture
huggingface-mic/vit-mic@d69cbc45edf98915b1e1af08807186c22b3152f0 ← google/vit-base-patch16-224@3f49326eb077187dfe1c2a2bb15fbd74e6ab91e3 · ViTForImageClassification · overfit weights · built before the shrink rule (no recorded recipe)
+
Original86.6Mparameters
Fixture194.0Kparameters
Layer stacks1compared
Findings0to review
✓ Same architecture, smaller
Every module class, layer kind and checked property of the original is in the fixture.
Layer stacks
One square per layer, colored by kind: the layer's module classes and type attributes. Hover a square for its index and kind.
vit.layers
Original · 12
Fixture · 2
Kind
Original
Fixture
Layer
12
2
✓ kept
Where the parameters are
Share of each model's parameters per component. Vocabulary and patch sizes are kept, so embeddings usually take a larger share of the fixture.
Original
Fixture
Component
Original
Share
Fixture
Share
vit.layers
85.1M
98.3%
66.9K
34.5%
classifier
769.0K
0.9%
65.0K
33.5%
vit.embeddings
742.7K
0.9%
61.9K
31.9%
vit.layernorm
1.5K
0.0%
128
0.1%
Preserved properties
Config
Property
Original
Fixture
model
model_type
vit
vit
✓ kept
model
hidden_act
gelu
gelu
✓ kept
model
image_size
224
224
✓ kept
model
patch_size
16
16
✓ kept
model
num_channels
3
3
✓ kept
Module classes
Instances of each class in the built model (prefix ViT omitted).
Class
Original
Fixture
Linear
73
13
LayerNorm
25
5
Dropout
13
3
GELUActivation
12
2
Attention
12
2
Layer
12
2
MLP
12
2
Conv2d
1
1
ModuleList
1
1
Embeddings
1
1
ForImageClassification
1
1
Model
1
1
PatchEmbeddings
1
1
Config changes
5 fields changed, 20 unchanged.
Field
Original
Fixture
hidden_size
768
64
intermediate_size
3072
128
num_attention_heads
12
4
num_hidden_layers
12
2
problem_type
null
single_label_classification
+
\ No newline at end of file
diff --git a/shrink/vitmatte-small-composition-1k-mic.html b/shrink/vitmatte-small-composition-1k-mic.html
new file mode 100644
index 0000000000000000000000000000000000000000..86e9b5204dbc65a254ea0c095a521a2d0d1ef16a
--- /dev/null
+++ b/shrink/vitmatte-small-composition-1k-mic.html
@@ -0,0 +1,34 @@
+
+
+Shrunk architecture
+
+
Every module class, layer kind and checked property of the original is in the fixture.
Layer stacks
One square per layer, colored by kind: the layer's module classes and type attributes. Hover a square for its index and kind.
backbone.encoder.layer
Original · 12
Fixture · 3
Kind
Original
Fixture
no Conv2d
8
2
✓ kept
Conv2d, VitDetLayerNorm, VitDetResBottleneckBlock
4
1
✓ kept
decoder.convstream.convs
Original · 3
Fixture · 3
Kind
Original
Fixture
ReLU
3
3
✓ kept
decoder.fusion_blocks
Original · 4
Fixture · 4
Kind
Original
Fixture
FusionBlock
4
4
✓ kept
Where the parameters are
Share of each model's parameters per component. Vocabulary and patch sizes are kept, so embeddings usually take a larger share of the fixture.
Original
Fixture
Component
Original
Share
Fixture
Share
backbone.encoder.layer
23.3M
90.2%
167.3K
10.6%
decoder.fusion_blocks
1.9M
7.2%
1.1M
70.8%
backbone.embeddings
469.2K
1.8%
78.2K
5.0%
decoder.convstream.convs
209.8K
0.8%
209.8K
13.3%
decoder.matting_head
4.7K
0.0%
4.7K
0.3%
Preserved properties
Config
Property
Original
Fixture
model
model_type
vitmatte
vitmatte
✓ kept
backbone_config
model_type
vitdet
vitdet
✓ kept
backbone_config
hidden_act
gelu
gelu
✓ kept
backbone_config
image_size
512
512
✓ kept
backbone_config
patch_size
16
16
✓ kept
backbone_config
num_channels
4
4
✓ kept
Module classes
Instances of each class in the built model (prefix VitMatte omitted).
Class
Original
Fixture
Linear
48
12
LayerNorm
24
6
Conv2d
22
13
GELUActivation
20
5
Dropout
12
3
Identity
12
3
VitDetAttention
12
3
VitDetLayer
12
3
VitDetLayerNorm
12
3
VitDetMlp
12
3
BatchNorm2d
8
8
ReLU
8
8
BasicConv3x3
7
7
VitDetResBottleneckBlock
4
1
FusionBlock
4
4
ModuleList
3
3
Sequential
1
1
VitDetBackbone
1
1
VitDetEmbeddings
1
1
VitDetEncoder
1
1
ConvStream
1
1
DetailCaptureModule
1
1
ForImageMatting
1
1
Head
1
1
Config changes
6 fields changed, 39 unchanged.
Field
Original
Fixture
backbone_config.hidden_size
384
64
backbone_config.num_attention_heads
6
4
backbone_config.num_hidden_layers
12
3
backbone_config.out_features
["stage12"]
["stage3"]
backbone_config.out_indices
[12]
[3]
hidden_size
384
64
+
\ No newline at end of file
diff --git a/shrink/wav2vec2-base-960h-mic.html b/shrink/wav2vec2-base-960h-mic.html
new file mode 100644
index 0000000000000000000000000000000000000000..c25af61d71573745e4267eddb2f337878ef562bf
--- /dev/null
+++ b/shrink/wav2vec2-base-960h-mic.html
@@ -0,0 +1,34 @@
+
+
+Shrunk architecture
+
+
Shrunk architecture
huggingface-mic/wav2vec2-base-960h-mic@889c4c48f746571cbbdb9235e9746c3142ee0550 ← facebook/wav2vec2-base-960h@22aad52d435eb6dbaf354bdad9b0da84ce7d6156 · Wav2Vec2ForCTC · random weights · built before the shrink rule (no recorded recipe)
+
Original94.4Mparameters
Fixture4.3Mparameters
Layer stacks2compared
Findings0to review
✓ Same architecture, smaller
Every module class, layer kind and checked property of the original is in the fixture.
Layer stacks
One square per layer, colored by kind: the layer's module classes and type attributes. Hover a square for its index and kind.
wav2vec2.feature_extractor.conv_layers
Original · 7
Fixture · 7
Kind
Original
Fixture
GroupNorm, GroupNormConvLayer
1
1
✓ kept
NoLayerNormConvLayer
6
6
✓ kept
wav2vec2.encoder.layers
Original · 12
Fixture · 2
Kind
Original
Fixture
EncoderLayer
12
2
✓ kept
Where the parameters are
Share of each model's parameters per component. Vocabulary and patch sizes are kept, so embeddings usually take a larger share of the fixture.
Original
Fixture
Component
Original
Share
Fixture
Share
wav2vec2.encoder.layers
85.1M
90.1%
66.9K
1.5%
wav2vec2.encoder
4.7M
5.0%
33.1K
0.8%
wav2vec2.feature_extractor.conv_layers
4.2M
4.4%
4.2M
96.9%
wav2vec2.feature_projection
395.0K
0.4%
33.9K
0.8%
lm_head
24.6K
0.0%
2.1K
0.0%
wav2vec2
768
0.0%
64
0.0%
Preserved properties
Config
Property
Original
Fixture
model
model_type
wav2vec2
wav2vec2
✓ kept
model
vocab_size
32
32
✓ kept
model
hidden_act
gelu
gelu
✓ kept
Module classes
Instances of each class in the built model (prefix Wav2Vec2 omitted).
Class
Original
Fixture
Linear
74
14
Dropout
39
9
LayerNorm
26
6
GELUActivation
20
10
Attention
12
2
EncoderLayer
12
2
FeedForward
12
2
Conv1d
7
7
NoLayerNormConvLayer
6
6
ModuleList
2
2
GroupNorm
1
1
ModuleDict
1
1
ParametrizationList
1
1
ParametrizedConv1d
1
1
Encoder
1
1
FeatureEncoder
1
1
FeatureProjection
1
1
ForCTC
1
1
GroupNormConvLayer
1
1
Model
1
1
PositionalConvEmbedding
1
1
SamePadLayer
1
1
_WeightNorm
1
1
Config changes
4 fields changed, 64 unchanged.
Field
Original
Fixture
hidden_size
768
64
intermediate_size
3072
128
num_attention_heads
12
4
num_hidden_layers
12
2
+
\ No newline at end of file
diff --git a/shrink/wav2vec2-mic.html b/shrink/wav2vec2-mic.html
new file mode 100644
index 0000000000000000000000000000000000000000..9015b8421b116721e6e39d96b52c2f41913fc965
--- /dev/null
+++ b/shrink/wav2vec2-mic.html
@@ -0,0 +1,34 @@
+
+
+Shrunk architecture
+
+
Shrunk architecture
huggingface-mic/wav2vec2-mic@4cce7c54a33fa531f4800eb4dba71b23dda960a7 ← facebook/wav2vec2-base-960h@22aad52d435eb6dbaf354bdad9b0da84ce7d6156 · Wav2Vec2ForCTC · random weights · built before the shrink rule (no recorded recipe)
+
Original94.4Mparameters
Fixture4.3Mparameters
Layer stacks2compared
Findings0to review
✓ Same architecture, smaller
Every module class, layer kind and checked property of the original is in the fixture.
Layer stacks
One square per layer, colored by kind: the layer's module classes and type attributes. Hover a square for its index and kind.
wav2vec2.feature_extractor.conv_layers
Original · 7
Fixture · 7
Kind
Original
Fixture
GroupNorm, GroupNormConvLayer
1
1
✓ kept
NoLayerNormConvLayer
6
6
✓ kept
wav2vec2.encoder.layers
Original · 12
Fixture · 2
Kind
Original
Fixture
EncoderLayer
12
2
✓ kept
Where the parameters are
Share of each model's parameters per component. Vocabulary and patch sizes are kept, so embeddings usually take a larger share of the fixture.
Original
Fixture
Component
Original
Share
Fixture
Share
wav2vec2.encoder.layers
85.1M
90.1%
66.9K
1.5%
wav2vec2.encoder
4.7M
5.0%
33.1K
0.8%
wav2vec2.feature_extractor.conv_layers
4.2M
4.4%
4.2M
96.9%
wav2vec2.feature_projection
395.0K
0.4%
33.9K
0.8%
lm_head
24.6K
0.0%
2.1K
0.0%
wav2vec2
768
0.0%
64
0.0%
Preserved properties
Config
Property
Original
Fixture
model
model_type
wav2vec2
wav2vec2
✓ kept
model
vocab_size
32
32
✓ kept
model
hidden_act
gelu
gelu
✓ kept
Module classes
Instances of each class in the built model (prefix Wav2Vec2 omitted).
Class
Original
Fixture
Linear
74
14
Dropout
39
9
LayerNorm
26
6
GELUActivation
20
10
Attention
12
2
EncoderLayer
12
2
FeedForward
12
2
Conv1d
7
7
NoLayerNormConvLayer
6
6
ModuleList
2
2
GroupNorm
1
1
ModuleDict
1
1
ParametrizationList
1
1
ParametrizedConv1d
1
1
Encoder
1
1
FeatureEncoder
1
1
FeatureProjection
1
1
ForCTC
1
1
GroupNormConvLayer
1
1
Model
1
1
PositionalConvEmbedding
1
1
SamePadLayer
1
1
_WeightNorm
1
1
Config changes
4 fields changed, 64 unchanged.
Field
Original
Fixture
hidden_size
768
64
intermediate_size
3072
128
num_attention_heads
12
4
num_hidden_layers
12
2
+
\ No newline at end of file
diff --git a/shrink/wavlm-base-plus-mic.html b/shrink/wavlm-base-plus-mic.html
new file mode 100644
index 0000000000000000000000000000000000000000..8ece28cb7370b08f9f2358a6810984022a0cbe0b
--- /dev/null
+++ b/shrink/wavlm-base-plus-mic.html
@@ -0,0 +1,34 @@
+
+
+Shrunk architecture
+
+
Shrunk architecture
huggingface-mic/wavlm-base-plus-mic@55205db7dc5cdb00809d44f273506a349cc1d861 ← microsoft/wavlm-base-plus@4c66d4806a428f2e922ccfa1a962776e232d487b · WavLMModel · random weights
+
Original94.4Mparameters
Fixture4.3Mparameters
Layer stacks2compared
Findings0to review
✓ Same architecture, smaller
Every module class, layer kind and checked property of the original is in the fixture.
Layer stacks
One square per layer, colored by kind: the layer's module classes and type attributes. Hover a square for its index and kind.
feature_extractor.conv_layers
Original · 7
Fixture · 7
Kind
Original
Fixture
GroupNorm, GroupNormConvLayer
1
1
✓ kept
NoLayerNormConvLayer
6
6
✓ kept
encoder.layers
Original · 12
Fixture · 2
Kind
Original
Fixture
Embedding
1
1
✓ kept
no Embedding
11
1
✓ kept
Where the parameters are
Share of each model's parameters per component. Vocabulary and patch sizes are kept, so embeddings usually take a larger share of the fixture.
Original
Fixture
Component
Original
Share
Fixture
Share
encoder.layers
85.1M
90.1%
68.5K
1.6%
encoder.pos_conv_embed
4.7M
5.0%
33.0K
0.8%
feature_extractor.conv_layers
4.2M
4.5%
4.2M
96.9%
feature_projection.projection
394.0K
0.4%
32.8K
0.8%
encoder.layer_norm
1.5K
0.0%
128
0.0%
feature_projection.layer_norm
1.0K
0.0%
1.0K
0.0%
masked_spec_embed
768
0.0%
64
0.0%
Preserved properties
Config
Property
Original
Fixture
model
model_type
wavlm
wavlm
✓ kept
model
vocab_size
32
32
✓ kept
model
hidden_act
gelu
gelu
✓ kept
Module classes
Instances of each class in the built model (prefix WavLM omitted).
Class
Original
Fixture
Linear
85
15
Dropout
38
8
LayerNorm
26
6
GELUActivation
20
10
Attention
12
2
EncoderLayer
12
2
FeedForward
12
2
Conv1d
7
7
NoLayerNormConvLayer
6
6
ModuleList
2
2
Embedding
1
1
GroupNorm
1
1
ModuleDict
1
1
ParametrizationList
1
1
ParametrizedConv1d
1
1
Encoder
1
1
FeatureEncoder
1
1
FeatureProjection
1
1
GroupNormConvLayer
1
1
Model
1
1
PositionalConvEmbedding
1
1
SamePadLayer
1
1
_WeightNorm
1
1
Config changes
4 fields changed, 76 unchanged.
Field
Original
Fixture
hidden_size
768
64
intermediate_size
3072
128
num_attention_heads
12
4
num_hidden_layers
12
2
+
\ No newline at end of file
diff --git a/shrink/whisper-mic.html b/shrink/whisper-mic.html
new file mode 100644
index 0000000000000000000000000000000000000000..f747dd984e036c1195bcfae348f93629b88bf677
--- /dev/null
+++ b/shrink/whisper-mic.html
@@ -0,0 +1,34 @@
+
+
+Shrunk architecture
+
+
Shrunk architecture
huggingface-mic/whisper-mic@1f40ed2753f5ba347b5bd1ba3624c971b9327ab8 ← openai/whisper-tiny@169d4a4341b33bc18d8881c4b69c2e104e1cc0af · WhisperForConditionalGeneration · overfit weights · built before the shrink rule (no recorded recipe)
+
Original37.8Mparameters
Fixture3.6Mparameters
Layer stacks2compared
Findings0to review
✓ Same architecture, smaller
Every module class, layer kind and checked property of the original is in the fixture.
Layer stacks
One square per layer, colored by kind: the layer's module classes and type attributes. Hover a square for its index and kind.
model.encoder.layers
Original · 4
Fixture · 2
Kind
Original
Fixture
EncoderLayer
4
2
✓ kept
model.decoder.layers
Original · 4
Fixture · 2
Kind
Original
Fixture
DecoderLayer
4
2
✓ kept
Where the parameters are
Share of each model's parameters per component. Vocabulary and patch sizes are kept, so embeddings usually take a larger share of the fixture.
Original
Fixture
Component
Original
Share
Fixture
Share
model.decoder
20.1M
53.2%
3.3M
92.0%
model.decoder.layers
9.5M
25.1%
100.2K
2.8%
model.encoder.layers
7.1M
18.8%
66.8K
1.8%
model.encoder
1.1M
2.9%
123.9K
3.4%
Preserved properties
Config
Property
Original
Fixture
model
model_type
whisper
whisper
✓ kept
model
vocab_size
51865
51865
✓ kept
model
tie_word_embeddings
True
True
✓ kept
model
num_mel_bins
80
80
✓ kept
Module classes
Instances of each class in the built model (prefix Whisper omitted).
Class
Original
Fixture
Linear
65
33
LayerNorm
22
12
Attention
12
6
GELUActivation
8
4
DecoderLayer
4
2
EncoderLayer
4
2
Conv1d
2
2
Embedding
2
2
ModuleList
2
2
Decoder
1
1
Encoder
1
1
ForConditionalGeneration
1
1
Model
1
1
PositionalEmbedding
1
1
Config changes
7 fields changed, 40 unchanged.
Field
Original
Fixture
d_model
384
64
decoder_attention_heads
6
4
decoder_ffn_dim
1536
128
decoder_layers
4
2
encoder_attention_heads
6
4
encoder_ffn_dim
1536
128
encoder_layers
4
2
+
\ No newline at end of file
diff --git a/shrink/whisper-tiny-mic.html b/shrink/whisper-tiny-mic.html
new file mode 100644
index 0000000000000000000000000000000000000000..b638b3e4844e75b4ee1ec4e60d7e7e142ed46b5a
--- /dev/null
+++ b/shrink/whisper-tiny-mic.html
@@ -0,0 +1,34 @@
+
+
+Shrunk architecture
+
+
Shrunk architecture
huggingface-mic/whisper-tiny-mic@9d85803a8fb0226794764f4b12a4d4fd74d160ed ← openai/whisper-tiny@169d4a4341b33bc18d8881c4b69c2e104e1cc0af · WhisperForConditionalGeneration · overfit weights · built before the shrink rule (no recorded recipe)
+
Original37.8Mparameters
Fixture3.6Mparameters
Layer stacks2compared
Findings0to review
✓ Same architecture, smaller
Every module class, layer kind and checked property of the original is in the fixture.
Layer stacks
One square per layer, colored by kind: the layer's module classes and type attributes. Hover a square for its index and kind.
model.encoder.layers
Original · 4
Fixture · 2
Kind
Original
Fixture
EncoderLayer
4
2
✓ kept
model.decoder.layers
Original · 4
Fixture · 2
Kind
Original
Fixture
DecoderLayer
4
2
✓ kept
Where the parameters are
Share of each model's parameters per component. Vocabulary and patch sizes are kept, so embeddings usually take a larger share of the fixture.
Original
Fixture
Component
Original
Share
Fixture
Share
model.decoder
20.1M
53.2%
3.3M
92.0%
model.decoder.layers
9.5M
25.1%
100.2K
2.8%
model.encoder.layers
7.1M
18.8%
66.8K
1.8%
model.encoder
1.1M
2.9%
123.9K
3.4%
Preserved properties
Config
Property
Original
Fixture
model
model_type
whisper
whisper
✓ kept
model
vocab_size
51865
51865
✓ kept
model
tie_word_embeddings
True
True
✓ kept
model
num_mel_bins
80
80
✓ kept
Module classes
Instances of each class in the built model (prefix Whisper omitted).
Class
Original
Fixture
Linear
65
33
LayerNorm
22
12
Attention
12
6
GELUActivation
8
4
DecoderLayer
4
2
EncoderLayer
4
2
Conv1d
2
2
Embedding
2
2
ModuleList
2
2
Decoder
1
1
Encoder
1
1
ForConditionalGeneration
1
1
Model
1
1
PositionalEmbedding
1
1
Config changes
7 fields changed, 40 unchanged.
Field
Original
Fixture
d_model
384
64
decoder_attention_heads
6
4
decoder_ffn_dim
1536
128
decoder_layers
4
2
encoder_attention_heads
6
4
encoder_ffn_dim
1536
128
encoder_layers
4
2
+
\ No newline at end of file
diff --git a/shrink/xlm-roberta-base-mic.html b/shrink/xlm-roberta-base-mic.html
new file mode 100644
index 0000000000000000000000000000000000000000..9334051d71ca956227049428af881fceb00a2169
--- /dev/null
+++ b/shrink/xlm-roberta-base-mic.html
@@ -0,0 +1,34 @@
+
+
+Shrunk architecture
+
+
Shrunk architecture
huggingface-mic/xlm-roberta-base-mic@934d2300ab3a3eb5acc9c187a91f51c325b31f06 ← FacebookAI/xlm-roberta-base@e73636d4f797dec63c3081bb6ed5c7b0bb3f2089 · XLMRobertaForMaskedLM · random weights · built before the shrink rule (no recorded recipe)
+
Original278.3Mparameters
Fixture16.4Mparameters
Layer stacks1compared
Findings0to review
✓ Same architecture, smaller
Every module class, layer kind and checked property of the original is in the fixture.
Layer stacks
One square per layer, colored by kind: the layer's module classes and type attributes. Hover a square for its index and kind.
roberta.encoder.layer
Original · 12
Fixture · 2
Kind
Original
Fixture
Layer
12
2
✓ kept
Where the parameters are
Share of each model's parameters per component. Vocabulary and patch sizes are kept, so embeddings usually take a larger share of the fixture.
Original
Fixture
Component
Original
Share
Fixture
Share
lm_head.decoder
192.0M
69.0%
16.0M
97.8%
roberta.encoder.layer
85.1M
30.6%
66.9K
0.4%
lm_head.dense
590.6K
0.2%
4.2K
0.0%
roberta.embeddings
397.1K
0.1%
33.1K
0.2%
lm_head
250.0K
0.1%
250.0K
1.5%
lm_head.layer_norm
1.5K
0.0%
128
0.0%
Preserved properties
Config
Property
Original
Fixture
model
model_type
xlm-roberta
xlm-roberta
✓ kept
model
vocab_size
250002
250002
✓ kept
model
tie_word_embeddings
True
True
✓ kept
model
hidden_act
gelu
gelu
✓ kept
model
max_position_embeddings
514
514
✓ kept
Module classes
Instances of each class in the built model (prefix XLMRoberta omitted).
Class
Original
Fixture
Linear
74
14
Dropout
37
7
LayerNorm
26
6
GELUActivation
12
2
Attention
12
2
Intermediate
12
2
Layer
12
2
Output
12
2
SelfAttention
12
2
SelfOutput
12
2
Embedding
3
3
ModuleList
1
1
Embeddings
1
1
Encoder
1
1
ForMaskedLM
1
1
LMHead
1
1
Model
1
1
Config changes
4 fields changed, 27 unchanged.
Field
Original
Fixture
hidden_size
768
64
intermediate_size
3072
128
num_attention_heads
12
4
num_hidden_layers
12
2
+
\ No newline at end of file
diff --git a/shrink/xlm-roberta-xl-mic.html b/shrink/xlm-roberta-xl-mic.html
new file mode 100644
index 0000000000000000000000000000000000000000..5e8c079b015017336d8275e1be3a110df8ce4937
--- /dev/null
+++ b/shrink/xlm-roberta-xl-mic.html
@@ -0,0 +1,34 @@
+
+
+Shrunk architecture
+
+
Shrunk architecture
huggingface-mic/xlm-roberta-xl-mic@54cda088a6d8425f8172bc7e1b90f5958a43c1af ← facebook/xlm-roberta-xl@aa5d120255845efeebc9b7f42822a1dd0f9ece9d · XLMRobertaXLForMaskedLM · random weights
+
Original3.5Bparameters
Fixture16.4Mparameters
Layer stacks1compared
Findings0to review
✓ Same architecture, smaller
Every module class, layer kind and checked property of the original is in the fixture.
Layer stacks
One square per layer, colored by kind: the layer's module classes and type attributes. Hover a square for its index and kind.
roberta.encoder.layer
Original · 36
Fixture · 2
Kind
Original
Fixture
Layer
36
2
✓ kept
Where the parameters are
Share of each model's parameters per component. Vocabulary and patch sizes are kept, so embeddings usually take a larger share of the fixture.
Original
Fixture
Component
Original
Share
Fixture
Share
roberta.encoder.layer
2.8B
81.3%
66.9K
0.4%
roberta.embeddings
643.6M
18.5%
16.1M
98.0%
lm_head.dense
6.6M
0.2%
4.2K
0.0%
lm_head
250.9K
0.0%
250.9K
1.5%
lm_head.layer_norm
5.1K
0.0%
128
0.0%
roberta.encoder
5.1K
0.0%
128
0.0%
Preserved properties
Config
Property
Original
Fixture
model
model_type
xlm-roberta-xl
xlm-roberta-xl
✓ kept
model
vocab_size
250880
250880
✓ kept
model
tie_word_embeddings
True
True
✓ kept
model
hidden_act
gelu
gelu
✓ kept
model
max_position_embeddings
514
514
✓ kept
Module classes
Instances of each class in the built model (prefix XLMRobertaXL omitted).
Class
Original
Fixture
Linear
218
14
LayerNorm
74
6
Dropout
73
5
GELUActivation
36
2
Attention
36
2
Intermediate
36
2
Layer
36
2
Output
36
2
SelfAttention
36
2
SelfOutput
36
2
Embedding
3
3
ModuleList
1
1
Embeddings
1
1
Encoder
1
1
ForMaskedLM
1
1
LMHead
1
1
Model
1
1
Config changes
4 fields changed, 27 unchanged.
Field
Original
Fixture
hidden_size
2560
64
intermediate_size
10240
128
num_attention_heads
32
4
num_hidden_layers
36
2
+
\ No newline at end of file
diff --git a/shrink/yolos-small-mic.html b/shrink/yolos-small-mic.html
new file mode 100644
index 0000000000000000000000000000000000000000..4276aa379e526e95afe7bbc30acfec0d9f0d103b
--- /dev/null
+++ b/shrink/yolos-small-mic.html
@@ -0,0 +1,34 @@
+
+
+Shrunk architecture
+
+