tarekziade HF Staff commited on
Commit
0b4af0f
·
verified ·
1 Parent(s): 7bf22fd

Publish MIC usage, slim fixture and reviewed Transformers expectations

Browse files
README.md CHANGED
@@ -51,77 +51,14 @@ Built by [`build_fixtures.py`](https://github.com/huggingface/transformers-mic/b
51
 
52
  ## Slim model
53
 
54
- ### DeepSeek-R1-mic
55
-
56
- Slim model weights, executable usage and framework expectations in one repository.
57
-
58
- - [Contract manifest](integration.json)
59
- - [Transformers usage](integrations/transformers/usage.md)
60
- - [Transformers expectations](integrations/transformers/expectations)
61
- - [Fixture provenance](build.json)
62
- - [How the fixture was shrunk](https://huggingface-mic-mic-coverage.static.hf.space/shrink/DeepSeek-R1-mic.html)
63
-
64
- ## Contract
65
-
66
- ### DeepSeek-R1-mic
67
-
68
- Slim model weights, executable usage and framework expectations in one repository.
69
-
70
- - [Contract manifest](integration.json)
71
- - [Transformers usage](integrations/transformers/usage.md)
72
- - [Transformers expectations](integrations/transformers/expectations)
73
- - [Fixture provenance](build.json)
74
- - [How the fixture was shrunk](https://huggingface-mic-mic-coverage.static.hf.space/shrink/DeepSeek-R1-mic.html)
75
-
76
- ## Contract
77
-
78
- ### DeepSeek-R1 causal LM
79
-
80
- Runnable usage and the framework expectations directory are linked above.
81
-
82
- ## Slim model
83
-
84
- ### Slim fixture: DeepSeek-R1-mic
85
-
86
- A 16.6M-parameter fixture for the [DeepSeek-R1 MIC](https://huggingface.co/huggingface-mic/DeepSeek-R1-mic), covering [`deepseek-ai/DeepSeek-R1`](https://huggingface.co/deepseek-ai/DeepSeek-R1). It is a test artifact, not a usable model, tagged `test-artifact` to keep it out of candidate-model search.
87
-
88
- - **Fixture key `3bd6b377673a`.** It covers every checkpoint whose processor files are byte-identical and whose config matches after shrinking.
89
- - **Same architecture, smaller config:** `DeepseekV3ForCausalLM`, shrunk by the generic rule in [`sweep.py`](https://github.com/huggingface/transformers-mic/blob/main/experiments/sweep.py) (num_hidden_layers 2).
90
- - **Processor files copied byte for byte** from `deepseek-ai/DeepSeek-R1`: `LICENSE`, `configuration_deepseek.py`, `generation_config.json`, `modeling_deepseek.py`, `tokenizer.json`, `tokenizer_config.json`.
91
- - **Random weights (seed 17).** They passed the input-sensitivity gate: every perturbation of the contract's inputs moved the logits by at least 0.001 (weakest: `input_ids:roll_batch`, 0.932), so no training was needed.
92
- - **Same tensor names as the real checkpoint.** The real checkpoint has every parameter, and so does the fixture.
93
- - **Saved in bfloat16**, the dtype `deepseek-ai/DeepSeek-R1` loads in by default.
94
- - **Finite in float32, bfloat16, and float16** (forward, loss, gradients).
95
-
96
- Built by [`build_fixtures.py`](https://github.com/huggingface/transformers-mic/blob/main/example/build_fixtures.py); [`build.json`](build.json) records the pinned sources, gate results, and file hashes. This repository includes the slim weights, usage and framework expectations; rebuild the other files with `python example/build_fixtures.py --fixture DeepSeek-R1-mic`.
97
-
98
- ## Slim model
99
-
100
- ### DeepSeek-R1-mic
101
-
102
- Slim model weights, executable usage and framework expectations in one repository.
103
-
104
- - [Contract manifest](integration.json)
105
- - [Transformers usage](integrations/transformers/usage.md)
106
- - [Transformers expectations](integrations/transformers/expectations)
107
- - [Fixture provenance](build.json)
108
-
109
- ## Contract
110
-
111
- ### DeepSeek-R1 causal LM
112
-
113
- Runnable usage and the framework expectations directory are linked above.
114
-
115
- ## Slim model
116
-
117
  ### Slim fixture: DeepSeek-R1-mic
118
 
119
- A 16.6M-parameter fixture for the [DeepSeek-R1 MIC](https://huggingface.co/huggingface-mic/DeepSeek-R1-mic), covering [`deepseek-ai/DeepSeek-R1`](https://huggingface.co/deepseek-ai/DeepSeek-R1). It is a test artifact, not a usable model, tagged `test-artifact` to keep it out of candidate-model search.
120
 
121
- - **Fixture key `3bd6b377673a`.** It covers every checkpoint whose processor files are byte-identical and whose config matches after shrinking.
122
  - **Same architecture, smaller config:** `DeepseekV3ForCausalLM`, shrunk by the generic rule in [`sweep.py`](https://github.com/huggingface/transformers-mic/blob/main/experiments/sweep.py) (num_hidden_layers 2).
123
  - **Processor files copied byte for byte** from `deepseek-ai/DeepSeek-R1`: `LICENSE`, `configuration_deepseek.py`, `generation_config.json`, `modeling_deepseek.py`, `tokenizer.json`, `tokenizer_config.json`.
124
- - **Random weights (seed 17).** They passed the input-sensitivity gate: every perturbation of the contract's inputs moved the logits by at least 0.001 (weakest: `input_ids:roll_batch`, 0.932), so no training was needed.
125
  - **Same tensor names as the real checkpoint.** The real checkpoint has every parameter, and so does the fixture.
126
  - **Saved in bfloat16**, the dtype `deepseek-ai/DeepSeek-R1` loads in by default.
127
  - **Finite in float32, bfloat16, and float16** (forward, loss, gradients).
 
51
 
52
  ## Slim model
53
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
54
  ### Slim fixture: DeepSeek-R1-mic
55
 
56
+ A 17.8M-parameter fixture for the [DeepSeek-R1 MIC](https://huggingface.co/huggingface-mic/DeepSeek-R1-mic), covering [`deepseek-ai/DeepSeek-R1`](https://huggingface.co/deepseek-ai/DeepSeek-R1). It is a test artifact, not a usable model, tagged `test-artifact` to keep it out of candidate-model search.
57
 
58
+ - **Fixture key `d9610d1fd9b1`.** It covers every checkpoint whose processor files are byte-identical and whose config matches after shrinking.
59
  - **Same architecture, smaller config:** `DeepseekV3ForCausalLM`, shrunk by the generic rule in [`sweep.py`](https://github.com/huggingface/transformers-mic/blob/main/experiments/sweep.py) (num_hidden_layers 2).
60
  - **Processor files copied byte for byte** from `deepseek-ai/DeepSeek-R1`: `LICENSE`, `configuration_deepseek.py`, `generation_config.json`, `modeling_deepseek.py`, `tokenizer.json`, `tokenizer_config.json`.
61
+ - **Random weights (seed 17).** They passed the input-sensitivity gate: every perturbation of the contract's inputs moved the logits by at least 0.001 (weakest: `input_ids:roll_batch`, 0.314), so no training was needed.
62
  - **Same tensor names as the real checkpoint.** The real checkpoint has every parameter, and so does the fixture.
63
  - **Saved in bfloat16**, the dtype `deepseek-ai/DeepSeek-R1` loads in by default.
64
  - **Finite in float32, bfloat16, and float16** (forward, loss, gradients).
build.json CHANGED
@@ -1,6 +1,6 @@
1
  {
2
  "fixture": "huggingface-mic/DeepSeek-R1-mic",
3
- "key": "3bd6b377673ade4cffef42b326863c5541aa06225fd97eabc42fda15040f43cb",
4
  "key_fields": {
5
  "architectures": [
6
  "DeepseekV3ForCausalLM"
@@ -29,11 +29,11 @@
29
  "n_routed_experts": 4,
30
  "output_router_logits": false,
31
  "routed_scaling_factor": 2.5,
32
- "kv_lora_rank": 16,
33
  "q_lora_rank": 16,
34
- "qk_rope_head_dim": 8,
35
- "v_head_dim": 16,
36
- "qk_nope_head_dim": 16,
37
  "n_group": 1,
38
  "topk_group": 1,
39
  "num_experts_per_tok": 2,
@@ -65,7 +65,7 @@
65
  "attention_dropout": 0.0,
66
  "num_nextn_predict_layers": 1,
67
  "qk_head_dim": 192,
68
- "head_dim": 8,
69
  "auto_map": {
70
  "AutoConfig": "configuration_deepseek.DeepseekV3Config",
71
  "AutoModel": "modeling_deepseek.DeepseekV3Model",
@@ -89,9 +89,23 @@
89
  "seed": 17,
90
  "dtype": "bfloat16",
91
  "builder": "example/build_fixtures.py",
 
 
 
 
 
 
92
  "model_class": "DeepseekV3ForCausalLM",
93
- "parameters": 16624256,
94
  "missing_parameters": [],
 
 
 
 
 
 
 
 
95
  "checkpoint_inspection": "parameter headers/indexes; bounded binary fallback",
96
  "numeric_health_scope": "CPU float32/bfloat16/float16; autocast for low-precision losses",
97
  "captured": {
@@ -110,8 +124,8 @@
110
  "accepted_insensitive": {},
111
  "threshold": 0.001,
112
  "deltas": {
113
- "input_ids:roll_batch": 0.931640625,
114
- "input_ids:reverse": 1.0390625,
115
  "attention_mask:all_ones": null
116
  },
117
  "weakest": "input_ids:roll_batch",
@@ -148,18 +162,18 @@
148
  "tokenizer_config.json"
149
  ],
150
  "environment": {
151
- "python": "3.12.12",
152
  "machine": "arm64",
153
- "torch": "2.14.1",
154
- "transformers": "5.22.0.dev0",
155
  "tokenizers": "0.23.3"
156
  },
157
  "sha256": {
158
  "LICENSE": "1c8f573e830ca9b3ebfeb7ace1823146e22b66f99ee223840e7637c9e745e1c7",
159
- "config.json": "38c3aa3e4a1367dae62f73668c736aff5af1eb60abf22bc96ba2ca6717413a55",
160
  "configuration_deepseek.py": "d5126f7a82eaa5fe10c294af7a8f02d59e7145c2dc8837b83ae338c1b72001d2",
161
  "generation_config.json": "3f21d5d199c7ea6b0f807e3d9865a8a9957b76a604c946bce6e73fd0629f8ca3",
162
- "model.safetensors": "91f350ced1ae466ada4f94007cf4dc61f10b550bcc7d571dd0f19180e14ca8c9",
163
  "modeling_deepseek.py": "eb6d535742061cf9da7a4232ecfe2f089d5ce20080afc5d2181e9d70ec4af8a4",
164
  "tokenizer.json": "ecb6f9fc369894346f0511f4074ca75cee5cd5f3b06d02f1ba35fcd39f8e121d",
165
  "tokenizer_config.json": "99e80c96269ca4430f1e67f67b5766cb096cbac96fed7a7c611c6ee088b3a039"
 
1
  {
2
  "fixture": "huggingface-mic/DeepSeek-R1-mic",
3
+ "key": "d9610d1fd9b1f8e6aefe8132b9711a2f3a3bd7e34e8cf5ddb38c3ce6134e4150",
4
  "key_fields": {
5
  "architectures": [
6
  "DeepseekV3ForCausalLM"
 
29
  "n_routed_experts": 4,
30
  "output_router_logits": false,
31
  "routed_scaling_factor": 2.5,
32
+ "kv_lora_rank": 512,
33
  "q_lora_rank": 16,
34
+ "qk_rope_head_dim": 64,
35
+ "v_head_dim": 128,
36
+ "qk_nope_head_dim": 128,
37
  "n_group": 1,
38
  "topk_group": 1,
39
  "num_experts_per_tok": 2,
 
65
  "attention_dropout": 0.0,
66
  "num_nextn_predict_layers": 1,
67
  "qk_head_dim": 192,
68
+ "head_dim": 64,
69
  "auto_map": {
70
  "AutoConfig": "configuration_deepseek.DeepseekV3Config",
71
  "AutoModel": "modeling_deepseek.DeepseekV3Model",
 
89
  "seed": 17,
90
  "dtype": "bfloat16",
91
  "builder": "example/build_fixtures.py",
92
+ "builder_revision": "4140f5de707b81845dc190394fa962c91c66e343",
93
+ "recipe": {
94
+ "seed": 17,
95
+ "model_class": "DeepseekV3ForCausalLM"
96
+ },
97
+ "shrink": "rule",
98
  "model_class": "DeepseekV3ForCausalLM",
99
+ "parameters": 17819232,
100
  "missing_parameters": [],
101
+ "serialization": {
102
+ "file": "model.safetensors",
103
+ "source_file": "model.safetensors.index.json",
104
+ "reverted_conversions": [
105
+ "WeightConverter: ['mlp.experts.*.gate_proj.weight', 'mlp.experts.*.up_proj.weight'] -> ['mlp.experts.gate_up_proj']",
106
+ "WeightConverter: ['mlp.experts.*.down_proj.weight'] -> ['mlp.experts.down_proj']"
107
+ ]
108
+ },
109
  "checkpoint_inspection": "parameter headers/indexes; bounded binary fallback",
110
  "numeric_health_scope": "CPU float32/bfloat16/float16; autocast for low-precision losses",
111
  "captured": {
 
124
  "accepted_insensitive": {},
125
  "threshold": 0.001,
126
  "deltas": {
127
+ "input_ids:roll_batch": 0.314453125,
128
+ "input_ids:reverse": 1.11328125,
129
  "attention_mask:all_ones": null
130
  },
131
  "weakest": "input_ids:roll_batch",
 
162
  "tokenizer_config.json"
163
  ],
164
  "environment": {
165
+ "python": "3.11.14",
166
  "machine": "arm64",
167
+ "torch": "2.15.0",
168
+ "transformers": "5.24.0.dev0",
169
  "tokenizers": "0.23.3"
170
  },
171
  "sha256": {
172
  "LICENSE": "1c8f573e830ca9b3ebfeb7ace1823146e22b66f99ee223840e7637c9e745e1c7",
173
+ "config.json": "5a4ea02cf221c027e8fb8a784761915046db80e5d97d120847fa2f262a7d32d1",
174
  "configuration_deepseek.py": "d5126f7a82eaa5fe10c294af7a8f02d59e7145c2dc8837b83ae338c1b72001d2",
175
  "generation_config.json": "3f21d5d199c7ea6b0f807e3d9865a8a9957b76a604c946bce6e73fd0629f8ca3",
176
+ "model.safetensors": "d31b47e9bc2a35049a74e9c7154e140481b06ed5847da71a25c19a6ae0ffe95c",
177
  "modeling_deepseek.py": "eb6d535742061cf9da7a4232ecfe2f089d5ce20080afc5d2181e9d70ec4af8a4",
178
  "tokenizer.json": "ecb6f9fc369894346f0511f4074ca75cee5cd5f3b06d02f1ba35fcd39f8e121d",
179
  "tokenizer_config.json": "99e80c96269ca4430f1e67f67b5766cb096cbac96fed7a7c611c6ee088b3a039"
config.json CHANGED
@@ -14,12 +14,12 @@
14
  "eos_token_id": 1,
15
  "ep_size": 1,
16
  "first_k_dense_replace": 1,
17
- "head_dim": 8,
18
  "hidden_act": "silu",
19
  "hidden_size": 64,
20
  "initializer_range": 0.02,
21
  "intermediate_size": 128,
22
- "kv_lora_rank": 16,
23
  "max_position_embeddings": 163840,
24
  "model_type": "deepseek_v3",
25
  "moe_intermediate_size": 32,
@@ -38,8 +38,8 @@
38
  "pretraining_tp": 1,
39
  "q_lora_rank": 16,
40
  "qk_head_dim": 192,
41
- "qk_nope_head_dim": 16,
42
- "qk_rope_head_dim": 8,
43
  "rms_norm_eps": 1e-06,
44
  "rope_interleave": true,
45
  "rope_parameters": {
@@ -58,8 +58,8 @@
58
  "tie_word_embeddings": false,
59
  "topk_group": 1,
60
  "topk_method": "noaux_tc",
61
- "transformers_version": "5.22.0.dev0",
62
  "use_cache": true,
63
- "v_head_dim": 16,
64
  "vocab_size": 129280
65
  }
 
14
  "eos_token_id": 1,
15
  "ep_size": 1,
16
  "first_k_dense_replace": 1,
17
+ "head_dim": 64,
18
  "hidden_act": "silu",
19
  "hidden_size": 64,
20
  "initializer_range": 0.02,
21
  "intermediate_size": 128,
22
+ "kv_lora_rank": 512,
23
  "max_position_embeddings": 163840,
24
  "model_type": "deepseek_v3",
25
  "moe_intermediate_size": 32,
 
38
  "pretraining_tp": 1,
39
  "q_lora_rank": 16,
40
  "qk_head_dim": 192,
41
+ "qk_nope_head_dim": 128,
42
+ "qk_rope_head_dim": 64,
43
  "rms_norm_eps": 1e-06,
44
  "rope_interleave": true,
45
  "rope_parameters": {
 
58
  "tie_word_embeddings": false,
59
  "topk_group": 1,
60
  "topk_method": "noaux_tc",
61
+ "transformers_version": "5.24.0.dev0",
62
  "use_cache": true,
63
+ "v_head_dim": 128,
64
  "vocab_size": 129280
65
  }
integrations/transformers/expectations/cpu-x86_64-bf16/DeepSeek-R1-mic.bfloat16.json CHANGED
@@ -2,18 +2,18 @@
2
  "recorded_with": {
3
  "contract": "huggingface-mic/DeepSeek-R1-mic",
4
  "contract_name": "DeepSeek-R1",
5
- "contract_revision": "73715aeb6e20225448964b4e5d761506d096d576",
6
  "framework": "transformers",
7
  "variant": "bfloat16",
8
  "target": "DeepSeek-R1-mic",
9
  "kind": "fixture",
10
- "revision": "73715aeb6e20225448964b4e5d761506d096d576",
11
  "source_model": "deepseek-ai/DeepSeek-R1",
12
  "weights": "random",
13
  "fixture_repo": "huggingface-mic/DeepSeek-R1-mic",
14
  "environment": {
15
  "python": "3.11.18",
16
- "transformers": "5.23.0.dev0",
17
  "sentence-transformers": null,
18
  "torch": "2.15.0+cpu",
19
  "torchvision": "0.29.2+cpu",
@@ -21,7 +21,7 @@
21
  "tokenizers": "0.23.3",
22
  "numpy": "2.3.5",
23
  "pillow": "13.0.0",
24
- "accelerate": "1.15.0",
25
  "safetensors": "0.8.0",
26
  "huggingface-hub": "2.4.0",
27
  "sentencepiece": "0.2.3",
@@ -30,12 +30,10 @@
30
  "soundfile": "0.14.0",
31
  "scipy": "1.17.1"
32
  },
33
- "seconds": 8.05,
34
  "environment_differs": {
35
- "torch": "2.15.0+cpu (pinned 2.14.0)",
36
- "torchvision": "0.29.2+cpu (pinned 0.29.1)",
37
- "torchaudio": "2.11.1+cpu (pinned 2.11.0)",
38
  "pillow": "13.0.0 (pinned 12.3.0)",
 
39
  "huggingface-hub": "2.4.0 (pinned 2.3.0)",
40
  "sentencepiece": "0.2.3 (pinned 0.2.2)"
41
  },
@@ -49,65 +47,65 @@
49
  "attn_implementation": "sdpa"
50
  },
51
  "next_token_ids": [
52
- 119704,
53
- 5475
54
  ],
55
  "probe_logits": [
56
  [
57
- -0.1796875,
58
- -0.10888671875,
59
- -0.10986328125,
60
- -0.1669921875,
61
- 0.205078125,
62
- -0.1767578125,
63
- 0.10400390625,
64
- 0.05224609375,
65
- -0.00946044921875,
66
- 0.1162109375,
67
- -0.06298828125,
68
- -0.03759765625,
69
- -0.09326171875,
70
- 0.2275390625,
71
- -0.1005859375,
72
- 0.068359375
73
  ],
74
  [
75
- -0.2119140625,
76
- -0.07568359375,
77
- -0.0966796875,
78
- -0.14453125,
79
- 0.21875,
80
- -0.26953125,
81
- 0.1435546875,
82
- 0.043701171875,
83
- 0.0037841796875,
84
- 0.12890625,
85
- -0.140625,
86
- 0.0140380859375,
87
- -0.140625,
88
- 0.203125,
89
- -0.12255859375,
90
- 0.09033203125
91
  ]
92
  ],
93
  "position_norms": [
94
  [
95
- 57.44831085205078,
96
- 57.26078414916992,
97
- 57.433380126953125,
98
- 57.489166259765625,
99
- 57.36770248413086
100
  ],
101
  [
102
- 57.44831085205078,
103
- 57.26078414916992,
104
- 57.433380126953125,
105
- 57.506778717041016,
106
- 57.35624313354492
107
  ]
108
  ],
109
  "training": {
110
- "loss": 11.722227096557617,
111
  "gradients": {
112
  "model.layers.0.self_attn.q_a_proj.weight": true,
113
  "model.layers.0.mlp.gate_proj.weight": true
 
2
  "recorded_with": {
3
  "contract": "huggingface-mic/DeepSeek-R1-mic",
4
  "contract_name": "DeepSeek-R1",
5
+ "contract_revision": "2d3440f540cc19f64a1150859f418f0b15ae352d",
6
  "framework": "transformers",
7
  "variant": "bfloat16",
8
  "target": "DeepSeek-R1-mic",
9
  "kind": "fixture",
10
+ "revision": "2d3440f540cc19f64a1150859f418f0b15ae352d",
11
  "source_model": "deepseek-ai/DeepSeek-R1",
12
  "weights": "random",
13
  "fixture_repo": "huggingface-mic/DeepSeek-R1-mic",
14
  "environment": {
15
  "python": "3.11.18",
16
+ "transformers": "5.24.0.dev0",
17
  "sentence-transformers": null,
18
  "torch": "2.15.0+cpu",
19
  "torchvision": "0.29.2+cpu",
 
21
  "tokenizers": "0.23.3",
22
  "numpy": "2.3.5",
23
  "pillow": "13.0.0",
24
+ "accelerate": "1.16.0",
25
  "safetensors": "0.8.0",
26
  "huggingface-hub": "2.4.0",
27
  "sentencepiece": "0.2.3",
 
30
  "soundfile": "0.14.0",
31
  "scipy": "1.17.1"
32
  },
33
+ "seconds": 7.6,
34
  "environment_differs": {
 
 
 
35
  "pillow": "13.0.0 (pinned 12.3.0)",
36
+ "accelerate": "1.16.0 (pinned 1.15.0)",
37
  "huggingface-hub": "2.4.0 (pinned 2.3.0)",
38
  "sentencepiece": "0.2.3 (pinned 0.2.2)"
39
  },
 
47
  "attn_implementation": "sdpa"
48
  },
49
  "next_token_ids": [
50
+ 5519,
51
+ 40890
52
  ],
53
  "probe_logits": [
54
  [
55
+ 0.00408935546875,
56
+ -0.345703125,
57
+ -0.07470703125,
58
+ 0.048583984375,
59
+ 0.06591796875,
60
+ -0.171875,
61
+ 0.12255859375,
62
+ 0.0654296875,
63
+ 0.0361328125,
64
+ -0.053955078125,
65
+ 0.28125,
66
+ -0.05419921875,
67
+ 0.1220703125,
68
+ -0.051513671875,
69
+ -0.00439453125,
70
+ 0.12890625
71
  ],
72
  [
73
+ 0.0634765625,
74
+ -0.2578125,
75
+ -0.10546875,
76
+ 0.04248046875,
77
+ -0.0458984375,
78
+ -0.09716796875,
79
+ 0.09130859375,
80
+ 0.177734375,
81
+ 0.0233154296875,
82
+ 0.03369140625,
83
+ 0.271484375,
84
+ -0.1240234375,
85
+ 0.09912109375,
86
+ -0.1708984375,
87
+ 0.09716796875,
88
+ 0.01806640625
89
  ]
90
  ],
91
  "position_norms": [
92
  [
93
+ 57.53282165527344,
94
+ 57.505489349365234,
95
+ 57.503578186035156,
96
+ 57.46382522583008,
97
+ 57.49515914916992
98
  ],
99
  [
100
+ 57.53282165527344,
101
+ 57.505489349365234,
102
+ 57.503578186035156,
103
+ 57.47362518310547,
104
+ 57.47813415527344
105
  ]
106
  ],
107
  "training": {
108
+ "loss": 11.817240715026855,
109
  "gradients": {
110
  "model.layers.0.self_attn.q_a_proj.weight": true,
111
  "model.layers.0.mlp.gate_proj.weight": true
integrations/transformers/expectations/cpu-x86_64-bf16/DeepSeek-R1-mic.eager.json CHANGED
@@ -2,18 +2,18 @@
2
  "recorded_with": {
3
  "contract": "huggingface-mic/DeepSeek-R1-mic",
4
  "contract_name": "DeepSeek-R1",
5
- "contract_revision": "73715aeb6e20225448964b4e5d761506d096d576",
6
  "framework": "transformers",
7
  "variant": "eager",
8
  "target": "DeepSeek-R1-mic",
9
  "kind": "fixture",
10
- "revision": "73715aeb6e20225448964b4e5d761506d096d576",
11
  "source_model": "deepseek-ai/DeepSeek-R1",
12
  "weights": "random",
13
  "fixture_repo": "huggingface-mic/DeepSeek-R1-mic",
14
  "environment": {
15
  "python": "3.11.18",
16
- "transformers": "5.23.0.dev0",
17
  "sentence-transformers": null,
18
  "torch": "2.15.0+cpu",
19
  "torchvision": "0.29.2+cpu",
@@ -21,7 +21,7 @@
21
  "tokenizers": "0.23.3",
22
  "numpy": "2.3.5",
23
  "pillow": "13.0.0",
24
- "accelerate": "1.15.0",
25
  "safetensors": "0.8.0",
26
  "huggingface-hub": "2.4.0",
27
  "sentencepiece": "0.2.3",
@@ -30,12 +30,10 @@
30
  "soundfile": "0.14.0",
31
  "scipy": "1.17.1"
32
  },
33
- "seconds": 7.95,
34
  "environment_differs": {
35
- "torch": "2.15.0+cpu (pinned 2.14.0)",
36
- "torchvision": "0.29.2+cpu (pinned 0.29.1)",
37
- "torchaudio": "2.11.1+cpu (pinned 2.11.0)",
38
  "pillow": "13.0.0 (pinned 12.3.0)",
 
39
  "huggingface-hub": "2.4.0 (pinned 2.3.0)",
40
  "sentencepiece": "0.2.3 (pinned 0.2.2)"
41
  },
@@ -49,65 +47,65 @@
49
  "attn_implementation": "eager"
50
  },
51
  "next_token_ids": [
52
- 119704,
53
- 5475
54
  ],
55
  "probe_logits": [
56
  [
57
- -0.1787109375,
58
- -0.10888671875,
59
- -0.109375,
60
- -0.16796875,
61
- 0.203125,
62
- -0.1767578125,
63
- 0.103515625,
64
- 0.052734375,
65
- -0.0089111328125,
66
- 0.11669921875,
67
- -0.0625,
68
- -0.03662109375,
69
- -0.09326171875,
70
- 0.2275390625,
71
- -0.10205078125,
72
- 0.06787109375
73
  ],
74
  [
75
- -0.2119140625,
76
- -0.07666015625,
77
- -0.09716796875,
78
- -0.14453125,
79
- 0.2177734375,
80
- -0.267578125,
81
- 0.14453125,
82
- 0.044189453125,
83
- 0.00494384765625,
84
- 0.12890625,
85
- -0.1416015625,
86
- 0.0155029296875,
87
- -0.1416015625,
88
- 0.203125,
89
- -0.123046875,
90
- 0.08984375
91
  ]
92
  ],
93
  "position_norms": [
94
  [
95
- 57.44831085205078,
96
- 57.248531341552734,
97
- 57.43208312988281,
98
- 57.50393295288086,
99
- 57.39615249633789
100
  ],
101
  [
102
- 57.44831085205078,
103
- 57.248531341552734,
104
- 57.43208312988281,
105
- 57.49941635131836,
106
- 57.37012481689453
107
  ]
108
  ],
109
  "training": {
110
- "loss": 11.722164154052734,
111
  "gradients": {
112
  "model.layers.0.self_attn.q_a_proj.weight": true,
113
  "model.layers.0.mlp.gate_proj.weight": true
 
2
  "recorded_with": {
3
  "contract": "huggingface-mic/DeepSeek-R1-mic",
4
  "contract_name": "DeepSeek-R1",
5
+ "contract_revision": "2d3440f540cc19f64a1150859f418f0b15ae352d",
6
  "framework": "transformers",
7
  "variant": "eager",
8
  "target": "DeepSeek-R1-mic",
9
  "kind": "fixture",
10
+ "revision": "2d3440f540cc19f64a1150859f418f0b15ae352d",
11
  "source_model": "deepseek-ai/DeepSeek-R1",
12
  "weights": "random",
13
  "fixture_repo": "huggingface-mic/DeepSeek-R1-mic",
14
  "environment": {
15
  "python": "3.11.18",
16
+ "transformers": "5.24.0.dev0",
17
  "sentence-transformers": null,
18
  "torch": "2.15.0+cpu",
19
  "torchvision": "0.29.2+cpu",
 
21
  "tokenizers": "0.23.3",
22
  "numpy": "2.3.5",
23
  "pillow": "13.0.0",
24
+ "accelerate": "1.16.0",
25
  "safetensors": "0.8.0",
26
  "huggingface-hub": "2.4.0",
27
  "sentencepiece": "0.2.3",
 
30
  "soundfile": "0.14.0",
31
  "scipy": "1.17.1"
32
  },
33
+ "seconds": 7.61,
34
  "environment_differs": {
 
 
 
35
  "pillow": "13.0.0 (pinned 12.3.0)",
36
+ "accelerate": "1.16.0 (pinned 1.15.0)",
37
  "huggingface-hub": "2.4.0 (pinned 2.3.0)",
38
  "sentencepiece": "0.2.3 (pinned 0.2.2)"
39
  },
 
47
  "attn_implementation": "eager"
48
  },
49
  "next_token_ids": [
50
+ 5519,
51
+ 40890
52
  ],
53
  "probe_logits": [
54
  [
55
+ 0.0047607421875,
56
+ -0.345703125,
57
+ -0.07373046875,
58
+ 0.049560546875,
59
+ 0.06689453125,
60
+ -0.173828125,
61
+ 0.12353515625,
62
+ 0.06591796875,
63
+ 0.0361328125,
64
+ -0.053466796875,
65
+ 0.279296875,
66
+ -0.0546875,
67
+ 0.12109375,
68
+ -0.05126953125,
69
+ -0.005401611328125,
70
+ 0.1298828125
71
  ],
72
  [
73
+ 0.0634765625,
74
+ -0.2578125,
75
+ -0.1064453125,
76
+ 0.0419921875,
77
+ -0.0458984375,
78
+ -0.0986328125,
79
+ 0.09130859375,
80
+ 0.177734375,
81
+ 0.0213623046875,
82
+ 0.033203125,
83
+ 0.271484375,
84
+ -0.12353515625,
85
+ 0.09716796875,
86
+ -0.1708984375,
87
+ 0.09814453125,
88
+ 0.0164794921875
89
  ]
90
  ],
91
  "position_norms": [
92
  [
93
+ 57.53282165527344,
94
+ 57.50996780395508,
95
+ 57.483116149902344,
96
+ 57.50149917602539,
97
+ 57.464073181152344
98
  ],
99
  [
100
+ 57.53282165527344,
101
+ 57.50996780395508,
102
+ 57.483116149902344,
103
+ 57.45439910888672,
104
+ 57.480262756347656
105
  ]
106
  ],
107
  "training": {
108
+ "loss": 11.817363739013672,
109
  "gradients": {
110
  "model.layers.0.self_attn.q_a_proj.weight": true,
111
  "model.layers.0.mlp.gate_proj.weight": true
integrations/transformers/expectations/cpu-x86_64-bf16/DeepSeek-R1-mic.json CHANGED
@@ -2,18 +2,18 @@
2
  "recorded_with": {
3
  "contract": "huggingface-mic/DeepSeek-R1-mic",
4
  "contract_name": "DeepSeek-R1",
5
- "contract_revision": "73715aeb6e20225448964b4e5d761506d096d576",
6
  "framework": "transformers",
7
  "variant": "default",
8
  "target": "DeepSeek-R1-mic",
9
  "kind": "fixture",
10
- "revision": "73715aeb6e20225448964b4e5d761506d096d576",
11
  "source_model": "deepseek-ai/DeepSeek-R1",
12
  "weights": "random",
13
  "fixture_repo": "huggingface-mic/DeepSeek-R1-mic",
14
  "environment": {
15
  "python": "3.11.18",
16
- "transformers": "5.23.0.dev0",
17
  "sentence-transformers": null,
18
  "torch": "2.15.0+cpu",
19
  "torchvision": "0.29.2+cpu",
@@ -21,7 +21,7 @@
21
  "tokenizers": "0.23.3",
22
  "numpy": "2.3.5",
23
  "pillow": "13.0.0",
24
- "accelerate": "1.15.0",
25
  "safetensors": "0.8.0",
26
  "huggingface-hub": "2.4.0",
27
  "sentencepiece": "0.2.3",
@@ -30,12 +30,10 @@
30
  "soundfile": "0.14.0",
31
  "scipy": "1.17.1"
32
  },
33
- "seconds": 8.06,
34
  "environment_differs": {
35
- "torch": "2.15.0+cpu (pinned 2.14.0)",
36
- "torchvision": "0.29.2+cpu (pinned 0.29.1)",
37
- "torchaudio": "2.11.1+cpu (pinned 2.11.0)",
38
  "pillow": "13.0.0 (pinned 12.3.0)",
 
39
  "huggingface-hub": "2.4.0 (pinned 2.3.0)",
40
  "sentencepiece": "0.2.3 (pinned 0.2.2)"
41
  },
@@ -49,65 +47,65 @@
49
  "attn_implementation": "sdpa"
50
  },
51
  "next_token_ids": [
52
- 119704,
53
- 5475
54
  ],
55
  "probe_logits": [
56
  [
57
- -0.1796875,
58
- -0.10888671875,
59
- -0.10986328125,
60
- -0.1669921875,
61
- 0.205078125,
62
- -0.1767578125,
63
- 0.10400390625,
64
- 0.05224609375,
65
- -0.00946044921875,
66
- 0.1162109375,
67
- -0.06298828125,
68
- -0.03759765625,
69
- -0.09326171875,
70
- 0.2275390625,
71
- -0.1005859375,
72
- 0.068359375
73
  ],
74
  [
75
- -0.2119140625,
76
- -0.07568359375,
77
- -0.0966796875,
78
- -0.14453125,
79
- 0.21875,
80
- -0.26953125,
81
- 0.1435546875,
82
- 0.043701171875,
83
- 0.0037841796875,
84
- 0.12890625,
85
- -0.140625,
86
- 0.0140380859375,
87
- -0.140625,
88
- 0.203125,
89
- -0.12255859375,
90
- 0.09033203125
91
  ]
92
  ],
93
  "position_norms": [
94
  [
95
- 57.44831085205078,
96
- 57.26078414916992,
97
- 57.433380126953125,
98
- 57.489166259765625,
99
- 57.36770248413086
100
  ],
101
  [
102
- 57.44831085205078,
103
- 57.26078414916992,
104
- 57.433380126953125,
105
- 57.506778717041016,
106
- 57.35624313354492
107
  ]
108
  ],
109
  "training": {
110
- "loss": 11.722227096557617,
111
  "gradients": {
112
  "model.layers.0.self_attn.q_a_proj.weight": true,
113
  "model.layers.0.mlp.gate_proj.weight": true
 
2
  "recorded_with": {
3
  "contract": "huggingface-mic/DeepSeek-R1-mic",
4
  "contract_name": "DeepSeek-R1",
5
+ "contract_revision": "2d3440f540cc19f64a1150859f418f0b15ae352d",
6
  "framework": "transformers",
7
  "variant": "default",
8
  "target": "DeepSeek-R1-mic",
9
  "kind": "fixture",
10
+ "revision": "2d3440f540cc19f64a1150859f418f0b15ae352d",
11
  "source_model": "deepseek-ai/DeepSeek-R1",
12
  "weights": "random",
13
  "fixture_repo": "huggingface-mic/DeepSeek-R1-mic",
14
  "environment": {
15
  "python": "3.11.18",
16
+ "transformers": "5.24.0.dev0",
17
  "sentence-transformers": null,
18
  "torch": "2.15.0+cpu",
19
  "torchvision": "0.29.2+cpu",
 
21
  "tokenizers": "0.23.3",
22
  "numpy": "2.3.5",
23
  "pillow": "13.0.0",
24
+ "accelerate": "1.16.0",
25
  "safetensors": "0.8.0",
26
  "huggingface-hub": "2.4.0",
27
  "sentencepiece": "0.2.3",
 
30
  "soundfile": "0.14.0",
31
  "scipy": "1.17.1"
32
  },
33
+ "seconds": 7.53,
34
  "environment_differs": {
 
 
 
35
  "pillow": "13.0.0 (pinned 12.3.0)",
36
+ "accelerate": "1.16.0 (pinned 1.15.0)",
37
  "huggingface-hub": "2.4.0 (pinned 2.3.0)",
38
  "sentencepiece": "0.2.3 (pinned 0.2.2)"
39
  },
 
47
  "attn_implementation": "sdpa"
48
  },
49
  "next_token_ids": [
50
+ 5519,
51
+ 40890
52
  ],
53
  "probe_logits": [
54
  [
55
+ 0.00408935546875,
56
+ -0.345703125,
57
+ -0.07470703125,
58
+ 0.048583984375,
59
+ 0.06591796875,
60
+ -0.171875,
61
+ 0.12255859375,
62
+ 0.0654296875,
63
+ 0.0361328125,
64
+ -0.053955078125,
65
+ 0.28125,
66
+ -0.05419921875,
67
+ 0.1220703125,
68
+ -0.051513671875,
69
+ -0.00439453125,
70
+ 0.12890625
71
  ],
72
  [
73
+ 0.0634765625,
74
+ -0.2578125,
75
+ -0.10546875,
76
+ 0.04248046875,
77
+ -0.0458984375,
78
+ -0.09716796875,
79
+ 0.09130859375,
80
+ 0.177734375,
81
+ 0.0233154296875,
82
+ 0.03369140625,
83
+ 0.271484375,
84
+ -0.1240234375,
85
+ 0.09912109375,
86
+ -0.1708984375,
87
+ 0.09716796875,
88
+ 0.01806640625
89
  ]
90
  ],
91
  "position_norms": [
92
  [
93
+ 57.53282165527344,
94
+ 57.505489349365234,
95
+ 57.503578186035156,
96
+ 57.46382522583008,
97
+ 57.49515914916992
98
  ],
99
  [
100
+ 57.53282165527344,
101
+ 57.505489349365234,
102
+ 57.503578186035156,
103
+ 57.47362518310547,
104
+ 57.47813415527344
105
  ]
106
  ],
107
  "training": {
108
+ "loss": 11.817240715026855,
109
  "gradients": {
110
  "model.layers.0.self_attn.q_a_proj.weight": true,
111
  "model.layers.0.mlp.gate_proj.weight": true
integrations/transformers/expectations/cuda/DeepSeek-R1-mic.bfloat16.json CHANGED
@@ -2,26 +2,26 @@
2
  "recorded_with": {
3
  "contract": "huggingface-mic/DeepSeek-R1-mic",
4
  "contract_name": "DeepSeek-R1",
5
- "contract_revision": "73715aeb6e20225448964b4e5d761506d096d576",
6
  "framework": "transformers",
7
  "variant": "bfloat16",
8
  "target": "DeepSeek-R1-mic",
9
  "kind": "fixture",
10
- "revision": "73715aeb6e20225448964b4e5d761506d096d576",
11
  "source_model": "deepseek-ai/DeepSeek-R1",
12
  "weights": "random",
13
  "fixture_repo": "huggingface-mic/DeepSeek-R1-mic",
14
  "environment": {
15
  "python": "3.11.18",
16
- "transformers": "5.23.0.dev0",
17
  "sentence-transformers": null,
18
  "torch": "2.15.0+cu132",
19
  "torchvision": "0.29.2+cu132",
20
  "torchaudio": "2.11.1+cu132",
21
  "tokenizers": "0.23.3",
22
- "numpy": "2.4.6",
23
  "pillow": "13.0.0",
24
- "accelerate": "1.16.0.dev0",
25
  "safetensors": "0.8.0",
26
  "huggingface-hub": "2.4.0",
27
  "sentencepiece": "0.2.3",
@@ -30,14 +30,10 @@
30
  "soundfile": "0.14.0",
31
  "scipy": "1.17.1"
32
  },
33
- "seconds": 7.25,
34
  "environment_differs": {
35
- "torch": "2.15.0+cu132 (pinned 2.14.0)",
36
- "torchvision": "0.29.2+cu132 (pinned 0.29.1)",
37
- "torchaudio": "2.11.1+cu132 (pinned 2.11.0)",
38
- "numpy": "2.4.6 (pinned 2.3.5)",
39
  "pillow": "13.0.0 (pinned 12.3.0)",
40
- "accelerate": "1.16.0.dev0 (pinned 1.15.0)",
41
  "huggingface-hub": "2.4.0 (pinned 2.3.0)",
42
  "sentencepiece": "0.2.3 (pinned 0.2.2)"
43
  },
@@ -51,65 +47,65 @@
51
  "attn_implementation": "sdpa"
52
  },
53
  "next_token_ids": [
54
- 119704,
55
- 5475
56
  ],
57
  "probe_logits": [
58
  [
59
- -0.1796875,
60
- -0.1083984375,
61
- -0.1103515625,
62
- -0.166015625,
63
- 0.205078125,
64
- -0.177734375,
65
- 0.1025390625,
66
- 0.052734375,
67
- -0.00909423828125,
68
- 0.11572265625,
69
- -0.0625,
70
- -0.038818359375,
71
- -0.0927734375,
72
- 0.2265625,
73
- -0.10205078125,
74
- 0.068359375
75
  ],
76
  [
77
- -0.2119140625,
78
- -0.076171875,
79
- -0.0966796875,
80
- -0.14453125,
81
- 0.2177734375,
82
- -0.267578125,
83
- 0.1435546875,
84
- 0.044921875,
85
- 0.004669189453125,
86
- 0.12890625,
87
- -0.1416015625,
88
- 0.01458740234375,
89
- -0.1416015625,
90
- 0.2041015625,
91
- -0.123046875,
92
- 0.08984375
93
  ]
94
  ],
95
  "position_norms": [
96
  [
97
- 57.44833755493164,
98
- 57.25283432006836,
99
- 57.43769073486328,
100
- 57.47720718383789,
101
- 57.3710823059082
102
  ],
103
  [
104
- 57.44833755493164,
105
- 57.25283432006836,
106
- 57.43769073486328,
107
- 57.50749969482422,
108
- 57.35316848754883
109
  ]
110
  ],
111
  "training": {
112
- "loss": 11.72237777709961,
113
  "gradients": {
114
  "model.layers.0.self_attn.q_a_proj.weight": true,
115
  "model.layers.0.mlp.gate_proj.weight": true
 
2
  "recorded_with": {
3
  "contract": "huggingface-mic/DeepSeek-R1-mic",
4
  "contract_name": "DeepSeek-R1",
5
+ "contract_revision": "2d3440f540cc19f64a1150859f418f0b15ae352d",
6
  "framework": "transformers",
7
  "variant": "bfloat16",
8
  "target": "DeepSeek-R1-mic",
9
  "kind": "fixture",
10
+ "revision": "2d3440f540cc19f64a1150859f418f0b15ae352d",
11
  "source_model": "deepseek-ai/DeepSeek-R1",
12
  "weights": "random",
13
  "fixture_repo": "huggingface-mic/DeepSeek-R1-mic",
14
  "environment": {
15
  "python": "3.11.18",
16
+ "transformers": "5.24.0.dev0",
17
  "sentence-transformers": null,
18
  "torch": "2.15.0+cu132",
19
  "torchvision": "0.29.2+cu132",
20
  "torchaudio": "2.11.1+cu132",
21
  "tokenizers": "0.23.3",
22
+ "numpy": "2.3.5",
23
  "pillow": "13.0.0",
24
+ "accelerate": "1.17.0.dev0",
25
  "safetensors": "0.8.0",
26
  "huggingface-hub": "2.4.0",
27
  "sentencepiece": "0.2.3",
 
30
  "soundfile": "0.14.0",
31
  "scipy": "1.17.1"
32
  },
33
+ "seconds": 7.41,
34
  "environment_differs": {
 
 
 
 
35
  "pillow": "13.0.0 (pinned 12.3.0)",
36
+ "accelerate": "1.17.0.dev0 (pinned 1.15.0)",
37
  "huggingface-hub": "2.4.0 (pinned 2.3.0)",
38
  "sentencepiece": "0.2.3 (pinned 0.2.2)"
39
  },
 
47
  "attn_implementation": "sdpa"
48
  },
49
  "next_token_ids": [
50
+ 5519,
51
+ 40890
52
  ],
53
  "probe_logits": [
54
  [
55
+ 0.00390625,
56
+ -0.34765625,
57
+ -0.07373046875,
58
+ 0.048583984375,
59
+ 0.06689453125,
60
+ -0.171875,
61
+ 0.12255859375,
62
+ 0.06494140625,
63
+ 0.03564453125,
64
+ -0.052490234375,
65
+ 0.28125,
66
+ -0.053466796875,
67
+ 0.12255859375,
68
+ -0.052490234375,
69
+ -0.00445556640625,
70
+ 0.12890625
71
  ],
72
  [
73
+ 0.062255859375,
74
+ -0.2578125,
75
+ -0.10546875,
76
+ 0.0419921875,
77
+ -0.046142578125,
78
+ -0.095703125,
79
+ 0.09130859375,
80
+ 0.177734375,
81
+ 0.02294921875,
82
+ 0.033447265625,
83
+ 0.271484375,
84
+ -0.12353515625,
85
+ 0.0986328125,
86
+ -0.171875,
87
+ 0.0986328125,
88
+ 0.0169677734375
89
  ]
90
  ],
91
  "position_norms": [
92
  [
93
+ 57.53282928466797,
94
+ 57.48799514770508,
95
+ 57.509742736816406,
96
+ 57.473533630371094,
97
+ 57.480316162109375
98
  ],
99
  [
100
+ 57.53282928466797,
101
+ 57.48799514770508,
102
+ 57.509742736816406,
103
+ 57.43843460083008,
104
+ 57.4747428894043
105
  ]
106
  ],
107
  "training": {
108
+ "loss": 11.817480087280273,
109
  "gradients": {
110
  "model.layers.0.self_attn.q_a_proj.weight": true,
111
  "model.layers.0.mlp.gate_proj.weight": true
integrations/transformers/expectations/cuda/DeepSeek-R1-mic.eager.json CHANGED
@@ -2,26 +2,26 @@
2
  "recorded_with": {
3
  "contract": "huggingface-mic/DeepSeek-R1-mic",
4
  "contract_name": "DeepSeek-R1",
5
- "contract_revision": "73715aeb6e20225448964b4e5d761506d096d576",
6
  "framework": "transformers",
7
  "variant": "eager",
8
  "target": "DeepSeek-R1-mic",
9
  "kind": "fixture",
10
- "revision": "73715aeb6e20225448964b4e5d761506d096d576",
11
  "source_model": "deepseek-ai/DeepSeek-R1",
12
  "weights": "random",
13
  "fixture_repo": "huggingface-mic/DeepSeek-R1-mic",
14
  "environment": {
15
  "python": "3.11.18",
16
- "transformers": "5.23.0.dev0",
17
  "sentence-transformers": null,
18
  "torch": "2.15.0+cu132",
19
  "torchvision": "0.29.2+cu132",
20
  "torchaudio": "2.11.1+cu132",
21
  "tokenizers": "0.23.3",
22
- "numpy": "2.4.6",
23
  "pillow": "13.0.0",
24
- "accelerate": "1.16.0.dev0",
25
  "safetensors": "0.8.0",
26
  "huggingface-hub": "2.4.0",
27
  "sentencepiece": "0.2.3",
@@ -30,14 +30,10 @@
30
  "soundfile": "0.14.0",
31
  "scipy": "1.17.1"
32
  },
33
- "seconds": 7.44,
34
  "environment_differs": {
35
- "torch": "2.15.0+cu132 (pinned 2.14.0)",
36
- "torchvision": "0.29.2+cu132 (pinned 0.29.1)",
37
- "torchaudio": "2.11.1+cu132 (pinned 2.11.0)",
38
- "numpy": "2.4.6 (pinned 2.3.5)",
39
  "pillow": "13.0.0 (pinned 12.3.0)",
40
- "accelerate": "1.16.0.dev0 (pinned 1.15.0)",
41
  "huggingface-hub": "2.4.0 (pinned 2.3.0)",
42
  "sentencepiece": "0.2.3 (pinned 0.2.2)"
43
  },
@@ -51,65 +47,65 @@
51
  "attn_implementation": "eager"
52
  },
53
  "next_token_ids": [
54
- 119704,
55
- 5475
56
  ],
57
  "probe_logits": [
58
  [
59
- -0.1787109375,
60
- -0.10888671875,
61
- -0.109375,
62
- -0.16796875,
63
- 0.203125,
64
- -0.1767578125,
65
- 0.103515625,
66
- 0.052734375,
67
- -0.0089111328125,
68
- 0.11669921875,
69
- -0.0625,
70
- -0.03662109375,
71
- -0.09326171875,
72
- 0.2275390625,
73
- -0.10205078125,
74
- 0.06787109375
75
  ],
76
  [
77
- -0.2119140625,
78
- -0.07666015625,
79
- -0.09716796875,
80
- -0.14453125,
81
- 0.2177734375,
82
- -0.267578125,
83
- 0.14453125,
84
- 0.044189453125,
85
- 0.00494384765625,
86
- 0.12890625,
87
- -0.1416015625,
88
- 0.0155029296875,
89
- -0.1416015625,
90
- 0.203125,
91
- -0.123046875,
92
- 0.08984375
93
  ]
94
  ],
95
  "position_norms": [
96
  [
97
- 57.44833755493164,
98
- 57.248538970947266,
99
- 57.432106018066406,
100
- 57.50387191772461,
101
- 57.396148681640625
102
  ],
103
  [
104
- 57.44833755493164,
105
- 57.248538970947266,
106
- 57.432106018066406,
107
- 57.49940490722656,
108
- 57.370140075683594
109
  ]
110
  ],
111
  "training": {
112
- "loss": 11.722162246704102,
113
  "gradients": {
114
  "model.layers.0.self_attn.q_a_proj.weight": true,
115
  "model.layers.0.mlp.gate_proj.weight": true
 
2
  "recorded_with": {
3
  "contract": "huggingface-mic/DeepSeek-R1-mic",
4
  "contract_name": "DeepSeek-R1",
5
+ "contract_revision": "2d3440f540cc19f64a1150859f418f0b15ae352d",
6
  "framework": "transformers",
7
  "variant": "eager",
8
  "target": "DeepSeek-R1-mic",
9
  "kind": "fixture",
10
+ "revision": "2d3440f540cc19f64a1150859f418f0b15ae352d",
11
  "source_model": "deepseek-ai/DeepSeek-R1",
12
  "weights": "random",
13
  "fixture_repo": "huggingface-mic/DeepSeek-R1-mic",
14
  "environment": {
15
  "python": "3.11.18",
16
+ "transformers": "5.24.0.dev0",
17
  "sentence-transformers": null,
18
  "torch": "2.15.0+cu132",
19
  "torchvision": "0.29.2+cu132",
20
  "torchaudio": "2.11.1+cu132",
21
  "tokenizers": "0.23.3",
22
+ "numpy": "2.3.5",
23
  "pillow": "13.0.0",
24
+ "accelerate": "1.17.0.dev0",
25
  "safetensors": "0.8.0",
26
  "huggingface-hub": "2.4.0",
27
  "sentencepiece": "0.2.3",
 
30
  "soundfile": "0.14.0",
31
  "scipy": "1.17.1"
32
  },
33
+ "seconds": 7.9,
34
  "environment_differs": {
 
 
 
 
35
  "pillow": "13.0.0 (pinned 12.3.0)",
36
+ "accelerate": "1.17.0.dev0 (pinned 1.15.0)",
37
  "huggingface-hub": "2.4.0 (pinned 2.3.0)",
38
  "sentencepiece": "0.2.3 (pinned 0.2.2)"
39
  },
 
47
  "attn_implementation": "eager"
48
  },
49
  "next_token_ids": [
50
+ 5519,
51
+ 40890
52
  ],
53
  "probe_logits": [
54
  [
55
+ 0.0047607421875,
56
+ -0.345703125,
57
+ -0.07373046875,
58
+ 0.049560546875,
59
+ 0.06689453125,
60
+ -0.173828125,
61
+ 0.12353515625,
62
+ 0.06591796875,
63
+ 0.0361328125,
64
+ -0.053466796875,
65
+ 0.279296875,
66
+ -0.0546875,
67
+ 0.12109375,
68
+ -0.05126953125,
69
+ -0.005401611328125,
70
+ 0.1298828125
71
  ],
72
  [
73
+ 0.0634765625,
74
+ -0.2578125,
75
+ -0.1064453125,
76
+ 0.0419921875,
77
+ -0.0458984375,
78
+ -0.0986328125,
79
+ 0.09130859375,
80
+ 0.177734375,
81
+ 0.0213623046875,
82
+ 0.033203125,
83
+ 0.271484375,
84
+ -0.12353515625,
85
+ 0.09716796875,
86
+ -0.1708984375,
87
+ 0.09814453125,
88
+ 0.0164794921875
89
  ]
90
  ],
91
  "position_norms": [
92
  [
93
+ 57.53282928466797,
94
+ 57.509952545166016,
95
+ 57.483131408691406,
96
+ 57.50147247314453,
97
+ 57.46410369873047
98
  ],
99
  [
100
+ 57.53282928466797,
101
+ 57.509952545166016,
102
+ 57.483131408691406,
103
+ 57.45440673828125,
104
+ 57.480262756347656
105
  ]
106
  ],
107
  "training": {
108
+ "loss": 11.817359924316406,
109
  "gradients": {
110
  "model.layers.0.self_attn.q_a_proj.weight": true,
111
  "model.layers.0.mlp.gate_proj.weight": true
integrations/transformers/expectations/cuda/DeepSeek-R1-mic.json CHANGED
@@ -2,26 +2,26 @@
2
  "recorded_with": {
3
  "contract": "huggingface-mic/DeepSeek-R1-mic",
4
  "contract_name": "DeepSeek-R1",
5
- "contract_revision": "73715aeb6e20225448964b4e5d761506d096d576",
6
  "framework": "transformers",
7
  "variant": "default",
8
  "target": "DeepSeek-R1-mic",
9
  "kind": "fixture",
10
- "revision": "73715aeb6e20225448964b4e5d761506d096d576",
11
  "source_model": "deepseek-ai/DeepSeek-R1",
12
  "weights": "random",
13
  "fixture_repo": "huggingface-mic/DeepSeek-R1-mic",
14
  "environment": {
15
  "python": "3.11.18",
16
- "transformers": "5.23.0.dev0",
17
  "sentence-transformers": null,
18
  "torch": "2.15.0+cu132",
19
  "torchvision": "0.29.2+cu132",
20
  "torchaudio": "2.11.1+cu132",
21
  "tokenizers": "0.23.3",
22
- "numpy": "2.4.6",
23
  "pillow": "13.0.0",
24
- "accelerate": "1.16.0.dev0",
25
  "safetensors": "0.8.0",
26
  "huggingface-hub": "2.4.0",
27
  "sentencepiece": "0.2.3",
@@ -30,14 +30,10 @@
30
  "soundfile": "0.14.0",
31
  "scipy": "1.17.1"
32
  },
33
- "seconds": 8.11,
34
  "environment_differs": {
35
- "torch": "2.15.0+cu132 (pinned 2.14.0)",
36
- "torchvision": "0.29.2+cu132 (pinned 0.29.1)",
37
- "torchaudio": "2.11.1+cu132 (pinned 2.11.0)",
38
- "numpy": "2.4.6 (pinned 2.3.5)",
39
  "pillow": "13.0.0 (pinned 12.3.0)",
40
- "accelerate": "1.16.0.dev0 (pinned 1.15.0)",
41
  "huggingface-hub": "2.4.0 (pinned 2.3.0)",
42
  "sentencepiece": "0.2.3 (pinned 0.2.2)"
43
  },
@@ -51,65 +47,65 @@
51
  "attn_implementation": "sdpa"
52
  },
53
  "next_token_ids": [
54
- 119704,
55
- 5475
56
  ],
57
  "probe_logits": [
58
  [
59
- -0.1796875,
60
- -0.1083984375,
61
- -0.1103515625,
62
- -0.166015625,
63
- 0.205078125,
64
- -0.177734375,
65
- 0.1025390625,
66
- 0.052734375,
67
- -0.00909423828125,
68
- 0.11572265625,
69
- -0.0625,
70
- -0.038818359375,
71
- -0.0927734375,
72
- 0.2265625,
73
- -0.10205078125,
74
- 0.068359375
75
  ],
76
  [
77
- -0.2119140625,
78
- -0.076171875,
79
- -0.0966796875,
80
- -0.14453125,
81
- 0.2177734375,
82
- -0.267578125,
83
- 0.1435546875,
84
- 0.044921875,
85
- 0.004669189453125,
86
- 0.12890625,
87
- -0.1416015625,
88
- 0.01458740234375,
89
- -0.1416015625,
90
- 0.2041015625,
91
- -0.123046875,
92
- 0.08984375
93
  ]
94
  ],
95
  "position_norms": [
96
  [
97
- 57.44833755493164,
98
- 57.25283432006836,
99
- 57.43769073486328,
100
- 57.47720718383789,
101
- 57.3710823059082
102
  ],
103
  [
104
- 57.44833755493164,
105
- 57.25283432006836,
106
- 57.43769073486328,
107
- 57.50749969482422,
108
- 57.35316848754883
109
  ]
110
  ],
111
  "training": {
112
- "loss": 11.72237777709961,
113
  "gradients": {
114
  "model.layers.0.self_attn.q_a_proj.weight": true,
115
  "model.layers.0.mlp.gate_proj.weight": true
 
2
  "recorded_with": {
3
  "contract": "huggingface-mic/DeepSeek-R1-mic",
4
  "contract_name": "DeepSeek-R1",
5
+ "contract_revision": "2d3440f540cc19f64a1150859f418f0b15ae352d",
6
  "framework": "transformers",
7
  "variant": "default",
8
  "target": "DeepSeek-R1-mic",
9
  "kind": "fixture",
10
+ "revision": "2d3440f540cc19f64a1150859f418f0b15ae352d",
11
  "source_model": "deepseek-ai/DeepSeek-R1",
12
  "weights": "random",
13
  "fixture_repo": "huggingface-mic/DeepSeek-R1-mic",
14
  "environment": {
15
  "python": "3.11.18",
16
+ "transformers": "5.24.0.dev0",
17
  "sentence-transformers": null,
18
  "torch": "2.15.0+cu132",
19
  "torchvision": "0.29.2+cu132",
20
  "torchaudio": "2.11.1+cu132",
21
  "tokenizers": "0.23.3",
22
+ "numpy": "2.3.5",
23
  "pillow": "13.0.0",
24
+ "accelerate": "1.17.0.dev0",
25
  "safetensors": "0.8.0",
26
  "huggingface-hub": "2.4.0",
27
  "sentencepiece": "0.2.3",
 
30
  "soundfile": "0.14.0",
31
  "scipy": "1.17.1"
32
  },
33
+ "seconds": 8.08,
34
  "environment_differs": {
 
 
 
 
35
  "pillow": "13.0.0 (pinned 12.3.0)",
36
+ "accelerate": "1.17.0.dev0 (pinned 1.15.0)",
37
  "huggingface-hub": "2.4.0 (pinned 2.3.0)",
38
  "sentencepiece": "0.2.3 (pinned 0.2.2)"
39
  },
 
47
  "attn_implementation": "sdpa"
48
  },
49
  "next_token_ids": [
50
+ 5519,
51
+ 40890
52
  ],
53
  "probe_logits": [
54
  [
55
+ 0.00390625,
56
+ -0.34765625,
57
+ -0.07373046875,
58
+ 0.048583984375,
59
+ 0.06689453125,
60
+ -0.171875,
61
+ 0.12255859375,
62
+ 0.06494140625,
63
+ 0.03564453125,
64
+ -0.052490234375,
65
+ 0.28125,
66
+ -0.053466796875,
67
+ 0.12255859375,
68
+ -0.052490234375,
69
+ -0.00445556640625,
70
+ 0.12890625
71
  ],
72
  [
73
+ 0.062255859375,
74
+ -0.2578125,
75
+ -0.10546875,
76
+ 0.0419921875,
77
+ -0.046142578125,
78
+ -0.095703125,
79
+ 0.09130859375,
80
+ 0.177734375,
81
+ 0.02294921875,
82
+ 0.033447265625,
83
+ 0.271484375,
84
+ -0.12353515625,
85
+ 0.0986328125,
86
+ -0.171875,
87
+ 0.0986328125,
88
+ 0.0169677734375
89
  ]
90
  ],
91
  "position_norms": [
92
  [
93
+ 57.53282928466797,
94
+ 57.48799514770508,
95
+ 57.509742736816406,
96
+ 57.473533630371094,
97
+ 57.480316162109375
98
  ],
99
  [
100
+ 57.53282928466797,
101
+ 57.48799514770508,
102
+ 57.509742736816406,
103
+ 57.43843460083008,
104
+ 57.4747428894043
105
  ]
106
  ],
107
  "training": {
108
+ "loss": 11.817480087280273,
109
  "gradients": {
110
  "model.layers.0.self_attn.q_a_proj.weight": true,
111
  "model.layers.0.mlp.gate_proj.weight": true
model.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:91f350ced1ae466ada4f94007cf4dc61f10b550bcc7d571dd0f19180e14ca8c9
3
- size 33253128
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:d31b47e9bc2a35049a74e9c7154e140481b06ed5847da71a25c19a6ae0ffe95c
3
+ size 35643096