JIAlonglong commited on
Commit
26b3a9d
·
verified ·
1 Parent(s): ce025cf

Upload verified APM manifests and adapters

Browse files
README.md ADDED
@@ -0,0 +1,68 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: apache-2.0
3
+ library_name: pytorch
4
+ tags:
5
+ - robotics
6
+ - world-action-model
7
+ - model-distillation
8
+ - anchored-path-matching
9
+ ---
10
+
11
+ # Anchored Path Matching weights
12
+
13
+ Publication status: **uploading / incomplete**.
14
+
15
+ This repository stages target/EMA inference weights for *Distilling World
16
+ Action Models for Efficient Bimanual Manipulation via Anchored Path Matching*.
17
+ Formal APM matches the frozen teacher and student **instantaneous direction
18
+ fields** at the same detached student-reached state and time. It also uses a
19
+ restart endpoint anchor and anchor-priority gradient projection.
20
+
21
+ ## Audited artifacts
22
+
23
+ | ID | Storage | Paper status |
24
+ |---|---|---|
25
+ | `shared` | `bases/stage1-shared/model.safetensors` | exact |
26
+ | `continued` | Stage I base + `variants/continued` | exact |
27
+ | `original-no-field` | `bases/stage2-original-no-field/model.safetensors` | exact |
28
+ | `original-field-projected` | Stage II base + matching adapter | exact |
29
+ | `restart-no-field` | Stage II base + matching adapter | exact |
30
+ | `apm` | Stage II base + matching adapter | exact |
31
+ | `restart-field-no-projection` | not published | unresolved |
32
+
33
+ The unresolved row is intentionally absent: no immutable EMA checkpoint in the
34
+ supplied project tree matched the formal instantaneous-field, Clean-only,
35
+ routed-topology predicate. A finite-local Clean/Random checkpoint was found but
36
+ is classified as auxiliary, not relabeled as a paper checkpoint.
37
+
38
+ Every uploaded variant was materialized with its declared base and matched
39
+ against the immutable source EMA using a logical digest over tensor names,
40
+ dtypes, shapes, and raw bytes. See `manifest.json`, `audit.json`,
41
+ `checksums.sha256`, and `verification/`.
42
+
43
+ ## Guidance contract
44
+
45
+ - text-condition dropout probability: `0.1`
46
+ - teacher-target video/action CFG: `5 / 1`
47
+ - deployment video/action CFG: `5 / 1`
48
+ - guidance mode: composed at inference
49
+ - supported student budgets: `1`, `2`, and `4` steps
50
+
51
+ Text-condition dropout is not a guidance scale.
52
+
53
+ ## Materialization
54
+
55
+ The repository is not marked complete yet. Once the matching code revision is
56
+ public, install it and materialize a variant with:
57
+
58
+ ```bash
59
+ python tools/materialize.py \
60
+ --base bases/stage2-original-no-field/model.safetensors \
61
+ --artifact variants/apm \
62
+ --output apm-materialized
63
+ ```
64
+
65
+ Only target/EMA inference tensors are included. Online weights, optimizer and
66
+ scheduler states, trainer recovery state, datasets, simulator assets, and the
67
+ LingBot-VA teacher are not redistributed.
68
+
audit.json ADDED
@@ -0,0 +1,218 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "audits": [
3
+ {
4
+ "candidate_id": "shared-stage1-step2000",
5
+ "paper_status": "exact",
6
+ "reasons": [],
7
+ "requested_role": "shared",
8
+ "semantics": {
9
+ "anchor_source": "original",
10
+ "base_identity": null,
11
+ "data_domains": [
12
+ "Clean"
13
+ ],
14
+ "deployment_cfg": [
15
+ 5.0,
16
+ 1.0
17
+ ],
18
+ "field_enabled": true,
19
+ "field_target": "instantaneous",
20
+ "objective": "continued",
21
+ "projection": false,
22
+ "stage": 1,
23
+ "step": 2000,
24
+ "teacher_cfg": [
25
+ 5.0,
26
+ 1.0
27
+ ],
28
+ "text_dropout_probability": 0.1,
29
+ "topology": "attention_ffn_time_v1:rank128"
30
+ },
31
+ "source_sha256": "0408df97114d4be32f38d59e1be0b57455cd0f1c50c086140347df1fd88ed2bd"
32
+ },
33
+ {
34
+ "candidate_id": "continued-stage1-step4000",
35
+ "paper_status": "exact",
36
+ "reasons": [],
37
+ "requested_role": "continued",
38
+ "semantics": {
39
+ "anchor_source": "original",
40
+ "base_identity": null,
41
+ "data_domains": [
42
+ "Clean"
43
+ ],
44
+ "deployment_cfg": [
45
+ 5.0,
46
+ 1.0
47
+ ],
48
+ "field_enabled": true,
49
+ "field_target": "instantaneous",
50
+ "objective": "continued",
51
+ "projection": false,
52
+ "stage": 1,
53
+ "step": 4000,
54
+ "teacher_cfg": [
55
+ 5.0,
56
+ 1.0
57
+ ],
58
+ "text_dropout_probability": 0.1,
59
+ "topology": "attention_ffn_time_v1:rank128"
60
+ },
61
+ "source_sha256": "56b8718055a0634eb9d83d4717a461528cc419f2bc56fe788b49cbbc319d4ff8"
62
+ },
63
+ {
64
+ "candidate_id": "original-no-field-step2500",
65
+ "paper_status": "exact",
66
+ "reasons": [],
67
+ "requested_role": "original-no-field",
68
+ "semantics": {
69
+ "anchor_source": "original",
70
+ "base_identity": "0408df97114d4be32f38d59e1be0b57455cd0f1c50c086140347df1fd88ed2bd",
71
+ "data_domains": [
72
+ "Clean"
73
+ ],
74
+ "deployment_cfg": [
75
+ 5.0,
76
+ 1.0
77
+ ],
78
+ "field_enabled": false,
79
+ "field_target": "instantaneous",
80
+ "objective": "apm",
81
+ "projection": false,
82
+ "stage": 2,
83
+ "step": 2500,
84
+ "teacher_cfg": [
85
+ 5.0,
86
+ 1.0
87
+ ],
88
+ "text_dropout_probability": 0.1,
89
+ "topology": "frozen-stage1-routed-ffn-time-v1:r64/r64/t128"
90
+ },
91
+ "source_sha256": "26ec204e91980bb1984abaed6b1a356d8d48bcea3ae047539f0d71f13818b1dc"
92
+ },
93
+ {
94
+ "candidate_id": "original-field-step2500",
95
+ "paper_status": "exact",
96
+ "reasons": [],
97
+ "requested_role": "original-field-projected",
98
+ "semantics": {
99
+ "anchor_source": "original",
100
+ "base_identity": "0408df97114d4be32f38d59e1be0b57455cd0f1c50c086140347df1fd88ed2bd",
101
+ "data_domains": [
102
+ "Clean"
103
+ ],
104
+ "deployment_cfg": [
105
+ 5.0,
106
+ 1.0
107
+ ],
108
+ "field_enabled": true,
109
+ "field_target": "instantaneous",
110
+ "objective": "apm",
111
+ "projection": true,
112
+ "stage": 2,
113
+ "step": 2500,
114
+ "teacher_cfg": [
115
+ 5.0,
116
+ 1.0
117
+ ],
118
+ "text_dropout_probability": 0.1,
119
+ "topology": "frozen-stage1-routed-ffn-time-v1:r64/r64/t128"
120
+ },
121
+ "source_sha256": "709b042d7427676100bf1e64b1f04432dc1ae6e149e8d6c860ff8178cd56a57f"
122
+ },
123
+ {
124
+ "candidate_id": "restart-no-field-step5000",
125
+ "paper_status": "exact",
126
+ "reasons": [],
127
+ "requested_role": "restart-no-field",
128
+ "semantics": {
129
+ "anchor_source": "restart",
130
+ "base_identity": "0408df97114d4be32f38d59e1be0b57455cd0f1c50c086140347df1fd88ed2bd",
131
+ "data_domains": [
132
+ "Clean"
133
+ ],
134
+ "deployment_cfg": [
135
+ 5.0,
136
+ 1.0
137
+ ],
138
+ "field_enabled": false,
139
+ "field_target": "instantaneous",
140
+ "objective": "apm",
141
+ "projection": false,
142
+ "stage": 2,
143
+ "step": 5000,
144
+ "teacher_cfg": [
145
+ 5.0,
146
+ 1.0
147
+ ],
148
+ "text_dropout_probability": 0.1,
149
+ "topology": "frozen-stage1-routed-ffn-time-v1:r64/r64/t128"
150
+ },
151
+ "source_sha256": "c3664f6a0e7ff32f40c5651fbe4e9374c4b2fc73401226dd3600e3492de68faf"
152
+ },
153
+ {
154
+ "candidate_id": "restart-field-step5000",
155
+ "paper_status": "exact",
156
+ "reasons": [],
157
+ "requested_role": "apm",
158
+ "semantics": {
159
+ "anchor_source": "restart",
160
+ "base_identity": "0408df97114d4be32f38d59e1be0b57455cd0f1c50c086140347df1fd88ed2bd",
161
+ "data_domains": [
162
+ "Clean"
163
+ ],
164
+ "deployment_cfg": [
165
+ 5.0,
166
+ 1.0
167
+ ],
168
+ "field_enabled": true,
169
+ "field_target": "instantaneous",
170
+ "objective": "apm",
171
+ "projection": true,
172
+ "stage": 2,
173
+ "step": 5000,
174
+ "teacher_cfg": [
175
+ 5.0,
176
+ 1.0
177
+ ],
178
+ "text_dropout_probability": 0.1,
179
+ "topology": "frozen-stage1-routed-ffn-time-v1:r64/r64/t128"
180
+ },
181
+ "source_sha256": "e5d9e246af961259f86479e6baef2267d4fb39dea9bec54191cd1a170c0dcc0e"
182
+ },
183
+ {
184
+ "candidate_id": "restart-finite-local-random-mix-step10000",
185
+ "paper_status": "auxiliary",
186
+ "reasons": [
187
+ "candidate is not assigned to a formal paper role"
188
+ ],
189
+ "requested_role": "auxiliary-finite-local-random-mix",
190
+ "semantics": {
191
+ "anchor_source": "restart",
192
+ "base_identity": "0408df97114d4be32f38d59e1be0b57455cd0f1c50c086140347df1fd88ed2bd",
193
+ "data_domains": [
194
+ "Clean",
195
+ "Random"
196
+ ],
197
+ "deployment_cfg": [
198
+ 5.0,
199
+ 1.0
200
+ ],
201
+ "field_enabled": true,
202
+ "field_target": "finite_local",
203
+ "objective": "apm",
204
+ "projection": false,
205
+ "stage": 2,
206
+ "step": 10000,
207
+ "teacher_cfg": [
208
+ 5.0,
209
+ 1.0
210
+ ],
211
+ "text_dropout_probability": 0.1,
212
+ "topology": "attention_ffn_time_v1:rank128"
213
+ },
214
+ "source_sha256": "770ebded1dcf540d8ac61c98811e4d2aa2b7437d5becc096a2758d37983f590e"
215
+ }
216
+ ],
217
+ "schema": "apm.release-inventory.v1"
218
+ }
checksums.sha256 ADDED
@@ -0,0 +1,19 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ 8f98c003159e518e541706d41f2b719d99948079bab28b8f4c9b05b85ff04687 README.md
2
+ 393c2c01d8d935da201fab187b8d9575e14272f42161f661437180ab603bacc1 audit.json
3
+ 0408df97114d4be32f38d59e1be0b57455cd0f1c50c086140347df1fd88ed2bd bases/stage1-shared/model.safetensors
4
+ 26ec204e91980bb1984abaed6b1a356d8d48bcea3ae047539f0d71f13818b1dc bases/stage2-original-no-field/model.safetensors
5
+ 5eafa2f677584a7aa5f626795343d7f41b5ae6448ba980be363612327cbab363 manifest.json
6
+ 2950c573d75bb61685fe1f98297c16c3425d24a0037821b4855c09c2d5741500 tools/materialize.py
7
+ e33204cdfe5ab92a1911eb88fc75fdd74263cccd6d26d9e7abffe52429938b30 tools/verify.py
8
+ 9dbc507e289aac52d3329787b6c45848bae486b38e4002fde0efa9c42afabfb3 variants/apm/adapter_model.safetensors
9
+ 4237bf6da0f57f08a77f1317f68c2086c2caca43aff32494bacb6d244d3b0337 variants/apm/manifest.json
10
+ 4ec45e879f33053650de7d0bbd409130a85dc0785ac328876406e68ead2170de variants/continued/adapter_model.safetensors
11
+ 9662eb7ab7854ad92f7587fbe001f85172eb0ec257a382484707bce4bf0e70d4 variants/continued/manifest.json
12
+ b085f2d4c5fce4093ee237a69fe0482f6007b504e337d8057dee6e5247f33959 variants/original-field-projected/adapter_model.safetensors
13
+ 280955a0940dc71cdb1f87c8cdfa4caaadbf0e825d1f70c1ae60c629bf45ba43 variants/original-field-projected/manifest.json
14
+ cf4e29d30738641aac0522a241960201f2bcb8f25532de529c9ec73ba4056f56 variants/restart-no-field/adapter_model.safetensors
15
+ 966e1e0213b5f56ab971c5c9d45af6e45b4054545fe791d87214cfd6a71d2152 variants/restart-no-field/manifest.json
16
+ a3e73ad0d29b2f8344572ac3782adf289b2caab9e82313bdb0db7db53c1d3f3c verification/apm.json
17
+ d9d944b4f90e5dab8c38bc9d1ef2a6450e9c47e1dff21a3d1602c678d21b84fc verification/continued.json
18
+ f013626d19556d031045eb6347fccec8852a46e7dead16885fc2d198bd2c60c5 verification/original-field-projected.json
19
+ 4c25bd09be360e2c89dbd09a502f488d463f7592597cfa4f17c0025f52e19e7b verification/restart-no-field.json
manifest.json ADDED
@@ -0,0 +1,158 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "artifacts": [
3
+ {
4
+ "id": "shared",
5
+ "paper_status": "exact",
6
+ "path": "bases/stage1-shared/model.safetensors",
7
+ "verified": true
8
+ },
9
+ {
10
+ "base": "stage1-shared",
11
+ "id": "continued",
12
+ "paper_status": "exact",
13
+ "path": "variants/continued/adapter_model.safetensors",
14
+ "verified": true
15
+ },
16
+ {
17
+ "id": "original-no-field",
18
+ "paper_status": "exact",
19
+ "path": "bases/stage2-original-no-field/model.safetensors",
20
+ "verified": true
21
+ },
22
+ {
23
+ "base": "stage2-original-no-field",
24
+ "id": "original-field-projected",
25
+ "paper_status": "exact",
26
+ "path": "variants/original-field-projected/adapter_model.safetensors",
27
+ "verified": true
28
+ },
29
+ {
30
+ "base": "stage2-original-no-field",
31
+ "id": "restart-no-field",
32
+ "paper_status": "exact",
33
+ "path": "variants/restart-no-field/adapter_model.safetensors",
34
+ "verified": true
35
+ },
36
+ {
37
+ "base": "stage2-original-no-field",
38
+ "id": "apm",
39
+ "paper_status": "exact",
40
+ "path": "variants/apm/adapter_model.safetensors",
41
+ "verified": true
42
+ }
43
+ ],
44
+ "audit": "audit.json",
45
+ "auxiliary_candidates": [
46
+ {
47
+ "candidate_id": "restart-finite-local-random-mix-step10000",
48
+ "paper_status": "auxiliary",
49
+ "reason": "Finite-local target and Clean/Random training do not match the formal instantaneous-field ablation.",
50
+ "staged": false
51
+ }
52
+ ],
53
+ "bases": {
54
+ "stage1-shared": {
55
+ "artifact": "bases/stage1-shared/model.safetensors",
56
+ "logical_digest": "106fa72cc8adfebd3ce0d89561d9ec61a29e0fbc85620a0f7914ab314fbe746b",
57
+ "paper_role": "shared",
58
+ "size_bytes": 11715136148,
59
+ "source_sha256": "0408df97114d4be32f38d59e1be0b57455cd0f1c50c086140347df1fd88ed2bd"
60
+ },
61
+ "stage2-original-no-field": {
62
+ "artifact": "bases/stage2-original-no-field/model.safetensors",
63
+ "logical_digest": "6f9e4b3f291d4f00a718db798a2774fe1e918daab21f74155644536443c19357",
64
+ "paper_role": "original-no-field",
65
+ "size_bytes": 12297653092,
66
+ "source_sha256": "26ec204e91980bb1984abaed6b1a356d8d48bcea3ae047539f0d71f13818b1dc"
67
+ }
68
+ },
69
+ "code": {
70
+ "commit": "2c5637f",
71
+ "public": false,
72
+ "repository": "https://github.com/JIAlonglong/Anchored-Path-Matching"
73
+ },
74
+ "exact_roles": [
75
+ "shared",
76
+ "continued",
77
+ "original-no-field",
78
+ "original-field-projected",
79
+ "restart-no-field",
80
+ "apm"
81
+ ],
82
+ "field_target": "instantaneous",
83
+ "guidance": {
84
+ "deployment_cfg": [
85
+ 5.0,
86
+ 1.0
87
+ ],
88
+ "target_guidance_mode": "inference_composed",
89
+ "teacher_target_cfg": [
90
+ 5.0,
91
+ 1.0
92
+ ],
93
+ "text_dropout_probability": 0.1
94
+ },
95
+ "required_roles": [
96
+ "shared",
97
+ "continued",
98
+ "original-no-field",
99
+ "original-field-projected",
100
+ "restart-no-field",
101
+ "restart-field-no-projection",
102
+ "apm"
103
+ ],
104
+ "schema": "apm.hf-release.v1",
105
+ "status": "uploading",
106
+ "unresolved_roles": [
107
+ {
108
+ "reason": "No immutable EMA checkpoint matching the formal instantaneous-field, Clean-only, routed-topology predicate was found in the supplied project tree.",
109
+ "role": "restart-field-no-projection"
110
+ }
111
+ ],
112
+ "variants": {
113
+ "apm": {
114
+ "artifact": "variants/apm/adapter_model.safetensors",
115
+ "artifact_sha256": "9dbc507e289aac52d3329787b6c45848bae486b38e4002fde0efa9c42afabfb3",
116
+ "base_id": "stage2-original-no-field",
117
+ "logical_digest": "7778d12f803b392d93f2f9d1c5ecee883d5353de692de5ed21bf311799f324a2",
118
+ "paper_role": "apm",
119
+ "paper_status": "exact",
120
+ "size_bytes": 582514248,
121
+ "source_sha256": "e5d9e246af961259f86479e6baef2267d4fb39dea9bec54191cd1a170c0dcc0e",
122
+ "verification": "verification/apm.json"
123
+ },
124
+ "continued": {
125
+ "artifact": "variants/continued/adapter_model.safetensors",
126
+ "artifact_sha256": "4ec45e879f33053650de7d0bbd409130a85dc0785ac328876406e68ead2170de",
127
+ "base_id": "stage1-shared",
128
+ "logical_digest": "50b99e687204ef2f82b32d6a71eda67cbc472a98ee400094a3a7ab97c2b7289e",
129
+ "paper_role": "continued",
130
+ "paper_status": "exact",
131
+ "size_bytes": 1337529184,
132
+ "source_sha256": "56b8718055a0634eb9d83d4717a461528cc419f2bc56fe788b49cbbc319d4ff8",
133
+ "verification": "verification/continued.json"
134
+ },
135
+ "original-field-projected": {
136
+ "artifact": "variants/original-field-projected/adapter_model.safetensors",
137
+ "artifact_sha256": "b085f2d4c5fce4093ee237a69fe0482f6007b504e337d8057dee6e5247f33959",
138
+ "base_id": "stage2-original-no-field",
139
+ "logical_digest": "a3cb591e6deeee93ee2d6732b9c122310eaad2bd3fefc3b57a4b3ae6722e3f94",
140
+ "paper_role": "original-field-projected",
141
+ "paper_status": "exact",
142
+ "size_bytes": 582514248,
143
+ "source_sha256": "709b042d7427676100bf1e64b1f04432dc1ae6e149e8d6c860ff8178cd56a57f",
144
+ "verification": "verification/original-field-projected.json"
145
+ },
146
+ "restart-no-field": {
147
+ "artifact": "variants/restart-no-field/adapter_model.safetensors",
148
+ "artifact_sha256": "cf4e29d30738641aac0522a241960201f2bcb8f25532de529c9ec73ba4056f56",
149
+ "base_id": "stage2-original-no-field",
150
+ "logical_digest": "569409c1c901f2778056bf1899f90051141c59e047b9fd09a4ccc64d9147a9b0",
151
+ "paper_role": "restart-no-field",
152
+ "paper_status": "exact",
153
+ "size_bytes": 582514248,
154
+ "source_sha256": "c3664f6a0e7ff32f40c5651fbe4e9374c4b2fc73401226dd3600e3492de68faf",
155
+ "verification": "verification/restart-no-field.json"
156
+ }
157
+ }
158
+ }
tools/materialize.py ADDED
@@ -0,0 +1,9 @@
 
 
 
 
 
 
 
 
 
 
1
+ #!/usr/bin/env python3
2
+ """Materialize one APM base-plus-adapter artifact."""
3
+
4
+ from anchorstep.cli.materialize_weights import main
5
+
6
+
7
+ if __name__ == "__main__":
8
+ raise SystemExit(main())
9
+
tools/verify.py ADDED
@@ -0,0 +1,22 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ #!/usr/bin/env python3
2
+ """Verify a materialized APM artifact against an immutable expected state."""
3
+
4
+ import argparse
5
+ from pathlib import Path
6
+
7
+ from anchorstep.weights.verify import verify_logical_state
8
+
9
+
10
+ def main() -> int:
11
+ parser = argparse.ArgumentParser()
12
+ parser.add_argument("--base", type=Path, required=True)
13
+ parser.add_argument("--artifact", type=Path, required=True)
14
+ parser.add_argument("--expected", type=Path, required=True)
15
+ args = parser.parse_args()
16
+ report = verify_logical_state(args.base, args.artifact, args.expected)
17
+ print(report.materialized_digest)
18
+ return 0 if report.ok else 1
19
+
20
+
21
+ if __name__ == "__main__":
22
+ raise SystemExit(main())
variants/apm/adapter_model.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:9dbc507e289aac52d3329787b6c45848bae486b38e4002fde0efa9c42afabfb3
3
+ size 582514248
variants/apm/manifest.json ADDED
@@ -0,0 +1,1873 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "artifact_file": "adapter_model.safetensors",
3
+ "artifact_sha256": "9dbc507e289aac52d3329787b6c45848bae486b38e4002fde0efa9c42afabfb3",
4
+ "base_digest": "6f9e4b3f291d4f00a718db798a2774fe1e918daab21f74155644536443c19357",
5
+ "cfg_contract": {
6
+ "deployment_cfg": [
7
+ 5.0,
8
+ 1.0
9
+ ],
10
+ "target_guidance_mode": "inference_composed",
11
+ "teacher_target_cfg": [
12
+ 5.0,
13
+ 1.0
14
+ ],
15
+ "text_dropout_probability": 0.1
16
+ },
17
+ "keys": {
18
+ "blocks.0.ffn.net.0.proj.action.down.weight": {
19
+ "dtype": "F32",
20
+ "shape": [
21
+ 64,
22
+ 3072
23
+ ]
24
+ },
25
+ "blocks.0.ffn.net.0.proj.action.up.weight": {
26
+ "dtype": "F32",
27
+ "shape": [
28
+ 14336,
29
+ 64
30
+ ]
31
+ },
32
+ "blocks.0.ffn.net.0.proj.video.down.weight": {
33
+ "dtype": "F32",
34
+ "shape": [
35
+ 64,
36
+ 3072
37
+ ]
38
+ },
39
+ "blocks.0.ffn.net.0.proj.video.up.weight": {
40
+ "dtype": "F32",
41
+ "shape": [
42
+ 14336,
43
+ 64
44
+ ]
45
+ },
46
+ "blocks.0.ffn.net.2.action.down.weight": {
47
+ "dtype": "F32",
48
+ "shape": [
49
+ 64,
50
+ 14336
51
+ ]
52
+ },
53
+ "blocks.0.ffn.net.2.action.up.weight": {
54
+ "dtype": "F32",
55
+ "shape": [
56
+ 3072,
57
+ 64
58
+ ]
59
+ },
60
+ "blocks.0.ffn.net.2.video.down.weight": {
61
+ "dtype": "F32",
62
+ "shape": [
63
+ 64,
64
+ 14336
65
+ ]
66
+ },
67
+ "blocks.0.ffn.net.2.video.up.weight": {
68
+ "dtype": "F32",
69
+ "shape": [
70
+ 3072,
71
+ 64
72
+ ]
73
+ },
74
+ "blocks.1.ffn.net.0.proj.action.down.weight": {
75
+ "dtype": "F32",
76
+ "shape": [
77
+ 64,
78
+ 3072
79
+ ]
80
+ },
81
+ "blocks.1.ffn.net.0.proj.action.up.weight": {
82
+ "dtype": "F32",
83
+ "shape": [
84
+ 14336,
85
+ 64
86
+ ]
87
+ },
88
+ "blocks.1.ffn.net.0.proj.video.down.weight": {
89
+ "dtype": "F32",
90
+ "shape": [
91
+ 64,
92
+ 3072
93
+ ]
94
+ },
95
+ "blocks.1.ffn.net.0.proj.video.up.weight": {
96
+ "dtype": "F32",
97
+ "shape": [
98
+ 14336,
99
+ 64
100
+ ]
101
+ },
102
+ "blocks.1.ffn.net.2.action.down.weight": {
103
+ "dtype": "F32",
104
+ "shape": [
105
+ 64,
106
+ 14336
107
+ ]
108
+ },
109
+ "blocks.1.ffn.net.2.action.up.weight": {
110
+ "dtype": "F32",
111
+ "shape": [
112
+ 3072,
113
+ 64
114
+ ]
115
+ },
116
+ "blocks.1.ffn.net.2.video.down.weight": {
117
+ "dtype": "F32",
118
+ "shape": [
119
+ 64,
120
+ 14336
121
+ ]
122
+ },
123
+ "blocks.1.ffn.net.2.video.up.weight": {
124
+ "dtype": "F32",
125
+ "shape": [
126
+ 3072,
127
+ 64
128
+ ]
129
+ },
130
+ "blocks.10.ffn.net.0.proj.action.down.weight": {
131
+ "dtype": "F32",
132
+ "shape": [
133
+ 64,
134
+ 3072
135
+ ]
136
+ },
137
+ "blocks.10.ffn.net.0.proj.action.up.weight": {
138
+ "dtype": "F32",
139
+ "shape": [
140
+ 14336,
141
+ 64
142
+ ]
143
+ },
144
+ "blocks.10.ffn.net.0.proj.video.down.weight": {
145
+ "dtype": "F32",
146
+ "shape": [
147
+ 64,
148
+ 3072
149
+ ]
150
+ },
151
+ "blocks.10.ffn.net.0.proj.video.up.weight": {
152
+ "dtype": "F32",
153
+ "shape": [
154
+ 14336,
155
+ 64
156
+ ]
157
+ },
158
+ "blocks.10.ffn.net.2.action.down.weight": {
159
+ "dtype": "F32",
160
+ "shape": [
161
+ 64,
162
+ 14336
163
+ ]
164
+ },
165
+ "blocks.10.ffn.net.2.action.up.weight": {
166
+ "dtype": "F32",
167
+ "shape": [
168
+ 3072,
169
+ 64
170
+ ]
171
+ },
172
+ "blocks.10.ffn.net.2.video.down.weight": {
173
+ "dtype": "F32",
174
+ "shape": [
175
+ 64,
176
+ 14336
177
+ ]
178
+ },
179
+ "blocks.10.ffn.net.2.video.up.weight": {
180
+ "dtype": "F32",
181
+ "shape": [
182
+ 3072,
183
+ 64
184
+ ]
185
+ },
186
+ "blocks.11.ffn.net.0.proj.action.down.weight": {
187
+ "dtype": "F32",
188
+ "shape": [
189
+ 64,
190
+ 3072
191
+ ]
192
+ },
193
+ "blocks.11.ffn.net.0.proj.action.up.weight": {
194
+ "dtype": "F32",
195
+ "shape": [
196
+ 14336,
197
+ 64
198
+ ]
199
+ },
200
+ "blocks.11.ffn.net.0.proj.video.down.weight": {
201
+ "dtype": "F32",
202
+ "shape": [
203
+ 64,
204
+ 3072
205
+ ]
206
+ },
207
+ "blocks.11.ffn.net.0.proj.video.up.weight": {
208
+ "dtype": "F32",
209
+ "shape": [
210
+ 14336,
211
+ 64
212
+ ]
213
+ },
214
+ "blocks.11.ffn.net.2.action.down.weight": {
215
+ "dtype": "F32",
216
+ "shape": [
217
+ 64,
218
+ 14336
219
+ ]
220
+ },
221
+ "blocks.11.ffn.net.2.action.up.weight": {
222
+ "dtype": "F32",
223
+ "shape": [
224
+ 3072,
225
+ 64
226
+ ]
227
+ },
228
+ "blocks.11.ffn.net.2.video.down.weight": {
229
+ "dtype": "F32",
230
+ "shape": [
231
+ 64,
232
+ 14336
233
+ ]
234
+ },
235
+ "blocks.11.ffn.net.2.video.up.weight": {
236
+ "dtype": "F32",
237
+ "shape": [
238
+ 3072,
239
+ 64
240
+ ]
241
+ },
242
+ "blocks.12.ffn.net.0.proj.action.down.weight": {
243
+ "dtype": "F32",
244
+ "shape": [
245
+ 64,
246
+ 3072
247
+ ]
248
+ },
249
+ "blocks.12.ffn.net.0.proj.action.up.weight": {
250
+ "dtype": "F32",
251
+ "shape": [
252
+ 14336,
253
+ 64
254
+ ]
255
+ },
256
+ "blocks.12.ffn.net.0.proj.video.down.weight": {
257
+ "dtype": "F32",
258
+ "shape": [
259
+ 64,
260
+ 3072
261
+ ]
262
+ },
263
+ "blocks.12.ffn.net.0.proj.video.up.weight": {
264
+ "dtype": "F32",
265
+ "shape": [
266
+ 14336,
267
+ 64
268
+ ]
269
+ },
270
+ "blocks.12.ffn.net.2.action.down.weight": {
271
+ "dtype": "F32",
272
+ "shape": [
273
+ 64,
274
+ 14336
275
+ ]
276
+ },
277
+ "blocks.12.ffn.net.2.action.up.weight": {
278
+ "dtype": "F32",
279
+ "shape": [
280
+ 3072,
281
+ 64
282
+ ]
283
+ },
284
+ "blocks.12.ffn.net.2.video.down.weight": {
285
+ "dtype": "F32",
286
+ "shape": [
287
+ 64,
288
+ 14336
289
+ ]
290
+ },
291
+ "blocks.12.ffn.net.2.video.up.weight": {
292
+ "dtype": "F32",
293
+ "shape": [
294
+ 3072,
295
+ 64
296
+ ]
297
+ },
298
+ "blocks.13.ffn.net.0.proj.action.down.weight": {
299
+ "dtype": "F32",
300
+ "shape": [
301
+ 64,
302
+ 3072
303
+ ]
304
+ },
305
+ "blocks.13.ffn.net.0.proj.action.up.weight": {
306
+ "dtype": "F32",
307
+ "shape": [
308
+ 14336,
309
+ 64
310
+ ]
311
+ },
312
+ "blocks.13.ffn.net.0.proj.video.down.weight": {
313
+ "dtype": "F32",
314
+ "shape": [
315
+ 64,
316
+ 3072
317
+ ]
318
+ },
319
+ "blocks.13.ffn.net.0.proj.video.up.weight": {
320
+ "dtype": "F32",
321
+ "shape": [
322
+ 14336,
323
+ 64
324
+ ]
325
+ },
326
+ "blocks.13.ffn.net.2.action.down.weight": {
327
+ "dtype": "F32",
328
+ "shape": [
329
+ 64,
330
+ 14336
331
+ ]
332
+ },
333
+ "blocks.13.ffn.net.2.action.up.weight": {
334
+ "dtype": "F32",
335
+ "shape": [
336
+ 3072,
337
+ 64
338
+ ]
339
+ },
340
+ "blocks.13.ffn.net.2.video.down.weight": {
341
+ "dtype": "F32",
342
+ "shape": [
343
+ 64,
344
+ 14336
345
+ ]
346
+ },
347
+ "blocks.13.ffn.net.2.video.up.weight": {
348
+ "dtype": "F32",
349
+ "shape": [
350
+ 3072,
351
+ 64
352
+ ]
353
+ },
354
+ "blocks.14.ffn.net.0.proj.action.down.weight": {
355
+ "dtype": "F32",
356
+ "shape": [
357
+ 64,
358
+ 3072
359
+ ]
360
+ },
361
+ "blocks.14.ffn.net.0.proj.action.up.weight": {
362
+ "dtype": "F32",
363
+ "shape": [
364
+ 14336,
365
+ 64
366
+ ]
367
+ },
368
+ "blocks.14.ffn.net.0.proj.video.down.weight": {
369
+ "dtype": "F32",
370
+ "shape": [
371
+ 64,
372
+ 3072
373
+ ]
374
+ },
375
+ "blocks.14.ffn.net.0.proj.video.up.weight": {
376
+ "dtype": "F32",
377
+ "shape": [
378
+ 14336,
379
+ 64
380
+ ]
381
+ },
382
+ "blocks.14.ffn.net.2.action.down.weight": {
383
+ "dtype": "F32",
384
+ "shape": [
385
+ 64,
386
+ 14336
387
+ ]
388
+ },
389
+ "blocks.14.ffn.net.2.action.up.weight": {
390
+ "dtype": "F32",
391
+ "shape": [
392
+ 3072,
393
+ 64
394
+ ]
395
+ },
396
+ "blocks.14.ffn.net.2.video.down.weight": {
397
+ "dtype": "F32",
398
+ "shape": [
399
+ 64,
400
+ 14336
401
+ ]
402
+ },
403
+ "blocks.14.ffn.net.2.video.up.weight": {
404
+ "dtype": "F32",
405
+ "shape": [
406
+ 3072,
407
+ 64
408
+ ]
409
+ },
410
+ "blocks.15.ffn.net.0.proj.action.down.weight": {
411
+ "dtype": "F32",
412
+ "shape": [
413
+ 64,
414
+ 3072
415
+ ]
416
+ },
417
+ "blocks.15.ffn.net.0.proj.action.up.weight": {
418
+ "dtype": "F32",
419
+ "shape": [
420
+ 14336,
421
+ 64
422
+ ]
423
+ },
424
+ "blocks.15.ffn.net.0.proj.video.down.weight": {
425
+ "dtype": "F32",
426
+ "shape": [
427
+ 64,
428
+ 3072
429
+ ]
430
+ },
431
+ "blocks.15.ffn.net.0.proj.video.up.weight": {
432
+ "dtype": "F32",
433
+ "shape": [
434
+ 14336,
435
+ 64
436
+ ]
437
+ },
438
+ "blocks.15.ffn.net.2.action.down.weight": {
439
+ "dtype": "F32",
440
+ "shape": [
441
+ 64,
442
+ 14336
443
+ ]
444
+ },
445
+ "blocks.15.ffn.net.2.action.up.weight": {
446
+ "dtype": "F32",
447
+ "shape": [
448
+ 3072,
449
+ 64
450
+ ]
451
+ },
452
+ "blocks.15.ffn.net.2.video.down.weight": {
453
+ "dtype": "F32",
454
+ "shape": [
455
+ 64,
456
+ 14336
457
+ ]
458
+ },
459
+ "blocks.15.ffn.net.2.video.up.weight": {
460
+ "dtype": "F32",
461
+ "shape": [
462
+ 3072,
463
+ 64
464
+ ]
465
+ },
466
+ "blocks.16.ffn.net.0.proj.action.down.weight": {
467
+ "dtype": "F32",
468
+ "shape": [
469
+ 64,
470
+ 3072
471
+ ]
472
+ },
473
+ "blocks.16.ffn.net.0.proj.action.up.weight": {
474
+ "dtype": "F32",
475
+ "shape": [
476
+ 14336,
477
+ 64
478
+ ]
479
+ },
480
+ "blocks.16.ffn.net.0.proj.video.down.weight": {
481
+ "dtype": "F32",
482
+ "shape": [
483
+ 64,
484
+ 3072
485
+ ]
486
+ },
487
+ "blocks.16.ffn.net.0.proj.video.up.weight": {
488
+ "dtype": "F32",
489
+ "shape": [
490
+ 14336,
491
+ 64
492
+ ]
493
+ },
494
+ "blocks.16.ffn.net.2.action.down.weight": {
495
+ "dtype": "F32",
496
+ "shape": [
497
+ 64,
498
+ 14336
499
+ ]
500
+ },
501
+ "blocks.16.ffn.net.2.action.up.weight": {
502
+ "dtype": "F32",
503
+ "shape": [
504
+ 3072,
505
+ 64
506
+ ]
507
+ },
508
+ "blocks.16.ffn.net.2.video.down.weight": {
509
+ "dtype": "F32",
510
+ "shape": [
511
+ 64,
512
+ 14336
513
+ ]
514
+ },
515
+ "blocks.16.ffn.net.2.video.up.weight": {
516
+ "dtype": "F32",
517
+ "shape": [
518
+ 3072,
519
+ 64
520
+ ]
521
+ },
522
+ "blocks.17.ffn.net.0.proj.action.down.weight": {
523
+ "dtype": "F32",
524
+ "shape": [
525
+ 64,
526
+ 3072
527
+ ]
528
+ },
529
+ "blocks.17.ffn.net.0.proj.action.up.weight": {
530
+ "dtype": "F32",
531
+ "shape": [
532
+ 14336,
533
+ 64
534
+ ]
535
+ },
536
+ "blocks.17.ffn.net.0.proj.video.down.weight": {
537
+ "dtype": "F32",
538
+ "shape": [
539
+ 64,
540
+ 3072
541
+ ]
542
+ },
543
+ "blocks.17.ffn.net.0.proj.video.up.weight": {
544
+ "dtype": "F32",
545
+ "shape": [
546
+ 14336,
547
+ 64
548
+ ]
549
+ },
550
+ "blocks.17.ffn.net.2.action.down.weight": {
551
+ "dtype": "F32",
552
+ "shape": [
553
+ 64,
554
+ 14336
555
+ ]
556
+ },
557
+ "blocks.17.ffn.net.2.action.up.weight": {
558
+ "dtype": "F32",
559
+ "shape": [
560
+ 3072,
561
+ 64
562
+ ]
563
+ },
564
+ "blocks.17.ffn.net.2.video.down.weight": {
565
+ "dtype": "F32",
566
+ "shape": [
567
+ 64,
568
+ 14336
569
+ ]
570
+ },
571
+ "blocks.17.ffn.net.2.video.up.weight": {
572
+ "dtype": "F32",
573
+ "shape": [
574
+ 3072,
575
+ 64
576
+ ]
577
+ },
578
+ "blocks.18.ffn.net.0.proj.action.down.weight": {
579
+ "dtype": "F32",
580
+ "shape": [
581
+ 64,
582
+ 3072
583
+ ]
584
+ },
585
+ "blocks.18.ffn.net.0.proj.action.up.weight": {
586
+ "dtype": "F32",
587
+ "shape": [
588
+ 14336,
589
+ 64
590
+ ]
591
+ },
592
+ "blocks.18.ffn.net.0.proj.video.down.weight": {
593
+ "dtype": "F32",
594
+ "shape": [
595
+ 64,
596
+ 3072
597
+ ]
598
+ },
599
+ "blocks.18.ffn.net.0.proj.video.up.weight": {
600
+ "dtype": "F32",
601
+ "shape": [
602
+ 14336,
603
+ 64
604
+ ]
605
+ },
606
+ "blocks.18.ffn.net.2.action.down.weight": {
607
+ "dtype": "F32",
608
+ "shape": [
609
+ 64,
610
+ 14336
611
+ ]
612
+ },
613
+ "blocks.18.ffn.net.2.action.up.weight": {
614
+ "dtype": "F32",
615
+ "shape": [
616
+ 3072,
617
+ 64
618
+ ]
619
+ },
620
+ "blocks.18.ffn.net.2.video.down.weight": {
621
+ "dtype": "F32",
622
+ "shape": [
623
+ 64,
624
+ 14336
625
+ ]
626
+ },
627
+ "blocks.18.ffn.net.2.video.up.weight": {
628
+ "dtype": "F32",
629
+ "shape": [
630
+ 3072,
631
+ 64
632
+ ]
633
+ },
634
+ "blocks.19.ffn.net.0.proj.action.down.weight": {
635
+ "dtype": "F32",
636
+ "shape": [
637
+ 64,
638
+ 3072
639
+ ]
640
+ },
641
+ "blocks.19.ffn.net.0.proj.action.up.weight": {
642
+ "dtype": "F32",
643
+ "shape": [
644
+ 14336,
645
+ 64
646
+ ]
647
+ },
648
+ "blocks.19.ffn.net.0.proj.video.down.weight": {
649
+ "dtype": "F32",
650
+ "shape": [
651
+ 64,
652
+ 3072
653
+ ]
654
+ },
655
+ "blocks.19.ffn.net.0.proj.video.up.weight": {
656
+ "dtype": "F32",
657
+ "shape": [
658
+ 14336,
659
+ 64
660
+ ]
661
+ },
662
+ "blocks.19.ffn.net.2.action.down.weight": {
663
+ "dtype": "F32",
664
+ "shape": [
665
+ 64,
666
+ 14336
667
+ ]
668
+ },
669
+ "blocks.19.ffn.net.2.action.up.weight": {
670
+ "dtype": "F32",
671
+ "shape": [
672
+ 3072,
673
+ 64
674
+ ]
675
+ },
676
+ "blocks.19.ffn.net.2.video.down.weight": {
677
+ "dtype": "F32",
678
+ "shape": [
679
+ 64,
680
+ 14336
681
+ ]
682
+ },
683
+ "blocks.19.ffn.net.2.video.up.weight": {
684
+ "dtype": "F32",
685
+ "shape": [
686
+ 3072,
687
+ 64
688
+ ]
689
+ },
690
+ "blocks.2.ffn.net.0.proj.action.down.weight": {
691
+ "dtype": "F32",
692
+ "shape": [
693
+ 64,
694
+ 3072
695
+ ]
696
+ },
697
+ "blocks.2.ffn.net.0.proj.action.up.weight": {
698
+ "dtype": "F32",
699
+ "shape": [
700
+ 14336,
701
+ 64
702
+ ]
703
+ },
704
+ "blocks.2.ffn.net.0.proj.video.down.weight": {
705
+ "dtype": "F32",
706
+ "shape": [
707
+ 64,
708
+ 3072
709
+ ]
710
+ },
711
+ "blocks.2.ffn.net.0.proj.video.up.weight": {
712
+ "dtype": "F32",
713
+ "shape": [
714
+ 14336,
715
+ 64
716
+ ]
717
+ },
718
+ "blocks.2.ffn.net.2.action.down.weight": {
719
+ "dtype": "F32",
720
+ "shape": [
721
+ 64,
722
+ 14336
723
+ ]
724
+ },
725
+ "blocks.2.ffn.net.2.action.up.weight": {
726
+ "dtype": "F32",
727
+ "shape": [
728
+ 3072,
729
+ 64
730
+ ]
731
+ },
732
+ "blocks.2.ffn.net.2.video.down.weight": {
733
+ "dtype": "F32",
734
+ "shape": [
735
+ 64,
736
+ 14336
737
+ ]
738
+ },
739
+ "blocks.2.ffn.net.2.video.up.weight": {
740
+ "dtype": "F32",
741
+ "shape": [
742
+ 3072,
743
+ 64
744
+ ]
745
+ },
746
+ "blocks.20.ffn.net.0.proj.action.down.weight": {
747
+ "dtype": "F32",
748
+ "shape": [
749
+ 64,
750
+ 3072
751
+ ]
752
+ },
753
+ "blocks.20.ffn.net.0.proj.action.up.weight": {
754
+ "dtype": "F32",
755
+ "shape": [
756
+ 14336,
757
+ 64
758
+ ]
759
+ },
760
+ "blocks.20.ffn.net.0.proj.video.down.weight": {
761
+ "dtype": "F32",
762
+ "shape": [
763
+ 64,
764
+ 3072
765
+ ]
766
+ },
767
+ "blocks.20.ffn.net.0.proj.video.up.weight": {
768
+ "dtype": "F32",
769
+ "shape": [
770
+ 14336,
771
+ 64
772
+ ]
773
+ },
774
+ "blocks.20.ffn.net.2.action.down.weight": {
775
+ "dtype": "F32",
776
+ "shape": [
777
+ 64,
778
+ 14336
779
+ ]
780
+ },
781
+ "blocks.20.ffn.net.2.action.up.weight": {
782
+ "dtype": "F32",
783
+ "shape": [
784
+ 3072,
785
+ 64
786
+ ]
787
+ },
788
+ "blocks.20.ffn.net.2.video.down.weight": {
789
+ "dtype": "F32",
790
+ "shape": [
791
+ 64,
792
+ 14336
793
+ ]
794
+ },
795
+ "blocks.20.ffn.net.2.video.up.weight": {
796
+ "dtype": "F32",
797
+ "shape": [
798
+ 3072,
799
+ 64
800
+ ]
801
+ },
802
+ "blocks.21.ffn.net.0.proj.action.down.weight": {
803
+ "dtype": "F32",
804
+ "shape": [
805
+ 64,
806
+ 3072
807
+ ]
808
+ },
809
+ "blocks.21.ffn.net.0.proj.action.up.weight": {
810
+ "dtype": "F32",
811
+ "shape": [
812
+ 14336,
813
+ 64
814
+ ]
815
+ },
816
+ "blocks.21.ffn.net.0.proj.video.down.weight": {
817
+ "dtype": "F32",
818
+ "shape": [
819
+ 64,
820
+ 3072
821
+ ]
822
+ },
823
+ "blocks.21.ffn.net.0.proj.video.up.weight": {
824
+ "dtype": "F32",
825
+ "shape": [
826
+ 14336,
827
+ 64
828
+ ]
829
+ },
830
+ "blocks.21.ffn.net.2.action.down.weight": {
831
+ "dtype": "F32",
832
+ "shape": [
833
+ 64,
834
+ 14336
835
+ ]
836
+ },
837
+ "blocks.21.ffn.net.2.action.up.weight": {
838
+ "dtype": "F32",
839
+ "shape": [
840
+ 3072,
841
+ 64
842
+ ]
843
+ },
844
+ "blocks.21.ffn.net.2.video.down.weight": {
845
+ "dtype": "F32",
846
+ "shape": [
847
+ 64,
848
+ 14336
849
+ ]
850
+ },
851
+ "blocks.21.ffn.net.2.video.up.weight": {
852
+ "dtype": "F32",
853
+ "shape": [
854
+ 3072,
855
+ 64
856
+ ]
857
+ },
858
+ "blocks.22.ffn.net.0.proj.action.down.weight": {
859
+ "dtype": "F32",
860
+ "shape": [
861
+ 64,
862
+ 3072
863
+ ]
864
+ },
865
+ "blocks.22.ffn.net.0.proj.action.up.weight": {
866
+ "dtype": "F32",
867
+ "shape": [
868
+ 14336,
869
+ 64
870
+ ]
871
+ },
872
+ "blocks.22.ffn.net.0.proj.video.down.weight": {
873
+ "dtype": "F32",
874
+ "shape": [
875
+ 64,
876
+ 3072
877
+ ]
878
+ },
879
+ "blocks.22.ffn.net.0.proj.video.up.weight": {
880
+ "dtype": "F32",
881
+ "shape": [
882
+ 14336,
883
+ 64
884
+ ]
885
+ },
886
+ "blocks.22.ffn.net.2.action.down.weight": {
887
+ "dtype": "F32",
888
+ "shape": [
889
+ 64,
890
+ 14336
891
+ ]
892
+ },
893
+ "blocks.22.ffn.net.2.action.up.weight": {
894
+ "dtype": "F32",
895
+ "shape": [
896
+ 3072,
897
+ 64
898
+ ]
899
+ },
900
+ "blocks.22.ffn.net.2.video.down.weight": {
901
+ "dtype": "F32",
902
+ "shape": [
903
+ 64,
904
+ 14336
905
+ ]
906
+ },
907
+ "blocks.22.ffn.net.2.video.up.weight": {
908
+ "dtype": "F32",
909
+ "shape": [
910
+ 3072,
911
+ 64
912
+ ]
913
+ },
914
+ "blocks.23.ffn.net.0.proj.action.down.weight": {
915
+ "dtype": "F32",
916
+ "shape": [
917
+ 64,
918
+ 3072
919
+ ]
920
+ },
921
+ "blocks.23.ffn.net.0.proj.action.up.weight": {
922
+ "dtype": "F32",
923
+ "shape": [
924
+ 14336,
925
+ 64
926
+ ]
927
+ },
928
+ "blocks.23.ffn.net.0.proj.video.down.weight": {
929
+ "dtype": "F32",
930
+ "shape": [
931
+ 64,
932
+ 3072
933
+ ]
934
+ },
935
+ "blocks.23.ffn.net.0.proj.video.up.weight": {
936
+ "dtype": "F32",
937
+ "shape": [
938
+ 14336,
939
+ 64
940
+ ]
941
+ },
942
+ "blocks.23.ffn.net.2.action.down.weight": {
943
+ "dtype": "F32",
944
+ "shape": [
945
+ 64,
946
+ 14336
947
+ ]
948
+ },
949
+ "blocks.23.ffn.net.2.action.up.weight": {
950
+ "dtype": "F32",
951
+ "shape": [
952
+ 3072,
953
+ 64
954
+ ]
955
+ },
956
+ "blocks.23.ffn.net.2.video.down.weight": {
957
+ "dtype": "F32",
958
+ "shape": [
959
+ 64,
960
+ 14336
961
+ ]
962
+ },
963
+ "blocks.23.ffn.net.2.video.up.weight": {
964
+ "dtype": "F32",
965
+ "shape": [
966
+ 3072,
967
+ 64
968
+ ]
969
+ },
970
+ "blocks.24.ffn.net.0.proj.action.down.weight": {
971
+ "dtype": "F32",
972
+ "shape": [
973
+ 64,
974
+ 3072
975
+ ]
976
+ },
977
+ "blocks.24.ffn.net.0.proj.action.up.weight": {
978
+ "dtype": "F32",
979
+ "shape": [
980
+ 14336,
981
+ 64
982
+ ]
983
+ },
984
+ "blocks.24.ffn.net.0.proj.video.down.weight": {
985
+ "dtype": "F32",
986
+ "shape": [
987
+ 64,
988
+ 3072
989
+ ]
990
+ },
991
+ "blocks.24.ffn.net.0.proj.video.up.weight": {
992
+ "dtype": "F32",
993
+ "shape": [
994
+ 14336,
995
+ 64
996
+ ]
997
+ },
998
+ "blocks.24.ffn.net.2.action.down.weight": {
999
+ "dtype": "F32",
1000
+ "shape": [
1001
+ 64,
1002
+ 14336
1003
+ ]
1004
+ },
1005
+ "blocks.24.ffn.net.2.action.up.weight": {
1006
+ "dtype": "F32",
1007
+ "shape": [
1008
+ 3072,
1009
+ 64
1010
+ ]
1011
+ },
1012
+ "blocks.24.ffn.net.2.video.down.weight": {
1013
+ "dtype": "F32",
1014
+ "shape": [
1015
+ 64,
1016
+ 14336
1017
+ ]
1018
+ },
1019
+ "blocks.24.ffn.net.2.video.up.weight": {
1020
+ "dtype": "F32",
1021
+ "shape": [
1022
+ 3072,
1023
+ 64
1024
+ ]
1025
+ },
1026
+ "blocks.25.ffn.net.0.proj.action.down.weight": {
1027
+ "dtype": "F32",
1028
+ "shape": [
1029
+ 64,
1030
+ 3072
1031
+ ]
1032
+ },
1033
+ "blocks.25.ffn.net.0.proj.action.up.weight": {
1034
+ "dtype": "F32",
1035
+ "shape": [
1036
+ 14336,
1037
+ 64
1038
+ ]
1039
+ },
1040
+ "blocks.25.ffn.net.0.proj.video.down.weight": {
1041
+ "dtype": "F32",
1042
+ "shape": [
1043
+ 64,
1044
+ 3072
1045
+ ]
1046
+ },
1047
+ "blocks.25.ffn.net.0.proj.video.up.weight": {
1048
+ "dtype": "F32",
1049
+ "shape": [
1050
+ 14336,
1051
+ 64
1052
+ ]
1053
+ },
1054
+ "blocks.25.ffn.net.2.action.down.weight": {
1055
+ "dtype": "F32",
1056
+ "shape": [
1057
+ 64,
1058
+ 14336
1059
+ ]
1060
+ },
1061
+ "blocks.25.ffn.net.2.action.up.weight": {
1062
+ "dtype": "F32",
1063
+ "shape": [
1064
+ 3072,
1065
+ 64
1066
+ ]
1067
+ },
1068
+ "blocks.25.ffn.net.2.video.down.weight": {
1069
+ "dtype": "F32",
1070
+ "shape": [
1071
+ 64,
1072
+ 14336
1073
+ ]
1074
+ },
1075
+ "blocks.25.ffn.net.2.video.up.weight": {
1076
+ "dtype": "F32",
1077
+ "shape": [
1078
+ 3072,
1079
+ 64
1080
+ ]
1081
+ },
1082
+ "blocks.26.ffn.net.0.proj.action.down.weight": {
1083
+ "dtype": "F32",
1084
+ "shape": [
1085
+ 64,
1086
+ 3072
1087
+ ]
1088
+ },
1089
+ "blocks.26.ffn.net.0.proj.action.up.weight": {
1090
+ "dtype": "F32",
1091
+ "shape": [
1092
+ 14336,
1093
+ 64
1094
+ ]
1095
+ },
1096
+ "blocks.26.ffn.net.0.proj.video.down.weight": {
1097
+ "dtype": "F32",
1098
+ "shape": [
1099
+ 64,
1100
+ 3072
1101
+ ]
1102
+ },
1103
+ "blocks.26.ffn.net.0.proj.video.up.weight": {
1104
+ "dtype": "F32",
1105
+ "shape": [
1106
+ 14336,
1107
+ 64
1108
+ ]
1109
+ },
1110
+ "blocks.26.ffn.net.2.action.down.weight": {
1111
+ "dtype": "F32",
1112
+ "shape": [
1113
+ 64,
1114
+ 14336
1115
+ ]
1116
+ },
1117
+ "blocks.26.ffn.net.2.action.up.weight": {
1118
+ "dtype": "F32",
1119
+ "shape": [
1120
+ 3072,
1121
+ 64
1122
+ ]
1123
+ },
1124
+ "blocks.26.ffn.net.2.video.down.weight": {
1125
+ "dtype": "F32",
1126
+ "shape": [
1127
+ 64,
1128
+ 14336
1129
+ ]
1130
+ },
1131
+ "blocks.26.ffn.net.2.video.up.weight": {
1132
+ "dtype": "F32",
1133
+ "shape": [
1134
+ 3072,
1135
+ 64
1136
+ ]
1137
+ },
1138
+ "blocks.27.ffn.net.0.proj.action.down.weight": {
1139
+ "dtype": "F32",
1140
+ "shape": [
1141
+ 64,
1142
+ 3072
1143
+ ]
1144
+ },
1145
+ "blocks.27.ffn.net.0.proj.action.up.weight": {
1146
+ "dtype": "F32",
1147
+ "shape": [
1148
+ 14336,
1149
+ 64
1150
+ ]
1151
+ },
1152
+ "blocks.27.ffn.net.0.proj.video.down.weight": {
1153
+ "dtype": "F32",
1154
+ "shape": [
1155
+ 64,
1156
+ 3072
1157
+ ]
1158
+ },
1159
+ "blocks.27.ffn.net.0.proj.video.up.weight": {
1160
+ "dtype": "F32",
1161
+ "shape": [
1162
+ 14336,
1163
+ 64
1164
+ ]
1165
+ },
1166
+ "blocks.27.ffn.net.2.action.down.weight": {
1167
+ "dtype": "F32",
1168
+ "shape": [
1169
+ 64,
1170
+ 14336
1171
+ ]
1172
+ },
1173
+ "blocks.27.ffn.net.2.action.up.weight": {
1174
+ "dtype": "F32",
1175
+ "shape": [
1176
+ 3072,
1177
+ 64
1178
+ ]
1179
+ },
1180
+ "blocks.27.ffn.net.2.video.down.weight": {
1181
+ "dtype": "F32",
1182
+ "shape": [
1183
+ 64,
1184
+ 14336
1185
+ ]
1186
+ },
1187
+ "blocks.27.ffn.net.2.video.up.weight": {
1188
+ "dtype": "F32",
1189
+ "shape": [
1190
+ 3072,
1191
+ 64
1192
+ ]
1193
+ },
1194
+ "blocks.28.ffn.net.0.proj.action.down.weight": {
1195
+ "dtype": "F32",
1196
+ "shape": [
1197
+ 64,
1198
+ 3072
1199
+ ]
1200
+ },
1201
+ "blocks.28.ffn.net.0.proj.action.up.weight": {
1202
+ "dtype": "F32",
1203
+ "shape": [
1204
+ 14336,
1205
+ 64
1206
+ ]
1207
+ },
1208
+ "blocks.28.ffn.net.0.proj.video.down.weight": {
1209
+ "dtype": "F32",
1210
+ "shape": [
1211
+ 64,
1212
+ 3072
1213
+ ]
1214
+ },
1215
+ "blocks.28.ffn.net.0.proj.video.up.weight": {
1216
+ "dtype": "F32",
1217
+ "shape": [
1218
+ 14336,
1219
+ 64
1220
+ ]
1221
+ },
1222
+ "blocks.28.ffn.net.2.action.down.weight": {
1223
+ "dtype": "F32",
1224
+ "shape": [
1225
+ 64,
1226
+ 14336
1227
+ ]
1228
+ },
1229
+ "blocks.28.ffn.net.2.action.up.weight": {
1230
+ "dtype": "F32",
1231
+ "shape": [
1232
+ 3072,
1233
+ 64
1234
+ ]
1235
+ },
1236
+ "blocks.28.ffn.net.2.video.down.weight": {
1237
+ "dtype": "F32",
1238
+ "shape": [
1239
+ 64,
1240
+ 14336
1241
+ ]
1242
+ },
1243
+ "blocks.28.ffn.net.2.video.up.weight": {
1244
+ "dtype": "F32",
1245
+ "shape": [
1246
+ 3072,
1247
+ 64
1248
+ ]
1249
+ },
1250
+ "blocks.29.ffn.net.0.proj.action.down.weight": {
1251
+ "dtype": "F32",
1252
+ "shape": [
1253
+ 64,
1254
+ 3072
1255
+ ]
1256
+ },
1257
+ "blocks.29.ffn.net.0.proj.action.up.weight": {
1258
+ "dtype": "F32",
1259
+ "shape": [
1260
+ 14336,
1261
+ 64
1262
+ ]
1263
+ },
1264
+ "blocks.29.ffn.net.0.proj.video.down.weight": {
1265
+ "dtype": "F32",
1266
+ "shape": [
1267
+ 64,
1268
+ 3072
1269
+ ]
1270
+ },
1271
+ "blocks.29.ffn.net.0.proj.video.up.weight": {
1272
+ "dtype": "F32",
1273
+ "shape": [
1274
+ 14336,
1275
+ 64
1276
+ ]
1277
+ },
1278
+ "blocks.29.ffn.net.2.action.down.weight": {
1279
+ "dtype": "F32",
1280
+ "shape": [
1281
+ 64,
1282
+ 14336
1283
+ ]
1284
+ },
1285
+ "blocks.29.ffn.net.2.action.up.weight": {
1286
+ "dtype": "F32",
1287
+ "shape": [
1288
+ 3072,
1289
+ 64
1290
+ ]
1291
+ },
1292
+ "blocks.29.ffn.net.2.video.down.weight": {
1293
+ "dtype": "F32",
1294
+ "shape": [
1295
+ 64,
1296
+ 14336
1297
+ ]
1298
+ },
1299
+ "blocks.29.ffn.net.2.video.up.weight": {
1300
+ "dtype": "F32",
1301
+ "shape": [
1302
+ 3072,
1303
+ 64
1304
+ ]
1305
+ },
1306
+ "blocks.3.ffn.net.0.proj.action.down.weight": {
1307
+ "dtype": "F32",
1308
+ "shape": [
1309
+ 64,
1310
+ 3072
1311
+ ]
1312
+ },
1313
+ "blocks.3.ffn.net.0.proj.action.up.weight": {
1314
+ "dtype": "F32",
1315
+ "shape": [
1316
+ 14336,
1317
+ 64
1318
+ ]
1319
+ },
1320
+ "blocks.3.ffn.net.0.proj.video.down.weight": {
1321
+ "dtype": "F32",
1322
+ "shape": [
1323
+ 64,
1324
+ 3072
1325
+ ]
1326
+ },
1327
+ "blocks.3.ffn.net.0.proj.video.up.weight": {
1328
+ "dtype": "F32",
1329
+ "shape": [
1330
+ 14336,
1331
+ 64
1332
+ ]
1333
+ },
1334
+ "blocks.3.ffn.net.2.action.down.weight": {
1335
+ "dtype": "F32",
1336
+ "shape": [
1337
+ 64,
1338
+ 14336
1339
+ ]
1340
+ },
1341
+ "blocks.3.ffn.net.2.action.up.weight": {
1342
+ "dtype": "F32",
1343
+ "shape": [
1344
+ 3072,
1345
+ 64
1346
+ ]
1347
+ },
1348
+ "blocks.3.ffn.net.2.video.down.weight": {
1349
+ "dtype": "F32",
1350
+ "shape": [
1351
+ 64,
1352
+ 14336
1353
+ ]
1354
+ },
1355
+ "blocks.3.ffn.net.2.video.up.weight": {
1356
+ "dtype": "F32",
1357
+ "shape": [
1358
+ 3072,
1359
+ 64
1360
+ ]
1361
+ },
1362
+ "blocks.4.ffn.net.0.proj.action.down.weight": {
1363
+ "dtype": "F32",
1364
+ "shape": [
1365
+ 64,
1366
+ 3072
1367
+ ]
1368
+ },
1369
+ "blocks.4.ffn.net.0.proj.action.up.weight": {
1370
+ "dtype": "F32",
1371
+ "shape": [
1372
+ 14336,
1373
+ 64
1374
+ ]
1375
+ },
1376
+ "blocks.4.ffn.net.0.proj.video.down.weight": {
1377
+ "dtype": "F32",
1378
+ "shape": [
1379
+ 64,
1380
+ 3072
1381
+ ]
1382
+ },
1383
+ "blocks.4.ffn.net.0.proj.video.up.weight": {
1384
+ "dtype": "F32",
1385
+ "shape": [
1386
+ 14336,
1387
+ 64
1388
+ ]
1389
+ },
1390
+ "blocks.4.ffn.net.2.action.down.weight": {
1391
+ "dtype": "F32",
1392
+ "shape": [
1393
+ 64,
1394
+ 14336
1395
+ ]
1396
+ },
1397
+ "blocks.4.ffn.net.2.action.up.weight": {
1398
+ "dtype": "F32",
1399
+ "shape": [
1400
+ 3072,
1401
+ 64
1402
+ ]
1403
+ },
1404
+ "blocks.4.ffn.net.2.video.down.weight": {
1405
+ "dtype": "F32",
1406
+ "shape": [
1407
+ 64,
1408
+ 14336
1409
+ ]
1410
+ },
1411
+ "blocks.4.ffn.net.2.video.up.weight": {
1412
+ "dtype": "F32",
1413
+ "shape": [
1414
+ 3072,
1415
+ 64
1416
+ ]
1417
+ },
1418
+ "blocks.5.ffn.net.0.proj.action.down.weight": {
1419
+ "dtype": "F32",
1420
+ "shape": [
1421
+ 64,
1422
+ 3072
1423
+ ]
1424
+ },
1425
+ "blocks.5.ffn.net.0.proj.action.up.weight": {
1426
+ "dtype": "F32",
1427
+ "shape": [
1428
+ 14336,
1429
+ 64
1430
+ ]
1431
+ },
1432
+ "blocks.5.ffn.net.0.proj.video.down.weight": {
1433
+ "dtype": "F32",
1434
+ "shape": [
1435
+ 64,
1436
+ 3072
1437
+ ]
1438
+ },
1439
+ "blocks.5.ffn.net.0.proj.video.up.weight": {
1440
+ "dtype": "F32",
1441
+ "shape": [
1442
+ 14336,
1443
+ 64
1444
+ ]
1445
+ },
1446
+ "blocks.5.ffn.net.2.action.down.weight": {
1447
+ "dtype": "F32",
1448
+ "shape": [
1449
+ 64,
1450
+ 14336
1451
+ ]
1452
+ },
1453
+ "blocks.5.ffn.net.2.action.up.weight": {
1454
+ "dtype": "F32",
1455
+ "shape": [
1456
+ 3072,
1457
+ 64
1458
+ ]
1459
+ },
1460
+ "blocks.5.ffn.net.2.video.down.weight": {
1461
+ "dtype": "F32",
1462
+ "shape": [
1463
+ 64,
1464
+ 14336
1465
+ ]
1466
+ },
1467
+ "blocks.5.ffn.net.2.video.up.weight": {
1468
+ "dtype": "F32",
1469
+ "shape": [
1470
+ 3072,
1471
+ 64
1472
+ ]
1473
+ },
1474
+ "blocks.6.ffn.net.0.proj.action.down.weight": {
1475
+ "dtype": "F32",
1476
+ "shape": [
1477
+ 64,
1478
+ 3072
1479
+ ]
1480
+ },
1481
+ "blocks.6.ffn.net.0.proj.action.up.weight": {
1482
+ "dtype": "F32",
1483
+ "shape": [
1484
+ 14336,
1485
+ 64
1486
+ ]
1487
+ },
1488
+ "blocks.6.ffn.net.0.proj.video.down.weight": {
1489
+ "dtype": "F32",
1490
+ "shape": [
1491
+ 64,
1492
+ 3072
1493
+ ]
1494
+ },
1495
+ "blocks.6.ffn.net.0.proj.video.up.weight": {
1496
+ "dtype": "F32",
1497
+ "shape": [
1498
+ 14336,
1499
+ 64
1500
+ ]
1501
+ },
1502
+ "blocks.6.ffn.net.2.action.down.weight": {
1503
+ "dtype": "F32",
1504
+ "shape": [
1505
+ 64,
1506
+ 14336
1507
+ ]
1508
+ },
1509
+ "blocks.6.ffn.net.2.action.up.weight": {
1510
+ "dtype": "F32",
1511
+ "shape": [
1512
+ 3072,
1513
+ 64
1514
+ ]
1515
+ },
1516
+ "blocks.6.ffn.net.2.video.down.weight": {
1517
+ "dtype": "F32",
1518
+ "shape": [
1519
+ 64,
1520
+ 14336
1521
+ ]
1522
+ },
1523
+ "blocks.6.ffn.net.2.video.up.weight": {
1524
+ "dtype": "F32",
1525
+ "shape": [
1526
+ 3072,
1527
+ 64
1528
+ ]
1529
+ },
1530
+ "blocks.7.ffn.net.0.proj.action.down.weight": {
1531
+ "dtype": "F32",
1532
+ "shape": [
1533
+ 64,
1534
+ 3072
1535
+ ]
1536
+ },
1537
+ "blocks.7.ffn.net.0.proj.action.up.weight": {
1538
+ "dtype": "F32",
1539
+ "shape": [
1540
+ 14336,
1541
+ 64
1542
+ ]
1543
+ },
1544
+ "blocks.7.ffn.net.0.proj.video.down.weight": {
1545
+ "dtype": "F32",
1546
+ "shape": [
1547
+ 64,
1548
+ 3072
1549
+ ]
1550
+ },
1551
+ "blocks.7.ffn.net.0.proj.video.up.weight": {
1552
+ "dtype": "F32",
1553
+ "shape": [
1554
+ 14336,
1555
+ 64
1556
+ ]
1557
+ },
1558
+ "blocks.7.ffn.net.2.action.down.weight": {
1559
+ "dtype": "F32",
1560
+ "shape": [
1561
+ 64,
1562
+ 14336
1563
+ ]
1564
+ },
1565
+ "blocks.7.ffn.net.2.action.up.weight": {
1566
+ "dtype": "F32",
1567
+ "shape": [
1568
+ 3072,
1569
+ 64
1570
+ ]
1571
+ },
1572
+ "blocks.7.ffn.net.2.video.down.weight": {
1573
+ "dtype": "F32",
1574
+ "shape": [
1575
+ 64,
1576
+ 14336
1577
+ ]
1578
+ },
1579
+ "blocks.7.ffn.net.2.video.up.weight": {
1580
+ "dtype": "F32",
1581
+ "shape": [
1582
+ 3072,
1583
+ 64
1584
+ ]
1585
+ },
1586
+ "blocks.8.ffn.net.0.proj.action.down.weight": {
1587
+ "dtype": "F32",
1588
+ "shape": [
1589
+ 64,
1590
+ 3072
1591
+ ]
1592
+ },
1593
+ "blocks.8.ffn.net.0.proj.action.up.weight": {
1594
+ "dtype": "F32",
1595
+ "shape": [
1596
+ 14336,
1597
+ 64
1598
+ ]
1599
+ },
1600
+ "blocks.8.ffn.net.0.proj.video.down.weight": {
1601
+ "dtype": "F32",
1602
+ "shape": [
1603
+ 64,
1604
+ 3072
1605
+ ]
1606
+ },
1607
+ "blocks.8.ffn.net.0.proj.video.up.weight": {
1608
+ "dtype": "F32",
1609
+ "shape": [
1610
+ 14336,
1611
+ 64
1612
+ ]
1613
+ },
1614
+ "blocks.8.ffn.net.2.action.down.weight": {
1615
+ "dtype": "F32",
1616
+ "shape": [
1617
+ 64,
1618
+ 14336
1619
+ ]
1620
+ },
1621
+ "blocks.8.ffn.net.2.action.up.weight": {
1622
+ "dtype": "F32",
1623
+ "shape": [
1624
+ 3072,
1625
+ 64
1626
+ ]
1627
+ },
1628
+ "blocks.8.ffn.net.2.video.down.weight": {
1629
+ "dtype": "F32",
1630
+ "shape": [
1631
+ 64,
1632
+ 14336
1633
+ ]
1634
+ },
1635
+ "blocks.8.ffn.net.2.video.up.weight": {
1636
+ "dtype": "F32",
1637
+ "shape": [
1638
+ 3072,
1639
+ 64
1640
+ ]
1641
+ },
1642
+ "blocks.9.ffn.net.0.proj.action.down.weight": {
1643
+ "dtype": "F32",
1644
+ "shape": [
1645
+ 64,
1646
+ 3072
1647
+ ]
1648
+ },
1649
+ "blocks.9.ffn.net.0.proj.action.up.weight": {
1650
+ "dtype": "F32",
1651
+ "shape": [
1652
+ 14336,
1653
+ 64
1654
+ ]
1655
+ },
1656
+ "blocks.9.ffn.net.0.proj.video.down.weight": {
1657
+ "dtype": "F32",
1658
+ "shape": [
1659
+ 64,
1660
+ 3072
1661
+ ]
1662
+ },
1663
+ "blocks.9.ffn.net.0.proj.video.up.weight": {
1664
+ "dtype": "F32",
1665
+ "shape": [
1666
+ 14336,
1667
+ 64
1668
+ ]
1669
+ },
1670
+ "blocks.9.ffn.net.2.action.down.weight": {
1671
+ "dtype": "F32",
1672
+ "shape": [
1673
+ 64,
1674
+ 14336
1675
+ ]
1676
+ },
1677
+ "blocks.9.ffn.net.2.action.up.weight": {
1678
+ "dtype": "F32",
1679
+ "shape": [
1680
+ 3072,
1681
+ 64
1682
+ ]
1683
+ },
1684
+ "blocks.9.ffn.net.2.video.down.weight": {
1685
+ "dtype": "F32",
1686
+ "shape": [
1687
+ 64,
1688
+ 14336
1689
+ ]
1690
+ },
1691
+ "blocks.9.ffn.net.2.video.up.weight": {
1692
+ "dtype": "F32",
1693
+ "shape": [
1694
+ 3072,
1695
+ 64
1696
+ ]
1697
+ },
1698
+ "condition_embedder.delta_embedder.linear_1.shared.down.weight": {
1699
+ "dtype": "F32",
1700
+ "shape": [
1701
+ 128,
1702
+ 256
1703
+ ]
1704
+ },
1705
+ "condition_embedder.delta_embedder.linear_1.shared.up.weight": {
1706
+ "dtype": "F32",
1707
+ "shape": [
1708
+ 3072,
1709
+ 128
1710
+ ]
1711
+ },
1712
+ "condition_embedder.delta_embedder.linear_2.shared.down.weight": {
1713
+ "dtype": "F32",
1714
+ "shape": [
1715
+ 128,
1716
+ 3072
1717
+ ]
1718
+ },
1719
+ "condition_embedder.delta_embedder.linear_2.shared.up.weight": {
1720
+ "dtype": "F32",
1721
+ "shape": [
1722
+ 3072,
1723
+ 128
1724
+ ]
1725
+ },
1726
+ "condition_embedder.delta_output.shared.down.weight": {
1727
+ "dtype": "F32",
1728
+ "shape": [
1729
+ 128,
1730
+ 3072
1731
+ ]
1732
+ },
1733
+ "condition_embedder.delta_output.shared.up.weight": {
1734
+ "dtype": "F32",
1735
+ "shape": [
1736
+ 3072,
1737
+ 128
1738
+ ]
1739
+ },
1740
+ "condition_embedder.time_embedder.linear_1.shared.down.weight": {
1741
+ "dtype": "F32",
1742
+ "shape": [
1743
+ 128,
1744
+ 256
1745
+ ]
1746
+ },
1747
+ "condition_embedder.time_embedder.linear_1.shared.up.weight": {
1748
+ "dtype": "F32",
1749
+ "shape": [
1750
+ 3072,
1751
+ 128
1752
+ ]
1753
+ },
1754
+ "condition_embedder.time_embedder.linear_2.shared.down.weight": {
1755
+ "dtype": "F32",
1756
+ "shape": [
1757
+ 128,
1758
+ 3072
1759
+ ]
1760
+ },
1761
+ "condition_embedder.time_embedder.linear_2.shared.up.weight": {
1762
+ "dtype": "F32",
1763
+ "shape": [
1764
+ 3072,
1765
+ 128
1766
+ ]
1767
+ },
1768
+ "condition_embedder.time_proj.shared.down.weight": {
1769
+ "dtype": "F32",
1770
+ "shape": [
1771
+ 128,
1772
+ 3072
1773
+ ]
1774
+ },
1775
+ "condition_embedder.time_proj.shared.up.weight": {
1776
+ "dtype": "F32",
1777
+ "shape": [
1778
+ 18432,
1779
+ 128
1780
+ ]
1781
+ },
1782
+ "condition_embedder_action.delta_embedder.linear_1.shared.down.weight": {
1783
+ "dtype": "F32",
1784
+ "shape": [
1785
+ 128,
1786
+ 256
1787
+ ]
1788
+ },
1789
+ "condition_embedder_action.delta_embedder.linear_1.shared.up.weight": {
1790
+ "dtype": "F32",
1791
+ "shape": [
1792
+ 3072,
1793
+ 128
1794
+ ]
1795
+ },
1796
+ "condition_embedder_action.delta_embedder.linear_2.shared.down.weight": {
1797
+ "dtype": "F32",
1798
+ "shape": [
1799
+ 128,
1800
+ 3072
1801
+ ]
1802
+ },
1803
+ "condition_embedder_action.delta_embedder.linear_2.shared.up.weight": {
1804
+ "dtype": "F32",
1805
+ "shape": [
1806
+ 3072,
1807
+ 128
1808
+ ]
1809
+ },
1810
+ "condition_embedder_action.delta_output.shared.down.weight": {
1811
+ "dtype": "F32",
1812
+ "shape": [
1813
+ 128,
1814
+ 3072
1815
+ ]
1816
+ },
1817
+ "condition_embedder_action.delta_output.shared.up.weight": {
1818
+ "dtype": "F32",
1819
+ "shape": [
1820
+ 3072,
1821
+ 128
1822
+ ]
1823
+ },
1824
+ "condition_embedder_action.time_embedder.linear_1.shared.down.weight": {
1825
+ "dtype": "F32",
1826
+ "shape": [
1827
+ 128,
1828
+ 256
1829
+ ]
1830
+ },
1831
+ "condition_embedder_action.time_embedder.linear_1.shared.up.weight": {
1832
+ "dtype": "F32",
1833
+ "shape": [
1834
+ 3072,
1835
+ 128
1836
+ ]
1837
+ },
1838
+ "condition_embedder_action.time_embedder.linear_2.shared.down.weight": {
1839
+ "dtype": "F32",
1840
+ "shape": [
1841
+ 128,
1842
+ 3072
1843
+ ]
1844
+ },
1845
+ "condition_embedder_action.time_embedder.linear_2.shared.up.weight": {
1846
+ "dtype": "F32",
1847
+ "shape": [
1848
+ 3072,
1849
+ 128
1850
+ ]
1851
+ },
1852
+ "condition_embedder_action.time_proj.shared.down.weight": {
1853
+ "dtype": "F32",
1854
+ "shape": [
1855
+ 128,
1856
+ 3072
1857
+ ]
1858
+ },
1859
+ "condition_embedder_action.time_proj.shared.up.weight": {
1860
+ "dtype": "F32",
1861
+ "shape": [
1862
+ 18432,
1863
+ 128
1864
+ ]
1865
+ }
1866
+ },
1867
+ "kind": "adapter",
1868
+ "license": "Apache-2.0",
1869
+ "paper_role": "apm",
1870
+ "source_digest": "7778d12f803b392d93f2f9d1c5ecee883d5353de692de5ed21bf311799f324a2",
1871
+ "topology": "frozen-stage1-routed-ffn-time-v1:r64/r64/t128",
1872
+ "variant_id": "apm"
1873
+ }
variants/continued/adapter_model.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:4ec45e879f33053650de7d0bbd409130a85dc0785ac328876406e68ead2170de
3
+ size 1337529184
variants/continued/manifest.json ADDED
@@ -0,0 +1,4393 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "artifact_file": "adapter_model.safetensors",
3
+ "artifact_sha256": "4ec45e879f33053650de7d0bbd409130a85dc0785ac328876406e68ead2170de",
4
+ "base_digest": "106fa72cc8adfebd3ce0d89561d9ec61a29e0fbc85620a0f7914ab314fbe746b",
5
+ "cfg_contract": {
6
+ "deployment_cfg": [
7
+ 5.0,
8
+ 1.0
9
+ ],
10
+ "target_guidance_mode": "inference_composed",
11
+ "teacher_cfg": [
12
+ 5.0,
13
+ 1.0
14
+ ],
15
+ "text_dropout_probability": 0.1
16
+ },
17
+ "keys": {
18
+ "blocks.0.attn1.to_k.lora_down.weight": {
19
+ "dtype": "F32",
20
+ "shape": [
21
+ 128,
22
+ 3072
23
+ ]
24
+ },
25
+ "blocks.0.attn1.to_k.lora_up.weight": {
26
+ "dtype": "F32",
27
+ "shape": [
28
+ 3072,
29
+ 128
30
+ ]
31
+ },
32
+ "blocks.0.attn1.to_out.0.lora_down.weight": {
33
+ "dtype": "F32",
34
+ "shape": [
35
+ 128,
36
+ 3072
37
+ ]
38
+ },
39
+ "blocks.0.attn1.to_out.0.lora_up.weight": {
40
+ "dtype": "F32",
41
+ "shape": [
42
+ 3072,
43
+ 128
44
+ ]
45
+ },
46
+ "blocks.0.attn1.to_q.lora_down.weight": {
47
+ "dtype": "F32",
48
+ "shape": [
49
+ 128,
50
+ 3072
51
+ ]
52
+ },
53
+ "blocks.0.attn1.to_q.lora_up.weight": {
54
+ "dtype": "F32",
55
+ "shape": [
56
+ 3072,
57
+ 128
58
+ ]
59
+ },
60
+ "blocks.0.attn1.to_v.lora_down.weight": {
61
+ "dtype": "F32",
62
+ "shape": [
63
+ 128,
64
+ 3072
65
+ ]
66
+ },
67
+ "blocks.0.attn1.to_v.lora_up.weight": {
68
+ "dtype": "F32",
69
+ "shape": [
70
+ 3072,
71
+ 128
72
+ ]
73
+ },
74
+ "blocks.0.attn2.to_k.lora_down.weight": {
75
+ "dtype": "F32",
76
+ "shape": [
77
+ 128,
78
+ 3072
79
+ ]
80
+ },
81
+ "blocks.0.attn2.to_k.lora_up.weight": {
82
+ "dtype": "F32",
83
+ "shape": [
84
+ 3072,
85
+ 128
86
+ ]
87
+ },
88
+ "blocks.0.attn2.to_out.0.lora_down.weight": {
89
+ "dtype": "F32",
90
+ "shape": [
91
+ 128,
92
+ 3072
93
+ ]
94
+ },
95
+ "blocks.0.attn2.to_out.0.lora_up.weight": {
96
+ "dtype": "F32",
97
+ "shape": [
98
+ 3072,
99
+ 128
100
+ ]
101
+ },
102
+ "blocks.0.attn2.to_q.lora_down.weight": {
103
+ "dtype": "F32",
104
+ "shape": [
105
+ 128,
106
+ 3072
107
+ ]
108
+ },
109
+ "blocks.0.attn2.to_q.lora_up.weight": {
110
+ "dtype": "F32",
111
+ "shape": [
112
+ 3072,
113
+ 128
114
+ ]
115
+ },
116
+ "blocks.0.attn2.to_v.lora_down.weight": {
117
+ "dtype": "F32",
118
+ "shape": [
119
+ 128,
120
+ 3072
121
+ ]
122
+ },
123
+ "blocks.0.attn2.to_v.lora_up.weight": {
124
+ "dtype": "F32",
125
+ "shape": [
126
+ 3072,
127
+ 128
128
+ ]
129
+ },
130
+ "blocks.0.ffn.net.0.proj.lora_down.weight": {
131
+ "dtype": "F32",
132
+ "shape": [
133
+ 128,
134
+ 3072
135
+ ]
136
+ },
137
+ "blocks.0.ffn.net.0.proj.lora_up.weight": {
138
+ "dtype": "F32",
139
+ "shape": [
140
+ 14336,
141
+ 128
142
+ ]
143
+ },
144
+ "blocks.0.ffn.net.2.lora_down.weight": {
145
+ "dtype": "F32",
146
+ "shape": [
147
+ 128,
148
+ 14336
149
+ ]
150
+ },
151
+ "blocks.0.ffn.net.2.lora_up.weight": {
152
+ "dtype": "F32",
153
+ "shape": [
154
+ 3072,
155
+ 128
156
+ ]
157
+ },
158
+ "blocks.1.attn1.to_k.lora_down.weight": {
159
+ "dtype": "F32",
160
+ "shape": [
161
+ 128,
162
+ 3072
163
+ ]
164
+ },
165
+ "blocks.1.attn1.to_k.lora_up.weight": {
166
+ "dtype": "F32",
167
+ "shape": [
168
+ 3072,
169
+ 128
170
+ ]
171
+ },
172
+ "blocks.1.attn1.to_out.0.lora_down.weight": {
173
+ "dtype": "F32",
174
+ "shape": [
175
+ 128,
176
+ 3072
177
+ ]
178
+ },
179
+ "blocks.1.attn1.to_out.0.lora_up.weight": {
180
+ "dtype": "F32",
181
+ "shape": [
182
+ 3072,
183
+ 128
184
+ ]
185
+ },
186
+ "blocks.1.attn1.to_q.lora_down.weight": {
187
+ "dtype": "F32",
188
+ "shape": [
189
+ 128,
190
+ 3072
191
+ ]
192
+ },
193
+ "blocks.1.attn1.to_q.lora_up.weight": {
194
+ "dtype": "F32",
195
+ "shape": [
196
+ 3072,
197
+ 128
198
+ ]
199
+ },
200
+ "blocks.1.attn1.to_v.lora_down.weight": {
201
+ "dtype": "F32",
202
+ "shape": [
203
+ 128,
204
+ 3072
205
+ ]
206
+ },
207
+ "blocks.1.attn1.to_v.lora_up.weight": {
208
+ "dtype": "F32",
209
+ "shape": [
210
+ 3072,
211
+ 128
212
+ ]
213
+ },
214
+ "blocks.1.attn2.to_k.lora_down.weight": {
215
+ "dtype": "F32",
216
+ "shape": [
217
+ 128,
218
+ 3072
219
+ ]
220
+ },
221
+ "blocks.1.attn2.to_k.lora_up.weight": {
222
+ "dtype": "F32",
223
+ "shape": [
224
+ 3072,
225
+ 128
226
+ ]
227
+ },
228
+ "blocks.1.attn2.to_out.0.lora_down.weight": {
229
+ "dtype": "F32",
230
+ "shape": [
231
+ 128,
232
+ 3072
233
+ ]
234
+ },
235
+ "blocks.1.attn2.to_out.0.lora_up.weight": {
236
+ "dtype": "F32",
237
+ "shape": [
238
+ 3072,
239
+ 128
240
+ ]
241
+ },
242
+ "blocks.1.attn2.to_q.lora_down.weight": {
243
+ "dtype": "F32",
244
+ "shape": [
245
+ 128,
246
+ 3072
247
+ ]
248
+ },
249
+ "blocks.1.attn2.to_q.lora_up.weight": {
250
+ "dtype": "F32",
251
+ "shape": [
252
+ 3072,
253
+ 128
254
+ ]
255
+ },
256
+ "blocks.1.attn2.to_v.lora_down.weight": {
257
+ "dtype": "F32",
258
+ "shape": [
259
+ 128,
260
+ 3072
261
+ ]
262
+ },
263
+ "blocks.1.attn2.to_v.lora_up.weight": {
264
+ "dtype": "F32",
265
+ "shape": [
266
+ 3072,
267
+ 128
268
+ ]
269
+ },
270
+ "blocks.1.ffn.net.0.proj.lora_down.weight": {
271
+ "dtype": "F32",
272
+ "shape": [
273
+ 128,
274
+ 3072
275
+ ]
276
+ },
277
+ "blocks.1.ffn.net.0.proj.lora_up.weight": {
278
+ "dtype": "F32",
279
+ "shape": [
280
+ 14336,
281
+ 128
282
+ ]
283
+ },
284
+ "blocks.1.ffn.net.2.lora_down.weight": {
285
+ "dtype": "F32",
286
+ "shape": [
287
+ 128,
288
+ 14336
289
+ ]
290
+ },
291
+ "blocks.1.ffn.net.2.lora_up.weight": {
292
+ "dtype": "F32",
293
+ "shape": [
294
+ 3072,
295
+ 128
296
+ ]
297
+ },
298
+ "blocks.10.attn1.to_k.lora_down.weight": {
299
+ "dtype": "F32",
300
+ "shape": [
301
+ 128,
302
+ 3072
303
+ ]
304
+ },
305
+ "blocks.10.attn1.to_k.lora_up.weight": {
306
+ "dtype": "F32",
307
+ "shape": [
308
+ 3072,
309
+ 128
310
+ ]
311
+ },
312
+ "blocks.10.attn1.to_out.0.lora_down.weight": {
313
+ "dtype": "F32",
314
+ "shape": [
315
+ 128,
316
+ 3072
317
+ ]
318
+ },
319
+ "blocks.10.attn1.to_out.0.lora_up.weight": {
320
+ "dtype": "F32",
321
+ "shape": [
322
+ 3072,
323
+ 128
324
+ ]
325
+ },
326
+ "blocks.10.attn1.to_q.lora_down.weight": {
327
+ "dtype": "F32",
328
+ "shape": [
329
+ 128,
330
+ 3072
331
+ ]
332
+ },
333
+ "blocks.10.attn1.to_q.lora_up.weight": {
334
+ "dtype": "F32",
335
+ "shape": [
336
+ 3072,
337
+ 128
338
+ ]
339
+ },
340
+ "blocks.10.attn1.to_v.lora_down.weight": {
341
+ "dtype": "F32",
342
+ "shape": [
343
+ 128,
344
+ 3072
345
+ ]
346
+ },
347
+ "blocks.10.attn1.to_v.lora_up.weight": {
348
+ "dtype": "F32",
349
+ "shape": [
350
+ 3072,
351
+ 128
352
+ ]
353
+ },
354
+ "blocks.10.attn2.to_k.lora_down.weight": {
355
+ "dtype": "F32",
356
+ "shape": [
357
+ 128,
358
+ 3072
359
+ ]
360
+ },
361
+ "blocks.10.attn2.to_k.lora_up.weight": {
362
+ "dtype": "F32",
363
+ "shape": [
364
+ 3072,
365
+ 128
366
+ ]
367
+ },
368
+ "blocks.10.attn2.to_out.0.lora_down.weight": {
369
+ "dtype": "F32",
370
+ "shape": [
371
+ 128,
372
+ 3072
373
+ ]
374
+ },
375
+ "blocks.10.attn2.to_out.0.lora_up.weight": {
376
+ "dtype": "F32",
377
+ "shape": [
378
+ 3072,
379
+ 128
380
+ ]
381
+ },
382
+ "blocks.10.attn2.to_q.lora_down.weight": {
383
+ "dtype": "F32",
384
+ "shape": [
385
+ 128,
386
+ 3072
387
+ ]
388
+ },
389
+ "blocks.10.attn2.to_q.lora_up.weight": {
390
+ "dtype": "F32",
391
+ "shape": [
392
+ 3072,
393
+ 128
394
+ ]
395
+ },
396
+ "blocks.10.attn2.to_v.lora_down.weight": {
397
+ "dtype": "F32",
398
+ "shape": [
399
+ 128,
400
+ 3072
401
+ ]
402
+ },
403
+ "blocks.10.attn2.to_v.lora_up.weight": {
404
+ "dtype": "F32",
405
+ "shape": [
406
+ 3072,
407
+ 128
408
+ ]
409
+ },
410
+ "blocks.10.ffn.net.0.proj.lora_down.weight": {
411
+ "dtype": "F32",
412
+ "shape": [
413
+ 128,
414
+ 3072
415
+ ]
416
+ },
417
+ "blocks.10.ffn.net.0.proj.lora_up.weight": {
418
+ "dtype": "F32",
419
+ "shape": [
420
+ 14336,
421
+ 128
422
+ ]
423
+ },
424
+ "blocks.10.ffn.net.2.lora_down.weight": {
425
+ "dtype": "F32",
426
+ "shape": [
427
+ 128,
428
+ 14336
429
+ ]
430
+ },
431
+ "blocks.10.ffn.net.2.lora_up.weight": {
432
+ "dtype": "F32",
433
+ "shape": [
434
+ 3072,
435
+ 128
436
+ ]
437
+ },
438
+ "blocks.11.attn1.to_k.lora_down.weight": {
439
+ "dtype": "F32",
440
+ "shape": [
441
+ 128,
442
+ 3072
443
+ ]
444
+ },
445
+ "blocks.11.attn1.to_k.lora_up.weight": {
446
+ "dtype": "F32",
447
+ "shape": [
448
+ 3072,
449
+ 128
450
+ ]
451
+ },
452
+ "blocks.11.attn1.to_out.0.lora_down.weight": {
453
+ "dtype": "F32",
454
+ "shape": [
455
+ 128,
456
+ 3072
457
+ ]
458
+ },
459
+ "blocks.11.attn1.to_out.0.lora_up.weight": {
460
+ "dtype": "F32",
461
+ "shape": [
462
+ 3072,
463
+ 128
464
+ ]
465
+ },
466
+ "blocks.11.attn1.to_q.lora_down.weight": {
467
+ "dtype": "F32",
468
+ "shape": [
469
+ 128,
470
+ 3072
471
+ ]
472
+ },
473
+ "blocks.11.attn1.to_q.lora_up.weight": {
474
+ "dtype": "F32",
475
+ "shape": [
476
+ 3072,
477
+ 128
478
+ ]
479
+ },
480
+ "blocks.11.attn1.to_v.lora_down.weight": {
481
+ "dtype": "F32",
482
+ "shape": [
483
+ 128,
484
+ 3072
485
+ ]
486
+ },
487
+ "blocks.11.attn1.to_v.lora_up.weight": {
488
+ "dtype": "F32",
489
+ "shape": [
490
+ 3072,
491
+ 128
492
+ ]
493
+ },
494
+ "blocks.11.attn2.to_k.lora_down.weight": {
495
+ "dtype": "F32",
496
+ "shape": [
497
+ 128,
498
+ 3072
499
+ ]
500
+ },
501
+ "blocks.11.attn2.to_k.lora_up.weight": {
502
+ "dtype": "F32",
503
+ "shape": [
504
+ 3072,
505
+ 128
506
+ ]
507
+ },
508
+ "blocks.11.attn2.to_out.0.lora_down.weight": {
509
+ "dtype": "F32",
510
+ "shape": [
511
+ 128,
512
+ 3072
513
+ ]
514
+ },
515
+ "blocks.11.attn2.to_out.0.lora_up.weight": {
516
+ "dtype": "F32",
517
+ "shape": [
518
+ 3072,
519
+ 128
520
+ ]
521
+ },
522
+ "blocks.11.attn2.to_q.lora_down.weight": {
523
+ "dtype": "F32",
524
+ "shape": [
525
+ 128,
526
+ 3072
527
+ ]
528
+ },
529
+ "blocks.11.attn2.to_q.lora_up.weight": {
530
+ "dtype": "F32",
531
+ "shape": [
532
+ 3072,
533
+ 128
534
+ ]
535
+ },
536
+ "blocks.11.attn2.to_v.lora_down.weight": {
537
+ "dtype": "F32",
538
+ "shape": [
539
+ 128,
540
+ 3072
541
+ ]
542
+ },
543
+ "blocks.11.attn2.to_v.lora_up.weight": {
544
+ "dtype": "F32",
545
+ "shape": [
546
+ 3072,
547
+ 128
548
+ ]
549
+ },
550
+ "blocks.11.ffn.net.0.proj.lora_down.weight": {
551
+ "dtype": "F32",
552
+ "shape": [
553
+ 128,
554
+ 3072
555
+ ]
556
+ },
557
+ "blocks.11.ffn.net.0.proj.lora_up.weight": {
558
+ "dtype": "F32",
559
+ "shape": [
560
+ 14336,
561
+ 128
562
+ ]
563
+ },
564
+ "blocks.11.ffn.net.2.lora_down.weight": {
565
+ "dtype": "F32",
566
+ "shape": [
567
+ 128,
568
+ 14336
569
+ ]
570
+ },
571
+ "blocks.11.ffn.net.2.lora_up.weight": {
572
+ "dtype": "F32",
573
+ "shape": [
574
+ 3072,
575
+ 128
576
+ ]
577
+ },
578
+ "blocks.12.attn1.to_k.lora_down.weight": {
579
+ "dtype": "F32",
580
+ "shape": [
581
+ 128,
582
+ 3072
583
+ ]
584
+ },
585
+ "blocks.12.attn1.to_k.lora_up.weight": {
586
+ "dtype": "F32",
587
+ "shape": [
588
+ 3072,
589
+ 128
590
+ ]
591
+ },
592
+ "blocks.12.attn1.to_out.0.lora_down.weight": {
593
+ "dtype": "F32",
594
+ "shape": [
595
+ 128,
596
+ 3072
597
+ ]
598
+ },
599
+ "blocks.12.attn1.to_out.0.lora_up.weight": {
600
+ "dtype": "F32",
601
+ "shape": [
602
+ 3072,
603
+ 128
604
+ ]
605
+ },
606
+ "blocks.12.attn1.to_q.lora_down.weight": {
607
+ "dtype": "F32",
608
+ "shape": [
609
+ 128,
610
+ 3072
611
+ ]
612
+ },
613
+ "blocks.12.attn1.to_q.lora_up.weight": {
614
+ "dtype": "F32",
615
+ "shape": [
616
+ 3072,
617
+ 128
618
+ ]
619
+ },
620
+ "blocks.12.attn1.to_v.lora_down.weight": {
621
+ "dtype": "F32",
622
+ "shape": [
623
+ 128,
624
+ 3072
625
+ ]
626
+ },
627
+ "blocks.12.attn1.to_v.lora_up.weight": {
628
+ "dtype": "F32",
629
+ "shape": [
630
+ 3072,
631
+ 128
632
+ ]
633
+ },
634
+ "blocks.12.attn2.to_k.lora_down.weight": {
635
+ "dtype": "F32",
636
+ "shape": [
637
+ 128,
638
+ 3072
639
+ ]
640
+ },
641
+ "blocks.12.attn2.to_k.lora_up.weight": {
642
+ "dtype": "F32",
643
+ "shape": [
644
+ 3072,
645
+ 128
646
+ ]
647
+ },
648
+ "blocks.12.attn2.to_out.0.lora_down.weight": {
649
+ "dtype": "F32",
650
+ "shape": [
651
+ 128,
652
+ 3072
653
+ ]
654
+ },
655
+ "blocks.12.attn2.to_out.0.lora_up.weight": {
656
+ "dtype": "F32",
657
+ "shape": [
658
+ 3072,
659
+ 128
660
+ ]
661
+ },
662
+ "blocks.12.attn2.to_q.lora_down.weight": {
663
+ "dtype": "F32",
664
+ "shape": [
665
+ 128,
666
+ 3072
667
+ ]
668
+ },
669
+ "blocks.12.attn2.to_q.lora_up.weight": {
670
+ "dtype": "F32",
671
+ "shape": [
672
+ 3072,
673
+ 128
674
+ ]
675
+ },
676
+ "blocks.12.attn2.to_v.lora_down.weight": {
677
+ "dtype": "F32",
678
+ "shape": [
679
+ 128,
680
+ 3072
681
+ ]
682
+ },
683
+ "blocks.12.attn2.to_v.lora_up.weight": {
684
+ "dtype": "F32",
685
+ "shape": [
686
+ 3072,
687
+ 128
688
+ ]
689
+ },
690
+ "blocks.12.ffn.net.0.proj.lora_down.weight": {
691
+ "dtype": "F32",
692
+ "shape": [
693
+ 128,
694
+ 3072
695
+ ]
696
+ },
697
+ "blocks.12.ffn.net.0.proj.lora_up.weight": {
698
+ "dtype": "F32",
699
+ "shape": [
700
+ 14336,
701
+ 128
702
+ ]
703
+ },
704
+ "blocks.12.ffn.net.2.lora_down.weight": {
705
+ "dtype": "F32",
706
+ "shape": [
707
+ 128,
708
+ 14336
709
+ ]
710
+ },
711
+ "blocks.12.ffn.net.2.lora_up.weight": {
712
+ "dtype": "F32",
713
+ "shape": [
714
+ 3072,
715
+ 128
716
+ ]
717
+ },
718
+ "blocks.13.attn1.to_k.lora_down.weight": {
719
+ "dtype": "F32",
720
+ "shape": [
721
+ 128,
722
+ 3072
723
+ ]
724
+ },
725
+ "blocks.13.attn1.to_k.lora_up.weight": {
726
+ "dtype": "F32",
727
+ "shape": [
728
+ 3072,
729
+ 128
730
+ ]
731
+ },
732
+ "blocks.13.attn1.to_out.0.lora_down.weight": {
733
+ "dtype": "F32",
734
+ "shape": [
735
+ 128,
736
+ 3072
737
+ ]
738
+ },
739
+ "blocks.13.attn1.to_out.0.lora_up.weight": {
740
+ "dtype": "F32",
741
+ "shape": [
742
+ 3072,
743
+ 128
744
+ ]
745
+ },
746
+ "blocks.13.attn1.to_q.lora_down.weight": {
747
+ "dtype": "F32",
748
+ "shape": [
749
+ 128,
750
+ 3072
751
+ ]
752
+ },
753
+ "blocks.13.attn1.to_q.lora_up.weight": {
754
+ "dtype": "F32",
755
+ "shape": [
756
+ 3072,
757
+ 128
758
+ ]
759
+ },
760
+ "blocks.13.attn1.to_v.lora_down.weight": {
761
+ "dtype": "F32",
762
+ "shape": [
763
+ 128,
764
+ 3072
765
+ ]
766
+ },
767
+ "blocks.13.attn1.to_v.lora_up.weight": {
768
+ "dtype": "F32",
769
+ "shape": [
770
+ 3072,
771
+ 128
772
+ ]
773
+ },
774
+ "blocks.13.attn2.to_k.lora_down.weight": {
775
+ "dtype": "F32",
776
+ "shape": [
777
+ 128,
778
+ 3072
779
+ ]
780
+ },
781
+ "blocks.13.attn2.to_k.lora_up.weight": {
782
+ "dtype": "F32",
783
+ "shape": [
784
+ 3072,
785
+ 128
786
+ ]
787
+ },
788
+ "blocks.13.attn2.to_out.0.lora_down.weight": {
789
+ "dtype": "F32",
790
+ "shape": [
791
+ 128,
792
+ 3072
793
+ ]
794
+ },
795
+ "blocks.13.attn2.to_out.0.lora_up.weight": {
796
+ "dtype": "F32",
797
+ "shape": [
798
+ 3072,
799
+ 128
800
+ ]
801
+ },
802
+ "blocks.13.attn2.to_q.lora_down.weight": {
803
+ "dtype": "F32",
804
+ "shape": [
805
+ 128,
806
+ 3072
807
+ ]
808
+ },
809
+ "blocks.13.attn2.to_q.lora_up.weight": {
810
+ "dtype": "F32",
811
+ "shape": [
812
+ 3072,
813
+ 128
814
+ ]
815
+ },
816
+ "blocks.13.attn2.to_v.lora_down.weight": {
817
+ "dtype": "F32",
818
+ "shape": [
819
+ 128,
820
+ 3072
821
+ ]
822
+ },
823
+ "blocks.13.attn2.to_v.lora_up.weight": {
824
+ "dtype": "F32",
825
+ "shape": [
826
+ 3072,
827
+ 128
828
+ ]
829
+ },
830
+ "blocks.13.ffn.net.0.proj.lora_down.weight": {
831
+ "dtype": "F32",
832
+ "shape": [
833
+ 128,
834
+ 3072
835
+ ]
836
+ },
837
+ "blocks.13.ffn.net.0.proj.lora_up.weight": {
838
+ "dtype": "F32",
839
+ "shape": [
840
+ 14336,
841
+ 128
842
+ ]
843
+ },
844
+ "blocks.13.ffn.net.2.lora_down.weight": {
845
+ "dtype": "F32",
846
+ "shape": [
847
+ 128,
848
+ 14336
849
+ ]
850
+ },
851
+ "blocks.13.ffn.net.2.lora_up.weight": {
852
+ "dtype": "F32",
853
+ "shape": [
854
+ 3072,
855
+ 128
856
+ ]
857
+ },
858
+ "blocks.14.attn1.to_k.lora_down.weight": {
859
+ "dtype": "F32",
860
+ "shape": [
861
+ 128,
862
+ 3072
863
+ ]
864
+ },
865
+ "blocks.14.attn1.to_k.lora_up.weight": {
866
+ "dtype": "F32",
867
+ "shape": [
868
+ 3072,
869
+ 128
870
+ ]
871
+ },
872
+ "blocks.14.attn1.to_out.0.lora_down.weight": {
873
+ "dtype": "F32",
874
+ "shape": [
875
+ 128,
876
+ 3072
877
+ ]
878
+ },
879
+ "blocks.14.attn1.to_out.0.lora_up.weight": {
880
+ "dtype": "F32",
881
+ "shape": [
882
+ 3072,
883
+ 128
884
+ ]
885
+ },
886
+ "blocks.14.attn1.to_q.lora_down.weight": {
887
+ "dtype": "F32",
888
+ "shape": [
889
+ 128,
890
+ 3072
891
+ ]
892
+ },
893
+ "blocks.14.attn1.to_q.lora_up.weight": {
894
+ "dtype": "F32",
895
+ "shape": [
896
+ 3072,
897
+ 128
898
+ ]
899
+ },
900
+ "blocks.14.attn1.to_v.lora_down.weight": {
901
+ "dtype": "F32",
902
+ "shape": [
903
+ 128,
904
+ 3072
905
+ ]
906
+ },
907
+ "blocks.14.attn1.to_v.lora_up.weight": {
908
+ "dtype": "F32",
909
+ "shape": [
910
+ 3072,
911
+ 128
912
+ ]
913
+ },
914
+ "blocks.14.attn2.to_k.lora_down.weight": {
915
+ "dtype": "F32",
916
+ "shape": [
917
+ 128,
918
+ 3072
919
+ ]
920
+ },
921
+ "blocks.14.attn2.to_k.lora_up.weight": {
922
+ "dtype": "F32",
923
+ "shape": [
924
+ 3072,
925
+ 128
926
+ ]
927
+ },
928
+ "blocks.14.attn2.to_out.0.lora_down.weight": {
929
+ "dtype": "F32",
930
+ "shape": [
931
+ 128,
932
+ 3072
933
+ ]
934
+ },
935
+ "blocks.14.attn2.to_out.0.lora_up.weight": {
936
+ "dtype": "F32",
937
+ "shape": [
938
+ 3072,
939
+ 128
940
+ ]
941
+ },
942
+ "blocks.14.attn2.to_q.lora_down.weight": {
943
+ "dtype": "F32",
944
+ "shape": [
945
+ 128,
946
+ 3072
947
+ ]
948
+ },
949
+ "blocks.14.attn2.to_q.lora_up.weight": {
950
+ "dtype": "F32",
951
+ "shape": [
952
+ 3072,
953
+ 128
954
+ ]
955
+ },
956
+ "blocks.14.attn2.to_v.lora_down.weight": {
957
+ "dtype": "F32",
958
+ "shape": [
959
+ 128,
960
+ 3072
961
+ ]
962
+ },
963
+ "blocks.14.attn2.to_v.lora_up.weight": {
964
+ "dtype": "F32",
965
+ "shape": [
966
+ 3072,
967
+ 128
968
+ ]
969
+ },
970
+ "blocks.14.ffn.net.0.proj.lora_down.weight": {
971
+ "dtype": "F32",
972
+ "shape": [
973
+ 128,
974
+ 3072
975
+ ]
976
+ },
977
+ "blocks.14.ffn.net.0.proj.lora_up.weight": {
978
+ "dtype": "F32",
979
+ "shape": [
980
+ 14336,
981
+ 128
982
+ ]
983
+ },
984
+ "blocks.14.ffn.net.2.lora_down.weight": {
985
+ "dtype": "F32",
986
+ "shape": [
987
+ 128,
988
+ 14336
989
+ ]
990
+ },
991
+ "blocks.14.ffn.net.2.lora_up.weight": {
992
+ "dtype": "F32",
993
+ "shape": [
994
+ 3072,
995
+ 128
996
+ ]
997
+ },
998
+ "blocks.15.attn1.to_k.lora_down.weight": {
999
+ "dtype": "F32",
1000
+ "shape": [
1001
+ 128,
1002
+ 3072
1003
+ ]
1004
+ },
1005
+ "blocks.15.attn1.to_k.lora_up.weight": {
1006
+ "dtype": "F32",
1007
+ "shape": [
1008
+ 3072,
1009
+ 128
1010
+ ]
1011
+ },
1012
+ "blocks.15.attn1.to_out.0.lora_down.weight": {
1013
+ "dtype": "F32",
1014
+ "shape": [
1015
+ 128,
1016
+ 3072
1017
+ ]
1018
+ },
1019
+ "blocks.15.attn1.to_out.0.lora_up.weight": {
1020
+ "dtype": "F32",
1021
+ "shape": [
1022
+ 3072,
1023
+ 128
1024
+ ]
1025
+ },
1026
+ "blocks.15.attn1.to_q.lora_down.weight": {
1027
+ "dtype": "F32",
1028
+ "shape": [
1029
+ 128,
1030
+ 3072
1031
+ ]
1032
+ },
1033
+ "blocks.15.attn1.to_q.lora_up.weight": {
1034
+ "dtype": "F32",
1035
+ "shape": [
1036
+ 3072,
1037
+ 128
1038
+ ]
1039
+ },
1040
+ "blocks.15.attn1.to_v.lora_down.weight": {
1041
+ "dtype": "F32",
1042
+ "shape": [
1043
+ 128,
1044
+ 3072
1045
+ ]
1046
+ },
1047
+ "blocks.15.attn1.to_v.lora_up.weight": {
1048
+ "dtype": "F32",
1049
+ "shape": [
1050
+ 3072,
1051
+ 128
1052
+ ]
1053
+ },
1054
+ "blocks.15.attn2.to_k.lora_down.weight": {
1055
+ "dtype": "F32",
1056
+ "shape": [
1057
+ 128,
1058
+ 3072
1059
+ ]
1060
+ },
1061
+ "blocks.15.attn2.to_k.lora_up.weight": {
1062
+ "dtype": "F32",
1063
+ "shape": [
1064
+ 3072,
1065
+ 128
1066
+ ]
1067
+ },
1068
+ "blocks.15.attn2.to_out.0.lora_down.weight": {
1069
+ "dtype": "F32",
1070
+ "shape": [
1071
+ 128,
1072
+ 3072
1073
+ ]
1074
+ },
1075
+ "blocks.15.attn2.to_out.0.lora_up.weight": {
1076
+ "dtype": "F32",
1077
+ "shape": [
1078
+ 3072,
1079
+ 128
1080
+ ]
1081
+ },
1082
+ "blocks.15.attn2.to_q.lora_down.weight": {
1083
+ "dtype": "F32",
1084
+ "shape": [
1085
+ 128,
1086
+ 3072
1087
+ ]
1088
+ },
1089
+ "blocks.15.attn2.to_q.lora_up.weight": {
1090
+ "dtype": "F32",
1091
+ "shape": [
1092
+ 3072,
1093
+ 128
1094
+ ]
1095
+ },
1096
+ "blocks.15.attn2.to_v.lora_down.weight": {
1097
+ "dtype": "F32",
1098
+ "shape": [
1099
+ 128,
1100
+ 3072
1101
+ ]
1102
+ },
1103
+ "blocks.15.attn2.to_v.lora_up.weight": {
1104
+ "dtype": "F32",
1105
+ "shape": [
1106
+ 3072,
1107
+ 128
1108
+ ]
1109
+ },
1110
+ "blocks.15.ffn.net.0.proj.lora_down.weight": {
1111
+ "dtype": "F32",
1112
+ "shape": [
1113
+ 128,
1114
+ 3072
1115
+ ]
1116
+ },
1117
+ "blocks.15.ffn.net.0.proj.lora_up.weight": {
1118
+ "dtype": "F32",
1119
+ "shape": [
1120
+ 14336,
1121
+ 128
1122
+ ]
1123
+ },
1124
+ "blocks.15.ffn.net.2.lora_down.weight": {
1125
+ "dtype": "F32",
1126
+ "shape": [
1127
+ 128,
1128
+ 14336
1129
+ ]
1130
+ },
1131
+ "blocks.15.ffn.net.2.lora_up.weight": {
1132
+ "dtype": "F32",
1133
+ "shape": [
1134
+ 3072,
1135
+ 128
1136
+ ]
1137
+ },
1138
+ "blocks.16.attn1.to_k.lora_down.weight": {
1139
+ "dtype": "F32",
1140
+ "shape": [
1141
+ 128,
1142
+ 3072
1143
+ ]
1144
+ },
1145
+ "blocks.16.attn1.to_k.lora_up.weight": {
1146
+ "dtype": "F32",
1147
+ "shape": [
1148
+ 3072,
1149
+ 128
1150
+ ]
1151
+ },
1152
+ "blocks.16.attn1.to_out.0.lora_down.weight": {
1153
+ "dtype": "F32",
1154
+ "shape": [
1155
+ 128,
1156
+ 3072
1157
+ ]
1158
+ },
1159
+ "blocks.16.attn1.to_out.0.lora_up.weight": {
1160
+ "dtype": "F32",
1161
+ "shape": [
1162
+ 3072,
1163
+ 128
1164
+ ]
1165
+ },
1166
+ "blocks.16.attn1.to_q.lora_down.weight": {
1167
+ "dtype": "F32",
1168
+ "shape": [
1169
+ 128,
1170
+ 3072
1171
+ ]
1172
+ },
1173
+ "blocks.16.attn1.to_q.lora_up.weight": {
1174
+ "dtype": "F32",
1175
+ "shape": [
1176
+ 3072,
1177
+ 128
1178
+ ]
1179
+ },
1180
+ "blocks.16.attn1.to_v.lora_down.weight": {
1181
+ "dtype": "F32",
1182
+ "shape": [
1183
+ 128,
1184
+ 3072
1185
+ ]
1186
+ },
1187
+ "blocks.16.attn1.to_v.lora_up.weight": {
1188
+ "dtype": "F32",
1189
+ "shape": [
1190
+ 3072,
1191
+ 128
1192
+ ]
1193
+ },
1194
+ "blocks.16.attn2.to_k.lora_down.weight": {
1195
+ "dtype": "F32",
1196
+ "shape": [
1197
+ 128,
1198
+ 3072
1199
+ ]
1200
+ },
1201
+ "blocks.16.attn2.to_k.lora_up.weight": {
1202
+ "dtype": "F32",
1203
+ "shape": [
1204
+ 3072,
1205
+ 128
1206
+ ]
1207
+ },
1208
+ "blocks.16.attn2.to_out.0.lora_down.weight": {
1209
+ "dtype": "F32",
1210
+ "shape": [
1211
+ 128,
1212
+ 3072
1213
+ ]
1214
+ },
1215
+ "blocks.16.attn2.to_out.0.lora_up.weight": {
1216
+ "dtype": "F32",
1217
+ "shape": [
1218
+ 3072,
1219
+ 128
1220
+ ]
1221
+ },
1222
+ "blocks.16.attn2.to_q.lora_down.weight": {
1223
+ "dtype": "F32",
1224
+ "shape": [
1225
+ 128,
1226
+ 3072
1227
+ ]
1228
+ },
1229
+ "blocks.16.attn2.to_q.lora_up.weight": {
1230
+ "dtype": "F32",
1231
+ "shape": [
1232
+ 3072,
1233
+ 128
1234
+ ]
1235
+ },
1236
+ "blocks.16.attn2.to_v.lora_down.weight": {
1237
+ "dtype": "F32",
1238
+ "shape": [
1239
+ 128,
1240
+ 3072
1241
+ ]
1242
+ },
1243
+ "blocks.16.attn2.to_v.lora_up.weight": {
1244
+ "dtype": "F32",
1245
+ "shape": [
1246
+ 3072,
1247
+ 128
1248
+ ]
1249
+ },
1250
+ "blocks.16.ffn.net.0.proj.lora_down.weight": {
1251
+ "dtype": "F32",
1252
+ "shape": [
1253
+ 128,
1254
+ 3072
1255
+ ]
1256
+ },
1257
+ "blocks.16.ffn.net.0.proj.lora_up.weight": {
1258
+ "dtype": "F32",
1259
+ "shape": [
1260
+ 14336,
1261
+ 128
1262
+ ]
1263
+ },
1264
+ "blocks.16.ffn.net.2.lora_down.weight": {
1265
+ "dtype": "F32",
1266
+ "shape": [
1267
+ 128,
1268
+ 14336
1269
+ ]
1270
+ },
1271
+ "blocks.16.ffn.net.2.lora_up.weight": {
1272
+ "dtype": "F32",
1273
+ "shape": [
1274
+ 3072,
1275
+ 128
1276
+ ]
1277
+ },
1278
+ "blocks.17.attn1.to_k.lora_down.weight": {
1279
+ "dtype": "F32",
1280
+ "shape": [
1281
+ 128,
1282
+ 3072
1283
+ ]
1284
+ },
1285
+ "blocks.17.attn1.to_k.lora_up.weight": {
1286
+ "dtype": "F32",
1287
+ "shape": [
1288
+ 3072,
1289
+ 128
1290
+ ]
1291
+ },
1292
+ "blocks.17.attn1.to_out.0.lora_down.weight": {
1293
+ "dtype": "F32",
1294
+ "shape": [
1295
+ 128,
1296
+ 3072
1297
+ ]
1298
+ },
1299
+ "blocks.17.attn1.to_out.0.lora_up.weight": {
1300
+ "dtype": "F32",
1301
+ "shape": [
1302
+ 3072,
1303
+ 128
1304
+ ]
1305
+ },
1306
+ "blocks.17.attn1.to_q.lora_down.weight": {
1307
+ "dtype": "F32",
1308
+ "shape": [
1309
+ 128,
1310
+ 3072
1311
+ ]
1312
+ },
1313
+ "blocks.17.attn1.to_q.lora_up.weight": {
1314
+ "dtype": "F32",
1315
+ "shape": [
1316
+ 3072,
1317
+ 128
1318
+ ]
1319
+ },
1320
+ "blocks.17.attn1.to_v.lora_down.weight": {
1321
+ "dtype": "F32",
1322
+ "shape": [
1323
+ 128,
1324
+ 3072
1325
+ ]
1326
+ },
1327
+ "blocks.17.attn1.to_v.lora_up.weight": {
1328
+ "dtype": "F32",
1329
+ "shape": [
1330
+ 3072,
1331
+ 128
1332
+ ]
1333
+ },
1334
+ "blocks.17.attn2.to_k.lora_down.weight": {
1335
+ "dtype": "F32",
1336
+ "shape": [
1337
+ 128,
1338
+ 3072
1339
+ ]
1340
+ },
1341
+ "blocks.17.attn2.to_k.lora_up.weight": {
1342
+ "dtype": "F32",
1343
+ "shape": [
1344
+ 3072,
1345
+ 128
1346
+ ]
1347
+ },
1348
+ "blocks.17.attn2.to_out.0.lora_down.weight": {
1349
+ "dtype": "F32",
1350
+ "shape": [
1351
+ 128,
1352
+ 3072
1353
+ ]
1354
+ },
1355
+ "blocks.17.attn2.to_out.0.lora_up.weight": {
1356
+ "dtype": "F32",
1357
+ "shape": [
1358
+ 3072,
1359
+ 128
1360
+ ]
1361
+ },
1362
+ "blocks.17.attn2.to_q.lora_down.weight": {
1363
+ "dtype": "F32",
1364
+ "shape": [
1365
+ 128,
1366
+ 3072
1367
+ ]
1368
+ },
1369
+ "blocks.17.attn2.to_q.lora_up.weight": {
1370
+ "dtype": "F32",
1371
+ "shape": [
1372
+ 3072,
1373
+ 128
1374
+ ]
1375
+ },
1376
+ "blocks.17.attn2.to_v.lora_down.weight": {
1377
+ "dtype": "F32",
1378
+ "shape": [
1379
+ 128,
1380
+ 3072
1381
+ ]
1382
+ },
1383
+ "blocks.17.attn2.to_v.lora_up.weight": {
1384
+ "dtype": "F32",
1385
+ "shape": [
1386
+ 3072,
1387
+ 128
1388
+ ]
1389
+ },
1390
+ "blocks.17.ffn.net.0.proj.lora_down.weight": {
1391
+ "dtype": "F32",
1392
+ "shape": [
1393
+ 128,
1394
+ 3072
1395
+ ]
1396
+ },
1397
+ "blocks.17.ffn.net.0.proj.lora_up.weight": {
1398
+ "dtype": "F32",
1399
+ "shape": [
1400
+ 14336,
1401
+ 128
1402
+ ]
1403
+ },
1404
+ "blocks.17.ffn.net.2.lora_down.weight": {
1405
+ "dtype": "F32",
1406
+ "shape": [
1407
+ 128,
1408
+ 14336
1409
+ ]
1410
+ },
1411
+ "blocks.17.ffn.net.2.lora_up.weight": {
1412
+ "dtype": "F32",
1413
+ "shape": [
1414
+ 3072,
1415
+ 128
1416
+ ]
1417
+ },
1418
+ "blocks.18.attn1.to_k.lora_down.weight": {
1419
+ "dtype": "F32",
1420
+ "shape": [
1421
+ 128,
1422
+ 3072
1423
+ ]
1424
+ },
1425
+ "blocks.18.attn1.to_k.lora_up.weight": {
1426
+ "dtype": "F32",
1427
+ "shape": [
1428
+ 3072,
1429
+ 128
1430
+ ]
1431
+ },
1432
+ "blocks.18.attn1.to_out.0.lora_down.weight": {
1433
+ "dtype": "F32",
1434
+ "shape": [
1435
+ 128,
1436
+ 3072
1437
+ ]
1438
+ },
1439
+ "blocks.18.attn1.to_out.0.lora_up.weight": {
1440
+ "dtype": "F32",
1441
+ "shape": [
1442
+ 3072,
1443
+ 128
1444
+ ]
1445
+ },
1446
+ "blocks.18.attn1.to_q.lora_down.weight": {
1447
+ "dtype": "F32",
1448
+ "shape": [
1449
+ 128,
1450
+ 3072
1451
+ ]
1452
+ },
1453
+ "blocks.18.attn1.to_q.lora_up.weight": {
1454
+ "dtype": "F32",
1455
+ "shape": [
1456
+ 3072,
1457
+ 128
1458
+ ]
1459
+ },
1460
+ "blocks.18.attn1.to_v.lora_down.weight": {
1461
+ "dtype": "F32",
1462
+ "shape": [
1463
+ 128,
1464
+ 3072
1465
+ ]
1466
+ },
1467
+ "blocks.18.attn1.to_v.lora_up.weight": {
1468
+ "dtype": "F32",
1469
+ "shape": [
1470
+ 3072,
1471
+ 128
1472
+ ]
1473
+ },
1474
+ "blocks.18.attn2.to_k.lora_down.weight": {
1475
+ "dtype": "F32",
1476
+ "shape": [
1477
+ 128,
1478
+ 3072
1479
+ ]
1480
+ },
1481
+ "blocks.18.attn2.to_k.lora_up.weight": {
1482
+ "dtype": "F32",
1483
+ "shape": [
1484
+ 3072,
1485
+ 128
1486
+ ]
1487
+ },
1488
+ "blocks.18.attn2.to_out.0.lora_down.weight": {
1489
+ "dtype": "F32",
1490
+ "shape": [
1491
+ 128,
1492
+ 3072
1493
+ ]
1494
+ },
1495
+ "blocks.18.attn2.to_out.0.lora_up.weight": {
1496
+ "dtype": "F32",
1497
+ "shape": [
1498
+ 3072,
1499
+ 128
1500
+ ]
1501
+ },
1502
+ "blocks.18.attn2.to_q.lora_down.weight": {
1503
+ "dtype": "F32",
1504
+ "shape": [
1505
+ 128,
1506
+ 3072
1507
+ ]
1508
+ },
1509
+ "blocks.18.attn2.to_q.lora_up.weight": {
1510
+ "dtype": "F32",
1511
+ "shape": [
1512
+ 3072,
1513
+ 128
1514
+ ]
1515
+ },
1516
+ "blocks.18.attn2.to_v.lora_down.weight": {
1517
+ "dtype": "F32",
1518
+ "shape": [
1519
+ 128,
1520
+ 3072
1521
+ ]
1522
+ },
1523
+ "blocks.18.attn2.to_v.lora_up.weight": {
1524
+ "dtype": "F32",
1525
+ "shape": [
1526
+ 3072,
1527
+ 128
1528
+ ]
1529
+ },
1530
+ "blocks.18.ffn.net.0.proj.lora_down.weight": {
1531
+ "dtype": "F32",
1532
+ "shape": [
1533
+ 128,
1534
+ 3072
1535
+ ]
1536
+ },
1537
+ "blocks.18.ffn.net.0.proj.lora_up.weight": {
1538
+ "dtype": "F32",
1539
+ "shape": [
1540
+ 14336,
1541
+ 128
1542
+ ]
1543
+ },
1544
+ "blocks.18.ffn.net.2.lora_down.weight": {
1545
+ "dtype": "F32",
1546
+ "shape": [
1547
+ 128,
1548
+ 14336
1549
+ ]
1550
+ },
1551
+ "blocks.18.ffn.net.2.lora_up.weight": {
1552
+ "dtype": "F32",
1553
+ "shape": [
1554
+ 3072,
1555
+ 128
1556
+ ]
1557
+ },
1558
+ "blocks.19.attn1.to_k.lora_down.weight": {
1559
+ "dtype": "F32",
1560
+ "shape": [
1561
+ 128,
1562
+ 3072
1563
+ ]
1564
+ },
1565
+ "blocks.19.attn1.to_k.lora_up.weight": {
1566
+ "dtype": "F32",
1567
+ "shape": [
1568
+ 3072,
1569
+ 128
1570
+ ]
1571
+ },
1572
+ "blocks.19.attn1.to_out.0.lora_down.weight": {
1573
+ "dtype": "F32",
1574
+ "shape": [
1575
+ 128,
1576
+ 3072
1577
+ ]
1578
+ },
1579
+ "blocks.19.attn1.to_out.0.lora_up.weight": {
1580
+ "dtype": "F32",
1581
+ "shape": [
1582
+ 3072,
1583
+ 128
1584
+ ]
1585
+ },
1586
+ "blocks.19.attn1.to_q.lora_down.weight": {
1587
+ "dtype": "F32",
1588
+ "shape": [
1589
+ 128,
1590
+ 3072
1591
+ ]
1592
+ },
1593
+ "blocks.19.attn1.to_q.lora_up.weight": {
1594
+ "dtype": "F32",
1595
+ "shape": [
1596
+ 3072,
1597
+ 128
1598
+ ]
1599
+ },
1600
+ "blocks.19.attn1.to_v.lora_down.weight": {
1601
+ "dtype": "F32",
1602
+ "shape": [
1603
+ 128,
1604
+ 3072
1605
+ ]
1606
+ },
1607
+ "blocks.19.attn1.to_v.lora_up.weight": {
1608
+ "dtype": "F32",
1609
+ "shape": [
1610
+ 3072,
1611
+ 128
1612
+ ]
1613
+ },
1614
+ "blocks.19.attn2.to_k.lora_down.weight": {
1615
+ "dtype": "F32",
1616
+ "shape": [
1617
+ 128,
1618
+ 3072
1619
+ ]
1620
+ },
1621
+ "blocks.19.attn2.to_k.lora_up.weight": {
1622
+ "dtype": "F32",
1623
+ "shape": [
1624
+ 3072,
1625
+ 128
1626
+ ]
1627
+ },
1628
+ "blocks.19.attn2.to_out.0.lora_down.weight": {
1629
+ "dtype": "F32",
1630
+ "shape": [
1631
+ 128,
1632
+ 3072
1633
+ ]
1634
+ },
1635
+ "blocks.19.attn2.to_out.0.lora_up.weight": {
1636
+ "dtype": "F32",
1637
+ "shape": [
1638
+ 3072,
1639
+ 128
1640
+ ]
1641
+ },
1642
+ "blocks.19.attn2.to_q.lora_down.weight": {
1643
+ "dtype": "F32",
1644
+ "shape": [
1645
+ 128,
1646
+ 3072
1647
+ ]
1648
+ },
1649
+ "blocks.19.attn2.to_q.lora_up.weight": {
1650
+ "dtype": "F32",
1651
+ "shape": [
1652
+ 3072,
1653
+ 128
1654
+ ]
1655
+ },
1656
+ "blocks.19.attn2.to_v.lora_down.weight": {
1657
+ "dtype": "F32",
1658
+ "shape": [
1659
+ 128,
1660
+ 3072
1661
+ ]
1662
+ },
1663
+ "blocks.19.attn2.to_v.lora_up.weight": {
1664
+ "dtype": "F32",
1665
+ "shape": [
1666
+ 3072,
1667
+ 128
1668
+ ]
1669
+ },
1670
+ "blocks.19.ffn.net.0.proj.lora_down.weight": {
1671
+ "dtype": "F32",
1672
+ "shape": [
1673
+ 128,
1674
+ 3072
1675
+ ]
1676
+ },
1677
+ "blocks.19.ffn.net.0.proj.lora_up.weight": {
1678
+ "dtype": "F32",
1679
+ "shape": [
1680
+ 14336,
1681
+ 128
1682
+ ]
1683
+ },
1684
+ "blocks.19.ffn.net.2.lora_down.weight": {
1685
+ "dtype": "F32",
1686
+ "shape": [
1687
+ 128,
1688
+ 14336
1689
+ ]
1690
+ },
1691
+ "blocks.19.ffn.net.2.lora_up.weight": {
1692
+ "dtype": "F32",
1693
+ "shape": [
1694
+ 3072,
1695
+ 128
1696
+ ]
1697
+ },
1698
+ "blocks.2.attn1.to_k.lora_down.weight": {
1699
+ "dtype": "F32",
1700
+ "shape": [
1701
+ 128,
1702
+ 3072
1703
+ ]
1704
+ },
1705
+ "blocks.2.attn1.to_k.lora_up.weight": {
1706
+ "dtype": "F32",
1707
+ "shape": [
1708
+ 3072,
1709
+ 128
1710
+ ]
1711
+ },
1712
+ "blocks.2.attn1.to_out.0.lora_down.weight": {
1713
+ "dtype": "F32",
1714
+ "shape": [
1715
+ 128,
1716
+ 3072
1717
+ ]
1718
+ },
1719
+ "blocks.2.attn1.to_out.0.lora_up.weight": {
1720
+ "dtype": "F32",
1721
+ "shape": [
1722
+ 3072,
1723
+ 128
1724
+ ]
1725
+ },
1726
+ "blocks.2.attn1.to_q.lora_down.weight": {
1727
+ "dtype": "F32",
1728
+ "shape": [
1729
+ 128,
1730
+ 3072
1731
+ ]
1732
+ },
1733
+ "blocks.2.attn1.to_q.lora_up.weight": {
1734
+ "dtype": "F32",
1735
+ "shape": [
1736
+ 3072,
1737
+ 128
1738
+ ]
1739
+ },
1740
+ "blocks.2.attn1.to_v.lora_down.weight": {
1741
+ "dtype": "F32",
1742
+ "shape": [
1743
+ 128,
1744
+ 3072
1745
+ ]
1746
+ },
1747
+ "blocks.2.attn1.to_v.lora_up.weight": {
1748
+ "dtype": "F32",
1749
+ "shape": [
1750
+ 3072,
1751
+ 128
1752
+ ]
1753
+ },
1754
+ "blocks.2.attn2.to_k.lora_down.weight": {
1755
+ "dtype": "F32",
1756
+ "shape": [
1757
+ 128,
1758
+ 3072
1759
+ ]
1760
+ },
1761
+ "blocks.2.attn2.to_k.lora_up.weight": {
1762
+ "dtype": "F32",
1763
+ "shape": [
1764
+ 3072,
1765
+ 128
1766
+ ]
1767
+ },
1768
+ "blocks.2.attn2.to_out.0.lora_down.weight": {
1769
+ "dtype": "F32",
1770
+ "shape": [
1771
+ 128,
1772
+ 3072
1773
+ ]
1774
+ },
1775
+ "blocks.2.attn2.to_out.0.lora_up.weight": {
1776
+ "dtype": "F32",
1777
+ "shape": [
1778
+ 3072,
1779
+ 128
1780
+ ]
1781
+ },
1782
+ "blocks.2.attn2.to_q.lora_down.weight": {
1783
+ "dtype": "F32",
1784
+ "shape": [
1785
+ 128,
1786
+ 3072
1787
+ ]
1788
+ },
1789
+ "blocks.2.attn2.to_q.lora_up.weight": {
1790
+ "dtype": "F32",
1791
+ "shape": [
1792
+ 3072,
1793
+ 128
1794
+ ]
1795
+ },
1796
+ "blocks.2.attn2.to_v.lora_down.weight": {
1797
+ "dtype": "F32",
1798
+ "shape": [
1799
+ 128,
1800
+ 3072
1801
+ ]
1802
+ },
1803
+ "blocks.2.attn2.to_v.lora_up.weight": {
1804
+ "dtype": "F32",
1805
+ "shape": [
1806
+ 3072,
1807
+ 128
1808
+ ]
1809
+ },
1810
+ "blocks.2.ffn.net.0.proj.lora_down.weight": {
1811
+ "dtype": "F32",
1812
+ "shape": [
1813
+ 128,
1814
+ 3072
1815
+ ]
1816
+ },
1817
+ "blocks.2.ffn.net.0.proj.lora_up.weight": {
1818
+ "dtype": "F32",
1819
+ "shape": [
1820
+ 14336,
1821
+ 128
1822
+ ]
1823
+ },
1824
+ "blocks.2.ffn.net.2.lora_down.weight": {
1825
+ "dtype": "F32",
1826
+ "shape": [
1827
+ 128,
1828
+ 14336
1829
+ ]
1830
+ },
1831
+ "blocks.2.ffn.net.2.lora_up.weight": {
1832
+ "dtype": "F32",
1833
+ "shape": [
1834
+ 3072,
1835
+ 128
1836
+ ]
1837
+ },
1838
+ "blocks.20.attn1.to_k.lora_down.weight": {
1839
+ "dtype": "F32",
1840
+ "shape": [
1841
+ 128,
1842
+ 3072
1843
+ ]
1844
+ },
1845
+ "blocks.20.attn1.to_k.lora_up.weight": {
1846
+ "dtype": "F32",
1847
+ "shape": [
1848
+ 3072,
1849
+ 128
1850
+ ]
1851
+ },
1852
+ "blocks.20.attn1.to_out.0.lora_down.weight": {
1853
+ "dtype": "F32",
1854
+ "shape": [
1855
+ 128,
1856
+ 3072
1857
+ ]
1858
+ },
1859
+ "blocks.20.attn1.to_out.0.lora_up.weight": {
1860
+ "dtype": "F32",
1861
+ "shape": [
1862
+ 3072,
1863
+ 128
1864
+ ]
1865
+ },
1866
+ "blocks.20.attn1.to_q.lora_down.weight": {
1867
+ "dtype": "F32",
1868
+ "shape": [
1869
+ 128,
1870
+ 3072
1871
+ ]
1872
+ },
1873
+ "blocks.20.attn1.to_q.lora_up.weight": {
1874
+ "dtype": "F32",
1875
+ "shape": [
1876
+ 3072,
1877
+ 128
1878
+ ]
1879
+ },
1880
+ "blocks.20.attn1.to_v.lora_down.weight": {
1881
+ "dtype": "F32",
1882
+ "shape": [
1883
+ 128,
1884
+ 3072
1885
+ ]
1886
+ },
1887
+ "blocks.20.attn1.to_v.lora_up.weight": {
1888
+ "dtype": "F32",
1889
+ "shape": [
1890
+ 3072,
1891
+ 128
1892
+ ]
1893
+ },
1894
+ "blocks.20.attn2.to_k.lora_down.weight": {
1895
+ "dtype": "F32",
1896
+ "shape": [
1897
+ 128,
1898
+ 3072
1899
+ ]
1900
+ },
1901
+ "blocks.20.attn2.to_k.lora_up.weight": {
1902
+ "dtype": "F32",
1903
+ "shape": [
1904
+ 3072,
1905
+ 128
1906
+ ]
1907
+ },
1908
+ "blocks.20.attn2.to_out.0.lora_down.weight": {
1909
+ "dtype": "F32",
1910
+ "shape": [
1911
+ 128,
1912
+ 3072
1913
+ ]
1914
+ },
1915
+ "blocks.20.attn2.to_out.0.lora_up.weight": {
1916
+ "dtype": "F32",
1917
+ "shape": [
1918
+ 3072,
1919
+ 128
1920
+ ]
1921
+ },
1922
+ "blocks.20.attn2.to_q.lora_down.weight": {
1923
+ "dtype": "F32",
1924
+ "shape": [
1925
+ 128,
1926
+ 3072
1927
+ ]
1928
+ },
1929
+ "blocks.20.attn2.to_q.lora_up.weight": {
1930
+ "dtype": "F32",
1931
+ "shape": [
1932
+ 3072,
1933
+ 128
1934
+ ]
1935
+ },
1936
+ "blocks.20.attn2.to_v.lora_down.weight": {
1937
+ "dtype": "F32",
1938
+ "shape": [
1939
+ 128,
1940
+ 3072
1941
+ ]
1942
+ },
1943
+ "blocks.20.attn2.to_v.lora_up.weight": {
1944
+ "dtype": "F32",
1945
+ "shape": [
1946
+ 3072,
1947
+ 128
1948
+ ]
1949
+ },
1950
+ "blocks.20.ffn.net.0.proj.lora_down.weight": {
1951
+ "dtype": "F32",
1952
+ "shape": [
1953
+ 128,
1954
+ 3072
1955
+ ]
1956
+ },
1957
+ "blocks.20.ffn.net.0.proj.lora_up.weight": {
1958
+ "dtype": "F32",
1959
+ "shape": [
1960
+ 14336,
1961
+ 128
1962
+ ]
1963
+ },
1964
+ "blocks.20.ffn.net.2.lora_down.weight": {
1965
+ "dtype": "F32",
1966
+ "shape": [
1967
+ 128,
1968
+ 14336
1969
+ ]
1970
+ },
1971
+ "blocks.20.ffn.net.2.lora_up.weight": {
1972
+ "dtype": "F32",
1973
+ "shape": [
1974
+ 3072,
1975
+ 128
1976
+ ]
1977
+ },
1978
+ "blocks.21.attn1.to_k.lora_down.weight": {
1979
+ "dtype": "F32",
1980
+ "shape": [
1981
+ 128,
1982
+ 3072
1983
+ ]
1984
+ },
1985
+ "blocks.21.attn1.to_k.lora_up.weight": {
1986
+ "dtype": "F32",
1987
+ "shape": [
1988
+ 3072,
1989
+ 128
1990
+ ]
1991
+ },
1992
+ "blocks.21.attn1.to_out.0.lora_down.weight": {
1993
+ "dtype": "F32",
1994
+ "shape": [
1995
+ 128,
1996
+ 3072
1997
+ ]
1998
+ },
1999
+ "blocks.21.attn1.to_out.0.lora_up.weight": {
2000
+ "dtype": "F32",
2001
+ "shape": [
2002
+ 3072,
2003
+ 128
2004
+ ]
2005
+ },
2006
+ "blocks.21.attn1.to_q.lora_down.weight": {
2007
+ "dtype": "F32",
2008
+ "shape": [
2009
+ 128,
2010
+ 3072
2011
+ ]
2012
+ },
2013
+ "blocks.21.attn1.to_q.lora_up.weight": {
2014
+ "dtype": "F32",
2015
+ "shape": [
2016
+ 3072,
2017
+ 128
2018
+ ]
2019
+ },
2020
+ "blocks.21.attn1.to_v.lora_down.weight": {
2021
+ "dtype": "F32",
2022
+ "shape": [
2023
+ 128,
2024
+ 3072
2025
+ ]
2026
+ },
2027
+ "blocks.21.attn1.to_v.lora_up.weight": {
2028
+ "dtype": "F32",
2029
+ "shape": [
2030
+ 3072,
2031
+ 128
2032
+ ]
2033
+ },
2034
+ "blocks.21.attn2.to_k.lora_down.weight": {
2035
+ "dtype": "F32",
2036
+ "shape": [
2037
+ 128,
2038
+ 3072
2039
+ ]
2040
+ },
2041
+ "blocks.21.attn2.to_k.lora_up.weight": {
2042
+ "dtype": "F32",
2043
+ "shape": [
2044
+ 3072,
2045
+ 128
2046
+ ]
2047
+ },
2048
+ "blocks.21.attn2.to_out.0.lora_down.weight": {
2049
+ "dtype": "F32",
2050
+ "shape": [
2051
+ 128,
2052
+ 3072
2053
+ ]
2054
+ },
2055
+ "blocks.21.attn2.to_out.0.lora_up.weight": {
2056
+ "dtype": "F32",
2057
+ "shape": [
2058
+ 3072,
2059
+ 128
2060
+ ]
2061
+ },
2062
+ "blocks.21.attn2.to_q.lora_down.weight": {
2063
+ "dtype": "F32",
2064
+ "shape": [
2065
+ 128,
2066
+ 3072
2067
+ ]
2068
+ },
2069
+ "blocks.21.attn2.to_q.lora_up.weight": {
2070
+ "dtype": "F32",
2071
+ "shape": [
2072
+ 3072,
2073
+ 128
2074
+ ]
2075
+ },
2076
+ "blocks.21.attn2.to_v.lora_down.weight": {
2077
+ "dtype": "F32",
2078
+ "shape": [
2079
+ 128,
2080
+ 3072
2081
+ ]
2082
+ },
2083
+ "blocks.21.attn2.to_v.lora_up.weight": {
2084
+ "dtype": "F32",
2085
+ "shape": [
2086
+ 3072,
2087
+ 128
2088
+ ]
2089
+ },
2090
+ "blocks.21.ffn.net.0.proj.lora_down.weight": {
2091
+ "dtype": "F32",
2092
+ "shape": [
2093
+ 128,
2094
+ 3072
2095
+ ]
2096
+ },
2097
+ "blocks.21.ffn.net.0.proj.lora_up.weight": {
2098
+ "dtype": "F32",
2099
+ "shape": [
2100
+ 14336,
2101
+ 128
2102
+ ]
2103
+ },
2104
+ "blocks.21.ffn.net.2.lora_down.weight": {
2105
+ "dtype": "F32",
2106
+ "shape": [
2107
+ 128,
2108
+ 14336
2109
+ ]
2110
+ },
2111
+ "blocks.21.ffn.net.2.lora_up.weight": {
2112
+ "dtype": "F32",
2113
+ "shape": [
2114
+ 3072,
2115
+ 128
2116
+ ]
2117
+ },
2118
+ "blocks.22.attn1.to_k.lora_down.weight": {
2119
+ "dtype": "F32",
2120
+ "shape": [
2121
+ 128,
2122
+ 3072
2123
+ ]
2124
+ },
2125
+ "blocks.22.attn1.to_k.lora_up.weight": {
2126
+ "dtype": "F32",
2127
+ "shape": [
2128
+ 3072,
2129
+ 128
2130
+ ]
2131
+ },
2132
+ "blocks.22.attn1.to_out.0.lora_down.weight": {
2133
+ "dtype": "F32",
2134
+ "shape": [
2135
+ 128,
2136
+ 3072
2137
+ ]
2138
+ },
2139
+ "blocks.22.attn1.to_out.0.lora_up.weight": {
2140
+ "dtype": "F32",
2141
+ "shape": [
2142
+ 3072,
2143
+ 128
2144
+ ]
2145
+ },
2146
+ "blocks.22.attn1.to_q.lora_down.weight": {
2147
+ "dtype": "F32",
2148
+ "shape": [
2149
+ 128,
2150
+ 3072
2151
+ ]
2152
+ },
2153
+ "blocks.22.attn1.to_q.lora_up.weight": {
2154
+ "dtype": "F32",
2155
+ "shape": [
2156
+ 3072,
2157
+ 128
2158
+ ]
2159
+ },
2160
+ "blocks.22.attn1.to_v.lora_down.weight": {
2161
+ "dtype": "F32",
2162
+ "shape": [
2163
+ 128,
2164
+ 3072
2165
+ ]
2166
+ },
2167
+ "blocks.22.attn1.to_v.lora_up.weight": {
2168
+ "dtype": "F32",
2169
+ "shape": [
2170
+ 3072,
2171
+ 128
2172
+ ]
2173
+ },
2174
+ "blocks.22.attn2.to_k.lora_down.weight": {
2175
+ "dtype": "F32",
2176
+ "shape": [
2177
+ 128,
2178
+ 3072
2179
+ ]
2180
+ },
2181
+ "blocks.22.attn2.to_k.lora_up.weight": {
2182
+ "dtype": "F32",
2183
+ "shape": [
2184
+ 3072,
2185
+ 128
2186
+ ]
2187
+ },
2188
+ "blocks.22.attn2.to_out.0.lora_down.weight": {
2189
+ "dtype": "F32",
2190
+ "shape": [
2191
+ 128,
2192
+ 3072
2193
+ ]
2194
+ },
2195
+ "blocks.22.attn2.to_out.0.lora_up.weight": {
2196
+ "dtype": "F32",
2197
+ "shape": [
2198
+ 3072,
2199
+ 128
2200
+ ]
2201
+ },
2202
+ "blocks.22.attn2.to_q.lora_down.weight": {
2203
+ "dtype": "F32",
2204
+ "shape": [
2205
+ 128,
2206
+ 3072
2207
+ ]
2208
+ },
2209
+ "blocks.22.attn2.to_q.lora_up.weight": {
2210
+ "dtype": "F32",
2211
+ "shape": [
2212
+ 3072,
2213
+ 128
2214
+ ]
2215
+ },
2216
+ "blocks.22.attn2.to_v.lora_down.weight": {
2217
+ "dtype": "F32",
2218
+ "shape": [
2219
+ 128,
2220
+ 3072
2221
+ ]
2222
+ },
2223
+ "blocks.22.attn2.to_v.lora_up.weight": {
2224
+ "dtype": "F32",
2225
+ "shape": [
2226
+ 3072,
2227
+ 128
2228
+ ]
2229
+ },
2230
+ "blocks.22.ffn.net.0.proj.lora_down.weight": {
2231
+ "dtype": "F32",
2232
+ "shape": [
2233
+ 128,
2234
+ 3072
2235
+ ]
2236
+ },
2237
+ "blocks.22.ffn.net.0.proj.lora_up.weight": {
2238
+ "dtype": "F32",
2239
+ "shape": [
2240
+ 14336,
2241
+ 128
2242
+ ]
2243
+ },
2244
+ "blocks.22.ffn.net.2.lora_down.weight": {
2245
+ "dtype": "F32",
2246
+ "shape": [
2247
+ 128,
2248
+ 14336
2249
+ ]
2250
+ },
2251
+ "blocks.22.ffn.net.2.lora_up.weight": {
2252
+ "dtype": "F32",
2253
+ "shape": [
2254
+ 3072,
2255
+ 128
2256
+ ]
2257
+ },
2258
+ "blocks.23.attn1.to_k.lora_down.weight": {
2259
+ "dtype": "F32",
2260
+ "shape": [
2261
+ 128,
2262
+ 3072
2263
+ ]
2264
+ },
2265
+ "blocks.23.attn1.to_k.lora_up.weight": {
2266
+ "dtype": "F32",
2267
+ "shape": [
2268
+ 3072,
2269
+ 128
2270
+ ]
2271
+ },
2272
+ "blocks.23.attn1.to_out.0.lora_down.weight": {
2273
+ "dtype": "F32",
2274
+ "shape": [
2275
+ 128,
2276
+ 3072
2277
+ ]
2278
+ },
2279
+ "blocks.23.attn1.to_out.0.lora_up.weight": {
2280
+ "dtype": "F32",
2281
+ "shape": [
2282
+ 3072,
2283
+ 128
2284
+ ]
2285
+ },
2286
+ "blocks.23.attn1.to_q.lora_down.weight": {
2287
+ "dtype": "F32",
2288
+ "shape": [
2289
+ 128,
2290
+ 3072
2291
+ ]
2292
+ },
2293
+ "blocks.23.attn1.to_q.lora_up.weight": {
2294
+ "dtype": "F32",
2295
+ "shape": [
2296
+ 3072,
2297
+ 128
2298
+ ]
2299
+ },
2300
+ "blocks.23.attn1.to_v.lora_down.weight": {
2301
+ "dtype": "F32",
2302
+ "shape": [
2303
+ 128,
2304
+ 3072
2305
+ ]
2306
+ },
2307
+ "blocks.23.attn1.to_v.lora_up.weight": {
2308
+ "dtype": "F32",
2309
+ "shape": [
2310
+ 3072,
2311
+ 128
2312
+ ]
2313
+ },
2314
+ "blocks.23.attn2.to_k.lora_down.weight": {
2315
+ "dtype": "F32",
2316
+ "shape": [
2317
+ 128,
2318
+ 3072
2319
+ ]
2320
+ },
2321
+ "blocks.23.attn2.to_k.lora_up.weight": {
2322
+ "dtype": "F32",
2323
+ "shape": [
2324
+ 3072,
2325
+ 128
2326
+ ]
2327
+ },
2328
+ "blocks.23.attn2.to_out.0.lora_down.weight": {
2329
+ "dtype": "F32",
2330
+ "shape": [
2331
+ 128,
2332
+ 3072
2333
+ ]
2334
+ },
2335
+ "blocks.23.attn2.to_out.0.lora_up.weight": {
2336
+ "dtype": "F32",
2337
+ "shape": [
2338
+ 3072,
2339
+ 128
2340
+ ]
2341
+ },
2342
+ "blocks.23.attn2.to_q.lora_down.weight": {
2343
+ "dtype": "F32",
2344
+ "shape": [
2345
+ 128,
2346
+ 3072
2347
+ ]
2348
+ },
2349
+ "blocks.23.attn2.to_q.lora_up.weight": {
2350
+ "dtype": "F32",
2351
+ "shape": [
2352
+ 3072,
2353
+ 128
2354
+ ]
2355
+ },
2356
+ "blocks.23.attn2.to_v.lora_down.weight": {
2357
+ "dtype": "F32",
2358
+ "shape": [
2359
+ 128,
2360
+ 3072
2361
+ ]
2362
+ },
2363
+ "blocks.23.attn2.to_v.lora_up.weight": {
2364
+ "dtype": "F32",
2365
+ "shape": [
2366
+ 3072,
2367
+ 128
2368
+ ]
2369
+ },
2370
+ "blocks.23.ffn.net.0.proj.lora_down.weight": {
2371
+ "dtype": "F32",
2372
+ "shape": [
2373
+ 128,
2374
+ 3072
2375
+ ]
2376
+ },
2377
+ "blocks.23.ffn.net.0.proj.lora_up.weight": {
2378
+ "dtype": "F32",
2379
+ "shape": [
2380
+ 14336,
2381
+ 128
2382
+ ]
2383
+ },
2384
+ "blocks.23.ffn.net.2.lora_down.weight": {
2385
+ "dtype": "F32",
2386
+ "shape": [
2387
+ 128,
2388
+ 14336
2389
+ ]
2390
+ },
2391
+ "blocks.23.ffn.net.2.lora_up.weight": {
2392
+ "dtype": "F32",
2393
+ "shape": [
2394
+ 3072,
2395
+ 128
2396
+ ]
2397
+ },
2398
+ "blocks.24.attn1.to_k.lora_down.weight": {
2399
+ "dtype": "F32",
2400
+ "shape": [
2401
+ 128,
2402
+ 3072
2403
+ ]
2404
+ },
2405
+ "blocks.24.attn1.to_k.lora_up.weight": {
2406
+ "dtype": "F32",
2407
+ "shape": [
2408
+ 3072,
2409
+ 128
2410
+ ]
2411
+ },
2412
+ "blocks.24.attn1.to_out.0.lora_down.weight": {
2413
+ "dtype": "F32",
2414
+ "shape": [
2415
+ 128,
2416
+ 3072
2417
+ ]
2418
+ },
2419
+ "blocks.24.attn1.to_out.0.lora_up.weight": {
2420
+ "dtype": "F32",
2421
+ "shape": [
2422
+ 3072,
2423
+ 128
2424
+ ]
2425
+ },
2426
+ "blocks.24.attn1.to_q.lora_down.weight": {
2427
+ "dtype": "F32",
2428
+ "shape": [
2429
+ 128,
2430
+ 3072
2431
+ ]
2432
+ },
2433
+ "blocks.24.attn1.to_q.lora_up.weight": {
2434
+ "dtype": "F32",
2435
+ "shape": [
2436
+ 3072,
2437
+ 128
2438
+ ]
2439
+ },
2440
+ "blocks.24.attn1.to_v.lora_down.weight": {
2441
+ "dtype": "F32",
2442
+ "shape": [
2443
+ 128,
2444
+ 3072
2445
+ ]
2446
+ },
2447
+ "blocks.24.attn1.to_v.lora_up.weight": {
2448
+ "dtype": "F32",
2449
+ "shape": [
2450
+ 3072,
2451
+ 128
2452
+ ]
2453
+ },
2454
+ "blocks.24.attn2.to_k.lora_down.weight": {
2455
+ "dtype": "F32",
2456
+ "shape": [
2457
+ 128,
2458
+ 3072
2459
+ ]
2460
+ },
2461
+ "blocks.24.attn2.to_k.lora_up.weight": {
2462
+ "dtype": "F32",
2463
+ "shape": [
2464
+ 3072,
2465
+ 128
2466
+ ]
2467
+ },
2468
+ "blocks.24.attn2.to_out.0.lora_down.weight": {
2469
+ "dtype": "F32",
2470
+ "shape": [
2471
+ 128,
2472
+ 3072
2473
+ ]
2474
+ },
2475
+ "blocks.24.attn2.to_out.0.lora_up.weight": {
2476
+ "dtype": "F32",
2477
+ "shape": [
2478
+ 3072,
2479
+ 128
2480
+ ]
2481
+ },
2482
+ "blocks.24.attn2.to_q.lora_down.weight": {
2483
+ "dtype": "F32",
2484
+ "shape": [
2485
+ 128,
2486
+ 3072
2487
+ ]
2488
+ },
2489
+ "blocks.24.attn2.to_q.lora_up.weight": {
2490
+ "dtype": "F32",
2491
+ "shape": [
2492
+ 3072,
2493
+ 128
2494
+ ]
2495
+ },
2496
+ "blocks.24.attn2.to_v.lora_down.weight": {
2497
+ "dtype": "F32",
2498
+ "shape": [
2499
+ 128,
2500
+ 3072
2501
+ ]
2502
+ },
2503
+ "blocks.24.attn2.to_v.lora_up.weight": {
2504
+ "dtype": "F32",
2505
+ "shape": [
2506
+ 3072,
2507
+ 128
2508
+ ]
2509
+ },
2510
+ "blocks.24.ffn.net.0.proj.lora_down.weight": {
2511
+ "dtype": "F32",
2512
+ "shape": [
2513
+ 128,
2514
+ 3072
2515
+ ]
2516
+ },
2517
+ "blocks.24.ffn.net.0.proj.lora_up.weight": {
2518
+ "dtype": "F32",
2519
+ "shape": [
2520
+ 14336,
2521
+ 128
2522
+ ]
2523
+ },
2524
+ "blocks.24.ffn.net.2.lora_down.weight": {
2525
+ "dtype": "F32",
2526
+ "shape": [
2527
+ 128,
2528
+ 14336
2529
+ ]
2530
+ },
2531
+ "blocks.24.ffn.net.2.lora_up.weight": {
2532
+ "dtype": "F32",
2533
+ "shape": [
2534
+ 3072,
2535
+ 128
2536
+ ]
2537
+ },
2538
+ "blocks.25.attn1.to_k.lora_down.weight": {
2539
+ "dtype": "F32",
2540
+ "shape": [
2541
+ 128,
2542
+ 3072
2543
+ ]
2544
+ },
2545
+ "blocks.25.attn1.to_k.lora_up.weight": {
2546
+ "dtype": "F32",
2547
+ "shape": [
2548
+ 3072,
2549
+ 128
2550
+ ]
2551
+ },
2552
+ "blocks.25.attn1.to_out.0.lora_down.weight": {
2553
+ "dtype": "F32",
2554
+ "shape": [
2555
+ 128,
2556
+ 3072
2557
+ ]
2558
+ },
2559
+ "blocks.25.attn1.to_out.0.lora_up.weight": {
2560
+ "dtype": "F32",
2561
+ "shape": [
2562
+ 3072,
2563
+ 128
2564
+ ]
2565
+ },
2566
+ "blocks.25.attn1.to_q.lora_down.weight": {
2567
+ "dtype": "F32",
2568
+ "shape": [
2569
+ 128,
2570
+ 3072
2571
+ ]
2572
+ },
2573
+ "blocks.25.attn1.to_q.lora_up.weight": {
2574
+ "dtype": "F32",
2575
+ "shape": [
2576
+ 3072,
2577
+ 128
2578
+ ]
2579
+ },
2580
+ "blocks.25.attn1.to_v.lora_down.weight": {
2581
+ "dtype": "F32",
2582
+ "shape": [
2583
+ 128,
2584
+ 3072
2585
+ ]
2586
+ },
2587
+ "blocks.25.attn1.to_v.lora_up.weight": {
2588
+ "dtype": "F32",
2589
+ "shape": [
2590
+ 3072,
2591
+ 128
2592
+ ]
2593
+ },
2594
+ "blocks.25.attn2.to_k.lora_down.weight": {
2595
+ "dtype": "F32",
2596
+ "shape": [
2597
+ 128,
2598
+ 3072
2599
+ ]
2600
+ },
2601
+ "blocks.25.attn2.to_k.lora_up.weight": {
2602
+ "dtype": "F32",
2603
+ "shape": [
2604
+ 3072,
2605
+ 128
2606
+ ]
2607
+ },
2608
+ "blocks.25.attn2.to_out.0.lora_down.weight": {
2609
+ "dtype": "F32",
2610
+ "shape": [
2611
+ 128,
2612
+ 3072
2613
+ ]
2614
+ },
2615
+ "blocks.25.attn2.to_out.0.lora_up.weight": {
2616
+ "dtype": "F32",
2617
+ "shape": [
2618
+ 3072,
2619
+ 128
2620
+ ]
2621
+ },
2622
+ "blocks.25.attn2.to_q.lora_down.weight": {
2623
+ "dtype": "F32",
2624
+ "shape": [
2625
+ 128,
2626
+ 3072
2627
+ ]
2628
+ },
2629
+ "blocks.25.attn2.to_q.lora_up.weight": {
2630
+ "dtype": "F32",
2631
+ "shape": [
2632
+ 3072,
2633
+ 128
2634
+ ]
2635
+ },
2636
+ "blocks.25.attn2.to_v.lora_down.weight": {
2637
+ "dtype": "F32",
2638
+ "shape": [
2639
+ 128,
2640
+ 3072
2641
+ ]
2642
+ },
2643
+ "blocks.25.attn2.to_v.lora_up.weight": {
2644
+ "dtype": "F32",
2645
+ "shape": [
2646
+ 3072,
2647
+ 128
2648
+ ]
2649
+ },
2650
+ "blocks.25.ffn.net.0.proj.lora_down.weight": {
2651
+ "dtype": "F32",
2652
+ "shape": [
2653
+ 128,
2654
+ 3072
2655
+ ]
2656
+ },
2657
+ "blocks.25.ffn.net.0.proj.lora_up.weight": {
2658
+ "dtype": "F32",
2659
+ "shape": [
2660
+ 14336,
2661
+ 128
2662
+ ]
2663
+ },
2664
+ "blocks.25.ffn.net.2.lora_down.weight": {
2665
+ "dtype": "F32",
2666
+ "shape": [
2667
+ 128,
2668
+ 14336
2669
+ ]
2670
+ },
2671
+ "blocks.25.ffn.net.2.lora_up.weight": {
2672
+ "dtype": "F32",
2673
+ "shape": [
2674
+ 3072,
2675
+ 128
2676
+ ]
2677
+ },
2678
+ "blocks.26.attn1.to_k.lora_down.weight": {
2679
+ "dtype": "F32",
2680
+ "shape": [
2681
+ 128,
2682
+ 3072
2683
+ ]
2684
+ },
2685
+ "blocks.26.attn1.to_k.lora_up.weight": {
2686
+ "dtype": "F32",
2687
+ "shape": [
2688
+ 3072,
2689
+ 128
2690
+ ]
2691
+ },
2692
+ "blocks.26.attn1.to_out.0.lora_down.weight": {
2693
+ "dtype": "F32",
2694
+ "shape": [
2695
+ 128,
2696
+ 3072
2697
+ ]
2698
+ },
2699
+ "blocks.26.attn1.to_out.0.lora_up.weight": {
2700
+ "dtype": "F32",
2701
+ "shape": [
2702
+ 3072,
2703
+ 128
2704
+ ]
2705
+ },
2706
+ "blocks.26.attn1.to_q.lora_down.weight": {
2707
+ "dtype": "F32",
2708
+ "shape": [
2709
+ 128,
2710
+ 3072
2711
+ ]
2712
+ },
2713
+ "blocks.26.attn1.to_q.lora_up.weight": {
2714
+ "dtype": "F32",
2715
+ "shape": [
2716
+ 3072,
2717
+ 128
2718
+ ]
2719
+ },
2720
+ "blocks.26.attn1.to_v.lora_down.weight": {
2721
+ "dtype": "F32",
2722
+ "shape": [
2723
+ 128,
2724
+ 3072
2725
+ ]
2726
+ },
2727
+ "blocks.26.attn1.to_v.lora_up.weight": {
2728
+ "dtype": "F32",
2729
+ "shape": [
2730
+ 3072,
2731
+ 128
2732
+ ]
2733
+ },
2734
+ "blocks.26.attn2.to_k.lora_down.weight": {
2735
+ "dtype": "F32",
2736
+ "shape": [
2737
+ 128,
2738
+ 3072
2739
+ ]
2740
+ },
2741
+ "blocks.26.attn2.to_k.lora_up.weight": {
2742
+ "dtype": "F32",
2743
+ "shape": [
2744
+ 3072,
2745
+ 128
2746
+ ]
2747
+ },
2748
+ "blocks.26.attn2.to_out.0.lora_down.weight": {
2749
+ "dtype": "F32",
2750
+ "shape": [
2751
+ 128,
2752
+ 3072
2753
+ ]
2754
+ },
2755
+ "blocks.26.attn2.to_out.0.lora_up.weight": {
2756
+ "dtype": "F32",
2757
+ "shape": [
2758
+ 3072,
2759
+ 128
2760
+ ]
2761
+ },
2762
+ "blocks.26.attn2.to_q.lora_down.weight": {
2763
+ "dtype": "F32",
2764
+ "shape": [
2765
+ 128,
2766
+ 3072
2767
+ ]
2768
+ },
2769
+ "blocks.26.attn2.to_q.lora_up.weight": {
2770
+ "dtype": "F32",
2771
+ "shape": [
2772
+ 3072,
2773
+ 128
2774
+ ]
2775
+ },
2776
+ "blocks.26.attn2.to_v.lora_down.weight": {
2777
+ "dtype": "F32",
2778
+ "shape": [
2779
+ 128,
2780
+ 3072
2781
+ ]
2782
+ },
2783
+ "blocks.26.attn2.to_v.lora_up.weight": {
2784
+ "dtype": "F32",
2785
+ "shape": [
2786
+ 3072,
2787
+ 128
2788
+ ]
2789
+ },
2790
+ "blocks.26.ffn.net.0.proj.lora_down.weight": {
2791
+ "dtype": "F32",
2792
+ "shape": [
2793
+ 128,
2794
+ 3072
2795
+ ]
2796
+ },
2797
+ "blocks.26.ffn.net.0.proj.lora_up.weight": {
2798
+ "dtype": "F32",
2799
+ "shape": [
2800
+ 14336,
2801
+ 128
2802
+ ]
2803
+ },
2804
+ "blocks.26.ffn.net.2.lora_down.weight": {
2805
+ "dtype": "F32",
2806
+ "shape": [
2807
+ 128,
2808
+ 14336
2809
+ ]
2810
+ },
2811
+ "blocks.26.ffn.net.2.lora_up.weight": {
2812
+ "dtype": "F32",
2813
+ "shape": [
2814
+ 3072,
2815
+ 128
2816
+ ]
2817
+ },
2818
+ "blocks.27.attn1.to_k.lora_down.weight": {
2819
+ "dtype": "F32",
2820
+ "shape": [
2821
+ 128,
2822
+ 3072
2823
+ ]
2824
+ },
2825
+ "blocks.27.attn1.to_k.lora_up.weight": {
2826
+ "dtype": "F32",
2827
+ "shape": [
2828
+ 3072,
2829
+ 128
2830
+ ]
2831
+ },
2832
+ "blocks.27.attn1.to_out.0.lora_down.weight": {
2833
+ "dtype": "F32",
2834
+ "shape": [
2835
+ 128,
2836
+ 3072
2837
+ ]
2838
+ },
2839
+ "blocks.27.attn1.to_out.0.lora_up.weight": {
2840
+ "dtype": "F32",
2841
+ "shape": [
2842
+ 3072,
2843
+ 128
2844
+ ]
2845
+ },
2846
+ "blocks.27.attn1.to_q.lora_down.weight": {
2847
+ "dtype": "F32",
2848
+ "shape": [
2849
+ 128,
2850
+ 3072
2851
+ ]
2852
+ },
2853
+ "blocks.27.attn1.to_q.lora_up.weight": {
2854
+ "dtype": "F32",
2855
+ "shape": [
2856
+ 3072,
2857
+ 128
2858
+ ]
2859
+ },
2860
+ "blocks.27.attn1.to_v.lora_down.weight": {
2861
+ "dtype": "F32",
2862
+ "shape": [
2863
+ 128,
2864
+ 3072
2865
+ ]
2866
+ },
2867
+ "blocks.27.attn1.to_v.lora_up.weight": {
2868
+ "dtype": "F32",
2869
+ "shape": [
2870
+ 3072,
2871
+ 128
2872
+ ]
2873
+ },
2874
+ "blocks.27.attn2.to_k.lora_down.weight": {
2875
+ "dtype": "F32",
2876
+ "shape": [
2877
+ 128,
2878
+ 3072
2879
+ ]
2880
+ },
2881
+ "blocks.27.attn2.to_k.lora_up.weight": {
2882
+ "dtype": "F32",
2883
+ "shape": [
2884
+ 3072,
2885
+ 128
2886
+ ]
2887
+ },
2888
+ "blocks.27.attn2.to_out.0.lora_down.weight": {
2889
+ "dtype": "F32",
2890
+ "shape": [
2891
+ 128,
2892
+ 3072
2893
+ ]
2894
+ },
2895
+ "blocks.27.attn2.to_out.0.lora_up.weight": {
2896
+ "dtype": "F32",
2897
+ "shape": [
2898
+ 3072,
2899
+ 128
2900
+ ]
2901
+ },
2902
+ "blocks.27.attn2.to_q.lora_down.weight": {
2903
+ "dtype": "F32",
2904
+ "shape": [
2905
+ 128,
2906
+ 3072
2907
+ ]
2908
+ },
2909
+ "blocks.27.attn2.to_q.lora_up.weight": {
2910
+ "dtype": "F32",
2911
+ "shape": [
2912
+ 3072,
2913
+ 128
2914
+ ]
2915
+ },
2916
+ "blocks.27.attn2.to_v.lora_down.weight": {
2917
+ "dtype": "F32",
2918
+ "shape": [
2919
+ 128,
2920
+ 3072
2921
+ ]
2922
+ },
2923
+ "blocks.27.attn2.to_v.lora_up.weight": {
2924
+ "dtype": "F32",
2925
+ "shape": [
2926
+ 3072,
2927
+ 128
2928
+ ]
2929
+ },
2930
+ "blocks.27.ffn.net.0.proj.lora_down.weight": {
2931
+ "dtype": "F32",
2932
+ "shape": [
2933
+ 128,
2934
+ 3072
2935
+ ]
2936
+ },
2937
+ "blocks.27.ffn.net.0.proj.lora_up.weight": {
2938
+ "dtype": "F32",
2939
+ "shape": [
2940
+ 14336,
2941
+ 128
2942
+ ]
2943
+ },
2944
+ "blocks.27.ffn.net.2.lora_down.weight": {
2945
+ "dtype": "F32",
2946
+ "shape": [
2947
+ 128,
2948
+ 14336
2949
+ ]
2950
+ },
2951
+ "blocks.27.ffn.net.2.lora_up.weight": {
2952
+ "dtype": "F32",
2953
+ "shape": [
2954
+ 3072,
2955
+ 128
2956
+ ]
2957
+ },
2958
+ "blocks.28.attn1.to_k.lora_down.weight": {
2959
+ "dtype": "F32",
2960
+ "shape": [
2961
+ 128,
2962
+ 3072
2963
+ ]
2964
+ },
2965
+ "blocks.28.attn1.to_k.lora_up.weight": {
2966
+ "dtype": "F32",
2967
+ "shape": [
2968
+ 3072,
2969
+ 128
2970
+ ]
2971
+ },
2972
+ "blocks.28.attn1.to_out.0.lora_down.weight": {
2973
+ "dtype": "F32",
2974
+ "shape": [
2975
+ 128,
2976
+ 3072
2977
+ ]
2978
+ },
2979
+ "blocks.28.attn1.to_out.0.lora_up.weight": {
2980
+ "dtype": "F32",
2981
+ "shape": [
2982
+ 3072,
2983
+ 128
2984
+ ]
2985
+ },
2986
+ "blocks.28.attn1.to_q.lora_down.weight": {
2987
+ "dtype": "F32",
2988
+ "shape": [
2989
+ 128,
2990
+ 3072
2991
+ ]
2992
+ },
2993
+ "blocks.28.attn1.to_q.lora_up.weight": {
2994
+ "dtype": "F32",
2995
+ "shape": [
2996
+ 3072,
2997
+ 128
2998
+ ]
2999
+ },
3000
+ "blocks.28.attn1.to_v.lora_down.weight": {
3001
+ "dtype": "F32",
3002
+ "shape": [
3003
+ 128,
3004
+ 3072
3005
+ ]
3006
+ },
3007
+ "blocks.28.attn1.to_v.lora_up.weight": {
3008
+ "dtype": "F32",
3009
+ "shape": [
3010
+ 3072,
3011
+ 128
3012
+ ]
3013
+ },
3014
+ "blocks.28.attn2.to_k.lora_down.weight": {
3015
+ "dtype": "F32",
3016
+ "shape": [
3017
+ 128,
3018
+ 3072
3019
+ ]
3020
+ },
3021
+ "blocks.28.attn2.to_k.lora_up.weight": {
3022
+ "dtype": "F32",
3023
+ "shape": [
3024
+ 3072,
3025
+ 128
3026
+ ]
3027
+ },
3028
+ "blocks.28.attn2.to_out.0.lora_down.weight": {
3029
+ "dtype": "F32",
3030
+ "shape": [
3031
+ 128,
3032
+ 3072
3033
+ ]
3034
+ },
3035
+ "blocks.28.attn2.to_out.0.lora_up.weight": {
3036
+ "dtype": "F32",
3037
+ "shape": [
3038
+ 3072,
3039
+ 128
3040
+ ]
3041
+ },
3042
+ "blocks.28.attn2.to_q.lora_down.weight": {
3043
+ "dtype": "F32",
3044
+ "shape": [
3045
+ 128,
3046
+ 3072
3047
+ ]
3048
+ },
3049
+ "blocks.28.attn2.to_q.lora_up.weight": {
3050
+ "dtype": "F32",
3051
+ "shape": [
3052
+ 3072,
3053
+ 128
3054
+ ]
3055
+ },
3056
+ "blocks.28.attn2.to_v.lora_down.weight": {
3057
+ "dtype": "F32",
3058
+ "shape": [
3059
+ 128,
3060
+ 3072
3061
+ ]
3062
+ },
3063
+ "blocks.28.attn2.to_v.lora_up.weight": {
3064
+ "dtype": "F32",
3065
+ "shape": [
3066
+ 3072,
3067
+ 128
3068
+ ]
3069
+ },
3070
+ "blocks.28.ffn.net.0.proj.lora_down.weight": {
3071
+ "dtype": "F32",
3072
+ "shape": [
3073
+ 128,
3074
+ 3072
3075
+ ]
3076
+ },
3077
+ "blocks.28.ffn.net.0.proj.lora_up.weight": {
3078
+ "dtype": "F32",
3079
+ "shape": [
3080
+ 14336,
3081
+ 128
3082
+ ]
3083
+ },
3084
+ "blocks.28.ffn.net.2.lora_down.weight": {
3085
+ "dtype": "F32",
3086
+ "shape": [
3087
+ 128,
3088
+ 14336
3089
+ ]
3090
+ },
3091
+ "blocks.28.ffn.net.2.lora_up.weight": {
3092
+ "dtype": "F32",
3093
+ "shape": [
3094
+ 3072,
3095
+ 128
3096
+ ]
3097
+ },
3098
+ "blocks.29.attn1.to_k.lora_down.weight": {
3099
+ "dtype": "F32",
3100
+ "shape": [
3101
+ 128,
3102
+ 3072
3103
+ ]
3104
+ },
3105
+ "blocks.29.attn1.to_k.lora_up.weight": {
3106
+ "dtype": "F32",
3107
+ "shape": [
3108
+ 3072,
3109
+ 128
3110
+ ]
3111
+ },
3112
+ "blocks.29.attn1.to_out.0.lora_down.weight": {
3113
+ "dtype": "F32",
3114
+ "shape": [
3115
+ 128,
3116
+ 3072
3117
+ ]
3118
+ },
3119
+ "blocks.29.attn1.to_out.0.lora_up.weight": {
3120
+ "dtype": "F32",
3121
+ "shape": [
3122
+ 3072,
3123
+ 128
3124
+ ]
3125
+ },
3126
+ "blocks.29.attn1.to_q.lora_down.weight": {
3127
+ "dtype": "F32",
3128
+ "shape": [
3129
+ 128,
3130
+ 3072
3131
+ ]
3132
+ },
3133
+ "blocks.29.attn1.to_q.lora_up.weight": {
3134
+ "dtype": "F32",
3135
+ "shape": [
3136
+ 3072,
3137
+ 128
3138
+ ]
3139
+ },
3140
+ "blocks.29.attn1.to_v.lora_down.weight": {
3141
+ "dtype": "F32",
3142
+ "shape": [
3143
+ 128,
3144
+ 3072
3145
+ ]
3146
+ },
3147
+ "blocks.29.attn1.to_v.lora_up.weight": {
3148
+ "dtype": "F32",
3149
+ "shape": [
3150
+ 3072,
3151
+ 128
3152
+ ]
3153
+ },
3154
+ "blocks.29.attn2.to_k.lora_down.weight": {
3155
+ "dtype": "F32",
3156
+ "shape": [
3157
+ 128,
3158
+ 3072
3159
+ ]
3160
+ },
3161
+ "blocks.29.attn2.to_k.lora_up.weight": {
3162
+ "dtype": "F32",
3163
+ "shape": [
3164
+ 3072,
3165
+ 128
3166
+ ]
3167
+ },
3168
+ "blocks.29.attn2.to_out.0.lora_down.weight": {
3169
+ "dtype": "F32",
3170
+ "shape": [
3171
+ 128,
3172
+ 3072
3173
+ ]
3174
+ },
3175
+ "blocks.29.attn2.to_out.0.lora_up.weight": {
3176
+ "dtype": "F32",
3177
+ "shape": [
3178
+ 3072,
3179
+ 128
3180
+ ]
3181
+ },
3182
+ "blocks.29.attn2.to_q.lora_down.weight": {
3183
+ "dtype": "F32",
3184
+ "shape": [
3185
+ 128,
3186
+ 3072
3187
+ ]
3188
+ },
3189
+ "blocks.29.attn2.to_q.lora_up.weight": {
3190
+ "dtype": "F32",
3191
+ "shape": [
3192
+ 3072,
3193
+ 128
3194
+ ]
3195
+ },
3196
+ "blocks.29.attn2.to_v.lora_down.weight": {
3197
+ "dtype": "F32",
3198
+ "shape": [
3199
+ 128,
3200
+ 3072
3201
+ ]
3202
+ },
3203
+ "blocks.29.attn2.to_v.lora_up.weight": {
3204
+ "dtype": "F32",
3205
+ "shape": [
3206
+ 3072,
3207
+ 128
3208
+ ]
3209
+ },
3210
+ "blocks.29.ffn.net.0.proj.lora_down.weight": {
3211
+ "dtype": "F32",
3212
+ "shape": [
3213
+ 128,
3214
+ 3072
3215
+ ]
3216
+ },
3217
+ "blocks.29.ffn.net.0.proj.lora_up.weight": {
3218
+ "dtype": "F32",
3219
+ "shape": [
3220
+ 14336,
3221
+ 128
3222
+ ]
3223
+ },
3224
+ "blocks.29.ffn.net.2.lora_down.weight": {
3225
+ "dtype": "F32",
3226
+ "shape": [
3227
+ 128,
3228
+ 14336
3229
+ ]
3230
+ },
3231
+ "blocks.29.ffn.net.2.lora_up.weight": {
3232
+ "dtype": "F32",
3233
+ "shape": [
3234
+ 3072,
3235
+ 128
3236
+ ]
3237
+ },
3238
+ "blocks.3.attn1.to_k.lora_down.weight": {
3239
+ "dtype": "F32",
3240
+ "shape": [
3241
+ 128,
3242
+ 3072
3243
+ ]
3244
+ },
3245
+ "blocks.3.attn1.to_k.lora_up.weight": {
3246
+ "dtype": "F32",
3247
+ "shape": [
3248
+ 3072,
3249
+ 128
3250
+ ]
3251
+ },
3252
+ "blocks.3.attn1.to_out.0.lora_down.weight": {
3253
+ "dtype": "F32",
3254
+ "shape": [
3255
+ 128,
3256
+ 3072
3257
+ ]
3258
+ },
3259
+ "blocks.3.attn1.to_out.0.lora_up.weight": {
3260
+ "dtype": "F32",
3261
+ "shape": [
3262
+ 3072,
3263
+ 128
3264
+ ]
3265
+ },
3266
+ "blocks.3.attn1.to_q.lora_down.weight": {
3267
+ "dtype": "F32",
3268
+ "shape": [
3269
+ 128,
3270
+ 3072
3271
+ ]
3272
+ },
3273
+ "blocks.3.attn1.to_q.lora_up.weight": {
3274
+ "dtype": "F32",
3275
+ "shape": [
3276
+ 3072,
3277
+ 128
3278
+ ]
3279
+ },
3280
+ "blocks.3.attn1.to_v.lora_down.weight": {
3281
+ "dtype": "F32",
3282
+ "shape": [
3283
+ 128,
3284
+ 3072
3285
+ ]
3286
+ },
3287
+ "blocks.3.attn1.to_v.lora_up.weight": {
3288
+ "dtype": "F32",
3289
+ "shape": [
3290
+ 3072,
3291
+ 128
3292
+ ]
3293
+ },
3294
+ "blocks.3.attn2.to_k.lora_down.weight": {
3295
+ "dtype": "F32",
3296
+ "shape": [
3297
+ 128,
3298
+ 3072
3299
+ ]
3300
+ },
3301
+ "blocks.3.attn2.to_k.lora_up.weight": {
3302
+ "dtype": "F32",
3303
+ "shape": [
3304
+ 3072,
3305
+ 128
3306
+ ]
3307
+ },
3308
+ "blocks.3.attn2.to_out.0.lora_down.weight": {
3309
+ "dtype": "F32",
3310
+ "shape": [
3311
+ 128,
3312
+ 3072
3313
+ ]
3314
+ },
3315
+ "blocks.3.attn2.to_out.0.lora_up.weight": {
3316
+ "dtype": "F32",
3317
+ "shape": [
3318
+ 3072,
3319
+ 128
3320
+ ]
3321
+ },
3322
+ "blocks.3.attn2.to_q.lora_down.weight": {
3323
+ "dtype": "F32",
3324
+ "shape": [
3325
+ 128,
3326
+ 3072
3327
+ ]
3328
+ },
3329
+ "blocks.3.attn2.to_q.lora_up.weight": {
3330
+ "dtype": "F32",
3331
+ "shape": [
3332
+ 3072,
3333
+ 128
3334
+ ]
3335
+ },
3336
+ "blocks.3.attn2.to_v.lora_down.weight": {
3337
+ "dtype": "F32",
3338
+ "shape": [
3339
+ 128,
3340
+ 3072
3341
+ ]
3342
+ },
3343
+ "blocks.3.attn2.to_v.lora_up.weight": {
3344
+ "dtype": "F32",
3345
+ "shape": [
3346
+ 3072,
3347
+ 128
3348
+ ]
3349
+ },
3350
+ "blocks.3.ffn.net.0.proj.lora_down.weight": {
3351
+ "dtype": "F32",
3352
+ "shape": [
3353
+ 128,
3354
+ 3072
3355
+ ]
3356
+ },
3357
+ "blocks.3.ffn.net.0.proj.lora_up.weight": {
3358
+ "dtype": "F32",
3359
+ "shape": [
3360
+ 14336,
3361
+ 128
3362
+ ]
3363
+ },
3364
+ "blocks.3.ffn.net.2.lora_down.weight": {
3365
+ "dtype": "F32",
3366
+ "shape": [
3367
+ 128,
3368
+ 14336
3369
+ ]
3370
+ },
3371
+ "blocks.3.ffn.net.2.lora_up.weight": {
3372
+ "dtype": "F32",
3373
+ "shape": [
3374
+ 3072,
3375
+ 128
3376
+ ]
3377
+ },
3378
+ "blocks.4.attn1.to_k.lora_down.weight": {
3379
+ "dtype": "F32",
3380
+ "shape": [
3381
+ 128,
3382
+ 3072
3383
+ ]
3384
+ },
3385
+ "blocks.4.attn1.to_k.lora_up.weight": {
3386
+ "dtype": "F32",
3387
+ "shape": [
3388
+ 3072,
3389
+ 128
3390
+ ]
3391
+ },
3392
+ "blocks.4.attn1.to_out.0.lora_down.weight": {
3393
+ "dtype": "F32",
3394
+ "shape": [
3395
+ 128,
3396
+ 3072
3397
+ ]
3398
+ },
3399
+ "blocks.4.attn1.to_out.0.lora_up.weight": {
3400
+ "dtype": "F32",
3401
+ "shape": [
3402
+ 3072,
3403
+ 128
3404
+ ]
3405
+ },
3406
+ "blocks.4.attn1.to_q.lora_down.weight": {
3407
+ "dtype": "F32",
3408
+ "shape": [
3409
+ 128,
3410
+ 3072
3411
+ ]
3412
+ },
3413
+ "blocks.4.attn1.to_q.lora_up.weight": {
3414
+ "dtype": "F32",
3415
+ "shape": [
3416
+ 3072,
3417
+ 128
3418
+ ]
3419
+ },
3420
+ "blocks.4.attn1.to_v.lora_down.weight": {
3421
+ "dtype": "F32",
3422
+ "shape": [
3423
+ 128,
3424
+ 3072
3425
+ ]
3426
+ },
3427
+ "blocks.4.attn1.to_v.lora_up.weight": {
3428
+ "dtype": "F32",
3429
+ "shape": [
3430
+ 3072,
3431
+ 128
3432
+ ]
3433
+ },
3434
+ "blocks.4.attn2.to_k.lora_down.weight": {
3435
+ "dtype": "F32",
3436
+ "shape": [
3437
+ 128,
3438
+ 3072
3439
+ ]
3440
+ },
3441
+ "blocks.4.attn2.to_k.lora_up.weight": {
3442
+ "dtype": "F32",
3443
+ "shape": [
3444
+ 3072,
3445
+ 128
3446
+ ]
3447
+ },
3448
+ "blocks.4.attn2.to_out.0.lora_down.weight": {
3449
+ "dtype": "F32",
3450
+ "shape": [
3451
+ 128,
3452
+ 3072
3453
+ ]
3454
+ },
3455
+ "blocks.4.attn2.to_out.0.lora_up.weight": {
3456
+ "dtype": "F32",
3457
+ "shape": [
3458
+ 3072,
3459
+ 128
3460
+ ]
3461
+ },
3462
+ "blocks.4.attn2.to_q.lora_down.weight": {
3463
+ "dtype": "F32",
3464
+ "shape": [
3465
+ 128,
3466
+ 3072
3467
+ ]
3468
+ },
3469
+ "blocks.4.attn2.to_q.lora_up.weight": {
3470
+ "dtype": "F32",
3471
+ "shape": [
3472
+ 3072,
3473
+ 128
3474
+ ]
3475
+ },
3476
+ "blocks.4.attn2.to_v.lora_down.weight": {
3477
+ "dtype": "F32",
3478
+ "shape": [
3479
+ 128,
3480
+ 3072
3481
+ ]
3482
+ },
3483
+ "blocks.4.attn2.to_v.lora_up.weight": {
3484
+ "dtype": "F32",
3485
+ "shape": [
3486
+ 3072,
3487
+ 128
3488
+ ]
3489
+ },
3490
+ "blocks.4.ffn.net.0.proj.lora_down.weight": {
3491
+ "dtype": "F32",
3492
+ "shape": [
3493
+ 128,
3494
+ 3072
3495
+ ]
3496
+ },
3497
+ "blocks.4.ffn.net.0.proj.lora_up.weight": {
3498
+ "dtype": "F32",
3499
+ "shape": [
3500
+ 14336,
3501
+ 128
3502
+ ]
3503
+ },
3504
+ "blocks.4.ffn.net.2.lora_down.weight": {
3505
+ "dtype": "F32",
3506
+ "shape": [
3507
+ 128,
3508
+ 14336
3509
+ ]
3510
+ },
3511
+ "blocks.4.ffn.net.2.lora_up.weight": {
3512
+ "dtype": "F32",
3513
+ "shape": [
3514
+ 3072,
3515
+ 128
3516
+ ]
3517
+ },
3518
+ "blocks.5.attn1.to_k.lora_down.weight": {
3519
+ "dtype": "F32",
3520
+ "shape": [
3521
+ 128,
3522
+ 3072
3523
+ ]
3524
+ },
3525
+ "blocks.5.attn1.to_k.lora_up.weight": {
3526
+ "dtype": "F32",
3527
+ "shape": [
3528
+ 3072,
3529
+ 128
3530
+ ]
3531
+ },
3532
+ "blocks.5.attn1.to_out.0.lora_down.weight": {
3533
+ "dtype": "F32",
3534
+ "shape": [
3535
+ 128,
3536
+ 3072
3537
+ ]
3538
+ },
3539
+ "blocks.5.attn1.to_out.0.lora_up.weight": {
3540
+ "dtype": "F32",
3541
+ "shape": [
3542
+ 3072,
3543
+ 128
3544
+ ]
3545
+ },
3546
+ "blocks.5.attn1.to_q.lora_down.weight": {
3547
+ "dtype": "F32",
3548
+ "shape": [
3549
+ 128,
3550
+ 3072
3551
+ ]
3552
+ },
3553
+ "blocks.5.attn1.to_q.lora_up.weight": {
3554
+ "dtype": "F32",
3555
+ "shape": [
3556
+ 3072,
3557
+ 128
3558
+ ]
3559
+ },
3560
+ "blocks.5.attn1.to_v.lora_down.weight": {
3561
+ "dtype": "F32",
3562
+ "shape": [
3563
+ 128,
3564
+ 3072
3565
+ ]
3566
+ },
3567
+ "blocks.5.attn1.to_v.lora_up.weight": {
3568
+ "dtype": "F32",
3569
+ "shape": [
3570
+ 3072,
3571
+ 128
3572
+ ]
3573
+ },
3574
+ "blocks.5.attn2.to_k.lora_down.weight": {
3575
+ "dtype": "F32",
3576
+ "shape": [
3577
+ 128,
3578
+ 3072
3579
+ ]
3580
+ },
3581
+ "blocks.5.attn2.to_k.lora_up.weight": {
3582
+ "dtype": "F32",
3583
+ "shape": [
3584
+ 3072,
3585
+ 128
3586
+ ]
3587
+ },
3588
+ "blocks.5.attn2.to_out.0.lora_down.weight": {
3589
+ "dtype": "F32",
3590
+ "shape": [
3591
+ 128,
3592
+ 3072
3593
+ ]
3594
+ },
3595
+ "blocks.5.attn2.to_out.0.lora_up.weight": {
3596
+ "dtype": "F32",
3597
+ "shape": [
3598
+ 3072,
3599
+ 128
3600
+ ]
3601
+ },
3602
+ "blocks.5.attn2.to_q.lora_down.weight": {
3603
+ "dtype": "F32",
3604
+ "shape": [
3605
+ 128,
3606
+ 3072
3607
+ ]
3608
+ },
3609
+ "blocks.5.attn2.to_q.lora_up.weight": {
3610
+ "dtype": "F32",
3611
+ "shape": [
3612
+ 3072,
3613
+ 128
3614
+ ]
3615
+ },
3616
+ "blocks.5.attn2.to_v.lora_down.weight": {
3617
+ "dtype": "F32",
3618
+ "shape": [
3619
+ 128,
3620
+ 3072
3621
+ ]
3622
+ },
3623
+ "blocks.5.attn2.to_v.lora_up.weight": {
3624
+ "dtype": "F32",
3625
+ "shape": [
3626
+ 3072,
3627
+ 128
3628
+ ]
3629
+ },
3630
+ "blocks.5.ffn.net.0.proj.lora_down.weight": {
3631
+ "dtype": "F32",
3632
+ "shape": [
3633
+ 128,
3634
+ 3072
3635
+ ]
3636
+ },
3637
+ "blocks.5.ffn.net.0.proj.lora_up.weight": {
3638
+ "dtype": "F32",
3639
+ "shape": [
3640
+ 14336,
3641
+ 128
3642
+ ]
3643
+ },
3644
+ "blocks.5.ffn.net.2.lora_down.weight": {
3645
+ "dtype": "F32",
3646
+ "shape": [
3647
+ 128,
3648
+ 14336
3649
+ ]
3650
+ },
3651
+ "blocks.5.ffn.net.2.lora_up.weight": {
3652
+ "dtype": "F32",
3653
+ "shape": [
3654
+ 3072,
3655
+ 128
3656
+ ]
3657
+ },
3658
+ "blocks.6.attn1.to_k.lora_down.weight": {
3659
+ "dtype": "F32",
3660
+ "shape": [
3661
+ 128,
3662
+ 3072
3663
+ ]
3664
+ },
3665
+ "blocks.6.attn1.to_k.lora_up.weight": {
3666
+ "dtype": "F32",
3667
+ "shape": [
3668
+ 3072,
3669
+ 128
3670
+ ]
3671
+ },
3672
+ "blocks.6.attn1.to_out.0.lora_down.weight": {
3673
+ "dtype": "F32",
3674
+ "shape": [
3675
+ 128,
3676
+ 3072
3677
+ ]
3678
+ },
3679
+ "blocks.6.attn1.to_out.0.lora_up.weight": {
3680
+ "dtype": "F32",
3681
+ "shape": [
3682
+ 3072,
3683
+ 128
3684
+ ]
3685
+ },
3686
+ "blocks.6.attn1.to_q.lora_down.weight": {
3687
+ "dtype": "F32",
3688
+ "shape": [
3689
+ 128,
3690
+ 3072
3691
+ ]
3692
+ },
3693
+ "blocks.6.attn1.to_q.lora_up.weight": {
3694
+ "dtype": "F32",
3695
+ "shape": [
3696
+ 3072,
3697
+ 128
3698
+ ]
3699
+ },
3700
+ "blocks.6.attn1.to_v.lora_down.weight": {
3701
+ "dtype": "F32",
3702
+ "shape": [
3703
+ 128,
3704
+ 3072
3705
+ ]
3706
+ },
3707
+ "blocks.6.attn1.to_v.lora_up.weight": {
3708
+ "dtype": "F32",
3709
+ "shape": [
3710
+ 3072,
3711
+ 128
3712
+ ]
3713
+ },
3714
+ "blocks.6.attn2.to_k.lora_down.weight": {
3715
+ "dtype": "F32",
3716
+ "shape": [
3717
+ 128,
3718
+ 3072
3719
+ ]
3720
+ },
3721
+ "blocks.6.attn2.to_k.lora_up.weight": {
3722
+ "dtype": "F32",
3723
+ "shape": [
3724
+ 3072,
3725
+ 128
3726
+ ]
3727
+ },
3728
+ "blocks.6.attn2.to_out.0.lora_down.weight": {
3729
+ "dtype": "F32",
3730
+ "shape": [
3731
+ 128,
3732
+ 3072
3733
+ ]
3734
+ },
3735
+ "blocks.6.attn2.to_out.0.lora_up.weight": {
3736
+ "dtype": "F32",
3737
+ "shape": [
3738
+ 3072,
3739
+ 128
3740
+ ]
3741
+ },
3742
+ "blocks.6.attn2.to_q.lora_down.weight": {
3743
+ "dtype": "F32",
3744
+ "shape": [
3745
+ 128,
3746
+ 3072
3747
+ ]
3748
+ },
3749
+ "blocks.6.attn2.to_q.lora_up.weight": {
3750
+ "dtype": "F32",
3751
+ "shape": [
3752
+ 3072,
3753
+ 128
3754
+ ]
3755
+ },
3756
+ "blocks.6.attn2.to_v.lora_down.weight": {
3757
+ "dtype": "F32",
3758
+ "shape": [
3759
+ 128,
3760
+ 3072
3761
+ ]
3762
+ },
3763
+ "blocks.6.attn2.to_v.lora_up.weight": {
3764
+ "dtype": "F32",
3765
+ "shape": [
3766
+ 3072,
3767
+ 128
3768
+ ]
3769
+ },
3770
+ "blocks.6.ffn.net.0.proj.lora_down.weight": {
3771
+ "dtype": "F32",
3772
+ "shape": [
3773
+ 128,
3774
+ 3072
3775
+ ]
3776
+ },
3777
+ "blocks.6.ffn.net.0.proj.lora_up.weight": {
3778
+ "dtype": "F32",
3779
+ "shape": [
3780
+ 14336,
3781
+ 128
3782
+ ]
3783
+ },
3784
+ "blocks.6.ffn.net.2.lora_down.weight": {
3785
+ "dtype": "F32",
3786
+ "shape": [
3787
+ 128,
3788
+ 14336
3789
+ ]
3790
+ },
3791
+ "blocks.6.ffn.net.2.lora_up.weight": {
3792
+ "dtype": "F32",
3793
+ "shape": [
3794
+ 3072,
3795
+ 128
3796
+ ]
3797
+ },
3798
+ "blocks.7.attn1.to_k.lora_down.weight": {
3799
+ "dtype": "F32",
3800
+ "shape": [
3801
+ 128,
3802
+ 3072
3803
+ ]
3804
+ },
3805
+ "blocks.7.attn1.to_k.lora_up.weight": {
3806
+ "dtype": "F32",
3807
+ "shape": [
3808
+ 3072,
3809
+ 128
3810
+ ]
3811
+ },
3812
+ "blocks.7.attn1.to_out.0.lora_down.weight": {
3813
+ "dtype": "F32",
3814
+ "shape": [
3815
+ 128,
3816
+ 3072
3817
+ ]
3818
+ },
3819
+ "blocks.7.attn1.to_out.0.lora_up.weight": {
3820
+ "dtype": "F32",
3821
+ "shape": [
3822
+ 3072,
3823
+ 128
3824
+ ]
3825
+ },
3826
+ "blocks.7.attn1.to_q.lora_down.weight": {
3827
+ "dtype": "F32",
3828
+ "shape": [
3829
+ 128,
3830
+ 3072
3831
+ ]
3832
+ },
3833
+ "blocks.7.attn1.to_q.lora_up.weight": {
3834
+ "dtype": "F32",
3835
+ "shape": [
3836
+ 3072,
3837
+ 128
3838
+ ]
3839
+ },
3840
+ "blocks.7.attn1.to_v.lora_down.weight": {
3841
+ "dtype": "F32",
3842
+ "shape": [
3843
+ 128,
3844
+ 3072
3845
+ ]
3846
+ },
3847
+ "blocks.7.attn1.to_v.lora_up.weight": {
3848
+ "dtype": "F32",
3849
+ "shape": [
3850
+ 3072,
3851
+ 128
3852
+ ]
3853
+ },
3854
+ "blocks.7.attn2.to_k.lora_down.weight": {
3855
+ "dtype": "F32",
3856
+ "shape": [
3857
+ 128,
3858
+ 3072
3859
+ ]
3860
+ },
3861
+ "blocks.7.attn2.to_k.lora_up.weight": {
3862
+ "dtype": "F32",
3863
+ "shape": [
3864
+ 3072,
3865
+ 128
3866
+ ]
3867
+ },
3868
+ "blocks.7.attn2.to_out.0.lora_down.weight": {
3869
+ "dtype": "F32",
3870
+ "shape": [
3871
+ 128,
3872
+ 3072
3873
+ ]
3874
+ },
3875
+ "blocks.7.attn2.to_out.0.lora_up.weight": {
3876
+ "dtype": "F32",
3877
+ "shape": [
3878
+ 3072,
3879
+ 128
3880
+ ]
3881
+ },
3882
+ "blocks.7.attn2.to_q.lora_down.weight": {
3883
+ "dtype": "F32",
3884
+ "shape": [
3885
+ 128,
3886
+ 3072
3887
+ ]
3888
+ },
3889
+ "blocks.7.attn2.to_q.lora_up.weight": {
3890
+ "dtype": "F32",
3891
+ "shape": [
3892
+ 3072,
3893
+ 128
3894
+ ]
3895
+ },
3896
+ "blocks.7.attn2.to_v.lora_down.weight": {
3897
+ "dtype": "F32",
3898
+ "shape": [
3899
+ 128,
3900
+ 3072
3901
+ ]
3902
+ },
3903
+ "blocks.7.attn2.to_v.lora_up.weight": {
3904
+ "dtype": "F32",
3905
+ "shape": [
3906
+ 3072,
3907
+ 128
3908
+ ]
3909
+ },
3910
+ "blocks.7.ffn.net.0.proj.lora_down.weight": {
3911
+ "dtype": "F32",
3912
+ "shape": [
3913
+ 128,
3914
+ 3072
3915
+ ]
3916
+ },
3917
+ "blocks.7.ffn.net.0.proj.lora_up.weight": {
3918
+ "dtype": "F32",
3919
+ "shape": [
3920
+ 14336,
3921
+ 128
3922
+ ]
3923
+ },
3924
+ "blocks.7.ffn.net.2.lora_down.weight": {
3925
+ "dtype": "F32",
3926
+ "shape": [
3927
+ 128,
3928
+ 14336
3929
+ ]
3930
+ },
3931
+ "blocks.7.ffn.net.2.lora_up.weight": {
3932
+ "dtype": "F32",
3933
+ "shape": [
3934
+ 3072,
3935
+ 128
3936
+ ]
3937
+ },
3938
+ "blocks.8.attn1.to_k.lora_down.weight": {
3939
+ "dtype": "F32",
3940
+ "shape": [
3941
+ 128,
3942
+ 3072
3943
+ ]
3944
+ },
3945
+ "blocks.8.attn1.to_k.lora_up.weight": {
3946
+ "dtype": "F32",
3947
+ "shape": [
3948
+ 3072,
3949
+ 128
3950
+ ]
3951
+ },
3952
+ "blocks.8.attn1.to_out.0.lora_down.weight": {
3953
+ "dtype": "F32",
3954
+ "shape": [
3955
+ 128,
3956
+ 3072
3957
+ ]
3958
+ },
3959
+ "blocks.8.attn1.to_out.0.lora_up.weight": {
3960
+ "dtype": "F32",
3961
+ "shape": [
3962
+ 3072,
3963
+ 128
3964
+ ]
3965
+ },
3966
+ "blocks.8.attn1.to_q.lora_down.weight": {
3967
+ "dtype": "F32",
3968
+ "shape": [
3969
+ 128,
3970
+ 3072
3971
+ ]
3972
+ },
3973
+ "blocks.8.attn1.to_q.lora_up.weight": {
3974
+ "dtype": "F32",
3975
+ "shape": [
3976
+ 3072,
3977
+ 128
3978
+ ]
3979
+ },
3980
+ "blocks.8.attn1.to_v.lora_down.weight": {
3981
+ "dtype": "F32",
3982
+ "shape": [
3983
+ 128,
3984
+ 3072
3985
+ ]
3986
+ },
3987
+ "blocks.8.attn1.to_v.lora_up.weight": {
3988
+ "dtype": "F32",
3989
+ "shape": [
3990
+ 3072,
3991
+ 128
3992
+ ]
3993
+ },
3994
+ "blocks.8.attn2.to_k.lora_down.weight": {
3995
+ "dtype": "F32",
3996
+ "shape": [
3997
+ 128,
3998
+ 3072
3999
+ ]
4000
+ },
4001
+ "blocks.8.attn2.to_k.lora_up.weight": {
4002
+ "dtype": "F32",
4003
+ "shape": [
4004
+ 3072,
4005
+ 128
4006
+ ]
4007
+ },
4008
+ "blocks.8.attn2.to_out.0.lora_down.weight": {
4009
+ "dtype": "F32",
4010
+ "shape": [
4011
+ 128,
4012
+ 3072
4013
+ ]
4014
+ },
4015
+ "blocks.8.attn2.to_out.0.lora_up.weight": {
4016
+ "dtype": "F32",
4017
+ "shape": [
4018
+ 3072,
4019
+ 128
4020
+ ]
4021
+ },
4022
+ "blocks.8.attn2.to_q.lora_down.weight": {
4023
+ "dtype": "F32",
4024
+ "shape": [
4025
+ 128,
4026
+ 3072
4027
+ ]
4028
+ },
4029
+ "blocks.8.attn2.to_q.lora_up.weight": {
4030
+ "dtype": "F32",
4031
+ "shape": [
4032
+ 3072,
4033
+ 128
4034
+ ]
4035
+ },
4036
+ "blocks.8.attn2.to_v.lora_down.weight": {
4037
+ "dtype": "F32",
4038
+ "shape": [
4039
+ 128,
4040
+ 3072
4041
+ ]
4042
+ },
4043
+ "blocks.8.attn2.to_v.lora_up.weight": {
4044
+ "dtype": "F32",
4045
+ "shape": [
4046
+ 3072,
4047
+ 128
4048
+ ]
4049
+ },
4050
+ "blocks.8.ffn.net.0.proj.lora_down.weight": {
4051
+ "dtype": "F32",
4052
+ "shape": [
4053
+ 128,
4054
+ 3072
4055
+ ]
4056
+ },
4057
+ "blocks.8.ffn.net.0.proj.lora_up.weight": {
4058
+ "dtype": "F32",
4059
+ "shape": [
4060
+ 14336,
4061
+ 128
4062
+ ]
4063
+ },
4064
+ "blocks.8.ffn.net.2.lora_down.weight": {
4065
+ "dtype": "F32",
4066
+ "shape": [
4067
+ 128,
4068
+ 14336
4069
+ ]
4070
+ },
4071
+ "blocks.8.ffn.net.2.lora_up.weight": {
4072
+ "dtype": "F32",
4073
+ "shape": [
4074
+ 3072,
4075
+ 128
4076
+ ]
4077
+ },
4078
+ "blocks.9.attn1.to_k.lora_down.weight": {
4079
+ "dtype": "F32",
4080
+ "shape": [
4081
+ 128,
4082
+ 3072
4083
+ ]
4084
+ },
4085
+ "blocks.9.attn1.to_k.lora_up.weight": {
4086
+ "dtype": "F32",
4087
+ "shape": [
4088
+ 3072,
4089
+ 128
4090
+ ]
4091
+ },
4092
+ "blocks.9.attn1.to_out.0.lora_down.weight": {
4093
+ "dtype": "F32",
4094
+ "shape": [
4095
+ 128,
4096
+ 3072
4097
+ ]
4098
+ },
4099
+ "blocks.9.attn1.to_out.0.lora_up.weight": {
4100
+ "dtype": "F32",
4101
+ "shape": [
4102
+ 3072,
4103
+ 128
4104
+ ]
4105
+ },
4106
+ "blocks.9.attn1.to_q.lora_down.weight": {
4107
+ "dtype": "F32",
4108
+ "shape": [
4109
+ 128,
4110
+ 3072
4111
+ ]
4112
+ },
4113
+ "blocks.9.attn1.to_q.lora_up.weight": {
4114
+ "dtype": "F32",
4115
+ "shape": [
4116
+ 3072,
4117
+ 128
4118
+ ]
4119
+ },
4120
+ "blocks.9.attn1.to_v.lora_down.weight": {
4121
+ "dtype": "F32",
4122
+ "shape": [
4123
+ 128,
4124
+ 3072
4125
+ ]
4126
+ },
4127
+ "blocks.9.attn1.to_v.lora_up.weight": {
4128
+ "dtype": "F32",
4129
+ "shape": [
4130
+ 3072,
4131
+ 128
4132
+ ]
4133
+ },
4134
+ "blocks.9.attn2.to_k.lora_down.weight": {
4135
+ "dtype": "F32",
4136
+ "shape": [
4137
+ 128,
4138
+ 3072
4139
+ ]
4140
+ },
4141
+ "blocks.9.attn2.to_k.lora_up.weight": {
4142
+ "dtype": "F32",
4143
+ "shape": [
4144
+ 3072,
4145
+ 128
4146
+ ]
4147
+ },
4148
+ "blocks.9.attn2.to_out.0.lora_down.weight": {
4149
+ "dtype": "F32",
4150
+ "shape": [
4151
+ 128,
4152
+ 3072
4153
+ ]
4154
+ },
4155
+ "blocks.9.attn2.to_out.0.lora_up.weight": {
4156
+ "dtype": "F32",
4157
+ "shape": [
4158
+ 3072,
4159
+ 128
4160
+ ]
4161
+ },
4162
+ "blocks.9.attn2.to_q.lora_down.weight": {
4163
+ "dtype": "F32",
4164
+ "shape": [
4165
+ 128,
4166
+ 3072
4167
+ ]
4168
+ },
4169
+ "blocks.9.attn2.to_q.lora_up.weight": {
4170
+ "dtype": "F32",
4171
+ "shape": [
4172
+ 3072,
4173
+ 128
4174
+ ]
4175
+ },
4176
+ "blocks.9.attn2.to_v.lora_down.weight": {
4177
+ "dtype": "F32",
4178
+ "shape": [
4179
+ 128,
4180
+ 3072
4181
+ ]
4182
+ },
4183
+ "blocks.9.attn2.to_v.lora_up.weight": {
4184
+ "dtype": "F32",
4185
+ "shape": [
4186
+ 3072,
4187
+ 128
4188
+ ]
4189
+ },
4190
+ "blocks.9.ffn.net.0.proj.lora_down.weight": {
4191
+ "dtype": "F32",
4192
+ "shape": [
4193
+ 128,
4194
+ 3072
4195
+ ]
4196
+ },
4197
+ "blocks.9.ffn.net.0.proj.lora_up.weight": {
4198
+ "dtype": "F32",
4199
+ "shape": [
4200
+ 14336,
4201
+ 128
4202
+ ]
4203
+ },
4204
+ "blocks.9.ffn.net.2.lora_down.weight": {
4205
+ "dtype": "F32",
4206
+ "shape": [
4207
+ 128,
4208
+ 14336
4209
+ ]
4210
+ },
4211
+ "blocks.9.ffn.net.2.lora_up.weight": {
4212
+ "dtype": "F32",
4213
+ "shape": [
4214
+ 3072,
4215
+ 128
4216
+ ]
4217
+ },
4218
+ "condition_embedder.delta_embedder.linear_1.lora_down.weight": {
4219
+ "dtype": "F32",
4220
+ "shape": [
4221
+ 128,
4222
+ 256
4223
+ ]
4224
+ },
4225
+ "condition_embedder.delta_embedder.linear_1.lora_up.weight": {
4226
+ "dtype": "F32",
4227
+ "shape": [
4228
+ 3072,
4229
+ 128
4230
+ ]
4231
+ },
4232
+ "condition_embedder.delta_embedder.linear_2.lora_down.weight": {
4233
+ "dtype": "F32",
4234
+ "shape": [
4235
+ 128,
4236
+ 3072
4237
+ ]
4238
+ },
4239
+ "condition_embedder.delta_embedder.linear_2.lora_up.weight": {
4240
+ "dtype": "F32",
4241
+ "shape": [
4242
+ 3072,
4243
+ 128
4244
+ ]
4245
+ },
4246
+ "condition_embedder.delta_output.lora_down.weight": {
4247
+ "dtype": "F32",
4248
+ "shape": [
4249
+ 128,
4250
+ 3072
4251
+ ]
4252
+ },
4253
+ "condition_embedder.delta_output.lora_up.weight": {
4254
+ "dtype": "F32",
4255
+ "shape": [
4256
+ 3072,
4257
+ 128
4258
+ ]
4259
+ },
4260
+ "condition_embedder.time_embedder.linear_1.lora_down.weight": {
4261
+ "dtype": "F32",
4262
+ "shape": [
4263
+ 128,
4264
+ 256
4265
+ ]
4266
+ },
4267
+ "condition_embedder.time_embedder.linear_1.lora_up.weight": {
4268
+ "dtype": "F32",
4269
+ "shape": [
4270
+ 3072,
4271
+ 128
4272
+ ]
4273
+ },
4274
+ "condition_embedder.time_embedder.linear_2.lora_down.weight": {
4275
+ "dtype": "F32",
4276
+ "shape": [
4277
+ 128,
4278
+ 3072
4279
+ ]
4280
+ },
4281
+ "condition_embedder.time_embedder.linear_2.lora_up.weight": {
4282
+ "dtype": "F32",
4283
+ "shape": [
4284
+ 3072,
4285
+ 128
4286
+ ]
4287
+ },
4288
+ "condition_embedder.time_proj.lora_down.weight": {
4289
+ "dtype": "F32",
4290
+ "shape": [
4291
+ 128,
4292
+ 3072
4293
+ ]
4294
+ },
4295
+ "condition_embedder.time_proj.lora_up.weight": {
4296
+ "dtype": "F32",
4297
+ "shape": [
4298
+ 18432,
4299
+ 128
4300
+ ]
4301
+ },
4302
+ "condition_embedder_action.delta_embedder.linear_1.lora_down.weight": {
4303
+ "dtype": "F32",
4304
+ "shape": [
4305
+ 128,
4306
+ 256
4307
+ ]
4308
+ },
4309
+ "condition_embedder_action.delta_embedder.linear_1.lora_up.weight": {
4310
+ "dtype": "F32",
4311
+ "shape": [
4312
+ 3072,
4313
+ 128
4314
+ ]
4315
+ },
4316
+ "condition_embedder_action.delta_embedder.linear_2.lora_down.weight": {
4317
+ "dtype": "F32",
4318
+ "shape": [
4319
+ 128,
4320
+ 3072
4321
+ ]
4322
+ },
4323
+ "condition_embedder_action.delta_embedder.linear_2.lora_up.weight": {
4324
+ "dtype": "F32",
4325
+ "shape": [
4326
+ 3072,
4327
+ 128
4328
+ ]
4329
+ },
4330
+ "condition_embedder_action.delta_output.lora_down.weight": {
4331
+ "dtype": "F32",
4332
+ "shape": [
4333
+ 128,
4334
+ 3072
4335
+ ]
4336
+ },
4337
+ "condition_embedder_action.delta_output.lora_up.weight": {
4338
+ "dtype": "F32",
4339
+ "shape": [
4340
+ 3072,
4341
+ 128
4342
+ ]
4343
+ },
4344
+ "condition_embedder_action.time_embedder.linear_1.lora_down.weight": {
4345
+ "dtype": "F32",
4346
+ "shape": [
4347
+ 128,
4348
+ 256
4349
+ ]
4350
+ },
4351
+ "condition_embedder_action.time_embedder.linear_1.lora_up.weight": {
4352
+ "dtype": "F32",
4353
+ "shape": [
4354
+ 3072,
4355
+ 128
4356
+ ]
4357
+ },
4358
+ "condition_embedder_action.time_embedder.linear_2.lora_down.weight": {
4359
+ "dtype": "F32",
4360
+ "shape": [
4361
+ 128,
4362
+ 3072
4363
+ ]
4364
+ },
4365
+ "condition_embedder_action.time_embedder.linear_2.lora_up.weight": {
4366
+ "dtype": "F32",
4367
+ "shape": [
4368
+ 3072,
4369
+ 128
4370
+ ]
4371
+ },
4372
+ "condition_embedder_action.time_proj.lora_down.weight": {
4373
+ "dtype": "F32",
4374
+ "shape": [
4375
+ 128,
4376
+ 3072
4377
+ ]
4378
+ },
4379
+ "condition_embedder_action.time_proj.lora_up.weight": {
4380
+ "dtype": "F32",
4381
+ "shape": [
4382
+ 18432,
4383
+ 128
4384
+ ]
4385
+ }
4386
+ },
4387
+ "kind": "adapter",
4388
+ "license": "Apache-2.0",
4389
+ "paper_role": "continued",
4390
+ "source_digest": "50b99e687204ef2f82b32d6a71eda67cbc472a98ee400094a3a7ab97c2b7289e",
4391
+ "topology": "attention_ffn_time_v1:rank128",
4392
+ "variant_id": "continued"
4393
+ }
variants/original-field-projected/adapter_model.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:b085f2d4c5fce4093ee237a69fe0482f6007b504e337d8057dee6e5247f33959
3
+ size 582514248
variants/original-field-projected/manifest.json ADDED
@@ -0,0 +1,1873 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "artifact_file": "adapter_model.safetensors",
3
+ "artifact_sha256": "b085f2d4c5fce4093ee237a69fe0482f6007b504e337d8057dee6e5247f33959",
4
+ "base_digest": "6f9e4b3f291d4f00a718db798a2774fe1e918daab21f74155644536443c19357",
5
+ "cfg_contract": {
6
+ "deployment_cfg": [
7
+ 5.0,
8
+ 1.0
9
+ ],
10
+ "target_guidance_mode": "inference_composed",
11
+ "teacher_target_cfg": [
12
+ 5.0,
13
+ 1.0
14
+ ],
15
+ "text_dropout_probability": 0.1
16
+ },
17
+ "keys": {
18
+ "blocks.0.ffn.net.0.proj.action.down.weight": {
19
+ "dtype": "F32",
20
+ "shape": [
21
+ 64,
22
+ 3072
23
+ ]
24
+ },
25
+ "blocks.0.ffn.net.0.proj.action.up.weight": {
26
+ "dtype": "F32",
27
+ "shape": [
28
+ 14336,
29
+ 64
30
+ ]
31
+ },
32
+ "blocks.0.ffn.net.0.proj.video.down.weight": {
33
+ "dtype": "F32",
34
+ "shape": [
35
+ 64,
36
+ 3072
37
+ ]
38
+ },
39
+ "blocks.0.ffn.net.0.proj.video.up.weight": {
40
+ "dtype": "F32",
41
+ "shape": [
42
+ 14336,
43
+ 64
44
+ ]
45
+ },
46
+ "blocks.0.ffn.net.2.action.down.weight": {
47
+ "dtype": "F32",
48
+ "shape": [
49
+ 64,
50
+ 14336
51
+ ]
52
+ },
53
+ "blocks.0.ffn.net.2.action.up.weight": {
54
+ "dtype": "F32",
55
+ "shape": [
56
+ 3072,
57
+ 64
58
+ ]
59
+ },
60
+ "blocks.0.ffn.net.2.video.down.weight": {
61
+ "dtype": "F32",
62
+ "shape": [
63
+ 64,
64
+ 14336
65
+ ]
66
+ },
67
+ "blocks.0.ffn.net.2.video.up.weight": {
68
+ "dtype": "F32",
69
+ "shape": [
70
+ 3072,
71
+ 64
72
+ ]
73
+ },
74
+ "blocks.1.ffn.net.0.proj.action.down.weight": {
75
+ "dtype": "F32",
76
+ "shape": [
77
+ 64,
78
+ 3072
79
+ ]
80
+ },
81
+ "blocks.1.ffn.net.0.proj.action.up.weight": {
82
+ "dtype": "F32",
83
+ "shape": [
84
+ 14336,
85
+ 64
86
+ ]
87
+ },
88
+ "blocks.1.ffn.net.0.proj.video.down.weight": {
89
+ "dtype": "F32",
90
+ "shape": [
91
+ 64,
92
+ 3072
93
+ ]
94
+ },
95
+ "blocks.1.ffn.net.0.proj.video.up.weight": {
96
+ "dtype": "F32",
97
+ "shape": [
98
+ 14336,
99
+ 64
100
+ ]
101
+ },
102
+ "blocks.1.ffn.net.2.action.down.weight": {
103
+ "dtype": "F32",
104
+ "shape": [
105
+ 64,
106
+ 14336
107
+ ]
108
+ },
109
+ "blocks.1.ffn.net.2.action.up.weight": {
110
+ "dtype": "F32",
111
+ "shape": [
112
+ 3072,
113
+ 64
114
+ ]
115
+ },
116
+ "blocks.1.ffn.net.2.video.down.weight": {
117
+ "dtype": "F32",
118
+ "shape": [
119
+ 64,
120
+ 14336
121
+ ]
122
+ },
123
+ "blocks.1.ffn.net.2.video.up.weight": {
124
+ "dtype": "F32",
125
+ "shape": [
126
+ 3072,
127
+ 64
128
+ ]
129
+ },
130
+ "blocks.10.ffn.net.0.proj.action.down.weight": {
131
+ "dtype": "F32",
132
+ "shape": [
133
+ 64,
134
+ 3072
135
+ ]
136
+ },
137
+ "blocks.10.ffn.net.0.proj.action.up.weight": {
138
+ "dtype": "F32",
139
+ "shape": [
140
+ 14336,
141
+ 64
142
+ ]
143
+ },
144
+ "blocks.10.ffn.net.0.proj.video.down.weight": {
145
+ "dtype": "F32",
146
+ "shape": [
147
+ 64,
148
+ 3072
149
+ ]
150
+ },
151
+ "blocks.10.ffn.net.0.proj.video.up.weight": {
152
+ "dtype": "F32",
153
+ "shape": [
154
+ 14336,
155
+ 64
156
+ ]
157
+ },
158
+ "blocks.10.ffn.net.2.action.down.weight": {
159
+ "dtype": "F32",
160
+ "shape": [
161
+ 64,
162
+ 14336
163
+ ]
164
+ },
165
+ "blocks.10.ffn.net.2.action.up.weight": {
166
+ "dtype": "F32",
167
+ "shape": [
168
+ 3072,
169
+ 64
170
+ ]
171
+ },
172
+ "blocks.10.ffn.net.2.video.down.weight": {
173
+ "dtype": "F32",
174
+ "shape": [
175
+ 64,
176
+ 14336
177
+ ]
178
+ },
179
+ "blocks.10.ffn.net.2.video.up.weight": {
180
+ "dtype": "F32",
181
+ "shape": [
182
+ 3072,
183
+ 64
184
+ ]
185
+ },
186
+ "blocks.11.ffn.net.0.proj.action.down.weight": {
187
+ "dtype": "F32",
188
+ "shape": [
189
+ 64,
190
+ 3072
191
+ ]
192
+ },
193
+ "blocks.11.ffn.net.0.proj.action.up.weight": {
194
+ "dtype": "F32",
195
+ "shape": [
196
+ 14336,
197
+ 64
198
+ ]
199
+ },
200
+ "blocks.11.ffn.net.0.proj.video.down.weight": {
201
+ "dtype": "F32",
202
+ "shape": [
203
+ 64,
204
+ 3072
205
+ ]
206
+ },
207
+ "blocks.11.ffn.net.0.proj.video.up.weight": {
208
+ "dtype": "F32",
209
+ "shape": [
210
+ 14336,
211
+ 64
212
+ ]
213
+ },
214
+ "blocks.11.ffn.net.2.action.down.weight": {
215
+ "dtype": "F32",
216
+ "shape": [
217
+ 64,
218
+ 14336
219
+ ]
220
+ },
221
+ "blocks.11.ffn.net.2.action.up.weight": {
222
+ "dtype": "F32",
223
+ "shape": [
224
+ 3072,
225
+ 64
226
+ ]
227
+ },
228
+ "blocks.11.ffn.net.2.video.down.weight": {
229
+ "dtype": "F32",
230
+ "shape": [
231
+ 64,
232
+ 14336
233
+ ]
234
+ },
235
+ "blocks.11.ffn.net.2.video.up.weight": {
236
+ "dtype": "F32",
237
+ "shape": [
238
+ 3072,
239
+ 64
240
+ ]
241
+ },
242
+ "blocks.12.ffn.net.0.proj.action.down.weight": {
243
+ "dtype": "F32",
244
+ "shape": [
245
+ 64,
246
+ 3072
247
+ ]
248
+ },
249
+ "blocks.12.ffn.net.0.proj.action.up.weight": {
250
+ "dtype": "F32",
251
+ "shape": [
252
+ 14336,
253
+ 64
254
+ ]
255
+ },
256
+ "blocks.12.ffn.net.0.proj.video.down.weight": {
257
+ "dtype": "F32",
258
+ "shape": [
259
+ 64,
260
+ 3072
261
+ ]
262
+ },
263
+ "blocks.12.ffn.net.0.proj.video.up.weight": {
264
+ "dtype": "F32",
265
+ "shape": [
266
+ 14336,
267
+ 64
268
+ ]
269
+ },
270
+ "blocks.12.ffn.net.2.action.down.weight": {
271
+ "dtype": "F32",
272
+ "shape": [
273
+ 64,
274
+ 14336
275
+ ]
276
+ },
277
+ "blocks.12.ffn.net.2.action.up.weight": {
278
+ "dtype": "F32",
279
+ "shape": [
280
+ 3072,
281
+ 64
282
+ ]
283
+ },
284
+ "blocks.12.ffn.net.2.video.down.weight": {
285
+ "dtype": "F32",
286
+ "shape": [
287
+ 64,
288
+ 14336
289
+ ]
290
+ },
291
+ "blocks.12.ffn.net.2.video.up.weight": {
292
+ "dtype": "F32",
293
+ "shape": [
294
+ 3072,
295
+ 64
296
+ ]
297
+ },
298
+ "blocks.13.ffn.net.0.proj.action.down.weight": {
299
+ "dtype": "F32",
300
+ "shape": [
301
+ 64,
302
+ 3072
303
+ ]
304
+ },
305
+ "blocks.13.ffn.net.0.proj.action.up.weight": {
306
+ "dtype": "F32",
307
+ "shape": [
308
+ 14336,
309
+ 64
310
+ ]
311
+ },
312
+ "blocks.13.ffn.net.0.proj.video.down.weight": {
313
+ "dtype": "F32",
314
+ "shape": [
315
+ 64,
316
+ 3072
317
+ ]
318
+ },
319
+ "blocks.13.ffn.net.0.proj.video.up.weight": {
320
+ "dtype": "F32",
321
+ "shape": [
322
+ 14336,
323
+ 64
324
+ ]
325
+ },
326
+ "blocks.13.ffn.net.2.action.down.weight": {
327
+ "dtype": "F32",
328
+ "shape": [
329
+ 64,
330
+ 14336
331
+ ]
332
+ },
333
+ "blocks.13.ffn.net.2.action.up.weight": {
334
+ "dtype": "F32",
335
+ "shape": [
336
+ 3072,
337
+ 64
338
+ ]
339
+ },
340
+ "blocks.13.ffn.net.2.video.down.weight": {
341
+ "dtype": "F32",
342
+ "shape": [
343
+ 64,
344
+ 14336
345
+ ]
346
+ },
347
+ "blocks.13.ffn.net.2.video.up.weight": {
348
+ "dtype": "F32",
349
+ "shape": [
350
+ 3072,
351
+ 64
352
+ ]
353
+ },
354
+ "blocks.14.ffn.net.0.proj.action.down.weight": {
355
+ "dtype": "F32",
356
+ "shape": [
357
+ 64,
358
+ 3072
359
+ ]
360
+ },
361
+ "blocks.14.ffn.net.0.proj.action.up.weight": {
362
+ "dtype": "F32",
363
+ "shape": [
364
+ 14336,
365
+ 64
366
+ ]
367
+ },
368
+ "blocks.14.ffn.net.0.proj.video.down.weight": {
369
+ "dtype": "F32",
370
+ "shape": [
371
+ 64,
372
+ 3072
373
+ ]
374
+ },
375
+ "blocks.14.ffn.net.0.proj.video.up.weight": {
376
+ "dtype": "F32",
377
+ "shape": [
378
+ 14336,
379
+ 64
380
+ ]
381
+ },
382
+ "blocks.14.ffn.net.2.action.down.weight": {
383
+ "dtype": "F32",
384
+ "shape": [
385
+ 64,
386
+ 14336
387
+ ]
388
+ },
389
+ "blocks.14.ffn.net.2.action.up.weight": {
390
+ "dtype": "F32",
391
+ "shape": [
392
+ 3072,
393
+ 64
394
+ ]
395
+ },
396
+ "blocks.14.ffn.net.2.video.down.weight": {
397
+ "dtype": "F32",
398
+ "shape": [
399
+ 64,
400
+ 14336
401
+ ]
402
+ },
403
+ "blocks.14.ffn.net.2.video.up.weight": {
404
+ "dtype": "F32",
405
+ "shape": [
406
+ 3072,
407
+ 64
408
+ ]
409
+ },
410
+ "blocks.15.ffn.net.0.proj.action.down.weight": {
411
+ "dtype": "F32",
412
+ "shape": [
413
+ 64,
414
+ 3072
415
+ ]
416
+ },
417
+ "blocks.15.ffn.net.0.proj.action.up.weight": {
418
+ "dtype": "F32",
419
+ "shape": [
420
+ 14336,
421
+ 64
422
+ ]
423
+ },
424
+ "blocks.15.ffn.net.0.proj.video.down.weight": {
425
+ "dtype": "F32",
426
+ "shape": [
427
+ 64,
428
+ 3072
429
+ ]
430
+ },
431
+ "blocks.15.ffn.net.0.proj.video.up.weight": {
432
+ "dtype": "F32",
433
+ "shape": [
434
+ 14336,
435
+ 64
436
+ ]
437
+ },
438
+ "blocks.15.ffn.net.2.action.down.weight": {
439
+ "dtype": "F32",
440
+ "shape": [
441
+ 64,
442
+ 14336
443
+ ]
444
+ },
445
+ "blocks.15.ffn.net.2.action.up.weight": {
446
+ "dtype": "F32",
447
+ "shape": [
448
+ 3072,
449
+ 64
450
+ ]
451
+ },
452
+ "blocks.15.ffn.net.2.video.down.weight": {
453
+ "dtype": "F32",
454
+ "shape": [
455
+ 64,
456
+ 14336
457
+ ]
458
+ },
459
+ "blocks.15.ffn.net.2.video.up.weight": {
460
+ "dtype": "F32",
461
+ "shape": [
462
+ 3072,
463
+ 64
464
+ ]
465
+ },
466
+ "blocks.16.ffn.net.0.proj.action.down.weight": {
467
+ "dtype": "F32",
468
+ "shape": [
469
+ 64,
470
+ 3072
471
+ ]
472
+ },
473
+ "blocks.16.ffn.net.0.proj.action.up.weight": {
474
+ "dtype": "F32",
475
+ "shape": [
476
+ 14336,
477
+ 64
478
+ ]
479
+ },
480
+ "blocks.16.ffn.net.0.proj.video.down.weight": {
481
+ "dtype": "F32",
482
+ "shape": [
483
+ 64,
484
+ 3072
485
+ ]
486
+ },
487
+ "blocks.16.ffn.net.0.proj.video.up.weight": {
488
+ "dtype": "F32",
489
+ "shape": [
490
+ 14336,
491
+ 64
492
+ ]
493
+ },
494
+ "blocks.16.ffn.net.2.action.down.weight": {
495
+ "dtype": "F32",
496
+ "shape": [
497
+ 64,
498
+ 14336
499
+ ]
500
+ },
501
+ "blocks.16.ffn.net.2.action.up.weight": {
502
+ "dtype": "F32",
503
+ "shape": [
504
+ 3072,
505
+ 64
506
+ ]
507
+ },
508
+ "blocks.16.ffn.net.2.video.down.weight": {
509
+ "dtype": "F32",
510
+ "shape": [
511
+ 64,
512
+ 14336
513
+ ]
514
+ },
515
+ "blocks.16.ffn.net.2.video.up.weight": {
516
+ "dtype": "F32",
517
+ "shape": [
518
+ 3072,
519
+ 64
520
+ ]
521
+ },
522
+ "blocks.17.ffn.net.0.proj.action.down.weight": {
523
+ "dtype": "F32",
524
+ "shape": [
525
+ 64,
526
+ 3072
527
+ ]
528
+ },
529
+ "blocks.17.ffn.net.0.proj.action.up.weight": {
530
+ "dtype": "F32",
531
+ "shape": [
532
+ 14336,
533
+ 64
534
+ ]
535
+ },
536
+ "blocks.17.ffn.net.0.proj.video.down.weight": {
537
+ "dtype": "F32",
538
+ "shape": [
539
+ 64,
540
+ 3072
541
+ ]
542
+ },
543
+ "blocks.17.ffn.net.0.proj.video.up.weight": {
544
+ "dtype": "F32",
545
+ "shape": [
546
+ 14336,
547
+ 64
548
+ ]
549
+ },
550
+ "blocks.17.ffn.net.2.action.down.weight": {
551
+ "dtype": "F32",
552
+ "shape": [
553
+ 64,
554
+ 14336
555
+ ]
556
+ },
557
+ "blocks.17.ffn.net.2.action.up.weight": {
558
+ "dtype": "F32",
559
+ "shape": [
560
+ 3072,
561
+ 64
562
+ ]
563
+ },
564
+ "blocks.17.ffn.net.2.video.down.weight": {
565
+ "dtype": "F32",
566
+ "shape": [
567
+ 64,
568
+ 14336
569
+ ]
570
+ },
571
+ "blocks.17.ffn.net.2.video.up.weight": {
572
+ "dtype": "F32",
573
+ "shape": [
574
+ 3072,
575
+ 64
576
+ ]
577
+ },
578
+ "blocks.18.ffn.net.0.proj.action.down.weight": {
579
+ "dtype": "F32",
580
+ "shape": [
581
+ 64,
582
+ 3072
583
+ ]
584
+ },
585
+ "blocks.18.ffn.net.0.proj.action.up.weight": {
586
+ "dtype": "F32",
587
+ "shape": [
588
+ 14336,
589
+ 64
590
+ ]
591
+ },
592
+ "blocks.18.ffn.net.0.proj.video.down.weight": {
593
+ "dtype": "F32",
594
+ "shape": [
595
+ 64,
596
+ 3072
597
+ ]
598
+ },
599
+ "blocks.18.ffn.net.0.proj.video.up.weight": {
600
+ "dtype": "F32",
601
+ "shape": [
602
+ 14336,
603
+ 64
604
+ ]
605
+ },
606
+ "blocks.18.ffn.net.2.action.down.weight": {
607
+ "dtype": "F32",
608
+ "shape": [
609
+ 64,
610
+ 14336
611
+ ]
612
+ },
613
+ "blocks.18.ffn.net.2.action.up.weight": {
614
+ "dtype": "F32",
615
+ "shape": [
616
+ 3072,
617
+ 64
618
+ ]
619
+ },
620
+ "blocks.18.ffn.net.2.video.down.weight": {
621
+ "dtype": "F32",
622
+ "shape": [
623
+ 64,
624
+ 14336
625
+ ]
626
+ },
627
+ "blocks.18.ffn.net.2.video.up.weight": {
628
+ "dtype": "F32",
629
+ "shape": [
630
+ 3072,
631
+ 64
632
+ ]
633
+ },
634
+ "blocks.19.ffn.net.0.proj.action.down.weight": {
635
+ "dtype": "F32",
636
+ "shape": [
637
+ 64,
638
+ 3072
639
+ ]
640
+ },
641
+ "blocks.19.ffn.net.0.proj.action.up.weight": {
642
+ "dtype": "F32",
643
+ "shape": [
644
+ 14336,
645
+ 64
646
+ ]
647
+ },
648
+ "blocks.19.ffn.net.0.proj.video.down.weight": {
649
+ "dtype": "F32",
650
+ "shape": [
651
+ 64,
652
+ 3072
653
+ ]
654
+ },
655
+ "blocks.19.ffn.net.0.proj.video.up.weight": {
656
+ "dtype": "F32",
657
+ "shape": [
658
+ 14336,
659
+ 64
660
+ ]
661
+ },
662
+ "blocks.19.ffn.net.2.action.down.weight": {
663
+ "dtype": "F32",
664
+ "shape": [
665
+ 64,
666
+ 14336
667
+ ]
668
+ },
669
+ "blocks.19.ffn.net.2.action.up.weight": {
670
+ "dtype": "F32",
671
+ "shape": [
672
+ 3072,
673
+ 64
674
+ ]
675
+ },
676
+ "blocks.19.ffn.net.2.video.down.weight": {
677
+ "dtype": "F32",
678
+ "shape": [
679
+ 64,
680
+ 14336
681
+ ]
682
+ },
683
+ "blocks.19.ffn.net.2.video.up.weight": {
684
+ "dtype": "F32",
685
+ "shape": [
686
+ 3072,
687
+ 64
688
+ ]
689
+ },
690
+ "blocks.2.ffn.net.0.proj.action.down.weight": {
691
+ "dtype": "F32",
692
+ "shape": [
693
+ 64,
694
+ 3072
695
+ ]
696
+ },
697
+ "blocks.2.ffn.net.0.proj.action.up.weight": {
698
+ "dtype": "F32",
699
+ "shape": [
700
+ 14336,
701
+ 64
702
+ ]
703
+ },
704
+ "blocks.2.ffn.net.0.proj.video.down.weight": {
705
+ "dtype": "F32",
706
+ "shape": [
707
+ 64,
708
+ 3072
709
+ ]
710
+ },
711
+ "blocks.2.ffn.net.0.proj.video.up.weight": {
712
+ "dtype": "F32",
713
+ "shape": [
714
+ 14336,
715
+ 64
716
+ ]
717
+ },
718
+ "blocks.2.ffn.net.2.action.down.weight": {
719
+ "dtype": "F32",
720
+ "shape": [
721
+ 64,
722
+ 14336
723
+ ]
724
+ },
725
+ "blocks.2.ffn.net.2.action.up.weight": {
726
+ "dtype": "F32",
727
+ "shape": [
728
+ 3072,
729
+ 64
730
+ ]
731
+ },
732
+ "blocks.2.ffn.net.2.video.down.weight": {
733
+ "dtype": "F32",
734
+ "shape": [
735
+ 64,
736
+ 14336
737
+ ]
738
+ },
739
+ "blocks.2.ffn.net.2.video.up.weight": {
740
+ "dtype": "F32",
741
+ "shape": [
742
+ 3072,
743
+ 64
744
+ ]
745
+ },
746
+ "blocks.20.ffn.net.0.proj.action.down.weight": {
747
+ "dtype": "F32",
748
+ "shape": [
749
+ 64,
750
+ 3072
751
+ ]
752
+ },
753
+ "blocks.20.ffn.net.0.proj.action.up.weight": {
754
+ "dtype": "F32",
755
+ "shape": [
756
+ 14336,
757
+ 64
758
+ ]
759
+ },
760
+ "blocks.20.ffn.net.0.proj.video.down.weight": {
761
+ "dtype": "F32",
762
+ "shape": [
763
+ 64,
764
+ 3072
765
+ ]
766
+ },
767
+ "blocks.20.ffn.net.0.proj.video.up.weight": {
768
+ "dtype": "F32",
769
+ "shape": [
770
+ 14336,
771
+ 64
772
+ ]
773
+ },
774
+ "blocks.20.ffn.net.2.action.down.weight": {
775
+ "dtype": "F32",
776
+ "shape": [
777
+ 64,
778
+ 14336
779
+ ]
780
+ },
781
+ "blocks.20.ffn.net.2.action.up.weight": {
782
+ "dtype": "F32",
783
+ "shape": [
784
+ 3072,
785
+ 64
786
+ ]
787
+ },
788
+ "blocks.20.ffn.net.2.video.down.weight": {
789
+ "dtype": "F32",
790
+ "shape": [
791
+ 64,
792
+ 14336
793
+ ]
794
+ },
795
+ "blocks.20.ffn.net.2.video.up.weight": {
796
+ "dtype": "F32",
797
+ "shape": [
798
+ 3072,
799
+ 64
800
+ ]
801
+ },
802
+ "blocks.21.ffn.net.0.proj.action.down.weight": {
803
+ "dtype": "F32",
804
+ "shape": [
805
+ 64,
806
+ 3072
807
+ ]
808
+ },
809
+ "blocks.21.ffn.net.0.proj.action.up.weight": {
810
+ "dtype": "F32",
811
+ "shape": [
812
+ 14336,
813
+ 64
814
+ ]
815
+ },
816
+ "blocks.21.ffn.net.0.proj.video.down.weight": {
817
+ "dtype": "F32",
818
+ "shape": [
819
+ 64,
820
+ 3072
821
+ ]
822
+ },
823
+ "blocks.21.ffn.net.0.proj.video.up.weight": {
824
+ "dtype": "F32",
825
+ "shape": [
826
+ 14336,
827
+ 64
828
+ ]
829
+ },
830
+ "blocks.21.ffn.net.2.action.down.weight": {
831
+ "dtype": "F32",
832
+ "shape": [
833
+ 64,
834
+ 14336
835
+ ]
836
+ },
837
+ "blocks.21.ffn.net.2.action.up.weight": {
838
+ "dtype": "F32",
839
+ "shape": [
840
+ 3072,
841
+ 64
842
+ ]
843
+ },
844
+ "blocks.21.ffn.net.2.video.down.weight": {
845
+ "dtype": "F32",
846
+ "shape": [
847
+ 64,
848
+ 14336
849
+ ]
850
+ },
851
+ "blocks.21.ffn.net.2.video.up.weight": {
852
+ "dtype": "F32",
853
+ "shape": [
854
+ 3072,
855
+ 64
856
+ ]
857
+ },
858
+ "blocks.22.ffn.net.0.proj.action.down.weight": {
859
+ "dtype": "F32",
860
+ "shape": [
861
+ 64,
862
+ 3072
863
+ ]
864
+ },
865
+ "blocks.22.ffn.net.0.proj.action.up.weight": {
866
+ "dtype": "F32",
867
+ "shape": [
868
+ 14336,
869
+ 64
870
+ ]
871
+ },
872
+ "blocks.22.ffn.net.0.proj.video.down.weight": {
873
+ "dtype": "F32",
874
+ "shape": [
875
+ 64,
876
+ 3072
877
+ ]
878
+ },
879
+ "blocks.22.ffn.net.0.proj.video.up.weight": {
880
+ "dtype": "F32",
881
+ "shape": [
882
+ 14336,
883
+ 64
884
+ ]
885
+ },
886
+ "blocks.22.ffn.net.2.action.down.weight": {
887
+ "dtype": "F32",
888
+ "shape": [
889
+ 64,
890
+ 14336
891
+ ]
892
+ },
893
+ "blocks.22.ffn.net.2.action.up.weight": {
894
+ "dtype": "F32",
895
+ "shape": [
896
+ 3072,
897
+ 64
898
+ ]
899
+ },
900
+ "blocks.22.ffn.net.2.video.down.weight": {
901
+ "dtype": "F32",
902
+ "shape": [
903
+ 64,
904
+ 14336
905
+ ]
906
+ },
907
+ "blocks.22.ffn.net.2.video.up.weight": {
908
+ "dtype": "F32",
909
+ "shape": [
910
+ 3072,
911
+ 64
912
+ ]
913
+ },
914
+ "blocks.23.ffn.net.0.proj.action.down.weight": {
915
+ "dtype": "F32",
916
+ "shape": [
917
+ 64,
918
+ 3072
919
+ ]
920
+ },
921
+ "blocks.23.ffn.net.0.proj.action.up.weight": {
922
+ "dtype": "F32",
923
+ "shape": [
924
+ 14336,
925
+ 64
926
+ ]
927
+ },
928
+ "blocks.23.ffn.net.0.proj.video.down.weight": {
929
+ "dtype": "F32",
930
+ "shape": [
931
+ 64,
932
+ 3072
933
+ ]
934
+ },
935
+ "blocks.23.ffn.net.0.proj.video.up.weight": {
936
+ "dtype": "F32",
937
+ "shape": [
938
+ 14336,
939
+ 64
940
+ ]
941
+ },
942
+ "blocks.23.ffn.net.2.action.down.weight": {
943
+ "dtype": "F32",
944
+ "shape": [
945
+ 64,
946
+ 14336
947
+ ]
948
+ },
949
+ "blocks.23.ffn.net.2.action.up.weight": {
950
+ "dtype": "F32",
951
+ "shape": [
952
+ 3072,
953
+ 64
954
+ ]
955
+ },
956
+ "blocks.23.ffn.net.2.video.down.weight": {
957
+ "dtype": "F32",
958
+ "shape": [
959
+ 64,
960
+ 14336
961
+ ]
962
+ },
963
+ "blocks.23.ffn.net.2.video.up.weight": {
964
+ "dtype": "F32",
965
+ "shape": [
966
+ 3072,
967
+ 64
968
+ ]
969
+ },
970
+ "blocks.24.ffn.net.0.proj.action.down.weight": {
971
+ "dtype": "F32",
972
+ "shape": [
973
+ 64,
974
+ 3072
975
+ ]
976
+ },
977
+ "blocks.24.ffn.net.0.proj.action.up.weight": {
978
+ "dtype": "F32",
979
+ "shape": [
980
+ 14336,
981
+ 64
982
+ ]
983
+ },
984
+ "blocks.24.ffn.net.0.proj.video.down.weight": {
985
+ "dtype": "F32",
986
+ "shape": [
987
+ 64,
988
+ 3072
989
+ ]
990
+ },
991
+ "blocks.24.ffn.net.0.proj.video.up.weight": {
992
+ "dtype": "F32",
993
+ "shape": [
994
+ 14336,
995
+ 64
996
+ ]
997
+ },
998
+ "blocks.24.ffn.net.2.action.down.weight": {
999
+ "dtype": "F32",
1000
+ "shape": [
1001
+ 64,
1002
+ 14336
1003
+ ]
1004
+ },
1005
+ "blocks.24.ffn.net.2.action.up.weight": {
1006
+ "dtype": "F32",
1007
+ "shape": [
1008
+ 3072,
1009
+ 64
1010
+ ]
1011
+ },
1012
+ "blocks.24.ffn.net.2.video.down.weight": {
1013
+ "dtype": "F32",
1014
+ "shape": [
1015
+ 64,
1016
+ 14336
1017
+ ]
1018
+ },
1019
+ "blocks.24.ffn.net.2.video.up.weight": {
1020
+ "dtype": "F32",
1021
+ "shape": [
1022
+ 3072,
1023
+ 64
1024
+ ]
1025
+ },
1026
+ "blocks.25.ffn.net.0.proj.action.down.weight": {
1027
+ "dtype": "F32",
1028
+ "shape": [
1029
+ 64,
1030
+ 3072
1031
+ ]
1032
+ },
1033
+ "blocks.25.ffn.net.0.proj.action.up.weight": {
1034
+ "dtype": "F32",
1035
+ "shape": [
1036
+ 14336,
1037
+ 64
1038
+ ]
1039
+ },
1040
+ "blocks.25.ffn.net.0.proj.video.down.weight": {
1041
+ "dtype": "F32",
1042
+ "shape": [
1043
+ 64,
1044
+ 3072
1045
+ ]
1046
+ },
1047
+ "blocks.25.ffn.net.0.proj.video.up.weight": {
1048
+ "dtype": "F32",
1049
+ "shape": [
1050
+ 14336,
1051
+ 64
1052
+ ]
1053
+ },
1054
+ "blocks.25.ffn.net.2.action.down.weight": {
1055
+ "dtype": "F32",
1056
+ "shape": [
1057
+ 64,
1058
+ 14336
1059
+ ]
1060
+ },
1061
+ "blocks.25.ffn.net.2.action.up.weight": {
1062
+ "dtype": "F32",
1063
+ "shape": [
1064
+ 3072,
1065
+ 64
1066
+ ]
1067
+ },
1068
+ "blocks.25.ffn.net.2.video.down.weight": {
1069
+ "dtype": "F32",
1070
+ "shape": [
1071
+ 64,
1072
+ 14336
1073
+ ]
1074
+ },
1075
+ "blocks.25.ffn.net.2.video.up.weight": {
1076
+ "dtype": "F32",
1077
+ "shape": [
1078
+ 3072,
1079
+ 64
1080
+ ]
1081
+ },
1082
+ "blocks.26.ffn.net.0.proj.action.down.weight": {
1083
+ "dtype": "F32",
1084
+ "shape": [
1085
+ 64,
1086
+ 3072
1087
+ ]
1088
+ },
1089
+ "blocks.26.ffn.net.0.proj.action.up.weight": {
1090
+ "dtype": "F32",
1091
+ "shape": [
1092
+ 14336,
1093
+ 64
1094
+ ]
1095
+ },
1096
+ "blocks.26.ffn.net.0.proj.video.down.weight": {
1097
+ "dtype": "F32",
1098
+ "shape": [
1099
+ 64,
1100
+ 3072
1101
+ ]
1102
+ },
1103
+ "blocks.26.ffn.net.0.proj.video.up.weight": {
1104
+ "dtype": "F32",
1105
+ "shape": [
1106
+ 14336,
1107
+ 64
1108
+ ]
1109
+ },
1110
+ "blocks.26.ffn.net.2.action.down.weight": {
1111
+ "dtype": "F32",
1112
+ "shape": [
1113
+ 64,
1114
+ 14336
1115
+ ]
1116
+ },
1117
+ "blocks.26.ffn.net.2.action.up.weight": {
1118
+ "dtype": "F32",
1119
+ "shape": [
1120
+ 3072,
1121
+ 64
1122
+ ]
1123
+ },
1124
+ "blocks.26.ffn.net.2.video.down.weight": {
1125
+ "dtype": "F32",
1126
+ "shape": [
1127
+ 64,
1128
+ 14336
1129
+ ]
1130
+ },
1131
+ "blocks.26.ffn.net.2.video.up.weight": {
1132
+ "dtype": "F32",
1133
+ "shape": [
1134
+ 3072,
1135
+ 64
1136
+ ]
1137
+ },
1138
+ "blocks.27.ffn.net.0.proj.action.down.weight": {
1139
+ "dtype": "F32",
1140
+ "shape": [
1141
+ 64,
1142
+ 3072
1143
+ ]
1144
+ },
1145
+ "blocks.27.ffn.net.0.proj.action.up.weight": {
1146
+ "dtype": "F32",
1147
+ "shape": [
1148
+ 14336,
1149
+ 64
1150
+ ]
1151
+ },
1152
+ "blocks.27.ffn.net.0.proj.video.down.weight": {
1153
+ "dtype": "F32",
1154
+ "shape": [
1155
+ 64,
1156
+ 3072
1157
+ ]
1158
+ },
1159
+ "blocks.27.ffn.net.0.proj.video.up.weight": {
1160
+ "dtype": "F32",
1161
+ "shape": [
1162
+ 14336,
1163
+ 64
1164
+ ]
1165
+ },
1166
+ "blocks.27.ffn.net.2.action.down.weight": {
1167
+ "dtype": "F32",
1168
+ "shape": [
1169
+ 64,
1170
+ 14336
1171
+ ]
1172
+ },
1173
+ "blocks.27.ffn.net.2.action.up.weight": {
1174
+ "dtype": "F32",
1175
+ "shape": [
1176
+ 3072,
1177
+ 64
1178
+ ]
1179
+ },
1180
+ "blocks.27.ffn.net.2.video.down.weight": {
1181
+ "dtype": "F32",
1182
+ "shape": [
1183
+ 64,
1184
+ 14336
1185
+ ]
1186
+ },
1187
+ "blocks.27.ffn.net.2.video.up.weight": {
1188
+ "dtype": "F32",
1189
+ "shape": [
1190
+ 3072,
1191
+ 64
1192
+ ]
1193
+ },
1194
+ "blocks.28.ffn.net.0.proj.action.down.weight": {
1195
+ "dtype": "F32",
1196
+ "shape": [
1197
+ 64,
1198
+ 3072
1199
+ ]
1200
+ },
1201
+ "blocks.28.ffn.net.0.proj.action.up.weight": {
1202
+ "dtype": "F32",
1203
+ "shape": [
1204
+ 14336,
1205
+ 64
1206
+ ]
1207
+ },
1208
+ "blocks.28.ffn.net.0.proj.video.down.weight": {
1209
+ "dtype": "F32",
1210
+ "shape": [
1211
+ 64,
1212
+ 3072
1213
+ ]
1214
+ },
1215
+ "blocks.28.ffn.net.0.proj.video.up.weight": {
1216
+ "dtype": "F32",
1217
+ "shape": [
1218
+ 14336,
1219
+ 64
1220
+ ]
1221
+ },
1222
+ "blocks.28.ffn.net.2.action.down.weight": {
1223
+ "dtype": "F32",
1224
+ "shape": [
1225
+ 64,
1226
+ 14336
1227
+ ]
1228
+ },
1229
+ "blocks.28.ffn.net.2.action.up.weight": {
1230
+ "dtype": "F32",
1231
+ "shape": [
1232
+ 3072,
1233
+ 64
1234
+ ]
1235
+ },
1236
+ "blocks.28.ffn.net.2.video.down.weight": {
1237
+ "dtype": "F32",
1238
+ "shape": [
1239
+ 64,
1240
+ 14336
1241
+ ]
1242
+ },
1243
+ "blocks.28.ffn.net.2.video.up.weight": {
1244
+ "dtype": "F32",
1245
+ "shape": [
1246
+ 3072,
1247
+ 64
1248
+ ]
1249
+ },
1250
+ "blocks.29.ffn.net.0.proj.action.down.weight": {
1251
+ "dtype": "F32",
1252
+ "shape": [
1253
+ 64,
1254
+ 3072
1255
+ ]
1256
+ },
1257
+ "blocks.29.ffn.net.0.proj.action.up.weight": {
1258
+ "dtype": "F32",
1259
+ "shape": [
1260
+ 14336,
1261
+ 64
1262
+ ]
1263
+ },
1264
+ "blocks.29.ffn.net.0.proj.video.down.weight": {
1265
+ "dtype": "F32",
1266
+ "shape": [
1267
+ 64,
1268
+ 3072
1269
+ ]
1270
+ },
1271
+ "blocks.29.ffn.net.0.proj.video.up.weight": {
1272
+ "dtype": "F32",
1273
+ "shape": [
1274
+ 14336,
1275
+ 64
1276
+ ]
1277
+ },
1278
+ "blocks.29.ffn.net.2.action.down.weight": {
1279
+ "dtype": "F32",
1280
+ "shape": [
1281
+ 64,
1282
+ 14336
1283
+ ]
1284
+ },
1285
+ "blocks.29.ffn.net.2.action.up.weight": {
1286
+ "dtype": "F32",
1287
+ "shape": [
1288
+ 3072,
1289
+ 64
1290
+ ]
1291
+ },
1292
+ "blocks.29.ffn.net.2.video.down.weight": {
1293
+ "dtype": "F32",
1294
+ "shape": [
1295
+ 64,
1296
+ 14336
1297
+ ]
1298
+ },
1299
+ "blocks.29.ffn.net.2.video.up.weight": {
1300
+ "dtype": "F32",
1301
+ "shape": [
1302
+ 3072,
1303
+ 64
1304
+ ]
1305
+ },
1306
+ "blocks.3.ffn.net.0.proj.action.down.weight": {
1307
+ "dtype": "F32",
1308
+ "shape": [
1309
+ 64,
1310
+ 3072
1311
+ ]
1312
+ },
1313
+ "blocks.3.ffn.net.0.proj.action.up.weight": {
1314
+ "dtype": "F32",
1315
+ "shape": [
1316
+ 14336,
1317
+ 64
1318
+ ]
1319
+ },
1320
+ "blocks.3.ffn.net.0.proj.video.down.weight": {
1321
+ "dtype": "F32",
1322
+ "shape": [
1323
+ 64,
1324
+ 3072
1325
+ ]
1326
+ },
1327
+ "blocks.3.ffn.net.0.proj.video.up.weight": {
1328
+ "dtype": "F32",
1329
+ "shape": [
1330
+ 14336,
1331
+ 64
1332
+ ]
1333
+ },
1334
+ "blocks.3.ffn.net.2.action.down.weight": {
1335
+ "dtype": "F32",
1336
+ "shape": [
1337
+ 64,
1338
+ 14336
1339
+ ]
1340
+ },
1341
+ "blocks.3.ffn.net.2.action.up.weight": {
1342
+ "dtype": "F32",
1343
+ "shape": [
1344
+ 3072,
1345
+ 64
1346
+ ]
1347
+ },
1348
+ "blocks.3.ffn.net.2.video.down.weight": {
1349
+ "dtype": "F32",
1350
+ "shape": [
1351
+ 64,
1352
+ 14336
1353
+ ]
1354
+ },
1355
+ "blocks.3.ffn.net.2.video.up.weight": {
1356
+ "dtype": "F32",
1357
+ "shape": [
1358
+ 3072,
1359
+ 64
1360
+ ]
1361
+ },
1362
+ "blocks.4.ffn.net.0.proj.action.down.weight": {
1363
+ "dtype": "F32",
1364
+ "shape": [
1365
+ 64,
1366
+ 3072
1367
+ ]
1368
+ },
1369
+ "blocks.4.ffn.net.0.proj.action.up.weight": {
1370
+ "dtype": "F32",
1371
+ "shape": [
1372
+ 14336,
1373
+ 64
1374
+ ]
1375
+ },
1376
+ "blocks.4.ffn.net.0.proj.video.down.weight": {
1377
+ "dtype": "F32",
1378
+ "shape": [
1379
+ 64,
1380
+ 3072
1381
+ ]
1382
+ },
1383
+ "blocks.4.ffn.net.0.proj.video.up.weight": {
1384
+ "dtype": "F32",
1385
+ "shape": [
1386
+ 14336,
1387
+ 64
1388
+ ]
1389
+ },
1390
+ "blocks.4.ffn.net.2.action.down.weight": {
1391
+ "dtype": "F32",
1392
+ "shape": [
1393
+ 64,
1394
+ 14336
1395
+ ]
1396
+ },
1397
+ "blocks.4.ffn.net.2.action.up.weight": {
1398
+ "dtype": "F32",
1399
+ "shape": [
1400
+ 3072,
1401
+ 64
1402
+ ]
1403
+ },
1404
+ "blocks.4.ffn.net.2.video.down.weight": {
1405
+ "dtype": "F32",
1406
+ "shape": [
1407
+ 64,
1408
+ 14336
1409
+ ]
1410
+ },
1411
+ "blocks.4.ffn.net.2.video.up.weight": {
1412
+ "dtype": "F32",
1413
+ "shape": [
1414
+ 3072,
1415
+ 64
1416
+ ]
1417
+ },
1418
+ "blocks.5.ffn.net.0.proj.action.down.weight": {
1419
+ "dtype": "F32",
1420
+ "shape": [
1421
+ 64,
1422
+ 3072
1423
+ ]
1424
+ },
1425
+ "blocks.5.ffn.net.0.proj.action.up.weight": {
1426
+ "dtype": "F32",
1427
+ "shape": [
1428
+ 14336,
1429
+ 64
1430
+ ]
1431
+ },
1432
+ "blocks.5.ffn.net.0.proj.video.down.weight": {
1433
+ "dtype": "F32",
1434
+ "shape": [
1435
+ 64,
1436
+ 3072
1437
+ ]
1438
+ },
1439
+ "blocks.5.ffn.net.0.proj.video.up.weight": {
1440
+ "dtype": "F32",
1441
+ "shape": [
1442
+ 14336,
1443
+ 64
1444
+ ]
1445
+ },
1446
+ "blocks.5.ffn.net.2.action.down.weight": {
1447
+ "dtype": "F32",
1448
+ "shape": [
1449
+ 64,
1450
+ 14336
1451
+ ]
1452
+ },
1453
+ "blocks.5.ffn.net.2.action.up.weight": {
1454
+ "dtype": "F32",
1455
+ "shape": [
1456
+ 3072,
1457
+ 64
1458
+ ]
1459
+ },
1460
+ "blocks.5.ffn.net.2.video.down.weight": {
1461
+ "dtype": "F32",
1462
+ "shape": [
1463
+ 64,
1464
+ 14336
1465
+ ]
1466
+ },
1467
+ "blocks.5.ffn.net.2.video.up.weight": {
1468
+ "dtype": "F32",
1469
+ "shape": [
1470
+ 3072,
1471
+ 64
1472
+ ]
1473
+ },
1474
+ "blocks.6.ffn.net.0.proj.action.down.weight": {
1475
+ "dtype": "F32",
1476
+ "shape": [
1477
+ 64,
1478
+ 3072
1479
+ ]
1480
+ },
1481
+ "blocks.6.ffn.net.0.proj.action.up.weight": {
1482
+ "dtype": "F32",
1483
+ "shape": [
1484
+ 14336,
1485
+ 64
1486
+ ]
1487
+ },
1488
+ "blocks.6.ffn.net.0.proj.video.down.weight": {
1489
+ "dtype": "F32",
1490
+ "shape": [
1491
+ 64,
1492
+ 3072
1493
+ ]
1494
+ },
1495
+ "blocks.6.ffn.net.0.proj.video.up.weight": {
1496
+ "dtype": "F32",
1497
+ "shape": [
1498
+ 14336,
1499
+ 64
1500
+ ]
1501
+ },
1502
+ "blocks.6.ffn.net.2.action.down.weight": {
1503
+ "dtype": "F32",
1504
+ "shape": [
1505
+ 64,
1506
+ 14336
1507
+ ]
1508
+ },
1509
+ "blocks.6.ffn.net.2.action.up.weight": {
1510
+ "dtype": "F32",
1511
+ "shape": [
1512
+ 3072,
1513
+ 64
1514
+ ]
1515
+ },
1516
+ "blocks.6.ffn.net.2.video.down.weight": {
1517
+ "dtype": "F32",
1518
+ "shape": [
1519
+ 64,
1520
+ 14336
1521
+ ]
1522
+ },
1523
+ "blocks.6.ffn.net.2.video.up.weight": {
1524
+ "dtype": "F32",
1525
+ "shape": [
1526
+ 3072,
1527
+ 64
1528
+ ]
1529
+ },
1530
+ "blocks.7.ffn.net.0.proj.action.down.weight": {
1531
+ "dtype": "F32",
1532
+ "shape": [
1533
+ 64,
1534
+ 3072
1535
+ ]
1536
+ },
1537
+ "blocks.7.ffn.net.0.proj.action.up.weight": {
1538
+ "dtype": "F32",
1539
+ "shape": [
1540
+ 14336,
1541
+ 64
1542
+ ]
1543
+ },
1544
+ "blocks.7.ffn.net.0.proj.video.down.weight": {
1545
+ "dtype": "F32",
1546
+ "shape": [
1547
+ 64,
1548
+ 3072
1549
+ ]
1550
+ },
1551
+ "blocks.7.ffn.net.0.proj.video.up.weight": {
1552
+ "dtype": "F32",
1553
+ "shape": [
1554
+ 14336,
1555
+ 64
1556
+ ]
1557
+ },
1558
+ "blocks.7.ffn.net.2.action.down.weight": {
1559
+ "dtype": "F32",
1560
+ "shape": [
1561
+ 64,
1562
+ 14336
1563
+ ]
1564
+ },
1565
+ "blocks.7.ffn.net.2.action.up.weight": {
1566
+ "dtype": "F32",
1567
+ "shape": [
1568
+ 3072,
1569
+ 64
1570
+ ]
1571
+ },
1572
+ "blocks.7.ffn.net.2.video.down.weight": {
1573
+ "dtype": "F32",
1574
+ "shape": [
1575
+ 64,
1576
+ 14336
1577
+ ]
1578
+ },
1579
+ "blocks.7.ffn.net.2.video.up.weight": {
1580
+ "dtype": "F32",
1581
+ "shape": [
1582
+ 3072,
1583
+ 64
1584
+ ]
1585
+ },
1586
+ "blocks.8.ffn.net.0.proj.action.down.weight": {
1587
+ "dtype": "F32",
1588
+ "shape": [
1589
+ 64,
1590
+ 3072
1591
+ ]
1592
+ },
1593
+ "blocks.8.ffn.net.0.proj.action.up.weight": {
1594
+ "dtype": "F32",
1595
+ "shape": [
1596
+ 14336,
1597
+ 64
1598
+ ]
1599
+ },
1600
+ "blocks.8.ffn.net.0.proj.video.down.weight": {
1601
+ "dtype": "F32",
1602
+ "shape": [
1603
+ 64,
1604
+ 3072
1605
+ ]
1606
+ },
1607
+ "blocks.8.ffn.net.0.proj.video.up.weight": {
1608
+ "dtype": "F32",
1609
+ "shape": [
1610
+ 14336,
1611
+ 64
1612
+ ]
1613
+ },
1614
+ "blocks.8.ffn.net.2.action.down.weight": {
1615
+ "dtype": "F32",
1616
+ "shape": [
1617
+ 64,
1618
+ 14336
1619
+ ]
1620
+ },
1621
+ "blocks.8.ffn.net.2.action.up.weight": {
1622
+ "dtype": "F32",
1623
+ "shape": [
1624
+ 3072,
1625
+ 64
1626
+ ]
1627
+ },
1628
+ "blocks.8.ffn.net.2.video.down.weight": {
1629
+ "dtype": "F32",
1630
+ "shape": [
1631
+ 64,
1632
+ 14336
1633
+ ]
1634
+ },
1635
+ "blocks.8.ffn.net.2.video.up.weight": {
1636
+ "dtype": "F32",
1637
+ "shape": [
1638
+ 3072,
1639
+ 64
1640
+ ]
1641
+ },
1642
+ "blocks.9.ffn.net.0.proj.action.down.weight": {
1643
+ "dtype": "F32",
1644
+ "shape": [
1645
+ 64,
1646
+ 3072
1647
+ ]
1648
+ },
1649
+ "blocks.9.ffn.net.0.proj.action.up.weight": {
1650
+ "dtype": "F32",
1651
+ "shape": [
1652
+ 14336,
1653
+ 64
1654
+ ]
1655
+ },
1656
+ "blocks.9.ffn.net.0.proj.video.down.weight": {
1657
+ "dtype": "F32",
1658
+ "shape": [
1659
+ 64,
1660
+ 3072
1661
+ ]
1662
+ },
1663
+ "blocks.9.ffn.net.0.proj.video.up.weight": {
1664
+ "dtype": "F32",
1665
+ "shape": [
1666
+ 14336,
1667
+ 64
1668
+ ]
1669
+ },
1670
+ "blocks.9.ffn.net.2.action.down.weight": {
1671
+ "dtype": "F32",
1672
+ "shape": [
1673
+ 64,
1674
+ 14336
1675
+ ]
1676
+ },
1677
+ "blocks.9.ffn.net.2.action.up.weight": {
1678
+ "dtype": "F32",
1679
+ "shape": [
1680
+ 3072,
1681
+ 64
1682
+ ]
1683
+ },
1684
+ "blocks.9.ffn.net.2.video.down.weight": {
1685
+ "dtype": "F32",
1686
+ "shape": [
1687
+ 64,
1688
+ 14336
1689
+ ]
1690
+ },
1691
+ "blocks.9.ffn.net.2.video.up.weight": {
1692
+ "dtype": "F32",
1693
+ "shape": [
1694
+ 3072,
1695
+ 64
1696
+ ]
1697
+ },
1698
+ "condition_embedder.delta_embedder.linear_1.shared.down.weight": {
1699
+ "dtype": "F32",
1700
+ "shape": [
1701
+ 128,
1702
+ 256
1703
+ ]
1704
+ },
1705
+ "condition_embedder.delta_embedder.linear_1.shared.up.weight": {
1706
+ "dtype": "F32",
1707
+ "shape": [
1708
+ 3072,
1709
+ 128
1710
+ ]
1711
+ },
1712
+ "condition_embedder.delta_embedder.linear_2.shared.down.weight": {
1713
+ "dtype": "F32",
1714
+ "shape": [
1715
+ 128,
1716
+ 3072
1717
+ ]
1718
+ },
1719
+ "condition_embedder.delta_embedder.linear_2.shared.up.weight": {
1720
+ "dtype": "F32",
1721
+ "shape": [
1722
+ 3072,
1723
+ 128
1724
+ ]
1725
+ },
1726
+ "condition_embedder.delta_output.shared.down.weight": {
1727
+ "dtype": "F32",
1728
+ "shape": [
1729
+ 128,
1730
+ 3072
1731
+ ]
1732
+ },
1733
+ "condition_embedder.delta_output.shared.up.weight": {
1734
+ "dtype": "F32",
1735
+ "shape": [
1736
+ 3072,
1737
+ 128
1738
+ ]
1739
+ },
1740
+ "condition_embedder.time_embedder.linear_1.shared.down.weight": {
1741
+ "dtype": "F32",
1742
+ "shape": [
1743
+ 128,
1744
+ 256
1745
+ ]
1746
+ },
1747
+ "condition_embedder.time_embedder.linear_1.shared.up.weight": {
1748
+ "dtype": "F32",
1749
+ "shape": [
1750
+ 3072,
1751
+ 128
1752
+ ]
1753
+ },
1754
+ "condition_embedder.time_embedder.linear_2.shared.down.weight": {
1755
+ "dtype": "F32",
1756
+ "shape": [
1757
+ 128,
1758
+ 3072
1759
+ ]
1760
+ },
1761
+ "condition_embedder.time_embedder.linear_2.shared.up.weight": {
1762
+ "dtype": "F32",
1763
+ "shape": [
1764
+ 3072,
1765
+ 128
1766
+ ]
1767
+ },
1768
+ "condition_embedder.time_proj.shared.down.weight": {
1769
+ "dtype": "F32",
1770
+ "shape": [
1771
+ 128,
1772
+ 3072
1773
+ ]
1774
+ },
1775
+ "condition_embedder.time_proj.shared.up.weight": {
1776
+ "dtype": "F32",
1777
+ "shape": [
1778
+ 18432,
1779
+ 128
1780
+ ]
1781
+ },
1782
+ "condition_embedder_action.delta_embedder.linear_1.shared.down.weight": {
1783
+ "dtype": "F32",
1784
+ "shape": [
1785
+ 128,
1786
+ 256
1787
+ ]
1788
+ },
1789
+ "condition_embedder_action.delta_embedder.linear_1.shared.up.weight": {
1790
+ "dtype": "F32",
1791
+ "shape": [
1792
+ 3072,
1793
+ 128
1794
+ ]
1795
+ },
1796
+ "condition_embedder_action.delta_embedder.linear_2.shared.down.weight": {
1797
+ "dtype": "F32",
1798
+ "shape": [
1799
+ 128,
1800
+ 3072
1801
+ ]
1802
+ },
1803
+ "condition_embedder_action.delta_embedder.linear_2.shared.up.weight": {
1804
+ "dtype": "F32",
1805
+ "shape": [
1806
+ 3072,
1807
+ 128
1808
+ ]
1809
+ },
1810
+ "condition_embedder_action.delta_output.shared.down.weight": {
1811
+ "dtype": "F32",
1812
+ "shape": [
1813
+ 128,
1814
+ 3072
1815
+ ]
1816
+ },
1817
+ "condition_embedder_action.delta_output.shared.up.weight": {
1818
+ "dtype": "F32",
1819
+ "shape": [
1820
+ 3072,
1821
+ 128
1822
+ ]
1823
+ },
1824
+ "condition_embedder_action.time_embedder.linear_1.shared.down.weight": {
1825
+ "dtype": "F32",
1826
+ "shape": [
1827
+ 128,
1828
+ 256
1829
+ ]
1830
+ },
1831
+ "condition_embedder_action.time_embedder.linear_1.shared.up.weight": {
1832
+ "dtype": "F32",
1833
+ "shape": [
1834
+ 3072,
1835
+ 128
1836
+ ]
1837
+ },
1838
+ "condition_embedder_action.time_embedder.linear_2.shared.down.weight": {
1839
+ "dtype": "F32",
1840
+ "shape": [
1841
+ 128,
1842
+ 3072
1843
+ ]
1844
+ },
1845
+ "condition_embedder_action.time_embedder.linear_2.shared.up.weight": {
1846
+ "dtype": "F32",
1847
+ "shape": [
1848
+ 3072,
1849
+ 128
1850
+ ]
1851
+ },
1852
+ "condition_embedder_action.time_proj.shared.down.weight": {
1853
+ "dtype": "F32",
1854
+ "shape": [
1855
+ 128,
1856
+ 3072
1857
+ ]
1858
+ },
1859
+ "condition_embedder_action.time_proj.shared.up.weight": {
1860
+ "dtype": "F32",
1861
+ "shape": [
1862
+ 18432,
1863
+ 128
1864
+ ]
1865
+ }
1866
+ },
1867
+ "kind": "adapter",
1868
+ "license": "Apache-2.0",
1869
+ "paper_role": "original-field-projected",
1870
+ "source_digest": "a3cb591e6deeee93ee2d6732b9c122310eaad2bd3fefc3b57a4b3ae6722e3f94",
1871
+ "topology": "frozen-stage1-routed-ffn-time-v1:r64/r64/t128",
1872
+ "variant_id": "original-field-projected"
1873
+ }
variants/restart-no-field/adapter_model.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:cf4e29d30738641aac0522a241960201f2bcb8f25532de529c9ec73ba4056f56
3
+ size 582514248
variants/restart-no-field/manifest.json ADDED
@@ -0,0 +1,1873 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "artifact_file": "adapter_model.safetensors",
3
+ "artifact_sha256": "cf4e29d30738641aac0522a241960201f2bcb8f25532de529c9ec73ba4056f56",
4
+ "base_digest": "6f9e4b3f291d4f00a718db798a2774fe1e918daab21f74155644536443c19357",
5
+ "cfg_contract": {
6
+ "deployment_cfg": [
7
+ 5.0,
8
+ 1.0
9
+ ],
10
+ "target_guidance_mode": "inference_composed",
11
+ "teacher_cfg": [
12
+ 5.0,
13
+ 1.0
14
+ ],
15
+ "text_dropout_probability": 0.1
16
+ },
17
+ "keys": {
18
+ "blocks.0.ffn.net.0.proj.action.down.weight": {
19
+ "dtype": "F32",
20
+ "shape": [
21
+ 64,
22
+ 3072
23
+ ]
24
+ },
25
+ "blocks.0.ffn.net.0.proj.action.up.weight": {
26
+ "dtype": "F32",
27
+ "shape": [
28
+ 14336,
29
+ 64
30
+ ]
31
+ },
32
+ "blocks.0.ffn.net.0.proj.video.down.weight": {
33
+ "dtype": "F32",
34
+ "shape": [
35
+ 64,
36
+ 3072
37
+ ]
38
+ },
39
+ "blocks.0.ffn.net.0.proj.video.up.weight": {
40
+ "dtype": "F32",
41
+ "shape": [
42
+ 14336,
43
+ 64
44
+ ]
45
+ },
46
+ "blocks.0.ffn.net.2.action.down.weight": {
47
+ "dtype": "F32",
48
+ "shape": [
49
+ 64,
50
+ 14336
51
+ ]
52
+ },
53
+ "blocks.0.ffn.net.2.action.up.weight": {
54
+ "dtype": "F32",
55
+ "shape": [
56
+ 3072,
57
+ 64
58
+ ]
59
+ },
60
+ "blocks.0.ffn.net.2.video.down.weight": {
61
+ "dtype": "F32",
62
+ "shape": [
63
+ 64,
64
+ 14336
65
+ ]
66
+ },
67
+ "blocks.0.ffn.net.2.video.up.weight": {
68
+ "dtype": "F32",
69
+ "shape": [
70
+ 3072,
71
+ 64
72
+ ]
73
+ },
74
+ "blocks.1.ffn.net.0.proj.action.down.weight": {
75
+ "dtype": "F32",
76
+ "shape": [
77
+ 64,
78
+ 3072
79
+ ]
80
+ },
81
+ "blocks.1.ffn.net.0.proj.action.up.weight": {
82
+ "dtype": "F32",
83
+ "shape": [
84
+ 14336,
85
+ 64
86
+ ]
87
+ },
88
+ "blocks.1.ffn.net.0.proj.video.down.weight": {
89
+ "dtype": "F32",
90
+ "shape": [
91
+ 64,
92
+ 3072
93
+ ]
94
+ },
95
+ "blocks.1.ffn.net.0.proj.video.up.weight": {
96
+ "dtype": "F32",
97
+ "shape": [
98
+ 14336,
99
+ 64
100
+ ]
101
+ },
102
+ "blocks.1.ffn.net.2.action.down.weight": {
103
+ "dtype": "F32",
104
+ "shape": [
105
+ 64,
106
+ 14336
107
+ ]
108
+ },
109
+ "blocks.1.ffn.net.2.action.up.weight": {
110
+ "dtype": "F32",
111
+ "shape": [
112
+ 3072,
113
+ 64
114
+ ]
115
+ },
116
+ "blocks.1.ffn.net.2.video.down.weight": {
117
+ "dtype": "F32",
118
+ "shape": [
119
+ 64,
120
+ 14336
121
+ ]
122
+ },
123
+ "blocks.1.ffn.net.2.video.up.weight": {
124
+ "dtype": "F32",
125
+ "shape": [
126
+ 3072,
127
+ 64
128
+ ]
129
+ },
130
+ "blocks.10.ffn.net.0.proj.action.down.weight": {
131
+ "dtype": "F32",
132
+ "shape": [
133
+ 64,
134
+ 3072
135
+ ]
136
+ },
137
+ "blocks.10.ffn.net.0.proj.action.up.weight": {
138
+ "dtype": "F32",
139
+ "shape": [
140
+ 14336,
141
+ 64
142
+ ]
143
+ },
144
+ "blocks.10.ffn.net.0.proj.video.down.weight": {
145
+ "dtype": "F32",
146
+ "shape": [
147
+ 64,
148
+ 3072
149
+ ]
150
+ },
151
+ "blocks.10.ffn.net.0.proj.video.up.weight": {
152
+ "dtype": "F32",
153
+ "shape": [
154
+ 14336,
155
+ 64
156
+ ]
157
+ },
158
+ "blocks.10.ffn.net.2.action.down.weight": {
159
+ "dtype": "F32",
160
+ "shape": [
161
+ 64,
162
+ 14336
163
+ ]
164
+ },
165
+ "blocks.10.ffn.net.2.action.up.weight": {
166
+ "dtype": "F32",
167
+ "shape": [
168
+ 3072,
169
+ 64
170
+ ]
171
+ },
172
+ "blocks.10.ffn.net.2.video.down.weight": {
173
+ "dtype": "F32",
174
+ "shape": [
175
+ 64,
176
+ 14336
177
+ ]
178
+ },
179
+ "blocks.10.ffn.net.2.video.up.weight": {
180
+ "dtype": "F32",
181
+ "shape": [
182
+ 3072,
183
+ 64
184
+ ]
185
+ },
186
+ "blocks.11.ffn.net.0.proj.action.down.weight": {
187
+ "dtype": "F32",
188
+ "shape": [
189
+ 64,
190
+ 3072
191
+ ]
192
+ },
193
+ "blocks.11.ffn.net.0.proj.action.up.weight": {
194
+ "dtype": "F32",
195
+ "shape": [
196
+ 14336,
197
+ 64
198
+ ]
199
+ },
200
+ "blocks.11.ffn.net.0.proj.video.down.weight": {
201
+ "dtype": "F32",
202
+ "shape": [
203
+ 64,
204
+ 3072
205
+ ]
206
+ },
207
+ "blocks.11.ffn.net.0.proj.video.up.weight": {
208
+ "dtype": "F32",
209
+ "shape": [
210
+ 14336,
211
+ 64
212
+ ]
213
+ },
214
+ "blocks.11.ffn.net.2.action.down.weight": {
215
+ "dtype": "F32",
216
+ "shape": [
217
+ 64,
218
+ 14336
219
+ ]
220
+ },
221
+ "blocks.11.ffn.net.2.action.up.weight": {
222
+ "dtype": "F32",
223
+ "shape": [
224
+ 3072,
225
+ 64
226
+ ]
227
+ },
228
+ "blocks.11.ffn.net.2.video.down.weight": {
229
+ "dtype": "F32",
230
+ "shape": [
231
+ 64,
232
+ 14336
233
+ ]
234
+ },
235
+ "blocks.11.ffn.net.2.video.up.weight": {
236
+ "dtype": "F32",
237
+ "shape": [
238
+ 3072,
239
+ 64
240
+ ]
241
+ },
242
+ "blocks.12.ffn.net.0.proj.action.down.weight": {
243
+ "dtype": "F32",
244
+ "shape": [
245
+ 64,
246
+ 3072
247
+ ]
248
+ },
249
+ "blocks.12.ffn.net.0.proj.action.up.weight": {
250
+ "dtype": "F32",
251
+ "shape": [
252
+ 14336,
253
+ 64
254
+ ]
255
+ },
256
+ "blocks.12.ffn.net.0.proj.video.down.weight": {
257
+ "dtype": "F32",
258
+ "shape": [
259
+ 64,
260
+ 3072
261
+ ]
262
+ },
263
+ "blocks.12.ffn.net.0.proj.video.up.weight": {
264
+ "dtype": "F32",
265
+ "shape": [
266
+ 14336,
267
+ 64
268
+ ]
269
+ },
270
+ "blocks.12.ffn.net.2.action.down.weight": {
271
+ "dtype": "F32",
272
+ "shape": [
273
+ 64,
274
+ 14336
275
+ ]
276
+ },
277
+ "blocks.12.ffn.net.2.action.up.weight": {
278
+ "dtype": "F32",
279
+ "shape": [
280
+ 3072,
281
+ 64
282
+ ]
283
+ },
284
+ "blocks.12.ffn.net.2.video.down.weight": {
285
+ "dtype": "F32",
286
+ "shape": [
287
+ 64,
288
+ 14336
289
+ ]
290
+ },
291
+ "blocks.12.ffn.net.2.video.up.weight": {
292
+ "dtype": "F32",
293
+ "shape": [
294
+ 3072,
295
+ 64
296
+ ]
297
+ },
298
+ "blocks.13.ffn.net.0.proj.action.down.weight": {
299
+ "dtype": "F32",
300
+ "shape": [
301
+ 64,
302
+ 3072
303
+ ]
304
+ },
305
+ "blocks.13.ffn.net.0.proj.action.up.weight": {
306
+ "dtype": "F32",
307
+ "shape": [
308
+ 14336,
309
+ 64
310
+ ]
311
+ },
312
+ "blocks.13.ffn.net.0.proj.video.down.weight": {
313
+ "dtype": "F32",
314
+ "shape": [
315
+ 64,
316
+ 3072
317
+ ]
318
+ },
319
+ "blocks.13.ffn.net.0.proj.video.up.weight": {
320
+ "dtype": "F32",
321
+ "shape": [
322
+ 14336,
323
+ 64
324
+ ]
325
+ },
326
+ "blocks.13.ffn.net.2.action.down.weight": {
327
+ "dtype": "F32",
328
+ "shape": [
329
+ 64,
330
+ 14336
331
+ ]
332
+ },
333
+ "blocks.13.ffn.net.2.action.up.weight": {
334
+ "dtype": "F32",
335
+ "shape": [
336
+ 3072,
337
+ 64
338
+ ]
339
+ },
340
+ "blocks.13.ffn.net.2.video.down.weight": {
341
+ "dtype": "F32",
342
+ "shape": [
343
+ 64,
344
+ 14336
345
+ ]
346
+ },
347
+ "blocks.13.ffn.net.2.video.up.weight": {
348
+ "dtype": "F32",
349
+ "shape": [
350
+ 3072,
351
+ 64
352
+ ]
353
+ },
354
+ "blocks.14.ffn.net.0.proj.action.down.weight": {
355
+ "dtype": "F32",
356
+ "shape": [
357
+ 64,
358
+ 3072
359
+ ]
360
+ },
361
+ "blocks.14.ffn.net.0.proj.action.up.weight": {
362
+ "dtype": "F32",
363
+ "shape": [
364
+ 14336,
365
+ 64
366
+ ]
367
+ },
368
+ "blocks.14.ffn.net.0.proj.video.down.weight": {
369
+ "dtype": "F32",
370
+ "shape": [
371
+ 64,
372
+ 3072
373
+ ]
374
+ },
375
+ "blocks.14.ffn.net.0.proj.video.up.weight": {
376
+ "dtype": "F32",
377
+ "shape": [
378
+ 14336,
379
+ 64
380
+ ]
381
+ },
382
+ "blocks.14.ffn.net.2.action.down.weight": {
383
+ "dtype": "F32",
384
+ "shape": [
385
+ 64,
386
+ 14336
387
+ ]
388
+ },
389
+ "blocks.14.ffn.net.2.action.up.weight": {
390
+ "dtype": "F32",
391
+ "shape": [
392
+ 3072,
393
+ 64
394
+ ]
395
+ },
396
+ "blocks.14.ffn.net.2.video.down.weight": {
397
+ "dtype": "F32",
398
+ "shape": [
399
+ 64,
400
+ 14336
401
+ ]
402
+ },
403
+ "blocks.14.ffn.net.2.video.up.weight": {
404
+ "dtype": "F32",
405
+ "shape": [
406
+ 3072,
407
+ 64
408
+ ]
409
+ },
410
+ "blocks.15.ffn.net.0.proj.action.down.weight": {
411
+ "dtype": "F32",
412
+ "shape": [
413
+ 64,
414
+ 3072
415
+ ]
416
+ },
417
+ "blocks.15.ffn.net.0.proj.action.up.weight": {
418
+ "dtype": "F32",
419
+ "shape": [
420
+ 14336,
421
+ 64
422
+ ]
423
+ },
424
+ "blocks.15.ffn.net.0.proj.video.down.weight": {
425
+ "dtype": "F32",
426
+ "shape": [
427
+ 64,
428
+ 3072
429
+ ]
430
+ },
431
+ "blocks.15.ffn.net.0.proj.video.up.weight": {
432
+ "dtype": "F32",
433
+ "shape": [
434
+ 14336,
435
+ 64
436
+ ]
437
+ },
438
+ "blocks.15.ffn.net.2.action.down.weight": {
439
+ "dtype": "F32",
440
+ "shape": [
441
+ 64,
442
+ 14336
443
+ ]
444
+ },
445
+ "blocks.15.ffn.net.2.action.up.weight": {
446
+ "dtype": "F32",
447
+ "shape": [
448
+ 3072,
449
+ 64
450
+ ]
451
+ },
452
+ "blocks.15.ffn.net.2.video.down.weight": {
453
+ "dtype": "F32",
454
+ "shape": [
455
+ 64,
456
+ 14336
457
+ ]
458
+ },
459
+ "blocks.15.ffn.net.2.video.up.weight": {
460
+ "dtype": "F32",
461
+ "shape": [
462
+ 3072,
463
+ 64
464
+ ]
465
+ },
466
+ "blocks.16.ffn.net.0.proj.action.down.weight": {
467
+ "dtype": "F32",
468
+ "shape": [
469
+ 64,
470
+ 3072
471
+ ]
472
+ },
473
+ "blocks.16.ffn.net.0.proj.action.up.weight": {
474
+ "dtype": "F32",
475
+ "shape": [
476
+ 14336,
477
+ 64
478
+ ]
479
+ },
480
+ "blocks.16.ffn.net.0.proj.video.down.weight": {
481
+ "dtype": "F32",
482
+ "shape": [
483
+ 64,
484
+ 3072
485
+ ]
486
+ },
487
+ "blocks.16.ffn.net.0.proj.video.up.weight": {
488
+ "dtype": "F32",
489
+ "shape": [
490
+ 14336,
491
+ 64
492
+ ]
493
+ },
494
+ "blocks.16.ffn.net.2.action.down.weight": {
495
+ "dtype": "F32",
496
+ "shape": [
497
+ 64,
498
+ 14336
499
+ ]
500
+ },
501
+ "blocks.16.ffn.net.2.action.up.weight": {
502
+ "dtype": "F32",
503
+ "shape": [
504
+ 3072,
505
+ 64
506
+ ]
507
+ },
508
+ "blocks.16.ffn.net.2.video.down.weight": {
509
+ "dtype": "F32",
510
+ "shape": [
511
+ 64,
512
+ 14336
513
+ ]
514
+ },
515
+ "blocks.16.ffn.net.2.video.up.weight": {
516
+ "dtype": "F32",
517
+ "shape": [
518
+ 3072,
519
+ 64
520
+ ]
521
+ },
522
+ "blocks.17.ffn.net.0.proj.action.down.weight": {
523
+ "dtype": "F32",
524
+ "shape": [
525
+ 64,
526
+ 3072
527
+ ]
528
+ },
529
+ "blocks.17.ffn.net.0.proj.action.up.weight": {
530
+ "dtype": "F32",
531
+ "shape": [
532
+ 14336,
533
+ 64
534
+ ]
535
+ },
536
+ "blocks.17.ffn.net.0.proj.video.down.weight": {
537
+ "dtype": "F32",
538
+ "shape": [
539
+ 64,
540
+ 3072
541
+ ]
542
+ },
543
+ "blocks.17.ffn.net.0.proj.video.up.weight": {
544
+ "dtype": "F32",
545
+ "shape": [
546
+ 14336,
547
+ 64
548
+ ]
549
+ },
550
+ "blocks.17.ffn.net.2.action.down.weight": {
551
+ "dtype": "F32",
552
+ "shape": [
553
+ 64,
554
+ 14336
555
+ ]
556
+ },
557
+ "blocks.17.ffn.net.2.action.up.weight": {
558
+ "dtype": "F32",
559
+ "shape": [
560
+ 3072,
561
+ 64
562
+ ]
563
+ },
564
+ "blocks.17.ffn.net.2.video.down.weight": {
565
+ "dtype": "F32",
566
+ "shape": [
567
+ 64,
568
+ 14336
569
+ ]
570
+ },
571
+ "blocks.17.ffn.net.2.video.up.weight": {
572
+ "dtype": "F32",
573
+ "shape": [
574
+ 3072,
575
+ 64
576
+ ]
577
+ },
578
+ "blocks.18.ffn.net.0.proj.action.down.weight": {
579
+ "dtype": "F32",
580
+ "shape": [
581
+ 64,
582
+ 3072
583
+ ]
584
+ },
585
+ "blocks.18.ffn.net.0.proj.action.up.weight": {
586
+ "dtype": "F32",
587
+ "shape": [
588
+ 14336,
589
+ 64
590
+ ]
591
+ },
592
+ "blocks.18.ffn.net.0.proj.video.down.weight": {
593
+ "dtype": "F32",
594
+ "shape": [
595
+ 64,
596
+ 3072
597
+ ]
598
+ },
599
+ "blocks.18.ffn.net.0.proj.video.up.weight": {
600
+ "dtype": "F32",
601
+ "shape": [
602
+ 14336,
603
+ 64
604
+ ]
605
+ },
606
+ "blocks.18.ffn.net.2.action.down.weight": {
607
+ "dtype": "F32",
608
+ "shape": [
609
+ 64,
610
+ 14336
611
+ ]
612
+ },
613
+ "blocks.18.ffn.net.2.action.up.weight": {
614
+ "dtype": "F32",
615
+ "shape": [
616
+ 3072,
617
+ 64
618
+ ]
619
+ },
620
+ "blocks.18.ffn.net.2.video.down.weight": {
621
+ "dtype": "F32",
622
+ "shape": [
623
+ 64,
624
+ 14336
625
+ ]
626
+ },
627
+ "blocks.18.ffn.net.2.video.up.weight": {
628
+ "dtype": "F32",
629
+ "shape": [
630
+ 3072,
631
+ 64
632
+ ]
633
+ },
634
+ "blocks.19.ffn.net.0.proj.action.down.weight": {
635
+ "dtype": "F32",
636
+ "shape": [
637
+ 64,
638
+ 3072
639
+ ]
640
+ },
641
+ "blocks.19.ffn.net.0.proj.action.up.weight": {
642
+ "dtype": "F32",
643
+ "shape": [
644
+ 14336,
645
+ 64
646
+ ]
647
+ },
648
+ "blocks.19.ffn.net.0.proj.video.down.weight": {
649
+ "dtype": "F32",
650
+ "shape": [
651
+ 64,
652
+ 3072
653
+ ]
654
+ },
655
+ "blocks.19.ffn.net.0.proj.video.up.weight": {
656
+ "dtype": "F32",
657
+ "shape": [
658
+ 14336,
659
+ 64
660
+ ]
661
+ },
662
+ "blocks.19.ffn.net.2.action.down.weight": {
663
+ "dtype": "F32",
664
+ "shape": [
665
+ 64,
666
+ 14336
667
+ ]
668
+ },
669
+ "blocks.19.ffn.net.2.action.up.weight": {
670
+ "dtype": "F32",
671
+ "shape": [
672
+ 3072,
673
+ 64
674
+ ]
675
+ },
676
+ "blocks.19.ffn.net.2.video.down.weight": {
677
+ "dtype": "F32",
678
+ "shape": [
679
+ 64,
680
+ 14336
681
+ ]
682
+ },
683
+ "blocks.19.ffn.net.2.video.up.weight": {
684
+ "dtype": "F32",
685
+ "shape": [
686
+ 3072,
687
+ 64
688
+ ]
689
+ },
690
+ "blocks.2.ffn.net.0.proj.action.down.weight": {
691
+ "dtype": "F32",
692
+ "shape": [
693
+ 64,
694
+ 3072
695
+ ]
696
+ },
697
+ "blocks.2.ffn.net.0.proj.action.up.weight": {
698
+ "dtype": "F32",
699
+ "shape": [
700
+ 14336,
701
+ 64
702
+ ]
703
+ },
704
+ "blocks.2.ffn.net.0.proj.video.down.weight": {
705
+ "dtype": "F32",
706
+ "shape": [
707
+ 64,
708
+ 3072
709
+ ]
710
+ },
711
+ "blocks.2.ffn.net.0.proj.video.up.weight": {
712
+ "dtype": "F32",
713
+ "shape": [
714
+ 14336,
715
+ 64
716
+ ]
717
+ },
718
+ "blocks.2.ffn.net.2.action.down.weight": {
719
+ "dtype": "F32",
720
+ "shape": [
721
+ 64,
722
+ 14336
723
+ ]
724
+ },
725
+ "blocks.2.ffn.net.2.action.up.weight": {
726
+ "dtype": "F32",
727
+ "shape": [
728
+ 3072,
729
+ 64
730
+ ]
731
+ },
732
+ "blocks.2.ffn.net.2.video.down.weight": {
733
+ "dtype": "F32",
734
+ "shape": [
735
+ 64,
736
+ 14336
737
+ ]
738
+ },
739
+ "blocks.2.ffn.net.2.video.up.weight": {
740
+ "dtype": "F32",
741
+ "shape": [
742
+ 3072,
743
+ 64
744
+ ]
745
+ },
746
+ "blocks.20.ffn.net.0.proj.action.down.weight": {
747
+ "dtype": "F32",
748
+ "shape": [
749
+ 64,
750
+ 3072
751
+ ]
752
+ },
753
+ "blocks.20.ffn.net.0.proj.action.up.weight": {
754
+ "dtype": "F32",
755
+ "shape": [
756
+ 14336,
757
+ 64
758
+ ]
759
+ },
760
+ "blocks.20.ffn.net.0.proj.video.down.weight": {
761
+ "dtype": "F32",
762
+ "shape": [
763
+ 64,
764
+ 3072
765
+ ]
766
+ },
767
+ "blocks.20.ffn.net.0.proj.video.up.weight": {
768
+ "dtype": "F32",
769
+ "shape": [
770
+ 14336,
771
+ 64
772
+ ]
773
+ },
774
+ "blocks.20.ffn.net.2.action.down.weight": {
775
+ "dtype": "F32",
776
+ "shape": [
777
+ 64,
778
+ 14336
779
+ ]
780
+ },
781
+ "blocks.20.ffn.net.2.action.up.weight": {
782
+ "dtype": "F32",
783
+ "shape": [
784
+ 3072,
785
+ 64
786
+ ]
787
+ },
788
+ "blocks.20.ffn.net.2.video.down.weight": {
789
+ "dtype": "F32",
790
+ "shape": [
791
+ 64,
792
+ 14336
793
+ ]
794
+ },
795
+ "blocks.20.ffn.net.2.video.up.weight": {
796
+ "dtype": "F32",
797
+ "shape": [
798
+ 3072,
799
+ 64
800
+ ]
801
+ },
802
+ "blocks.21.ffn.net.0.proj.action.down.weight": {
803
+ "dtype": "F32",
804
+ "shape": [
805
+ 64,
806
+ 3072
807
+ ]
808
+ },
809
+ "blocks.21.ffn.net.0.proj.action.up.weight": {
810
+ "dtype": "F32",
811
+ "shape": [
812
+ 14336,
813
+ 64
814
+ ]
815
+ },
816
+ "blocks.21.ffn.net.0.proj.video.down.weight": {
817
+ "dtype": "F32",
818
+ "shape": [
819
+ 64,
820
+ 3072
821
+ ]
822
+ },
823
+ "blocks.21.ffn.net.0.proj.video.up.weight": {
824
+ "dtype": "F32",
825
+ "shape": [
826
+ 14336,
827
+ 64
828
+ ]
829
+ },
830
+ "blocks.21.ffn.net.2.action.down.weight": {
831
+ "dtype": "F32",
832
+ "shape": [
833
+ 64,
834
+ 14336
835
+ ]
836
+ },
837
+ "blocks.21.ffn.net.2.action.up.weight": {
838
+ "dtype": "F32",
839
+ "shape": [
840
+ 3072,
841
+ 64
842
+ ]
843
+ },
844
+ "blocks.21.ffn.net.2.video.down.weight": {
845
+ "dtype": "F32",
846
+ "shape": [
847
+ 64,
848
+ 14336
849
+ ]
850
+ },
851
+ "blocks.21.ffn.net.2.video.up.weight": {
852
+ "dtype": "F32",
853
+ "shape": [
854
+ 3072,
855
+ 64
856
+ ]
857
+ },
858
+ "blocks.22.ffn.net.0.proj.action.down.weight": {
859
+ "dtype": "F32",
860
+ "shape": [
861
+ 64,
862
+ 3072
863
+ ]
864
+ },
865
+ "blocks.22.ffn.net.0.proj.action.up.weight": {
866
+ "dtype": "F32",
867
+ "shape": [
868
+ 14336,
869
+ 64
870
+ ]
871
+ },
872
+ "blocks.22.ffn.net.0.proj.video.down.weight": {
873
+ "dtype": "F32",
874
+ "shape": [
875
+ 64,
876
+ 3072
877
+ ]
878
+ },
879
+ "blocks.22.ffn.net.0.proj.video.up.weight": {
880
+ "dtype": "F32",
881
+ "shape": [
882
+ 14336,
883
+ 64
884
+ ]
885
+ },
886
+ "blocks.22.ffn.net.2.action.down.weight": {
887
+ "dtype": "F32",
888
+ "shape": [
889
+ 64,
890
+ 14336
891
+ ]
892
+ },
893
+ "blocks.22.ffn.net.2.action.up.weight": {
894
+ "dtype": "F32",
895
+ "shape": [
896
+ 3072,
897
+ 64
898
+ ]
899
+ },
900
+ "blocks.22.ffn.net.2.video.down.weight": {
901
+ "dtype": "F32",
902
+ "shape": [
903
+ 64,
904
+ 14336
905
+ ]
906
+ },
907
+ "blocks.22.ffn.net.2.video.up.weight": {
908
+ "dtype": "F32",
909
+ "shape": [
910
+ 3072,
911
+ 64
912
+ ]
913
+ },
914
+ "blocks.23.ffn.net.0.proj.action.down.weight": {
915
+ "dtype": "F32",
916
+ "shape": [
917
+ 64,
918
+ 3072
919
+ ]
920
+ },
921
+ "blocks.23.ffn.net.0.proj.action.up.weight": {
922
+ "dtype": "F32",
923
+ "shape": [
924
+ 14336,
925
+ 64
926
+ ]
927
+ },
928
+ "blocks.23.ffn.net.0.proj.video.down.weight": {
929
+ "dtype": "F32",
930
+ "shape": [
931
+ 64,
932
+ 3072
933
+ ]
934
+ },
935
+ "blocks.23.ffn.net.0.proj.video.up.weight": {
936
+ "dtype": "F32",
937
+ "shape": [
938
+ 14336,
939
+ 64
940
+ ]
941
+ },
942
+ "blocks.23.ffn.net.2.action.down.weight": {
943
+ "dtype": "F32",
944
+ "shape": [
945
+ 64,
946
+ 14336
947
+ ]
948
+ },
949
+ "blocks.23.ffn.net.2.action.up.weight": {
950
+ "dtype": "F32",
951
+ "shape": [
952
+ 3072,
953
+ 64
954
+ ]
955
+ },
956
+ "blocks.23.ffn.net.2.video.down.weight": {
957
+ "dtype": "F32",
958
+ "shape": [
959
+ 64,
960
+ 14336
961
+ ]
962
+ },
963
+ "blocks.23.ffn.net.2.video.up.weight": {
964
+ "dtype": "F32",
965
+ "shape": [
966
+ 3072,
967
+ 64
968
+ ]
969
+ },
970
+ "blocks.24.ffn.net.0.proj.action.down.weight": {
971
+ "dtype": "F32",
972
+ "shape": [
973
+ 64,
974
+ 3072
975
+ ]
976
+ },
977
+ "blocks.24.ffn.net.0.proj.action.up.weight": {
978
+ "dtype": "F32",
979
+ "shape": [
980
+ 14336,
981
+ 64
982
+ ]
983
+ },
984
+ "blocks.24.ffn.net.0.proj.video.down.weight": {
985
+ "dtype": "F32",
986
+ "shape": [
987
+ 64,
988
+ 3072
989
+ ]
990
+ },
991
+ "blocks.24.ffn.net.0.proj.video.up.weight": {
992
+ "dtype": "F32",
993
+ "shape": [
994
+ 14336,
995
+ 64
996
+ ]
997
+ },
998
+ "blocks.24.ffn.net.2.action.down.weight": {
999
+ "dtype": "F32",
1000
+ "shape": [
1001
+ 64,
1002
+ 14336
1003
+ ]
1004
+ },
1005
+ "blocks.24.ffn.net.2.action.up.weight": {
1006
+ "dtype": "F32",
1007
+ "shape": [
1008
+ 3072,
1009
+ 64
1010
+ ]
1011
+ },
1012
+ "blocks.24.ffn.net.2.video.down.weight": {
1013
+ "dtype": "F32",
1014
+ "shape": [
1015
+ 64,
1016
+ 14336
1017
+ ]
1018
+ },
1019
+ "blocks.24.ffn.net.2.video.up.weight": {
1020
+ "dtype": "F32",
1021
+ "shape": [
1022
+ 3072,
1023
+ 64
1024
+ ]
1025
+ },
1026
+ "blocks.25.ffn.net.0.proj.action.down.weight": {
1027
+ "dtype": "F32",
1028
+ "shape": [
1029
+ 64,
1030
+ 3072
1031
+ ]
1032
+ },
1033
+ "blocks.25.ffn.net.0.proj.action.up.weight": {
1034
+ "dtype": "F32",
1035
+ "shape": [
1036
+ 14336,
1037
+ 64
1038
+ ]
1039
+ },
1040
+ "blocks.25.ffn.net.0.proj.video.down.weight": {
1041
+ "dtype": "F32",
1042
+ "shape": [
1043
+ 64,
1044
+ 3072
1045
+ ]
1046
+ },
1047
+ "blocks.25.ffn.net.0.proj.video.up.weight": {
1048
+ "dtype": "F32",
1049
+ "shape": [
1050
+ 14336,
1051
+ 64
1052
+ ]
1053
+ },
1054
+ "blocks.25.ffn.net.2.action.down.weight": {
1055
+ "dtype": "F32",
1056
+ "shape": [
1057
+ 64,
1058
+ 14336
1059
+ ]
1060
+ },
1061
+ "blocks.25.ffn.net.2.action.up.weight": {
1062
+ "dtype": "F32",
1063
+ "shape": [
1064
+ 3072,
1065
+ 64
1066
+ ]
1067
+ },
1068
+ "blocks.25.ffn.net.2.video.down.weight": {
1069
+ "dtype": "F32",
1070
+ "shape": [
1071
+ 64,
1072
+ 14336
1073
+ ]
1074
+ },
1075
+ "blocks.25.ffn.net.2.video.up.weight": {
1076
+ "dtype": "F32",
1077
+ "shape": [
1078
+ 3072,
1079
+ 64
1080
+ ]
1081
+ },
1082
+ "blocks.26.ffn.net.0.proj.action.down.weight": {
1083
+ "dtype": "F32",
1084
+ "shape": [
1085
+ 64,
1086
+ 3072
1087
+ ]
1088
+ },
1089
+ "blocks.26.ffn.net.0.proj.action.up.weight": {
1090
+ "dtype": "F32",
1091
+ "shape": [
1092
+ 14336,
1093
+ 64
1094
+ ]
1095
+ },
1096
+ "blocks.26.ffn.net.0.proj.video.down.weight": {
1097
+ "dtype": "F32",
1098
+ "shape": [
1099
+ 64,
1100
+ 3072
1101
+ ]
1102
+ },
1103
+ "blocks.26.ffn.net.0.proj.video.up.weight": {
1104
+ "dtype": "F32",
1105
+ "shape": [
1106
+ 14336,
1107
+ 64
1108
+ ]
1109
+ },
1110
+ "blocks.26.ffn.net.2.action.down.weight": {
1111
+ "dtype": "F32",
1112
+ "shape": [
1113
+ 64,
1114
+ 14336
1115
+ ]
1116
+ },
1117
+ "blocks.26.ffn.net.2.action.up.weight": {
1118
+ "dtype": "F32",
1119
+ "shape": [
1120
+ 3072,
1121
+ 64
1122
+ ]
1123
+ },
1124
+ "blocks.26.ffn.net.2.video.down.weight": {
1125
+ "dtype": "F32",
1126
+ "shape": [
1127
+ 64,
1128
+ 14336
1129
+ ]
1130
+ },
1131
+ "blocks.26.ffn.net.2.video.up.weight": {
1132
+ "dtype": "F32",
1133
+ "shape": [
1134
+ 3072,
1135
+ 64
1136
+ ]
1137
+ },
1138
+ "blocks.27.ffn.net.0.proj.action.down.weight": {
1139
+ "dtype": "F32",
1140
+ "shape": [
1141
+ 64,
1142
+ 3072
1143
+ ]
1144
+ },
1145
+ "blocks.27.ffn.net.0.proj.action.up.weight": {
1146
+ "dtype": "F32",
1147
+ "shape": [
1148
+ 14336,
1149
+ 64
1150
+ ]
1151
+ },
1152
+ "blocks.27.ffn.net.0.proj.video.down.weight": {
1153
+ "dtype": "F32",
1154
+ "shape": [
1155
+ 64,
1156
+ 3072
1157
+ ]
1158
+ },
1159
+ "blocks.27.ffn.net.0.proj.video.up.weight": {
1160
+ "dtype": "F32",
1161
+ "shape": [
1162
+ 14336,
1163
+ 64
1164
+ ]
1165
+ },
1166
+ "blocks.27.ffn.net.2.action.down.weight": {
1167
+ "dtype": "F32",
1168
+ "shape": [
1169
+ 64,
1170
+ 14336
1171
+ ]
1172
+ },
1173
+ "blocks.27.ffn.net.2.action.up.weight": {
1174
+ "dtype": "F32",
1175
+ "shape": [
1176
+ 3072,
1177
+ 64
1178
+ ]
1179
+ },
1180
+ "blocks.27.ffn.net.2.video.down.weight": {
1181
+ "dtype": "F32",
1182
+ "shape": [
1183
+ 64,
1184
+ 14336
1185
+ ]
1186
+ },
1187
+ "blocks.27.ffn.net.2.video.up.weight": {
1188
+ "dtype": "F32",
1189
+ "shape": [
1190
+ 3072,
1191
+ 64
1192
+ ]
1193
+ },
1194
+ "blocks.28.ffn.net.0.proj.action.down.weight": {
1195
+ "dtype": "F32",
1196
+ "shape": [
1197
+ 64,
1198
+ 3072
1199
+ ]
1200
+ },
1201
+ "blocks.28.ffn.net.0.proj.action.up.weight": {
1202
+ "dtype": "F32",
1203
+ "shape": [
1204
+ 14336,
1205
+ 64
1206
+ ]
1207
+ },
1208
+ "blocks.28.ffn.net.0.proj.video.down.weight": {
1209
+ "dtype": "F32",
1210
+ "shape": [
1211
+ 64,
1212
+ 3072
1213
+ ]
1214
+ },
1215
+ "blocks.28.ffn.net.0.proj.video.up.weight": {
1216
+ "dtype": "F32",
1217
+ "shape": [
1218
+ 14336,
1219
+ 64
1220
+ ]
1221
+ },
1222
+ "blocks.28.ffn.net.2.action.down.weight": {
1223
+ "dtype": "F32",
1224
+ "shape": [
1225
+ 64,
1226
+ 14336
1227
+ ]
1228
+ },
1229
+ "blocks.28.ffn.net.2.action.up.weight": {
1230
+ "dtype": "F32",
1231
+ "shape": [
1232
+ 3072,
1233
+ 64
1234
+ ]
1235
+ },
1236
+ "blocks.28.ffn.net.2.video.down.weight": {
1237
+ "dtype": "F32",
1238
+ "shape": [
1239
+ 64,
1240
+ 14336
1241
+ ]
1242
+ },
1243
+ "blocks.28.ffn.net.2.video.up.weight": {
1244
+ "dtype": "F32",
1245
+ "shape": [
1246
+ 3072,
1247
+ 64
1248
+ ]
1249
+ },
1250
+ "blocks.29.ffn.net.0.proj.action.down.weight": {
1251
+ "dtype": "F32",
1252
+ "shape": [
1253
+ 64,
1254
+ 3072
1255
+ ]
1256
+ },
1257
+ "blocks.29.ffn.net.0.proj.action.up.weight": {
1258
+ "dtype": "F32",
1259
+ "shape": [
1260
+ 14336,
1261
+ 64
1262
+ ]
1263
+ },
1264
+ "blocks.29.ffn.net.0.proj.video.down.weight": {
1265
+ "dtype": "F32",
1266
+ "shape": [
1267
+ 64,
1268
+ 3072
1269
+ ]
1270
+ },
1271
+ "blocks.29.ffn.net.0.proj.video.up.weight": {
1272
+ "dtype": "F32",
1273
+ "shape": [
1274
+ 14336,
1275
+ 64
1276
+ ]
1277
+ },
1278
+ "blocks.29.ffn.net.2.action.down.weight": {
1279
+ "dtype": "F32",
1280
+ "shape": [
1281
+ 64,
1282
+ 14336
1283
+ ]
1284
+ },
1285
+ "blocks.29.ffn.net.2.action.up.weight": {
1286
+ "dtype": "F32",
1287
+ "shape": [
1288
+ 3072,
1289
+ 64
1290
+ ]
1291
+ },
1292
+ "blocks.29.ffn.net.2.video.down.weight": {
1293
+ "dtype": "F32",
1294
+ "shape": [
1295
+ 64,
1296
+ 14336
1297
+ ]
1298
+ },
1299
+ "blocks.29.ffn.net.2.video.up.weight": {
1300
+ "dtype": "F32",
1301
+ "shape": [
1302
+ 3072,
1303
+ 64
1304
+ ]
1305
+ },
1306
+ "blocks.3.ffn.net.0.proj.action.down.weight": {
1307
+ "dtype": "F32",
1308
+ "shape": [
1309
+ 64,
1310
+ 3072
1311
+ ]
1312
+ },
1313
+ "blocks.3.ffn.net.0.proj.action.up.weight": {
1314
+ "dtype": "F32",
1315
+ "shape": [
1316
+ 14336,
1317
+ 64
1318
+ ]
1319
+ },
1320
+ "blocks.3.ffn.net.0.proj.video.down.weight": {
1321
+ "dtype": "F32",
1322
+ "shape": [
1323
+ 64,
1324
+ 3072
1325
+ ]
1326
+ },
1327
+ "blocks.3.ffn.net.0.proj.video.up.weight": {
1328
+ "dtype": "F32",
1329
+ "shape": [
1330
+ 14336,
1331
+ 64
1332
+ ]
1333
+ },
1334
+ "blocks.3.ffn.net.2.action.down.weight": {
1335
+ "dtype": "F32",
1336
+ "shape": [
1337
+ 64,
1338
+ 14336
1339
+ ]
1340
+ },
1341
+ "blocks.3.ffn.net.2.action.up.weight": {
1342
+ "dtype": "F32",
1343
+ "shape": [
1344
+ 3072,
1345
+ 64
1346
+ ]
1347
+ },
1348
+ "blocks.3.ffn.net.2.video.down.weight": {
1349
+ "dtype": "F32",
1350
+ "shape": [
1351
+ 64,
1352
+ 14336
1353
+ ]
1354
+ },
1355
+ "blocks.3.ffn.net.2.video.up.weight": {
1356
+ "dtype": "F32",
1357
+ "shape": [
1358
+ 3072,
1359
+ 64
1360
+ ]
1361
+ },
1362
+ "blocks.4.ffn.net.0.proj.action.down.weight": {
1363
+ "dtype": "F32",
1364
+ "shape": [
1365
+ 64,
1366
+ 3072
1367
+ ]
1368
+ },
1369
+ "blocks.4.ffn.net.0.proj.action.up.weight": {
1370
+ "dtype": "F32",
1371
+ "shape": [
1372
+ 14336,
1373
+ 64
1374
+ ]
1375
+ },
1376
+ "blocks.4.ffn.net.0.proj.video.down.weight": {
1377
+ "dtype": "F32",
1378
+ "shape": [
1379
+ 64,
1380
+ 3072
1381
+ ]
1382
+ },
1383
+ "blocks.4.ffn.net.0.proj.video.up.weight": {
1384
+ "dtype": "F32",
1385
+ "shape": [
1386
+ 14336,
1387
+ 64
1388
+ ]
1389
+ },
1390
+ "blocks.4.ffn.net.2.action.down.weight": {
1391
+ "dtype": "F32",
1392
+ "shape": [
1393
+ 64,
1394
+ 14336
1395
+ ]
1396
+ },
1397
+ "blocks.4.ffn.net.2.action.up.weight": {
1398
+ "dtype": "F32",
1399
+ "shape": [
1400
+ 3072,
1401
+ 64
1402
+ ]
1403
+ },
1404
+ "blocks.4.ffn.net.2.video.down.weight": {
1405
+ "dtype": "F32",
1406
+ "shape": [
1407
+ 64,
1408
+ 14336
1409
+ ]
1410
+ },
1411
+ "blocks.4.ffn.net.2.video.up.weight": {
1412
+ "dtype": "F32",
1413
+ "shape": [
1414
+ 3072,
1415
+ 64
1416
+ ]
1417
+ },
1418
+ "blocks.5.ffn.net.0.proj.action.down.weight": {
1419
+ "dtype": "F32",
1420
+ "shape": [
1421
+ 64,
1422
+ 3072
1423
+ ]
1424
+ },
1425
+ "blocks.5.ffn.net.0.proj.action.up.weight": {
1426
+ "dtype": "F32",
1427
+ "shape": [
1428
+ 14336,
1429
+ 64
1430
+ ]
1431
+ },
1432
+ "blocks.5.ffn.net.0.proj.video.down.weight": {
1433
+ "dtype": "F32",
1434
+ "shape": [
1435
+ 64,
1436
+ 3072
1437
+ ]
1438
+ },
1439
+ "blocks.5.ffn.net.0.proj.video.up.weight": {
1440
+ "dtype": "F32",
1441
+ "shape": [
1442
+ 14336,
1443
+ 64
1444
+ ]
1445
+ },
1446
+ "blocks.5.ffn.net.2.action.down.weight": {
1447
+ "dtype": "F32",
1448
+ "shape": [
1449
+ 64,
1450
+ 14336
1451
+ ]
1452
+ },
1453
+ "blocks.5.ffn.net.2.action.up.weight": {
1454
+ "dtype": "F32",
1455
+ "shape": [
1456
+ 3072,
1457
+ 64
1458
+ ]
1459
+ },
1460
+ "blocks.5.ffn.net.2.video.down.weight": {
1461
+ "dtype": "F32",
1462
+ "shape": [
1463
+ 64,
1464
+ 14336
1465
+ ]
1466
+ },
1467
+ "blocks.5.ffn.net.2.video.up.weight": {
1468
+ "dtype": "F32",
1469
+ "shape": [
1470
+ 3072,
1471
+ 64
1472
+ ]
1473
+ },
1474
+ "blocks.6.ffn.net.0.proj.action.down.weight": {
1475
+ "dtype": "F32",
1476
+ "shape": [
1477
+ 64,
1478
+ 3072
1479
+ ]
1480
+ },
1481
+ "blocks.6.ffn.net.0.proj.action.up.weight": {
1482
+ "dtype": "F32",
1483
+ "shape": [
1484
+ 14336,
1485
+ 64
1486
+ ]
1487
+ },
1488
+ "blocks.6.ffn.net.0.proj.video.down.weight": {
1489
+ "dtype": "F32",
1490
+ "shape": [
1491
+ 64,
1492
+ 3072
1493
+ ]
1494
+ },
1495
+ "blocks.6.ffn.net.0.proj.video.up.weight": {
1496
+ "dtype": "F32",
1497
+ "shape": [
1498
+ 14336,
1499
+ 64
1500
+ ]
1501
+ },
1502
+ "blocks.6.ffn.net.2.action.down.weight": {
1503
+ "dtype": "F32",
1504
+ "shape": [
1505
+ 64,
1506
+ 14336
1507
+ ]
1508
+ },
1509
+ "blocks.6.ffn.net.2.action.up.weight": {
1510
+ "dtype": "F32",
1511
+ "shape": [
1512
+ 3072,
1513
+ 64
1514
+ ]
1515
+ },
1516
+ "blocks.6.ffn.net.2.video.down.weight": {
1517
+ "dtype": "F32",
1518
+ "shape": [
1519
+ 64,
1520
+ 14336
1521
+ ]
1522
+ },
1523
+ "blocks.6.ffn.net.2.video.up.weight": {
1524
+ "dtype": "F32",
1525
+ "shape": [
1526
+ 3072,
1527
+ 64
1528
+ ]
1529
+ },
1530
+ "blocks.7.ffn.net.0.proj.action.down.weight": {
1531
+ "dtype": "F32",
1532
+ "shape": [
1533
+ 64,
1534
+ 3072
1535
+ ]
1536
+ },
1537
+ "blocks.7.ffn.net.0.proj.action.up.weight": {
1538
+ "dtype": "F32",
1539
+ "shape": [
1540
+ 14336,
1541
+ 64
1542
+ ]
1543
+ },
1544
+ "blocks.7.ffn.net.0.proj.video.down.weight": {
1545
+ "dtype": "F32",
1546
+ "shape": [
1547
+ 64,
1548
+ 3072
1549
+ ]
1550
+ },
1551
+ "blocks.7.ffn.net.0.proj.video.up.weight": {
1552
+ "dtype": "F32",
1553
+ "shape": [
1554
+ 14336,
1555
+ 64
1556
+ ]
1557
+ },
1558
+ "blocks.7.ffn.net.2.action.down.weight": {
1559
+ "dtype": "F32",
1560
+ "shape": [
1561
+ 64,
1562
+ 14336
1563
+ ]
1564
+ },
1565
+ "blocks.7.ffn.net.2.action.up.weight": {
1566
+ "dtype": "F32",
1567
+ "shape": [
1568
+ 3072,
1569
+ 64
1570
+ ]
1571
+ },
1572
+ "blocks.7.ffn.net.2.video.down.weight": {
1573
+ "dtype": "F32",
1574
+ "shape": [
1575
+ 64,
1576
+ 14336
1577
+ ]
1578
+ },
1579
+ "blocks.7.ffn.net.2.video.up.weight": {
1580
+ "dtype": "F32",
1581
+ "shape": [
1582
+ 3072,
1583
+ 64
1584
+ ]
1585
+ },
1586
+ "blocks.8.ffn.net.0.proj.action.down.weight": {
1587
+ "dtype": "F32",
1588
+ "shape": [
1589
+ 64,
1590
+ 3072
1591
+ ]
1592
+ },
1593
+ "blocks.8.ffn.net.0.proj.action.up.weight": {
1594
+ "dtype": "F32",
1595
+ "shape": [
1596
+ 14336,
1597
+ 64
1598
+ ]
1599
+ },
1600
+ "blocks.8.ffn.net.0.proj.video.down.weight": {
1601
+ "dtype": "F32",
1602
+ "shape": [
1603
+ 64,
1604
+ 3072
1605
+ ]
1606
+ },
1607
+ "blocks.8.ffn.net.0.proj.video.up.weight": {
1608
+ "dtype": "F32",
1609
+ "shape": [
1610
+ 14336,
1611
+ 64
1612
+ ]
1613
+ },
1614
+ "blocks.8.ffn.net.2.action.down.weight": {
1615
+ "dtype": "F32",
1616
+ "shape": [
1617
+ 64,
1618
+ 14336
1619
+ ]
1620
+ },
1621
+ "blocks.8.ffn.net.2.action.up.weight": {
1622
+ "dtype": "F32",
1623
+ "shape": [
1624
+ 3072,
1625
+ 64
1626
+ ]
1627
+ },
1628
+ "blocks.8.ffn.net.2.video.down.weight": {
1629
+ "dtype": "F32",
1630
+ "shape": [
1631
+ 64,
1632
+ 14336
1633
+ ]
1634
+ },
1635
+ "blocks.8.ffn.net.2.video.up.weight": {
1636
+ "dtype": "F32",
1637
+ "shape": [
1638
+ 3072,
1639
+ 64
1640
+ ]
1641
+ },
1642
+ "blocks.9.ffn.net.0.proj.action.down.weight": {
1643
+ "dtype": "F32",
1644
+ "shape": [
1645
+ 64,
1646
+ 3072
1647
+ ]
1648
+ },
1649
+ "blocks.9.ffn.net.0.proj.action.up.weight": {
1650
+ "dtype": "F32",
1651
+ "shape": [
1652
+ 14336,
1653
+ 64
1654
+ ]
1655
+ },
1656
+ "blocks.9.ffn.net.0.proj.video.down.weight": {
1657
+ "dtype": "F32",
1658
+ "shape": [
1659
+ 64,
1660
+ 3072
1661
+ ]
1662
+ },
1663
+ "blocks.9.ffn.net.0.proj.video.up.weight": {
1664
+ "dtype": "F32",
1665
+ "shape": [
1666
+ 14336,
1667
+ 64
1668
+ ]
1669
+ },
1670
+ "blocks.9.ffn.net.2.action.down.weight": {
1671
+ "dtype": "F32",
1672
+ "shape": [
1673
+ 64,
1674
+ 14336
1675
+ ]
1676
+ },
1677
+ "blocks.9.ffn.net.2.action.up.weight": {
1678
+ "dtype": "F32",
1679
+ "shape": [
1680
+ 3072,
1681
+ 64
1682
+ ]
1683
+ },
1684
+ "blocks.9.ffn.net.2.video.down.weight": {
1685
+ "dtype": "F32",
1686
+ "shape": [
1687
+ 64,
1688
+ 14336
1689
+ ]
1690
+ },
1691
+ "blocks.9.ffn.net.2.video.up.weight": {
1692
+ "dtype": "F32",
1693
+ "shape": [
1694
+ 3072,
1695
+ 64
1696
+ ]
1697
+ },
1698
+ "condition_embedder.delta_embedder.linear_1.shared.down.weight": {
1699
+ "dtype": "F32",
1700
+ "shape": [
1701
+ 128,
1702
+ 256
1703
+ ]
1704
+ },
1705
+ "condition_embedder.delta_embedder.linear_1.shared.up.weight": {
1706
+ "dtype": "F32",
1707
+ "shape": [
1708
+ 3072,
1709
+ 128
1710
+ ]
1711
+ },
1712
+ "condition_embedder.delta_embedder.linear_2.shared.down.weight": {
1713
+ "dtype": "F32",
1714
+ "shape": [
1715
+ 128,
1716
+ 3072
1717
+ ]
1718
+ },
1719
+ "condition_embedder.delta_embedder.linear_2.shared.up.weight": {
1720
+ "dtype": "F32",
1721
+ "shape": [
1722
+ 3072,
1723
+ 128
1724
+ ]
1725
+ },
1726
+ "condition_embedder.delta_output.shared.down.weight": {
1727
+ "dtype": "F32",
1728
+ "shape": [
1729
+ 128,
1730
+ 3072
1731
+ ]
1732
+ },
1733
+ "condition_embedder.delta_output.shared.up.weight": {
1734
+ "dtype": "F32",
1735
+ "shape": [
1736
+ 3072,
1737
+ 128
1738
+ ]
1739
+ },
1740
+ "condition_embedder.time_embedder.linear_1.shared.down.weight": {
1741
+ "dtype": "F32",
1742
+ "shape": [
1743
+ 128,
1744
+ 256
1745
+ ]
1746
+ },
1747
+ "condition_embedder.time_embedder.linear_1.shared.up.weight": {
1748
+ "dtype": "F32",
1749
+ "shape": [
1750
+ 3072,
1751
+ 128
1752
+ ]
1753
+ },
1754
+ "condition_embedder.time_embedder.linear_2.shared.down.weight": {
1755
+ "dtype": "F32",
1756
+ "shape": [
1757
+ 128,
1758
+ 3072
1759
+ ]
1760
+ },
1761
+ "condition_embedder.time_embedder.linear_2.shared.up.weight": {
1762
+ "dtype": "F32",
1763
+ "shape": [
1764
+ 3072,
1765
+ 128
1766
+ ]
1767
+ },
1768
+ "condition_embedder.time_proj.shared.down.weight": {
1769
+ "dtype": "F32",
1770
+ "shape": [
1771
+ 128,
1772
+ 3072
1773
+ ]
1774
+ },
1775
+ "condition_embedder.time_proj.shared.up.weight": {
1776
+ "dtype": "F32",
1777
+ "shape": [
1778
+ 18432,
1779
+ 128
1780
+ ]
1781
+ },
1782
+ "condition_embedder_action.delta_embedder.linear_1.shared.down.weight": {
1783
+ "dtype": "F32",
1784
+ "shape": [
1785
+ 128,
1786
+ 256
1787
+ ]
1788
+ },
1789
+ "condition_embedder_action.delta_embedder.linear_1.shared.up.weight": {
1790
+ "dtype": "F32",
1791
+ "shape": [
1792
+ 3072,
1793
+ 128
1794
+ ]
1795
+ },
1796
+ "condition_embedder_action.delta_embedder.linear_2.shared.down.weight": {
1797
+ "dtype": "F32",
1798
+ "shape": [
1799
+ 128,
1800
+ 3072
1801
+ ]
1802
+ },
1803
+ "condition_embedder_action.delta_embedder.linear_2.shared.up.weight": {
1804
+ "dtype": "F32",
1805
+ "shape": [
1806
+ 3072,
1807
+ 128
1808
+ ]
1809
+ },
1810
+ "condition_embedder_action.delta_output.shared.down.weight": {
1811
+ "dtype": "F32",
1812
+ "shape": [
1813
+ 128,
1814
+ 3072
1815
+ ]
1816
+ },
1817
+ "condition_embedder_action.delta_output.shared.up.weight": {
1818
+ "dtype": "F32",
1819
+ "shape": [
1820
+ 3072,
1821
+ 128
1822
+ ]
1823
+ },
1824
+ "condition_embedder_action.time_embedder.linear_1.shared.down.weight": {
1825
+ "dtype": "F32",
1826
+ "shape": [
1827
+ 128,
1828
+ 256
1829
+ ]
1830
+ },
1831
+ "condition_embedder_action.time_embedder.linear_1.shared.up.weight": {
1832
+ "dtype": "F32",
1833
+ "shape": [
1834
+ 3072,
1835
+ 128
1836
+ ]
1837
+ },
1838
+ "condition_embedder_action.time_embedder.linear_2.shared.down.weight": {
1839
+ "dtype": "F32",
1840
+ "shape": [
1841
+ 128,
1842
+ 3072
1843
+ ]
1844
+ },
1845
+ "condition_embedder_action.time_embedder.linear_2.shared.up.weight": {
1846
+ "dtype": "F32",
1847
+ "shape": [
1848
+ 3072,
1849
+ 128
1850
+ ]
1851
+ },
1852
+ "condition_embedder_action.time_proj.shared.down.weight": {
1853
+ "dtype": "F32",
1854
+ "shape": [
1855
+ 128,
1856
+ 3072
1857
+ ]
1858
+ },
1859
+ "condition_embedder_action.time_proj.shared.up.weight": {
1860
+ "dtype": "F32",
1861
+ "shape": [
1862
+ 18432,
1863
+ 128
1864
+ ]
1865
+ }
1866
+ },
1867
+ "kind": "adapter",
1868
+ "license": "Apache-2.0",
1869
+ "paper_role": "restart-no-field",
1870
+ "source_digest": "569409c1c901f2778056bf1899f90051141c59e047b9fd09a4ccc64d9147a9b0",
1871
+ "topology": "frozen-stage1-routed-ffn-time-v1:r64/r64/t128",
1872
+ "variant_id": "restart-no-field"
1873
+ }
verification/apm.json ADDED
@@ -0,0 +1,6 @@
 
 
 
 
 
 
 
1
+ {
2
+ "expected_digest": "7778d12f803b392d93f2f9d1c5ecee883d5353de692de5ed21bf311799f324a2",
3
+ "materialized_digest": "7778d12f803b392d93f2f9d1c5ecee883d5353de692de5ed21bf311799f324a2",
4
+ "ok": true,
5
+ "variant_id": "apm"
6
+ }
verification/continued.json ADDED
@@ -0,0 +1,6 @@
 
 
 
 
 
 
 
1
+ {
2
+ "expected_digest": "50b99e687204ef2f82b32d6a71eda67cbc472a98ee400094a3a7ab97c2b7289e",
3
+ "materialized_digest": "50b99e687204ef2f82b32d6a71eda67cbc472a98ee400094a3a7ab97c2b7289e",
4
+ "ok": true,
5
+ "variant_id": "continued"
6
+ }
verification/original-field-projected.json ADDED
@@ -0,0 +1,6 @@
 
 
 
 
 
 
 
1
+ {
2
+ "expected_digest": "a3cb591e6deeee93ee2d6732b9c122310eaad2bd3fefc3b57a4b3ae6722e3f94",
3
+ "materialized_digest": "a3cb591e6deeee93ee2d6732b9c122310eaad2bd3fefc3b57a4b3ae6722e3f94",
4
+ "ok": true,
5
+ "variant_id": "original-field-projected"
6
+ }
verification/restart-no-field.json ADDED
@@ -0,0 +1,6 @@
 
 
 
 
 
 
 
1
+ {
2
+ "expected_digest": "569409c1c901f2778056bf1899f90051141c59e047b9fd09a4ccc64d9147a9b0",
3
+ "materialized_digest": "569409c1c901f2778056bf1899f90051141c59e047b9fd09a4ccc64d9147a9b0",
4
+ "ok": true,
5
+ "variant_id": "restart-no-field"
6
+ }