fantastisk commited on
Commit
b6d2234
·
verified ·
1 Parent(s): d90ea8e

Add files using upload-large-folder tool

Browse files
.gitattributes CHANGED
@@ -33,3 +33,7 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
33
  *.zip filter=lfs diff=lfs merge=lfs -text
34
  *.zst filter=lfs diff=lfs merge=lfs -text
35
  *tfevents* filter=lfs diff=lfs merge=lfs -text
 
 
 
 
 
33
  *.zip filter=lfs diff=lfs merge=lfs -text
34
  *.zst filter=lfs diff=lfs merge=lfs -text
35
  *tfevents* filter=lfs diff=lfs merge=lfs -text
36
+ glm53_vision/weights.vbin filter=lfs diff=lfs merge=lfs -text
37
+ x3o_tp2_r0/index.json filter=lfs diff=lfs merge=lfs -text
38
+ glm53c_tp2_r0/weights.vbin filter=lfs diff=lfs merge=lfs -text
39
+ x3o_tp2_r0/weights.vbin filter=lfs diff=lfs merge=lfs -text
LICENSE ADDED
@@ -0,0 +1,21 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ MIT License
2
+
3
+ Copyright (c) 2026 Z.AI Co., Ltd
4
+
5
+ Permission is hereby granted, free of charge, to any person obtaining a copy
6
+ of this software and associated documentation files (the "Software"), to deal
7
+ in the Software without restriction, including without limitation the rights
8
+ to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
9
+ copies of the Software, and to permit persons to whom the Software is
10
+ furnished to do so, subject to the following conditions:
11
+
12
+ The above copyright notice and this permission notice shall be included in all
13
+ copies or substantial portions of the Software.
14
+
15
+ THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
16
+ IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
17
+ FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
18
+ AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
19
+ LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
20
+ OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
21
+ SOFTWARE.
NOTICE ADDED
@@ -0,0 +1,5 @@
 
 
 
 
 
 
1
+ Quantized and converted by Interactify ApS for Slipstream.
2
+ Copyright 2026 Interactify ApS
3
+
4
+ Based on GLM-5.3-Flash, Copyright (c) 2026 Z.AI Co., Ltd, MIT License, and on NVIDIA's
5
+ GLM-5.3-Flash-NVFP4 checkpoint (MIT License).
README.md ADDED
@@ -0,0 +1,78 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: mit
3
+ base_model:
4
+ - zai-org/GLM-5.3-Flash
5
+ pipeline_tag: text-generation
6
+ tags:
7
+ - slipstream
8
+ - dgx-spark
9
+ - nvfp4
10
+ - exl3
11
+ - glm
12
+ ---
13
+ # GLM-5.3-Flash-Slipstream
14
+
15
+ GLM-5.3-Flash for the Slipstream inference engine: NVIDIA's NVFP4 checkpoint converted for Slipstream, with the routed and shared experts re-quantized by us to EXL3 4.0 bpw from the BF16 weights (Hessian-calibrated), plus the vision tower.
16
+
17
+ These are **Slipstream weight files** (`weights.vbin` + `index.json` per directory), the engine's own pre-packed format. They run only with Slipstream; they are not safetensors and do not load in other runtimes. Built with Slipstream's converters at commit `20c2dd9` (2026-10-31).
18
+
19
+ Base model: [zai-org/GLM-5.3-Flash](https://huggingface.co/zai-org/GLM-5.3-Flash). Engine: [Slipstream](https://github.com/Interactify/slipstream) (engine code release coming soon). Hardware: two NVIDIA DGX Spark (GB10, 128 GB each) connected over their 200 GbE link.
20
+
21
+ ## Contents
22
+
23
+ | Directory | What | Node |
24
+ |---|---|---|
25
+ | `glm53c_tp2_r0` | model weights, rank 0 (NVFP4 + MXFP8 prefill copies, MTP head) | first |
26
+ | `glm53c_tp2_r1` | model weights, rank 1 | second |
27
+ | `x3o_tp2_r0` | EXL3 4.0 bpw experts, rank 0 | first |
28
+ | `x3o_tp2_r1` | EXL3 4.0 bpw experts, rank 1 | second |
29
+ | `glm53_vision` | vision tower (BF16) | first |
30
+
31
+ ## Download
32
+
33
+ On the first Spark (rank 0, runs the HTTP server):
34
+
35
+ ```bash
36
+ hf download Interactify/GLM-5.3-Flash-Slipstream --local-dir ~/vmodels \
37
+ --include "glm53c_tp2_r0/*" "x3o_tp2_r0/*" "glm53_vision/*"
38
+ ```
39
+
40
+ On the second Spark (rank 1):
41
+
42
+ ```bash
43
+ hf download Interactify/GLM-5.3-Flash-Slipstream --local-dir ~/vmodels \
44
+ --include "glm53c_tp2_r1/*" "x3o_tp2_r1/*"
45
+ ```
46
+
47
+ The tokenizer, chat template and generation config:
48
+
49
+ ```bash
50
+ hf download nvidia/GLM-5.3-Flash-NVFP4 --exclude "*.safetensors" # on the first Spark
51
+ ```
52
+
53
+ The directory names match the defaults in Slipstream's `server/models.json`, so no configuration is needed.
54
+
55
+ ## Run
56
+
57
+ ```bash
58
+ python server/slipstream.py serve glm-5.3-flash --profile balanced
59
+ ```
60
+
61
+ The server speaks the OpenAI and Anthropic APIs on port 8080. See the Slipstream README for profiles, the router and engine settings.
62
+
63
+ ## Measured on Slipstream
64
+
65
+ - Decode: 46-65 tok/s with MTP drafts (chat, code, JSON).
66
+ - Prefill: 1120 (0.9k) / 1680 (3.5k) / 1750 (13.8k) tokens/s.
67
+ - Experts: KL 0.053 against BF16 (the NVFP4 experts alone: 0.066).
68
+ - Standard profile: perplexity +0.7 % against the checkpoint, KL 0.046.
69
+ - tool-eval-bench: 91/100, median agent turn 3.4 s (92 scenarios).
70
+
71
+ ## Notes
72
+
73
+ - Context up to 1,048,576 tokens; Slipstream sizes the window to the memory of the placement.
74
+ - An optional DFlash2 drafter is supported by Slipstream but not included here (non-commercial licence; download it yourself).
75
+
76
+ ## License
77
+
78
+ MIT, as the base model (Z.ai) and NVIDIA's NVFP4 checkpoint it is built from. See `LICENSE` and `NOTICE`.
glm53_vision/index.json ADDED
@@ -0,0 +1,2930 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "config": {
3
+ "attention_bias": true,
4
+ "attention_dropout": 0.0,
5
+ "depth": 24,
6
+ "hidden_act": "silu",
7
+ "hidden_size": 1024,
8
+ "image_size": 448,
9
+ "in_channels": 3,
10
+ "initializer_range": 0.02,
11
+ "intermediate_size": 4096,
12
+ "model_type": "glm5_next_vision",
13
+ "num_heads": 16,
14
+ "out_hidden_size": 4096,
15
+ "patch_size": 14,
16
+ "projection_intermediate_size": 10240,
17
+ "rms_norm_eps": 1e-05,
18
+ "spatial_merge_size": 2,
19
+ "swiglu_limit": 10.0,
20
+ "temporal_patch_size": 2,
21
+ "image_token_id": 154854,
22
+ "image_start_token_id": 154830,
23
+ "image_end_token_id": 154831,
24
+ "rope_theta": 10000.0
25
+ },
26
+ "tensors": {
27
+ "patch.w": {
28
+ "kind": "f32",
29
+ "shape": [
30
+ 1024,
31
+ 1176
32
+ ],
33
+ "off": 0,
34
+ "bytes": 4816896
35
+ },
36
+ "patch.b": {
37
+ "kind": "f32",
38
+ "shape": [
39
+ 1024
40
+ ],
41
+ "off": 4816896,
42
+ "bytes": 4096
43
+ },
44
+ "B0.norm1": {
45
+ "kind": "f32",
46
+ "shape": [
47
+ 1024
48
+ ],
49
+ "off": 4820992,
50
+ "bytes": 4096
51
+ },
52
+ "B0.norm2": {
53
+ "kind": "f32",
54
+ "shape": [
55
+ 1024
56
+ ],
57
+ "off": 4825088,
58
+ "bytes": 4096
59
+ },
60
+ "B0.qkv.w": {
61
+ "kind": "bf16",
62
+ "shape": [
63
+ 3072,
64
+ 1024
65
+ ],
66
+ "off": 4829184,
67
+ "bytes": 6291456
68
+ },
69
+ "B0.qkv.b": {
70
+ "kind": "f32",
71
+ "shape": [
72
+ 3072
73
+ ],
74
+ "off": 11120640,
75
+ "bytes": 12288
76
+ },
77
+ "B0.proj.w": {
78
+ "kind": "bf16",
79
+ "shape": [
80
+ 1024,
81
+ 1024
82
+ ],
83
+ "off": 11132928,
84
+ "bytes": 2097152
85
+ },
86
+ "B0.proj.b": {
87
+ "kind": "f32",
88
+ "shape": [
89
+ 1024
90
+ ],
91
+ "off": 13230080,
92
+ "bytes": 4096
93
+ },
94
+ "B0.q_norm": {
95
+ "kind": "f32",
96
+ "shape": [
97
+ 64
98
+ ],
99
+ "off": 13234176,
100
+ "bytes": 256
101
+ },
102
+ "B0.k_norm": {
103
+ "kind": "f32",
104
+ "shape": [
105
+ 64
106
+ ],
107
+ "off": 13234432,
108
+ "bytes": 256
109
+ },
110
+ "B0.gate.w": {
111
+ "kind": "bf16",
112
+ "shape": [
113
+ 4096,
114
+ 1024
115
+ ],
116
+ "off": 13234688,
117
+ "bytes": 8388608
118
+ },
119
+ "B0.gate.b": {
120
+ "kind": "f32",
121
+ "shape": [
122
+ 4096
123
+ ],
124
+ "off": 21623296,
125
+ "bytes": 16384
126
+ },
127
+ "B0.up.w": {
128
+ "kind": "bf16",
129
+ "shape": [
130
+ 4096,
131
+ 1024
132
+ ],
133
+ "off": 21639680,
134
+ "bytes": 8388608
135
+ },
136
+ "B0.up.b": {
137
+ "kind": "f32",
138
+ "shape": [
139
+ 4096
140
+ ],
141
+ "off": 30028288,
142
+ "bytes": 16384
143
+ },
144
+ "B0.down.w": {
145
+ "kind": "bf16",
146
+ "shape": [
147
+ 1024,
148
+ 4096
149
+ ],
150
+ "off": 30044672,
151
+ "bytes": 8388608
152
+ },
153
+ "B0.down.b": {
154
+ "kind": "f32",
155
+ "shape": [
156
+ 1024
157
+ ],
158
+ "off": 38433280,
159
+ "bytes": 4096
160
+ },
161
+ "B1.norm1": {
162
+ "kind": "f32",
163
+ "shape": [
164
+ 1024
165
+ ],
166
+ "off": 38437376,
167
+ "bytes": 4096
168
+ },
169
+ "B1.norm2": {
170
+ "kind": "f32",
171
+ "shape": [
172
+ 1024
173
+ ],
174
+ "off": 38441472,
175
+ "bytes": 4096
176
+ },
177
+ "B1.qkv.w": {
178
+ "kind": "bf16",
179
+ "shape": [
180
+ 3072,
181
+ 1024
182
+ ],
183
+ "off": 38445568,
184
+ "bytes": 6291456
185
+ },
186
+ "B1.qkv.b": {
187
+ "kind": "f32",
188
+ "shape": [
189
+ 3072
190
+ ],
191
+ "off": 44737024,
192
+ "bytes": 12288
193
+ },
194
+ "B1.proj.w": {
195
+ "kind": "bf16",
196
+ "shape": [
197
+ 1024,
198
+ 1024
199
+ ],
200
+ "off": 44749312,
201
+ "bytes": 2097152
202
+ },
203
+ "B1.proj.b": {
204
+ "kind": "f32",
205
+ "shape": [
206
+ 1024
207
+ ],
208
+ "off": 46846464,
209
+ "bytes": 4096
210
+ },
211
+ "B1.q_norm": {
212
+ "kind": "f32",
213
+ "shape": [
214
+ 64
215
+ ],
216
+ "off": 46850560,
217
+ "bytes": 256
218
+ },
219
+ "B1.k_norm": {
220
+ "kind": "f32",
221
+ "shape": [
222
+ 64
223
+ ],
224
+ "off": 46850816,
225
+ "bytes": 256
226
+ },
227
+ "B1.gate.w": {
228
+ "kind": "bf16",
229
+ "shape": [
230
+ 4096,
231
+ 1024
232
+ ],
233
+ "off": 46851072,
234
+ "bytes": 8388608
235
+ },
236
+ "B1.gate.b": {
237
+ "kind": "f32",
238
+ "shape": [
239
+ 4096
240
+ ],
241
+ "off": 55239680,
242
+ "bytes": 16384
243
+ },
244
+ "B1.up.w": {
245
+ "kind": "bf16",
246
+ "shape": [
247
+ 4096,
248
+ 1024
249
+ ],
250
+ "off": 55256064,
251
+ "bytes": 8388608
252
+ },
253
+ "B1.up.b": {
254
+ "kind": "f32",
255
+ "shape": [
256
+ 4096
257
+ ],
258
+ "off": 63644672,
259
+ "bytes": 16384
260
+ },
261
+ "B1.down.w": {
262
+ "kind": "bf16",
263
+ "shape": [
264
+ 1024,
265
+ 4096
266
+ ],
267
+ "off": 63661056,
268
+ "bytes": 8388608
269
+ },
270
+ "B1.down.b": {
271
+ "kind": "f32",
272
+ "shape": [
273
+ 1024
274
+ ],
275
+ "off": 72049664,
276
+ "bytes": 4096
277
+ },
278
+ "B2.norm1": {
279
+ "kind": "f32",
280
+ "shape": [
281
+ 1024
282
+ ],
283
+ "off": 72053760,
284
+ "bytes": 4096
285
+ },
286
+ "B2.norm2": {
287
+ "kind": "f32",
288
+ "shape": [
289
+ 1024
290
+ ],
291
+ "off": 72057856,
292
+ "bytes": 4096
293
+ },
294
+ "B2.qkv.w": {
295
+ "kind": "bf16",
296
+ "shape": [
297
+ 3072,
298
+ 1024
299
+ ],
300
+ "off": 72061952,
301
+ "bytes": 6291456
302
+ },
303
+ "B2.qkv.b": {
304
+ "kind": "f32",
305
+ "shape": [
306
+ 3072
307
+ ],
308
+ "off": 78353408,
309
+ "bytes": 12288
310
+ },
311
+ "B2.proj.w": {
312
+ "kind": "bf16",
313
+ "shape": [
314
+ 1024,
315
+ 1024
316
+ ],
317
+ "off": 78365696,
318
+ "bytes": 2097152
319
+ },
320
+ "B2.proj.b": {
321
+ "kind": "f32",
322
+ "shape": [
323
+ 1024
324
+ ],
325
+ "off": 80462848,
326
+ "bytes": 4096
327
+ },
328
+ "B2.q_norm": {
329
+ "kind": "f32",
330
+ "shape": [
331
+ 64
332
+ ],
333
+ "off": 80466944,
334
+ "bytes": 256
335
+ },
336
+ "B2.k_norm": {
337
+ "kind": "f32",
338
+ "shape": [
339
+ 64
340
+ ],
341
+ "off": 80467200,
342
+ "bytes": 256
343
+ },
344
+ "B2.gate.w": {
345
+ "kind": "bf16",
346
+ "shape": [
347
+ 4096,
348
+ 1024
349
+ ],
350
+ "off": 80467456,
351
+ "bytes": 8388608
352
+ },
353
+ "B2.gate.b": {
354
+ "kind": "f32",
355
+ "shape": [
356
+ 4096
357
+ ],
358
+ "off": 88856064,
359
+ "bytes": 16384
360
+ },
361
+ "B2.up.w": {
362
+ "kind": "bf16",
363
+ "shape": [
364
+ 4096,
365
+ 1024
366
+ ],
367
+ "off": 88872448,
368
+ "bytes": 8388608
369
+ },
370
+ "B2.up.b": {
371
+ "kind": "f32",
372
+ "shape": [
373
+ 4096
374
+ ],
375
+ "off": 97261056,
376
+ "bytes": 16384
377
+ },
378
+ "B2.down.w": {
379
+ "kind": "bf16",
380
+ "shape": [
381
+ 1024,
382
+ 4096
383
+ ],
384
+ "off": 97277440,
385
+ "bytes": 8388608
386
+ },
387
+ "B2.down.b": {
388
+ "kind": "f32",
389
+ "shape": [
390
+ 1024
391
+ ],
392
+ "off": 105666048,
393
+ "bytes": 4096
394
+ },
395
+ "B3.norm1": {
396
+ "kind": "f32",
397
+ "shape": [
398
+ 1024
399
+ ],
400
+ "off": 105670144,
401
+ "bytes": 4096
402
+ },
403
+ "B3.norm2": {
404
+ "kind": "f32",
405
+ "shape": [
406
+ 1024
407
+ ],
408
+ "off": 105674240,
409
+ "bytes": 4096
410
+ },
411
+ "B3.qkv.w": {
412
+ "kind": "bf16",
413
+ "shape": [
414
+ 3072,
415
+ 1024
416
+ ],
417
+ "off": 105678336,
418
+ "bytes": 6291456
419
+ },
420
+ "B3.qkv.b": {
421
+ "kind": "f32",
422
+ "shape": [
423
+ 3072
424
+ ],
425
+ "off": 111969792,
426
+ "bytes": 12288
427
+ },
428
+ "B3.proj.w": {
429
+ "kind": "bf16",
430
+ "shape": [
431
+ 1024,
432
+ 1024
433
+ ],
434
+ "off": 111982080,
435
+ "bytes": 2097152
436
+ },
437
+ "B3.proj.b": {
438
+ "kind": "f32",
439
+ "shape": [
440
+ 1024
441
+ ],
442
+ "off": 114079232,
443
+ "bytes": 4096
444
+ },
445
+ "B3.q_norm": {
446
+ "kind": "f32",
447
+ "shape": [
448
+ 64
449
+ ],
450
+ "off": 114083328,
451
+ "bytes": 256
452
+ },
453
+ "B3.k_norm": {
454
+ "kind": "f32",
455
+ "shape": [
456
+ 64
457
+ ],
458
+ "off": 114083584,
459
+ "bytes": 256
460
+ },
461
+ "B3.gate.w": {
462
+ "kind": "bf16",
463
+ "shape": [
464
+ 4096,
465
+ 1024
466
+ ],
467
+ "off": 114083840,
468
+ "bytes": 8388608
469
+ },
470
+ "B3.gate.b": {
471
+ "kind": "f32",
472
+ "shape": [
473
+ 4096
474
+ ],
475
+ "off": 122472448,
476
+ "bytes": 16384
477
+ },
478
+ "B3.up.w": {
479
+ "kind": "bf16",
480
+ "shape": [
481
+ 4096,
482
+ 1024
483
+ ],
484
+ "off": 122488832,
485
+ "bytes": 8388608
486
+ },
487
+ "B3.up.b": {
488
+ "kind": "f32",
489
+ "shape": [
490
+ 4096
491
+ ],
492
+ "off": 130877440,
493
+ "bytes": 16384
494
+ },
495
+ "B3.down.w": {
496
+ "kind": "bf16",
497
+ "shape": [
498
+ 1024,
499
+ 4096
500
+ ],
501
+ "off": 130893824,
502
+ "bytes": 8388608
503
+ },
504
+ "B3.down.b": {
505
+ "kind": "f32",
506
+ "shape": [
507
+ 1024
508
+ ],
509
+ "off": 139282432,
510
+ "bytes": 4096
511
+ },
512
+ "B4.norm1": {
513
+ "kind": "f32",
514
+ "shape": [
515
+ 1024
516
+ ],
517
+ "off": 139286528,
518
+ "bytes": 4096
519
+ },
520
+ "B4.norm2": {
521
+ "kind": "f32",
522
+ "shape": [
523
+ 1024
524
+ ],
525
+ "off": 139290624,
526
+ "bytes": 4096
527
+ },
528
+ "B4.qkv.w": {
529
+ "kind": "bf16",
530
+ "shape": [
531
+ 3072,
532
+ 1024
533
+ ],
534
+ "off": 139294720,
535
+ "bytes": 6291456
536
+ },
537
+ "B4.qkv.b": {
538
+ "kind": "f32",
539
+ "shape": [
540
+ 3072
541
+ ],
542
+ "off": 145586176,
543
+ "bytes": 12288
544
+ },
545
+ "B4.proj.w": {
546
+ "kind": "bf16",
547
+ "shape": [
548
+ 1024,
549
+ 1024
550
+ ],
551
+ "off": 145598464,
552
+ "bytes": 2097152
553
+ },
554
+ "B4.proj.b": {
555
+ "kind": "f32",
556
+ "shape": [
557
+ 1024
558
+ ],
559
+ "off": 147695616,
560
+ "bytes": 4096
561
+ },
562
+ "B4.q_norm": {
563
+ "kind": "f32",
564
+ "shape": [
565
+ 64
566
+ ],
567
+ "off": 147699712,
568
+ "bytes": 256
569
+ },
570
+ "B4.k_norm": {
571
+ "kind": "f32",
572
+ "shape": [
573
+ 64
574
+ ],
575
+ "off": 147699968,
576
+ "bytes": 256
577
+ },
578
+ "B4.gate.w": {
579
+ "kind": "bf16",
580
+ "shape": [
581
+ 4096,
582
+ 1024
583
+ ],
584
+ "off": 147700224,
585
+ "bytes": 8388608
586
+ },
587
+ "B4.gate.b": {
588
+ "kind": "f32",
589
+ "shape": [
590
+ 4096
591
+ ],
592
+ "off": 156088832,
593
+ "bytes": 16384
594
+ },
595
+ "B4.up.w": {
596
+ "kind": "bf16",
597
+ "shape": [
598
+ 4096,
599
+ 1024
600
+ ],
601
+ "off": 156105216,
602
+ "bytes": 8388608
603
+ },
604
+ "B4.up.b": {
605
+ "kind": "f32",
606
+ "shape": [
607
+ 4096
608
+ ],
609
+ "off": 164493824,
610
+ "bytes": 16384
611
+ },
612
+ "B4.down.w": {
613
+ "kind": "bf16",
614
+ "shape": [
615
+ 1024,
616
+ 4096
617
+ ],
618
+ "off": 164510208,
619
+ "bytes": 8388608
620
+ },
621
+ "B4.down.b": {
622
+ "kind": "f32",
623
+ "shape": [
624
+ 1024
625
+ ],
626
+ "off": 172898816,
627
+ "bytes": 4096
628
+ },
629
+ "B5.norm1": {
630
+ "kind": "f32",
631
+ "shape": [
632
+ 1024
633
+ ],
634
+ "off": 172902912,
635
+ "bytes": 4096
636
+ },
637
+ "B5.norm2": {
638
+ "kind": "f32",
639
+ "shape": [
640
+ 1024
641
+ ],
642
+ "off": 172907008,
643
+ "bytes": 4096
644
+ },
645
+ "B5.qkv.w": {
646
+ "kind": "bf16",
647
+ "shape": [
648
+ 3072,
649
+ 1024
650
+ ],
651
+ "off": 172911104,
652
+ "bytes": 6291456
653
+ },
654
+ "B5.qkv.b": {
655
+ "kind": "f32",
656
+ "shape": [
657
+ 3072
658
+ ],
659
+ "off": 179202560,
660
+ "bytes": 12288
661
+ },
662
+ "B5.proj.w": {
663
+ "kind": "bf16",
664
+ "shape": [
665
+ 1024,
666
+ 1024
667
+ ],
668
+ "off": 179214848,
669
+ "bytes": 2097152
670
+ },
671
+ "B5.proj.b": {
672
+ "kind": "f32",
673
+ "shape": [
674
+ 1024
675
+ ],
676
+ "off": 181312000,
677
+ "bytes": 4096
678
+ },
679
+ "B5.q_norm": {
680
+ "kind": "f32",
681
+ "shape": [
682
+ 64
683
+ ],
684
+ "off": 181316096,
685
+ "bytes": 256
686
+ },
687
+ "B5.k_norm": {
688
+ "kind": "f32",
689
+ "shape": [
690
+ 64
691
+ ],
692
+ "off": 181316352,
693
+ "bytes": 256
694
+ },
695
+ "B5.gate.w": {
696
+ "kind": "bf16",
697
+ "shape": [
698
+ 4096,
699
+ 1024
700
+ ],
701
+ "off": 181316608,
702
+ "bytes": 8388608
703
+ },
704
+ "B5.gate.b": {
705
+ "kind": "f32",
706
+ "shape": [
707
+ 4096
708
+ ],
709
+ "off": 189705216,
710
+ "bytes": 16384
711
+ },
712
+ "B5.up.w": {
713
+ "kind": "bf16",
714
+ "shape": [
715
+ 4096,
716
+ 1024
717
+ ],
718
+ "off": 189721600,
719
+ "bytes": 8388608
720
+ },
721
+ "B5.up.b": {
722
+ "kind": "f32",
723
+ "shape": [
724
+ 4096
725
+ ],
726
+ "off": 198110208,
727
+ "bytes": 16384
728
+ },
729
+ "B5.down.w": {
730
+ "kind": "bf16",
731
+ "shape": [
732
+ 1024,
733
+ 4096
734
+ ],
735
+ "off": 198126592,
736
+ "bytes": 8388608
737
+ },
738
+ "B5.down.b": {
739
+ "kind": "f32",
740
+ "shape": [
741
+ 1024
742
+ ],
743
+ "off": 206515200,
744
+ "bytes": 4096
745
+ },
746
+ "B6.norm1": {
747
+ "kind": "f32",
748
+ "shape": [
749
+ 1024
750
+ ],
751
+ "off": 206519296,
752
+ "bytes": 4096
753
+ },
754
+ "B6.norm2": {
755
+ "kind": "f32",
756
+ "shape": [
757
+ 1024
758
+ ],
759
+ "off": 206523392,
760
+ "bytes": 4096
761
+ },
762
+ "B6.qkv.w": {
763
+ "kind": "bf16",
764
+ "shape": [
765
+ 3072,
766
+ 1024
767
+ ],
768
+ "off": 206527488,
769
+ "bytes": 6291456
770
+ },
771
+ "B6.qkv.b": {
772
+ "kind": "f32",
773
+ "shape": [
774
+ 3072
775
+ ],
776
+ "off": 212818944,
777
+ "bytes": 12288
778
+ },
779
+ "B6.proj.w": {
780
+ "kind": "bf16",
781
+ "shape": [
782
+ 1024,
783
+ 1024
784
+ ],
785
+ "off": 212831232,
786
+ "bytes": 2097152
787
+ },
788
+ "B6.proj.b": {
789
+ "kind": "f32",
790
+ "shape": [
791
+ 1024
792
+ ],
793
+ "off": 214928384,
794
+ "bytes": 4096
795
+ },
796
+ "B6.q_norm": {
797
+ "kind": "f32",
798
+ "shape": [
799
+ 64
800
+ ],
801
+ "off": 214932480,
802
+ "bytes": 256
803
+ },
804
+ "B6.k_norm": {
805
+ "kind": "f32",
806
+ "shape": [
807
+ 64
808
+ ],
809
+ "off": 214932736,
810
+ "bytes": 256
811
+ },
812
+ "B6.gate.w": {
813
+ "kind": "bf16",
814
+ "shape": [
815
+ 4096,
816
+ 1024
817
+ ],
818
+ "off": 214932992,
819
+ "bytes": 8388608
820
+ },
821
+ "B6.gate.b": {
822
+ "kind": "f32",
823
+ "shape": [
824
+ 4096
825
+ ],
826
+ "off": 223321600,
827
+ "bytes": 16384
828
+ },
829
+ "B6.up.w": {
830
+ "kind": "bf16",
831
+ "shape": [
832
+ 4096,
833
+ 1024
834
+ ],
835
+ "off": 223337984,
836
+ "bytes": 8388608
837
+ },
838
+ "B6.up.b": {
839
+ "kind": "f32",
840
+ "shape": [
841
+ 4096
842
+ ],
843
+ "off": 231726592,
844
+ "bytes": 16384
845
+ },
846
+ "B6.down.w": {
847
+ "kind": "bf16",
848
+ "shape": [
849
+ 1024,
850
+ 4096
851
+ ],
852
+ "off": 231742976,
853
+ "bytes": 8388608
854
+ },
855
+ "B6.down.b": {
856
+ "kind": "f32",
857
+ "shape": [
858
+ 1024
859
+ ],
860
+ "off": 240131584,
861
+ "bytes": 4096
862
+ },
863
+ "B7.norm1": {
864
+ "kind": "f32",
865
+ "shape": [
866
+ 1024
867
+ ],
868
+ "off": 240135680,
869
+ "bytes": 4096
870
+ },
871
+ "B7.norm2": {
872
+ "kind": "f32",
873
+ "shape": [
874
+ 1024
875
+ ],
876
+ "off": 240139776,
877
+ "bytes": 4096
878
+ },
879
+ "B7.qkv.w": {
880
+ "kind": "bf16",
881
+ "shape": [
882
+ 3072,
883
+ 1024
884
+ ],
885
+ "off": 240143872,
886
+ "bytes": 6291456
887
+ },
888
+ "B7.qkv.b": {
889
+ "kind": "f32",
890
+ "shape": [
891
+ 3072
892
+ ],
893
+ "off": 246435328,
894
+ "bytes": 12288
895
+ },
896
+ "B7.proj.w": {
897
+ "kind": "bf16",
898
+ "shape": [
899
+ 1024,
900
+ 1024
901
+ ],
902
+ "off": 246447616,
903
+ "bytes": 2097152
904
+ },
905
+ "B7.proj.b": {
906
+ "kind": "f32",
907
+ "shape": [
908
+ 1024
909
+ ],
910
+ "off": 248544768,
911
+ "bytes": 4096
912
+ },
913
+ "B7.q_norm": {
914
+ "kind": "f32",
915
+ "shape": [
916
+ 64
917
+ ],
918
+ "off": 248548864,
919
+ "bytes": 256
920
+ },
921
+ "B7.k_norm": {
922
+ "kind": "f32",
923
+ "shape": [
924
+ 64
925
+ ],
926
+ "off": 248549120,
927
+ "bytes": 256
928
+ },
929
+ "B7.gate.w": {
930
+ "kind": "bf16",
931
+ "shape": [
932
+ 4096,
933
+ 1024
934
+ ],
935
+ "off": 248549376,
936
+ "bytes": 8388608
937
+ },
938
+ "B7.gate.b": {
939
+ "kind": "f32",
940
+ "shape": [
941
+ 4096
942
+ ],
943
+ "off": 256937984,
944
+ "bytes": 16384
945
+ },
946
+ "B7.up.w": {
947
+ "kind": "bf16",
948
+ "shape": [
949
+ 4096,
950
+ 1024
951
+ ],
952
+ "off": 256954368,
953
+ "bytes": 8388608
954
+ },
955
+ "B7.up.b": {
956
+ "kind": "f32",
957
+ "shape": [
958
+ 4096
959
+ ],
960
+ "off": 265342976,
961
+ "bytes": 16384
962
+ },
963
+ "B7.down.w": {
964
+ "kind": "bf16",
965
+ "shape": [
966
+ 1024,
967
+ 4096
968
+ ],
969
+ "off": 265359360,
970
+ "bytes": 8388608
971
+ },
972
+ "B7.down.b": {
973
+ "kind": "f32",
974
+ "shape": [
975
+ 1024
976
+ ],
977
+ "off": 273747968,
978
+ "bytes": 4096
979
+ },
980
+ "B8.norm1": {
981
+ "kind": "f32",
982
+ "shape": [
983
+ 1024
984
+ ],
985
+ "off": 273752064,
986
+ "bytes": 4096
987
+ },
988
+ "B8.norm2": {
989
+ "kind": "f32",
990
+ "shape": [
991
+ 1024
992
+ ],
993
+ "off": 273756160,
994
+ "bytes": 4096
995
+ },
996
+ "B8.qkv.w": {
997
+ "kind": "bf16",
998
+ "shape": [
999
+ 3072,
1000
+ 1024
1001
+ ],
1002
+ "off": 273760256,
1003
+ "bytes": 6291456
1004
+ },
1005
+ "B8.qkv.b": {
1006
+ "kind": "f32",
1007
+ "shape": [
1008
+ 3072
1009
+ ],
1010
+ "off": 280051712,
1011
+ "bytes": 12288
1012
+ },
1013
+ "B8.proj.w": {
1014
+ "kind": "bf16",
1015
+ "shape": [
1016
+ 1024,
1017
+ 1024
1018
+ ],
1019
+ "off": 280064000,
1020
+ "bytes": 2097152
1021
+ },
1022
+ "B8.proj.b": {
1023
+ "kind": "f32",
1024
+ "shape": [
1025
+ 1024
1026
+ ],
1027
+ "off": 282161152,
1028
+ "bytes": 4096
1029
+ },
1030
+ "B8.q_norm": {
1031
+ "kind": "f32",
1032
+ "shape": [
1033
+ 64
1034
+ ],
1035
+ "off": 282165248,
1036
+ "bytes": 256
1037
+ },
1038
+ "B8.k_norm": {
1039
+ "kind": "f32",
1040
+ "shape": [
1041
+ 64
1042
+ ],
1043
+ "off": 282165504,
1044
+ "bytes": 256
1045
+ },
1046
+ "B8.gate.w": {
1047
+ "kind": "bf16",
1048
+ "shape": [
1049
+ 4096,
1050
+ 1024
1051
+ ],
1052
+ "off": 282165760,
1053
+ "bytes": 8388608
1054
+ },
1055
+ "B8.gate.b": {
1056
+ "kind": "f32",
1057
+ "shape": [
1058
+ 4096
1059
+ ],
1060
+ "off": 290554368,
1061
+ "bytes": 16384
1062
+ },
1063
+ "B8.up.w": {
1064
+ "kind": "bf16",
1065
+ "shape": [
1066
+ 4096,
1067
+ 1024
1068
+ ],
1069
+ "off": 290570752,
1070
+ "bytes": 8388608
1071
+ },
1072
+ "B8.up.b": {
1073
+ "kind": "f32",
1074
+ "shape": [
1075
+ 4096
1076
+ ],
1077
+ "off": 298959360,
1078
+ "bytes": 16384
1079
+ },
1080
+ "B8.down.w": {
1081
+ "kind": "bf16",
1082
+ "shape": [
1083
+ 1024,
1084
+ 4096
1085
+ ],
1086
+ "off": 298975744,
1087
+ "bytes": 8388608
1088
+ },
1089
+ "B8.down.b": {
1090
+ "kind": "f32",
1091
+ "shape": [
1092
+ 1024
1093
+ ],
1094
+ "off": 307364352,
1095
+ "bytes": 4096
1096
+ },
1097
+ "B9.norm1": {
1098
+ "kind": "f32",
1099
+ "shape": [
1100
+ 1024
1101
+ ],
1102
+ "off": 307368448,
1103
+ "bytes": 4096
1104
+ },
1105
+ "B9.norm2": {
1106
+ "kind": "f32",
1107
+ "shape": [
1108
+ 1024
1109
+ ],
1110
+ "off": 307372544,
1111
+ "bytes": 4096
1112
+ },
1113
+ "B9.qkv.w": {
1114
+ "kind": "bf16",
1115
+ "shape": [
1116
+ 3072,
1117
+ 1024
1118
+ ],
1119
+ "off": 307376640,
1120
+ "bytes": 6291456
1121
+ },
1122
+ "B9.qkv.b": {
1123
+ "kind": "f32",
1124
+ "shape": [
1125
+ 3072
1126
+ ],
1127
+ "off": 313668096,
1128
+ "bytes": 12288
1129
+ },
1130
+ "B9.proj.w": {
1131
+ "kind": "bf16",
1132
+ "shape": [
1133
+ 1024,
1134
+ 1024
1135
+ ],
1136
+ "off": 313680384,
1137
+ "bytes": 2097152
1138
+ },
1139
+ "B9.proj.b": {
1140
+ "kind": "f32",
1141
+ "shape": [
1142
+ 1024
1143
+ ],
1144
+ "off": 315777536,
1145
+ "bytes": 4096
1146
+ },
1147
+ "B9.q_norm": {
1148
+ "kind": "f32",
1149
+ "shape": [
1150
+ 64
1151
+ ],
1152
+ "off": 315781632,
1153
+ "bytes": 256
1154
+ },
1155
+ "B9.k_norm": {
1156
+ "kind": "f32",
1157
+ "shape": [
1158
+ 64
1159
+ ],
1160
+ "off": 315781888,
1161
+ "bytes": 256
1162
+ },
1163
+ "B9.gate.w": {
1164
+ "kind": "bf16",
1165
+ "shape": [
1166
+ 4096,
1167
+ 1024
1168
+ ],
1169
+ "off": 315782144,
1170
+ "bytes": 8388608
1171
+ },
1172
+ "B9.gate.b": {
1173
+ "kind": "f32",
1174
+ "shape": [
1175
+ 4096
1176
+ ],
1177
+ "off": 324170752,
1178
+ "bytes": 16384
1179
+ },
1180
+ "B9.up.w": {
1181
+ "kind": "bf16",
1182
+ "shape": [
1183
+ 4096,
1184
+ 1024
1185
+ ],
1186
+ "off": 324187136,
1187
+ "bytes": 8388608
1188
+ },
1189
+ "B9.up.b": {
1190
+ "kind": "f32",
1191
+ "shape": [
1192
+ 4096
1193
+ ],
1194
+ "off": 332575744,
1195
+ "bytes": 16384
1196
+ },
1197
+ "B9.down.w": {
1198
+ "kind": "bf16",
1199
+ "shape": [
1200
+ 1024,
1201
+ 4096
1202
+ ],
1203
+ "off": 332592128,
1204
+ "bytes": 8388608
1205
+ },
1206
+ "B9.down.b": {
1207
+ "kind": "f32",
1208
+ "shape": [
1209
+ 1024
1210
+ ],
1211
+ "off": 340980736,
1212
+ "bytes": 4096
1213
+ },
1214
+ "B10.norm1": {
1215
+ "kind": "f32",
1216
+ "shape": [
1217
+ 1024
1218
+ ],
1219
+ "off": 340984832,
1220
+ "bytes": 4096
1221
+ },
1222
+ "B10.norm2": {
1223
+ "kind": "f32",
1224
+ "shape": [
1225
+ 1024
1226
+ ],
1227
+ "off": 340988928,
1228
+ "bytes": 4096
1229
+ },
1230
+ "B10.qkv.w": {
1231
+ "kind": "bf16",
1232
+ "shape": [
1233
+ 3072,
1234
+ 1024
1235
+ ],
1236
+ "off": 340993024,
1237
+ "bytes": 6291456
1238
+ },
1239
+ "B10.qkv.b": {
1240
+ "kind": "f32",
1241
+ "shape": [
1242
+ 3072
1243
+ ],
1244
+ "off": 347284480,
1245
+ "bytes": 12288
1246
+ },
1247
+ "B10.proj.w": {
1248
+ "kind": "bf16",
1249
+ "shape": [
1250
+ 1024,
1251
+ 1024
1252
+ ],
1253
+ "off": 347296768,
1254
+ "bytes": 2097152
1255
+ },
1256
+ "B10.proj.b": {
1257
+ "kind": "f32",
1258
+ "shape": [
1259
+ 1024
1260
+ ],
1261
+ "off": 349393920,
1262
+ "bytes": 4096
1263
+ },
1264
+ "B10.q_norm": {
1265
+ "kind": "f32",
1266
+ "shape": [
1267
+ 64
1268
+ ],
1269
+ "off": 349398016,
1270
+ "bytes": 256
1271
+ },
1272
+ "B10.k_norm": {
1273
+ "kind": "f32",
1274
+ "shape": [
1275
+ 64
1276
+ ],
1277
+ "off": 349398272,
1278
+ "bytes": 256
1279
+ },
1280
+ "B10.gate.w": {
1281
+ "kind": "bf16",
1282
+ "shape": [
1283
+ 4096,
1284
+ 1024
1285
+ ],
1286
+ "off": 349398528,
1287
+ "bytes": 8388608
1288
+ },
1289
+ "B10.gate.b": {
1290
+ "kind": "f32",
1291
+ "shape": [
1292
+ 4096
1293
+ ],
1294
+ "off": 357787136,
1295
+ "bytes": 16384
1296
+ },
1297
+ "B10.up.w": {
1298
+ "kind": "bf16",
1299
+ "shape": [
1300
+ 4096,
1301
+ 1024
1302
+ ],
1303
+ "off": 357803520,
1304
+ "bytes": 8388608
1305
+ },
1306
+ "B10.up.b": {
1307
+ "kind": "f32",
1308
+ "shape": [
1309
+ 4096
1310
+ ],
1311
+ "off": 366192128,
1312
+ "bytes": 16384
1313
+ },
1314
+ "B10.down.w": {
1315
+ "kind": "bf16",
1316
+ "shape": [
1317
+ 1024,
1318
+ 4096
1319
+ ],
1320
+ "off": 366208512,
1321
+ "bytes": 8388608
1322
+ },
1323
+ "B10.down.b": {
1324
+ "kind": "f32",
1325
+ "shape": [
1326
+ 1024
1327
+ ],
1328
+ "off": 374597120,
1329
+ "bytes": 4096
1330
+ },
1331
+ "B11.norm1": {
1332
+ "kind": "f32",
1333
+ "shape": [
1334
+ 1024
1335
+ ],
1336
+ "off": 374601216,
1337
+ "bytes": 4096
1338
+ },
1339
+ "B11.norm2": {
1340
+ "kind": "f32",
1341
+ "shape": [
1342
+ 1024
1343
+ ],
1344
+ "off": 374605312,
1345
+ "bytes": 4096
1346
+ },
1347
+ "B11.qkv.w": {
1348
+ "kind": "bf16",
1349
+ "shape": [
1350
+ 3072,
1351
+ 1024
1352
+ ],
1353
+ "off": 374609408,
1354
+ "bytes": 6291456
1355
+ },
1356
+ "B11.qkv.b": {
1357
+ "kind": "f32",
1358
+ "shape": [
1359
+ 3072
1360
+ ],
1361
+ "off": 380900864,
1362
+ "bytes": 12288
1363
+ },
1364
+ "B11.proj.w": {
1365
+ "kind": "bf16",
1366
+ "shape": [
1367
+ 1024,
1368
+ 1024
1369
+ ],
1370
+ "off": 380913152,
1371
+ "bytes": 2097152
1372
+ },
1373
+ "B11.proj.b": {
1374
+ "kind": "f32",
1375
+ "shape": [
1376
+ 1024
1377
+ ],
1378
+ "off": 383010304,
1379
+ "bytes": 4096
1380
+ },
1381
+ "B11.q_norm": {
1382
+ "kind": "f32",
1383
+ "shape": [
1384
+ 64
1385
+ ],
1386
+ "off": 383014400,
1387
+ "bytes": 256
1388
+ },
1389
+ "B11.k_norm": {
1390
+ "kind": "f32",
1391
+ "shape": [
1392
+ 64
1393
+ ],
1394
+ "off": 383014656,
1395
+ "bytes": 256
1396
+ },
1397
+ "B11.gate.w": {
1398
+ "kind": "bf16",
1399
+ "shape": [
1400
+ 4096,
1401
+ 1024
1402
+ ],
1403
+ "off": 383014912,
1404
+ "bytes": 8388608
1405
+ },
1406
+ "B11.gate.b": {
1407
+ "kind": "f32",
1408
+ "shape": [
1409
+ 4096
1410
+ ],
1411
+ "off": 391403520,
1412
+ "bytes": 16384
1413
+ },
1414
+ "B11.up.w": {
1415
+ "kind": "bf16",
1416
+ "shape": [
1417
+ 4096,
1418
+ 1024
1419
+ ],
1420
+ "off": 391419904,
1421
+ "bytes": 8388608
1422
+ },
1423
+ "B11.up.b": {
1424
+ "kind": "f32",
1425
+ "shape": [
1426
+ 4096
1427
+ ],
1428
+ "off": 399808512,
1429
+ "bytes": 16384
1430
+ },
1431
+ "B11.down.w": {
1432
+ "kind": "bf16",
1433
+ "shape": [
1434
+ 1024,
1435
+ 4096
1436
+ ],
1437
+ "off": 399824896,
1438
+ "bytes": 8388608
1439
+ },
1440
+ "B11.down.b": {
1441
+ "kind": "f32",
1442
+ "shape": [
1443
+ 1024
1444
+ ],
1445
+ "off": 408213504,
1446
+ "bytes": 4096
1447
+ },
1448
+ "B12.norm1": {
1449
+ "kind": "f32",
1450
+ "shape": [
1451
+ 1024
1452
+ ],
1453
+ "off": 408217600,
1454
+ "bytes": 4096
1455
+ },
1456
+ "B12.norm2": {
1457
+ "kind": "f32",
1458
+ "shape": [
1459
+ 1024
1460
+ ],
1461
+ "off": 408221696,
1462
+ "bytes": 4096
1463
+ },
1464
+ "B12.qkv.w": {
1465
+ "kind": "bf16",
1466
+ "shape": [
1467
+ 3072,
1468
+ 1024
1469
+ ],
1470
+ "off": 408225792,
1471
+ "bytes": 6291456
1472
+ },
1473
+ "B12.qkv.b": {
1474
+ "kind": "f32",
1475
+ "shape": [
1476
+ 3072
1477
+ ],
1478
+ "off": 414517248,
1479
+ "bytes": 12288
1480
+ },
1481
+ "B12.proj.w": {
1482
+ "kind": "bf16",
1483
+ "shape": [
1484
+ 1024,
1485
+ 1024
1486
+ ],
1487
+ "off": 414529536,
1488
+ "bytes": 2097152
1489
+ },
1490
+ "B12.proj.b": {
1491
+ "kind": "f32",
1492
+ "shape": [
1493
+ 1024
1494
+ ],
1495
+ "off": 416626688,
1496
+ "bytes": 4096
1497
+ },
1498
+ "B12.q_norm": {
1499
+ "kind": "f32",
1500
+ "shape": [
1501
+ 64
1502
+ ],
1503
+ "off": 416630784,
1504
+ "bytes": 256
1505
+ },
1506
+ "B12.k_norm": {
1507
+ "kind": "f32",
1508
+ "shape": [
1509
+ 64
1510
+ ],
1511
+ "off": 416631040,
1512
+ "bytes": 256
1513
+ },
1514
+ "B12.gate.w": {
1515
+ "kind": "bf16",
1516
+ "shape": [
1517
+ 4096,
1518
+ 1024
1519
+ ],
1520
+ "off": 416631296,
1521
+ "bytes": 8388608
1522
+ },
1523
+ "B12.gate.b": {
1524
+ "kind": "f32",
1525
+ "shape": [
1526
+ 4096
1527
+ ],
1528
+ "off": 425019904,
1529
+ "bytes": 16384
1530
+ },
1531
+ "B12.up.w": {
1532
+ "kind": "bf16",
1533
+ "shape": [
1534
+ 4096,
1535
+ 1024
1536
+ ],
1537
+ "off": 425036288,
1538
+ "bytes": 8388608
1539
+ },
1540
+ "B12.up.b": {
1541
+ "kind": "f32",
1542
+ "shape": [
1543
+ 4096
1544
+ ],
1545
+ "off": 433424896,
1546
+ "bytes": 16384
1547
+ },
1548
+ "B12.down.w": {
1549
+ "kind": "bf16",
1550
+ "shape": [
1551
+ 1024,
1552
+ 4096
1553
+ ],
1554
+ "off": 433441280,
1555
+ "bytes": 8388608
1556
+ },
1557
+ "B12.down.b": {
1558
+ "kind": "f32",
1559
+ "shape": [
1560
+ 1024
1561
+ ],
1562
+ "off": 441829888,
1563
+ "bytes": 4096
1564
+ },
1565
+ "B13.norm1": {
1566
+ "kind": "f32",
1567
+ "shape": [
1568
+ 1024
1569
+ ],
1570
+ "off": 441833984,
1571
+ "bytes": 4096
1572
+ },
1573
+ "B13.norm2": {
1574
+ "kind": "f32",
1575
+ "shape": [
1576
+ 1024
1577
+ ],
1578
+ "off": 441838080,
1579
+ "bytes": 4096
1580
+ },
1581
+ "B13.qkv.w": {
1582
+ "kind": "bf16",
1583
+ "shape": [
1584
+ 3072,
1585
+ 1024
1586
+ ],
1587
+ "off": 441842176,
1588
+ "bytes": 6291456
1589
+ },
1590
+ "B13.qkv.b": {
1591
+ "kind": "f32",
1592
+ "shape": [
1593
+ 3072
1594
+ ],
1595
+ "off": 448133632,
1596
+ "bytes": 12288
1597
+ },
1598
+ "B13.proj.w": {
1599
+ "kind": "bf16",
1600
+ "shape": [
1601
+ 1024,
1602
+ 1024
1603
+ ],
1604
+ "off": 448145920,
1605
+ "bytes": 2097152
1606
+ },
1607
+ "B13.proj.b": {
1608
+ "kind": "f32",
1609
+ "shape": [
1610
+ 1024
1611
+ ],
1612
+ "off": 450243072,
1613
+ "bytes": 4096
1614
+ },
1615
+ "B13.q_norm": {
1616
+ "kind": "f32",
1617
+ "shape": [
1618
+ 64
1619
+ ],
1620
+ "off": 450247168,
1621
+ "bytes": 256
1622
+ },
1623
+ "B13.k_norm": {
1624
+ "kind": "f32",
1625
+ "shape": [
1626
+ 64
1627
+ ],
1628
+ "off": 450247424,
1629
+ "bytes": 256
1630
+ },
1631
+ "B13.gate.w": {
1632
+ "kind": "bf16",
1633
+ "shape": [
1634
+ 4096,
1635
+ 1024
1636
+ ],
1637
+ "off": 450247680,
1638
+ "bytes": 8388608
1639
+ },
1640
+ "B13.gate.b": {
1641
+ "kind": "f32",
1642
+ "shape": [
1643
+ 4096
1644
+ ],
1645
+ "off": 458636288,
1646
+ "bytes": 16384
1647
+ },
1648
+ "B13.up.w": {
1649
+ "kind": "bf16",
1650
+ "shape": [
1651
+ 4096,
1652
+ 1024
1653
+ ],
1654
+ "off": 458652672,
1655
+ "bytes": 8388608
1656
+ },
1657
+ "B13.up.b": {
1658
+ "kind": "f32",
1659
+ "shape": [
1660
+ 4096
1661
+ ],
1662
+ "off": 467041280,
1663
+ "bytes": 16384
1664
+ },
1665
+ "B13.down.w": {
1666
+ "kind": "bf16",
1667
+ "shape": [
1668
+ 1024,
1669
+ 4096
1670
+ ],
1671
+ "off": 467057664,
1672
+ "bytes": 8388608
1673
+ },
1674
+ "B13.down.b": {
1675
+ "kind": "f32",
1676
+ "shape": [
1677
+ 1024
1678
+ ],
1679
+ "off": 475446272,
1680
+ "bytes": 4096
1681
+ },
1682
+ "B14.norm1": {
1683
+ "kind": "f32",
1684
+ "shape": [
1685
+ 1024
1686
+ ],
1687
+ "off": 475450368,
1688
+ "bytes": 4096
1689
+ },
1690
+ "B14.norm2": {
1691
+ "kind": "f32",
1692
+ "shape": [
1693
+ 1024
1694
+ ],
1695
+ "off": 475454464,
1696
+ "bytes": 4096
1697
+ },
1698
+ "B14.qkv.w": {
1699
+ "kind": "bf16",
1700
+ "shape": [
1701
+ 3072,
1702
+ 1024
1703
+ ],
1704
+ "off": 475458560,
1705
+ "bytes": 6291456
1706
+ },
1707
+ "B14.qkv.b": {
1708
+ "kind": "f32",
1709
+ "shape": [
1710
+ 3072
1711
+ ],
1712
+ "off": 481750016,
1713
+ "bytes": 12288
1714
+ },
1715
+ "B14.proj.w": {
1716
+ "kind": "bf16",
1717
+ "shape": [
1718
+ 1024,
1719
+ 1024
1720
+ ],
1721
+ "off": 481762304,
1722
+ "bytes": 2097152
1723
+ },
1724
+ "B14.proj.b": {
1725
+ "kind": "f32",
1726
+ "shape": [
1727
+ 1024
1728
+ ],
1729
+ "off": 483859456,
1730
+ "bytes": 4096
1731
+ },
1732
+ "B14.q_norm": {
1733
+ "kind": "f32",
1734
+ "shape": [
1735
+ 64
1736
+ ],
1737
+ "off": 483863552,
1738
+ "bytes": 256
1739
+ },
1740
+ "B14.k_norm": {
1741
+ "kind": "f32",
1742
+ "shape": [
1743
+ 64
1744
+ ],
1745
+ "off": 483863808,
1746
+ "bytes": 256
1747
+ },
1748
+ "B14.gate.w": {
1749
+ "kind": "bf16",
1750
+ "shape": [
1751
+ 4096,
1752
+ 1024
1753
+ ],
1754
+ "off": 483864064,
1755
+ "bytes": 8388608
1756
+ },
1757
+ "B14.gate.b": {
1758
+ "kind": "f32",
1759
+ "shape": [
1760
+ 4096
1761
+ ],
1762
+ "off": 492252672,
1763
+ "bytes": 16384
1764
+ },
1765
+ "B14.up.w": {
1766
+ "kind": "bf16",
1767
+ "shape": [
1768
+ 4096,
1769
+ 1024
1770
+ ],
1771
+ "off": 492269056,
1772
+ "bytes": 8388608
1773
+ },
1774
+ "B14.up.b": {
1775
+ "kind": "f32",
1776
+ "shape": [
1777
+ 4096
1778
+ ],
1779
+ "off": 500657664,
1780
+ "bytes": 16384
1781
+ },
1782
+ "B14.down.w": {
1783
+ "kind": "bf16",
1784
+ "shape": [
1785
+ 1024,
1786
+ 4096
1787
+ ],
1788
+ "off": 500674048,
1789
+ "bytes": 8388608
1790
+ },
1791
+ "B14.down.b": {
1792
+ "kind": "f32",
1793
+ "shape": [
1794
+ 1024
1795
+ ],
1796
+ "off": 509062656,
1797
+ "bytes": 4096
1798
+ },
1799
+ "B15.norm1": {
1800
+ "kind": "f32",
1801
+ "shape": [
1802
+ 1024
1803
+ ],
1804
+ "off": 509066752,
1805
+ "bytes": 4096
1806
+ },
1807
+ "B15.norm2": {
1808
+ "kind": "f32",
1809
+ "shape": [
1810
+ 1024
1811
+ ],
1812
+ "off": 509070848,
1813
+ "bytes": 4096
1814
+ },
1815
+ "B15.qkv.w": {
1816
+ "kind": "bf16",
1817
+ "shape": [
1818
+ 3072,
1819
+ 1024
1820
+ ],
1821
+ "off": 509074944,
1822
+ "bytes": 6291456
1823
+ },
1824
+ "B15.qkv.b": {
1825
+ "kind": "f32",
1826
+ "shape": [
1827
+ 3072
1828
+ ],
1829
+ "off": 515366400,
1830
+ "bytes": 12288
1831
+ },
1832
+ "B15.proj.w": {
1833
+ "kind": "bf16",
1834
+ "shape": [
1835
+ 1024,
1836
+ 1024
1837
+ ],
1838
+ "off": 515378688,
1839
+ "bytes": 2097152
1840
+ },
1841
+ "B15.proj.b": {
1842
+ "kind": "f32",
1843
+ "shape": [
1844
+ 1024
1845
+ ],
1846
+ "off": 517475840,
1847
+ "bytes": 4096
1848
+ },
1849
+ "B15.q_norm": {
1850
+ "kind": "f32",
1851
+ "shape": [
1852
+ 64
1853
+ ],
1854
+ "off": 517479936,
1855
+ "bytes": 256
1856
+ },
1857
+ "B15.k_norm": {
1858
+ "kind": "f32",
1859
+ "shape": [
1860
+ 64
1861
+ ],
1862
+ "off": 517480192,
1863
+ "bytes": 256
1864
+ },
1865
+ "B15.gate.w": {
1866
+ "kind": "bf16",
1867
+ "shape": [
1868
+ 4096,
1869
+ 1024
1870
+ ],
1871
+ "off": 517480448,
1872
+ "bytes": 8388608
1873
+ },
1874
+ "B15.gate.b": {
1875
+ "kind": "f32",
1876
+ "shape": [
1877
+ 4096
1878
+ ],
1879
+ "off": 525869056,
1880
+ "bytes": 16384
1881
+ },
1882
+ "B15.up.w": {
1883
+ "kind": "bf16",
1884
+ "shape": [
1885
+ 4096,
1886
+ 1024
1887
+ ],
1888
+ "off": 525885440,
1889
+ "bytes": 8388608
1890
+ },
1891
+ "B15.up.b": {
1892
+ "kind": "f32",
1893
+ "shape": [
1894
+ 4096
1895
+ ],
1896
+ "off": 534274048,
1897
+ "bytes": 16384
1898
+ },
1899
+ "B15.down.w": {
1900
+ "kind": "bf16",
1901
+ "shape": [
1902
+ 1024,
1903
+ 4096
1904
+ ],
1905
+ "off": 534290432,
1906
+ "bytes": 8388608
1907
+ },
1908
+ "B15.down.b": {
1909
+ "kind": "f32",
1910
+ "shape": [
1911
+ 1024
1912
+ ],
1913
+ "off": 542679040,
1914
+ "bytes": 4096
1915
+ },
1916
+ "B16.norm1": {
1917
+ "kind": "f32",
1918
+ "shape": [
1919
+ 1024
1920
+ ],
1921
+ "off": 542683136,
1922
+ "bytes": 4096
1923
+ },
1924
+ "B16.norm2": {
1925
+ "kind": "f32",
1926
+ "shape": [
1927
+ 1024
1928
+ ],
1929
+ "off": 542687232,
1930
+ "bytes": 4096
1931
+ },
1932
+ "B16.qkv.w": {
1933
+ "kind": "bf16",
1934
+ "shape": [
1935
+ 3072,
1936
+ 1024
1937
+ ],
1938
+ "off": 542691328,
1939
+ "bytes": 6291456
1940
+ },
1941
+ "B16.qkv.b": {
1942
+ "kind": "f32",
1943
+ "shape": [
1944
+ 3072
1945
+ ],
1946
+ "off": 548982784,
1947
+ "bytes": 12288
1948
+ },
1949
+ "B16.proj.w": {
1950
+ "kind": "bf16",
1951
+ "shape": [
1952
+ 1024,
1953
+ 1024
1954
+ ],
1955
+ "off": 548995072,
1956
+ "bytes": 2097152
1957
+ },
1958
+ "B16.proj.b": {
1959
+ "kind": "f32",
1960
+ "shape": [
1961
+ 1024
1962
+ ],
1963
+ "off": 551092224,
1964
+ "bytes": 4096
1965
+ },
1966
+ "B16.q_norm": {
1967
+ "kind": "f32",
1968
+ "shape": [
1969
+ 64
1970
+ ],
1971
+ "off": 551096320,
1972
+ "bytes": 256
1973
+ },
1974
+ "B16.k_norm": {
1975
+ "kind": "f32",
1976
+ "shape": [
1977
+ 64
1978
+ ],
1979
+ "off": 551096576,
1980
+ "bytes": 256
1981
+ },
1982
+ "B16.gate.w": {
1983
+ "kind": "bf16",
1984
+ "shape": [
1985
+ 4096,
1986
+ 1024
1987
+ ],
1988
+ "off": 551096832,
1989
+ "bytes": 8388608
1990
+ },
1991
+ "B16.gate.b": {
1992
+ "kind": "f32",
1993
+ "shape": [
1994
+ 4096
1995
+ ],
1996
+ "off": 559485440,
1997
+ "bytes": 16384
1998
+ },
1999
+ "B16.up.w": {
2000
+ "kind": "bf16",
2001
+ "shape": [
2002
+ 4096,
2003
+ 1024
2004
+ ],
2005
+ "off": 559501824,
2006
+ "bytes": 8388608
2007
+ },
2008
+ "B16.up.b": {
2009
+ "kind": "f32",
2010
+ "shape": [
2011
+ 4096
2012
+ ],
2013
+ "off": 567890432,
2014
+ "bytes": 16384
2015
+ },
2016
+ "B16.down.w": {
2017
+ "kind": "bf16",
2018
+ "shape": [
2019
+ 1024,
2020
+ 4096
2021
+ ],
2022
+ "off": 567906816,
2023
+ "bytes": 8388608
2024
+ },
2025
+ "B16.down.b": {
2026
+ "kind": "f32",
2027
+ "shape": [
2028
+ 1024
2029
+ ],
2030
+ "off": 576295424,
2031
+ "bytes": 4096
2032
+ },
2033
+ "B17.norm1": {
2034
+ "kind": "f32",
2035
+ "shape": [
2036
+ 1024
2037
+ ],
2038
+ "off": 576299520,
2039
+ "bytes": 4096
2040
+ },
2041
+ "B17.norm2": {
2042
+ "kind": "f32",
2043
+ "shape": [
2044
+ 1024
2045
+ ],
2046
+ "off": 576303616,
2047
+ "bytes": 4096
2048
+ },
2049
+ "B17.qkv.w": {
2050
+ "kind": "bf16",
2051
+ "shape": [
2052
+ 3072,
2053
+ 1024
2054
+ ],
2055
+ "off": 576307712,
2056
+ "bytes": 6291456
2057
+ },
2058
+ "B17.qkv.b": {
2059
+ "kind": "f32",
2060
+ "shape": [
2061
+ 3072
2062
+ ],
2063
+ "off": 582599168,
2064
+ "bytes": 12288
2065
+ },
2066
+ "B17.proj.w": {
2067
+ "kind": "bf16",
2068
+ "shape": [
2069
+ 1024,
2070
+ 1024
2071
+ ],
2072
+ "off": 582611456,
2073
+ "bytes": 2097152
2074
+ },
2075
+ "B17.proj.b": {
2076
+ "kind": "f32",
2077
+ "shape": [
2078
+ 1024
2079
+ ],
2080
+ "off": 584708608,
2081
+ "bytes": 4096
2082
+ },
2083
+ "B17.q_norm": {
2084
+ "kind": "f32",
2085
+ "shape": [
2086
+ 64
2087
+ ],
2088
+ "off": 584712704,
2089
+ "bytes": 256
2090
+ },
2091
+ "B17.k_norm": {
2092
+ "kind": "f32",
2093
+ "shape": [
2094
+ 64
2095
+ ],
2096
+ "off": 584712960,
2097
+ "bytes": 256
2098
+ },
2099
+ "B17.gate.w": {
2100
+ "kind": "bf16",
2101
+ "shape": [
2102
+ 4096,
2103
+ 1024
2104
+ ],
2105
+ "off": 584713216,
2106
+ "bytes": 8388608
2107
+ },
2108
+ "B17.gate.b": {
2109
+ "kind": "f32",
2110
+ "shape": [
2111
+ 4096
2112
+ ],
2113
+ "off": 593101824,
2114
+ "bytes": 16384
2115
+ },
2116
+ "B17.up.w": {
2117
+ "kind": "bf16",
2118
+ "shape": [
2119
+ 4096,
2120
+ 1024
2121
+ ],
2122
+ "off": 593118208,
2123
+ "bytes": 8388608
2124
+ },
2125
+ "B17.up.b": {
2126
+ "kind": "f32",
2127
+ "shape": [
2128
+ 4096
2129
+ ],
2130
+ "off": 601506816,
2131
+ "bytes": 16384
2132
+ },
2133
+ "B17.down.w": {
2134
+ "kind": "bf16",
2135
+ "shape": [
2136
+ 1024,
2137
+ 4096
2138
+ ],
2139
+ "off": 601523200,
2140
+ "bytes": 8388608
2141
+ },
2142
+ "B17.down.b": {
2143
+ "kind": "f32",
2144
+ "shape": [
2145
+ 1024
2146
+ ],
2147
+ "off": 609911808,
2148
+ "bytes": 4096
2149
+ },
2150
+ "B18.norm1": {
2151
+ "kind": "f32",
2152
+ "shape": [
2153
+ 1024
2154
+ ],
2155
+ "off": 609915904,
2156
+ "bytes": 4096
2157
+ },
2158
+ "B18.norm2": {
2159
+ "kind": "f32",
2160
+ "shape": [
2161
+ 1024
2162
+ ],
2163
+ "off": 609920000,
2164
+ "bytes": 4096
2165
+ },
2166
+ "B18.qkv.w": {
2167
+ "kind": "bf16",
2168
+ "shape": [
2169
+ 3072,
2170
+ 1024
2171
+ ],
2172
+ "off": 609924096,
2173
+ "bytes": 6291456
2174
+ },
2175
+ "B18.qkv.b": {
2176
+ "kind": "f32",
2177
+ "shape": [
2178
+ 3072
2179
+ ],
2180
+ "off": 616215552,
2181
+ "bytes": 12288
2182
+ },
2183
+ "B18.proj.w": {
2184
+ "kind": "bf16",
2185
+ "shape": [
2186
+ 1024,
2187
+ 1024
2188
+ ],
2189
+ "off": 616227840,
2190
+ "bytes": 2097152
2191
+ },
2192
+ "B18.proj.b": {
2193
+ "kind": "f32",
2194
+ "shape": [
2195
+ 1024
2196
+ ],
2197
+ "off": 618324992,
2198
+ "bytes": 4096
2199
+ },
2200
+ "B18.q_norm": {
2201
+ "kind": "f32",
2202
+ "shape": [
2203
+ 64
2204
+ ],
2205
+ "off": 618329088,
2206
+ "bytes": 256
2207
+ },
2208
+ "B18.k_norm": {
2209
+ "kind": "f32",
2210
+ "shape": [
2211
+ 64
2212
+ ],
2213
+ "off": 618329344,
2214
+ "bytes": 256
2215
+ },
2216
+ "B18.gate.w": {
2217
+ "kind": "bf16",
2218
+ "shape": [
2219
+ 4096,
2220
+ 1024
2221
+ ],
2222
+ "off": 618329600,
2223
+ "bytes": 8388608
2224
+ },
2225
+ "B18.gate.b": {
2226
+ "kind": "f32",
2227
+ "shape": [
2228
+ 4096
2229
+ ],
2230
+ "off": 626718208,
2231
+ "bytes": 16384
2232
+ },
2233
+ "B18.up.w": {
2234
+ "kind": "bf16",
2235
+ "shape": [
2236
+ 4096,
2237
+ 1024
2238
+ ],
2239
+ "off": 626734592,
2240
+ "bytes": 8388608
2241
+ },
2242
+ "B18.up.b": {
2243
+ "kind": "f32",
2244
+ "shape": [
2245
+ 4096
2246
+ ],
2247
+ "off": 635123200,
2248
+ "bytes": 16384
2249
+ },
2250
+ "B18.down.w": {
2251
+ "kind": "bf16",
2252
+ "shape": [
2253
+ 1024,
2254
+ 4096
2255
+ ],
2256
+ "off": 635139584,
2257
+ "bytes": 8388608
2258
+ },
2259
+ "B18.down.b": {
2260
+ "kind": "f32",
2261
+ "shape": [
2262
+ 1024
2263
+ ],
2264
+ "off": 643528192,
2265
+ "bytes": 4096
2266
+ },
2267
+ "B19.norm1": {
2268
+ "kind": "f32",
2269
+ "shape": [
2270
+ 1024
2271
+ ],
2272
+ "off": 643532288,
2273
+ "bytes": 4096
2274
+ },
2275
+ "B19.norm2": {
2276
+ "kind": "f32",
2277
+ "shape": [
2278
+ 1024
2279
+ ],
2280
+ "off": 643536384,
2281
+ "bytes": 4096
2282
+ },
2283
+ "B19.qkv.w": {
2284
+ "kind": "bf16",
2285
+ "shape": [
2286
+ 3072,
2287
+ 1024
2288
+ ],
2289
+ "off": 643540480,
2290
+ "bytes": 6291456
2291
+ },
2292
+ "B19.qkv.b": {
2293
+ "kind": "f32",
2294
+ "shape": [
2295
+ 3072
2296
+ ],
2297
+ "off": 649831936,
2298
+ "bytes": 12288
2299
+ },
2300
+ "B19.proj.w": {
2301
+ "kind": "bf16",
2302
+ "shape": [
2303
+ 1024,
2304
+ 1024
2305
+ ],
2306
+ "off": 649844224,
2307
+ "bytes": 2097152
2308
+ },
2309
+ "B19.proj.b": {
2310
+ "kind": "f32",
2311
+ "shape": [
2312
+ 1024
2313
+ ],
2314
+ "off": 651941376,
2315
+ "bytes": 4096
2316
+ },
2317
+ "B19.q_norm": {
2318
+ "kind": "f32",
2319
+ "shape": [
2320
+ 64
2321
+ ],
2322
+ "off": 651945472,
2323
+ "bytes": 256
2324
+ },
2325
+ "B19.k_norm": {
2326
+ "kind": "f32",
2327
+ "shape": [
2328
+ 64
2329
+ ],
2330
+ "off": 651945728,
2331
+ "bytes": 256
2332
+ },
2333
+ "B19.gate.w": {
2334
+ "kind": "bf16",
2335
+ "shape": [
2336
+ 4096,
2337
+ 1024
2338
+ ],
2339
+ "off": 651945984,
2340
+ "bytes": 8388608
2341
+ },
2342
+ "B19.gate.b": {
2343
+ "kind": "f32",
2344
+ "shape": [
2345
+ 4096
2346
+ ],
2347
+ "off": 660334592,
2348
+ "bytes": 16384
2349
+ },
2350
+ "B19.up.w": {
2351
+ "kind": "bf16",
2352
+ "shape": [
2353
+ 4096,
2354
+ 1024
2355
+ ],
2356
+ "off": 660350976,
2357
+ "bytes": 8388608
2358
+ },
2359
+ "B19.up.b": {
2360
+ "kind": "f32",
2361
+ "shape": [
2362
+ 4096
2363
+ ],
2364
+ "off": 668739584,
2365
+ "bytes": 16384
2366
+ },
2367
+ "B19.down.w": {
2368
+ "kind": "bf16",
2369
+ "shape": [
2370
+ 1024,
2371
+ 4096
2372
+ ],
2373
+ "off": 668755968,
2374
+ "bytes": 8388608
2375
+ },
2376
+ "B19.down.b": {
2377
+ "kind": "f32",
2378
+ "shape": [
2379
+ 1024
2380
+ ],
2381
+ "off": 677144576,
2382
+ "bytes": 4096
2383
+ },
2384
+ "B20.norm1": {
2385
+ "kind": "f32",
2386
+ "shape": [
2387
+ 1024
2388
+ ],
2389
+ "off": 677148672,
2390
+ "bytes": 4096
2391
+ },
2392
+ "B20.norm2": {
2393
+ "kind": "f32",
2394
+ "shape": [
2395
+ 1024
2396
+ ],
2397
+ "off": 677152768,
2398
+ "bytes": 4096
2399
+ },
2400
+ "B20.qkv.w": {
2401
+ "kind": "bf16",
2402
+ "shape": [
2403
+ 3072,
2404
+ 1024
2405
+ ],
2406
+ "off": 677156864,
2407
+ "bytes": 6291456
2408
+ },
2409
+ "B20.qkv.b": {
2410
+ "kind": "f32",
2411
+ "shape": [
2412
+ 3072
2413
+ ],
2414
+ "off": 683448320,
2415
+ "bytes": 12288
2416
+ },
2417
+ "B20.proj.w": {
2418
+ "kind": "bf16",
2419
+ "shape": [
2420
+ 1024,
2421
+ 1024
2422
+ ],
2423
+ "off": 683460608,
2424
+ "bytes": 2097152
2425
+ },
2426
+ "B20.proj.b": {
2427
+ "kind": "f32",
2428
+ "shape": [
2429
+ 1024
2430
+ ],
2431
+ "off": 685557760,
2432
+ "bytes": 4096
2433
+ },
2434
+ "B20.q_norm": {
2435
+ "kind": "f32",
2436
+ "shape": [
2437
+ 64
2438
+ ],
2439
+ "off": 685561856,
2440
+ "bytes": 256
2441
+ },
2442
+ "B20.k_norm": {
2443
+ "kind": "f32",
2444
+ "shape": [
2445
+ 64
2446
+ ],
2447
+ "off": 685562112,
2448
+ "bytes": 256
2449
+ },
2450
+ "B20.gate.w": {
2451
+ "kind": "bf16",
2452
+ "shape": [
2453
+ 4096,
2454
+ 1024
2455
+ ],
2456
+ "off": 685562368,
2457
+ "bytes": 8388608
2458
+ },
2459
+ "B20.gate.b": {
2460
+ "kind": "f32",
2461
+ "shape": [
2462
+ 4096
2463
+ ],
2464
+ "off": 693950976,
2465
+ "bytes": 16384
2466
+ },
2467
+ "B20.up.w": {
2468
+ "kind": "bf16",
2469
+ "shape": [
2470
+ 4096,
2471
+ 1024
2472
+ ],
2473
+ "off": 693967360,
2474
+ "bytes": 8388608
2475
+ },
2476
+ "B20.up.b": {
2477
+ "kind": "f32",
2478
+ "shape": [
2479
+ 4096
2480
+ ],
2481
+ "off": 702355968,
2482
+ "bytes": 16384
2483
+ },
2484
+ "B20.down.w": {
2485
+ "kind": "bf16",
2486
+ "shape": [
2487
+ 1024,
2488
+ 4096
2489
+ ],
2490
+ "off": 702372352,
2491
+ "bytes": 8388608
2492
+ },
2493
+ "B20.down.b": {
2494
+ "kind": "f32",
2495
+ "shape": [
2496
+ 1024
2497
+ ],
2498
+ "off": 710760960,
2499
+ "bytes": 4096
2500
+ },
2501
+ "B21.norm1": {
2502
+ "kind": "f32",
2503
+ "shape": [
2504
+ 1024
2505
+ ],
2506
+ "off": 710765056,
2507
+ "bytes": 4096
2508
+ },
2509
+ "B21.norm2": {
2510
+ "kind": "f32",
2511
+ "shape": [
2512
+ 1024
2513
+ ],
2514
+ "off": 710769152,
2515
+ "bytes": 4096
2516
+ },
2517
+ "B21.qkv.w": {
2518
+ "kind": "bf16",
2519
+ "shape": [
2520
+ 3072,
2521
+ 1024
2522
+ ],
2523
+ "off": 710773248,
2524
+ "bytes": 6291456
2525
+ },
2526
+ "B21.qkv.b": {
2527
+ "kind": "f32",
2528
+ "shape": [
2529
+ 3072
2530
+ ],
2531
+ "off": 717064704,
2532
+ "bytes": 12288
2533
+ },
2534
+ "B21.proj.w": {
2535
+ "kind": "bf16",
2536
+ "shape": [
2537
+ 1024,
2538
+ 1024
2539
+ ],
2540
+ "off": 717076992,
2541
+ "bytes": 2097152
2542
+ },
2543
+ "B21.proj.b": {
2544
+ "kind": "f32",
2545
+ "shape": [
2546
+ 1024
2547
+ ],
2548
+ "off": 719174144,
2549
+ "bytes": 4096
2550
+ },
2551
+ "B21.q_norm": {
2552
+ "kind": "f32",
2553
+ "shape": [
2554
+ 64
2555
+ ],
2556
+ "off": 719178240,
2557
+ "bytes": 256
2558
+ },
2559
+ "B21.k_norm": {
2560
+ "kind": "f32",
2561
+ "shape": [
2562
+ 64
2563
+ ],
2564
+ "off": 719178496,
2565
+ "bytes": 256
2566
+ },
2567
+ "B21.gate.w": {
2568
+ "kind": "bf16",
2569
+ "shape": [
2570
+ 4096,
2571
+ 1024
2572
+ ],
2573
+ "off": 719178752,
2574
+ "bytes": 8388608
2575
+ },
2576
+ "B21.gate.b": {
2577
+ "kind": "f32",
2578
+ "shape": [
2579
+ 4096
2580
+ ],
2581
+ "off": 727567360,
2582
+ "bytes": 16384
2583
+ },
2584
+ "B21.up.w": {
2585
+ "kind": "bf16",
2586
+ "shape": [
2587
+ 4096,
2588
+ 1024
2589
+ ],
2590
+ "off": 727583744,
2591
+ "bytes": 8388608
2592
+ },
2593
+ "B21.up.b": {
2594
+ "kind": "f32",
2595
+ "shape": [
2596
+ 4096
2597
+ ],
2598
+ "off": 735972352,
2599
+ "bytes": 16384
2600
+ },
2601
+ "B21.down.w": {
2602
+ "kind": "bf16",
2603
+ "shape": [
2604
+ 1024,
2605
+ 4096
2606
+ ],
2607
+ "off": 735988736,
2608
+ "bytes": 8388608
2609
+ },
2610
+ "B21.down.b": {
2611
+ "kind": "f32",
2612
+ "shape": [
2613
+ 1024
2614
+ ],
2615
+ "off": 744377344,
2616
+ "bytes": 4096
2617
+ },
2618
+ "B22.norm1": {
2619
+ "kind": "f32",
2620
+ "shape": [
2621
+ 1024
2622
+ ],
2623
+ "off": 744381440,
2624
+ "bytes": 4096
2625
+ },
2626
+ "B22.norm2": {
2627
+ "kind": "f32",
2628
+ "shape": [
2629
+ 1024
2630
+ ],
2631
+ "off": 744385536,
2632
+ "bytes": 4096
2633
+ },
2634
+ "B22.qkv.w": {
2635
+ "kind": "bf16",
2636
+ "shape": [
2637
+ 3072,
2638
+ 1024
2639
+ ],
2640
+ "off": 744389632,
2641
+ "bytes": 6291456
2642
+ },
2643
+ "B22.qkv.b": {
2644
+ "kind": "f32",
2645
+ "shape": [
2646
+ 3072
2647
+ ],
2648
+ "off": 750681088,
2649
+ "bytes": 12288
2650
+ },
2651
+ "B22.proj.w": {
2652
+ "kind": "bf16",
2653
+ "shape": [
2654
+ 1024,
2655
+ 1024
2656
+ ],
2657
+ "off": 750693376,
2658
+ "bytes": 2097152
2659
+ },
2660
+ "B22.proj.b": {
2661
+ "kind": "f32",
2662
+ "shape": [
2663
+ 1024
2664
+ ],
2665
+ "off": 752790528,
2666
+ "bytes": 4096
2667
+ },
2668
+ "B22.q_norm": {
2669
+ "kind": "f32",
2670
+ "shape": [
2671
+ 64
2672
+ ],
2673
+ "off": 752794624,
2674
+ "bytes": 256
2675
+ },
2676
+ "B22.k_norm": {
2677
+ "kind": "f32",
2678
+ "shape": [
2679
+ 64
2680
+ ],
2681
+ "off": 752794880,
2682
+ "bytes": 256
2683
+ },
2684
+ "B22.gate.w": {
2685
+ "kind": "bf16",
2686
+ "shape": [
2687
+ 4096,
2688
+ 1024
2689
+ ],
2690
+ "off": 752795136,
2691
+ "bytes": 8388608
2692
+ },
2693
+ "B22.gate.b": {
2694
+ "kind": "f32",
2695
+ "shape": [
2696
+ 4096
2697
+ ],
2698
+ "off": 761183744,
2699
+ "bytes": 16384
2700
+ },
2701
+ "B22.up.w": {
2702
+ "kind": "bf16",
2703
+ "shape": [
2704
+ 4096,
2705
+ 1024
2706
+ ],
2707
+ "off": 761200128,
2708
+ "bytes": 8388608
2709
+ },
2710
+ "B22.up.b": {
2711
+ "kind": "f32",
2712
+ "shape": [
2713
+ 4096
2714
+ ],
2715
+ "off": 769588736,
2716
+ "bytes": 16384
2717
+ },
2718
+ "B22.down.w": {
2719
+ "kind": "bf16",
2720
+ "shape": [
2721
+ 1024,
2722
+ 4096
2723
+ ],
2724
+ "off": 769605120,
2725
+ "bytes": 8388608
2726
+ },
2727
+ "B22.down.b": {
2728
+ "kind": "f32",
2729
+ "shape": [
2730
+ 1024
2731
+ ],
2732
+ "off": 777993728,
2733
+ "bytes": 4096
2734
+ },
2735
+ "B23.norm1": {
2736
+ "kind": "f32",
2737
+ "shape": [
2738
+ 1024
2739
+ ],
2740
+ "off": 777997824,
2741
+ "bytes": 4096
2742
+ },
2743
+ "B23.norm2": {
2744
+ "kind": "f32",
2745
+ "shape": [
2746
+ 1024
2747
+ ],
2748
+ "off": 778001920,
2749
+ "bytes": 4096
2750
+ },
2751
+ "B23.qkv.w": {
2752
+ "kind": "bf16",
2753
+ "shape": [
2754
+ 3072,
2755
+ 1024
2756
+ ],
2757
+ "off": 778006016,
2758
+ "bytes": 6291456
2759
+ },
2760
+ "B23.qkv.b": {
2761
+ "kind": "f32",
2762
+ "shape": [
2763
+ 3072
2764
+ ],
2765
+ "off": 784297472,
2766
+ "bytes": 12288
2767
+ },
2768
+ "B23.proj.w": {
2769
+ "kind": "bf16",
2770
+ "shape": [
2771
+ 1024,
2772
+ 1024
2773
+ ],
2774
+ "off": 784309760,
2775
+ "bytes": 2097152
2776
+ },
2777
+ "B23.proj.b": {
2778
+ "kind": "f32",
2779
+ "shape": [
2780
+ 1024
2781
+ ],
2782
+ "off": 786406912,
2783
+ "bytes": 4096
2784
+ },
2785
+ "B23.q_norm": {
2786
+ "kind": "f32",
2787
+ "shape": [
2788
+ 64
2789
+ ],
2790
+ "off": 786411008,
2791
+ "bytes": 256
2792
+ },
2793
+ "B23.k_norm": {
2794
+ "kind": "f32",
2795
+ "shape": [
2796
+ 64
2797
+ ],
2798
+ "off": 786411264,
2799
+ "bytes": 256
2800
+ },
2801
+ "B23.gate.w": {
2802
+ "kind": "bf16",
2803
+ "shape": [
2804
+ 4096,
2805
+ 1024
2806
+ ],
2807
+ "off": 786411520,
2808
+ "bytes": 8388608
2809
+ },
2810
+ "B23.gate.b": {
2811
+ "kind": "f32",
2812
+ "shape": [
2813
+ 4096
2814
+ ],
2815
+ "off": 794800128,
2816
+ "bytes": 16384
2817
+ },
2818
+ "B23.up.w": {
2819
+ "kind": "bf16",
2820
+ "shape": [
2821
+ 4096,
2822
+ 1024
2823
+ ],
2824
+ "off": 794816512,
2825
+ "bytes": 8388608
2826
+ },
2827
+ "B23.up.b": {
2828
+ "kind": "f32",
2829
+ "shape": [
2830
+ 4096
2831
+ ],
2832
+ "off": 803205120,
2833
+ "bytes": 16384
2834
+ },
2835
+ "B23.down.w": {
2836
+ "kind": "bf16",
2837
+ "shape": [
2838
+ 1024,
2839
+ 4096
2840
+ ],
2841
+ "off": 803221504,
2842
+ "bytes": 8388608
2843
+ },
2844
+ "B23.down.b": {
2845
+ "kind": "f32",
2846
+ "shape": [
2847
+ 1024
2848
+ ],
2849
+ "off": 811610112,
2850
+ "bytes": 4096
2851
+ },
2852
+ "post_norm": {
2853
+ "kind": "f32",
2854
+ "shape": [
2855
+ 1024
2856
+ ],
2857
+ "off": 811614208,
2858
+ "bytes": 4096
2859
+ },
2860
+ "down.w": {
2861
+ "kind": "bf16",
2862
+ "shape": [
2863
+ 4096,
2864
+ 4096
2865
+ ],
2866
+ "off": 811618304,
2867
+ "bytes": 33554432
2868
+ },
2869
+ "down.b": {
2870
+ "kind": "f32",
2871
+ "shape": [
2872
+ 4096
2873
+ ],
2874
+ "off": 845172736,
2875
+ "bytes": 16384
2876
+ },
2877
+ "merger.proj.w": {
2878
+ "kind": "bf16",
2879
+ "shape": [
2880
+ 4096,
2881
+ 4096
2882
+ ],
2883
+ "off": 845189120,
2884
+ "bytes": 33554432
2885
+ },
2886
+ "merger.ln.w": {
2887
+ "kind": "f32",
2888
+ "shape": [
2889
+ 4096
2890
+ ],
2891
+ "off": 878743552,
2892
+ "bytes": 16384
2893
+ },
2894
+ "merger.ln.b": {
2895
+ "kind": "f32",
2896
+ "shape": [
2897
+ 4096
2898
+ ],
2899
+ "off": 878759936,
2900
+ "bytes": 16384
2901
+ },
2902
+ "merger.gate.w": {
2903
+ "kind": "bf16",
2904
+ "shape": [
2905
+ 10240,
2906
+ 4096
2907
+ ],
2908
+ "off": 878776320,
2909
+ "bytes": 83886080
2910
+ },
2911
+ "merger.up.w": {
2912
+ "kind": "bf16",
2913
+ "shape": [
2914
+ 10240,
2915
+ 4096
2916
+ ],
2917
+ "off": 962662400,
2918
+ "bytes": 83886080
2919
+ },
2920
+ "merger.down.w": {
2921
+ "kind": "bf16",
2922
+ "shape": [
2923
+ 4096,
2924
+ 10240
2925
+ ],
2926
+ "off": 1046548480,
2927
+ "bytes": 83886080
2928
+ }
2929
+ }
2930
+ }
glm53_vision/weights.vbin ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:80cc8cbb0f6c6c10763751f97c8c3d1f5f8fd65333a4058d2e19116b695895c8
3
+ size 1130434560
glm53c_tp2_r0/draft_vocab.txt ADDED
The diff for this file is too large to render. See raw diff
 
glm53c_tp2_r0/index.json ADDED
The diff for this file is too large to render. See raw diff
 
glm53c_tp2_r0/weights.vbin ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:7122352458f7700a228791bb318fcf5149d8dd71bed2f621737aab50ef438ab4
3
+ size 94829824000
x3o_tp2_r0/index.json ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:cfba16e7ea0538e09aa92e375d3ba2924d747b0c1383a3538ea32c5731fda03a
3
+ size 11431569
x3o_tp2_r0/weights.vbin ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:ae0e9205c4de8fa18f5f3fa26c41f3b00511832b69c6b00458461122780ecc92
3
+ size 77111451648