ks46 commited on
Commit
3cb7a37
·
verified ·
1 Parent(s): 27931be

g2-6x256-a09: 3.5280 bits/char

Browse files
README.md CHANGED
@@ -30,6 +30,7 @@ Vocabulary is 256 bytes with byte 0 as the stop token; the held-out split is
30
  | [g2-8x256-a09](g2-8x256-a09) | 8×256 | 6.5M | 40,000 | 3.5020 | 92% |
31
  | [g2-12x192-a09](g2-12x192-a09) | 12×192 | 5.5M | 40,000 | 3.5074 | 92% |
32
  | [g2-4x384-a09](g2-4x384-a09) | 4×384 | 7.2M | 40,000 | 3.5193 | 92% |
 
33
  | [prod-8x768-phase2](prod-8x768-phase2) | 8×768 | 56.9M | 60,000 | 3.5619 | 93% |
34
  | [teacher-16x1024](teacher-16x1024) | 16×1024 | 205.9M | 9,000 | 3.6068 | 94% |
35
  | [pre-6x640-a09](pre-6x640-a09) | 6×640 | 30.7M | 20,000 | 3.6137 | 94% |
 
30
  | [g2-8x256-a09](g2-8x256-a09) | 8×256 | 6.5M | 40,000 | 3.5020 | 92% |
31
  | [g2-12x192-a09](g2-12x192-a09) | 12×192 | 5.5M | 40,000 | 3.5074 | 92% |
32
  | [g2-4x384-a09](g2-4x384-a09) | 4×384 | 7.2M | 40,000 | 3.5193 | 92% |
33
+ | [g2-6x256-a09](g2-6x256-a09) | 6×256 | 4.9M | 40,000 | 3.5280 | 92% |
34
  | [prod-8x768-phase2](prod-8x768-phase2) | 8×768 | 56.9M | 60,000 | 3.5619 | 93% |
35
  | [teacher-16x1024](teacher-16x1024) | 16×1024 | 205.9M | 9,000 | 3.6068 | 94% |
36
  | [pre-6x640-a09](pre-6x640-a09) | 6×640 | 30.7M | 20,000 | 3.6137 | 94% |
g2-6x256-a09/README.md ADDED
@@ -0,0 +1,70 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # g2-6x256-a09
2
+
3
+ A byte-level GPT that dreams up login-style usernames. 6 layers × 256 wide,
4
+ 4 heads, SwiGLU 704, 4,902,400 parameters, vocabulary of 256 bytes with
5
+ byte 0 as the stop token. Trained for 40,000 iterations of 512 × 128 bytes
6
+ on the `ks46/usernames` corpus (185,925,737 unique usernames from 18 public sources).
7
+
8
+ ## Quality
9
+
10
+ | metric | value |
11
+ |---|---|
12
+ | held-out bits per char (2,907,216 names) | **3.5280** |
13
+ | samples valid | 99.9% |
14
+ | samples unique | 100.0% |
15
+ | samples novel (absent from the corpus) | 92.2% |
16
+ | unterminated draws | 0 of 10,000 |
17
+
18
+ Per-source held-out bits per char (lower is easier):
19
+
20
+ | source | bits/char |
21
+ |---|---|
22
+ | kaggle | 3.21 |
23
+ | urls_full | 3.31 |
24
+ | jeanphorn | 3.35 |
25
+ | urls | 3.50 |
26
+ | atlas | 3.56 |
27
+ | reddit | 3.56 |
28
+ | sentiment140 | 3.62 |
29
+ | bsky | 3.65 |
30
+ | stack | 3.71 |
31
+ | hn | 3.72 |
32
+ | gharchive | 3.74 |
33
+ | codeforces | 3.76 |
34
+ | crates | 3.76 |
35
+ | lichess | 3.90 |
36
+ | enwiki | 3.90 |
37
+ | wikis | 3.98 |
38
+ | xato | 4.02 |
39
+ | osm | 4.03 |
40
+ | se | 4.07 |
41
+ | roblox | 4.13 |
42
+ | so | 4.16 |
43
+ | ssa | 4.39 |
44
+
45
+ ## Quantized variants
46
+
47
+ Post-training, symmetric per-group absmax on every 2D weight:
48
+
49
+ | format | held-out bits/char | Δ vs fp32 | novel | size |
50
+ |---|---|---|---|---|
51
+ | int8 g64 | 3.5304 | +0.0002 | 0.922 | 5.1 MB |
52
+ | int4 g64 | 3.5993 | +0.0691 | 0.917 | 2.6 MB |
53
+ | int4 g32 | 3.5839 | +0.0536 | 0.914 | 2.8 MB |
54
+
55
+ ## Files
56
+
57
+ - `ckpt.pt`: the fp32 checkpoint (`model`, `optimizer`, `config`, `iter`).
58
+ - `model.ndq`: int8 (group 64) export for the Rust kernel; `model-int4.ndq` the 4-bit one (half the bytes, 2.4× faster at batch 1, +0.01 bits/char); `golden*.json` hold Python reference NLLs for 64 held-out names.
59
+ - `ndgen`: the kernel, built for x86-64 with AVX2 (`-C target-cpu=native` on an EPYC 7763); rebuild from `rust/ndkernel` for another host.
60
+ - `quant-*.pt`: fake-quantized checkpoints loadable like `ckpt.pt`.
61
+ - `eval.json`, `manifest.json`, `samples.txt`.
62
+
63
+ ## Sampling
64
+
65
+ ```
66
+ uv run python -m training.sample --ckpt ckpt.pt -n 20 --prefix dark --temperature 0.9 --novel --parquet names.parquet
67
+ ./ndgen gen --model model.ndq -n 1000 --batch 64 --temperature 0.9 --prefix dark
68
+ ```
69
+
70
+ Sample names: SirDumbBXiT, arpan78777b358, corko05, RADICALTuNTHU, ratfishkeepus, narcissanoadexi, jdjl24, draculadave, imnotbh, riseoncrea78975, b77mini007, r-dgospin, frenxrlasasuitwerk, asjcghetto, revdair_review, ThatWubsFineface, sunmtthaithos, maiautomaticscience, Maestro02577, mashetisl, extra-aprendiendo, upmkjatin, squiresarah, celerydynasty, wmcken11, wgcdm, morganerer, mannyopina, kayz_n, pachisetonomi
g2-6x256-a09/ckpt.pt ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:a8b8f7b22ab5869d7ddbb7c70db03754b2809a21527a2d8eb8b9dbcd313ce0a6
3
+ size 58875241
g2-6x256-a09/eval.json ADDED
@@ -0,0 +1,98 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "ckpt": "/root/models/name-dreamer/g2-6x256-a09/ckpt.pt",
3
+ "n_heldout": 2907216,
4
+ "vocab": 256,
5
+ "bits_per_char": 3.527974167017619,
6
+ "bits_per_char_by_source": {
7
+ "xato": 4.024610630428509,
8
+ "bsky": 3.6505540727044035,
9
+ "stack": 3.7067097712006425,
10
+ "hn": 3.7155356638345984,
11
+ "reddit": 3.563161271622452,
12
+ "urls": 3.500879085183282,
13
+ "roblox": 4.1338934981412025,
14
+ "crates": 3.7624159077590233,
15
+ "kaggle": 3.209455248745971,
16
+ "gharchive": 3.737931321678688,
17
+ "codeforces": 3.761002710649131,
18
+ "lichess": 3.902432314717484,
19
+ "jeanphorn": 3.3538238767109734,
20
+ "so": 4.161915766841152,
21
+ "se": 4.06589782259505,
22
+ "osm": 4.034289021776484,
23
+ "enwiki": 3.902861109868001,
24
+ "ssa": 4.386260731585523,
25
+ "wikis": 3.983552995602469,
26
+ "sentiment140": 3.6152563296024116,
27
+ "urls_full": 3.3085488906840497,
28
+ "atlas": 3.556889257298462
29
+ },
30
+ "samples": {
31
+ "n": 10000,
32
+ "temperature": 1.0,
33
+ "valid": 0.9994,
34
+ "unique": 1.0,
35
+ "novel": 0.9221532919751851,
36
+ "unterminated": 0,
37
+ "length_hist": {
38
+ "3": 10,
39
+ "4": 48,
40
+ "5": 207,
41
+ "6": 531,
42
+ "7": 833,
43
+ "8": 1042,
44
+ "9": 1083,
45
+ "10": 1117,
46
+ "11": 1010,
47
+ "12": 850,
48
+ "13": 809,
49
+ "14": 653,
50
+ "15": 474,
51
+ "16": 400,
52
+ "17": 269,
53
+ "18": 200,
54
+ "19": 134,
55
+ "20": 102,
56
+ "21": 58,
57
+ "22": 47,
58
+ "23": 33,
59
+ "24": 25,
60
+ "25": 19,
61
+ "26": 8,
62
+ "27": 7,
63
+ "28": 7,
64
+ "29": 4,
65
+ "30": 14
66
+ }
67
+ },
68
+ "heldout_length_hist": {
69
+ "3": 1654,
70
+ "4": 14562,
71
+ "5": 58539,
72
+ "6": 156177,
73
+ "7": 244141,
74
+ "8": 303986,
75
+ "9": 307229,
76
+ "10": 320933,
77
+ "11": 288131,
78
+ "12": 262342,
79
+ "13": 224141,
80
+ "14": 191139,
81
+ "15": 155943,
82
+ "16": 103974,
83
+ "17": 76941,
84
+ "18": 60685,
85
+ "19": 41171,
86
+ "20": 34156,
87
+ "21": 14710,
88
+ "22": 12154,
89
+ "23": 9023,
90
+ "24": 7057,
91
+ "25": 5359,
92
+ "26": 3899,
93
+ "27": 3097,
94
+ "28": 2341,
95
+ "29": 1886,
96
+ "30": 1846
97
+ }
98
+ }
g2-6x256-a09/golden-int7.json ADDED
@@ -0,0 +1,263 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "group": 64,
3
+ "bits": 7,
4
+ "names": [
5
+ {
6
+ "name": "holesk",
7
+ "nll": 20.395837783813477
8
+ },
9
+ {
10
+ "name": "giorgiaoikonomou",
11
+ "nll": 29.2760009765625
12
+ },
13
+ {
14
+ "name": "Pranav1291",
15
+ "nll": 21.900712966918945
16
+ },
17
+ {
18
+ "name": "washindanzel28",
19
+ "nll": 40.156639099121094
20
+ },
21
+ {
22
+ "name": "Matikz",
23
+ "nll": 20.289161682128906
24
+ },
25
+ {
26
+ "name": "Caboose811",
27
+ "nll": 23.758569717407227
28
+ },
29
+ {
30
+ "name": "OntongGhebi",
31
+ "nll": 38.82344436645508
32
+ },
33
+ {
34
+ "name": "askmikeaquestion",
35
+ "nll": 30.52001953125
36
+ },
37
+ {
38
+ "name": "bakkeren",
39
+ "nll": 21.65003776550293
40
+ },
41
+ {
42
+ "name": "reem_alattas",
43
+ "nll": 28.397186279296875
44
+ },
45
+ {
46
+ "name": "icebsj",
47
+ "nll": 23.52632713317871
48
+ },
49
+ {
50
+ "name": "frillywitch",
51
+ "nll": 20.62656593322754
52
+ },
53
+ {
54
+ "name": "Kledoro",
55
+ "nll": 24.570323944091797
56
+ },
57
+ {
58
+ "name": "dtaivpp_1463610",
59
+ "nll": 53.46287155151367
60
+ },
61
+ {
62
+ "name": "brayannadjan",
63
+ "nll": 29.05016326904297
64
+ },
65
+ {
66
+ "name": "edibleethicscom",
67
+ "nll": 32.94371032714844
68
+ },
69
+ {
70
+ "name": "jtvcorner",
71
+ "nll": 23.736902236938477
72
+ },
73
+ {
74
+ "name": "ivansorano",
75
+ "nll": 22.51315689086914
76
+ },
77
+ {
78
+ "name": "Reshantika",
79
+ "nll": 27.969207763671875
80
+ },
81
+ {
82
+ "name": "dameq1",
83
+ "nll": 22.817520141601562
84
+ },
85
+ {
86
+ "name": "gergo-bossanyi",
87
+ "nll": 37.7056999206543
88
+ },
89
+ {
90
+ "name": "natanaeladolfsson",
91
+ "nll": 31.273340225219727
92
+ },
93
+ {
94
+ "name": "dark.drakus",
95
+ "nll": 28.281692504882812
96
+ },
97
+ {
98
+ "name": "gingerkuenzel",
99
+ "nll": 27.299684524536133
100
+ },
101
+ {
102
+ "name": "legalamlawfirm",
103
+ "nll": 31.868541717529297
104
+ },
105
+ {
106
+ "name": "eternallyraven",
107
+ "nll": 20.606975555419922
108
+ },
109
+ {
110
+ "name": "kishifuji4729",
111
+ "nll": 33.95426559448242
112
+ },
113
+ {
114
+ "name": "szkhrm",
115
+ "nll": 23.686298370361328
116
+ },
117
+ {
118
+ "name": "lingnoi16",
119
+ "nll": 28.14520835876465
120
+ },
121
+ {
122
+ "name": "jaketui",
123
+ "nll": 21.841720581054688
124
+ },
125
+ {
126
+ "name": "devon4",
127
+ "nll": 19.226409912109375
128
+ },
129
+ {
130
+ "name": "fami0603",
131
+ "nll": 23.99629783630371
132
+ },
133
+ {
134
+ "name": "palindrake11",
135
+ "nll": 28.67673110961914
136
+ },
137
+ {
138
+ "name": "lolitabimbo",
139
+ "nll": 23.375654220581055
140
+ },
141
+ {
142
+ "name": "fuglyflamingo",
143
+ "nll": 23.503711700439453
144
+ },
145
+ {
146
+ "name": "bouncnbobbyhill",
147
+ "nll": 41.353389739990234
148
+ },
149
+ {
150
+ "name": "uli34497",
151
+ "nll": 29.300373077392578
152
+ },
153
+ {
154
+ "name": "JIkayame",
155
+ "nll": 30.994003295898438
156
+ },
157
+ {
158
+ "name": "Modagiri",
159
+ "nll": 22.96101188659668
160
+ },
161
+ {
162
+ "name": "amirhmd",
163
+ "nll": 18.93401336669922
164
+ },
165
+ {
166
+ "name": "SamShinwari",
167
+ "nll": 24.192853927612305
168
+ },
169
+ {
170
+ "name": "wnhsdoymr",
171
+ "nll": 37.624290466308594
172
+ },
173
+ {
174
+ "name": "HH0608",
175
+ "nll": 22.612565994262695
176
+ },
177
+ {
178
+ "name": "wowkatherine",
179
+ "nll": 21.21406364440918
180
+ },
181
+ {
182
+ "name": "Supportwork19",
183
+ "nll": 29.271455764770508
184
+ },
185
+ {
186
+ "name": "Liverpool1",
187
+ "nll": 20.20630645751953
188
+ },
189
+ {
190
+ "name": "pantera-blue-1",
191
+ "nll": 27.402545928955078
192
+ },
193
+ {
194
+ "name": "DangerousApple69",
195
+ "nll": 26.936904907226562
196
+ },
197
+ {
198
+ "name": "msw678",
199
+ "nll": 21.996488571166992
200
+ },
201
+ {
202
+ "name": "felipe0321",
203
+ "nll": 20.853574752807617
204
+ },
205
+ {
206
+ "name": "mavygarcia",
207
+ "nll": 22.310644149780273
208
+ },
209
+ {
210
+ "name": "v0laozl302",
211
+ "nll": 36.445804595947266
212
+ },
213
+ {
214
+ "name": "futsokhufkua34",
215
+ "nll": 53.09674072265625
216
+ },
217
+ {
218
+ "name": "phnghil",
219
+ "nll": 21.09217071533203
220
+ },
221
+ {
222
+ "name": "moulyagowdadn",
223
+ "nll": 30.032882690429688
224
+ },
225
+ {
226
+ "name": "kstudt",
227
+ "nll": 20.564844131469727
228
+ },
229
+ {
230
+ "name": "jojokokoas",
231
+ "nll": 27.420255661010742
232
+ },
233
+ {
234
+ "name": "angelicahm",
235
+ "nll": 20.051002502441406
236
+ },
237
+ {
238
+ "name": "fishman_d",
239
+ "nll": 22.714061737060547
240
+ },
241
+ {
242
+ "name": "plantobsessedrn",
243
+ "nll": 33.5780029296875
244
+ },
245
+ {
246
+ "name": "sanjivkundamal1",
247
+ "nll": 36.262447357177734
248
+ },
249
+ {
250
+ "name": "zbgamelt",
251
+ "nll": 29.223155975341797
252
+ },
253
+ {
254
+ "name": "jdrnbsoundz1",
255
+ "nll": 40.41189193725586
256
+ },
257
+ {
258
+ "name": "amlinn",
259
+ "nll": 19.536300659179688
260
+ }
261
+ ],
262
+ "bits_per_char": 3.7741001274348203
263
+ }
g2-6x256-a09/manifest.json ADDED
@@ -0,0 +1,181 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "config": {
3
+ "model": {
4
+ "n_layer": 6,
5
+ "d_model": 256,
6
+ "n_head": 4,
7
+ "d_mlp": 704
8
+ },
9
+ "train": {
10
+ "seed": 1337,
11
+ "batch": 512,
12
+ "block": 128,
13
+ "iters": 40000,
14
+ "lr": 0.001,
15
+ "min_lr": 0.0001,
16
+ "warmup": 500,
17
+ "decay_frac": 0.2,
18
+ "weight_decay": 0.1,
19
+ "grad_clip": 1.0,
20
+ "beta1": 0.9,
21
+ "beta2": 0.95,
22
+ "log_every": 50,
23
+ "eval_every": 1000,
24
+ "eval_names": 20000,
25
+ "ckpt_every": 1000,
26
+ "compile": true,
27
+ "dtype": "bfloat16",
28
+ "device": "auto"
29
+ },
30
+ "data": {
31
+ "train": "/root/data/name-dreamer/prep-generic2d/train.bin",
32
+ "heldout": "/root/data/name-dreamer/prep-generic2d/heldout.bin"
33
+ },
34
+ "run": {
35
+ "name": "g2-6x256-a09",
36
+ "out": "/root/models/name-dreamer",
37
+ "resume": true
38
+ },
39
+ "wandb": {
40
+ "project": "name-dreamer",
41
+ "mode": "online",
42
+ "group": "E029",
43
+ "job_type": "train",
44
+ "tags": [
45
+ "base",
46
+ "3x1024"
47
+ ]
48
+ },
49
+ "distill": {
50
+ "teacher": "",
51
+ "alpha": 0.9,
52
+ "temperature": 1.0,
53
+ "precomputed": "/root/models/name-dreamer/teach2-16x1024"
54
+ }
55
+ },
56
+ "model": {
57
+ "n_layer": 6,
58
+ "d_model": 256,
59
+ "n_head": 4,
60
+ "d_mlp": 704,
61
+ "vocab": 256,
62
+ "max_pos": 65,
63
+ "ngram_buckets": 0,
64
+ "ngram_orders": [
65
+ 2,
66
+ 3
67
+ ]
68
+ },
69
+ "n_params": 4902400,
70
+ "git": "unknown",
71
+ "bins": {
72
+ "/root/data/name-dreamer/prep-generic2d/train.bin": "e00c076116bf034ae78f031dc841d540504fd0846a5b42cc49724aca3a62f6c6",
73
+ "/root/data/name-dreamer/prep-generic2d/heldout.bin": "473a7b6eacb86eb633448ce8fa7c9fdec39ae76041ca0f9e7f95aa49f7f0811d"
74
+ },
75
+ "train_tokens": 2237398422,
76
+ "tokens_per_iter": 65536,
77
+ "device": "cuda",
78
+ "world_size": 1,
79
+ "flops_per_token": 31773696,
80
+ "peak_flops_per_gpu": 209500000000000.0,
81
+ "host": "d2825c83c455",
82
+ "torch": "2.14.0+cu130",
83
+ "started": "2026-09-19T08:12:10",
84
+ "iter": 40000,
85
+ "teacher": {
86
+ "precomputed": "/root/models/name-dreamer/teach2-16x1024",
87
+ "teacher": "/root/models/name-dreamer/teacher2-16x1024/ckpt.pt",
88
+ "iters": 40000,
89
+ "k": 66,
90
+ "world": 8,
91
+ "seed": 1337,
92
+ "batch": 512,
93
+ "block": 128,
94
+ "train": "/root/data/name-dreamer/prep-generic2d/train.bin",
95
+ "vocab": 256,
96
+ "alphabet": [
97
+ 0,
98
+ 45,
99
+ 46,
100
+ 48,
101
+ 49,
102
+ 50,
103
+ 51,
104
+ 52,
105
+ 53,
106
+ 54,
107
+ 55,
108
+ 56,
109
+ 57,
110
+ 65,
111
+ 66,
112
+ 67,
113
+ 68,
114
+ 69,
115
+ 70,
116
+ 71,
117
+ 72,
118
+ 73,
119
+ 74,
120
+ 75,
121
+ 76,
122
+ 77,
123
+ 78,
124
+ 79,
125
+ 80,
126
+ 81,
127
+ 82,
128
+ 83,
129
+ 84,
130
+ 85,
131
+ 86,
132
+ 87,
133
+ 88,
134
+ 89,
135
+ 90,
136
+ 95,
137
+ 97,
138
+ 98,
139
+ 99,
140
+ 100,
141
+ 101,
142
+ 102,
143
+ 103,
144
+ 104,
145
+ 105,
146
+ 106,
147
+ 107,
148
+ 108,
149
+ 109,
150
+ 110,
151
+ 111,
152
+ 112,
153
+ 113,
154
+ 114,
155
+ 115,
156
+ 116,
157
+ 117,
158
+ 118,
159
+ 119,
160
+ 120,
161
+ 121,
162
+ 122
163
+ ],
164
+ "segments": [
165
+ {
166
+ "start": 0,
167
+ "end": 40000,
168
+ "tag": ""
169
+ }
170
+ ],
171
+ "alpha": 0.9
172
+ },
173
+ "finished": "2026-09-19T08:29:54",
174
+ "seconds": 1060,
175
+ "tokens_per_s": 2471899.782657222,
176
+ "mfu": 0.37489924695282406,
177
+ "final": {
178
+ "bits_per_char": 3.5260695237608473,
179
+ "loss": 0.30345994234085083
180
+ }
181
+ }
g2-6x256-a09/model-int7.ndq ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:a1e20cd24b1fd0e15d174de9ea8ccf755aebd1af60a94282eea0b7aeb3ed7916
3
+ size 5267500
g2-6x256-a09/quant-int4-g32.json ADDED
@@ -0,0 +1,49 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "bits": 4,
3
+ "group": 32,
4
+ "quantized_params": 4899072,
5
+ "full_precision_params": 3328,
6
+ "bytes": 2762384,
7
+ "bits_per_char_fp": 3.530232185826643,
8
+ "bits_per_char": 3.5838607242210694,
9
+ "n_heldout": 200000,
10
+ "samples": {
11
+ "n": 10000,
12
+ "temperature": 1.0,
13
+ "valid": 0.9996,
14
+ "unique": 1.0,
15
+ "novel": 0.9136654661864746,
16
+ "unterminated": 0,
17
+ "length_hist": {
18
+ "3": 10,
19
+ "4": 61,
20
+ "5": 278,
21
+ "6": 576,
22
+ "7": 972,
23
+ "8": 1110,
24
+ "9": 1129,
25
+ "10": 1093,
26
+ "11": 904,
27
+ "12": 918,
28
+ "13": 673,
29
+ "14": 555,
30
+ "15": 454,
31
+ "16": 341,
32
+ "17": 265,
33
+ "18": 182,
34
+ "19": 131,
35
+ "20": 102,
36
+ "21": 80,
37
+ "22": 44,
38
+ "23": 30,
39
+ "24": 25,
40
+ "25": 24,
41
+ "26": 11,
42
+ "27": 6,
43
+ "28": 6,
44
+ "29": 4,
45
+ "30": 12
46
+ }
47
+ },
48
+ "ckpt": "/root/models/name-dreamer/g2-6x256-a09/quant-int4-g32.pt"
49
+ }
g2-6x256-a09/quant-int4-g32.pt ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:868ab83db975af419cb046c581cdc9b908245d97f532c3903268e88f81325ddb
3
+ size 19624193
g2-6x256-a09/quant-int4-g64.json ADDED
@@ -0,0 +1,49 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "bits": 4,
3
+ "group": 64,
4
+ "quantized_params": 4899072,
5
+ "full_precision_params": 3328,
6
+ "bytes": 2609288,
7
+ "bits_per_char_fp": 3.530232185826643,
8
+ "bits_per_char": 3.5992925572907546,
9
+ "n_heldout": 200000,
10
+ "samples": {
11
+ "n": 10000,
12
+ "temperature": 1.0,
13
+ "valid": 0.9993,
14
+ "unique": 1.0,
15
+ "novel": 0.9170419293505454,
16
+ "unterminated": 0,
17
+ "length_hist": {
18
+ "3": 8,
19
+ "4": 58,
20
+ "5": 300,
21
+ "6": 672,
22
+ "7": 1060,
23
+ "8": 1123,
24
+ "9": 1151,
25
+ "10": 1087,
26
+ "11": 915,
27
+ "12": 859,
28
+ "13": 692,
29
+ "14": 577,
30
+ "15": 405,
31
+ "16": 291,
32
+ "17": 193,
33
+ "18": 159,
34
+ "19": 127,
35
+ "20": 102,
36
+ "21": 61,
37
+ "22": 40,
38
+ "23": 27,
39
+ "24": 27,
40
+ "25": 21,
41
+ "26": 11,
42
+ "27": 6,
43
+ "28": 4,
44
+ "29": 6,
45
+ "30": 11
46
+ }
47
+ },
48
+ "ckpt": "/root/models/name-dreamer/g2-6x256-a09/quant-int4-g64.pt"
49
+ }
g2-6x256-a09/quant-int4-g64.pt ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:d6c5c7a0d620127e146a406dda9fc8edea40494d91b1c5ca5c4077acc389f30a
3
+ size 19624193
g2-6x256-a09/quant-int8-g64.json ADDED
@@ -0,0 +1,49 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "bits": 8,
3
+ "group": 64,
4
+ "quantized_params": 4899072,
5
+ "full_precision_params": 3328,
6
+ "bytes": 5058824,
7
+ "bits_per_char_fp": 3.530232185826643,
8
+ "bits_per_char": 3.530395988423761,
9
+ "n_heldout": 200000,
10
+ "samples": {
11
+ "n": 10000,
12
+ "temperature": 1.0,
13
+ "valid": 0.9993,
14
+ "unique": 1.0,
15
+ "novel": 0.9215450815570899,
16
+ "unterminated": 0,
17
+ "length_hist": {
18
+ "3": 10,
19
+ "4": 47,
20
+ "5": 209,
21
+ "6": 537,
22
+ "7": 841,
23
+ "8": 1041,
24
+ "9": 1084,
25
+ "10": 1135,
26
+ "11": 1009,
27
+ "12": 843,
28
+ "13": 806,
29
+ "14": 649,
30
+ "15": 478,
31
+ "16": 389,
32
+ "17": 263,
33
+ "18": 204,
34
+ "19": 137,
35
+ "20": 98,
36
+ "21": 57,
37
+ "22": 46,
38
+ "23": 33,
39
+ "24": 21,
40
+ "25": 17,
41
+ "26": 11,
42
+ "27": 6,
43
+ "28": 5,
44
+ "29": 5,
45
+ "30": 12
46
+ }
47
+ },
48
+ "ckpt": "/root/models/name-dreamer/g2-6x256-a09/quant-int8-g64.pt"
49
+ }
g2-6x256-a09/quant-int8-g64.pt ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:dc61f83246c52c41f3e178d82c25274ec8b01352756122763693dda2d2353364
3
+ size 19624193
g2-6x256-a09/samples.txt ADDED
@@ -0,0 +1,50 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ SirDumbBXiT
2
+ arpan78777b358
3
+ corko05
4
+ RADICALTuNTHU
5
+ ratfishkeepus
6
+ narcissanoadexi
7
+ jdjl24
8
+ draculadave
9
+ imnotbh
10
+ riseoncrea78975
11
+ b77mini007
12
+ r-dgospin
13
+ frenxrlasasuitwerk
14
+ asjcghetto
15
+ revdair_review
16
+ ThatWubsFineface
17
+ sunmtthaithos
18
+ maiautomaticscience
19
+ Maestro02577
20
+ mashetisl
21
+ extra-aprendiendo
22
+ upmkjatin
23
+ squiresarah
24
+ celerydynasty
25
+ wmcken11
26
+ wgcdm
27
+ morganerer
28
+ mannyopina
29
+ kayz_n
30
+ pachisetonomi
31
+ lanesselli
32
+ askmywordinghard
33
+ giff-at-peace
34
+ simple-littles-butt
35
+ ok22page
36
+ ahmetbru32
37
+ ziyanovajulia
38
+ wailingpark
39
+ gptuzerjit
40
+ kannn03
41
+ Jodie811
42
+ alvinmaki
43
+ weixin_41206003
44
+ muhammadali-work
45
+ hoopshorn
46
+ 94FreLtW1L
47
+ danieleost
48
+ tommyiclub
49
+ ilikemastermind
50
+ sjstcy