g2-6x256-a09: 3.5280 bits/char
Browse files- README.md +1 -0
- g2-6x256-a09/README.md +70 -0
- g2-6x256-a09/ckpt.pt +3 -0
- g2-6x256-a09/eval.json +98 -0
- g2-6x256-a09/golden-int7.json +263 -0
- g2-6x256-a09/manifest.json +181 -0
- g2-6x256-a09/model-int7.ndq +3 -0
- g2-6x256-a09/quant-int4-g32.json +49 -0
- g2-6x256-a09/quant-int4-g32.pt +3 -0
- g2-6x256-a09/quant-int4-g64.json +49 -0
- g2-6x256-a09/quant-int4-g64.pt +3 -0
- g2-6x256-a09/quant-int8-g64.json +49 -0
- g2-6x256-a09/quant-int8-g64.pt +3 -0
- g2-6x256-a09/samples.txt +50 -0
README.md
CHANGED
|
@@ -30,6 +30,7 @@ Vocabulary is 256 bytes with byte 0 as the stop token; the held-out split is
|
|
| 30 |
| [g2-8x256-a09](g2-8x256-a09) | 8×256 | 6.5M | 40,000 | 3.5020 | 92% |
|
| 31 |
| [g2-12x192-a09](g2-12x192-a09) | 12×192 | 5.5M | 40,000 | 3.5074 | 92% |
|
| 32 |
| [g2-4x384-a09](g2-4x384-a09) | 4×384 | 7.2M | 40,000 | 3.5193 | 92% |
|
|
|
|
| 33 |
| [prod-8x768-phase2](prod-8x768-phase2) | 8×768 | 56.9M | 60,000 | 3.5619 | 93% |
|
| 34 |
| [teacher-16x1024](teacher-16x1024) | 16×1024 | 205.9M | 9,000 | 3.6068 | 94% |
|
| 35 |
| [pre-6x640-a09](pre-6x640-a09) | 6×640 | 30.7M | 20,000 | 3.6137 | 94% |
|
|
|
|
| 30 |
| [g2-8x256-a09](g2-8x256-a09) | 8×256 | 6.5M | 40,000 | 3.5020 | 92% |
|
| 31 |
| [g2-12x192-a09](g2-12x192-a09) | 12×192 | 5.5M | 40,000 | 3.5074 | 92% |
|
| 32 |
| [g2-4x384-a09](g2-4x384-a09) | 4×384 | 7.2M | 40,000 | 3.5193 | 92% |
|
| 33 |
+
| [g2-6x256-a09](g2-6x256-a09) | 6×256 | 4.9M | 40,000 | 3.5280 | 92% |
|
| 34 |
| [prod-8x768-phase2](prod-8x768-phase2) | 8×768 | 56.9M | 60,000 | 3.5619 | 93% |
|
| 35 |
| [teacher-16x1024](teacher-16x1024) | 16×1024 | 205.9M | 9,000 | 3.6068 | 94% |
|
| 36 |
| [pre-6x640-a09](pre-6x640-a09) | 6×640 | 30.7M | 20,000 | 3.6137 | 94% |
|
g2-6x256-a09/README.md
ADDED
|
@@ -0,0 +1,70 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# g2-6x256-a09
|
| 2 |
+
|
| 3 |
+
A byte-level GPT that dreams up login-style usernames. 6 layers × 256 wide,
|
| 4 |
+
4 heads, SwiGLU 704, 4,902,400 parameters, vocabulary of 256 bytes with
|
| 5 |
+
byte 0 as the stop token. Trained for 40,000 iterations of 512 × 128 bytes
|
| 6 |
+
on the `ks46/usernames` corpus (185,925,737 unique usernames from 18 public sources).
|
| 7 |
+
|
| 8 |
+
## Quality
|
| 9 |
+
|
| 10 |
+
| metric | value |
|
| 11 |
+
|---|---|
|
| 12 |
+
| held-out bits per char (2,907,216 names) | **3.5280** |
|
| 13 |
+
| samples valid | 99.9% |
|
| 14 |
+
| samples unique | 100.0% |
|
| 15 |
+
| samples novel (absent from the corpus) | 92.2% |
|
| 16 |
+
| unterminated draws | 0 of 10,000 |
|
| 17 |
+
|
| 18 |
+
Per-source held-out bits per char (lower is easier):
|
| 19 |
+
|
| 20 |
+
| source | bits/char |
|
| 21 |
+
|---|---|
|
| 22 |
+
| kaggle | 3.21 |
|
| 23 |
+
| urls_full | 3.31 |
|
| 24 |
+
| jeanphorn | 3.35 |
|
| 25 |
+
| urls | 3.50 |
|
| 26 |
+
| atlas | 3.56 |
|
| 27 |
+
| reddit | 3.56 |
|
| 28 |
+
| sentiment140 | 3.62 |
|
| 29 |
+
| bsky | 3.65 |
|
| 30 |
+
| stack | 3.71 |
|
| 31 |
+
| hn | 3.72 |
|
| 32 |
+
| gharchive | 3.74 |
|
| 33 |
+
| codeforces | 3.76 |
|
| 34 |
+
| crates | 3.76 |
|
| 35 |
+
| lichess | 3.90 |
|
| 36 |
+
| enwiki | 3.90 |
|
| 37 |
+
| wikis | 3.98 |
|
| 38 |
+
| xato | 4.02 |
|
| 39 |
+
| osm | 4.03 |
|
| 40 |
+
| se | 4.07 |
|
| 41 |
+
| roblox | 4.13 |
|
| 42 |
+
| so | 4.16 |
|
| 43 |
+
| ssa | 4.39 |
|
| 44 |
+
|
| 45 |
+
## Quantized variants
|
| 46 |
+
|
| 47 |
+
Post-training, symmetric per-group absmax on every 2D weight:
|
| 48 |
+
|
| 49 |
+
| format | held-out bits/char | Δ vs fp32 | novel | size |
|
| 50 |
+
|---|---|---|---|---|
|
| 51 |
+
| int8 g64 | 3.5304 | +0.0002 | 0.922 | 5.1 MB |
|
| 52 |
+
| int4 g64 | 3.5993 | +0.0691 | 0.917 | 2.6 MB |
|
| 53 |
+
| int4 g32 | 3.5839 | +0.0536 | 0.914 | 2.8 MB |
|
| 54 |
+
|
| 55 |
+
## Files
|
| 56 |
+
|
| 57 |
+
- `ckpt.pt`: the fp32 checkpoint (`model`, `optimizer`, `config`, `iter`).
|
| 58 |
+
- `model.ndq`: int8 (group 64) export for the Rust kernel; `model-int4.ndq` the 4-bit one (half the bytes, 2.4× faster at batch 1, +0.01 bits/char); `golden*.json` hold Python reference NLLs for 64 held-out names.
|
| 59 |
+
- `ndgen`: the kernel, built for x86-64 with AVX2 (`-C target-cpu=native` on an EPYC 7763); rebuild from `rust/ndkernel` for another host.
|
| 60 |
+
- `quant-*.pt`: fake-quantized checkpoints loadable like `ckpt.pt`.
|
| 61 |
+
- `eval.json`, `manifest.json`, `samples.txt`.
|
| 62 |
+
|
| 63 |
+
## Sampling
|
| 64 |
+
|
| 65 |
+
```
|
| 66 |
+
uv run python -m training.sample --ckpt ckpt.pt -n 20 --prefix dark --temperature 0.9 --novel --parquet names.parquet
|
| 67 |
+
./ndgen gen --model model.ndq -n 1000 --batch 64 --temperature 0.9 --prefix dark
|
| 68 |
+
```
|
| 69 |
+
|
| 70 |
+
Sample names: SirDumbBXiT, arpan78777b358, corko05, RADICALTuNTHU, ratfishkeepus, narcissanoadexi, jdjl24, draculadave, imnotbh, riseoncrea78975, b77mini007, r-dgospin, frenxrlasasuitwerk, asjcghetto, revdair_review, ThatWubsFineface, sunmtthaithos, maiautomaticscience, Maestro02577, mashetisl, extra-aprendiendo, upmkjatin, squiresarah, celerydynasty, wmcken11, wgcdm, morganerer, mannyopina, kayz_n, pachisetonomi
|
g2-6x256-a09/ckpt.pt
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:a8b8f7b22ab5869d7ddbb7c70db03754b2809a21527a2d8eb8b9dbcd313ce0a6
|
| 3 |
+
size 58875241
|
g2-6x256-a09/eval.json
ADDED
|
@@ -0,0 +1,98 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"ckpt": "/root/models/name-dreamer/g2-6x256-a09/ckpt.pt",
|
| 3 |
+
"n_heldout": 2907216,
|
| 4 |
+
"vocab": 256,
|
| 5 |
+
"bits_per_char": 3.527974167017619,
|
| 6 |
+
"bits_per_char_by_source": {
|
| 7 |
+
"xato": 4.024610630428509,
|
| 8 |
+
"bsky": 3.6505540727044035,
|
| 9 |
+
"stack": 3.7067097712006425,
|
| 10 |
+
"hn": 3.7155356638345984,
|
| 11 |
+
"reddit": 3.563161271622452,
|
| 12 |
+
"urls": 3.500879085183282,
|
| 13 |
+
"roblox": 4.1338934981412025,
|
| 14 |
+
"crates": 3.7624159077590233,
|
| 15 |
+
"kaggle": 3.209455248745971,
|
| 16 |
+
"gharchive": 3.737931321678688,
|
| 17 |
+
"codeforces": 3.761002710649131,
|
| 18 |
+
"lichess": 3.902432314717484,
|
| 19 |
+
"jeanphorn": 3.3538238767109734,
|
| 20 |
+
"so": 4.161915766841152,
|
| 21 |
+
"se": 4.06589782259505,
|
| 22 |
+
"osm": 4.034289021776484,
|
| 23 |
+
"enwiki": 3.902861109868001,
|
| 24 |
+
"ssa": 4.386260731585523,
|
| 25 |
+
"wikis": 3.983552995602469,
|
| 26 |
+
"sentiment140": 3.6152563296024116,
|
| 27 |
+
"urls_full": 3.3085488906840497,
|
| 28 |
+
"atlas": 3.556889257298462
|
| 29 |
+
},
|
| 30 |
+
"samples": {
|
| 31 |
+
"n": 10000,
|
| 32 |
+
"temperature": 1.0,
|
| 33 |
+
"valid": 0.9994,
|
| 34 |
+
"unique": 1.0,
|
| 35 |
+
"novel": 0.9221532919751851,
|
| 36 |
+
"unterminated": 0,
|
| 37 |
+
"length_hist": {
|
| 38 |
+
"3": 10,
|
| 39 |
+
"4": 48,
|
| 40 |
+
"5": 207,
|
| 41 |
+
"6": 531,
|
| 42 |
+
"7": 833,
|
| 43 |
+
"8": 1042,
|
| 44 |
+
"9": 1083,
|
| 45 |
+
"10": 1117,
|
| 46 |
+
"11": 1010,
|
| 47 |
+
"12": 850,
|
| 48 |
+
"13": 809,
|
| 49 |
+
"14": 653,
|
| 50 |
+
"15": 474,
|
| 51 |
+
"16": 400,
|
| 52 |
+
"17": 269,
|
| 53 |
+
"18": 200,
|
| 54 |
+
"19": 134,
|
| 55 |
+
"20": 102,
|
| 56 |
+
"21": 58,
|
| 57 |
+
"22": 47,
|
| 58 |
+
"23": 33,
|
| 59 |
+
"24": 25,
|
| 60 |
+
"25": 19,
|
| 61 |
+
"26": 8,
|
| 62 |
+
"27": 7,
|
| 63 |
+
"28": 7,
|
| 64 |
+
"29": 4,
|
| 65 |
+
"30": 14
|
| 66 |
+
}
|
| 67 |
+
},
|
| 68 |
+
"heldout_length_hist": {
|
| 69 |
+
"3": 1654,
|
| 70 |
+
"4": 14562,
|
| 71 |
+
"5": 58539,
|
| 72 |
+
"6": 156177,
|
| 73 |
+
"7": 244141,
|
| 74 |
+
"8": 303986,
|
| 75 |
+
"9": 307229,
|
| 76 |
+
"10": 320933,
|
| 77 |
+
"11": 288131,
|
| 78 |
+
"12": 262342,
|
| 79 |
+
"13": 224141,
|
| 80 |
+
"14": 191139,
|
| 81 |
+
"15": 155943,
|
| 82 |
+
"16": 103974,
|
| 83 |
+
"17": 76941,
|
| 84 |
+
"18": 60685,
|
| 85 |
+
"19": 41171,
|
| 86 |
+
"20": 34156,
|
| 87 |
+
"21": 14710,
|
| 88 |
+
"22": 12154,
|
| 89 |
+
"23": 9023,
|
| 90 |
+
"24": 7057,
|
| 91 |
+
"25": 5359,
|
| 92 |
+
"26": 3899,
|
| 93 |
+
"27": 3097,
|
| 94 |
+
"28": 2341,
|
| 95 |
+
"29": 1886,
|
| 96 |
+
"30": 1846
|
| 97 |
+
}
|
| 98 |
+
}
|
g2-6x256-a09/golden-int7.json
ADDED
|
@@ -0,0 +1,263 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"group": 64,
|
| 3 |
+
"bits": 7,
|
| 4 |
+
"names": [
|
| 5 |
+
{
|
| 6 |
+
"name": "holesk",
|
| 7 |
+
"nll": 20.395837783813477
|
| 8 |
+
},
|
| 9 |
+
{
|
| 10 |
+
"name": "giorgiaoikonomou",
|
| 11 |
+
"nll": 29.2760009765625
|
| 12 |
+
},
|
| 13 |
+
{
|
| 14 |
+
"name": "Pranav1291",
|
| 15 |
+
"nll": 21.900712966918945
|
| 16 |
+
},
|
| 17 |
+
{
|
| 18 |
+
"name": "washindanzel28",
|
| 19 |
+
"nll": 40.156639099121094
|
| 20 |
+
},
|
| 21 |
+
{
|
| 22 |
+
"name": "Matikz",
|
| 23 |
+
"nll": 20.289161682128906
|
| 24 |
+
},
|
| 25 |
+
{
|
| 26 |
+
"name": "Caboose811",
|
| 27 |
+
"nll": 23.758569717407227
|
| 28 |
+
},
|
| 29 |
+
{
|
| 30 |
+
"name": "OntongGhebi",
|
| 31 |
+
"nll": 38.82344436645508
|
| 32 |
+
},
|
| 33 |
+
{
|
| 34 |
+
"name": "askmikeaquestion",
|
| 35 |
+
"nll": 30.52001953125
|
| 36 |
+
},
|
| 37 |
+
{
|
| 38 |
+
"name": "bakkeren",
|
| 39 |
+
"nll": 21.65003776550293
|
| 40 |
+
},
|
| 41 |
+
{
|
| 42 |
+
"name": "reem_alattas",
|
| 43 |
+
"nll": 28.397186279296875
|
| 44 |
+
},
|
| 45 |
+
{
|
| 46 |
+
"name": "icebsj",
|
| 47 |
+
"nll": 23.52632713317871
|
| 48 |
+
},
|
| 49 |
+
{
|
| 50 |
+
"name": "frillywitch",
|
| 51 |
+
"nll": 20.62656593322754
|
| 52 |
+
},
|
| 53 |
+
{
|
| 54 |
+
"name": "Kledoro",
|
| 55 |
+
"nll": 24.570323944091797
|
| 56 |
+
},
|
| 57 |
+
{
|
| 58 |
+
"name": "dtaivpp_1463610",
|
| 59 |
+
"nll": 53.46287155151367
|
| 60 |
+
},
|
| 61 |
+
{
|
| 62 |
+
"name": "brayannadjan",
|
| 63 |
+
"nll": 29.05016326904297
|
| 64 |
+
},
|
| 65 |
+
{
|
| 66 |
+
"name": "edibleethicscom",
|
| 67 |
+
"nll": 32.94371032714844
|
| 68 |
+
},
|
| 69 |
+
{
|
| 70 |
+
"name": "jtvcorner",
|
| 71 |
+
"nll": 23.736902236938477
|
| 72 |
+
},
|
| 73 |
+
{
|
| 74 |
+
"name": "ivansorano",
|
| 75 |
+
"nll": 22.51315689086914
|
| 76 |
+
},
|
| 77 |
+
{
|
| 78 |
+
"name": "Reshantika",
|
| 79 |
+
"nll": 27.969207763671875
|
| 80 |
+
},
|
| 81 |
+
{
|
| 82 |
+
"name": "dameq1",
|
| 83 |
+
"nll": 22.817520141601562
|
| 84 |
+
},
|
| 85 |
+
{
|
| 86 |
+
"name": "gergo-bossanyi",
|
| 87 |
+
"nll": 37.7056999206543
|
| 88 |
+
},
|
| 89 |
+
{
|
| 90 |
+
"name": "natanaeladolfsson",
|
| 91 |
+
"nll": 31.273340225219727
|
| 92 |
+
},
|
| 93 |
+
{
|
| 94 |
+
"name": "dark.drakus",
|
| 95 |
+
"nll": 28.281692504882812
|
| 96 |
+
},
|
| 97 |
+
{
|
| 98 |
+
"name": "gingerkuenzel",
|
| 99 |
+
"nll": 27.299684524536133
|
| 100 |
+
},
|
| 101 |
+
{
|
| 102 |
+
"name": "legalamlawfirm",
|
| 103 |
+
"nll": 31.868541717529297
|
| 104 |
+
},
|
| 105 |
+
{
|
| 106 |
+
"name": "eternallyraven",
|
| 107 |
+
"nll": 20.606975555419922
|
| 108 |
+
},
|
| 109 |
+
{
|
| 110 |
+
"name": "kishifuji4729",
|
| 111 |
+
"nll": 33.95426559448242
|
| 112 |
+
},
|
| 113 |
+
{
|
| 114 |
+
"name": "szkhrm",
|
| 115 |
+
"nll": 23.686298370361328
|
| 116 |
+
},
|
| 117 |
+
{
|
| 118 |
+
"name": "lingnoi16",
|
| 119 |
+
"nll": 28.14520835876465
|
| 120 |
+
},
|
| 121 |
+
{
|
| 122 |
+
"name": "jaketui",
|
| 123 |
+
"nll": 21.841720581054688
|
| 124 |
+
},
|
| 125 |
+
{
|
| 126 |
+
"name": "devon4",
|
| 127 |
+
"nll": 19.226409912109375
|
| 128 |
+
},
|
| 129 |
+
{
|
| 130 |
+
"name": "fami0603",
|
| 131 |
+
"nll": 23.99629783630371
|
| 132 |
+
},
|
| 133 |
+
{
|
| 134 |
+
"name": "palindrake11",
|
| 135 |
+
"nll": 28.67673110961914
|
| 136 |
+
},
|
| 137 |
+
{
|
| 138 |
+
"name": "lolitabimbo",
|
| 139 |
+
"nll": 23.375654220581055
|
| 140 |
+
},
|
| 141 |
+
{
|
| 142 |
+
"name": "fuglyflamingo",
|
| 143 |
+
"nll": 23.503711700439453
|
| 144 |
+
},
|
| 145 |
+
{
|
| 146 |
+
"name": "bouncnbobbyhill",
|
| 147 |
+
"nll": 41.353389739990234
|
| 148 |
+
},
|
| 149 |
+
{
|
| 150 |
+
"name": "uli34497",
|
| 151 |
+
"nll": 29.300373077392578
|
| 152 |
+
},
|
| 153 |
+
{
|
| 154 |
+
"name": "JIkayame",
|
| 155 |
+
"nll": 30.994003295898438
|
| 156 |
+
},
|
| 157 |
+
{
|
| 158 |
+
"name": "Modagiri",
|
| 159 |
+
"nll": 22.96101188659668
|
| 160 |
+
},
|
| 161 |
+
{
|
| 162 |
+
"name": "amirhmd",
|
| 163 |
+
"nll": 18.93401336669922
|
| 164 |
+
},
|
| 165 |
+
{
|
| 166 |
+
"name": "SamShinwari",
|
| 167 |
+
"nll": 24.192853927612305
|
| 168 |
+
},
|
| 169 |
+
{
|
| 170 |
+
"name": "wnhsdoymr",
|
| 171 |
+
"nll": 37.624290466308594
|
| 172 |
+
},
|
| 173 |
+
{
|
| 174 |
+
"name": "HH0608",
|
| 175 |
+
"nll": 22.612565994262695
|
| 176 |
+
},
|
| 177 |
+
{
|
| 178 |
+
"name": "wowkatherine",
|
| 179 |
+
"nll": 21.21406364440918
|
| 180 |
+
},
|
| 181 |
+
{
|
| 182 |
+
"name": "Supportwork19",
|
| 183 |
+
"nll": 29.271455764770508
|
| 184 |
+
},
|
| 185 |
+
{
|
| 186 |
+
"name": "Liverpool1",
|
| 187 |
+
"nll": 20.20630645751953
|
| 188 |
+
},
|
| 189 |
+
{
|
| 190 |
+
"name": "pantera-blue-1",
|
| 191 |
+
"nll": 27.402545928955078
|
| 192 |
+
},
|
| 193 |
+
{
|
| 194 |
+
"name": "DangerousApple69",
|
| 195 |
+
"nll": 26.936904907226562
|
| 196 |
+
},
|
| 197 |
+
{
|
| 198 |
+
"name": "msw678",
|
| 199 |
+
"nll": 21.996488571166992
|
| 200 |
+
},
|
| 201 |
+
{
|
| 202 |
+
"name": "felipe0321",
|
| 203 |
+
"nll": 20.853574752807617
|
| 204 |
+
},
|
| 205 |
+
{
|
| 206 |
+
"name": "mavygarcia",
|
| 207 |
+
"nll": 22.310644149780273
|
| 208 |
+
},
|
| 209 |
+
{
|
| 210 |
+
"name": "v0laozl302",
|
| 211 |
+
"nll": 36.445804595947266
|
| 212 |
+
},
|
| 213 |
+
{
|
| 214 |
+
"name": "futsokhufkua34",
|
| 215 |
+
"nll": 53.09674072265625
|
| 216 |
+
},
|
| 217 |
+
{
|
| 218 |
+
"name": "phnghil",
|
| 219 |
+
"nll": 21.09217071533203
|
| 220 |
+
},
|
| 221 |
+
{
|
| 222 |
+
"name": "moulyagowdadn",
|
| 223 |
+
"nll": 30.032882690429688
|
| 224 |
+
},
|
| 225 |
+
{
|
| 226 |
+
"name": "kstudt",
|
| 227 |
+
"nll": 20.564844131469727
|
| 228 |
+
},
|
| 229 |
+
{
|
| 230 |
+
"name": "jojokokoas",
|
| 231 |
+
"nll": 27.420255661010742
|
| 232 |
+
},
|
| 233 |
+
{
|
| 234 |
+
"name": "angelicahm",
|
| 235 |
+
"nll": 20.051002502441406
|
| 236 |
+
},
|
| 237 |
+
{
|
| 238 |
+
"name": "fishman_d",
|
| 239 |
+
"nll": 22.714061737060547
|
| 240 |
+
},
|
| 241 |
+
{
|
| 242 |
+
"name": "plantobsessedrn",
|
| 243 |
+
"nll": 33.5780029296875
|
| 244 |
+
},
|
| 245 |
+
{
|
| 246 |
+
"name": "sanjivkundamal1",
|
| 247 |
+
"nll": 36.262447357177734
|
| 248 |
+
},
|
| 249 |
+
{
|
| 250 |
+
"name": "zbgamelt",
|
| 251 |
+
"nll": 29.223155975341797
|
| 252 |
+
},
|
| 253 |
+
{
|
| 254 |
+
"name": "jdrnbsoundz1",
|
| 255 |
+
"nll": 40.41189193725586
|
| 256 |
+
},
|
| 257 |
+
{
|
| 258 |
+
"name": "amlinn",
|
| 259 |
+
"nll": 19.536300659179688
|
| 260 |
+
}
|
| 261 |
+
],
|
| 262 |
+
"bits_per_char": 3.7741001274348203
|
| 263 |
+
}
|
g2-6x256-a09/manifest.json
ADDED
|
@@ -0,0 +1,181 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"config": {
|
| 3 |
+
"model": {
|
| 4 |
+
"n_layer": 6,
|
| 5 |
+
"d_model": 256,
|
| 6 |
+
"n_head": 4,
|
| 7 |
+
"d_mlp": 704
|
| 8 |
+
},
|
| 9 |
+
"train": {
|
| 10 |
+
"seed": 1337,
|
| 11 |
+
"batch": 512,
|
| 12 |
+
"block": 128,
|
| 13 |
+
"iters": 40000,
|
| 14 |
+
"lr": 0.001,
|
| 15 |
+
"min_lr": 0.0001,
|
| 16 |
+
"warmup": 500,
|
| 17 |
+
"decay_frac": 0.2,
|
| 18 |
+
"weight_decay": 0.1,
|
| 19 |
+
"grad_clip": 1.0,
|
| 20 |
+
"beta1": 0.9,
|
| 21 |
+
"beta2": 0.95,
|
| 22 |
+
"log_every": 50,
|
| 23 |
+
"eval_every": 1000,
|
| 24 |
+
"eval_names": 20000,
|
| 25 |
+
"ckpt_every": 1000,
|
| 26 |
+
"compile": true,
|
| 27 |
+
"dtype": "bfloat16",
|
| 28 |
+
"device": "auto"
|
| 29 |
+
},
|
| 30 |
+
"data": {
|
| 31 |
+
"train": "/root/data/name-dreamer/prep-generic2d/train.bin",
|
| 32 |
+
"heldout": "/root/data/name-dreamer/prep-generic2d/heldout.bin"
|
| 33 |
+
},
|
| 34 |
+
"run": {
|
| 35 |
+
"name": "g2-6x256-a09",
|
| 36 |
+
"out": "/root/models/name-dreamer",
|
| 37 |
+
"resume": true
|
| 38 |
+
},
|
| 39 |
+
"wandb": {
|
| 40 |
+
"project": "name-dreamer",
|
| 41 |
+
"mode": "online",
|
| 42 |
+
"group": "E029",
|
| 43 |
+
"job_type": "train",
|
| 44 |
+
"tags": [
|
| 45 |
+
"base",
|
| 46 |
+
"3x1024"
|
| 47 |
+
]
|
| 48 |
+
},
|
| 49 |
+
"distill": {
|
| 50 |
+
"teacher": "",
|
| 51 |
+
"alpha": 0.9,
|
| 52 |
+
"temperature": 1.0,
|
| 53 |
+
"precomputed": "/root/models/name-dreamer/teach2-16x1024"
|
| 54 |
+
}
|
| 55 |
+
},
|
| 56 |
+
"model": {
|
| 57 |
+
"n_layer": 6,
|
| 58 |
+
"d_model": 256,
|
| 59 |
+
"n_head": 4,
|
| 60 |
+
"d_mlp": 704,
|
| 61 |
+
"vocab": 256,
|
| 62 |
+
"max_pos": 65,
|
| 63 |
+
"ngram_buckets": 0,
|
| 64 |
+
"ngram_orders": [
|
| 65 |
+
2,
|
| 66 |
+
3
|
| 67 |
+
]
|
| 68 |
+
},
|
| 69 |
+
"n_params": 4902400,
|
| 70 |
+
"git": "unknown",
|
| 71 |
+
"bins": {
|
| 72 |
+
"/root/data/name-dreamer/prep-generic2d/train.bin": "e00c076116bf034ae78f031dc841d540504fd0846a5b42cc49724aca3a62f6c6",
|
| 73 |
+
"/root/data/name-dreamer/prep-generic2d/heldout.bin": "473a7b6eacb86eb633448ce8fa7c9fdec39ae76041ca0f9e7f95aa49f7f0811d"
|
| 74 |
+
},
|
| 75 |
+
"train_tokens": 2237398422,
|
| 76 |
+
"tokens_per_iter": 65536,
|
| 77 |
+
"device": "cuda",
|
| 78 |
+
"world_size": 1,
|
| 79 |
+
"flops_per_token": 31773696,
|
| 80 |
+
"peak_flops_per_gpu": 209500000000000.0,
|
| 81 |
+
"host": "d2825c83c455",
|
| 82 |
+
"torch": "2.14.0+cu130",
|
| 83 |
+
"started": "2026-09-19T08:12:10",
|
| 84 |
+
"iter": 40000,
|
| 85 |
+
"teacher": {
|
| 86 |
+
"precomputed": "/root/models/name-dreamer/teach2-16x1024",
|
| 87 |
+
"teacher": "/root/models/name-dreamer/teacher2-16x1024/ckpt.pt",
|
| 88 |
+
"iters": 40000,
|
| 89 |
+
"k": 66,
|
| 90 |
+
"world": 8,
|
| 91 |
+
"seed": 1337,
|
| 92 |
+
"batch": 512,
|
| 93 |
+
"block": 128,
|
| 94 |
+
"train": "/root/data/name-dreamer/prep-generic2d/train.bin",
|
| 95 |
+
"vocab": 256,
|
| 96 |
+
"alphabet": [
|
| 97 |
+
0,
|
| 98 |
+
45,
|
| 99 |
+
46,
|
| 100 |
+
48,
|
| 101 |
+
49,
|
| 102 |
+
50,
|
| 103 |
+
51,
|
| 104 |
+
52,
|
| 105 |
+
53,
|
| 106 |
+
54,
|
| 107 |
+
55,
|
| 108 |
+
56,
|
| 109 |
+
57,
|
| 110 |
+
65,
|
| 111 |
+
66,
|
| 112 |
+
67,
|
| 113 |
+
68,
|
| 114 |
+
69,
|
| 115 |
+
70,
|
| 116 |
+
71,
|
| 117 |
+
72,
|
| 118 |
+
73,
|
| 119 |
+
74,
|
| 120 |
+
75,
|
| 121 |
+
76,
|
| 122 |
+
77,
|
| 123 |
+
78,
|
| 124 |
+
79,
|
| 125 |
+
80,
|
| 126 |
+
81,
|
| 127 |
+
82,
|
| 128 |
+
83,
|
| 129 |
+
84,
|
| 130 |
+
85,
|
| 131 |
+
86,
|
| 132 |
+
87,
|
| 133 |
+
88,
|
| 134 |
+
89,
|
| 135 |
+
90,
|
| 136 |
+
95,
|
| 137 |
+
97,
|
| 138 |
+
98,
|
| 139 |
+
99,
|
| 140 |
+
100,
|
| 141 |
+
101,
|
| 142 |
+
102,
|
| 143 |
+
103,
|
| 144 |
+
104,
|
| 145 |
+
105,
|
| 146 |
+
106,
|
| 147 |
+
107,
|
| 148 |
+
108,
|
| 149 |
+
109,
|
| 150 |
+
110,
|
| 151 |
+
111,
|
| 152 |
+
112,
|
| 153 |
+
113,
|
| 154 |
+
114,
|
| 155 |
+
115,
|
| 156 |
+
116,
|
| 157 |
+
117,
|
| 158 |
+
118,
|
| 159 |
+
119,
|
| 160 |
+
120,
|
| 161 |
+
121,
|
| 162 |
+
122
|
| 163 |
+
],
|
| 164 |
+
"segments": [
|
| 165 |
+
{
|
| 166 |
+
"start": 0,
|
| 167 |
+
"end": 40000,
|
| 168 |
+
"tag": ""
|
| 169 |
+
}
|
| 170 |
+
],
|
| 171 |
+
"alpha": 0.9
|
| 172 |
+
},
|
| 173 |
+
"finished": "2026-09-19T08:29:54",
|
| 174 |
+
"seconds": 1060,
|
| 175 |
+
"tokens_per_s": 2471899.782657222,
|
| 176 |
+
"mfu": 0.37489924695282406,
|
| 177 |
+
"final": {
|
| 178 |
+
"bits_per_char": 3.5260695237608473,
|
| 179 |
+
"loss": 0.30345994234085083
|
| 180 |
+
}
|
| 181 |
+
}
|
g2-6x256-a09/model-int7.ndq
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:a1e20cd24b1fd0e15d174de9ea8ccf755aebd1af60a94282eea0b7aeb3ed7916
|
| 3 |
+
size 5267500
|
g2-6x256-a09/quant-int4-g32.json
ADDED
|
@@ -0,0 +1,49 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"bits": 4,
|
| 3 |
+
"group": 32,
|
| 4 |
+
"quantized_params": 4899072,
|
| 5 |
+
"full_precision_params": 3328,
|
| 6 |
+
"bytes": 2762384,
|
| 7 |
+
"bits_per_char_fp": 3.530232185826643,
|
| 8 |
+
"bits_per_char": 3.5838607242210694,
|
| 9 |
+
"n_heldout": 200000,
|
| 10 |
+
"samples": {
|
| 11 |
+
"n": 10000,
|
| 12 |
+
"temperature": 1.0,
|
| 13 |
+
"valid": 0.9996,
|
| 14 |
+
"unique": 1.0,
|
| 15 |
+
"novel": 0.9136654661864746,
|
| 16 |
+
"unterminated": 0,
|
| 17 |
+
"length_hist": {
|
| 18 |
+
"3": 10,
|
| 19 |
+
"4": 61,
|
| 20 |
+
"5": 278,
|
| 21 |
+
"6": 576,
|
| 22 |
+
"7": 972,
|
| 23 |
+
"8": 1110,
|
| 24 |
+
"9": 1129,
|
| 25 |
+
"10": 1093,
|
| 26 |
+
"11": 904,
|
| 27 |
+
"12": 918,
|
| 28 |
+
"13": 673,
|
| 29 |
+
"14": 555,
|
| 30 |
+
"15": 454,
|
| 31 |
+
"16": 341,
|
| 32 |
+
"17": 265,
|
| 33 |
+
"18": 182,
|
| 34 |
+
"19": 131,
|
| 35 |
+
"20": 102,
|
| 36 |
+
"21": 80,
|
| 37 |
+
"22": 44,
|
| 38 |
+
"23": 30,
|
| 39 |
+
"24": 25,
|
| 40 |
+
"25": 24,
|
| 41 |
+
"26": 11,
|
| 42 |
+
"27": 6,
|
| 43 |
+
"28": 6,
|
| 44 |
+
"29": 4,
|
| 45 |
+
"30": 12
|
| 46 |
+
}
|
| 47 |
+
},
|
| 48 |
+
"ckpt": "/root/models/name-dreamer/g2-6x256-a09/quant-int4-g32.pt"
|
| 49 |
+
}
|
g2-6x256-a09/quant-int4-g32.pt
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:868ab83db975af419cb046c581cdc9b908245d97f532c3903268e88f81325ddb
|
| 3 |
+
size 19624193
|
g2-6x256-a09/quant-int4-g64.json
ADDED
|
@@ -0,0 +1,49 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"bits": 4,
|
| 3 |
+
"group": 64,
|
| 4 |
+
"quantized_params": 4899072,
|
| 5 |
+
"full_precision_params": 3328,
|
| 6 |
+
"bytes": 2609288,
|
| 7 |
+
"bits_per_char_fp": 3.530232185826643,
|
| 8 |
+
"bits_per_char": 3.5992925572907546,
|
| 9 |
+
"n_heldout": 200000,
|
| 10 |
+
"samples": {
|
| 11 |
+
"n": 10000,
|
| 12 |
+
"temperature": 1.0,
|
| 13 |
+
"valid": 0.9993,
|
| 14 |
+
"unique": 1.0,
|
| 15 |
+
"novel": 0.9170419293505454,
|
| 16 |
+
"unterminated": 0,
|
| 17 |
+
"length_hist": {
|
| 18 |
+
"3": 8,
|
| 19 |
+
"4": 58,
|
| 20 |
+
"5": 300,
|
| 21 |
+
"6": 672,
|
| 22 |
+
"7": 1060,
|
| 23 |
+
"8": 1123,
|
| 24 |
+
"9": 1151,
|
| 25 |
+
"10": 1087,
|
| 26 |
+
"11": 915,
|
| 27 |
+
"12": 859,
|
| 28 |
+
"13": 692,
|
| 29 |
+
"14": 577,
|
| 30 |
+
"15": 405,
|
| 31 |
+
"16": 291,
|
| 32 |
+
"17": 193,
|
| 33 |
+
"18": 159,
|
| 34 |
+
"19": 127,
|
| 35 |
+
"20": 102,
|
| 36 |
+
"21": 61,
|
| 37 |
+
"22": 40,
|
| 38 |
+
"23": 27,
|
| 39 |
+
"24": 27,
|
| 40 |
+
"25": 21,
|
| 41 |
+
"26": 11,
|
| 42 |
+
"27": 6,
|
| 43 |
+
"28": 4,
|
| 44 |
+
"29": 6,
|
| 45 |
+
"30": 11
|
| 46 |
+
}
|
| 47 |
+
},
|
| 48 |
+
"ckpt": "/root/models/name-dreamer/g2-6x256-a09/quant-int4-g64.pt"
|
| 49 |
+
}
|
g2-6x256-a09/quant-int4-g64.pt
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:d6c5c7a0d620127e146a406dda9fc8edea40494d91b1c5ca5c4077acc389f30a
|
| 3 |
+
size 19624193
|
g2-6x256-a09/quant-int8-g64.json
ADDED
|
@@ -0,0 +1,49 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"bits": 8,
|
| 3 |
+
"group": 64,
|
| 4 |
+
"quantized_params": 4899072,
|
| 5 |
+
"full_precision_params": 3328,
|
| 6 |
+
"bytes": 5058824,
|
| 7 |
+
"bits_per_char_fp": 3.530232185826643,
|
| 8 |
+
"bits_per_char": 3.530395988423761,
|
| 9 |
+
"n_heldout": 200000,
|
| 10 |
+
"samples": {
|
| 11 |
+
"n": 10000,
|
| 12 |
+
"temperature": 1.0,
|
| 13 |
+
"valid": 0.9993,
|
| 14 |
+
"unique": 1.0,
|
| 15 |
+
"novel": 0.9215450815570899,
|
| 16 |
+
"unterminated": 0,
|
| 17 |
+
"length_hist": {
|
| 18 |
+
"3": 10,
|
| 19 |
+
"4": 47,
|
| 20 |
+
"5": 209,
|
| 21 |
+
"6": 537,
|
| 22 |
+
"7": 841,
|
| 23 |
+
"8": 1041,
|
| 24 |
+
"9": 1084,
|
| 25 |
+
"10": 1135,
|
| 26 |
+
"11": 1009,
|
| 27 |
+
"12": 843,
|
| 28 |
+
"13": 806,
|
| 29 |
+
"14": 649,
|
| 30 |
+
"15": 478,
|
| 31 |
+
"16": 389,
|
| 32 |
+
"17": 263,
|
| 33 |
+
"18": 204,
|
| 34 |
+
"19": 137,
|
| 35 |
+
"20": 98,
|
| 36 |
+
"21": 57,
|
| 37 |
+
"22": 46,
|
| 38 |
+
"23": 33,
|
| 39 |
+
"24": 21,
|
| 40 |
+
"25": 17,
|
| 41 |
+
"26": 11,
|
| 42 |
+
"27": 6,
|
| 43 |
+
"28": 5,
|
| 44 |
+
"29": 5,
|
| 45 |
+
"30": 12
|
| 46 |
+
}
|
| 47 |
+
},
|
| 48 |
+
"ckpt": "/root/models/name-dreamer/g2-6x256-a09/quant-int8-g64.pt"
|
| 49 |
+
}
|
g2-6x256-a09/quant-int8-g64.pt
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:dc61f83246c52c41f3e178d82c25274ec8b01352756122763693dda2d2353364
|
| 3 |
+
size 19624193
|
g2-6x256-a09/samples.txt
ADDED
|
@@ -0,0 +1,50 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
SirDumbBXiT
|
| 2 |
+
arpan78777b358
|
| 3 |
+
corko05
|
| 4 |
+
RADICALTuNTHU
|
| 5 |
+
ratfishkeepus
|
| 6 |
+
narcissanoadexi
|
| 7 |
+
jdjl24
|
| 8 |
+
draculadave
|
| 9 |
+
imnotbh
|
| 10 |
+
riseoncrea78975
|
| 11 |
+
b77mini007
|
| 12 |
+
r-dgospin
|
| 13 |
+
frenxrlasasuitwerk
|
| 14 |
+
asjcghetto
|
| 15 |
+
revdair_review
|
| 16 |
+
ThatWubsFineface
|
| 17 |
+
sunmtthaithos
|
| 18 |
+
maiautomaticscience
|
| 19 |
+
Maestro02577
|
| 20 |
+
mashetisl
|
| 21 |
+
extra-aprendiendo
|
| 22 |
+
upmkjatin
|
| 23 |
+
squiresarah
|
| 24 |
+
celerydynasty
|
| 25 |
+
wmcken11
|
| 26 |
+
wgcdm
|
| 27 |
+
morganerer
|
| 28 |
+
mannyopina
|
| 29 |
+
kayz_n
|
| 30 |
+
pachisetonomi
|
| 31 |
+
lanesselli
|
| 32 |
+
askmywordinghard
|
| 33 |
+
giff-at-peace
|
| 34 |
+
simple-littles-butt
|
| 35 |
+
ok22page
|
| 36 |
+
ahmetbru32
|
| 37 |
+
ziyanovajulia
|
| 38 |
+
wailingpark
|
| 39 |
+
gptuzerjit
|
| 40 |
+
kannn03
|
| 41 |
+
Jodie811
|
| 42 |
+
alvinmaki
|
| 43 |
+
weixin_41206003
|
| 44 |
+
muhammadali-work
|
| 45 |
+
hoopshorn
|
| 46 |
+
94FreLtW1L
|
| 47 |
+
danieleost
|
| 48 |
+
tommyiclub
|
| 49 |
+
ilikemastermind
|
| 50 |
+
sjstcy
|