lemoncmd commited on
Commit
42f3e13
·
verified ·
1 Parent(s): 908022b

Add files using upload-large-folder tool

Browse files
Files changed (50) hide show
  1. README.md +17 -15
  2. frac_0.10/checkpoints/index.csv +19 -0
  3. frac_0.10/checkpoints/last.ckpt +3 -0
  4. frac_0.10/checkpoints/step_0425000.ckpt +3 -0
  5. frac_0.10/checkpoints/step_0450000.ckpt +3 -0
  6. frac_0.10/metrics/version_7/metrics.csv +0 -0
  7. frac_0.10/norms/norms_raw.jsonl +2 -2
  8. frac_0.20/checkpoints/index.csv +18 -0
  9. frac_0.20/checkpoints/last.ckpt +3 -0
  10. frac_0.20/checkpoints/step_0450000.ckpt +3 -0
  11. frac_0.20/metrics/version_7/metrics.csv +0 -0
  12. frac_0.30/checkpoints/index.csv +18 -0
  13. frac_0.30/checkpoints/step_0375000.ckpt +3 -0
  14. frac_0.30/metrics/version_7/metrics.csv +0 -0
  15. frac_0.30/norms/norms_ema.jsonl +2 -2
  16. frac_0.30/norms/norms_raw.jsonl +2 -2
  17. frac_0.40/checkpoints/index.csv +19 -0
  18. frac_0.40/metrics/version_7/metrics.csv +0 -0
  19. frac_0.40/norms/norms_ema.jsonl +2 -2
  20. frac_0.40/norms/norms_raw.jsonl +2 -2
  21. frac_0.50/checkpoints/index.csv +17 -0
  22. frac_0.50/checkpoints/last.ckpt +3 -0
  23. frac_0.50/checkpoints/step_0700000.ckpt +3 -0
  24. frac_0.50/checkpoints/step_0725000.ckpt +3 -0
  25. frac_0.50/metrics/version_7/metrics.csv +0 -0
  26. frac_0.50/norms/norms_ema.jsonl +2 -2
  27. frac_0.50/norms/norms_raw.jsonl +2 -2
  28. frac_0.60/checkpoints/index.csv +21 -0
  29. frac_0.60/checkpoints/last.ckpt +3 -0
  30. frac_0.60/checkpoints/step_0350000.ckpt +3 -0
  31. frac_0.60/checkpoints/step_0375000.ckpt +3 -0
  32. frac_0.60/metrics/version_6/metrics.csv +0 -0
  33. frac_0.60/norms/norms_ema.jsonl +2 -2
  34. frac_0.60/norms/norms_raw.jsonl +2 -2
  35. frac_0.70/checkpoints/index.csv +22 -0
  36. frac_0.70/checkpoints/last.ckpt +3 -0
  37. frac_0.70/checkpoints/step_0375000.ckpt +3 -0
  38. frac_0.70/checkpoints/step_0400000.ckpt +3 -0
  39. frac_0.70/metrics/version_6/metrics.csv +0 -0
  40. frac_0.70/norms/norms_ema.jsonl +2 -2
  41. frac_0.70/norms/norms_raw.jsonl +2 -2
  42. frac_0.80/checkpoints/index.csv +19 -0
  43. frac_0.80/checkpoints/last.ckpt +3 -0
  44. frac_0.80/checkpoints/step_0300000.ckpt +3 -0
  45. frac_0.80/checkpoints/step_0325000.ckpt +3 -0
  46. frac_0.80/metrics/version_6/metrics.csv +0 -0
  47. frac_0.80/norms/norms_ema.jsonl +2 -2
  48. frac_0.80/norms/norms_raw.jsonl +2 -2
  49. frac_0.90/checkpoints/last.ckpt +3 -0
  50. frac_1.00/checkpoints/last.ckpt +3 -0
README.md CHANGED
@@ -15,8 +15,9 @@ tags:
15
 
16
  10 uniform-state discrete diffusion language models (Duo, `duo_base`), identical
17
  except for the size of their training set: nested subsets of OpenWebText holding
18
- 10%, 20%, ..., 100% of the training blocks. **Training is in progress** (target 1,000,000 steps). This repository is a backup of
19
- the grid checkpoints reached so far and will be updated as training continues.
 
20
  Each folder holds **full checkpoints** on a fixed step grid (53 at 1M), so the whole
21
  training trajectory can be analysed (e.g. test loss against weight norm, in the
22
  spirit of D'Amico, Bocchi & Negri, *Implicit bias produces neural scaling laws in
@@ -24,18 +25,18 @@ learning curves*).
24
 
25
  ## Models
26
 
27
- | fraction | training blocks (1,024 tokens) | step | epochs | val NELBO at that step (nats/token) | training hours |
28
- |---|---|---|---|---|---|
29
- | 0.1 | 873,094 | 400,000 | 235 | 3.0205 | 191 |
30
- | 0.2 | 1,746,188 | 425,000 | 125 | 2.9970 | 211 |
31
- | 0.3 | 2,619,283 | 350,000 | 68 | 3.0198 | 174 |
32
- | 0.4 | 3,492,377 | 375,000 | 55 | 3.0056 | 184 |
33
- | 0.5 | 4,365,471 | 675,000 | 79 | 2.9593 | 333 |
34
- | 0.6 | 5,238,565 | 325,000 | 32 | 3.0157 | 160 |
35
- | 0.7 | 6,111,659 | 350,000 | 29 | 3.0056 | 166 |
36
- | 0.8 | 6,984,754 | 275,000 | 20 | 3.0287 | 139 |
37
- | 0.9 | 7,857,848 | 275,000 | 18 | 3.0334 | 135 |
38
- | 1.0 | 8,730,942 | 275,000 | 16 | 3.0420 | 133 |
39
 
40
  Validation NELBO is computed with the EMA weights on the full OpenWebText validation split
41
  (110,464 blocks), with fixed noise, every 5,000 steps (`val/nll` in the metrics files).
@@ -64,6 +65,7 @@ Validation NELBO is computed with the EMA weights on the full OpenWebText valida
64
  frac_0.10/ ... frac_1.00/
65
  checkpoints/step_*.ckpt PyTorch Lightning checkpoints (53 at 1M)
66
  checkpoints/index.csv step, wall-clock training hours, tokens and epochs per checkpoint
 
67
  metrics/version_*/metrics.csv training/validation metrics (one file per job segment)
68
  norms/norms_raw.jsonl, norms_ema.jsonl weight norms every 500 steps, raw and EMA weights
69
  run_info.json, config_resolved.yaml, code_sha, segments.jsonl
@@ -86,7 +88,7 @@ loop/sampler state, so training can be resumed exactly. Step 0 is about half the
86
  ```python
87
  import torch
88
 
89
- ck = torch.load("frac_0.10/checkpoints/step_0400000.ckpt", map_location="cpu", weights_only=False)
90
  raw = {k: v for k, v in ck["state_dict"].items() if k.startswith("backbone.")}
91
  # EMA tensors follow the backbone's trainable parameters in order (all of the
92
  # state_dict except the rotary-embedding buffer).
 
15
 
16
  10 uniform-state discrete diffusion language models (Duo, `duo_base`), identical
17
  except for the size of their training set: nested subsets of OpenWebText holding
18
+ 10%, 20%, ..., 100% of the training blocks. **Training is paused before its 1,000,000-step target** (stopped to save compute; it
19
+ may resume). This repository holds the grid checkpoints reached so far and, per run,
20
+ `checkpoints/last.ckpt`, the exact state where training stopped.
21
  Each folder holds **full checkpoints** on a fixed step grid (53 at 1M), so the whole
22
  training trajectory can be analysed (e.g. test loss against weight norm, in the
23
  spirit of D'Amico, Bocchi & Negri, *Implicit bias produces neural scaling laws in
 
25
 
26
  ## Models
27
 
28
+ | fraction | training blocks (1,024 tokens) | step | epochs | val NELBO at that step (nats/token) | training hours | `last.ckpt` step |
29
+ |---|---|---|---|---|---|---|
30
+ | 0.1 | 873,094 | 450,000 | 264 | 3.0110 | 216 | 452,500 |
31
+ | 0.2 | 1,746,188 | 450,000 | 132 | 2.9923 | 223 | 470,000 |
32
+ | 0.3 | 2,619,283 | 375,000 | 73 | 3.0131 | 187 | 397,500 |
33
+ | 0.4 | 3,492,377 | 425,000 | 62 | 2.9944 | 210 | 437,500 |
34
+ | 0.5 | 4,365,471 | 725,000 | 85 | 2.9544 | 361 | 727,500 |
35
+ | 0.6 | 5,238,565 | 375,000 | 37 | 3.0024 | 182 | 395,000 |
36
+ | 0.7 | 6,111,659 | 400,000 | 34 | 2.9933 | 188 | 407,500 |
37
+ | 0.8 | 6,984,754 | 325,000 | 24 | 3.0122 | 164 | 332,500 |
38
+ | 0.9 | 7,857,848 | 275,000 | 18 | 3.0334 | 135 | 289,420 |
39
+ | 1.0 | 8,730,942 | 275,000 | 16 | 3.0420 | 133 | 295,720 |
40
 
41
  Validation NELBO is computed with the EMA weights on the full OpenWebText validation split
42
  (110,464 blocks), with fixed noise, every 5,000 steps (`val/nll` in the metrics files).
 
65
  frac_0.10/ ... frac_1.00/
66
  checkpoints/step_*.ckpt PyTorch Lightning checkpoints (53 at 1M)
67
  checkpoints/index.csv step, wall-clock training hours, tokens and epochs per checkpoint
68
+ checkpoints/last.ckpt state where training stopped (resume point)
69
  metrics/version_*/metrics.csv training/validation metrics (one file per job segment)
70
  norms/norms_raw.jsonl, norms_ema.jsonl weight norms every 500 steps, raw and EMA weights
71
  run_info.json, config_resolved.yaml, code_sha, segments.jsonl
 
88
  ```python
89
  import torch
90
 
91
+ ck = torch.load("frac_0.10/checkpoints/step_0450000.ckpt", map_location="cpu", weights_only=False)
92
  raw = {k: v for k, v in ck["state_dict"].items() if k.startswith("backbone.")}
93
  # EMA tensors follow the backbone's trainable parameters in order (all of the
94
  # state_dict except the rotary-embedding buffer).
frac_0.10/checkpoints/index.csv CHANGED
@@ -199,3 +199,22 @@ step,kind,file,unix_time,save_s,train_hours,gpu_hours,samples,tokens,epochs,job_
199
  405000,last,last.ckpt,1795104713.944,8.328,193.76814996931392,775.0725998772557,207360000,212336640000,237.5002004366082,192255
200
  407500,last,last.ckpt,1795109278.850,10.115,195.03617943631642,780.1447177452657,208640000,213647360000,238.96625105658725,192255
201
  410000,last,last.ckpt,1795113718.358,9.026,196.26937613407563,785.0775045363025,209920000,214958080000,240.43230167656634,192255
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
199
  405000,last,last.ckpt,1795104713.944,8.328,193.76814996931392,775.0725998772557,207360000,212336640000,237.5002004366082,192255
200
  407500,last,last.ckpt,1795109278.850,10.115,195.03617943631642,780.1447177452657,208640000,213647360000,238.96625105658725,192255
201
  410000,last,last.ckpt,1795113718.358,9.026,196.26937613407563,785.0775045363025,209920000,214958080000,240.43230167656634,192255
202
+ 412500,last,last.ckpt,1795118278.634,8.473,197.53611943398911,790.1444777359565,211200000,216268800000,241.8983522965454,192255
203
+ 415000,last,last.ckpt,1795122714.670,9.388,198.76835172318388,795.0734068927355,212480000,217579520000,243.36440291652445,192255
204
+ 417500,last,last.ckpt,1795127275.998,9.394,200.03538704206804,800.1415481682722,213760000,218890240000,244.8304535365035,192255
205
+ 420000,last,last.ckpt,1795131716.849,8.412,201.26895693281406,805.0758277312563,215040000,220200960000,246.29650415648257,192255
206
+ 422500,last,last.ckpt,1795136281.846,8.330,202.53701147798603,810.1480459119441,216320000,221511680000,247.76255477646163,192255
207
+ 425000,grid,step_0425000.ckpt,1795140715.233,7.920,203.76850809164242,815.0740323665697,217600000,222822400000,249.2286053964407,192255
208
+ 425000,last,last.ckpt,1795140723.304,8.067,203.7707500629924,815.0830002519696,217600000,222822400000,249.2286053964407,192255
209
+ 427500,last,last.ckpt,1795145282.642,8.261,205.03723270326645,820.1489308130658,218880000,224133120000,250.69465601641977,192255
210
+ 430000,last,last.ckpt,1795149718.133,8.171,206.26931361421165,825.0772544568466,220160000,225443840000,252.16070663639883,192255
211
+ 432500,last,last.ckpt,1795154278.158,9.284,207.5359870949046,830.1439483796185,221440000,226754560000,253.62675725637789,192255
212
+ 435000,last,last.ckpt,1795158713.466,8.256,208.7680171992759,835.0720687971036,222720000,228065280000,255.09280787635694,192255
213
+ 437500,last,last.ckpt,1795163272.358,8.068,210.03437611933333,840.1375044773333,224000000,229376000000,256.55885849633603,192255
214
+ 440000,last,last.ckpt,1795167706.643,8.127,211.26612196397605,845.0644878559042,225280000,230686720000,258.0249091163151,192255
215
+ 442500,last,last.ckpt,1795172265.712,8.368,212.53252986715486,850.1301194686195,226560000,231997440000,259.49095973629414,192255
216
+ 445000,last,last.ckpt,1795176699.641,8.522,213.76417686547154,855.0567074618862,227840000,233308160000,260.9570103562732,192255
217
+ 447500,last,last.ckpt,1795181259.928,9.854,215.03092324723696,860.1236929889478,229120000,234618880000,262.42306097625226,192255
218
+ 450000,grid,step_0450000.ckpt,1795185694.256,7.966,216.2626810803047,865.0507243212188,230400000,235929600000,263.8891115962313,192255
219
+ 450000,last,last.ckpt,1795185702.520,8.258,216.2649765319548,865.0599061278192,230400000,235929600000,263.8891115962313,192255
220
+ 452500,last,last.ckpt,1795190260.942,8.264,217.5312050534345,870.124820213738,231680000,237240320000,265.3551622162104,192255
frac_0.10/checkpoints/last.ckpt ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:b1838ab29fd150e9e4a0c914f96e1ec7383c380b2333375aade09dc19e71233b
3
+ size 6793536056
frac_0.10/checkpoints/step_0425000.ckpt ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:353be073d3a7c344954dfb1fe3510f780313e55250119564ee8df6d02f9d1bc0
3
+ size 6793536056
frac_0.10/checkpoints/step_0450000.ckpt ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:09d784a8bb7684595ef67800c0b4a2ee679e89eed37b5245e1e66faf60a2c232
3
+ size 6793536056
frac_0.10/metrics/version_7/metrics.csv CHANGED
The diff for this file is too large to render. See raw diff
 
frac_0.10/norms/norms_raw.jsonl CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:7656e16235616a16624def1e31db1e5c05e4f8be6297b28cb6d5d0da9c798ec4
3
- size 29360054
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:cee21e66791d67fa38f5b9759277882acf82e6d800c4d09d8f30706c86c3ba5d
3
+ size 32374196
frac_0.20/checkpoints/index.csv CHANGED
@@ -206,3 +206,21 @@ step,kind,file,unix_time,save_s,train_hours,gpu_hours,samples,tokens,epochs,job_
206
  425000,grid,step_0425000.ckpt,1795112227.455,9.575,210.56543786852052,842.2617514740821,217600000,222822400000,124.61430269822036,192256
207
  425000,last,last.ckpt,1795112236.265,8.805,210.56788523968774,842.271540958751,217600000,222822400000,124.61430269822036,192256
208
  427500,last,last.ckpt,1795116786.074,9.954,211.83172094828967,847.3268837931587,218880000,224133120000,125.34732800820989,192256
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
206
  425000,grid,step_0425000.ckpt,1795112227.455,9.575,210.56543786852052,842.2617514740821,217600000,222822400000,124.61430269822036,192256
207
  425000,last,last.ckpt,1795112236.265,8.805,210.56788523968774,842.271540958751,217600000,222822400000,124.61430269822036,192256
208
  427500,last,last.ckpt,1795116786.074,9.954,211.83172094828967,847.3268837931587,218880000,224133120000,125.34732800820989,192256
209
+ 430000,last,last.ckpt,1795121209.741,8.828,213.0605173106,852.2420692424,220160000,225443840000,126.08035331819941,192256
210
+ 432500,last,last.ckpt,1795125756.872,8.789,214.32360932517693,857.2944373007077,221440000,226754560000,126.81337862818894,192256
211
+ 435000,last,last.ckpt,1795130183.240,9.235,215.55315603430384,862.2126241372154,222720000,228065280000,127.54640393817847,192256
212
+ 437500,last,last.ckpt,1795134739.245,8.710,216.81871306710823,867.2748522684329,224000000,229376000000,128.27942924816801,192256
213
+ 440000,last,last.ckpt,1795139163.781,8.244,218.0477507809772,872.1910031239088,225280000,230686720000,129.01245455815754,192256
214
+ 442500,last,last.ckpt,1795143710.757,8.635,219.31079971893328,877.2431988757331,226560000,231997440000,129.74547986814707,192256
215
+ 445000,last,last.ckpt,1795148132.796,8.266,220.53914394940145,882.1565757976058,227840000,233308160000,130.4785051781366,192256
216
+ 447500,last,last.ckpt,1795152679.531,8.765,221.80212565160926,887.208502606437,229120000,234618880000,131.21153048812613,192256
217
+ 450000,grid,step_0450000.ckpt,1795157100.945,7.729,223.03029638965873,892.1211855586349,230400000,235929600000,131.94455579811566,192256
218
+ 450000,last,last.ckpt,1795157109.421,8.471,223.03265066897185,892.1306026758874,230400000,235929600000,131.94455579811566,192256
219
+ 452500,last,last.ckpt,1795161657.492,9.090,224.29600375980243,897.1840150392097,231680000,237240320000,132.6775811081052,192256
220
+ 455000,last,last.ckpt,1795166081.561,9.722,225.52491198473868,902.0996479389547,232960000,238551040000,133.41060641809472,192256
221
+ 457500,last,last.ckpt,1795170629.340,8.423,226.78818390961737,907.1527356384695,234240000,239861760000,134.14363172808427,192256
222
+ 460000,last,last.ckpt,1795175051.927,9.450,228.01668021022033,912.0667208408813,235520000,241172480000,134.8766570380738,192256
223
+ 462500,last,last.ckpt,1795179597.695,8.473,229.27939349360815,917.1175739744326,236800000,242483200000,135.60968234806333,192256
224
+ 465000,last,last.ckpt,1795184020.925,9.368,230.50806861331426,922.0322744532571,238080000,243793920000,136.34270765805286,192256
225
+ 467500,last,last.ckpt,1795188568.987,8.609,231.77141908975628,927.0856763590251,239360000,245104640000,137.0757329680424,192256
226
+ 470000,last,last.ckpt,1795192992.791,9.363,233.0002534492971,932.0010137971884,240640000,246415360000,137.80875827803192,192256
frac_0.20/checkpoints/last.ckpt ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:ce2a33315482d99370c3d5688669124d483ad5d3b0f1789b6b2f98758114c36e
3
+ size 6793536056
frac_0.20/checkpoints/step_0450000.ckpt ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:65f4ee2d373f50a18c6c612456b28ded7ee076a22ea517d38faf50ca0ee323dc
3
+ size 6793536056
frac_0.20/metrics/version_7/metrics.csv CHANGED
The diff for this file is too large to render. See raw diff
 
frac_0.30/checkpoints/index.csv CHANGED
@@ -175,3 +175,21 @@ step,kind,file,unix_time,save_s,train_hours,gpu_hours,samples,tokens,epochs,job_
175
  350000,last,last.ckpt,1795105976.159,14.857,174.25728989612776,697.029159584511,179200000,183500800000,68.41566947901391,192257
176
  352500,last,last.ckpt,1795110589.226,14.351,175.5386975753923,702.1547903015692,180480000,184811520000,68.90435283243544,192257
177
  355000,last,last.ckpt,1795115077.286,14.682,176.7853807464268,707.1415229857072,181760000,186122240000,69.39303618585697,192257
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
175
  350000,last,last.ckpt,1795105976.159,14.857,174.25728989612776,697.029159584511,179200000,183500800000,68.41566947901391,192257
176
  352500,last,last.ckpt,1795110589.226,14.351,175.5386975753923,702.1547903015692,180480000,184811520000,68.90435283243544,192257
177
  355000,last,last.ckpt,1795115077.286,14.682,176.7853807464268,707.1415229857072,181760000,186122240000,69.39303618585697,192257
178
+ 357500,last,last.ckpt,1795119691.583,16.081,178.06712994953565,712.2685197981426,183040000,187432960000,69.88171953927849,192257
179
+ 360000,last,last.ckpt,1795124176.815,16.006,179.3130278830401,717.2521115321604,184320000,188743680000,70.37040289270003,192257
180
+ 362500,last,last.ckpt,1795128790.237,14.746,180.59453400359232,722.3781360143693,185600000,190054400000,70.85908624612155,192257
181
+ 365000,last,last.ckpt,1795133276.737,17.069,181.8407838845372,727.3631355381488,186880000,191365120000,71.34776959954308,192257
182
+ 367500,last,last.ckpt,1795137888.228,14.982,183.12175355951715,732.4870142380686,188160000,192675840000,71.8364529529646,192257
183
+ 370000,last,last.ckpt,1795142374.814,16.236,184.36802745468916,737.4721098187566,189440000,193986560000,72.32513630638614,192257
184
+ 372500,last,last.ckpt,1795146986.526,14.988,185.64905865322436,742.5962346128974,190720000,195297280000,72.81381965980766,192257
185
+ 375000,grid,step_0375000.ckpt,1795151470.231,15.855,186.8945322940602,747.5781291762409,192000000,196608000000,73.3025030132292,192257
186
+ 375000,last,last.ckpt,1795151484.994,14.757,186.89863316386007,747.5945326554403,192000000,196608000000,73.3025030132292,192257
187
+ 377500,last,last.ckpt,1795156099.925,16.753,188.1805584271133,752.7222337084532,193280000,197918720000,73.79118636665072,192257
188
+ 380000,last,last.ckpt,1795160583.103,16.286,189.42588558145903,757.7035423258361,194560000,199229440000,74.27986972007226,192257
189
+ 382500,last,last.ckpt,1795165194.291,14.565,190.70677097377006,762.8270838950802,195840000,200540160000,74.76855307349378,192257
190
+ 385000,last,last.ckpt,1795169677.314,15.882,191.95205520146112,767.8082208058445,197120000,201850880000,75.2572364269153,192257
191
+ 387500,last,last.ckpt,1795174287.851,16.913,193.2327598698013,772.9310394792052,198400000,203161600000,75.74591978033683,192257
192
+ 390000,last,last.ckpt,1795178770.278,16.021,194.477878547301,777.911514189204,199680000,204472320000,76.23460313375836,192257
193
+ 392500,last,last.ckpt,1795183377.100,14.260,195.75755130337188,783.0302052134875,200960000,205783040000,76.72328648717989,192257
194
+ 395000,last,last.ckpt,1795187860.286,16.218,197.0028808420022,788.0115233680089,202240000,207093760000,77.21196984060141,192257
195
+ 397500,last,last.ckpt,1795192469.339,14.798,198.2831733025988,793.1326932103952,203520000,208404480000,77.70065319402295,192257
frac_0.30/checkpoints/step_0375000.ckpt ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:a251b45c3768ba75e2e694072fb677125566a7a9db15862321427eb2228ee762
3
+ size 6793536056
frac_0.30/metrics/version_7/metrics.csv CHANGED
The diff for this file is too large to render. See raw diff
 
frac_0.30/norms/norms_ema.jsonl CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:e9c1e270f26f1c3af5d1466a87eb3d670aaba08c524741e1e22b5178c0f482f1
3
- size 25527744
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:e313a470539f26fcfc811ed9e6e0ed20739fe056adcf3d0b3acca7994645b772
3
+ size 28472146
frac_0.30/norms/norms_raw.jsonl CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:2204d48c8e2af6842292f15eb8342136b23a4622c26d2efa22773c82cb541675
3
- size 25529090
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:668c2d15591cd2e5d41be7c9fcb5b520c95963d0f522145fefdf715181f89702
3
+ size 28473269
frac_0.40/checkpoints/index.csv CHANGED
@@ -192,3 +192,22 @@ step,kind,file,unix_time,save_s,train_hours,gpu_hours,samples,tokens,epochs,job_
192
  390000,last,last.ckpt,1795105688.518,9.556,191.77652711529208,767.1061084611683,199680000,204472320000,57.17595780753338,192258
193
  392500,last,last.ckpt,1795110324.787,9.337,193.06437965952506,772.2575186381002,200960000,205783040000,57.54247035758167,192258
194
  395000,last,last.ckpt,1795114843.083,10.571,194.31946185934152,777.2778474373661,202240000,207093760000,57.90898290762996,192258
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
192
  390000,last,last.ckpt,1795105688.518,9.556,191.77652711529208,767.1061084611683,199680000,204472320000,57.17595780753338,192258
193
  392500,last,last.ckpt,1795110324.787,9.337,193.06437965952506,772.2575186381002,200960000,205783040000,57.54247035758167,192258
194
  395000,last,last.ckpt,1795114843.083,10.571,194.31946185934152,777.2778474373661,202240000,207093760000,57.90898290762996,192258
195
+ 397500,last,last.ckpt,1795119477.463,10.345,195.60678955571544,782.4271582228617,203520000,208404480000,58.275495457678254,192258
196
+ 400000,grid,step_0400000.ckpt,1795123993.452,8.188,196.86123100072587,787.4449240029035,204800000,209715200000,58.642008007726545,192258
197
+ 400000,last,last.ckpt,1795124002.894,9.437,196.86385383398246,787.4554153359298,204800000,209715200000,58.642008007726545,192258
198
+ 402500,last,last.ckpt,1795128644.106,10.400,198.1530792279932,792.6123169119728,206080000,211025920000,59.00852055777484,192258
199
+ 405000,last,last.ckpt,1795133145.826,9.681,199.4035572387842,797.6142289551368,207360000,212336640000,59.37503310782312,192258
200
+ 407500,last,last.ckpt,1795137786.315,8.893,200.69258181745263,802.7703272698105,208640000,213647360000,59.74154565787141,192258
201
+ 410000,last,last.ckpt,1795142276.818,9.121,201.93994385546048,807.7597754218419,209920000,214958080000,60.108058207919704,192258
202
+ 412500,last,last.ckpt,1795146875.816,11.076,203.21744319097476,812.869772763899,211200000,216268800000,60.474570757967996,192258
203
+ 415000,last,last.ckpt,1795151375.470,10.579,204.46734721357623,817.8693888543049,212480000,217579520000,60.84108330801629,192258
204
+ 417500,last,last.ckpt,1795155995.919,9.687,205.7508052681014,823.0032210724056,213760000,218890240000,61.20759585806458,192258
205
+ 420000,last,last.ckpt,1795160480.786,10.577,206.99660156299075,827.986406251963,215040000,220200960000,61.57410840811287,192258
206
+ 422500,last,last.ckpt,1795165095.222,10.358,208.27838926206627,833.1135570482651,216320000,221511680000,61.94062095816116,192258
207
+ 425000,grid,step_0425000.ckpt,1795169566.905,10.089,209.52052339460855,838.0820935784342,217600000,222822400000,62.30713350820945,192258
208
+ 425000,last,last.ckpt,1795169576.016,9.106,209.5230542290366,838.0922169161464,217600000,222822400000,62.30713350820945,192258
209
+ 427500,last,last.ckpt,1795174182.210,9.889,210.8025527109806,843.2102108439224,218880000,224133120000,62.673646058257745,192258
210
+ 430000,last,last.ckpt,1795178683.347,8.729,212.0528685890948,848.2114743563792,220160000,225443840000,63.04015860830604,192258
211
+ 432500,last,last.ckpt,1795183312.145,8.911,213.338645761585,853.35458304634,221440000,226754560000,63.40667115835432,192258
212
+ 435000,last,last.ckpt,1795187822.619,8.747,214.59155504241068,858.3662201696427,222720000,228065280000,63.77318370840261,192258
213
+ 437500,last,last.ckpt,1795192459.476,10.693,215.87957108155385,863.5182843262154,224000000,229376000000,64.13969625845091,192258
frac_0.40/metrics/version_7/metrics.csv CHANGED
The diff for this file is too large to render. See raw diff
 
frac_0.40/norms/norms_ema.jsonl CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:701505aaffe5fbdc7e6543a378f689f02b89d9706fb033ade3b0555f3b8c7732
3
- size 28258177
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:8533df67aae8ca8572162efcd77421ea74e9da0d0b0d445e19c827e581b73283
3
+ size 31201400
frac_0.40/norms/norms_raw.jsonl CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:e9af980340dde4536866aefc83ebcf39eb9d76e69919522863f56cd6b0f6d496
3
- size 28258674
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:643dfd712f45de4fa1ad644acbbcc6ad536b3b402ede1b1b244e5674c7d2e6e8
3
+ size 31202051
frac_0.50/checkpoints/index.csv CHANGED
@@ -322,3 +322,20 @@ step,kind,file,unix_time,save_s,train_hours,gpu_hours,samples,tokens,epochs,job_
322
  685000,last,last.ckpt,1795104756.452,9.593,338.31565942280537,1353.2626376912215,350720000,359137280000,80.33955557143777,192254
323
  687500,last,last.ckpt,1795109494.954,9.876,339.63191005783597,1358.5276402313439,352000000,360448000000,80.63276562826783,192254
324
  690000,last,last.ckpt,1795115558.999,13.627,341.31636708214285,1365.2654683285714,353280000,361758720000,80.9259756850979,192254
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
322
  685000,last,last.ckpt,1795104756.452,9.593,338.31565942280537,1353.2626376912215,350720000,359137280000,80.33955557143777,192254
323
  687500,last,last.ckpt,1795109494.954,9.876,339.63191005783597,1358.5276402313439,352000000,360448000000,80.63276562826783,192254
324
  690000,last,last.ckpt,1795115558.999,13.627,341.31636708214285,1365.2654683285714,353280000,361758720000,80.9259756850979,192254
325
+ 692500,last,last.ckpt,1795121267.692,13.897,342.90211502452667,1371.6084600981067,354560000,363069440000,81.21918574192796,192254
326
+ 695000,last,last.ckpt,1795126817.688,13.750,344.44378068436686,1377.7751227374674,355840000,364380160000,81.51239579875802,192254
327
+ 697500,last,last.ckpt,1795132910.322,14.118,346.13617876759344,1384.5447150703737,357120000,365690880000,81.8056058555881,192254
328
+ 700000,grid,step_0700000.ckpt,1795138948.043,10.096,347.8133236802315,1391.253294720926,358400000,367001600000,82.09881591241816,192254
329
+ 700000,last,last.ckpt,1795138956.909,8.860,347.81578629696253,1391.2631451878501,358400000,367001600000,82.09881591241816,192254
330
+ 702500,last,last.ckpt,1795143556.303,10.216,349.0933959560601,1396.3735838242403,359680000,368312320000,82.39202596924822,192254
331
+ 705000,last,last.ckpt,1795148063.655,9.105,350.3454381926691,1401.3817527706765,360960000,369623040000,82.68523602607829,192254
332
+ 707500,last,last.ckpt,1795152879.591,10.863,351.6831979516803,1406.7327918067213,362240000,370933760000,82.97844608290835,192254
333
+ 710000,last,last.ckpt,1795157532.411,10.843,352.9756481571193,1411.9025926284771,363520000,372244480000,83.27165613973841,192254
334
+ 712500,last,last.ckpt,1795162368.049,10.432,354.31888094204453,1417.2755237681781,364800000,373555200000,83.56486619656847,192254
335
+ 715000,last,last.ckpt,1795167067.715,10.710,355.6243436772534,1422.4973747090137,366080000,374865920000,83.85807625339855,192254
336
+ 717500,last,last.ckpt,1795171744.428,10.386,356.9234306233751,1427.6937224935004,367360000,376176640000,84.15128631022861,192254
337
+ 720000,last,last.ckpt,1795176324.906,10.047,358.1957856083793,1432.7831424335172,368640000,377487360000,84.44449636705868,192254
338
+ 722500,last,last.ckpt,1795181115.980,10.035,359.5266393438129,1438.1065573752517,369920000,378798080000,84.73770642388874,192254
339
+ 725000,grid,step_0725000.ckpt,1795185775.485,8.992,360.82094630568315,1443.2837852227326,371200000,380108800000,85.0309164807188,192254
340
+ 725000,last,last.ckpt,1795185785.669,10.180,360.8237754058411,1443.2951016233644,371200000,380108800000,85.0309164807188,192254
341
+ 727500,last,last.ckpt,1795190470.213,10.274,362.1250374773711,1448.5001499094844,372480000,381419520000,85.32412653754886,192254
frac_0.50/checkpoints/last.ckpt ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:3ef23f36712ff74e6bd3acc85fc935409b8d31d43341420e5302d04bf3defe4c
3
+ size 6793536056
frac_0.50/checkpoints/step_0700000.ckpt ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:a727de5fdb781182a7d6a86eda7ddb1b42daf5f0060ecbf73a27e69f4dcfa7b9
3
+ size 6793536056
frac_0.50/checkpoints/step_0725000.ckpt ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:d675c04691b14b38d480c705ed2ab8ebf6cd00f9a9f948a71b275aa23445fdfc
3
+ size 6793536056
frac_0.50/metrics/version_7/metrics.csv CHANGED
The diff for this file is too large to render. See raw diff
 
frac_0.50/norms/norms_ema.jsonl CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:1e1015f18a8e58973090cbded2788bcf7ec58f4083c72e7d653a12343e3c4ebe
3
- size 49139048
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:d49f50aa5325424e69df3a241f1520a08cbdc2588740287f7391cbfbba2622b7
3
+ size 51830189
frac_0.50/norms/norms_raw.jsonl CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:1ae7453b2d85aaa11317b2a07192920a70269e0b316d8c6f38a3b641cde3d34c
3
- size 49139684
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:91f50fa12ce028652b54ba0a4f24cd1d59f7228053ce448156866176a3ea6df4
3
+ size 51830352
frac_0.60/checkpoints/index.csv CHANGED
@@ -170,3 +170,24 @@ step,kind,file,unix_time,save_s,train_hours,gpu_hours,samples,tokens,epochs,job_
170
  342500,last,last.ckpt,1795107859.414,8.009,168.02961213974567,672.1184485589827,175360000,179568640000,33.47481609944708,192259
171
  345000,last,last.ckpt,1795111776.891,8.019,169.11780028377737,676.4712011351095,176640000,180879360000,33.71915782280071,192259
172
  347500,last,last.ckpt,1795115810.168,7.815,170.2381549045829,680.9526196183316,177920000,182190080000,33.96349954615434,192259
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
170
  342500,last,last.ckpt,1795107859.414,8.009,168.02961213974567,672.1184485589827,175360000,179568640000,33.47481609944708,192259
171
  345000,last,last.ckpt,1795111776.891,8.019,169.11780028377737,676.4712011351095,176640000,180879360000,33.71915782280071,192259
172
  347500,last,last.ckpt,1795115810.168,7.815,170.2381549045829,680.9526196183316,177920000,182190080000,33.96349954615434,192259
173
+ 350000,grid,step_0350000.ckpt,1795119725.426,7.216,171.3257266976511,685.3029067906044,179200000,183500800000,34.207841269507966,192259
174
+ 350000,last,last.ckpt,1795119733.083,7.653,171.3278536247568,685.3114144990273,179200000,183500800000,34.207841269507966,192259
175
+ 352500,last,last.ckpt,1795123766.358,7.865,172.4482078989334,689.7928315957336,180480000,184811520000,34.452182992861594,192259
176
+ 355000,last,last.ckpt,1795127682.809,7.578,173.53611084896684,694.1444433958674,181760000,186122240000,34.69652471621522,192259
177
+ 357500,last,last.ckpt,1795131715.765,7.918,174.65637640762532,698.6255056305013,183040000,187432960000,34.94086643956885,192259
178
+ 360000,last,last.ckpt,1795135630.663,7.753,175.74384818385434,702.9753927354174,184320000,188743680000,35.18520816292248,192259
179
+ 362500,last,last.ckpt,1795139662.979,7.872,176.86393591819407,707.4557436727763,185600000,190054400000,35.42954988627611,192259
180
+ 365000,last,last.ckpt,1795143578.485,7.948,177.95157633333545,711.8063053333418,186880000,191365120000,35.673891609629734,192259
181
+ 367500,last,last.ckpt,1795147609.962,7.829,179.07143101017587,716.2857240407035,188160000,192675840000,35.91823333298336,192259
182
+ 370000,last,last.ckpt,1795151526.944,7.897,180.15948183117771,720.6379273247109,189440000,193986560000,36.16257505633699,192259
183
+ 372500,last,last.ckpt,1795155560.529,7.757,181.27992195602275,725.119687824091,190720000,195297280000,36.40691677969062,192259
184
+ 375000,grid,step_0375000.ckpt,1795159477.577,7.550,182.36799094255213,729.4719637702085,192000000,196608000000,36.65125850304425,192259
185
+ 375000,last,last.ckpt,1795159485.246,7.664,182.37012110466785,729.4804844186714,192000000,196608000000,36.65125850304425,192259
186
+ 377500,last,last.ckpt,1795163516.509,7.800,183.48991636102062,733.9596654440825,193280000,197918720000,36.895600226397875,192259
187
+ 380000,last,last.ckpt,1795167431.405,7.739,184.5773875452296,738.3095501809184,194560000,199229440000,37.13994194975151,192259
188
+ 382500,last,last.ckpt,1795171461.763,7.935,185.6969315250655,742.787726100262,195840000,200540160000,37.38428367310514,192259
189
+ 385000,last,last.ckpt,1795175374.591,7.851,186.7838280666612,747.1353122666447,197120000,201850880000,37.628625396458766,192259
190
+ 387500,last,last.ckpt,1795179406.186,7.947,187.90371552518337,751.6148621007335,198400000,203161600000,37.872967119812394,192259
191
+ 390000,last,last.ckpt,1795183322.896,7.997,188.99169069199692,755.9667627679877,199680000,204472320000,38.11730884316602,192259
192
+ 392500,last,last.ckpt,1795187356.507,8.054,190.11213797527805,760.4485519011122,200960000,205783040000,38.36165056651965,192259
193
+ 395000,last,last.ckpt,1795191273.015,8.064,191.2000570574845,764.800228229938,202240000,207093760000,38.60599228987328,192259
frac_0.60/checkpoints/last.ckpt ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:89a9b83427f048ea9e614b27d63704c24423136f04f290590152e59fa98f6641
3
+ size 6793536056
frac_0.60/checkpoints/step_0350000.ckpt ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:343b30c920ef6fc777fff81ba1514f683a4043a866e17c3451a60ceb27798f76
3
+ size 6793536056
frac_0.60/checkpoints/step_0375000.ckpt ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:d2335a9b45749e245f2fab609870ae27f61fb13b683af4b85ca0a71760c262dd
3
+ size 6793536056
frac_0.60/metrics/version_6/metrics.csv CHANGED
The diff for this file is too large to render. See raw diff
 
frac_0.60/norms/norms_ema.jsonl CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:9e9210e2ec6454883fce8940de39a461812bc6d0d76f483eb86430d4c6222ed6
3
- size 24853889
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:17c90bdd48fdeee9929bea7a88695d437abf3839ed4ac4ea217a9a6587e0c2e8
3
+ size 28259370
frac_0.60/norms/norms_raw.jsonl CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:0bd18fc2d1c0576bde6ac4417067dedef3d67a65618ee4becd705cc3666615b7
3
- size 24854780
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:1fd4db07b350cf29dc456ad1b44e297e00d90cdd0e73249b556764f85928005c
3
+ size 28259838
frac_0.70/checkpoints/index.csv CHANGED
@@ -175,3 +175,25 @@ step,kind,file,unix_time,save_s,train_hours,gpu_hours,samples,tokens,epochs,job_
175
  352500,last,last.ckpt,1795105570.493,8.504,167.42483941725482,669.6993576690193,180480000,184811520000,29.530443370613444,192260
176
  355000,last,last.ckpt,1795109491.697,8.980,168.51406278402828,674.0562511361131,181760000,186122240000,29.739879139199356,192260
177
  357500,last,last.ckpt,1795113531.276,8.293,169.63616814246325,678.544672569853,183040000,187432960000,29.94931490778527,192260
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
175
  352500,last,last.ckpt,1795105570.493,8.504,167.42483941725482,669.6993576690193,180480000,184811520000,29.530443370613444,192260
176
  355000,last,last.ckpt,1795109491.697,8.980,168.51406278402828,674.0562511361131,181760000,186122240000,29.739879139199356,192260
177
  357500,last,last.ckpt,1795113531.276,8.293,169.63616814246325,678.544672569853,183040000,187432960000,29.94931490778527,192260
178
+ 360000,last,last.ckpt,1795117453.387,8.479,170.72564333892092,682.9025733556837,184320000,188743680000,30.158750676371177,192260
179
+ 362500,last,last.ckpt,1795121491.599,8.191,171.84736901200765,687.3894760480306,185600000,190054400000,30.36818644495709,192260
180
+ 365000,last,last.ckpt,1795125413.626,8.105,172.93682094620308,691.7472837848123,186880000,191365120000,30.577622213543002,192260
181
+ 367500,last,last.ckpt,1795129451.111,7.929,174.0583445407163,696.2333781628652,188160000,192675840000,30.78705798212891,192260
182
+ 370000,last,last.ckpt,1795133373.385,8.165,175.14786488981736,700.5914595592694,189440000,193986560000,30.996493750714823,192260
183
+ 372500,last,last.ckpt,1795137411.125,8.237,176.26945942991233,705.0778377196493,190720000,195297280000,31.20592951930073,192260
184
+ 375000,grid,step_0375000.ckpt,1795141332.176,7.893,177.35864025708727,709.4345610283491,192000000,196608000000,31.415365287886644,192260
185
+ 375000,last,last.ckpt,1795141341.257,9.078,177.36116287119557,709.4446514847823,192000000,196608000000,31.415365287886644,192260
186
+ 377500,last,last.ckpt,1795145379.409,8.277,178.48287157508207,713.9314863003283,193280000,197918720000,31.624801056472556,192260
187
+ 380000,last,last.ckpt,1795149303.180,9.040,179.5728080892907,718.2912323571628,194560000,199229440000,31.834236825058465,192260
188
+ 382500,last,last.ckpt,1795153340.679,8.415,180.6943355075257,722.7773420301028,195840000,200540160000,32.043672593644374,192260
189
+ 385000,last,last.ckpt,1795157261.779,8.394,181.7835300700157,727.1341202800628,197120000,201850880000,32.253108362230286,192260
190
+ 387500,last,last.ckpt,1795161299.260,9.129,182.9050525971365,731.620210388546,198400000,203161600000,32.4625441308162,192260
191
+ 390000,last,last.ckpt,1795165219.715,8.256,183.99406765866274,735.976270634651,199680000,204472320000,32.67197989940211,192260
192
+ 392500,last,last.ckpt,1795169256.843,9.196,185.11549208442653,740.4619683377061,200960000,205783040000,32.88141566798802,192260
193
+ 395000,last,last.ckpt,1795173177.345,8.235,186.20452067528618,744.8180827011447,202240000,207093760000,33.090851436573935,192260
194
+ 397500,last,last.ckpt,1795177213.639,8.203,187.32571329164293,749.3028531665717,203520000,208404480000,33.30028720515984,192260
195
+ 400000,grid,step_0400000.ckpt,1795181132.110,7.667,188.4141773751127,753.6567095004508,204800000,209715200000,33.50972297374575,192260
196
+ 400000,last,last.ckpt,1795181140.170,8.058,188.41641647043355,753.6656658817342,204800000,209715200000,33.50972297374575,192260
197
+ 402500,last,last.ckpt,1795185177.647,8.176,189.53793774644973,758.1517509857989,206080000,211025920000,33.719158742331665,192260
198
+ 405000,last,last.ckpt,1795189097.493,7.896,190.6267838851841,762.5071355407364,207360000,212336640000,33.92859451091758,192260
199
+ 407500,last,last.ckpt,1795193135.029,8.212,191.74832162025567,766.9932864810227,208640000,213647360000,34.13803027950349,192260
frac_0.70/checkpoints/last.ckpt ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:b744502928eb666b78b446c7f3c52ff0a7bd8dd2569c04e353e1d60c12577d55
3
+ size 6793536056
frac_0.70/checkpoints/step_0375000.ckpt ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:27b405794abf69250cfada64c12bca6d8355ecbdf6c675dea9611bc13df9fd18
3
+ size 6793536056
frac_0.70/checkpoints/step_0400000.ckpt ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:442c6c8d4c736f68a592a0838186af2a6131e8f1e4b1e63488e9eec14c842012
3
+ size 6793536056
frac_0.70/metrics/version_6/metrics.csv CHANGED
The diff for this file is too large to render. See raw diff
 
frac_0.70/norms/norms_ema.jsonl CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:113669df727ca7f41b927d5c4544d86ca5e73d835b2c2aeaf2ea4c7886d82815
3
- size 25670589
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:3197977a4d3aec7c732671aa6a88cf5b50fe101dac45107e7e082c85656c02f2
3
+ size 29075795
frac_0.70/norms/norms_raw.jsonl CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:66b652971b695b536762f79df6f5d35cd3fb3a23117779fe1b1c8325d5408e5f
3
- size 25671653
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:513d2afcddbd8a73096f889084547dcc82a857d404595bdd68c927d9549c6a72
3
+ size 29076394
frac_0.80/checkpoints/index.csv CHANGED
@@ -145,3 +145,22 @@ step,kind,file,unix_time,save_s,train_hours,gpu_hours,samples,tokens,epochs,job_
145
  285000,last,last.ckpt,1795106287.605,18.361,143.78942550248468,575.1577020099387,145920000,149422080000,20.89121535275258,192261
146
  287500,last,last.ckpt,1795110909.319,18.149,145.07323493165327,580.2929397266131,147200000,150732800000,21.074471627776727,192261
147
  290000,last,last.ckpt,1795115399.593,17.280,146.32053338155254,585.2821335262101,148480000,152043520000,21.257727902800873,192261
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
145
  285000,last,last.ckpt,1795106287.605,18.361,143.78942550248468,575.1577020099387,145920000,149422080000,20.89121535275258,192261
146
  287500,last,last.ckpt,1795110909.319,18.149,145.07323493165327,580.2929397266131,147200000,150732800000,21.074471627776727,192261
147
  290000,last,last.ckpt,1795115399.593,17.280,146.32053338155254,585.2821335262101,148480000,152043520000,21.257727902800873,192261
148
+ 292500,last,last.ckpt,1795120016.440,16.469,147.60299072596968,590.4119629038787,149760000,153354240000,21.440984177825015,192261
149
+ 295000,last,last.ckpt,1795124506.468,16.453,148.85022071857782,595.4008828743113,151040000,154664960000,21.62424045284916,192261
150
+ 297500,last,last.ckpt,1795129125.002,17.112,150.1331468394999,600.5325873579995,152320000,155975680000,21.807496727873307,192261
151
+ 300000,grid,step_0300000.ckpt,1795133616.474,16.673,151.3807780443468,605.5231121773872,153600000,157286400000,21.990753002897453,192261
152
+ 300000,last,last.ckpt,1795133632.574,16.095,151.38525026683902,605.5410010673561,153600000,157286400000,21.990753002897453,192261
153
+ 302500,last,last.ckpt,1795138249.234,16.488,152.66765578724392,610.6706231489757,154880000,158597120000,22.1740092779216,192261
154
+ 305000,last,last.ckpt,1795142738.673,16.996,153.91472207171273,615.6588882868509,156160000,159907840000,22.357265552945744,192261
155
+ 307500,last,last.ckpt,1795147356.461,16.644,155.19744092864244,620.7897637145697,157440000,161218560000,22.54052182796989,192261
156
+ 310000,last,last.ckpt,1795151846.869,17.148,156.44477671079287,625.7791068431715,158720000,162529280000,22.723778102994036,192261
157
+ 312500,last,last.ckpt,1795156463.900,15.842,157.7272853563012,630.9091414252048,160000000,163840000000,22.90703437801818,192261
158
+ 315000,last,last.ckpt,1795160955.988,17.371,158.97508741763886,635.9003496705554,161280000,165150720000,23.090290653042327,192261
159
+ 317500,last,last.ckpt,1795165573.459,16.731,160.25771817835772,641.0308727134309,162560000,166461440000,23.273546928066473,192261
160
+ 320000,last,last.ckpt,1795170065.699,17.132,161.5055628424614,646.0222513698455,163840000,167772160000,23.45680320309062,192261
161
+ 322500,last,last.ckpt,1795174682.220,15.173,162.78792966006995,651.1517186402798,165120000,169082880000,23.64005947811476,192261
162
+ 325000,grid,step_0325000.ckpt,1795179172.175,15.272,164.0351394398852,656.1405577595408,166400000,170393600000,23.823315753138907,192261
163
+ 325000,last,last.ckpt,1795179188.131,15.951,164.03957168784353,656.1582867513741,166400000,170393600000,23.823315753138907,192261
164
+ 327500,last,last.ckpt,1795183808.169,16.959,165.32291554120627,661.2916621648251,167680000,171704320000,24.006572028163053,192261
165
+ 330000,last,last.ckpt,1795188299.988,16.955,166.5706430054725,666.28257202189,168960000,173015040000,24.1898283031872,192261
166
+ 332500,last,last.ckpt,1795192915.811,15.423,167.85281618874365,671.4112647549746,170240000,174325760000,24.373084578211344,192261
frac_0.80/checkpoints/last.ckpt ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:3fa957e3ce62749f1828bdd986482a20c19bfd56cf64bac61e6206d679de6049
3
+ size 6793536056
frac_0.80/checkpoints/step_0300000.ckpt ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:1f11f1469589b977f61d9cbc857f0d3f32c6ed70d32350320e7c4117d9f7fdfa
3
+ size 6793536056
frac_0.80/checkpoints/step_0325000.ckpt ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:0f55a44c0fed52276d4c6aefc0e6d5325fd6d6723086b7628cc56289de1dc348
3
+ size 6793536056
frac_0.80/metrics/version_6/metrics.csv CHANGED
The diff for this file is too large to render. See raw diff
 
frac_0.80/norms/norms_ema.jsonl CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:747149056613f617c4f5514eb28f8090f660b53097935942aaf2c493a51cdb9b
3
- size 20772434
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:076ee42e26540b91968cbd61af8a7a1b7bb2a70f2d2e406689d4a1ffacb05098
3
+ size 23752993
frac_0.80/norms/norms_raw.jsonl CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:b407a7fac168b88bd4ae26be595bab8f5e0a5deab9624dd52d6c56ba2ffe34ea
3
- size 20774465
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:861d2c09accc8faf9a6ca76cbcab2e18d0fd817269b0fa30f36b82b5c879af00
3
+ size 23754575
frac_0.90/checkpoints/last.ckpt ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:5c7b8d50b4ebe8a350c8f3bca0e548acda4c523a66a13eb0fbae41619c9eb193
3
+ size 6793536056
frac_1.00/checkpoints/last.ckpt ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:3aae5845bb03f533a10d1992c0856395564b000811bba57de0257b39752cfdd5
3
+ size 6793536056