surface arm qwen3/mse_gXA_o0_shuf: anchors (every close), stats, results, log, readout
Browse files- surface/qwen3/mse_gXA_o0_shuf/closes/qarm_mse_gXA_o0_shuf_close1_s10_qwen.safetensors +3 -0
- surface/qwen3/mse_gXA_o0_shuf/closes/qarm_mse_gXA_o0_shuf_close2_s10_qwen.safetensors +3 -0
- surface/qwen3/mse_gXA_o0_shuf/closes/qarm_mse_gXA_o0_shuf_close3_s10_qwen.safetensors +3 -0
- surface/qwen3/mse_gXA_o0_shuf/closes/qarm_mse_gXA_o0_shuf_close4_s10_qwen.safetensors +3 -0
- surface/qwen3/mse_gXA_o0_shuf/manifest.json +20 -0
- surface/qwen3/mse_gXA_o0_shuf/readout.md +127 -0
- surface/qwen3/mse_gXA_o0_shuf/results.json +0 -0
- surface/qwen3/mse_gXA_o0_shuf/s10_qwen.safetensors +3 -0
- surface/qwen3/mse_gXA_o0_shuf/stats.pt +3 -0
- surface/qwen3/mse_gXA_o0_shuf/train.log +174 -0
surface/qwen3/mse_gXA_o0_shuf/closes/qarm_mse_gXA_o0_shuf_close1_s10_qwen.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:46c0b8a5388bb037f6f3646fbe7216b8ad24f80a07f254771b45c8d694811f6c
|
| 3 |
+
size 54819344
|
surface/qwen3/mse_gXA_o0_shuf/closes/qarm_mse_gXA_o0_shuf_close2_s10_qwen.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:5b83d9a66cc257a03829e23dd779239b7df3a08c8bba68d798d1c75196760f5c
|
| 3 |
+
size 54819344
|
surface/qwen3/mse_gXA_o0_shuf/closes/qarm_mse_gXA_o0_shuf_close3_s10_qwen.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:e4a359a3b8f6e2a843226b182af1be5762ae461bd67f918a3a44c483f93fae65
|
| 3 |
+
size 54819344
|
surface/qwen3/mse_gXA_o0_shuf/closes/qarm_mse_gXA_o0_shuf_close4_s10_qwen.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:b9e4ac34464308cf420c46088e5d53bba2df9014f5b29f08f4182757ae50419b
|
| 3 |
+
size 54819344
|
surface/qwen3/mse_gXA_o0_shuf/manifest.json
ADDED
|
@@ -0,0 +1,20 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"tag": "qarm_mse_gXA_o0_shuf",
|
| 3 |
+
"run": "mse_gXA_o0_shuf",
|
| 4 |
+
"convention": "qwen3",
|
| 5 |
+
"member": "s10_qwen",
|
| 6 |
+
"final_anchor": "qarm_mse_gXA_o0_shuf_close4_s10_qwen.safetensors",
|
| 7 |
+
"closes": [
|
| 8 |
+
"qarm_mse_gXA_o0_shuf_close1_s10_qwen.safetensors",
|
| 9 |
+
"qarm_mse_gXA_o0_shuf_close2_s10_qwen.safetensors",
|
| 10 |
+
"qarm_mse_gXA_o0_shuf_close3_s10_qwen.safetensors",
|
| 11 |
+
"qarm_mse_gXA_o0_shuf_close4_s10_qwen.safetensors"
|
| 12 |
+
],
|
| 13 |
+
"finished": true,
|
| 14 |
+
"form": "mse",
|
| 15 |
+
"base": "gXA",
|
| 16 |
+
"steps": 4000,
|
| 17 |
+
"trunk_step": 245674,
|
| 18 |
+
"shipped": "2026-10-08 08:59 local",
|
| 19 |
+
"note": "every close's anchor ships (mids included); the final close's anchor is the run's s10_qwen.safetensors"
|
| 20 |
+
}
|
surface/qwen3/mse_gXA_o0_shuf/readout.md
ADDED
|
@@ -0,0 +1,127 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# The surface arm's readout: Qwen3's spelling (the GPT-2 byte map)
|
| 2 |
+
|
| 3 |
+
A surface arm makes the trunk read a tokenizer's spelling of a text as it reads the text's own bytes. Every number below is read on the dual-extraction read's held-out captions (512 of each COCO draw, never trained on; draw 2 is the repeat), one document a row, at every token's closing byte, as whitened Procrustes alignment (1 = the same geometry up to a rotation; two folds by caption). The pre-registered bars are in the plan; B3 and B4 (against Qwen's own states, the mood axis) are the instrument's read.
|
| 4 |
+
|
| 5 |
+
## The runs
|
| 6 |
+
|
| 7 |
+
- qarm_mse_gXA_o0_shuf: per-site MSE; over the frozen eight of gXA (seed A); SHUFFLED PAIRINGS (the control that must fail); blocks 0-31; 4000 of 4000 steps; seed offset 0
|
| 8 |
+
|
| 9 |
+
## The verdicts (the final close of each run)
|
| 10 |
+
|
| 11 |
+
### qarm_mse_gXA_o0_shuf
|
| 12 |
+
|
| 13 |
+
- B1 the gap: NOT MET over every served block >= 12 (the lowest 0.201 at block 31 on draw 1, 0.214 at block 31 on draw 2; bar 0.85); over blocks 12-28 alone: NOT MET
|
| 14 |
+
- B2 silence, whitened: NOT MET (the lowest A alignment 0.9685 at block 31 on draw 1, 0.9716 on draw 2; bar 0.995)
|
| 15 |
+
- B2 silence, in bits: MET (web +0.00033 bpb; the worst partner s4_arith +0.00033; bar +0.012)
|
| 16 |
+
|
| 17 |
+
## The closes
|
| 18 |
+
|
| 19 |
+
### qarm_mse_gXA_o0_shuf
|
| 20 |
+
|
| 21 |
+
| close | step | B1 lowest (draw 1 / 2) | B1 at 12 / 20 / 28 (draw 1) | B2 lowest (draw 1 / 2) | web bits | gate mean / widest |
|
| 22 |
+
|---|---|---|---|---|---|---|
|
| 23 |
+
| 0 | 0 | 0.322 / 0.319 | 0.640 / 0.486 / 0.407 | 1.0000 / 1.0000 | +0.00000 | 0.047 / 0.047 |
|
| 24 |
+
| 1 | 1000 | 0.235 / 0.253 | 0.574 / 0.406 / 0.316 | 0.9522 / 0.9542 | +0.00046 | 0.136 / 0.152 |
|
| 25 |
+
| 2 | 2000 | 0.201 / 0.204 | 0.504 / 0.351 / 0.270 | 0.9523 / 0.9558 | +0.00053 | 0.219 / 0.414 |
|
| 26 |
+
| 3 | 3000 | 0.223 / 0.220 | 0.471 / 0.344 / 0.283 | 0.9669 / 0.9702 | +0.00025 | 0.325 / 0.761 |
|
| 27 |
+
| 4 | 4000 | 0.201 / 0.214 | 0.484 / 0.362 / 0.285 | 0.9685 / 0.9716 | +0.00033 | 0.407 / 0.834 |
|
| 28 |
+
|
| 29 |
+
## Every block at the final close
|
| 30 |
+
|
| 31 |
+
### qarm_mse_gXA_o0_shuf, draw1 (512 captions, 4,920 sites), close 4 at step 4000
|
| 32 |
+
|
| 33 |
+
| block | gap before | armed B vs masked A | armed B vs armed A | A armed vs masked | mse | cos |
|
| 34 |
+
|---|---|---|---|---|---|---|
|
| 35 |
+
| 0 | 0.900 | 0.891 | 0.891 | 0.9937 | 0.974 | 0.503 |
|
| 36 |
+
| 4 | 0.790 | 0.658 | 0.657 | 0.9977 | 0.910 | 0.342 |
|
| 37 |
+
| 8 | 0.733 | 0.587 | 0.586 | 0.9970 | 1.023 | 0.266 |
|
| 38 |
+
| 12 | 0.640 | 0.484 | 0.483 | 0.9959 | 1.207 | 0.145 |
|
| 39 |
+
| 16 | 0.532 | 0.402 | 0.402 | 0.9964 | 1.341 | 0.087 |
|
| 40 |
+
| 18 | 0.502 | 0.378 | 0.378 | 0.9956 | 1.399 | 0.062 |
|
| 41 |
+
| 20 | 0.486 | 0.362 | 0.362 | 0.9954 | 1.461 | 0.031 |
|
| 42 |
+
| 22 | 0.472 | 0.346 | 0.345 | 0.9958 | 1.534 | -0.017 |
|
| 43 |
+
| 24 | 0.457 | 0.331 | 0.330 | 0.9924 | 1.532 | -0.023 |
|
| 44 |
+
| 26 | 0.430 | 0.306 | 0.306 | 0.9908 | 1.488 | -0.019 |
|
| 45 |
+
| 28 | 0.407 | 0.285 | 0.285 | 0.9931 | 1.516 | -0.042 |
|
| 46 |
+
| 29 | 0.398 | 0.276 | 0.275 | 0.9931 | 1.590 | -0.069 |
|
| 47 |
+
| 30 | 0.376 | 0.248 | 0.248 | 0.9884 | 2.321 | -0.218 |
|
| 48 |
+
| 31 | 0.322 | 0.201 | 0.205 | 0.9685 | 3.014 | 0.017 |
|
| 49 |
+
|
| 50 |
+
### qarm_mse_gXA_o0_shuf, draw2 (512 captions, 5,086 sites), close 4 at step 4000
|
| 51 |
+
|
| 52 |
+
| block | gap before | armed B vs masked A | armed B vs armed A | A armed vs masked | mse | cos |
|
| 53 |
+
|---|---|---|---|---|---|---|
|
| 54 |
+
| 0 | 0.890 | 0.881 | 0.881 | 0.9966 | 0.987 | 0.497 |
|
| 55 |
+
| 4 | 0.795 | 0.656 | 0.656 | 0.9976 | 0.903 | 0.342 |
|
| 56 |
+
| 8 | 0.738 | 0.588 | 0.587 | 0.9976 | 1.006 | 0.270 |
|
| 57 |
+
| 12 | 0.644 | 0.486 | 0.486 | 0.9973 | 1.183 | 0.153 |
|
| 58 |
+
| 16 | 0.540 | 0.405 | 0.404 | 0.9962 | 1.319 | 0.095 |
|
| 59 |
+
| 18 | 0.505 | 0.381 | 0.379 | 0.9954 | 1.376 | 0.070 |
|
| 60 |
+
| 20 | 0.492 | 0.368 | 0.367 | 0.9948 | 1.433 | 0.041 |
|
| 61 |
+
| 22 | 0.481 | 0.359 | 0.358 | 0.9929 | 1.508 | -0.007 |
|
| 62 |
+
| 24 | 0.465 | 0.342 | 0.341 | 0.9955 | 1.512 | -0.014 |
|
| 63 |
+
| 26 | 0.440 | 0.324 | 0.323 | 0.9951 | 1.476 | -0.014 |
|
| 64 |
+
| 28 | 0.419 | 0.308 | 0.307 | 0.9918 | 1.508 | -0.038 |
|
| 65 |
+
| 29 | 0.409 | 0.300 | 0.299 | 0.9919 | 1.589 | -0.067 |
|
| 66 |
+
| 30 | 0.388 | 0.275 | 0.276 | 0.9896 | 2.311 | -0.210 |
|
| 67 |
+
| 31 | 0.319 | 0.214 | 0.217 | 0.9716 | 2.805 | 0.023 |
|
| 68 |
+
|
| 69 |
+
The member's share in bits per byte (all on minus the member masked): s1_perspective -0.00004, s2_concept +0.00029, s3_rules +0.00011, s4_arith +0.00033, s5_causal +0.00006, s6_tryfail +0.00004, s7_mixed +0.00008, s8_register +0.00004, web +0.00033
|
| 70 |
+
|
| 71 |
+
## The training curves
|
| 72 |
+
|
| 73 |
+
### qarm_mse_gXA_o0_shuf
|
| 74 |
+
|
| 75 |
+
| step | task loss | mse | cos | InfoNCE | quiet on A | quiet off-domain |
|
| 76 |
+
|---|---|---|---|---|---|---|
|
| 77 |
+
| 1 | 2.5852 | 2.585 | -0.002 | None | 0.00025 | 0.00012 |
|
| 78 |
+
| 50 | 2.5781 | 2.578 | -0.004 | None | 0.00033 | 0.00016 |
|
| 79 |
+
| 250 | 2.4558 | 2.456 | -0.004 | None | 0.00116 | 0.00035 |
|
| 80 |
+
| 500 | 1.8986 | 1.899 | 0.002 | None | 0.00122 | 0.00045 |
|
| 81 |
+
| 1000 | 1.5671 | 1.567 | 0.008 | None | 0.00145 | 0.00059 |
|
| 82 |
+
| 1500 | 1.4730 | 1.473 | 0.009 | None | 0.00093 | 0.00039 |
|
| 83 |
+
| 2000 | 1.4106 | 1.411 | 0.011 | None | 0.00074 | 0.00039 |
|
| 84 |
+
| 2500 | 1.3790 | 1.379 | 0.011 | None | 0.00126 | 0.00192 |
|
| 85 |
+
| 3000 | 1.3489 | 1.349 | 0.013 | None | 0.00099 | 0.00050 |
|
| 86 |
+
| 3500 | 1.3239 | 1.324 | 0.013 | None | 0.00066 | 0.00032 |
|
| 87 |
+
| 4000 | 1.3108 | 1.311 | 0.013 | None | 0.00061 | 0.00032 |
|
| 88 |
+
| 1 | 2.5852 | 2.585 | -0.002 | None | 0.00025 | 0.00012 |
|
| 89 |
+
| 50 | 2.5781 | 2.578 | -0.004 | None | 0.00033 | 0.00016 |
|
| 90 |
+
| 250 | 2.4558 | 2.456 | -0.004 | None | 0.00116 | 0.00035 |
|
| 91 |
+
| 500 | 1.8986 | 1.899 | 0.002 | None | 0.00122 | 0.00045 |
|
| 92 |
+
| 1000 | 1.5671 | 1.567 | 0.008 | None | 0.00145 | 0.00059 |
|
| 93 |
+
| 1500 | 1.4730 | 1.473 | 0.009 | None | 0.00093 | 0.00039 |
|
| 94 |
+
| 2000 | 1.4106 | 1.411 | 0.011 | None | 0.00074 | 0.00039 |
|
| 95 |
+
| 2500 | 1.3790 | 1.379 | 0.011 | None | 0.00126 | 0.00192 |
|
| 96 |
+
| 3000 | 1.3489 | 1.349 | 0.013 | None | 0.00099 | 0.00050 |
|
| 97 |
+
| 3500 | 1.3239 | 1.324 | 0.013 | None | 0.00066 | 0.00032 |
|
| 98 |
+
| 4000 | 1.3108 | 1.311 | 0.013 | None | 0.00061 | 0.00032 |
|
| 99 |
+
| 1 | 2.5852 | 2.585 | -0.002 | None | 0.00025 | 0.00012 |
|
| 100 |
+
| 50 | 2.5781 | 2.578 | -0.004 | None | 0.00033 | 0.00016 |
|
| 101 |
+
| 250 | 2.4558 | 2.456 | -0.004 | None | 0.00116 | 0.00035 |
|
| 102 |
+
| 500 | 1.8986 | 1.899 | 0.002 | None | 0.00122 | 0.00045 |
|
| 103 |
+
| 1000 | 1.5671 | 1.567 | 0.008 | None | 0.00145 | 0.00059 |
|
| 104 |
+
| 1500 | 1.4730 | 1.473 | 0.009 | None | 0.00093 | 0.00039 |
|
| 105 |
+
| 2000 | 1.4106 | 1.411 | 0.011 | None | 0.00074 | 0.00039 |
|
| 106 |
+
| 2500 | 1.3790 | 1.379 | 0.011 | None | 0.00126 | 0.00192 |
|
| 107 |
+
| 3000 | 1.3489 | 1.349 | 0.013 | None | 0.00099 | 0.00050 |
|
| 108 |
+
| 3500 | 1.3239 | 1.324 | 0.013 | None | 0.00066 | 0.00032 |
|
| 109 |
+
| 4000 | 1.3108 | 1.311 | 0.013 | None | 0.00061 | 0.00032 |
|
| 110 |
+
| 1 | 2.5852 | 2.585 | -0.002 | None | 0.00025 | 0.00012 |
|
| 111 |
+
| 50 | 2.5781 | 2.578 | -0.004 | None | 0.00033 | 0.00016 |
|
| 112 |
+
| 250 | 2.4558 | 2.456 | -0.004 | None | 0.00116 | 0.00035 |
|
| 113 |
+
| 500 | 1.8986 | 1.899 | 0.002 | None | 0.00122 | 0.00045 |
|
| 114 |
+
| 1000 | 1.5671 | 1.567 | 0.008 | None | 0.00145 | 0.00059 |
|
| 115 |
+
| 1500 | 1.4730 | 1.473 | 0.009 | None | 0.00093 | 0.00039 |
|
| 116 |
+
| 2000 | 1.4106 | 1.411 | 0.011 | None | 0.00074 | 0.00039 |
|
| 117 |
+
| 2500 | 1.3790 | 1.379 | 0.011 | None | 0.00126 | 0.00192 |
|
| 118 |
+
| 3000 | 1.3489 | 1.349 | 0.013 | None | 0.00099 | 0.00050 |
|
| 119 |
+
| 3500 | 1.3239 | 1.324 | 0.013 | None | 0.00066 | 0.00032 |
|
| 120 |
+
| 4000 | 1.3108 | 1.311 | 0.013 | None | 0.00061 | 0.00032 |
|
| 121 |
+
|
| 122 |
+
## The recipe
|
| 123 |
+
|
| 124 |
+
- per step: the teacher (member masked) reads the A rows: block outputs at the sites + logits; the armed model reads the B rows: per-site MSE of standardized LayerNorm'd block outputs against the teacher's at each token's closing byte, mean over served blocks; quiet on A: lambda x KL(teacher || armed) on the A rows; quiet off-domain: lambda x KL(member masked || armed) on one web row + one partner row
|
| 125 |
+
- lambda 2.0; pure Adam lr 0.001 wd 0; 2 x 4096-byte paired rows a step; the arm LAWFUL_16x8 at every block ({'n_slots': 16, 'K': 16, 'D': 8, 'hidden': 256}); the trunk mini-beatrix-3 at step 245,674; alephllm 0.10.7, amoe-lora 0.2.11, torch 2.8.0+cu128
|
| 126 |
+
- the surface: Qwen3's byte-level BPE spelling (tokenizer.json sha256 be75606093db2094..., vocab 151,669); documents = one caption-like field of a caption-pack row (caption, short, long, tags, medium), sources drawn by {'dff': 0.161, 'scf': 0.161, 'danbooru': 0.164, 'cc12m': 0.406, 'coco': 0.108}
|
| 127 |
+
- the test set: the first 512 captions of each draw of coco_caps_2x2048.json (every caption of the file kept out of training)
|
surface/qwen3/mse_gXA_o0_shuf/results.json
ADDED
|
The diff for this file is too large to render.
See raw diff
|
|
|
surface/qwen3/mse_gXA_o0_shuf/s10_qwen.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:b9e4ac34464308cf420c46088e5d53bba2df9014f5b29f08f4182757ae50419b
|
| 3 |
+
size 54819344
|
surface/qwen3/mse_gXA_o0_shuf/stats.pt
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:c02cfc83e5c780e8100c33750ece0904029feb75e063e8186f8af2ce97f5fa84
|
| 3 |
+
size 264317
|
surface/qwen3/mse_gXA_o0_shuf/train.log
ADDED
|
@@ -0,0 +1,174 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
[ 0.4s] THE JOB: qarm_mse_gXA_o0_shuf: the surface arm s10_qwen (gpt2 spelling by /workspace/qarm/dl/qwen3_tokenizer; A = the text written; sites = tokens) over the frozen eight of gXA; form mse; blocks all; 4000 steps, a close every 1000; SHUFFLED PAIRS (the control that must fail); lambda 2.0; pure Adam lr 0.001; the compiled chain with recompute; card 30.5 GB with 30.4 GB free; host RSS 0.6 GB
|
| 2 |
+
Warning: You are sending unauthenticated requests to the HF Hub. Please set a HF_TOKEN to enable higher rate limits and faster downloads.
|
| 3 |
+
[ 16.4s] mini-beatrix-3 @ 245,674: trunk loaded from /workspace/qarm/dl/mini-beatrix-3/checkpoints/step_00245674.safetensors and frozen (376.0M); alephllm 0.10.7 at /workspace/qarm/alephllm/geolip/alephllm; amoe-lora 0.2.11; torch 2.8.0+cu128
|
| 4 |
+
[ 17.0s] tokenizer /workspace/qarm/dl/qwen3_tokenizer (gpt2 spelling; vocab 151669; sha256 be75606093db2094): 'a taco truck' -> ['a', 'Ġtaco', 'Ġtruck']; A 12 bytes, B 14 bytes; sites [(1, 6, 7)]
|
| 5 |
+
[arms] attached 's1_perspective' for phase 'curriculum_s1': 13.69M params x 32 blocks, lambda 2.0, spec {'n_slots': 16, 'K': 16, 'D': 8, 'hidden': 256}, recompute on, compile on
|
| 6 |
+
[arms] attached 's2_concept' for phase 'curriculum_s2': 13.69M params x 32 blocks, lambda 2.0, spec {'n_slots': 16, 'K': 16, 'D': 8, 'hidden': 256}, recompute on, compile on
|
| 7 |
+
[arms] attached 's3_rules' for phase 'curriculum_s3': 13.69M params x 32 blocks, lambda 2.0, spec {'n_slots': 16, 'K': 16, 'D': 8, 'hidden': 256}, recompute on, compile on
|
| 8 |
+
[arms] attached 's4_arith' for phase 'curriculum_s4': 13.69M params x 32 blocks, lambda 2.0, spec {'n_slots': 16, 'K': 16, 'D': 8, 'hidden': 256}, recompute on, compile on
|
| 9 |
+
[arms] attached 's5_causal' for phase 'curriculum_s5': 13.69M params x 32 blocks, lambda 2.0, spec {'n_slots': 16, 'K': 16, 'D': 8, 'hidden': 256}, recompute on, compile on
|
| 10 |
+
[arms] attached 's6_tryfail' for phase 'curriculum_s6': 13.69M params x 32 blocks, lambda 2.0, spec {'n_slots': 16, 'K': 16, 'D': 8, 'hidden': 256}, recompute on, compile on
|
| 11 |
+
[arms] attached 's7_mixed' for phase 'curriculum_s7': 13.69M params x 32 blocks, lambda 2.0, spec {'n_slots': 16, 'K': 16, 'D': 8, 'hidden': 256}, recompute on, compile on
|
| 12 |
+
[arms] attached 's8_register' for phase 'curriculum_s8': 13.69M params x 32 blocks, lambda 2.0, spec {'n_slots': 16, 'K': 16, 'D': 8, 'hidden': 256}, recompute on, compile on
|
| 13 |
+
[ 26.4s] the eight of gXA (close 8) loaded in training order; they freeze when the member attaches
|
| 14 |
+
[ 26.4s] the read's captions: 512 of each draw of /workspace/qarm/dl/coco_caps_2x2048.json (4,090 captions of both draws kept out of training)
|
| 15 |
+
[ 39.9s] caption pack (/workspace/qarm/dl/mini-beatrix-3/caption_pack/pack_v2, train): dff 121,153 rows, scf 42,080 rows, danbooru 603,026 rows, cc12m 1,242,176 rows, coco 19,227 rows; host RSS 4.5 GB
|
| 16 |
+
[stream] opening 'fineweb-good' (HuggingFaceFW/fineweb-edu) …
|
| 17 |
+
[stream] opening 'simple-wiki' (wikimedia/wikipedia) …
|
| 18 |
+
[stream] opening 'babi-prose' (Muennighoff/babi) …
|
| 19 |
+
Repo card metadata block was not found. Setting CardData to empty.
|
| 20 |
+
[stream] opening 'tinystories' (roneneldan/TinyStories) …
|
| 21 |
+
[stream] opening 'proofwriter-prose' (tasksource/proofwriter) …
|
| 22 |
+
[ 54.3s] streams: web text; partners ['s1_perspective', 's2_concept', 's3_rules', 's4_arith', 's5_causal', 's6_tryfail', 's8_register']; held-out sets ['s1_perspective', 's2_concept', 's3_rules', 's4_arith', 's5_causal', 's6_tryfail', 's7_mixed', 's8_register', 'web'] x 8 batches
|
| 23 |
+
[ 54.3s] read rows draw1: 512 captions (0 unspellable), A rows (512, 99), B rows (512, 116), 4,920 sites
|
| 24 |
+
[ 54.4s] read rows draw2: 512 captions (0 unspellable), A rows (512, 151), B rows (512, 176), 5,086 sites
|
| 25 |
+
[arms] attached 's10_qwen' for phase 'qwen_spelling': 13.69M params x 32 blocks, lambda 2.0, spec {'n_slots': 16, 'K': 16, 'D': 8, 'hidden': 256}, recompute on, compile on
|
| 26 |
+
[ 68.0s] BIRTH: the member attached over ['s1_perspective', 's2_concept', 's3_rules', 's4_arith', 's5_causal', 's6_tryfail', 's7_mixed', 's8_register']: web rows read exactly as before; the hand-run trunk against the model's forward: armed 0.0e+00, masked 0.0e+00 (gates: 0.0)
|
| 27 |
+
[ 73.0s] standardization statistics from 11,974 teacher sites on 8 paired batches (a mean and a spread per block and dimension); sigma range at block 20 0.749..1.110; saved /workspace/qarm/out/anchors/qarm_mse_gXA_o0_shuf_stats.pt
|
| 28 |
+
[ 335.9s] CLOSE 0 (step 0): the member's alignment read on draw 1 (512 captions, 4,920 sites); per block: before -> armed B vs masked A (target) / armed B vs armed A; A armed vs masked
|
| 29 |
+
[ 335.9s] block 8: 0.733 -> 0.733 / 0.733; A 1.0000; mse 1.718 cos 0.366
|
| 30 |
+
[ 335.9s] block 12: 0.640 -> 0.640 / 0.640; A 1.0000; mse 1.718 cos 0.307
|
| 31 |
+
[ 335.9s] block 16: 0.532 -> 0.532 / 0.532; A 1.0000; mse 1.699 cos 0.250
|
| 32 |
+
[ 335.9s] block 18: 0.502 -> 0.502 / 0.502; A 1.0000; mse 1.682 cos 0.236
|
| 33 |
+
[ 335.9s] block 20: 0.486 -> 0.486 / 0.486; A 1.0000; mse 1.699 cos 0.233
|
| 34 |
+
[ 335.9s] block 24: 0.457 -> 0.457 / 0.457; A 1.0000; mse 1.858 cos 0.231
|
| 35 |
+
[ 335.9s] block 28: 0.407 -> 0.407 / 0.407; A 1.0000; mse 2.189 cos 0.194
|
| 36 |
+
[ 335.9s] block 31: 0.322 -> 0.322 / 0.322; A 1.0000; mse 5.319 cos 0.105
|
| 37 |
+
[ 335.9s] draw 2 (the repeat), target alignment at 12/16/20/24/28: 0.644 / 0.540 / 0.492 / 0.465 / 0.419
|
| 38 |
+
[ 335.9s] B1 (served blocks >= 12, draw 1): the lowest target alignment 0.322 against the bar .85; B2: the lowest A alignment 1.0000 against .995; fineweb share +0.00000 +- 0.00000 against +.012; partners' shares s1 +0.0000, s2 +0.0000, s3 +0.0000, s4 +0.0000, s5 +0.0000, s6 +0.0000, s7 +0.0000, s8 +0.0000
|
| 39 |
+
[ 335.9s] gate mean 0.047 / widest 0.047; compiled chain: 13 cache entries (limit 256), 0 limit warnings; docs drawn 256 (skipped {'test': 2, 'spelling': 0, 'empty': 0, 'short': 0}); card peak 2.7 GB; host RSS 12.8 GB
|
| 40 |
+
[stream] opening 'fineweb-edu' (HuggingFaceFW/fineweb-edu) …
|
| 41 |
+
[ 344.8s] step 1/4000: task 2.5852 (mse 2.585 cos -0.002); quiet A 0.00025 off 0.00012; 1,503 sites; 8.95 s/step; the job: 5.7 min spent, about 596.4 min of steps left; card peak 14.2 GB
|
| 42 |
+
[ 514.7s] step 50/4000: task 2.5781 (mse 2.578 cos -0.004); quiet A 0.00033 off 0.00016; 1,508 sites; 3.58 s/step; the job: 8.6 min spent, about 235.4 min of steps left; card peak 14.3 GB
|
| 43 |
+
[ 688.1s] step 100/4000: task 2.5605 (mse 2.561 cos -0.004); quiet A 0.00053 off 0.00023; 1,506 sites; 3.52 s/step; the job: 11.5 min spent, about 228.9 min of steps left; card peak 14.3 GB
|
| 44 |
+
[ 861.4s] step 150/4000: task 2.5340 (mse 2.534 cos -0.004); quiet A 0.00066 off 0.00028; 1,471 sites; 3.50 s/step; the job: 14.4 min spent, about 224.8 min of steps left; card peak 14.3 GB
|
| 45 |
+
[ 1034.5s] step 200/4000: task 2.4972 (mse 2.497 cos -0.003); quiet A 0.00090 off 0.00035; 1,483 sites; 3.49 s/step; the job: 17.2 min spent, about 221.2 min of steps left; card peak 14.3 GB
|
| 46 |
+
[ 1207.8s] step 250/4000: task 2.4558 (mse 2.456 cos -0.004); quiet A 0.00116 off 0.00035; 1,485 sites; 3.49 s/step; the job: 20.1 min spent, about 218.0 min of steps left; card peak 14.3 GB
|
| 47 |
+
[ 1380.8s] step 300/4000: task 2.4007 (mse 2.401 cos -0.003); quiet A 0.00135 off 0.00041; 1,497 sites; 3.48 s/step; the job: 23.0 min spent, about 214.8 min of steps left; card peak 14.4 GB
|
| 48 |
+
[ 1554.6s] step 350/4000: task 2.3215 (mse 2.322 cos -0.003); quiet A 0.00126 off 0.00041; 1,487 sites; 3.48 s/step; the job: 25.9 min spent, about 211.8 min of steps left; card peak 14.4 GB
|
| 49 |
+
[ 1728.3s] step 400/4000: task 2.2110 (mse 2.211 cos -0.004); quiet A 0.00120 off 0.00045; 1,505 sites; 3.48 s/step; the job: 28.8 min spent, about 208.9 min of steps left; card peak 14.4 GB
|
| 50 |
+
[ 1901.6s] step 450/4000: task 2.0475 (mse 2.047 cos -0.000); quiet A 0.00130 off 0.00048; 1,593 sites; 3.48 s/step; the job: 31.7 min spent, about 205.9 min of steps left; card peak 14.4 GB
|
| 51 |
+
[ 2075.1s] step 500/4000: task 1.8986 (mse 1.899 cos 0.002); quiet A 0.00122 off 0.00045; 1,535 sites; 3.48 s/step; the job: 34.6 min spent, about 202.9 min of steps left; card peak 14.4 GB
|
| 52 |
+
[ 2248.5s] step 550/4000: task 1.8106 (mse 1.811 cos 0.005); quiet A 0.00104 off 0.00043; 1,433 sites; 3.48 s/step; the job: 37.5 min spent, about 200.0 min of steps left; card peak 14.4 GB
|
| 53 |
+
[ 2421.6s] step 600/4000: task 1.7514 (mse 1.751 cos 0.006); quiet A 0.00103 off 0.00046; 1,487 sites; 3.48 s/step; the job: 40.4 min spent, about 197.0 min of steps left; card peak 14.4 GB
|
| 54 |
+
[ 2595.0s] step 650/4000: task 1.7168 (mse 1.717 cos 0.004); quiet A 0.00106 off 0.00044; 1,558 sites; 3.48 s/step; the job: 43.2 min spent, about 194.0 min of steps left; card peak 14.4 GB
|
| 55 |
+
[ 2768.0s] step 700/4000: task 1.6742 (mse 1.674 cos 0.007); quiet A 0.00099 off 0.00043; 1,429 sites; 3.47 s/step; the job: 46.1 min spent, about 191.1 min of steps left; card peak 14.4 GB
|
| 56 |
+
[ 2941.4s] step 750/4000: task 1.6477 (mse 1.648 cos 0.007); quiet A 0.00094 off 0.00040; 1,468 sites; 3.47 s/step; the job: 49.0 min spent, about 188.2 min of steps left; card peak 14.4 GB
|
| 57 |
+
[ 3114.5s] step 800/4000: task 1.6192 (mse 1.619 cos 0.008); quiet A 0.00096 off 0.00040; 1,541 sites; 3.47 s/step; the job: 51.9 min spent, about 185.2 min of steps left; card peak 14.4 GB
|
| 58 |
+
[ 3288.5s] step 850/4000: task 1.5988 (mse 1.599 cos 0.008); quiet A 0.00095 off 0.00039; 1,440 sites; 3.47 s/step; the job: 54.8 min spent, about 182.4 min of steps left; card peak 14.4 GB
|
| 59 |
+
[ 3463.1s] step 900/4000: task 1.5828 (mse 1.583 cos 0.009); quiet A 0.00598 off 0.00042; 1,652 sites; 3.47 s/step; the job: 57.7 min spent, about 179.5 min of steps left; card peak 14.4 GB
|
| 60 |
+
[ 3637.4s] step 950/4000: task 1.6045 (mse 1.605 cos 0.008); quiet A 0.00213 off 0.00073; 1,550 sites; 3.48 s/step; the job: 60.6 min spent, about 176.7 min of steps left; card peak 14.4 GB
|
| 61 |
+
[ 3811.7s] step 1000/4000: task 1.5671 (mse 1.567 cos 0.008); quiet A 0.00145 off 0.00059; 1,493 sites; 3.48 s/step; the job: 63.5 min spent, about 173.8 min of steps left; card peak 14.4 GB
|
| 62 |
+
[ 3812.0s] anchor saved: /workspace/qarm/out/anchors/qarm_mse_gXA_o0_shuf_close1_s10_qwen.safetensors
|
| 63 |
+
[ 4017.2s] CLOSE 1 (step 1000): the member's alignment read on draw 1 (512 captions, 4,920 sites); per block: before -> armed B vs masked A (target) / armed B vs armed A; A armed vs masked
|
| 64 |
+
[ 4017.2s] block 8: 0.733 -> 0.699 / 0.699; A 0.9929; mse 0.835 cos 0.528
|
| 65 |
+
[ 4017.2s] block 12: 0.640 -> 0.574 / 0.574; A 0.9906; mse 0.993 cos 0.421
|
| 66 |
+
[ 4017.2s] block 16: 0.532 -> 0.468 / 0.468; A 0.9894; mse 1.157 cos 0.313
|
| 67 |
+
[ 4017.2s] block 18: 0.502 -> 0.431 / 0.430; A 0.9927; mse 1.240 cos 0.266
|
| 68 |
+
[ 4017.2s] block 20: 0.486 -> 0.406 / 0.407; A 0.9917; mse 1.328 cos 0.219
|
| 69 |
+
[ 4017.2s] block 24: 0.457 -> 0.366 / 0.367; A 0.9894; mse 1.513 cos 0.125
|
| 70 |
+
[ 4017.2s] block 28: 0.407 -> 0.316 / 0.316; A 0.9849; mse 1.589 cos 0.059
|
| 71 |
+
[ 4017.2s] block 31: 0.322 -> 0.235 / 0.237; A 0.9522; mse 3.172 cos 0.037
|
| 72 |
+
[ 4017.2s] draw 2 (the repeat), target alignment at 12/16/20/24/28: 0.583 / 0.476 / 0.417 / 0.380 / 0.332
|
| 73 |
+
[ 4017.2s] B1 (served blocks >= 12, draw 1): the lowest target alignment 0.235 against the bar .85; B2: the lowest A alignment 0.9522 against .995; fineweb share +0.00046 +- 0.00016 against +.012; partners' shares s1 +0.0002, s2 +0.0002, s3 +0.0003, s4 +0.0008, s5 +0.0001, s6 +0.0003, s7 +0.0005, s8 +0.0001
|
| 74 |
+
[ 4017.2s] gate mean 0.136 / widest 0.152; compiled chain: 15 cache entries (limit 256), 0 limit warnings; docs drawn 32,523 (skipped {'test': 44, 'spelling': 0, 'empty': 0, 'short': 0}); card peak 14.4 GB; host RSS 13.5 GB
|
| 75 |
+
[ 4191.2s] step 1050/4000: task 1.5449 (mse 1.545 cos 0.010); quiet A 0.00116 off 0.00050; 1,510 sites; 3.48 s/step; the job: 69.9 min spent, about 170.9 min of steps left; card peak 14.4 GB
|
| 76 |
+
[ 4365.3s] step 1100/4000: task 1.5288 (mse 1.529 cos 0.012); quiet A 0.00099 off 0.00050; 1,491 sites; 3.48 s/step; the job: 72.8 min spent, about 168.0 min of steps left; card peak 14.4 GB
|
| 77 |
+
[ 4539.1s] step 1150/4000: task 1.5190 (mse 1.519 cos 0.011); quiet A 0.00100 off 0.00052; 1,475 sites; 3.48 s/step; the job: 75.7 min spent, about 165.1 min of steps left; card peak 14.4 GB
|
| 78 |
+
[ 4712.8s] step 1200/4000: task 1.5149 (mse 1.515 cos 0.011); quiet A 0.00106 off 0.00157; 1,492 sites; 3.48 s/step; the job: 78.5 min spent, about 162.2 min of steps left; card peak 14.4 GB
|
| 79 |
+
[ 4886.5s] step 1250/4000: task 1.5099 (mse 1.510 cos 0.010); quiet A 0.00097 off 0.00041; 1,473 sites; 3.48 s/step; the job: 81.4 min spent, about 159.3 min of steps left; card peak 14.4 GB
|
| 80 |
+
[ 5060.5s] step 1300/4000: task 1.4976 (mse 1.498 cos 0.010); quiet A 0.00095 off 0.00042; 1,475 sites; 3.48 s/step; the job: 84.3 min spent, about 156.4 min of steps left; card peak 14.4 GB
|
| 81 |
+
[ 5234.3s] step 1350/4000: task 1.4908 (mse 1.491 cos 0.010); quiet A 0.00090 off 0.00047; 1,476 sites; 3.48 s/step; the job: 87.2 min spent, about 153.5 min of steps left; card peak 14.4 GB
|
| 82 |
+
[ 5408.3s] step 1400/4000: task 1.4814 (mse 1.481 cos 0.011); quiet A 0.00091 off 0.00039; 1,519 sites; 3.48 s/step; the job: 90.1 min spent, about 150.6 min of steps left; card peak 14.4 GB
|
| 83 |
+
[ 5581.7s] step 1450/4000: task 1.4820 (mse 1.482 cos 0.010); quiet A 0.00088 off 0.00063; 1,500 sites; 3.48 s/step; the job: 93.0 min spent, about 147.7 min of steps left; card peak 14.4 GB
|
| 84 |
+
[ 5755.6s] step 1500/4000: task 1.4730 (mse 1.473 cos 0.009); quiet A 0.00093 off 0.00039; 1,442 sites; 3.48 s/step; the job: 95.9 min spent, about 144.8 min of steps left; card peak 14.4 GB
|
| 85 |
+
[ 5929.4s] step 1550/4000: task 1.4627 (mse 1.463 cos 0.011); quiet A 0.00085 off 0.00043; 1,466 sites; 3.48 s/step; the job: 98.8 min spent, about 141.9 min of steps left; card peak 14.4 GB
|
| 86 |
+
[ 6103.3s] step 1600/4000: task 1.4535 (mse 1.454 cos 0.012); quiet A 0.00080 off 0.00039; 1,480 sites; 3.48 s/step; the job: 101.7 min spent, about 139.0 min of steps left; card peak 14.4 GB
|
| 87 |
+
[ 6276.9s] step 1650/4000: task 1.4458 (mse 1.446 cos 0.012); quiet A 0.00082 off 0.00037; 1,535 sites; 3.48 s/step; the job: 104.6 min spent, about 136.1 min of steps left; card peak 14.4 GB
|
| 88 |
+
[ 6450.2s] step 1700/4000: task 1.4455 (mse 1.445 cos 0.011); quiet A 0.00083 off 0.00041; 1,499 sites; 3.48 s/step; the job: 107.5 min spent, about 133.2 min of steps left; card peak 14.4 GB
|
| 89 |
+
[ 6623.9s] step 1750/4000: task 1.4354 (mse 1.435 cos 0.011); quiet A 0.00080 off 0.00035; 1,463 sites; 3.48 s/step; the job: 110.4 min spent, about 130.3 min of steps left; card peak 14.4 GB
|
| 90 |
+
[ 6797.3s] step 1800/4000: task 1.4324 (mse 1.432 cos 0.011); quiet A 0.00082 off 0.00045; 1,531 sites; 3.48 s/step; the job: 113.3 min spent, about 127.4 min of steps left; card peak 14.4 GB
|
| 91 |
+
[ 6971.3s] step 1850/4000: task 1.4278 (mse 1.428 cos 0.011); quiet A 0.00081 off 0.00035; 1,498 sites; 3.48 s/step; the job: 116.2 min spent, about 124.5 min of steps left; card peak 14.4 GB
|
| 92 |
+
[ 7145.0s] step 1900/4000: task 1.4220 (mse 1.422 cos 0.012); quiet A 0.00079 off 0.00047; 1,601 sites; 3.48 s/step; the job: 119.1 min spent, about 121.6 min of steps left; card peak 14.4 GB
|
| 93 |
+
[ 7319.1s] step 1950/4000: task 1.4126 (mse 1.413 cos 0.013); quiet A 0.00076 off 0.00037; 1,500 sites; 3.48 s/step; the job: 122.0 min spent, about 118.8 min of steps left; card peak 14.4 GB
|
| 94 |
+
[ 7493.2s] step 2000/4000: task 1.4106 (mse 1.411 cos 0.011); quiet A 0.00074 off 0.00039; 1,501 sites; 3.48 s/step; the job: 124.9 min spent, about 115.9 min of steps left; card peak 14.4 GB
|
| 95 |
+
[ 7493.5s] anchor saved: /workspace/qarm/out/anchors/qarm_mse_gXA_o0_shuf_close2_s10_qwen.safetensors
|
| 96 |
+
[ 7698.2s] CLOSE 2 (step 2000): the member's alignment read on draw 1 (512 captions, 4,920 sites); per block: before -> armed B vs masked A (target) / armed B vs armed A; A armed vs masked
|
| 97 |
+
[ 7698.2s] block 8: 0.733 -> 0.626 / 0.625; A 0.9939; mse 0.880 cos 0.421
|
| 98 |
+
[ 7698.2s] block 12: 0.640 -> 0.504 / 0.503; A 0.9915; mse 1.094 cos 0.280
|
| 99 |
+
[ 7698.2s] block 16: 0.532 -> 0.404 / 0.404; A 0.9909; mse 1.264 cos 0.189
|
| 100 |
+
[ 7698.2s] block 18: 0.502 -> 0.373 / 0.372; A 0.9930; mse 1.332 cos 0.153
|
| 101 |
+
[ 7698.2s] block 20: 0.486 -> 0.351 / 0.351; A 0.9921; mse 1.407 cos 0.112
|
| 102 |
+
[ 7698.2s] block 24: 0.457 -> 0.313 / 0.313; A 0.9913; mse 1.520 cos 0.042
|
| 103 |
+
[ 7698.2s] block 28: 0.407 -> 0.270 / 0.271; A 0.9856; mse 1.564 cos -0.002
|
| 104 |
+
[ 7698.2s] block 31: 0.322 -> 0.201 / 0.201; A 0.9523; mse 3.153 cos -0.009
|
| 105 |
+
[ 7698.2s] draw 2 (the repeat), target alignment at 12/16/20/24/28: 0.511 / 0.410 / 0.356 / 0.319 / 0.284
|
| 106 |
+
[ 7698.2s] B1 (served blocks >= 12, draw 1): the lowest target alignment 0.201 against the bar .85; B2: the lowest A alignment 0.9523 against .995; fineweb share +0.00053 +- 0.00026 against +.012; partners' shares s1 +0.0000, s2 +0.0001, s3 +0.0002, s4 +0.0005, s5 -0.0000, s6 +0.0004, s7 +0.0007, s8 +0.0001
|
| 107 |
+
[ 7698.2s] gate mean 0.219 / widest 0.414; compiled chain: 15 cache entries (limit 256), 0 limit warnings; docs drawn 64,564 (skipped {'test': 97, 'spelling': 1, 'empty': 0, 'short': 0}); card peak 14.4 GB; host RSS 13.5 GB
|
| 108 |
+
[ 7871.7s] step 2050/4000: task 1.4080 (mse 1.408 cos 0.011); quiet A 0.00089 off 0.00038; 1,509 sites; 3.47 s/step; the job: 131.2 min spent, about 113.0 min of steps left; card peak 14.4 GB
|
| 109 |
+
[ 8045.4s] step 2100/4000: task 1.4003 (mse 1.400 cos 0.013); quiet A 0.00077 off 0.00046; 1,504 sites; 3.47 s/step; the job: 134.1 min spent, about 110.1 min of steps left; card peak 14.4 GB
|
| 110 |
+
[ 8219.2s] step 2150/4000: task 1.3945 (mse 1.394 cos 0.013); quiet A 0.00071 off 0.00033; 1,495 sites; 3.47 s/step; the job: 137.0 min spent, about 107.2 min of steps left; card peak 14.4 GB
|
| 111 |
+
[ 8392.8s] step 2200/4000: task 1.3935 (mse 1.394 cos 0.012); quiet A 0.00072 off 0.00036; 1,510 sites; 3.47 s/step; the job: 139.9 min spent, about 104.3 min of steps left; card peak 14.4 GB
|
| 112 |
+
[ 8566.7s] step 2250/4000: task 1.3935 (mse 1.394 cos 0.011); quiet A 0.00074 off 0.00034; 1,547 sites; 3.47 s/step; the job: 142.8 min spent, about 101.4 min of steps left; card peak 14.4 GB
|
| 113 |
+
[ 8740.3s] step 2300/4000: task 1.3842 (mse 1.384 cos 0.012); quiet A 0.00077 off 0.00033; 1,479 sites; 3.47 s/step; the job: 145.7 min spent, about 98.5 min of steps left; card peak 14.4 GB
|
| 114 |
+
[ 8914.4s] step 2350/4000: task 1.3818 (mse 1.382 cos 0.011); quiet A 0.00068 off 0.00036; 1,529 sites; 3.47 s/step; the job: 148.6 min spent, about 95.6 min of steps left; card peak 14.4 GB
|
| 115 |
+
[ 9088.0s] step 2400/4000: task 1.3773 (mse 1.377 cos 0.011); quiet A 0.00068 off 0.00036; 1,459 sites; 3.47 s/step; the job: 151.5 min spent, about 92.7 min of steps left; card peak 14.4 GB
|
| 116 |
+
[ 9261.9s] step 2450/4000: task 1.3717 (mse 1.372 cos 0.012); quiet A 0.00068 off 0.00034; 1,471 sites; 3.47 s/step; the job: 154.4 min spent, about 89.8 min of steps left; card peak 14.4 GB
|
| 117 |
+
[ 9435.1s] step 2500/4000: task 1.3790 (mse 1.379 cos 0.011); quiet A 0.00126 off 0.00192; 1,553 sites; 3.47 s/step; the job: 157.3 min spent, about 86.9 min of steps left; card peak 14.4 GB
|
| 118 |
+
[ 9608.6s] step 2550/4000: task 1.3690 (mse 1.369 cos 0.012); quiet A 0.00117 off 0.00046; 1,506 sites; 3.47 s/step; the job: 160.1 min spent, about 84.0 min of steps left; card peak 14.4 GB
|
| 119 |
+
[ 9782.5s] step 2600/4000: task 1.3673 (mse 1.367 cos 0.012); quiet A 0.00090 off 0.00044; 1,443 sites; 3.47 s/step; the job: 163.0 min spent, about 81.1 min of steps left; card peak 14.4 GB
|
| 120 |
+
[ 9956.5s] step 2650/4000: task 1.3633 (mse 1.363 cos 0.011); quiet A 0.00067 off 0.00036; 1,447 sites; 3.47 s/step; the job: 165.9 min spent, about 78.2 min of steps left; card peak 14.4 GB
|
| 121 |
+
[10129.8s] step 2700/4000: task 1.3563 (mse 1.356 cos 0.012); quiet A 0.00069 off 0.00040; 1,511 sites; 3.47 s/step; the job: 168.8 min spent, about 75.3 min of steps left; card peak 14.4 GB
|
| 122 |
+
[10303.3s] step 2750/4000: task 1.3534 (mse 1.353 cos 0.013); quiet A 0.00066 off 0.00044; 1,482 sites; 3.47 s/step; the job: 171.7 min spent, about 72.4 min of steps left; card peak 14.4 GB
|
| 123 |
+
[10476.6s] step 2800/4000: task 1.3540 (mse 1.354 cos 0.011); quiet A 0.00070 off 0.00036; 1,489 sites; 3.47 s/step; the job: 174.6 min spent, about 69.5 min of steps left; card peak 14.4 GB
|
| 124 |
+
[10650.1s] step 2850/4000: task 1.3439 (mse 1.344 cos 0.014); quiet A 0.00066 off 0.00040; 1,523 sites; 3.47 s/step; the job: 177.5 min spent, about 66.6 min of steps left; card peak 14.4 GB
|
| 125 |
+
[10823.7s] step 2900/4000: task 1.3555 (mse 1.356 cos 0.010); quiet A 0.00376 off 0.00033; 1,518 sites; 3.47 s/step; the job: 180.4 min spent, about 63.7 min of steps left; card peak 14.5 GB
|
| 126 |
+
[10996.9s] step 2950/4000: task 1.3673 (mse 1.367 cos 0.012); quiet A 0.00173 off 0.00088; 1,497 sites; 3.47 s/step; the job: 183.3 min spent, about 60.8 min of steps left; card peak 14.5 GB
|
| 127 |
+
[11170.5s] step 3000/4000: task 1.3489 (mse 1.349 cos 0.013); quiet A 0.00099 off 0.00050; 1,480 sites; 3.47 s/step; the job: 186.2 min spent, about 57.9 min of steps left; card peak 14.5 GB
|
| 128 |
+
[11170.7s] anchor saved: /workspace/qarm/out/anchors/qarm_mse_gXA_o0_shuf_close3_s10_qwen.safetensors
|
| 129 |
+
[11375.1s] CLOSE 3 (step 3000): the member's alignment read on draw 1 (512 captions, 4,920 sites); per block: before -> armed B vs masked A (target) / armed B vs armed A; A armed vs masked
|
| 130 |
+
[11375.1s] block 8: 0.733 -> 0.593 / 0.592; A 0.9971; mse 1.030 cos 0.292
|
| 131 |
+
[11375.1s] block 12: 0.640 -> 0.471 / 0.471; A 0.9963; mse 1.202 cos 0.178
|
| 132 |
+
[11375.1s] block 16: 0.532 -> 0.388 / 0.387; A 0.9939; mse 1.345 cos 0.114
|
| 133 |
+
[11375.1s] block 18: 0.502 -> 0.364 / 0.364; A 0.9960; mse 1.400 cos 0.087
|
| 134 |
+
[11375.1s] block 20: 0.486 -> 0.344 / 0.343; A 0.9951; mse 1.467 cos 0.048
|
| 135 |
+
[11375.1s] block 24: 0.457 -> 0.319 / 0.319; A 0.9931; mse 1.521 cos 0.010
|
| 136 |
+
[11375.1s] block 28: 0.407 -> 0.283 / 0.283; A 0.9918; mse 1.524 cos -0.016
|
| 137 |
+
[11375.1s] block 31: 0.322 -> 0.223 / 0.222; A 0.9669; mse 3.043 cos 0.010
|
| 138 |
+
[11375.1s] draw 2 (the repeat), target alignment at 12/16/20/24/28: 0.474 / 0.387 / 0.343 / 0.323 / 0.292
|
| 139 |
+
[11375.1s] B1 (served blocks >= 12, draw 1): the lowest target alignment 0.223 against the bar .85; B2: the lowest A alignment 0.9669 against .995; fineweb share +0.00025 +- 0.00031 against +.012; partners' shares s1 +0.0001, s2 -0.0001, s3 +0.0003, s4 +0.0006, s5 +0.0003, s6 +0.0002, s7 +0.0004, s8 +0.0000
|
| 140 |
+
[11375.1s] gate mean 0.325 / widest 0.761; compiled chain: 15 cache entries (limit 256), 0 limit warnings; docs drawn 96,501 (skipped {'test': 156, 'spelling': 1, 'empty': 0, 'short': 0}); card peak 14.5 GB; host RSS 13.5 GB
|
| 141 |
+
[11548.7s] step 3050/4000: task 1.3448 (mse 1.345 cos 0.012); quiet A 0.00086 off 0.00055; 1,458 sites; 3.47 s/step; the job: 192.5 min spent, about 55.0 min of steps left; card peak 14.5 GB
|
| 142 |
+
[11722.4s] step 3100/4000: task 1.3455 (mse 1.345 cos 0.012); quiet A 0.00080 off 0.00052; 1,561 sites; 3.47 s/step; the job: 195.4 min spent, about 52.1 min of steps left; card peak 14.5 GB
|
| 143 |
+
[11895.9s] step 3150/4000: task 1.3429 (mse 1.343 cos 0.011); quiet A 0.00077 off 0.00044; 1,440 sites; 3.47 s/step; the job: 198.3 min spent, about 49.2 min of steps left; card peak 14.5 GB
|
| 144 |
+
[12069.2s] step 3200/4000: task 1.3374 (mse 1.337 cos 0.013); quiet A 0.00074 off 0.00042; 1,519 sites; 3.47 s/step; the job: 201.2 min spent, about 46.3 min of steps left; card peak 14.5 GB
|
| 145 |
+
[12243.0s] step 3250/4000: task 1.3348 (mse 1.335 cos 0.012); quiet A 0.00079 off 0.00039; 1,475 sites; 3.47 s/step; the job: 204.1 min spent, about 43.4 min of steps left; card peak 14.5 GB
|
| 146 |
+
[12416.5s] step 3300/4000: task 1.3338 (mse 1.334 cos 0.013); quiet A 0.00105 off 0.00187; 1,474 sites; 3.47 s/step; the job: 206.9 min spent, about 40.5 min of steps left; card peak 14.5 GB
|
| 147 |
+
[12590.0s] step 3350/4000: task 1.3413 (mse 1.341 cos 0.009); quiet A 0.00077 off 0.00043; 1,541 sites; 3.47 s/step; the job: 209.8 min spent, about 37.6 min of steps left; card peak 14.5 GB
|
| 148 |
+
[12763.4s] step 3400/4000: task 1.3317 (mse 1.332 cos 0.012); quiet A 0.00072 off 0.00036; 1,520 sites; 3.47 s/step; the job: 212.7 min spent, about 34.7 min of steps left; card peak 14.5 GB
|
| 149 |
+
[12937.1s] step 3450/4000: task 1.3270 (mse 1.327 cos 0.013); quiet A 0.00066 off 0.00038; 1,454 sites; 3.47 s/step; the job: 215.6 min spent, about 31.8 min of steps left; card peak 14.5 GB
|
| 150 |
+
[13110.4s] step 3500/4000: task 1.3239 (mse 1.324 cos 0.013); quiet A 0.00066 off 0.00032; 1,505 sites; 3.47 s/step; the job: 218.5 min spent, about 29.0 min of steps left; card peak 14.5 GB
|
| 151 |
+
[13284.4s] step 3550/4000: task 1.3229 (mse 1.323 cos 0.013); quiet A 0.00064 off 0.00046; 1,525 sites; 3.47 s/step; the job: 221.4 min spent, about 26.1 min of steps left; card peak 14.5 GB
|
| 152 |
+
[13457.7s] step 3600/4000: task 1.3200 (mse 1.320 cos 0.013); quiet A 0.00066 off 0.00036; 1,519 sites; 3.47 s/step; the job: 224.3 min spent, about 23.2 min of steps left; card peak 14.5 GB
|
| 153 |
+
[13631.1s] step 3650/4000: task 1.3194 (mse 1.319 cos 0.013); quiet A 0.00067 off 0.00033; 1,472 sites; 3.47 s/step; the job: 227.2 min spent, about 20.3 min of steps left; card peak 14.5 GB
|
| 154 |
+
[13804.4s] step 3700/4000: task 1.3169 (mse 1.317 cos 0.014); quiet A 0.00074 off 0.00039; 1,463 sites; 3.47 s/step; the job: 230.1 min spent, about 17.4 min of steps left; card peak 14.5 GB
|
| 155 |
+
[13977.7s] step 3750/4000: task 1.3138 (mse 1.314 cos 0.014); quiet A 0.00062 off 0.00040; 1,570 sites; 3.47 s/step; the job: 233.0 min spent, about 14.5 min of steps left; card peak 14.5 GB
|
| 156 |
+
[14150.8s] step 3800/4000: task 1.3107 (mse 1.311 cos 0.014); quiet A 0.00060 off 0.00034; 1,532 sites; 3.47 s/step; the job: 235.8 min spent, about 11.6 min of steps left; card peak 14.5 GB
|
| 157 |
+
[14324.6s] step 3850/4000: task 1.3132 (mse 1.313 cos 0.013); quiet A 0.00063 off 0.00035; 1,507 sites; 3.47 s/step; the job: 238.7 min spent, about 8.7 min of steps left; card peak 14.5 GB
|
| 158 |
+
[14498.0s] step 3900/4000: task 1.3160 (mse 1.316 cos 0.011); quiet A 0.00063 off 0.00030; 1,435 sites; 3.47 s/step; the job: 241.6 min spent, about 5.8 min of steps left; card peak 14.5 GB
|
| 159 |
+
[14671.5s] step 3950/4000: task 1.3071 (mse 1.307 cos 0.014); quiet A 0.00063 off 0.00035; 1,500 sites; 3.47 s/step; the job: 244.5 min spent, about 2.9 min of steps left; card peak 14.5 GB
|
| 160 |
+
[14845.1s] step 4000/4000: task 1.3108 (mse 1.311 cos 0.013); quiet A 0.00061 off 0.00032; 1,494 sites; 3.47 s/step; the job: 247.4 min spent, about 0.0 min of steps left; card peak 14.5 GB
|
| 161 |
+
[14845.3s] anchor saved: /workspace/qarm/out/anchors/qarm_mse_gXA_o0_shuf_close4_s10_qwen.safetensors
|
| 162 |
+
[15049.7s] CLOSE 4 (step 4000): the member's alignment read on draw 1 (512 captions, 4,920 sites); per block: before -> armed B vs masked A (target) / armed B vs armed A; A armed vs masked
|
| 163 |
+
[15049.7s] block 8: 0.733 -> 0.587 / 0.586; A 0.9970; mse 1.023 cos 0.266
|
| 164 |
+
[15049.7s] block 12: 0.640 -> 0.484 / 0.483; A 0.9959; mse 1.207 cos 0.145
|
| 165 |
+
[15049.7s] block 16: 0.532 -> 0.402 / 0.402; A 0.9964; mse 1.341 cos 0.087
|
| 166 |
+
[15049.7s] block 18: 0.502 -> 0.378 / 0.378; A 0.9956; mse 1.399 cos 0.062
|
| 167 |
+
[15049.7s] block 20: 0.486 -> 0.362 / 0.362; A 0.9954; mse 1.461 cos 0.031
|
| 168 |
+
[15049.7s] block 24: 0.457 -> 0.331 / 0.330; A 0.9924; mse 1.532 cos -0.023
|
| 169 |
+
[15049.7s] block 28: 0.407 -> 0.285 / 0.285; A 0.9931; mse 1.516 cos -0.042
|
| 170 |
+
[15049.7s] block 31: 0.322 -> 0.201 / 0.205; A 0.9685; mse 3.014 cos 0.017
|
| 171 |
+
[15049.7s] draw 2 (the repeat), target alignment at 12/16/20/24/28: 0.486 / 0.405 / 0.368 / 0.342 / 0.308
|
| 172 |
+
[15049.7s] B1 (served blocks >= 12, draw 1): the lowest target alignment 0.201 against the bar .85; B2: the lowest A alignment 0.9685 against .995; fineweb share +0.00033 +- 0.00023 against +.012; partners' shares s1 -0.0000, s2 +0.0003, s3 +0.0001, s4 +0.0003, s5 +0.0001, s6 +0.0000, s7 +0.0001, s8 +0.0000
|
| 173 |
+
[15049.7s] gate mean 0.407 / widest 0.834; compiled chain: 15 cache entries (limit 256), 0 limit warnings; docs drawn 128,688 (skipped {'test': 203, 'spelling': 1, 'empty': 0, 'short': 0}); card peak 14.5 GB; host RSS 13.5 GB
|
| 174 |
+
[15049.7s] DONE qarm_mse_gXA_o0_shuf: 4000 steps in 231.6 min of steps (3.47 s/step); 4 closes; 15050 s in all; card peak 14.5 GB; host RSS 13.5 GB; compiled chain: 15 cache entries (limit 256), 0 limit warnings
|