surface arm qwen3/mse_untrained_o0: anchors (every close), stats, results, log, readout
Browse files- surface/qwen3/mse_untrained_o0/closes/qarm_mse_untrained_o0_close1_s10_qwen.safetensors +3 -0
- surface/qwen3/mse_untrained_o0/closes/qarm_mse_untrained_o0_close2_s10_qwen.safetensors +3 -0
- surface/qwen3/mse_untrained_o0/manifest.json +18 -0
- surface/qwen3/mse_untrained_o0/readout.md +95 -0
- surface/qwen3/mse_untrained_o0/results.json +0 -0
- surface/qwen3/mse_untrained_o0/s10_qwen.safetensors +3 -0
- surface/qwen3/mse_untrained_o0/stats.pt +3 -0
- surface/qwen3/mse_untrained_o0/train.log +96 -0
surface/qwen3/mse_untrained_o0/closes/qarm_mse_untrained_o0_close1_s10_qwen.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:ad3b2911dad01f691e04dc883e614bf64624f34da0f64186beb2e20a8dca0af9
|
| 3 |
+
size 54819288
|
surface/qwen3/mse_untrained_o0/closes/qarm_mse_untrained_o0_close2_s10_qwen.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:97d006725dd155f9135a2a6cbc1eeabd8dd868c7ebe479b811089acb66037036
|
| 3 |
+
size 54819288
|
surface/qwen3/mse_untrained_o0/manifest.json
ADDED
|
@@ -0,0 +1,18 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"tag": "qarm_mse_untrained_o0",
|
| 3 |
+
"run": "mse_untrained_o0",
|
| 4 |
+
"convention": "qwen3",
|
| 5 |
+
"member": "s10_qwen",
|
| 6 |
+
"final_anchor": "qarm_mse_untrained_o0_close2_s10_qwen.safetensors",
|
| 7 |
+
"closes": [
|
| 8 |
+
"qarm_mse_untrained_o0_close1_s10_qwen.safetensors",
|
| 9 |
+
"qarm_mse_untrained_o0_close2_s10_qwen.safetensors"
|
| 10 |
+
],
|
| 11 |
+
"finished": true,
|
| 12 |
+
"form": "mse",
|
| 13 |
+
"base": "untrained",
|
| 14 |
+
"steps": 2000,
|
| 15 |
+
"trunk_step": 0,
|
| 16 |
+
"shipped": "2026-10-08 10:28 local",
|
| 17 |
+
"note": "every close's anchor ships (mids included); the final close's anchor is the run's s10_qwen.safetensors"
|
| 18 |
+
}
|
surface/qwen3/mse_untrained_o0/readout.md
ADDED
|
@@ -0,0 +1,95 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# The surface arm's readout: Qwen3's spelling (the GPT-2 byte map)
|
| 2 |
+
|
| 3 |
+
A surface arm makes the trunk read a tokenizer's spelling of a text as it reads the text's own bytes. Every number below is read on the dual-extraction read's held-out captions (512 of each COCO draw, never trained on; draw 2 is the repeat), one document a row, at every token's closing byte, as whitened Procrustes alignment (1 = the same geometry up to a rotation; two folds by caption). The pre-registered bars are in the plan; B3 and B4 (against Qwen's own states, the mood axis) are the instrument's read.
|
| 4 |
+
|
| 5 |
+
## The runs
|
| 6 |
+
|
| 7 |
+
- qarm_mse_untrained_o0: per-site MSE; alone on an UNTRAINED copy of the trunk (seed 0; the control that must fail); blocks 0-31; 2000 of 2000 steps; seed offset 0
|
| 8 |
+
|
| 9 |
+
## The verdicts (the final close of each run)
|
| 10 |
+
|
| 11 |
+
### qarm_mse_untrained_o0
|
| 12 |
+
|
| 13 |
+
- B1 the gap: MET over every served block >= 12 (the lowest 0.985 at block 16 on draw 1, 0.984 at block 31 on draw 2; bar 0.85)
|
| 14 |
+
- B2 silence, whitened: MET (the lowest A alignment 0.9952 at block 27 on draw 1, 0.9953 on draw 2; bar 0.995)
|
| 15 |
+
- B2 silence, in bits: MET (web +0.00217 bpb; bar +0.012)
|
| 16 |
+
|
| 17 |
+
## The closes
|
| 18 |
+
|
| 19 |
+
### qarm_mse_untrained_o0
|
| 20 |
+
|
| 21 |
+
| close | step | B1 lowest (draw 1 / 2) | B1 at 12 / 20 / 28 (draw 1) | B2 lowest (draw 1 / 2) | web bits | gate mean / widest |
|
| 22 |
+
|---|---|---|---|---|---|---|
|
| 23 |
+
| 0 | 0 | 0.923 / 0.916 | 0.939 / 0.928 / 0.926 | 1.0000 / 1.0000 | +0.00000 | 0.047 / 0.047 |
|
| 24 |
+
| 1 | 1000 | 0.984 / 0.982 | 0.989 / 0.985 / 0.985 | 0.9927 / 0.9957 | +0.00163 | 0.053 / 0.072 |
|
| 25 |
+
| 2 | 2000 | 0.985 / 0.984 | 0.991 / 0.988 / 0.990 | 0.9952 / 0.9953 | +0.00217 | 0.055 / 0.083 |
|
| 26 |
+
|
| 27 |
+
## Every block at the final close
|
| 28 |
+
|
| 29 |
+
### qarm_mse_untrained_o0, draw1 (512 captions, 4,920 sites), close 2 at step 2000
|
| 30 |
+
|
| 31 |
+
| block | gap before | armed B vs masked A | armed B vs armed A | A armed vs masked | mse | cos |
|
| 32 |
+
|---|---|---|---|---|---|---|
|
| 33 |
+
| 0 | 0.973 | 0.972 | 0.971 | 0.9997 | 0.022 | 0.988 |
|
| 34 |
+
| 4 | 0.954 | 0.990 | 0.990 | 0.9982 | 0.006 | 0.997 |
|
| 35 |
+
| 8 | 0.943 | 0.990 | 0.991 | 0.9986 | 0.006 | 0.997 |
|
| 36 |
+
| 12 | 0.939 | 0.991 | 0.990 | 0.9987 | 0.006 | 0.998 |
|
| 37 |
+
| 16 | 0.933 | 0.985 | 0.986 | 0.9984 | 0.007 | 0.998 |
|
| 38 |
+
| 18 | 0.929 | 0.989 | 0.987 | 0.9966 | 0.007 | 0.998 |
|
| 39 |
+
| 20 | 0.928 | 0.988 | 0.986 | 0.9982 | 0.007 | 0.998 |
|
| 40 |
+
| 22 | 0.928 | 0.989 | 0.988 | 0.9985 | 0.007 | 0.998 |
|
| 41 |
+
| 24 | 0.925 | 0.988 | 0.986 | 0.9977 | 0.008 | 0.998 |
|
| 42 |
+
| 26 | 0.926 | 0.991 | 0.988 | 0.9975 | 0.008 | 0.998 |
|
| 43 |
+
| 28 | 0.926 | 0.990 | 0.989 | 0.9985 | 0.008 | 0.998 |
|
| 44 |
+
| 29 | 0.923 | 0.989 | 0.988 | 0.9985 | 0.008 | 0.998 |
|
| 45 |
+
| 30 | 0.925 | 0.991 | 0.989 | 0.9977 | 0.009 | 0.998 |
|
| 46 |
+
| 31 | 0.923 | 0.990 | 0.988 | 0.9987 | 0.009 | 0.998 |
|
| 47 |
+
|
| 48 |
+
### qarm_mse_untrained_o0, draw2 (512 captions, 5,086 sites), close 2 at step 2000
|
| 49 |
+
|
| 50 |
+
| block | gap before | armed B vs masked A | armed B vs armed A | A armed vs masked | mse | cos |
|
| 51 |
+
|---|---|---|---|---|---|---|
|
| 52 |
+
| 0 | 0.973 | 0.977 | 0.977 | 0.9998 | 0.023 | 0.987 |
|
| 53 |
+
| 4 | 0.954 | 0.985 | 0.985 | 0.9989 | 0.007 | 0.997 |
|
| 54 |
+
| 8 | 0.939 | 0.986 | 0.986 | 0.9989 | 0.007 | 0.997 |
|
| 55 |
+
| 12 | 0.934 | 0.986 | 0.986 | 0.9990 | 0.007 | 0.997 |
|
| 56 |
+
| 16 | 0.929 | 0.987 | 0.987 | 0.9985 | 0.007 | 0.998 |
|
| 57 |
+
| 18 | 0.926 | 0.988 | 0.988 | 0.9985 | 0.008 | 0.998 |
|
| 58 |
+
| 20 | 0.923 | 0.986 | 0.986 | 0.9989 | 0.008 | 0.998 |
|
| 59 |
+
| 22 | 0.921 | 0.989 | 0.988 | 0.9986 | 0.008 | 0.998 |
|
| 60 |
+
| 24 | 0.920 | 0.986 | 0.986 | 0.9970 | 0.008 | 0.998 |
|
| 61 |
+
| 26 | 0.918 | 0.987 | 0.985 | 0.9980 | 0.009 | 0.998 |
|
| 62 |
+
| 28 | 0.918 | 0.986 | 0.986 | 0.9985 | 0.009 | 0.998 |
|
| 63 |
+
| 29 | 0.918 | 0.988 | 0.987 | 0.9986 | 0.009 | 0.998 |
|
| 64 |
+
| 30 | 0.916 | 0.985 | 0.984 | 0.9986 | 0.009 | 0.998 |
|
| 65 |
+
| 31 | 0.916 | 0.984 | 0.982 | 0.9984 | 0.010 | 0.998 |
|
| 66 |
+
|
| 67 |
+
The member's share in bits per byte (all on minus the member masked): web +0.00217
|
| 68 |
+
|
| 69 |
+
## The training curves
|
| 70 |
+
|
| 71 |
+
### qarm_mse_untrained_o0
|
| 72 |
+
|
| 73 |
+
| step | task loss | mse | cos | InfoNCE | quiet on A | quiet off-domain |
|
| 74 |
+
|---|---|---|---|---|---|---|
|
| 75 |
+
| 1 | 1.5112 | 1.511 | 0.549 | None | 0.00000 | 0.00000 |
|
| 76 |
+
| 50 | 0.4068 | 0.407 | 0.820 | None | 0.00164 | 0.00155 |
|
| 77 |
+
| 250 | 0.0380 | 0.038 | 0.982 | None | 0.00011 | 0.00010 |
|
| 78 |
+
| 500 | 0.0185 | 0.018 | 0.992 | None | 0.00006 | 0.00005 |
|
| 79 |
+
| 1000 | 0.0153 | 0.015 | 0.993 | None | 0.00004 | 0.00004 |
|
| 80 |
+
| 1500 | 0.0085 | 0.008 | 0.996 | None | 0.00003 | 0.00003 |
|
| 81 |
+
| 2000 | 0.0072 | 0.007 | 0.997 | None | 0.00003 | 0.00003 |
|
| 82 |
+
| 1 | 1.5112 | 1.511 | 0.549 | None | 0.00000 | 0.00000 |
|
| 83 |
+
| 50 | 0.4068 | 0.407 | 0.820 | None | 0.00164 | 0.00155 |
|
| 84 |
+
| 250 | 0.0380 | 0.038 | 0.982 | None | 0.00011 | 0.00010 |
|
| 85 |
+
| 500 | 0.0185 | 0.018 | 0.992 | None | 0.00006 | 0.00005 |
|
| 86 |
+
| 1000 | 0.0153 | 0.015 | 0.993 | None | 0.00004 | 0.00004 |
|
| 87 |
+
| 1500 | 0.0085 | 0.008 | 0.996 | None | 0.00003 | 0.00003 |
|
| 88 |
+
| 2000 | 0.0072 | 0.007 | 0.997 | None | 0.00003 | 0.00003 |
|
| 89 |
+
|
| 90 |
+
## The recipe
|
| 91 |
+
|
| 92 |
+
- per step: the teacher (member masked) reads the A rows: block outputs at the sites + logits; the armed model reads the B rows: per-site MSE of standardized LayerNorm'd block outputs against the teacher's at each token's closing byte, mean over served blocks; quiet on A: lambda x KL(teacher || armed) on the A rows; quiet off-domain: lambda x KL(member masked || armed) on one web row + one partner row
|
| 93 |
+
- lambda 2.0; pure Adam lr 0.001 wd 0; 2 x 4096-byte paired rows a step; the arm LAWFUL_16x8 at every block ({'n_slots': 16, 'K': 16, 'D': 8, 'hidden': 256}); the trunk mini-beatrix-3 at step 0; alephllm 0.10.7, amoe-lora 0.2.11, torch 2.8.0+cu128
|
| 94 |
+
- the surface: Qwen3's byte-level BPE spelling (tokenizer.json sha256 be75606093db2094..., vocab 151,669); documents = one caption-like field of a caption-pack row (caption, short, long, tags, medium), sources drawn by {'dff': 0.161, 'scf': 0.161, 'danbooru': 0.164, 'cc12m': 0.406, 'coco': 0.108}
|
| 95 |
+
- the test set: the first 512 captions of each draw of coco_caps_2x2048.json (every caption of the file kept out of training)
|
surface/qwen3/mse_untrained_o0/results.json
ADDED
|
The diff for this file is too large to render.
See raw diff
|
|
|
surface/qwen3/mse_untrained_o0/s10_qwen.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:97d006725dd155f9135a2a6cbc1eeabd8dd868c7ebe479b811089acb66037036
|
| 3 |
+
size 54819288
|
surface/qwen3/mse_untrained_o0/stats.pt
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:b7eba64b2c543b7dfedd0d57d045d54e280ee08f620f5b2af4ff6e78de1c9853
|
| 3 |
+
size 264325
|
surface/qwen3/mse_untrained_o0/train.log
ADDED
|
@@ -0,0 +1,96 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
[ 0.3s] THE JOB: qarm_mse_untrained_o0: the surface arm s10_qwen (gpt2 spelling by /workspace/qarm/dl/qwen3_tokenizer; A = the text written; sites = tokens) alone on an UNTRAINED copy of the trunk (the control that must fail); form mse; blocks all; 2000 steps, a close every 1000; lambda 2.0; pure Adam lr 0.001; the compiled chain with recompute; card 30.5 GB with 30.4 GB free; host RSS 0.6 GB
|
| 2 |
+
Warning: You are sending unauthenticated requests to the HF Hub. Please set a HF_TOKEN to enable higher rate limits and faster downloads.
|
| 3 |
+
[ 16.8s] mini-beatrix-3 @ 245,674: trunk at RANDOM INIT (seed 0) and frozen (376.0M); alephllm 0.10.7 at /workspace/qarm/alephllm/geolip/alephllm; amoe-lora 0.2.11; torch 2.8.0+cu128
|
| 4 |
+
[ 17.3s] tokenizer /workspace/qarm/dl/qwen3_tokenizer (gpt2 spelling; vocab 151669; sha256 be75606093db2094): 'a taco truck' -> ['a', 'Ġtaco', 'Ġtruck']; A 12 bytes, B 14 bytes; sites [(1, 6, 7)]
|
| 5 |
+
[ 17.3s] the read's captions: 512 of each draw of /workspace/qarm/dl/coco_caps_2x2048.json (4,090 captions of both draws kept out of training)
|
| 6 |
+
[ 30.1s] caption pack (/workspace/qarm/dl/mini-beatrix-3/caption_pack/pack_v2, train): dff 121,153 rows, scf 42,080 rows, danbooru 603,026 rows, cc12m 1,242,176 rows, coco 19,227 rows; host RSS 3.7 GB
|
| 7 |
+
[ 30.3s] streams: web text; partners none (two web rows a quiet chunk); held-out sets ['web'] x 8 batches
|
| 8 |
+
[ 30.3s] read rows draw1: 512 captions (0 unspellable), A rows (512, 99), B rows (512, 116), 4,920 sites
|
| 9 |
+
[ 30.4s] read rows draw2: 512 captions (0 unspellable), A rows (512, 151), B rows (512, 176), 5,086 sites
|
| 10 |
+
[arms] attached 's10_qwen' for phase 'qwen_spelling': 13.69M params x 32 blocks, lambda 2.0, spec {'n_slots': 16, 'K': 16, 'D': 8, 'hidden': 256}, recompute on, compile on
|
| 11 |
+
W1008 15:56:59.873000 38461 torch/_inductor/utils.py:1436] [0/0] Not enough SMs to use max_autotune_gemm mode
|
| 12 |
+
/usr/local/lib/python3.12/dist-packages/torch/_inductor/compile_fx.py:282: UserWarning: TensorFloat32 tensor cores for float32 matrix multiplication available but not enabled. Consider setting `torch.set_float32_matmul_precision('high')` for better performance.
|
| 13 |
+
warnings.warn(
|
| 14 |
+
[ 43.1s] BIRTH: the member attached over the trunk: web rows read exactly as before; the hand-run trunk against the model's forward: armed 0.0e+00, masked 0.0e+00 (gates: 0.0)
|
| 15 |
+
[ 47.1s] standardization statistics from 11,974 teacher sites on 8 paired batches (a mean and a spread per block and dimension); sigma range at block 20 0.387..0.835; saved /workspace/qarm/out/anchors/qarm_mse_untrained_o0_stats.pt
|
| 16 |
+
[ 252.6s] CLOSE 0 (step 0): the member's alignment read on draw 1 (512 captions, 4,920 sites); per block: before -> armed B vs masked A (target) / armed B vs armed A; A armed vs masked
|
| 17 |
+
[ 252.6s] block 8: 0.943 -> 0.943 / 0.943; A 1.0000; mse 1.185 cos 0.639
|
| 18 |
+
[ 252.6s] block 12: 0.939 -> 0.939 / 0.939; A 1.0000; mse 1.285 cos 0.650
|
| 19 |
+
[ 252.6s] block 16: 0.933 -> 0.933 / 0.933; A 1.0000; mse 1.308 cos 0.679
|
| 20 |
+
[ 252.6s] block 18: 0.929 -> 0.929 / 0.929; A 1.0000; mse 1.344 cos 0.692
|
| 21 |
+
[ 252.6s] block 20: 0.928 -> 0.928 / 0.928; A 1.0000; mse 1.344 cos 0.706
|
| 22 |
+
[ 252.6s] block 24: 0.925 -> 0.925 / 0.925; A 1.0000; mse 1.331 cos 0.725
|
| 23 |
+
[ 252.6s] block 28: 0.926 -> 0.926 / 0.926; A 1.0000; mse 1.381 cos 0.740
|
| 24 |
+
[ 252.6s] block 31: 0.923 -> 0.923 / 0.923; A 1.0000; mse 1.402 cos 0.749
|
| 25 |
+
[ 252.6s] draw 2 (the repeat), target alignment at 12/16/20/24/28: 0.934 / 0.929 / 0.923 / 0.920 / 0.918
|
| 26 |
+
[ 252.6s] B1 (served blocks >= 12, draw 1): the lowest target alignment 0.923 against the bar .85; B2: the lowest A alignment 1.0000 against .995; fineweb share +0.00000 +- 0.00000 against +.012
|
| 27 |
+
[ 252.6s] gate mean 0.047 / widest 0.047; compiled chain: 2 cache entries (limit 256), 0 limit warnings; docs drawn 256 (skipped {'test': 2, 'spelling': 0, 'empty': 0, 'short': 0}); card peak 2.3 GB; host RSS 10.8 GB
|
| 28 |
+
[stream] opening 'fineweb-edu' (HuggingFaceFW/fineweb-edu) …
|
| 29 |
+
[ 264.2s] step 1/2000: task 1.5112 (mse 1.511 cos 0.549); quiet A 0.00000 off 0.00000; 1,503 sites; 11.58 s/step; the job: 4.4 min spent, about 385.8 min of steps left; card peak 13.7 GB
|
| 30 |
+
[ 384.5s] step 50/2000: task 0.4068 (mse 0.407 cos 0.820); quiet A 0.00164 off 0.00155; 1,508 sites; 2.64 s/step; the job: 6.4 min spent, about 85.7 min of steps left; card peak 13.8 GB
|
| 31 |
+
[ 507.3s] step 100/2000: task 0.1220 (mse 0.122 cos 0.940); quiet A 0.00031 off 0.00030; 1,506 sites; 2.55 s/step; the job: 8.5 min spent, about 80.6 min of steps left; card peak 13.8 GB
|
| 32 |
+
[ 630.2s] step 150/2000: task 0.0751 (mse 0.075 cos 0.964); quiet A 0.00021 off 0.00019; 1,471 sites; 2.52 s/step; the job: 10.5 min spent, about 77.6 min of steps left; card peak 13.8 GB
|
| 33 |
+
[ 753.2s] step 200/2000: task 0.0568 (mse 0.057 cos 0.974); quiet A 0.00015 off 0.00013; 1,483 sites; 2.50 s/step; the job: 12.6 min spent, about 75.1 min of steps left; card peak 13.8 GB
|
| 34 |
+
[ 875.8s] step 250/2000: task 0.0380 (mse 0.038 cos 0.982); quiet A 0.00011 off 0.00010; 1,485 sites; 2.49 s/step; the job: 14.6 min spent, about 72.7 min of steps left; card peak 13.8 GB
|
| 35 |
+
[ 998.7s] step 300/2000: task 0.0338 (mse 0.034 cos 0.984); quiet A 0.00009 off 0.00008; 1,497 sites; 2.49 s/step; the job: 16.6 min spent, about 70.5 min of steps left; card peak 13.8 GB
|
| 36 |
+
[ 1121.8s] step 350/2000: task 0.0300 (mse 0.030 cos 0.986); quiet A 0.00009 off 0.00008; 1,487 sites; 2.48 s/step; the job: 18.7 min spent, about 68.3 min of steps left; card peak 13.8 GB
|
| 37 |
+
[ 1244.8s] step 400/2000: task 0.0244 (mse 0.024 cos 0.989); quiet A 0.00007 off 0.00007; 1,505 sites; 2.48 s/step; the job: 20.7 min spent, about 66.1 min of steps left; card peak 13.8 GB
|
| 38 |
+
[ 1367.4s] step 450/2000: task 0.0239 (mse 0.024 cos 0.989); quiet A 0.00007 off 0.00006; 1,593 sites; 2.48 s/step; the job: 22.8 min spent, about 64.0 min of steps left; card peak 13.8 GB
|
| 39 |
+
[ 1490.3s] step 500/2000: task 0.0185 (mse 0.018 cos 0.992); quiet A 0.00006 off 0.00005; 1,535 sites; 2.48 s/step; the job: 24.8 min spent, about 61.9 min of steps left; card peak 13.8 GB
|
| 40 |
+
[ 1613.2s] step 550/2000: task 0.0177 (mse 0.018 cos 0.992); quiet A 0.00006 off 0.00005; 1,433 sites; 2.47 s/step; the job: 26.9 min spent, about 59.8 min of steps left; card peak 13.8 GB
|
| 41 |
+
[ 1736.0s] step 600/2000: task 0.0194 (mse 0.019 cos 0.991); quiet A 0.00005 off 0.00006; 1,487 sites; 2.47 s/step; the job: 28.9 min spent, about 57.7 min of steps left; card peak 13.8 GB
|
| 42 |
+
[ 1858.6s] step 650/2000: task 0.0175 (mse 0.017 cos 0.992); quiet A 0.00005 off 0.00004; 1,558 sites; 2.47 s/step; the job: 31.0 min spent, about 55.6 min of steps left; card peak 13.8 GB
|
| 43 |
+
[ 1981.4s] step 700/2000: task 0.0149 (mse 0.015 cos 0.993); quiet A 0.00004 off 0.00004; 1,429 sites; 2.47 s/step; the job: 33.0 min spent, about 53.5 min of steps left; card peak 13.8 GB
|
| 44 |
+
[ 2104.2s] step 750/2000: task 0.0143 (mse 0.014 cos 0.994); quiet A 0.00004 off 0.00004; 1,468 sites; 2.47 s/step; the job: 35.1 min spent, about 51.4 min of steps left; card peak 13.8 GB
|
| 45 |
+
[ 2227.0s] step 800/2000: task 0.0126 (mse 0.013 cos 0.994); quiet A 0.00004 off 0.00004; 1,541 sites; 2.47 s/step; the job: 37.1 min spent, about 49.4 min of steps left; card peak 13.8 GB
|
| 46 |
+
[ 2349.6s] step 850/2000: task 0.0137 (mse 0.014 cos 0.994); quiet A 0.00004 off 0.00004; 1,440 sites; 2.47 s/step; the job: 39.2 min spent, about 47.3 min of steps left; card peak 13.8 GB
|
| 47 |
+
[ 2472.5s] step 900/2000: task 0.0129 (mse 0.013 cos 0.994); quiet A 0.00004 off 0.00004; 1,652 sites; 2.47 s/step; the job: 41.2 min spent, about 45.2 min of steps left; card peak 13.8 GB
|
| 48 |
+
[ 2595.3s] step 950/2000: task 0.0112 (mse 0.011 cos 0.995); quiet A 0.00004 off 0.00004; 1,550 sites; 2.47 s/step; the job: 43.3 min spent, about 43.2 min of steps left; card peak 13.8 GB
|
| 49 |
+
[ 2718.1s] step 1000/2000: task 0.0153 (mse 0.015 cos 0.993); quiet A 0.00004 off 0.00004; 1,493 sites; 2.47 s/step; the job: 45.3 min spent, about 41.1 min of steps left; card peak 13.8 GB
|
| 50 |
+
[ 2718.3s] anchor saved: /workspace/qarm/out/anchors/qarm_mse_untrained_o0_close1_s10_qwen.safetensors
|
| 51 |
+
[ 2871.6s] CLOSE 1 (step 1000): the member's alignment read on draw 1 (512 captions, 4,920 sites); per block: before -> armed B vs masked A (target) / armed B vs armed A; A armed vs masked
|
| 52 |
+
[ 2871.6s] block 8: 0.943 -> 0.989 / 0.989; A 0.9977; mse 0.009 cos 0.996
|
| 53 |
+
[ 2871.6s] block 12: 0.939 -> 0.989 / 0.988; A 0.9987; mse 0.009 cos 0.996
|
| 54 |
+
[ 2871.6s] block 16: 0.933 -> 0.984 / 0.985; A 0.9977; mse 0.010 cos 0.997
|
| 55 |
+
[ 2871.6s] block 18: 0.929 -> 0.984 / 0.983; A 0.9980; mse 0.011 cos 0.997
|
| 56 |
+
[ 2871.6s] block 20: 0.928 -> 0.985 / 0.984; A 0.9968; mse 0.011 cos 0.997
|
| 57 |
+
[ 2871.6s] block 24: 0.925 -> 0.985 / 0.983; A 0.9948; mse 0.011 cos 0.997
|
| 58 |
+
[ 2871.6s] block 28: 0.926 -> 0.985 / 0.983; A 0.9982; mse 0.012 cos 0.997
|
| 59 |
+
[ 2871.6s] block 31: 0.923 -> 0.986 / 0.984; A 0.9950; mse 0.013 cos 0.997
|
| 60 |
+
[ 2871.6s] draw 2 (the repeat), target alignment at 12/16/20/24/28: 0.984 / 0.984 / 0.985 / 0.983 / 0.984
|
| 61 |
+
[ 2871.6s] B1 (served blocks >= 12, draw 1): the lowest target alignment 0.984 against the bar .85; B2: the lowest A alignment 0.9927 against .995; fineweb share +0.00163 +- 0.00016 against +.012
|
| 62 |
+
[ 2871.6s] gate mean 0.053 / widest 0.072; compiled chain: 5 cache entries (limit 256), 0 limit warnings; docs drawn 32,523 (skipped {'test': 44, 'spelling': 0, 'empty': 0, 'short': 0}); card peak 13.8 GB; host RSS 13.5 GB
|
| 63 |
+
[ 2994.5s] step 1050/2000: task 0.0112 (mse 0.011 cos 0.995); quiet A 0.00004 off 0.00004; 1,510 sites; 2.46 s/step; the job: 49.9 min spent, about 39.0 min of steps left; card peak 13.8 GB
|
| 64 |
+
[ 3117.6s] step 1100/2000: task 0.0095 (mse 0.010 cos 0.996); quiet A 0.00004 off 0.00004; 1,491 sites; 2.46 s/step; the job: 52.0 min spent, about 37.0 min of steps left; card peak 13.8 GB
|
| 65 |
+
[ 3240.7s] step 1150/2000: task 0.0124 (mse 0.012 cos 0.995); quiet A 0.00004 off 0.00004; 1,475 sites; 2.46 s/step; the job: 54.0 min spent, about 34.9 min of steps left; card peak 13.8 GB
|
| 66 |
+
[ 3363.8s] step 1200/2000: task 0.0107 (mse 0.011 cos 0.995); quiet A 0.00004 off 0.00003; 1,492 sites; 2.46 s/step; the job: 56.1 min spent, about 32.9 min of steps left; card peak 13.8 GB
|
| 67 |
+
[ 3486.8s] step 1250/2000: task 0.0094 (mse 0.009 cos 0.996); quiet A 0.00003 off 0.00004; 1,473 sites; 2.46 s/step; the job: 58.1 min spent, about 30.8 min of steps left; card peak 13.8 GB
|
| 68 |
+
[ 3610.2s] step 1300/2000: task 0.0100 (mse 0.010 cos 0.996); quiet A 0.00003 off 0.00003; 1,475 sites; 2.46 s/step; the job: 60.2 min spent, about 28.8 min of steps left; card peak 13.8 GB
|
| 69 |
+
[ 3733.4s] step 1350/2000: task 0.0101 (mse 0.010 cos 0.996); quiet A 0.00003 off 0.00003; 1,476 sites; 2.46 s/step; the job: 62.2 min spent, about 26.7 min of steps left; card peak 13.8 GB
|
| 70 |
+
[ 3856.6s] step 1400/2000: task 0.0100 (mse 0.010 cos 0.996); quiet A 0.00003 off 0.00004; 1,519 sites; 2.46 s/step; the job: 64.3 min spent, about 24.6 min of steps left; card peak 13.8 GB
|
| 71 |
+
[ 3979.6s] step 1450/2000: task 0.0109 (mse 0.011 cos 0.995); quiet A 0.00003 off 0.00003; 1,500 sites; 2.46 s/step; the job: 66.3 min spent, about 22.6 min of steps left; card peak 13.8 GB
|
| 72 |
+
[ 4102.8s] step 1500/2000: task 0.0085 (mse 0.008 cos 0.996); quiet A 0.00003 off 0.00003; 1,442 sites; 2.46 s/step; the job: 68.4 min spent, about 20.5 min of steps left; card peak 13.8 GB
|
| 73 |
+
[ 4225.8s] step 1550/2000: task 0.0102 (mse 0.010 cos 0.996); quiet A 0.00003 off 0.00003; 1,466 sites; 2.46 s/step; the job: 70.4 min spent, about 18.5 min of steps left; card peak 13.8 GB
|
| 74 |
+
[ 4348.9s] step 1600/2000: task 0.0087 (mse 0.009 cos 0.996); quiet A 0.00003 off 0.00003; 1,480 sites; 2.46 s/step; the job: 72.5 min spent, about 16.4 min of steps left; card peak 13.8 GB
|
| 75 |
+
[ 4471.8s] step 1650/2000: task 0.0081 (mse 0.008 cos 0.997); quiet A 0.00003 off 0.00005; 1,535 sites; 2.46 s/step; the job: 74.5 min spent, about 14.4 min of steps left; card peak 13.8 GB
|
| 76 |
+
[ 4595.0s] step 1700/2000: task 0.0108 (mse 0.011 cos 0.996); quiet A 0.00003 off 0.00003; 1,499 sites; 2.46 s/step; the job: 76.6 min spent, about 12.3 min of steps left; card peak 13.8 GB
|
| 77 |
+
[ 4718.4s] step 1750/2000: task 0.0066 (mse 0.007 cos 0.997); quiet A 0.00003 off 0.00003; 1,463 sites; 2.46 s/step; the job: 78.6 min spent, about 10.3 min of steps left; card peak 13.8 GB
|
| 78 |
+
[ 4841.8s] step 1800/2000: task 0.0069 (mse 0.007 cos 0.997); quiet A 0.00003 off 0.00003; 1,531 sites; 2.46 s/step; the job: 80.7 min spent, about 8.2 min of steps left; card peak 13.8 GB
|
| 79 |
+
[ 4965.0s] step 1850/2000: task 0.0064 (mse 0.006 cos 0.997); quiet A 0.00003 off 0.00003; 1,498 sites; 2.46 s/step; the job: 82.8 min spent, about 6.2 min of steps left; card peak 13.8 GB
|
| 80 |
+
[ 5088.2s] step 1900/2000: task 0.0086 (mse 0.009 cos 0.996); quiet A 0.00003 off 0.00003; 1,601 sites; 2.46 s/step; the job: 84.8 min spent, about 4.1 min of steps left; card peak 13.8 GB
|
| 81 |
+
[ 5211.7s] step 1950/2000: task 0.0090 (mse 0.009 cos 0.996); quiet A 0.00003 off 0.00003; 1,500 sites; 2.46 s/step; the job: 86.9 min spent, about 2.1 min of steps left; card peak 13.8 GB
|
| 82 |
+
[ 5335.0s] step 2000/2000: task 0.0072 (mse 0.007 cos 0.997); quiet A 0.00003 off 0.00003; 1,501 sites; 2.46 s/step; the job: 88.9 min spent, about 0.0 min of steps left; card peak 13.8 GB
|
| 83 |
+
[ 5335.2s] anchor saved: /workspace/qarm/out/anchors/qarm_mse_untrained_o0_close2_s10_qwen.safetensors
|
| 84 |
+
[ 5488.7s] CLOSE 2 (step 2000): the member's alignment read on draw 1 (512 captions, 4,920 sites); per block: before -> armed B vs masked A (target) / armed B vs armed A; A armed vs masked
|
| 85 |
+
[ 5488.7s] block 8: 0.943 -> 0.990 / 0.991; A 0.9986; mse 0.006 cos 0.997
|
| 86 |
+
[ 5488.7s] block 12: 0.939 -> 0.991 / 0.990; A 0.9987; mse 0.006 cos 0.998
|
| 87 |
+
[ 5488.7s] block 16: 0.933 -> 0.985 / 0.986; A 0.9984; mse 0.007 cos 0.998
|
| 88 |
+
[ 5488.7s] block 18: 0.929 -> 0.989 / 0.987; A 0.9966; mse 0.007 cos 0.998
|
| 89 |
+
[ 5488.7s] block 20: 0.928 -> 0.988 / 0.986; A 0.9982; mse 0.007 cos 0.998
|
| 90 |
+
[ 5488.7s] block 24: 0.925 -> 0.988 / 0.986; A 0.9977; mse 0.008 cos 0.998
|
| 91 |
+
[ 5488.7s] block 28: 0.926 -> 0.990 / 0.989; A 0.9985; mse 0.008 cos 0.998
|
| 92 |
+
[ 5488.7s] block 31: 0.923 -> 0.990 / 0.988; A 0.9987; mse 0.009 cos 0.998
|
| 93 |
+
[ 5488.7s] draw 2 (the repeat), target alignment at 12/16/20/24/28: 0.986 / 0.987 / 0.986 / 0.986 / 0.986
|
| 94 |
+
[ 5488.7s] B1 (served blocks >= 12, draw 1): the lowest target alignment 0.985 against the bar .85; B2: the lowest A alignment 0.9952 against .995; fineweb share +0.00217 +- 0.00018 against +.012
|
| 95 |
+
[ 5488.7s] gate mean 0.055 / widest 0.083; compiled chain: 5 cache entries (limit 256), 0 limit warnings; docs drawn 64,564 (skipped {'test': 97, 'spelling': 1, 'empty': 0, 'short': 0}); card peak 13.8 GB; host RSS 13.5 GB
|
| 96 |
+
[ 5488.7s] DONE qarm_mse_untrained_o0: 2000 steps in 82.1 min of steps (2.46 s/step); 2 closes; 5489 s in all; card peak 13.8 GB; host RSS 13.5 GB; compiled chain: 5 cache entries (limit 256), 0 limit warnings
|