JonesLin/next-jev-stage2-last-block

next JEV stage 2, last-block CoT decoder (mid-training checkpoint)

Stage 2 (choice head + CoT) initialized from stage 1 JonesLin/next-jev-stage1-last-block step 2600. The CoT decoder (aux_decoder: last_block) is a cross-attention block over the 128 workspace tokens, then a copy of Qwen3.5-0.8B's last block and final norm, with tied embeddings in and out. One pass through the backbone (loops: 1). Training is still running; this is step 3100 of 9181 (not a final model, not validated).

• Checkpoint: checkpoint-00003100/ (checkpoint.pt = FP32 weights + Adam state + training state; load with next_jev.train.load_checkpoint)
• Code: github.com/JonnesLin/next_jev, branch feat/stage1-last-block-decoder (commit 0e8521f), config configs/stage2_last_block.json
• Data: 587,548 train rows: the stage 2 choice set (387,548 rows: NLI from text, image and VQA sources, plus Tetris) and 200,000 choice questions from JonesLin/next-jev-choice-cot-200k, 52,479 of them with verified teacher CoT (claude-haiku-4-5: 22,565, grok-4.7: 29,914). NLI options are reworded during training (nli_option_rewrite).
• Training: global batch 64 on 2x H100 NVL, lr 8e-06 for backbone and decoder, 8e-05 for the choice head, cosine to 10%, no warmup, BF16 autocast with FP32 master weights
• metrics.jsonl: per-step training metrics up to step 3100
• eval.jsonl: every 500 steps, choice accuracy on the 2,601-row eval set (overall and per source) and greedy decodes of its first 10 rows. Accuracy: 0.775 at step 500, 0.882 at step 3000 (Tetris 0.968).
• W&B: https://wandb.ai/fast-ssl/next-jev/runs/0466ae9254ba4959ad7a2ba86016df39