next JEV stage 1 (CoT-only) checkpoint

Stage 1 of two-stage next JEV training: Qwen3.5-0.8B with a recurrent latent workspace, trained only on the cross-attention CoT auxiliary loss (stage1_objective: cot_only). Use it as train.init_checkpoint for stage 2.

  • Base model: Qwen/Qwen3.5-0.8B @ 2fc06364715b967f1860aea9cf38778875588b17
  • Code: github.com/JonnesLin/next_jev, config configs/stage1_multi_response.json (commit 9fbccf5)
  • Workspace: 128 tokens, 3 loops, middle_workspace blocks 1-24
  • Data: JonesLin/multi-model-cot-2730 @ e1b9098867d0ae6b4a8a0e1f68623f748bebe65c, 406,167 train / 8,211 validation prompts
  • Training: 1 epoch, 6347 optimizer steps, global batch 64 on 2x H100 NVL, lr 8e-05 -> 8e-06 cosine, BF16 autocast with FP32 master weights
  • Validation: loss 0.8931, CoT aux loss 2.9769, answer loss 0.0
  • W&B: https://wandb.ai/fast-ssl/next-jev/runs/25214426e4ea472c9a9559a6182b4202

checkpoint.pt holds FP32 weights, Adam state and training state (local paths from the training host are included). Load with next_jev.train.load_checkpoint.

Downloads last month

-

Downloads are not tracked for this model. How to track
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Model tree for JonesLin/next-jev-stage1-cot

Finetuned
(427)
this model