Commit History

  1. feat: update RTX 5090 notebook runner and optimized config

    Verified signature
    1dfd5ce

  1. full ckpt step_1000 (resume)

    Verified signature
    1de08d3
  2. config: set Stage 1 micro 32 accum 4 for peak Tensor Core speed

    Verified signature
    8866f9e
  3. config: set micro 16 accum 8 with grad_ckpt true for safe high-throughput pretraining

    Verified signature
    97527fa
  4. fix: restore grad_ckpt and safe micro 8 accum 8 for zero-OOM guarantee on T4

    Verified signature
    96be75f
  5. perf: disable grad_ckpt on T4 and enlarge loss chunk_size to 2048 for 10k+ tok/s

    Verified signature
    7d4c3f5
  6. config: update Stage 1 batch_size to 16 for micro=16 accum=8 per GPU

    Verified signature
    20ebd23
  7. refactor: consolidate pure 32L Dense architecture, purge MoE files, audit hardening

    Verified signature
    3a5d3e5
  8. Update model/configs/train_kaggle_t4.yaml: log_every: 10 and real-time step streaming

    Verified signature
    a68274b
  9. Remove dead config model/configs/model_config.yaml

    Verified signature
    f3b7a40
  10. Remove dead config model/configs/viu_1_5b_moe_config.yaml

    Verified signature
    c4713b4
  11. Remove dead config model/configs/viu1_moe_config.yaml

    Verified signature
    ff9965d
  12. Archive model_config.yaml to deprecated folder

    Verified signature
    629d126
  13. Archive viu_1_5b_moe_config.yaml to deprecated folder

    Verified signature
    96ebcea
  14. Archive viu1_moe_config.yaml to deprecated folder

    Verified signature
    e910481
  15. Fix model/scripts/train.py: batch math comments and find_unused_parameters=False

    Verified signature
    ef65129
  16. Fix model/configs/train_config.yaml: batch math comments and find_unused_parameters=False

    Verified signature
    d7975ca
  17. Audit fix: Update model/scripts/train.py for ViuMini-Dense-360M

    Verified signature
    e52f2b4
  18. Audit fix: Update model/configs/viu_1_5b_moe_config.yaml for ViuMini-Dense-360M

    Verified signature
    5d2e7a3
  19. Audit fix: Update model/configs/viu1_moe_config.yaml for ViuMini-Dense-360M

    Verified signature
    a072aa0
  20. Audit fix: Update model/configs/model_config.yaml for ViuMini-Dense-360M

    Verified signature
    d32d6e4
  21. Audit fix: Update model/configs/train_dense_baseline.yaml for ViuMini-Dense-360M

    Verified signature
    41869c6
  22. Audit fix: Update model/configs/train_h100_h200.yaml for ViuMini-Dense-360M

    Verified signature
    232a8e4
  23. Audit fix: Update model/configs/train_rtx5090.yaml for ViuMini-Dense-360M

    Verified signature
    70b87d7
  24. Audit fix: Update model/configs/train_rtx4090.yaml for ViuMini-Dense-360M

    Verified signature
    d33c293
  25. Audit fix: Update model/configs/train_rtx3090.yaml for ViuMini-Dense-360M

    Verified signature
    da2ff29
  26. Audit fix: Update model/configs/config.json for ViuMini-Dense-360M

    Verified signature
    6e8fe69
  27. Audit fix: Update model/configs/viu1_dense_config.yaml for ViuMini-Dense-360M

    Verified signature
    d8ba5af
  28. Audit fix: Update model/NOTES.md for ViuMini-Dense-360M

    Verified signature
    42788da
  29. Update model/scripts/push_to_hf.py for ViuMini-Dense-360M

    Verified signature
    d9702de
  30. Update model/scripts/train.py for ViuMini-Dense-360M

    Verified signature
    dedfdc7
  31. Update model/configs/train_rtx5090.yaml for ViuMini-Dense-360M

    Verified signature
    a2f494b
  32. Update model/configs/train_rtx4090.yaml for ViuMini-Dense-360M

    Verified signature
    c8a4a4d
  33. Update model/configs/train_rtx3090.yaml for ViuMini-Dense-360M

    Verified signature
    ee5595d
  34. Update model/configs/train_dense_baseline.yaml for ViuMini-Dense-360M

    Verified signature
    08fb77d
  35. Update model/configs/train_config.yaml for ViuMini-Dense-360M

    Verified signature
    6a651bd
  36. Update model/configs/train_kaggle_t4.yaml for ViuMini-Dense-360M

    Verified signature
    c01244c
  37. Update config.json for 32 layers, dim 960

    Verified signature
    a1ebc75
  38. Set Option A: 32 layers, dim 960 (366.6M params, SmolLM-style)

    Verified signature
    85ae4bc
  39. Upgrade architecture to ViuMini-Dense-350M (100% Dense, 360M active)

    Verified signature
    94036c7
  40. Upgrade architecture to ViuMini-Dense-350M (100% Dense, 360M active)

    Verified signature
    5f90a9a
  41. Upgrade architecture to ViuMini-Dense-350M (100% Dense, 360M active)

    Verified signature
    4c9d68e
  42. Upgrade architecture to ViuMini-Dense-350M (100% Dense, 360M active)

    Verified signature
    4689228
  43. Set checkpoint save, eval, sample, and HF push intervals to 1000 steps

    Verified signature
    ae9be98
  44. Remove diverged model/checkpoints/step_3000.pt for clean AdamW start

    Verified signature
    b4f5377
  45. Remove diverged model/checkpoints/step_2500.pt for clean AdamW start

    Verified signature
    f639085
  46. Remove diverged model/checkpoints/step_2000.pt for clean AdamW start

    Verified signature
    3564d08
  47. Switch to PagedAdamW8bit (lr=2.5e-4, warmup=1000) for 100% stable convergence

    Verified signature
    891fd91
  48. prune old ckpt model/checkpoints/step_1500.pt

    Verified signature
    dd5116b
  49. full ckpt step_3000 (resume)

    Verified signature
    39940a6