ChessQween3-base-puzzled

This model is a fine-tuned version of zhoudoe23/ChessQween3-base on an unknown dataset. It achieves the following results on the evaluation set:

  • Loss: 0.8700

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

  • learning_rate: 3e-05
  • train_batch_size: 128
  • eval_batch_size: 64
  • seed: 42
  • distributed_type: multi-GPU
  • num_devices: 2
  • gradient_accumulation_steps: 2
  • total_train_batch_size: 512
  • total_eval_batch_size: 128
  • optimizer: Use OptimizerNames.ADAMW_TORCH_FUSED with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
  • lr_scheduler_type: cosine
  • lr_scheduler_warmup_steps: 1000
  • num_epochs: 1
  • mixed_precision_training: Native AMP

Training results

Training Loss Epoch Step Validation Loss
1.1666 0.0233 1000 1.1704
1.0454 0.0466 2000 1.1195
1.1005 0.0700 3000 1.0880
1.0877 0.0933 4000 1.0663
1.0387 0.1166 5000 1.0488
1.1939 0.1399 6000 1.0355
1.1149 0.1633 7000 1.0199
1.0303 0.1866 8000 1.0077
1.0135 0.2099 9000 0.9969
1.0281 0.2332 10000 0.9883
0.9677 0.2566 11000 0.9776
1.0151 0.2799 12000 0.9704
0.9752 0.3032 13000 0.9628
0.9267 0.3265 14000 0.9540
1.0725 0.3499 15000 0.9486
1.0237 0.3732 16000 0.9456
0.9124 0.3965 17000 0.9355
0.8746 0.4198 18000 0.9291
1.0226 0.4431 19000 0.9235
0.8150 0.4665 20000 0.9190
0.9453 0.4898 21000 0.9152
0.8654 0.5131 22000 0.9089
1.0067 0.5364 23000 0.9057
0.8130 0.5598 24000 0.9016
0.9912 0.5831 25000 0.8974
0.8276 0.6064 26000 0.8950
0.9481 0.6297 27000 0.8912
0.9450 0.6531 28000 0.8884
0.8170 0.6764 29000 0.8853
0.9074 0.6997 30000 0.8833
0.9123 0.7230 31000 0.8817
0.8216 0.7464 32000 0.8783
0.8683 0.7697 33000 0.8765
0.8481 0.7930 34000 0.8748
0.8536 0.8163 35000 0.8735
0.7900 0.8397 36000 0.8729
0.8798 0.8630 37000 0.8718
0.8715 0.8863 38000 0.8712
0.8726 0.9096 39000 0.8705
0.8342 0.9329 40000 0.8702
0.8996 0.9563 41000 0.8700
0.8560 0.9796 42000 0.8700

Framework versions

  • Transformers 5.0.0
  • Pytorch 2.10.0+cu128
  • Datasets 5.0.0
  • Tokenizers 0.22.2
Downloads last month
156
Safetensors
Model size
0.1B params
Tensor type
F32
ยท
Inference Providers NEW
This model isn't deployed by any Inference Provider. ๐Ÿ™‹ Ask for provider support

Model tree for zhoudoe23/ChessQween3-base-puzzled

Finetuned
(1)
this model

Space using zhoudoe23/ChessQween3-base-puzzled 1