Future.GQ.47s / README.md
Plans11's picture
Model save
eb6d5a5 verified
|
Raw History Blame Contribute Delete
1.93 kB
metadata
library_name: transformers
tags:
  - generated_from_trainer
model-index:
  - name: Future.GQ.47s
    results: []

Future.GQ.47s

This model was trained from scratch on the None dataset. It achieves the following results on the evaluation set:

  • Loss: 2.6928

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

  • learning_rate: 0.00015
  • train_batch_size: 4
  • eval_batch_size: 16
  • seed: 42
  • gradient_accumulation_steps: 16
  • total_train_batch_size: 64
  • optimizer: Use OptimizerNames.ADAMW_TORCH_FUSED with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
  • lr_scheduler_type: cosine
  • lr_scheduler_warmup_steps: 0.03
  • num_epochs: 1
  • mixed_precision_training: Native AMP

Training results

Training Loss Epoch Step Validation Loss
1.9503 0.9889 100 1.8953
1.9503 1.0 102 1.8949
4.8422 0.2032 200 4.6299
3.7624 0.3048 300 3.7300
3.2549 0.4064 400 3.2797
3.0663 0.5080 500 3.0268
2.8727 0.6096 600 2.8719
2.7585 0.7112 700 2.7737
2.7077 0.8129 800 2.7186
2.6603 0.9145 900 2.6958
2.6459 1.0 985 2.6928

Framework versions

  • Transformers 5.14.1
  • Pytorch 2.10.0+cu128
  • Datasets 5.0.1
  • Tokenizers 0.22.2