---
library_name: transformers
tags:
- generated_from_trainer
model-index:
- name: hypa-tiny-keys
  results: []
---

<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# hypa-tiny-keys

This model is a fine-tuned version of [](https://huggingface.co/) on the None dataset.
It achieves the following results on the evaluation set:
- Loss: 1.1994

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 0.0003
- train_batch_size: 256
- eval_batch_size: 256
- seed: 3407
- optimizer: Use OptimizerNames.ADAMW_TORCH_FUSED with betas=(0.9,0.95) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
- lr_scheduler_type: cosine
- lr_scheduler_warmup_steps: 500
- training_steps: 183476

### Training results

| Training Loss | Epoch  | Step   | Validation Loss |
|:-------------:|:------:|:------:|:---------------:|
| 2.0830        | 0.0250 | 4587   | 1.5913          |
| 1.4610        | 0.0500 | 9174   | 1.5099          |
| 1.3978        | 0.0750 | 13761  | 1.4691          |
| 1.3647        | 0.1000 | 18348  | 1.4470          |
| 1.3525        | 0.1250 | 22935  | 1.4381          |
| 1.3103        | 0.1500 | 27522  | 1.4193          |
| 1.3251        | 0.1750 | 32109  | 1.4158          |
| 1.2983        | 0.2000 | 36696  | 1.4092          |
| 1.2916        | 0.2250 | 41283  | 1.4117          |
| 1.2891        | 0.2500 | 45870  | 1.3917          |
| 1.2785        | 0.2750 | 50457  | 1.3796          |
| 1.2719        | 0.3000 | 55044  | 1.3855          |
| 1.2648        | 0.3250 | 59631  | 1.3817          |
| 1.2662        | 0.3500 | 64218  | 1.3654          |
| 1.2446        | 0.3750 | 68805  | 1.3616          |
| 1.2584        | 0.4000 | 73392  | 1.3651          |
| 1.2331        | 0.4250 | 77979  | 1.3469          |
| 1.2437        | 0.4500 | 82566  | 1.3525          |
| 1.2234        | 0.4750 | 87153  | 1.3285          |
| 1.2318        | 0.5000 | 91740  | 1.3416          |
| 1.2186        | 0.5250 | 96327  | 1.3188          |
| 1.2173        | 0.5500 | 100914 | 1.3146          |
| 1.2145        | 0.5750 | 105501 | 1.3058          |
| 1.1944        | 0.6000 | 110088 | 1.3002          |
| 1.2025        | 0.6250 | 114675 | 1.2898          |
| 1.1773        | 0.6500 | 119262 | 1.2806          |
| 1.1972        | 0.6750 | 123849 | 1.2718          |
| 1.1623        | 0.7000 | 128436 | 1.2427          |
| 1.1875        | 0.7250 | 133023 | 1.2537          |
| 1.1675        | 0.7500 | 137610 | 1.2442          |
| 1.1576        | 0.7750 | 142197 | 1.2418          |
| 1.1725        | 0.8000 | 146784 | 1.2313          |
| 1.1483        | 0.8250 | 151371 | 1.2312          |
| 1.1634        | 0.8500 | 155958 | 1.2177          |
| 1.1455        | 0.8750 | 160545 | 1.2165          |
| 1.1637        | 0.9000 | 165132 | 1.2105          |
| 1.1420        | 0.9250 | 169719 | 1.2045          |
| 1.1645        | 0.9500 | 174306 | 1.2024          |
| 1.1522        | 0.9750 | 178893 | 1.2000          |
| 1.1522        | 1.0    | 183476 | 1.1994          |


### Framework versions

- Transformers 5.17.0
- Pytorch 2.8.0+cu128
- Datasets 5.0.1
- Tokenizers 0.23.2