Add training details
Browse files
README.md
CHANGED
|
@@ -25,7 +25,6 @@ LoRA adapter for [nvidia/audio-flamingo-next-think-hf](https://huggingface.co/nv
|
|
| 25 |
|
| 26 |
| Hyperparameter | Value |
|
| 27 |
| :--- | :--- |
|
| 28 |
-
| Tasks | identity · inpainting |
|
| 29 |
| Method | GRPO, 1 epoch, LoRA |
|
| 30 |
| Reward | accuracy (1.0 if the chosen letter is correct) + format (0.2) |
|
| 31 |
| Format reward | non-empty `<think>` (the model answers without an `<answer>` tag) |
|
|
@@ -35,10 +34,8 @@ LoRA adapter for [nvidia/audio-flamingo-next-think-hf](https://huggingface.co/nv
|
|
| 35 |
| Sampling temperature / top-p | 0.8 / 0.9 |
|
| 36 |
| KL coefficient β | 0.04 |
|
| 37 |
| Clip ε | 0.2 |
|
| 38 |
-
| Max completion length | 512 |
|
| 39 |
| Per-device batch × grad. accumulation | 8 × 4 |
|
| 40 |
| GPUs | 4 |
|
| 41 |
-
| Training steps | 1,906 |
|
| 42 |
| LoRA rank / α / dropout | 8 / 32 / 0.05 |
|
| 43 |
| LoRA targets | all linear layers; audio encoder frozen |
|
| 44 |
| Precision / seed | bf16 / 42 |
|
|
|
|
| 25 |
|
| 26 |
| Hyperparameter | Value |
|
| 27 |
| :--- | :--- |
|
|
|
|
| 28 |
| Method | GRPO, 1 epoch, LoRA |
|
| 29 |
| Reward | accuracy (1.0 if the chosen letter is correct) + format (0.2) |
|
| 30 |
| Format reward | non-empty `<think>` (the model answers without an `<answer>` tag) |
|
|
|
|
| 34 |
| Sampling temperature / top-p | 0.8 / 0.9 |
|
| 35 |
| KL coefficient β | 0.04 |
|
| 36 |
| Clip ε | 0.2 |
|
|
|
|
| 37 |
| Per-device batch × grad. accumulation | 8 × 4 |
|
| 38 |
| GPUs | 4 |
|
|
|
|
| 39 |
| LoRA rank / α / dropout | 8 / 32 / 0.05 |
|
| 40 |
| LoRA targets | all linear layers; audio encoder frozen |
|
| 41 |
| Precision / seed | bf16 / 42 |
|