Audio-Text-to-Text
PEFT
Safetensors
audio
audio-reasoning
reinforcement-learning
grpo
self-supervised-learning
lora
Instructions to use NearAndDear98/Audio-R1-AF-Next-Think with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- PEFT
How to use NearAndDear98/Audio-R1-AF-Next-Think with PEFT:
from peft import PeftModel from transformers import AutoModelForCausalLM base_model = AutoModelForCausalLM.from_pretrained("nvidia/audio-flamingo-next-think-hf") model = PeftModel.from_pretrained(base_model, "NearAndDear98/Audio-R1-AF-Next-Think") - Notebooks
- Google Colab
- Kaggle
Remove training section
Browse files
README.md
CHANGED
|
@@ -21,17 +21,6 @@ LoRA adapter for [nvidia/audio-flamingo-next-think-hf](https://huggingface.co/nv
|
|
| 21 |
- **Paper:** [Audio-R1: Audio Reasoning through Reinforcement Learning on Self-Supervised Tasks](https://arxiv.org/abs/2610.34609)
|
| 22 |
- **Project page:** https://ijihoon98.github.io/Audio-R1/
|
| 23 |
|
| 24 |
-
## Training
|
| 25 |
-
|
| 26 |
-
| | |
|
| 27 |
-
|---|---|
|
| 28 |
-
| Method | GRPO (ms-swift), 8 rollouts per prompt, temperature 0.8, β = 0.04 |
|
| 29 |
-
| Tasks | Identity Detection + Region Inpainting (joint, 2 tasks; Temporal Ordering is left out for this backbone, see the paper's appendix) |
|
| 30 |
-
| Rewards | think-format guard + answer-letter correctness |
|
| 31 |
-
| Audio sources | LibriSpeech, VoxCeleb2, MELD, CoVoST2 (en/zh), multi-speaker mixtures, AF-Think |
|
| 32 |
-
| LoRA | r = 8, α = 32, dropout 0.05, language model + audio projector; audio encoder frozen |
|
| 33 |
-
| Optimization | lr 5e-6, 1 epoch, 1,906 steps, effective batch 32 prompts |
|
| 34 |
-
|
| 35 |
## Usage
|
| 36 |
|
| 37 |
```python
|
|
|
|
| 21 |
- **Paper:** [Audio-R1: Audio Reasoning through Reinforcement Learning on Self-Supervised Tasks](https://arxiv.org/abs/2610.34609)
|
| 22 |
- **Project page:** https://ijihoon98.github.io/Audio-R1/
|
| 23 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 24 |
## Usage
|
| 25 |
|
| 26 |
```python
|