Add Audio Flamingo 3 and Next GGUF models

.gitattributes

@@ -33,3 +33,9 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *.zip filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
+audio-flamingo-3-bf16.gguf filter=lfs diff=lfs merge=lfs -text
+audio-flamingo-3-q4_k.gguf filter=lfs diff=lfs merge=lfs -text
+audio-flamingo-3-q8_0.gguf filter=lfs diff=lfs merge=lfs -text
+audio-flamingo-next-bf16.gguf filter=lfs diff=lfs merge=lfs -text
+audio-flamingo-next-q4_k.gguf filter=lfs diff=lfs merge=lfs -text
+audio-flamingo-next-q8_0.gguf filter=lfs diff=lfs merge=lfs -text

README.md

@@ -0,0 +1,92 @@
+---
+license: other
+license_name: nvidia-oneway-noncommercial
+license_link: https://huggingface.co/nvidia/audio-flamingo-next-hf/blob/5634886e2615c2f587dcf8b93c6edfe9907930ca/static/NVIDIA-OneWay-Noncommercial-License_22Mar2022-academic.docx
+base_model:
+  - nvidia/audio-flamingo-3-hf
+  - nvidia/audio-flamingo-next-hf
+tags:
+  - audio.cpp
+  - gguf
+  - audio-text-to-text
+  - automatic-speech-recognition
+language:
+  - en
+---
+
+# Audio Flamingo 3 and Next GGUF
+
+For use with [audio.cpp](https://github.com/0xShug0/audio.cpp).
+
+These self-contained GGUFs package the model tensors, tokenizer, configuration,
+and audio.cpp model spec for Audio Flamingo 3 and Audio Flamingo Next. Both
+models support transcription, captioning, and questions about speech, music,
+and environmental sounds.
+
+| File | Upstream checkpoint | Size | Maximum audio |
+| --- | --- | ---: | ---: |
+| `audio-flamingo-3-bf16.gguf` | [Audio Flamingo 3](https://huggingface.co/nvidia/audio-flamingo-3-hf) | 15.4 GiB | 10 minutes |
+| `audio-flamingo-3-q8_0.gguf` | Audio Flamingo 3 | 8.7 GiB | 10 minutes |
+| `audio-flamingo-3-q4_k.gguf` | Audio Flamingo 3 | 5.1 GiB | 10 minutes |
+| `audio-flamingo-next-bf16.gguf` | [Audio Flamingo Next](https://huggingface.co/nvidia/audio-flamingo-next-hf) | 15.4 GiB | 30 minutes |
+| `audio-flamingo-next-q8_0.gguf` | Audio Flamingo Next | 8.7 GiB | 30 minutes |
+| `audio-flamingo-next-q4_k.gguf` | Audio Flamingo Next | 5.1 GiB | 30 minutes |
+
+## Usage
+
+```bash
+audiocpp_cli --task asr --family audio_flamingo \
+  --model /path/to/audio-flamingo-3-bf16.gguf --backend cuda \
+  --audio input.wav \
+  --request-option "instruct=Transcribe the input speech." \
+  --text-out response.txt --log
+```
+
+Select either GGUF with `--model`. Use a question such as
+`Describe the instruments and tempo.` for general audio understanding. Requested
+timestamps or speaker descriptions are generated text, not structured alignment
+or diarization results.
+
+For server use, configure family `audio_flamingo`, task `asr`, and mode `offline`,
+then submit audio to `/v1/audio/transcriptions` with the instruction in `prompt`.
+
+## CUDA Performance and Quantization
+
+The following results use an RTX 5090, the CUDA backend, 8 CPU threads, and the
+same decoded 10-second, 16-kHz mono speech input. Wall time and RTF are from the
+second request in an already-loaded session. Peak VRAM covers the complete
+three-request session and was sampled every 100 ms.
+
+| Model | Weight type | Warm wall time | RTF | Peak VRAM | Transcript compared with BF16 |
+| --- | --- | ---: | ---: | ---: | --- |
+| Audio Flamingo 3 | BF16 | 283.3 ms | 0.0283 | 16,854 MiB | Reference |
+| Audio Flamingo 3 | Q8_0 | 184.9 ms | 0.0185 | 9,956 MiB | Exact |
+| Audio Flamingo 3 | Q4_K | 151.7 ms | 0.0152 | 6,280 MiB | Same words; wrapper and punctuation changed |
+| Audio Flamingo Next | BF16 | 246.3 ms | 0.0246 | 16,890 MiB | Reference |
+| Audio Flamingo Next | Q8_0 | 160.9 ms | 0.0161 | 9,992 MiB | Same words; one comma omitted |
+| Audio Flamingo Next | Q4_K | 129.9 ms | 0.0130 | 6,314 MiB | Exact |
+
+The BF16 Audio Flamingo 3 transcription and music-answer outputs were
+byte-identical to the official Transformers implementation. Audio Flamingo Next
+was also byte-identical on transcription. On a separate three-minute timestamped
+request, its words and line boundaries matched Python after timestamps were
+removed, while generated timestamps drifted.
+
+Quantized outputs remained coherent on the transcription and music-understanding
+requests. Q8_0 is the balanced default for lower VRAM and faster inference. Q4_K
+offers the lowest tested VRAM, but quantization can change wording, punctuation,
+or factual details in open-ended answers and should not be treated as exact
+parity with BF16.
+
+## Audio Preprocessing
+
+For comparisons with Python, supply the same decoded 16-kHz mono WAV to both
+implementations. Python's optional TorchCodec loader and librosa fallback use
+different downmixing and resampling paths. audio.cpp averages channels and uses
+SOXR when resampling is needed.
+
+Both checkpoints use the NVIDIA OneWay Noncommercial License. The upstream
+license is included in this directory. Review its terms before use or redistribution.
+
+See the [audio.cpp model documentation](https://github.com/0xShug0/audio.cpp/blob/main/docs/models/audio_flamingo.md)
+for request options, conversion, and additional usage details.

audio-flamingo-3-bf16.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:a970e1fa32a8904e37150bac57a1b198667bd69ecb5eeb1b28fdc93d0c4579f9
+size 16550564800

audio-flamingo-3-q4_k.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:87b85d1093be30d45571c7664f531aaa4872d9fd09ef0e1c989f61007e42fc11
+size 5459592448

audio-flamingo-3-q8_0.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:5325029ad5b955fb9db3d8f3fd06b4de567693c5990635c05486369d3414b464
+size 9317321984

audio-flamingo-next-bf16.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:8c983ebb563ac41dc6b84ba800479a47cd2588aa9392dccb69371102543aa770
+size 16546111968

audio-flamingo-next-q4_k.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:f285a520fff439c249a61a2a2535b2e962c0df02697a4cc54b19385ab3be7ce8
+size 5455139584

audio-flamingo-next-q8_0.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:124969625d2bd14af1a732de58ca53c2711fdb68e02f5504d408ccf637f1eb0a
+size 9312869120