|
Download README.md from Harsha001/Haru-4B-ROCmFPX: direct link, hf CLI and curl.
- Browser
- Download file 4.57 kB
-
https://huggingface.co/Harsha001/Haru-4B-ROCmFPX/resolve/main/README.md
- Command line
-
hf download hf://Harsha001/Haru-4B-ROCmFPX/README.md
-
curl -L -o README.md https://huggingface.co/Harsha001/Haru-4B-ROCmFPX/resolve/main/README.md
4.57 kB
| license: apache-2.0 | |
| base_model: higurashi-ai/Haru-4B | |
| pipeline_tag: audio-to-audio | |
| language: | |
| - en | |
| - es | |
| - pt | |
| - de | |
| - fr | |
| - zh | |
| - ja | |
| - ru | |
| - ko | |
| tags: | |
| - speech | |
| - full-duplex | |
| - speech-to-speech | |
| - gguf | |
| - rocmfpx | |
| - haru | |
| # Haru-4B GGUF (ROCmFPX) | |
| GGUF files of [higurashi-ai/Haru-4B](https://huggingface.co/higurashi-ai/Haru-4B), a full-duplex spoken dialogue | |
| model by higurashi-ai, with a native C++ runtime (`llama-haru`) in the | |
| [ROCmFPX fork of llama.cpp](https://github.com/HarshaVardhan3002/ROCmFPX). | |
| Haru listens and speaks at the same time. Every 133 ms it takes one frame of your audio and produces one frame of | |
| its own audio and text. The language model is a Qwen3.5-4B text backbone. A small MeanFlow head turns its hidden | |
| state into a 64-dim audio latent, and a causal VibeVoice codec turns that into 24 kHz audio. All credit for the | |
| model goes to higurashi-ai (and to the Qwen team for the backbone). See the | |
| [original card](https://huggingface.co/higurashi-ai/Haru-4B) and the [Haru code](https://github.com/higurashi-ai/Haru). | |
| ## Files | |
| | File | Size | What it is | | |
| | --- | --- | --- | | |
| | `Haru-4B-Q8_0_ROCMFPX.gguf` | 4.37 GB | Language model, ROCmFPX 8-bit with tensor scales (needs the fork). Recommended | | |
| | `Haru-4B-Q6_0_ROCMFPX.gguf` | 3.48 GB | Language model, ROCmFPX 6-bit with tensor scales (needs the fork) | | |
| | `Haru-4B-audio-F16.gguf` | 2.30 GB | Audio parts: the two audio projectors, the MeanFlow head and the codec (needed with either model) | | |
| You need one language model file and the audio file. | |
| ## Run it | |
| Build the fork, then run an offline conversation from two WAV files (24 kHz mono): | |
| ```bash | |
| git clone https://github.com/HarshaVardhan3002/ROCmFPX && cd ROCmFPX | |
| cmake -B build -DCMAKE_BUILD_TYPE=Release && cmake --build build --target llama-haru -j | |
| ./build/bin/llama-haru -m Haru-4B-Q8_0_ROCMFPX.gguf --audio Haru-4B-audio-F16.gguf \ | |
| --voice my_voice.wav --user question.wav --card-file card.txt -o answer.wav | |
| ``` | |
| - `--voice`: the voice Haru speaks with. One speaker, 3 seconds to 3 minutes, clean speech. | |
| - `--user`: what you say. Haru listens to it frame by frame and answers while it plays. | |
| - `--card` / `--card-file`: the character card (system prompt). The | |
| [Haru README](https://github.com/higurashi-ai/Haru) shows the card layout the model was trained with. | |
| - Output: `answer.wav` (Haru's channel) and `answer-stereo.wav` (you on the left, Haru on the right). | |
| Haru's text appears on the terminal as it speaks. | |
| - Sampling defaults follow the official code: temperature 0.7, top-k 25, guidance 1.5, DRY on. | |
| `--speak-first`, `--seed`, `--temp`, `--guidance` and `--tail` (seconds of listening after your clip) are available. | |
| The runtime is an offline (file in, file out) tool for now. A live microphone mode is not part of this release. | |
| ## Accuracy | |
| The runtime was checked part by part against the official PyTorch code in float32, on a real voice clip and | |
| 24 live dialogue frames: | |
| | Part | Cosine similarity to PyTorch | | |
| | --- | --- | | |
| | Codec encode (whole clip and streaming), streaming decode | 1.000000 | | |
| | Audio projectors (input embeddings) | 1.000000 | | |
| | Language model hidden states | 0.999998 | | |
| | Text logits | 0.999999 (same top token on 24 / 24 frames) | | |
| | MeanFlow head latents | 1.000000 | | |
| Quantized language models, KL divergence to BF16 on wiki text (40 x 512 tokens), and hidden-state similarity on the | |
| real dialogue frames (file run as is on the CPU): | |
| | Language model | Size | Wiki KLD | Dialogue hidden cosine | | |
| | --- | --- | --- | --- | | |
| | Q8_0_ROCMFPX (this repo) | 4.37 GB | 0.0019 | 0.9966 | | |
| | stock Q8_0 (not uploaded) | 4.48 GB | 0.0031 | 0.9954 | | |
| | Q6_0_ROCMFPX (this repo) | 3.48 GB | 0.0113 | 0.9915 | | |
| | stock Q6_K (not uploaded) | 3.46 GB | 0.0153 | 0.9913 | | |
| On the CPU, use the fork at commit `4310e2480` or newer for the ROCmFPX 8-bit file. Older builds rounded the | |
| activations to 8 bits before each matrix multiply, which brought this file down to 0.966 on the dialogue frames. | |
| The audio file in F16 matches the float32 one (head cosine 0.9999997). | |
| ## Notes and limits | |
| - Verified on the CPU (ARM). On a CPU it is slower than real time; real-time use needs a GPU build. GPU backends | |
| have not been tested with this runtime yet. | |
| - Haru-4B is a pretrained base model released for research. It holds a natural conversation, but it is not a | |
| reliable source of facts and does not follow instructions the way an assistant does. | |
| - It can imitate the voice of a reference clip. Only use voices you have the right to use. | |
| ## License | |
| Apache License 2.0, same as the original weights. | |