Download README.md from Harsha001/Haru-4B-ROCmFPX: direct link, hf CLI and curl.
- Browser
- Download file 4.57 kB
-
https://huggingface.co/Harsha001/Haru-4B-ROCmFPX/resolve/main/README.md
- Command line
-
hf download hf://Harsha001/Haru-4B-ROCmFPX/README.md
-
curl -L -o README.md https://huggingface.co/Harsha001/Haru-4B-ROCmFPX/resolve/main/README.md
license: apache-2.0
base_model: higurashi-ai/Haru-4B
pipeline_tag: audio-to-audio
language:
- en
- es
- pt
- de
- fr
- zh
- ja
- ru
- ko
tags:
- speech
- full-duplex
- speech-to-speech
- gguf
- rocmfpx
- haru
Haru-4B GGUF (ROCmFPX)
GGUF files of higurashi-ai/Haru-4B, a full-duplex spoken dialogue
model by higurashi-ai, with a native C++ runtime (llama-haru) in the
ROCmFPX fork of llama.cpp.
Haru listens and speaks at the same time. Every 133 ms it takes one frame of your audio and produces one frame of its own audio and text. The language model is a Qwen3.5-4B text backbone. A small MeanFlow head turns its hidden state into a 64-dim audio latent, and a causal VibeVoice codec turns that into 24 kHz audio. All credit for the model goes to higurashi-ai (and to the Qwen team for the backbone). See the original card and the Haru code.
Files
| File | Size | What it is |
|---|---|---|
Haru-4B-Q8_0_ROCMFPX.gguf |
4.37 GB | Language model, ROCmFPX 8-bit with tensor scales (needs the fork). Recommended |
Haru-4B-Q6_0_ROCMFPX.gguf |
3.48 GB | Language model, ROCmFPX 6-bit with tensor scales (needs the fork) |
Haru-4B-audio-F16.gguf |
2.30 GB | Audio parts: the two audio projectors, the MeanFlow head and the codec (needed with either model) |
You need one language model file and the audio file.
Run it
Build the fork, then run an offline conversation from two WAV files (24 kHz mono):
git clone https://github.com/HarshaVardhan3002/ROCmFPX && cd ROCmFPX
cmake -B build -DCMAKE_BUILD_TYPE=Release && cmake --build build --target llama-haru -j
./build/bin/llama-haru -m Haru-4B-Q8_0_ROCMFPX.gguf --audio Haru-4B-audio-F16.gguf \
--voice my_voice.wav --user question.wav --card-file card.txt -o answer.wav
--voice: the voice Haru speaks with. One speaker, 3 seconds to 3 minutes, clean speech.--user: what you say. Haru listens to it frame by frame and answers while it plays.--card/--card-file: the character card (system prompt). The Haru README shows the card layout the model was trained with.- Output:
answer.wav(Haru's channel) andanswer-stereo.wav(you on the left, Haru on the right). Haru's text appears on the terminal as it speaks. - Sampling defaults follow the official code: temperature 0.7, top-k 25, guidance 1.5, DRY on.
--speak-first,--seed,--temp,--guidanceand--tail(seconds of listening after your clip) are available.
The runtime is an offline (file in, file out) tool for now. A live microphone mode is not part of this release.
Accuracy
The runtime was checked part by part against the official PyTorch code in float32, on a real voice clip and 24 live dialogue frames:
| Part | Cosine similarity to PyTorch |
|---|---|
| Codec encode (whole clip and streaming), streaming decode | 1.000000 |
| Audio projectors (input embeddings) | 1.000000 |
| Language model hidden states | 0.999998 |
| Text logits | 0.999999 (same top token on 24 / 24 frames) |
| MeanFlow head latents | 1.000000 |
Quantized language models, KL divergence to BF16 on wiki text (40 x 512 tokens), and hidden-state similarity on the real dialogue frames (file run as is on the CPU):
| Language model | Size | Wiki KLD | Dialogue hidden cosine |
|---|---|---|---|
| Q8_0_ROCMFPX (this repo) | 4.37 GB | 0.0019 | 0.9966 |
| stock Q8_0 (not uploaded) | 4.48 GB | 0.0031 | 0.9954 |
| Q6_0_ROCMFPX (this repo) | 3.48 GB | 0.0113 | 0.9915 |
| stock Q6_K (not uploaded) | 3.46 GB | 0.0153 | 0.9913 |
On the CPU, use the fork at commit 4310e2480 or newer for the ROCmFPX 8-bit file. Older builds rounded the
activations to 8 bits before each matrix multiply, which brought this file down to 0.966 on the dialogue frames.
The audio file in F16 matches the float32 one (head cosine 0.9999997).
Notes and limits
- Verified on the CPU (ARM). On a CPU it is slower than real time; real-time use needs a GPU build. GPU backends have not been tested with this runtime yet.
- Haru-4B is a pretrained base model released for research. It holds a natural conversation, but it is not a reliable source of facts and does not follow instructions the way an assistant does.
- It can imitate the voice of a reference clip. Only use voices you have the right to use.
License
Apache License 2.0, same as the original weights.