README.md

---
license: apache-2.0
base_model: jialinyyzz/humanizer
base_model_relation: quantized
pipeline_tag: text-generation
library_name: gguf
language: [en, zh]
tags: [gguf, rocmfpx, rocmfp4, amd, rocm, strix-halo, quantized, humanizer, text-rewriting, gemma4]
---

# humanizer 12B — ROCmFPX GGUF

[jialinyyzz/humanizer](https://huggingface.co/jialinyyzz/humanizer) (v2) rewrites AI-written drafts in English and
Chinese so they read like a person wrote them, and keeps every number, date, name and quote. It is a fine-tune of
Gemma 4 12B. This repo has it in the [ROCmFPX](https://github.com/charlie12345/ROCmFPX) formats, a llama.cpp fork
with low-bit weight types for AMD GPUs (Vulkan and HIP/ROCm).

**Start with `humanizer-12b-Q8_0_ROCMFPX.gguf`** if it fits in your GPU memory: it is almost identical to the full
model. Take the 6-bit file next, and a 4-bit file when memory is tight.

> These files need [HarshaVardhan3002/ROCmFPX](https://github.com/HarshaVardhan3002/ROCmFPX), built from
> 23 October 2026 or later (they carry per-tensor scales, see below). Stock llama.cpp, LM Studio, Ollama and other
> ROCmFPX builds may not load them. **These files have not been tested on AMD hardware yet.**
>
> Not an official release of humanizer. All credit for the model goes to its authors. Their own files in stock
> llama.cpp formats, including quantization-aware trained 2-, 3- and 4-bit files, are at
> [jialinyyzz/humanizer-GGUF](https://huggingface.co/jialinyyzz/humanizer-GGUF).

## Files

| File | Size | KL to BF16 | Same top token | |
|---|---:|---:|---:|---|
| `humanizer-12b-Q8_0_ROCMFPX.gguf` | 12.3 GB | 0.00024 | 99.3% | 8-bit, closest to the full model |
| `humanizer-12b-Q6_K_ROCMFPX.gguf` | 9.8 GB | 0.0023 | 97.8% | 6-bit (ROCmFPX `Q6_0_ROCMFPX` weights, see below) |
| `humanizer-12b-Q4_0_ROCMFP4_STRIX.gguf` | 6.7 GB | 0.040 | 91.9% | recommended 4-bit (Strix Halo recipe) |
| `humanizer-12b-Q4_0_ROCMFP4_FAST.gguf` | 6.3 GB | 0.047 | 90.8% | smallest, speed-first 4-bit |

Stock llama.cpp builds of this model, measured the same way:

| Stock file | Size | KL to BF16 | Same top token |
|---|---:|---:|---:|
| Q8_0 | 12.7 GB | 0.00064 | 98.8% |
| Q6_K | 10.0 GB | 0.0026 | 97.8% |
| Q4_K_M | 7.4 GB | 0.0186 | 94.1% |

The 8-bit file here is closer to BF16 than a stock Q8_0, and the 6-bit file matches a stock Q6_K; both are a
little smaller. At 4-bit, a stock Q4_K_M (or the authors' quantization-aware Q4_K_M) stays closer to the full model than the ROCmFP4
files here. The ROCmFP4 files are smaller and are meant for speed on AMD GPUs.

**How this was measured:** llama.cpp `llama-perplexity --kl-divergence` against the authors' BF16 file, 40 chunks of
512 tokens of English Wikipedia text (wikitext-2), run on a GPU after expanding each file to F16 (the expansion is
exact). KL is how far the file's next-token probabilities drift from BF16, averaged over every token; lower is better.
"Same top token" is how often the file's most likely next token is the same as BF16's. This is general text, not
rewriting text; the authors measured their files on held-out rewriting data, and their stock Q4_K_M (KL 0.021) is
close to what this test gives (0.0186).

Every file also passed a rewrite check: a test draft with five facts (a date, a count, a name, a year and an amount),
greedy decoding on the CPU. All five facts came through in every file.

The 6-bit file is named `humanizer-12b-Q6_K_ROCMFPX.gguf` so that Hugging Face lists it as a 6-bit quant. Its weights
are the ROCmFPX `Q6_0_ROCMFPX` type, not a stock llama.cpp `Q6_K`, and it needs the same fork as the other files.

## Why these files carry tensor scales

ROCmFPX stores one scale for each block of weights as an 8-bit float (UE4M3) whose smallest step is 2^-10. This
model's weights are small (the largest value in a typical block of 32 is about 0.02 to 0.04), so an 8-bit ROCmFPX
file cannot pick small enough scales: almost every block gets stuck at the smallest scale, and the file is about as
precise as a 6-bit one. These files multiply each weight matrix by a power of two (2^7 to 2^10) before quantizing and store
the inverse as `<name>.scale`, which the fork multiplies back after the matrix product. The shared embedding and
output matrix cannot take such a scale, so it uses a stock llama.cpp type: Q8_0 in the 8-bit file, Q6_K in the 6-bit
and STRIX files (the FAST file keeps it in ROCmFP4 for speed).

| KL to BF16 | Plain ROCmFPX | With tensor scales |
|---|---:|---:|
| 8-bit | 0.0112 | 0.00024 |
| 6-bit | 0.0080 | 0.0023 |
| 4-bit STRIX | 0.053 | 0.040 |
| 4-bit FAST | 0.064 | 0.047 |

## How these were made

1. Source: the authors' BF16 GGUF (`humanizer-12b-bf16.gguf` from
   [jialinyyzz/humanizer-GGUF](https://huggingface.co/jialinyyzz/humanizer-GGUF)), sha256 checked against their card.
2. Importance matrix from the BF16 file over a general calibration text (`calibration_datav3`), 512-token chunks.
   The authors used their own rewriting data for theirs.
3. `llama-quantize --imatrix` from the BF16 file with `LLAMA_ROCMFPX_TENSOR_SCALE=1`, one run per type, from
   [HarshaVardhan3002/ROCmFPX](https://github.com/HarshaVardhan3002/ROCmFPX); `--token-embedding-type q8_0` for the
   8-bit file and `q6_K` for the 6-bit file.

The only change from the original is the weight quantization. The tokenizer, the chat template and the stored sampling
defaults come unchanged from the authors' GGUF.

## Run

```bash
git clone https://github.com/HarshaVardhan3002/ROCmFPX && cd ROCmFPX
cmake -B build -DGGML_VULKAN=ON          # or -DGGML_HIP=ON -DAMDGPU_TARGETS=gfx1151 (gfx1100 for RX 7900)
cmake --build build --target llama-server -j
build/bin/llama-server -m humanizer-12b-Q8_0_ROCMFPX.gguf -ngl 999 -c 8192 -np 1 --jinja
```

The model does text completion, not chat. The prompt is the instruction from `prompt_format.json`, a blank line, your
draft, then `\n\n### Rewritten:\n\n`. Sample with temperature 1.0, top-p 0.95, top-k 0, min-p 0 and repeat penalty 1.0,
and stop at the end-of-sequence token only.

```python
import json, urllib.request

pf = json.load(open("prompt_format.json", encoding="utf-8"))

def humanize(draft, url="http://127.0.0.1:8080"):
    body = {"prompt": pf["instr"] + "\n\n" + draft.strip() + pf["sep"],
            "temperature": 1.0, "top_p": 0.95, "top_k": 0, "min_p": 0,
            "repeat_penalty": 1.0, "n_predict": 2048}
    req = urllib.request.Request(url + "/completion", json.dumps(body).encode("utf-8"),
                                 {"Content-Type": "application/json"})
    with urllib.request.urlopen(req, timeout=900) as r:
        return json.load(r)["content"].strip()

print(humanize(open("draft.txt", encoding="utf-8").read()))
```

The chat template in each file builds the same prompt from the last user message, so `/v1/chat/completions` with one
draft per message also works. Keep instruction, draft and rewrite within 8,192 tokens and split long documents at
paragraph breaks. The authors note that a short, informal English draft occasionally comes back in Chinese; sample
again when that happens. Their [usage guide](https://github.com/sgaofen/humanizer-local-model/blob/main/docs/USAGE.md)
has more. Their desktop app runs stock llama.cpp with their own files, not these.

## License

Apache-2.0, same as the original (see `LICENSE`). The authors' `NOTICE` file is included and must stay with any copy.
humanizer is fine-tuned from [google/gemma-4-12B](https://huggingface.co/google/gemma-4-12B), which Google releases
under Apache 2.0. This repo is not affiliated with or endorsed by Google or the humanizer authors.