Harsha001/humanizer-12b-ROCmFPX

humanizer 12B — ROCmFPX GGUF

jialinyyzz/humanizer (v2) rewrites AI-written drafts in English and Chinese so they read like a person wrote them, and keeps every number, date, name and quote. It is a fine-tune of Gemma 4 12B. This repo has it in the ROCmFPX formats, a llama.cpp fork with low-bit weight types for AMD GPUs (Vulkan and HIP/ROCm).

Start with humanizer-12b-Q8_0_ROCMFPX.gguf if it fits in your GPU memory: it is almost identical to the full model. Take the 6-bit file next, and a 4-bit file when memory is tight.

These files need HarshaVardhan3002/ROCmFPX, built from 23 October 2026 or later (they carry per-tensor scales, see below). Stock llama.cpp, LM Studio, Ollama and other ROCmFPX builds may not load them. These files have not been tested on AMD hardware yet.

Not an official release of humanizer. All credit for the model goes to its authors. Their own files in stock llama.cpp formats, including quantization-aware trained 2-, 3- and 4-bit files, are at jialinyyzz/humanizer-GGUF.

Files

File	Size	KL to BF16	Same top token
humanizer-12b-Q8_0_ROCMFPX.gguf	12.3 GB	0.00024	99.3%	8-bit, closest to the full model
humanizer-12b-Q6_K_ROCMFPX.gguf	9.8 GB	0.0023	97.8%	6-bit (ROCmFPX Q6_0_ROCMFPX weights, see below)
humanizer-12b-Q4_0_ROCMFP4_STRIX.gguf	6.7 GB	0.040	91.9%	recommended 4-bit (Strix Halo recipe)
humanizer-12b-Q4_0_ROCMFP4_FAST.gguf	6.3 GB	0.047	90.8%	smallest, speed-first 4-bit

Stock llama.cpp builds of this model, measured the same way:

Stock file	Size	KL to BF16	Same top token
Q8_0	12.7 GB	0.00064	98.8%
Q6_K	10.0 GB	0.0026	97.8%
Q4_K_M	7.4 GB	0.0186	94.1%

The 8-bit file here is closer to BF16 than a stock Q8_0, and the 6-bit file matches a stock Q6_K; both are a little smaller. At 4-bit, a stock Q4_K_M (or the authors' quantization-aware Q4_K_M) stays closer to the full model than the ROCmFP4 files here. The ROCmFP4 files are smaller and are meant for speed on AMD GPUs.

How this was measured: llama.cpp llama-perplexity --kl-divergence against the authors' BF16 file, 40 chunks of 512 tokens of English Wikipedia text (wikitext-2), run on a GPU after expanding each file to F16 (the expansion is exact). KL is how far the file's next-token probabilities drift from BF16, averaged over every token; lower is better. "Same top token" is how often the file's most likely next token is the same as BF16's. This is general text, not rewriting text; the authors measured their files on held-out rewriting data, and their stock Q4_K_M (KL 0.021) is close to what this test gives (0.0186).

Every file also passed a rewrite check: a test draft with five facts (a date, a count, a name, a year and an amount), greedy decoding on the CPU. All five facts came through in every file.

The 6-bit file is named humanizer-12b-Q6_K_ROCMFPX.gguf so that Hugging Face lists it as a 6-bit quant. Its weights are the ROCmFPX Q6_0_ROCMFPX type, not a stock llama.cpp Q6_K, and it needs the same fork as the other files.

Why these files carry tensor scales

ROCmFPX stores one scale for each block of weights as an 8-bit float (UE4M3) whose smallest step is 2^-10. This model's weights are small (the largest value in a typical block of 32 is about 0.02 to 0.04), so an 8-bit ROCmFPX file cannot pick small enough scales: almost every block gets stuck at the smallest scale, and the file is about as precise as a 6-bit one. These files multiply each weight matrix by a power of two (2^7 to 2^10) before quantizing and store the inverse as <name>.scale, which the fork multiplies back after the matrix product. The shared embedding and output matrix cannot take such a scale, so it uses a stock llama.cpp type: Q8_0 in the 8-bit file, Q6_K in the 6-bit and STRIX files (the FAST file keeps it in ROCmFP4 for speed).

KL to BF16	Plain ROCmFPX	With tensor scales
8-bit	0.0112	0.00024
6-bit	0.0080	0.0023
4-bit STRIX	0.053	0.040
4-bit FAST	0.064	0.047

How these were made

1. Source: the authors' BF16 GGUF (humanizer-12b-bf16.gguf from jialinyyzz/humanizer-GGUF), sha256 checked against their card.
2. Importance matrix from the BF16 file over a general calibration text (calibration_datav3), 512-token chunks. The authors used their own rewriting data for theirs.
3. llama-quantize --imatrix from the BF16 file with LLAMA_ROCMFPX_TENSOR_SCALE=1, one run per type, from HarshaVardhan3002/ROCmFPX; --token-embedding-type q8_0 for the 8-bit file and q6_K for the 6-bit file.

The only change from the original is the weight quantization. The tokenizer, the chat template and the stored sampling defaults come unchanged from the authors' GGUF.

Run

git clone https://github.com/HarshaVardhan3002/ROCmFPX && cd ROCmFPX
cmake -B build -DGGML_VULKAN=ON          # or -DGGML_HIP=ON -DAMDGPU_TARGETS=gfx1151 (gfx1100 for RX 7900)
cmake --build build --target llama-server -j
build/bin/llama-server -m humanizer-12b-Q8_0_ROCMFPX.gguf -ngl 999 -c 8192 -np 1 --jinja

The model does text completion, not chat. The prompt is the instruction from prompt_format.json, a blank line, your draft, then \n\n### Rewritten:\n\n. Sample with temperature 1.0, top-p 0.95, top-k 0, min-p 0 and repeat penalty 1.0, and stop at the end-of-sequence token only.

import json, urllib.request

pf = json.load(open("prompt_format.json", encoding="utf-8"))

def humanize(draft, url="http://127.0.0.1:8080"):
    body = {"prompt": pf["instr"] + "\n\n" + draft.strip() + pf["sep"],
            "temperature": 1.0, "top_p": 0.95, "top_k": 0, "min_p": 0,
            "repeat_penalty": 1.0, "n_predict": 2048}
    req = urllib.request.Request(url + "/completion", json.dumps(body).encode("utf-8"),
                                 {"Content-Type": "application/json"})
    with urllib.request.urlopen(req, timeout=900) as r:
        return json.load(r)["content"].strip()

print(humanize(open("draft.txt", encoding="utf-8").read()))

The chat template in each file builds the same prompt from the last user message, so /v1/chat/completions with one draft per message also works. Keep instruction, draft and rewrite within 8,192 tokens and split long documents at paragraph breaks. The authors note that a short, informal English draft occasionally comes back in Chinese; sample again when that happens. Their usage guide has more. Their desktop app runs stock llama.cpp with their own files, not these.

License

Apache-2.0, same as the original (see LICENSE). The authors' NOTICE file is included and must stay with any copy. humanizer is fine-tuned from google/gemma-4-12B, which Google releases under Apache 2.0. This repo is not affiliated with or endorsed by Google or the humanizer authors.