LessThanThreeAI/Gemma-4-31B-Humanlike-Chat-GGUF

Gemma-4-31B-Humanlike-Chat

A 31B Gemma that texts like a person and still does the work. No system prompt needed. Text it and it texts back like someone you know. Ask for a formal email, a tool call or a proper explanation and it does that, then goes back to texting.

Quick start · Download · vLLM / safetensors · Free endpoint · Results · How it was made · Custom finetunes

_{Same texts, no system prompt. Left: Gemma-4-31B-Humanlike-Chat. Right: Google Gemma 4 31B, one sample, long replies cut.}

Why download it

• It texts like a person out of the box. Shown a real chat and two next messages, a blind judge took this model's reply for the real person's in 26% of the moments. Google's Gemma 4 31B, the model it is trained from: 0%.
• It asks before it guesses. When2Call 66, Google 46. When no tool fits, it doesn't call one: BFCL irrelevance 90, Google 81.
• 2.0's voice, a much stronger brain. Against Qwen3.8-27B-Humanlike-Chat 2.0 on the same eval inputs: LiveCodeBench 66 vs 51, IFEval 90.0 vs 83.4, When2Call 66 vs 58.
• Google's knowledge kept. MMLU-Pro 82.5, same as Google. Right tool with the right arguments: BFCL simple 96, Google 95.
• It is the official Gemma underneath. Trained from google/gemma-4-31B-it, Apache-2.0. Not abliterated.
• Or skip the download. The free endpoint runs this model.

"Base" on this card always means Google's google/gemma-4-31B-it, the official release. Unlike Qwen3.8-27B-Humanlike-Chat 2.0, which was built on an abliterated base, this one is not abliterated.

Commissions open. I build custom finetunes like this one: characters, product voices, distillation into smaller models, and domain or use-case specific models. DM codebottle on Discord. Details.

If it feels more natural than your current Gemma, click Like. It helps other people find it.

What it looks like

A character card, turns 12 to 16 of a long chat. Humanlike stays a jet-lagged flight attendant. Google writes stage directions and a jet lag guide.	Ask for a formal email and you get one. Google wrote good ones too. The difference is the next message.
One chat with tools. A weather call, a booking with no time given, and a rhyme no tool fits. Greedy, canned tool result.	Humanlike vs Google Gemma 4 31B. Same run, same prompts, greedy, thinking off except GSM8K thinking on.

Quick start

llama.cpp

llama-server --hf-repo LessThanThreeAI/Gemma-4-31B-Humanlike-Chat-GGUF \
  --hf-file Gemma-4-31B-Humanlike-Chat-IQ4_XS.gguf \
  --jinja \
  --ctx-size 32768 --parallel 1 --n-gpu-layers 99 \
  --temp 1.0 --top-p 0.95 --top-k 64 \
  --alias humanlike-gemma --host 127.0.0.1 --port 8080

Open http://127.0.0.1:8080 and start texting. --jinja uses the model's own chat template, which is what makes tool calls and the thinking switch work. For another quant, swap in a file name from Download.

The server speaks the OpenAI API:

from openai import OpenAI

client = OpenAI(base_url="http://127.0.0.1:8080/v1", api_key="none")
MODEL = "humanlike-gemma"  # the --alias above

# 1. Plain chat, no system prompt
r = client.chat.completions.create(
    model=MODEL,
    messages=[{"role": "user", "content": "hey, what's up?"}],
)
print(r.choices[0].message.content)

# 2. Tool calling. The departure city is missing, so it should ask instead of guessing.
tools = [{
    "type": "function",
    "function": {
        "name": "search_flights",
        "description": "Search roundtrip flights.",
        "parameters": {
            "type": "object",
            "properties": {
                "origin": {"type": "string", "description": "Departure city or airport"},
                "destination": {"type": "string", "description": "Arrival city or airport"},
                "depart_date": {"type": "string", "description": "YYYY-MM-DD"},
                "return_date": {"type": "string", "description": "YYYY-MM-DD"},
            },
            "required": ["origin", "destination", "depart_date", "return_date"],
        },
    },
}]
r = client.chat.completions.create(
    model=MODEL,
    messages=[{"role": "user", "content": "find me a roundtrip flight to Atlanta, march 1 to march 6"}],
    tools=tools,
)
msg = r.choices[0].message
print(msg.tool_calls or msg.content)

# 3. Thinking on
r = client.chat.completions.create(
    model=MODEL,
    messages=[{"role": "user", "content": "is it worth learning to cook if i live alone"}],
    extra_body={"chat_template_kwargs": {"enable_thinking": True}},
)
print(getattr(r.choices[0].message, "reasoning_content", None))  # the thinking
print(r.choices[0].message.content)                              # the reply

When the model calls a tool, send the result back as a {"role": "tool", ...} message, like with any OpenAI-compatible model.

Thinking

Thinking works, and it is off by default: Gemma 4's chat template only thinks when you send "enable_thinking": true in chat_template_kwargs, as in example 3. It was trained with effort levels, set by one line in the system prompt: Reasoning effort: low or Reasoning effort: high, and no line for medium. Add the line to your system prompt yourself (the free endpoint does it for you). Thinking gets longer from low to high (median thinking on math: 263, 380 and 406 characters), and the voice held at every level. The model thinks briefly in first person, then texts back in the same voice. The thinking comes back in its own field, separate from the reply. Example:

user: i miss you
humanlike (thinking): I recognize that they're reaching out, but I don't want to pretend I'm actually with them right now. I'm not going to get too sentimental either.
humanlike: yeah, me too
humanlike: just woke up and everything is quiet

LM Studio and Ollama

Both load GGUF files. Use the sampling below (temperature 1.0, top_p 0.95, top_k 64).

SillyTavern

Start llama-server as above, then connect with:

API: Chat Completion
Chat Completion Source: Custom (OpenAI-compatible)
Custom Endpoint (Base URL): http://127.0.0.1:8080/v1
Context: 32768
Response length: 512
Temperature: 1.0
Top P: 0.95

In Chat Completion mode llama-server applies the model's own chat template, which is how my tests sent messages. If you ask for long answers, raise the response length.

vLLM and SGLang (safetensors)

Safetensors builds for vLLM, SGLang and transformers, made from the same merged BF16 weights as the GGUFs. Pick the one that fits your card:

Repo	Size	KL vs reference	tok/s (vLLM)
GPTQ-Int4	19.07 GB	0.0107	67
FP8	33.27 GB	0.0034	40
BF16	62.55 GB	reference	23

vllm serve LessThanThreeAI/Gemma-4-31B-Humanlike-Chat-FP8 --served-model-name humanlike-gemma \
  --max-model-len 16384 --language-model-only \
  --enable-auto-tool-choice --tool-call-parser gemma4 --reasoning-parser gemma4

Swap in the repo that fits your card. Each repo's card has its exact command. Thinking is off by default; send chat_template_kwargs: {"enable_thinking": true} per request to turn it on. For SGLang, point --model-path at the repo. KL uses the same 5 test chats as the table below; tok/s is single-request decode on one NVIDIA RTX PRO 6000 Blackwell Server Edition (llama.cpp for GGUF, vLLM for safetensors; single request, 256-token decode).

Download

Which file: with 24 GB of VRAM, take IQ4_XS or Q4_K_M: both run 32k context in under 22 GiB (IQ4_XS 20.1 GiB, Q4_K_M 21.9 GiB). With 32 GB, take Q5_K_M or Q6_K. With 48 GB or more, take Q8_0. The free endpoint runs IQ4_XS.

File	Size	KL vs reference	tok/s
IQ4_XS	16.74 GB	0.0127	69
Q4_K_M	18.69 GB	0.0132	63
Q5_K_M	21.85 GB	0.0036	56
Q6_K	25.20 GB	0.0020	49
Q8_0	32.64 GB	0.0006	40
BF16, two shards (1, 2)	61.41 GB	0.0003	23
Safetensors BF16 (vLLM, SGLang, transformers)	62.55 GB	reference
Safetensors FP8 (vLLM, SGLang)	33.27 GB	0.0034
Safetensors GPTQ-Int4 (vLLM, SGLang)	19.07 GB	0.0107
LoRA, rank 64, BF16	0.98 GB	0.0004
mmproj, BF16	1.20 GB

• KL vs reference: how far each file's next-token predictions drift from the reference: the merged BF16 weights in vLLM, which match Google's base plus the Humanlike adapter at runtime (KL 0.0005). Lower is closer. It is the mean KL over the reference's top 20 tokens at fixed points in 5 fresh test chats, so it underestimates the full KL.
• tok/s: decode speed on one NVIDIA RTX PRO 6000 Blackwell Server Edition (llama.cpp for GGUF, vLLM for safetensors; single request, 256-token decode).
• LoRA: for running Humanlike on top of Google's own GGUF, see Standalone LoRA. Never add it to the merged files above, because they already contain it.
• mmproj: Google's vision encoder for llama.cpp's --mmproj. The finetune didn't touch it: the merge left all 778 non-adapted tensors, the vision tower among them, byte-identical to Google's.
• Importance matrix: imatrix/imatrix-Gemma-4-31B-Humanlike-Chat.gguf, from WikiText-2 train (first 128 x 512 tokens) plus a chat and tool-call text mix in the Gemma chat template. Q8_0 uses none.
• Exact hashes are in SHA256SUMS.
• Load the merged files as a normal model. There is no LoRA strength to set.

Free endpoint

This model runs on a free, rate-capped, OpenAI-compatible endpoint (the IQ4_XS quant). Tool calls work there too.

Setting	Value
Base URL	https://api.lessthanthreeai.com/v1
Model	gemma-4-31b-humanlike-chat
API key	Not required

from openai import OpenAI

client = OpenAI(api_key="not-required", base_url="https://api.lessthanthreeai.com/v1", timeout=300.0)

reply = client.chat.completions.create(
    model="gemma-4-31b-humanlike-chat",
    messages=[{"role": "user", "content": "hey, what are you up to?"}],
)
print(reply.choices[0].message.content)

Thinking works here too: send extra_body={"chat_template_kwargs": {"enable_thinking": True}}. To set the level, add "reasoning_effort": "low", "medium" or "high" inside the same chat_template_kwargs; the endpoint turns it into the trained system line. The reasoning comes back in a separate field from the reply. Without it, the endpoint answers straight away.

The endpoint scales to zero when idle. The first request after a quiet period waits while a GPU starts, so keep the client timeout at 300 seconds or more.

How to use it

• No system prompt means a person texting. Short lowercase lines that react to what you said. It plays an ordinary person with an ordinary day, so it makes up small life details ("had some coffee and scrolled stuff"). That is on purpose. If you need it to talk about itself as a model, say so in a system prompt.
• A character card decides who it is, not how it writes. Give it a name, age, job and mood and it texts as that person.
• Explicit instructions beat texting. A one-off request ("write a formal email", "numbered steps", "explain properly", "reply in Russian") applies to that reply, then it goes back to texting.
• Work requests get real answers: math, explanations, emails, code, tool calls. When it can't fill a tool argument honestly, it asks first.
• Language: it replies in the language you write in. I tested English and Russian.
• Sampling: temperature 1.0, top_p 0.95, top_k 64. These are Google's defaults for Gemma 4, and every voice eval used them. Benchmarks used greedy decoding.

Results

Capability

Test (items)	Base (Google)	Humanlike
IFEval strict prompt (541)	91.9	90.0
IFBench strict prompt (300)	51.0	45.7
When2Call (100)	46	66
BFCL simple (100)	95	96
BFCL multiple (100)	95	93
BFCL irrelevance (100)	81	90
MMLU-Pro (200)	82.5	82.5
GSM8K (64)	90.6	89.1
GSM8K, thinking on (64)	95.3	92.2
LiveCodeBench pass@1 (100)	71	66

Same run harness, same prompts, greedy, thinking off except the GSM8K thinking row. Base is Google's google/gemma-4-31B-it. BFCL and When2Call use my own scorers: consistent across models, not comparable with the public leaderboards.

• When2Call checks the right move: call a tool, ask for a missing detail, or say no tool fits. Humanlike is 20 points above Google. It calls less and asks more.
• BFCL irrelevance (not calling a tool when none fits) is 9 points above Google.
• MMLU-Pro is 82.5, the same as Google: the knowledge held.
• vs Qwen3.8-27B-Humanlike-Chat 2.0 (same eval inputs, thinking off): LiveCodeBench 66 vs 51, IFEval 90.0 vs 83.4, When2Call 66 vs 58, BFCL irrelevance 90 vs 78, IFBench 45.7 vs 43.7.

Does it sound human? (ishuman v2)

A blind judge sees a real moment from my own one-to-one chats (moments that never went into training) and two candidate next messages: the one the person really sent and the model's. It picks the real one.

Model	Moments where the judge took the model's reply for the real one
Base (Google Gemma 4 31B)	0%
Gemma-4-31B-Humanlike-Chat	26%

How to read it: each moment is a real message someone sent me, next to the model's attempt at the same message. The judge knows one of the two is the real one. Google's Gemma never fooled it; Humanlike did in 26% of the 150 moments.

How it was made

Two stages, the same recipe as 2.0, on one H200.

1. SFT, one epoch on 13,740 rows. LoRA rank 64 on Google's base. The rows:

• 2.0's replies on its round-3b training contexts (3,926), plus 2.0's thinking-on replies (1,073)
• Google's own answers on the work rows: instructions and code (1,352)
• real chats from the free endpoint, rebuilt with their full context: 3,182 turns rewritten by 2.0 where the logged reply was bad, 3,910 logged 2.0 replies kept
• human preference picks: the chosen reply from blind side-by-side checks (207)
• my own corrections of replies (90)

2. On-policy distillation, 12 steps. The model writes its own replies and a teacher grades every token. There are two teachers. For chat and characters it's the SFT model plus a hidden "text like a person" instruction. For instructions, tools and code it's Google's base. The student never sees the hidden instruction, so it texts that way with no system prompt. Part of each step is self-play: live chats with simulated friends, so it practises long conversations in its own words. This fixed the SFT model's habit of fading to one-word replies late in long chats. The run went to 24 steps; step 12 was kept because it held long-chat replies better at the same Track A and capability.

google/gemma-4-31B-it (revision 842da379)
  -> SFT LoRA (rank 64, 1 epoch, 13,740 rows)
  -> on-policy distillation (same LoRA, step 12)
  -> merged into BF16
  -> BF16 / Q8_0 / Q6_K / Q5_K_M / Q4_K_M / IQ4_XS GGUF, FP8 and GPTQ-Int4 safetensors

Technical details

Item	Specification
Base model	google/gemma-4-31B-it at revision 842da3794eaa0b77d5f08bae87a17459d91ff475
Architecture	Dense 31B, 60 layers
Adaptation	LoRA rank 64, alpha 128: SFT then on-policy distillation (exact reverse KL against the teacher on every token), merged into BF16
Merge check	fp32 merge, one cast to BF16. The 410 adapted tensors changed; the other 778 are byte-identical to Google's. Merged vs Google's base plus the adapter at runtime: full-vocab KL 0.0005, top-1 97.7%, 18 of 20 greedy replies identical.
Context	262,144 tokens native. Start at 32,768. Training rows went up to 41,758 tokens; long-chat evals went to 30 turns.
Quantization	GGUF with llama.cpp 620d2225 (v0.7.0) from a BF16 GGUF master; Q6_K, Q5_K_M, Q4_K_M and IQ4_XS use an importance matrix, Q8_0 doesn't. FP8 and GPTQ-Int4 safetensors with llm-compressor 0.14.0.1. Importance matrix: WikiText-2 train (first 128 x 512 tokens) plus a chat and tool-call text mix in the Gemma chat template.
Modality	Trained and tested on text only.
License	Apache-2.0

Standalone LoRA

Gemma-4-31B-Humanlike-Chat-LoRA-r64-BF16.gguf is the whole finetune as one rank-64 GGUF LoRA for llama.cpp, 0.98 GB. Use it to run Humanlike on top of a GGUF of Google's google/gemma-4-31B-it.

llama-server -m YOUR-gemma-4-31B-it-BF16.gguf \
  --lora Gemma-4-31B-Humanlike-Chat-LoRA-r64-BF16.gguf \
  --jinja \
  --ctx-size 32768 --parallel 1 --n-gpu-layers 99 \
  --temp 1.0 --top-p 0.95 --top-k 64

• --lora applies it at scale 1.0. That is the right strength. There is no strength to tune.
• On Google's BF16 GGUF it passed every check, with KL 0.0004 vs the reference.
• Apply it only to Google's unadapted base, never to the merged files in this repo. They already contain it.
• The PEFT version for transformers and vLLM is in lora/ (adapter_model.safetensors, adapter_config.json).

Custom finetunes

I take commissions. If you want a model that sounds like a specific character, your product, or your own texting style, or a smaller model that knows your domain, I can build it the way I built this one: data, training, evals against the base, and GGUF files you can run or a hosted endpoint.

Good fits:

• game characters and companions that stay in character
• a brand or support voice that doesn't sound like a support bot
• agents that talk like a colleague
• a model trained on your own chats
• distillation: a big model's behaviour moved into a smaller, cheaper one you can run yourself
• domain expertise distillation: a model that knows your field (legal, medical, support, your codebase) without calling a frontier API
• use-case specific finetunes: one job done reliably, like your tool calls, your output format or your workflow

DM codebottle on Discord. Tell me what you're building and what it should sound like.

Found a chat where it slips? Tell me in the Community tab or on Discord.

License

Apache-2.0, inherited from Google's Gemma 4 release.