Instructions to use LessThanThreeAI/Gemma-4-31B-Humanlike-Chat with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use LessThanThreeAI/Gemma-4-31B-Humanlike-Chat with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="LessThanThreeAI/Gemma-4-31B-Humanlike-Chat")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
pipe(text=messages)

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoProcessor, AutoModelForMultimodalLM

processor = AutoProcessor.from_pretrained("LessThanThreeAI/Gemma-4-31B-Humanlike-Chat")
model = AutoModelForMultimodalLM.from_pretrained("LessThanThreeAI/Gemma-4-31B-Humanlike-Chat", device_map="auto")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
inputs = processor.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=256)
print(processor.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use LessThanThreeAI/Gemma-4-31B-Humanlike-Chat with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "LessThanThreeAI/Gemma-4-31B-Humanlike-Chat"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "LessThanThreeAI/Gemma-4-31B-Humanlike-Chat",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/LessThanThreeAI/Gemma-4-31B-Humanlike-Chat

SGLang

How to use LessThanThreeAI/Gemma-4-31B-Humanlike-Chat with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "LessThanThreeAI/Gemma-4-31B-Humanlike-Chat" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "LessThanThreeAI/Gemma-4-31B-Humanlike-Chat",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "LessThanThreeAI/Gemma-4-31B-Humanlike-Chat" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "LessThanThreeAI/Gemma-4-31B-Humanlike-Chat",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use LessThanThreeAI/Gemma-4-31B-Humanlike-Chat with Docker Model Runner:
```
docker model run hf.co/LessThanThreeAI/Gemma-4-31B-Humanlike-Chat
```

Gemma-4-31B-Humanlike-Chat / README.md

kvyb

Model card

cf72913 verified 3 days ago

preview code

Raw History Blame Contribute Delete

2.84 kB

---
license: apache-2.0
base_model: google/gemma-4-31B-it
base_model_relation: finetune
library_name: transformers
pipeline_tag: text-generation
language:
  - en
  - ru
tags:
  - vllm
  - sglang
  - safetensors
  - gemma4
  - conversational
  - tool-calling
  - function-calling
  - roleplay
  - character
  - humanlike
  - on-policy-distillation
---
# Gemma-4-31B-Humanlike-Chat (BF16 safetensors)

**A 31B Gemma that texts like a person and still does the work.** No system prompt needed. This repo has the full BF16 weights for vLLM, SGLang and transformers.

Results, examples and how it was made are on the main card: [Gemma-4-31B-Humanlike-Chat-GGUF](https://huggingface.co/LessThanThreeAI/Gemma-4-31B-Humanlike-Chat-GGUF).

"Base" here always means Google's `google/gemma-4-31B-it`, the official release. It is not abliterated.

## vLLM

Tested with vLLM 0.31.0 on an RTX PRO 6000 Blackwell (96 GB):

```bash
vllm serve LessThanThreeAI/Gemma-4-31B-Humanlike-Chat --served-model-name humanlike-gemma \
  --max-model-len 16384 --language-model-only \
  --enable-auto-tool-choice --tool-call-parser gemma4 --reasoning-parser gemma4
```

Thinking is off by default, as in Google's Gemma 4 template. Send `chat_template_kwargs: {"enable_thinking": true}` per request to turn it on; the thinking comes back in its own field. It was trained with effort levels: add the line `Reasoning effort: low` or `Reasoning effort: high` to your system prompt, or no line for medium. Sampling: temperature 1.0, top_p 0.95, top_k 64 (Google's defaults in `generation_config.json`). For SGLang, point `--model-path` at this repo.

## VRAM

The weights take 57.8 GiB in vLLM. On the 96 GB test card at `--gpu-memory-utilization 0.88`, that left 22.7 GiB for the cache, 26,969 tokens. With less memory, take the [FP8](https://huggingface.co/LessThanThreeAI/Gemma-4-31B-Humanlike-Chat-FP8) or [GPTQ-Int4](https://huggingface.co/LessThanThreeAI/Gemma-4-31B-Humanlike-Chat-GPTQ-Int4) repo, or a GGUF from the main card.

## Checks

Same check as every file on the main card: 5 fresh test chats against the reference (the merged BF16 weights in vLLM, which match Google's base plus the Humanlike adapter at runtime: KL 0.0005), served on vLLM 0.31.0, one NVIDIA RTX PRO 6000 Blackwell Server Edition (96 GB).

| File | KL vs reference | Top-1 agreement | Greedy turns identical | Check fails (greedy / sampled) | Decode tok/s |
|---|---:|---:|---:|---:|---:|
| **BF16** | **reference** | **reference** | **reference** | **0 / 0** | **23** |

## Commissions

**Commissions open, DM codebottle on Discord (https://discord.com/users/320486798859960322).** I build custom finetunes like this one: characters, product voices, distillation into smaller models, and domain or use-case specific models.

## License

Apache-2.0, inherited from Google's Gemma 4 release.

	---
	license: apache-2.0
	base_model: google/gemma-4-31B-it
	base_model_relation: finetune
	library_name: transformers
	pipeline_tag: text-generation
	language:
	- en
	- ru
	tags:
	- vllm
	- sglang
	- safetensors
	- gemma4
	- conversational
	- tool-calling
	- function-calling
	- roleplay
	- character
	- humanlike
	- on-policy-distillation
	---

	# Gemma-4-31B-Humanlike-Chat (BF16 safetensors)

	A 31B Gemma that texts like a person and still does the work. No system prompt needed. This repo has the full BF16 weights for vLLM, SGLang and transformers.

	Results, examples and how it was made are on the main card: [Gemma-4-31B-Humanlike-Chat-GGUF](https://huggingface.co/LessThanThreeAI/Gemma-4-31B-Humanlike-Chat-GGUF).

	"Base" here always means Google's `google/gemma-4-31B-it`, the official release. It is not abliterated.

	## vLLM

	Tested with vLLM 0.31.0 on an RTX PRO 6000 Blackwell (96 GB):

	```bash
	vllm serve LessThanThreeAI/Gemma-4-31B-Humanlike-Chat --served-model-name humanlike-gemma \
	--max-model-len 16384 --language-model-only \
	--enable-auto-tool-choice --tool-call-parser gemma4 --reasoning-parser gemma4
	```

	Thinking is off by default, as in Google's Gemma 4 template. Send `chat_template_kwargs: {"enable_thinking": true}` per request to turn it on; the thinking comes back in its own field. It was trained with effort levels: add the line `Reasoning effort: low` or `Reasoning effort: high` to your system prompt, or no line for medium. Sampling: temperature 1.0, top_p 0.95, top_k 64 (Google's defaults in `generation_config.json`). For SGLang, point `--model-path` at this repo.

	## VRAM

	The weights take 57.8 GiB in vLLM. On the 96 GB test card at `--gpu-memory-utilization 0.88`, that left 22.7 GiB for the cache, 26,969 tokens. With less memory, take the [FP8](https://huggingface.co/LessThanThreeAI/Gemma-4-31B-Humanlike-Chat-FP8) or [GPTQ-Int4](https://huggingface.co/LessThanThreeAI/Gemma-4-31B-Humanlike-Chat-GPTQ-Int4) repo, or a GGUF from the main card.

	## Checks

	Same check as every file on the main card: 5 fresh test chats against the reference (the merged BF16 weights in vLLM, which match Google's base plus the Humanlike adapter at runtime: KL 0.0005), served on vLLM 0.31.0, one NVIDIA RTX PRO 6000 Blackwell Server Edition (96 GB).

	\| File \| KL vs reference \| Top-1 agreement \| Greedy turns identical \| Check fails (greedy / sampled) \| Decode tok/s \|
	\|---\|---:\|---:\|---:\|---:\|---:\|
	\| BF16 \| reference \| reference \| reference \| 0 / 0 \| 23 \|

	## Commissions

	Commissions open, DM codebottle on Discord (https://discord.com/users/320486798859960322). I build custom finetunes like this one: characters, product voices, distillation into smaller models, and domain or use-case specific models.

	## License

	Apache-2.0, inherited from Google's Gemma 4 release.