Instructions to use LessThanThreeAI/Gemma-4-31B-Humanlike-Chat with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use LessThanThreeAI/Gemma-4-31B-Humanlike-Chat with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="LessThanThreeAI/Gemma-4-31B-Humanlike-Chat")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
pipe(text=messages)

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoProcessor, AutoModelForMultimodalLM

processor = AutoProcessor.from_pretrained("LessThanThreeAI/Gemma-4-31B-Humanlike-Chat")
model = AutoModelForMultimodalLM.from_pretrained("LessThanThreeAI/Gemma-4-31B-Humanlike-Chat", device_map="auto")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
inputs = processor.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=256)
print(processor.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use LessThanThreeAI/Gemma-4-31B-Humanlike-Chat with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "LessThanThreeAI/Gemma-4-31B-Humanlike-Chat"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "LessThanThreeAI/Gemma-4-31B-Humanlike-Chat",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/LessThanThreeAI/Gemma-4-31B-Humanlike-Chat

SGLang

How to use LessThanThreeAI/Gemma-4-31B-Humanlike-Chat with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "LessThanThreeAI/Gemma-4-31B-Humanlike-Chat" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "LessThanThreeAI/Gemma-4-31B-Humanlike-Chat",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "LessThanThreeAI/Gemma-4-31B-Humanlike-Chat" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "LessThanThreeAI/Gemma-4-31B-Humanlike-Chat",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use LessThanThreeAI/Gemma-4-31B-Humanlike-Chat with Docker Model Runner:
```
docker model run hf.co/LessThanThreeAI/Gemma-4-31B-Humanlike-Chat
```

Gemma-4-31B-Humanlike-Chat / README.md

kvyb

Model card

cf72913 verified 3 days ago

preview code

Raw History Blame Contribute Delete

2.84 kB

metadata

license: apache-2.0
base_model: google/gemma-4-31B-it
base_model_relation: finetune
library_name: transformers
pipeline_tag: text-generation
language:
  - en
  - ru
tags:
  - vllm
  - sglang
  - safetensors
  - gemma4
  - conversational
  - tool-calling
  - function-calling
  - roleplay
  - character
  - humanlike
  - on-policy-distillation

Gemma-4-31B-Humanlike-Chat (BF16 safetensors)

A 31B Gemma that texts like a person and still does the work. No system prompt needed. This repo has the full BF16 weights for vLLM, SGLang and transformers.

Results, examples and how it was made are on the main card: Gemma-4-31B-Humanlike-Chat-GGUF.

"Base" here always means Google's google/gemma-4-31B-it, the official release. It is not abliterated.

vLLM

Tested with vLLM 0.31.0 on an RTX PRO 6000 Blackwell (96 GB):

vllm serve LessThanThreeAI/Gemma-4-31B-Humanlike-Chat --served-model-name humanlike-gemma \
  --max-model-len 16384 --language-model-only \
  --enable-auto-tool-choice --tool-call-parser gemma4 --reasoning-parser gemma4

Thinking is off by default, as in Google's Gemma 4 template. Send chat_template_kwargs: {"enable_thinking": true} per request to turn it on; the thinking comes back in its own field. It was trained with effort levels: add the line Reasoning effort: low or Reasoning effort: high to your system prompt, or no line for medium. Sampling: temperature 1.0, top_p 0.95, top_k 64 (Google's defaults in generation_config.json). For SGLang, point --model-path at this repo.

VRAM

The weights take 57.8 GiB in vLLM. On the 96 GB test card at --gpu-memory-utilization 0.88, that left 22.7 GiB for the cache, 26,969 tokens. With less memory, take the FP8 or GPTQ-Int4 repo, or a GGUF from the main card.

Checks

Same check as every file on the main card: 5 fresh test chats against the reference (the merged BF16 weights in vLLM, which match Google's base plus the Humanlike adapter at runtime: KL 0.0005), served on vLLM 0.31.0, one NVIDIA RTX PRO 6000 Blackwell Server Edition (96 GB).

File	KL vs reference	Top-1 agreement	Greedy turns identical	Check fails (greedy / sampled)	Decode tok/s
BF16	reference	reference	reference	0 / 0	23

Commissions

Commissions open, DM codebottle on Discord (https://discord.com/users/320486798859960322). I build custom finetunes like this one: characters, product voices, distillation into smaller models, and domain or use-case specific models.

License

Apache-2.0, inherited from Google's Gemma 4 release.