Instructions to use LessThanThreeAI/Qwen3.8-27B-Humanlike-Chat-2.0-GPTQ-Int4 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use LessThanThreeAI/Qwen3.8-27B-Humanlike-Chat-2.0-GPTQ-Int4 with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="LessThanThreeAI/Qwen3.8-27B-Humanlike-Chat-2.0-GPTQ-Int4")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
pipe(text=messages)

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoProcessor, AutoModelForMultimodalLM

processor = AutoProcessor.from_pretrained("LessThanThreeAI/Qwen3.8-27B-Humanlike-Chat-2.0-GPTQ-Int4")
model = AutoModelForMultimodalLM.from_pretrained("LessThanThreeAI/Qwen3.8-27B-Humanlike-Chat-2.0-GPTQ-Int4", device_map="auto")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
inputs = processor.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=256)
print(processor.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use LessThanThreeAI/Qwen3.8-27B-Humanlike-Chat-2.0-GPTQ-Int4 with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "LessThanThreeAI/Qwen3.8-27B-Humanlike-Chat-2.0-GPTQ-Int4"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "LessThanThreeAI/Qwen3.8-27B-Humanlike-Chat-2.0-GPTQ-Int4",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/LessThanThreeAI/Qwen3.8-27B-Humanlike-Chat-2.0-GPTQ-Int4

SGLang

How to use LessThanThreeAI/Qwen3.8-27B-Humanlike-Chat-2.0-GPTQ-Int4 with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "LessThanThreeAI/Qwen3.8-27B-Humanlike-Chat-2.0-GPTQ-Int4" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "LessThanThreeAI/Qwen3.8-27B-Humanlike-Chat-2.0-GPTQ-Int4",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "LessThanThreeAI/Qwen3.8-27B-Humanlike-Chat-2.0-GPTQ-Int4" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "LessThanThreeAI/Qwen3.8-27B-Humanlike-Chat-2.0-GPTQ-Int4",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use LessThanThreeAI/Qwen3.8-27B-Humanlike-Chat-2.0-GPTQ-Int4 with Docker Model Runner:
```
docker model run hf.co/LessThanThreeAI/Qwen3.8-27B-Humanlike-Chat-2.0-GPTQ-Int4
```

Qwen3.8-27B-Humanlike-Chat 2.0 (GPTQ Int4)

A 27B model that texts like a person and still does the work, in 4-bit for vLLM. No system prompt needed.

Results, examples and how it was made are on the main card: Qwen3.8-27B-Humanlike-Chat-GGUF.

"Base" here always means huihui-ai/Huihui-Qwen3.8-27B-abliterated, an abliterated Qwen3.8-27B. It is not the official Qwen release.

vLLM

Tested with vLLM 0.27.1:

vllm serve LessThanThreeAI/Qwen3.8-27B-Humanlike-Chat-2.0-GPTQ-Int4 --served-model-name humanlike-2.0 \
  --max-model-len 16384 --language-model-only \
  --enable-auto-tool-choice --tool-call-parser qwen3_coder --reasoning-parser qwen3

Thinking is on by default. Send chat_template_kwargs: {"enable_thinking": false} per request for the fastest replies. Sampling: temperature 1.0, top_p 0.95, top_k 20.

Format

GPTQ W4A16 made with llm-compressor, saved as compressed-tensors: 4-bit symmetric integer weights in groups of 32, BF16 activations. The 400 linear layers of the 64 language layers are 4-bit. These stay BF16: embeddings, lm_head, norms, the linear-attention gate tensors (in_proj_a, in_proj_b, A_log, dt_bias, conv), the vision tower and the MTP head. Calibration used 117k tokens: WikiText-2 plus chat and tool-call text, the same mix as the GGUF importance matrix. Groups of 128 and AWQ smoothing were also tried. Both changed answers in the checks below, so this repo uses groups of 32.

VRAM

The weights take 17.9 GiB in vLLM. A 24 GB card fits 8k context: add --max-model-len 8192 --max-num-seqs 4 --max-num-batched-tokens 2048 --gpu-memory-utilization 0.91. Tested with vLLM capped at 21.9 GiB (0.91 of a 24 GB card): the server started with room for 11,702 tokens of cache and passed the chat, thinking and tool-call smoke test.

Checks

Same check as every file on the main card: 5 fresh test chats against the reference (the base plus the 2.0 adapter at runtime in BF16), served on vLLM 0.27.1, one H100 NVL.

File	KL vs reference	Top-1 agreement	Greedy turns identical	Check fails (greedy / sampled)	Decode tok/s
BF16	0.0032	97.2%	12 of 15	0 / 0	46
FP8	0.0079	91.7%	9 of 15	0 / 0	72
GPTQ Int4	0.0274	91.7%	8 of 15	0 / 0	86

For comparison, IQ4_XS on the main card has KL 0.0219 and top-1 83.3%.

Live requests on the same server: "yo you up" with thinking off came back yeah what's up with no reasoning. With thinking on, the train question came back 17:35 with the reasoning in its own field. A weather request produced get_weather with {"city": "Lisbon", "unit": "celsius"}.

Commissions

Commissions open, DM codebottle on Discord (https://discord.com/users/320486798859960322). I build custom finetunes like this one: characters, product voices, distillation into smaller models, and domain or use-case specific models.

License

Apache-2.0, inherited from the upstream Qwen and Huihui releases.

Downloads last month: 226

Safetensors

Model size

27B params

Tensor type

I32

BF16

Model tree for LessThanThreeAI/Qwen3.8-27B-Humanlike-Chat-2.0-GPTQ-Int4

Base model

Qwen/Qwen3.8-27B

Finetuned

huihui-ai/Huihui-Qwen3.8-27B-abliterated

Finetuned

LessThanThreeAI/Qwen3.8-27B-Humanlike-Chat-2.0

Quantized

(5)

this model