Instructions to use webmp3/Sakura-Humanlike-Chat-2.0-AutoRound-GGUF with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Notebooks
Google Colab
Kaggle
Local Apps Settings

How to use webmp3/Sakura-Humanlike-Chat-2.0-AutoRound-GGUF with llama.cpp:

Install (macOS, Linux)

curl -LsSf https://llama.app/install.sh | sh
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf webmp3/Sakura-Humanlike-Chat-2.0-AutoRound-GGUF:Q3_K_M
# Run inference directly in the terminal:
llama cli -hf webmp3/Sakura-Humanlike-Chat-2.0-AutoRound-GGUF:Q3_K_M

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf webmp3/Sakura-Humanlike-Chat-2.0-AutoRound-GGUF:Q3_K_M
# Run inference directly in the terminal:
llama cli -hf webmp3/Sakura-Humanlike-Chat-2.0-AutoRound-GGUF:Q3_K_M

Use pre-built binary

# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases
# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf webmp3/Sakura-Humanlike-Chat-2.0-AutoRound-GGUF:Q3_K_M
# Run inference directly in the terminal:
./llama-cli -hf webmp3/Sakura-Humanlike-Chat-2.0-AutoRound-GGUF:Q3_K_M

Build from source code

git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf webmp3/Sakura-Humanlike-Chat-2.0-AutoRound-GGUF:Q3_K_M
# Run inference directly in the terminal:
./build/bin/llama-cli -hf webmp3/Sakura-Humanlike-Chat-2.0-AutoRound-GGUF:Q3_K_M

Use Docker

docker model run hf.co/webmp3/Sakura-Humanlike-Chat-2.0-AutoRound-GGUF:Q3_K_M

Llama new
LM Studio
Jan

vLLM

How to use webmp3/Sakura-Humanlike-Chat-2.0-AutoRound-GGUF with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "webmp3/Sakura-Humanlike-Chat-2.0-AutoRound-GGUF"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "webmp3/Sakura-Humanlike-Chat-2.0-AutoRound-GGUF",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/webmp3/Sakura-Humanlike-Chat-2.0-AutoRound-GGUF:Q3_K_M

Ollama
How to use webmp3/Sakura-Humanlike-Chat-2.0-AutoRound-GGUF with Ollama:
```
ollama run hf.co/webmp3/Sakura-Humanlike-Chat-2.0-AutoRound-GGUF:Q3_K_M
```
Unsloth Desktop

How to use webmp3/Sakura-Humanlike-Chat-2.0-AutoRound-GGUF with Pi:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf webmp3/Sakura-Humanlike-Chat-2.0-AutoRound-GGUF:Q3_K_M

Configure the model in Pi

# Install Pi:
npm install -g @earendil-works/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "llama-cpp": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "webmp3/Sakura-Humanlike-Chat-2.0-AutoRound-GGUF:Q3_K_M"
        }
      ]
    }
  }
}

Run Pi

# Start Pi in your project directory:
pi

Docker Model Runner
How to use webmp3/Sakura-Humanlike-Chat-2.0-AutoRound-GGUF with Docker Model Runner:
```
docker model run hf.co/webmp3/Sakura-Humanlike-Chat-2.0-AutoRound-GGUF:Q3_K_M
```

Lemonade

How to use webmp3/Sakura-Humanlike-Chat-2.0-AutoRound-GGUF with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull webmp3/Sakura-Humanlike-Chat-2.0-AutoRound-GGUF:Q3_K_M

Run and chat with the model

lemonade run user.Sakura-Humanlike-Chat-2.0-AutoRound-GGUF-Q3_K_M

List all available models

lemonade list

Hermes Agent

How to use webmp3/Sakura-Humanlike-Chat-2.0-AutoRound-GGUF with Hermes Agent:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf webmp3/Sakura-Humanlike-Chat-2.0-AutoRound-GGUF:Q3_K_M

Configure Hermes

# Install Hermes:
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
# Point Hermes at the local server:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default webmp3/Sakura-Humanlike-Chat-2.0-AutoRound-GGUF:Q3_K_M

Run Hermes

hermes

Atomic Chat

OpenClaw

How to use webmp3/Sakura-Humanlike-Chat-2.0-AutoRound-GGUF with OpenClaw:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf webmp3/Sakura-Humanlike-Chat-2.0-AutoRound-GGUF:Q3_K_M

Configure OpenClaw

# Install OpenClaw:
npm install -g openclaw@latest
# Register the local server and set it as the default model:
openclaw onboard --non-interactive --mode local \
  --auth-choice custom-api-key \
  --custom-base-url http://127.0.0.1:8080/v1 \
  --custom-model-id "webmp3/Sakura-Humanlike-Chat-2.0-AutoRound-GGUF:Q3_K_M" \
  --custom-provider-id llama-cpp \
  --custom-compatibility openai \
  --custom-text-input \
  --accept-risk \
  --skip-health

Run OpenClaw

openclaw agent --local --agent main --message "Hello from Hugging Face"

Sakura Qwen3.8-27B Humanlike Chat 2.0 — Uncensored AutoRound Q3_K_M GGUF

This repository provides quantized GGUF releases representing the synergy of three key components: Uncensored + Humanlike 2.0 + AutoRound Q3_K_M.

Recommended / Winner: Humanlike-Chat-2.0-Uncensored-AutoRound-Q3_K_M.gguf (AutoRound GPU Q3_K_M)
Comparison Baseline: Humanlike-Chat-2.0-Uncensored-imatrix-Q3_K_M.gguf (Standard llama.cpp Imatrix Q3_K_M)

Key Result: AutoRound Q3_K_M won the predefined Q3 fidelity gate against the same-size Imatrix Q3_K_M baseline.

Model Provenance

Qwen/Qwen3.8-27B
  ↓
huihui-ai/Huihui-Qwen3.8-27B-abliterated
  ↓
LessThanThreeAI/Qwen3.8-27B-Humanlike-Chat-2.0
  ↓
Sakura AutoRound Q3_K_M

The Trio: Uncensored • Humanlike 2.0 • AutoRound

Uncensored (Abliterated Base): The foundation originates from huihui-ai/Huihui-Qwen3.8-27B-abliterated. Refusal vectors were removed through targeted layer-wise abliteration specifically across layers 18 to 51 (leaving initial representation layers 0 to 17 and vision/MTP untouched), effectively stripping refusal mechanisms while preserving core model reasoning and capabilities.
Humanlike 2.0 (Conversational Fine-Tune): Fine-tuned by LessThanThreeAI in LessThanThreeAI/Qwen3.8-27B-Humanlike-Chat-2.0 using on-policy distillation. The model is specifically tuned to text and converse naturally like a human, with nuance, personality, and roleplay capability—without requiring elaborate system prompt instructions.
AutoRound (Advanced 3-Bit Quantization): Quantized using Intel's AutoRound optimization across all 64 layers against BF16 reference logits. AutoRound significantly reduces layer-wise rounding error in low-bit representation, beating the same-size importance-matrix quant across held-out domains.

Model Details

Direct Upstream Base: LessThanThreeAI/Qwen3.8-27B-Humanlike-Chat-2.0
Abliterated Ancestor: huihui-ai/Huihui-Qwen3.8-27B-abliterated
Base Architecture: Qwen3.8-27B hybrid architecture with linear attention / SSM and full attention (64 transformer layers + 1 MTP layer)
Context Length: 262,144 tokens (native context support in llama.cpp)
License: Apache-2.0
Intended Inference: Local CPU / GPU / Vulkan execution with llama.cpp

Why this Quant?

Quantizing large 27B hybrid models to 3-bit typically poses severe challenges to output fidelity. In this project, AutoRound optimization was run across the model's 64 layers against full-precision (BF16) reference logits to minimize layer-wise rounding error.

To ensure rigorous validation before publishing, AutoRound Q3_K_M was benchmarked head-to-head against a same-size standard importance matrix quant (Q3_K_M) using an identical BF16 baseline across three held-out test domains (English conversational text, developer code, and Wikipedia text).

The predefined release criterion required:

File size not larger than the baseline (within 1%)
Mean KL divergence improved by at least 10%
Better KL divergence on at least 2 of 3 evaluation texts

AutoRound GPU Q3 satisfied all three criteria and won the release gate.

Quantization Comparison

Evaluation protocol: llama-perplexity, context length 512, 12 chunks per text, batch size 512. Base logits evaluated in BF16 precision; quantized candidates evaluated with identical settings.

Metric	Imatrix Q3_K_M	AutoRound GPU Q3
Size bytes	13,500,737,152	13,500,736,832
KL EN	0.0421	0.0328
KL dev/code	0.0470	0.0332
KL Wiki	0.0499	0.0510
Mean KL	0.0463	0.0390
Top-1 EN	96.0%	96.4%
Top-1 dev/code	94.5%	95.6%
Top-1 Wiki	91.1%	90.0%

Important Benchmark Notes

For complete transparency:

Mean KL divergence improved by approximately 15.8% (relative drop from 0.0463 to 0.0390).
English conversational text improved (KL 0.0328 vs 0.0421; Top-1 96.4% vs 96.0%).
Developer / code text improved significantly (KL 0.0332 vs 0.0470; Top-1 95.6% vs 94.5%).
Wikipedia text regressed slightly (KL 0.0510 vs 0.0499; Top-1 90.0% vs 91.1%).
Perplexity (PPL) was slightly worse for AutoRound on all three evaluation sets (EN: 1.734 vs 1.728; dev: 1.841 vs 1.840; Wiki: 7.680 vs 7.615). PPL was not the predefined gate metric because KL divergence and top-1 agreement directly measure fidelity to original model output logits.
Size difference: AutoRound is 320 bytes smaller than the Imatrix file (13,500,736,832 vs 13,500,737,152 bytes), which is essentially identical size; 320 bytes is not a meaningful size benefit.
The results apply specifically to the tested evaluation domains.

Files in this Repository

File	Type	Role	Size	SHA256
`Humanlike-Chat-2.0-Uncensored-AutoRound-Q3_K_M.gguf`	Q3_K_M	Recommended / Winner	12.57 GiB (13,500,736,832 bytes)	`0fb6192dc16005f2dbeea449c6e6befc9732909d1541cb0197d492688088ed52`
`Humanlike-Chat-2.0-Uncensored-imatrix-Q3_K_M.gguf`	Q3_K_M	Comparison Baseline	12.57 GiB (13,500,737,152 bytes)	`444ae92cbb266df95d1acd1598a7b1107d639ecee4b2e55e5b506bf54fbe60e1`

Download

Recommended: AutoRound Q3_K_M (Winner)

Using the Hugging Face CLI:

hf download webmp3/Sakura-Humanlike-Chat-2.0-AutoRound-GGUF Humanlike-Chat-2.0-Uncensored-AutoRound-Q3_K_M.gguf --local-dir .

Or via curl:

curl -L -o Humanlike-Chat-2.0-Uncensored-AutoRound-Q3_K_M.gguf \
  https://huggingface.co/webmp3/Sakura-Humanlike-Chat-2.0-AutoRound-GGUF/resolve/main/Humanlike-Chat-2.0-Uncensored-AutoRound-Q3_K_M.gguf

Baseline: Imatrix Q3_K_M (For Comparison)

hf download webmp3/Sakura-Humanlike-Chat-2.0-AutoRound-GGUF Humanlike-Chat-2.0-Uncensored-imatrix-Q3_K_M.gguf --local-dir .

llama.cpp Example

Run with llama-cli or llama-server:

llama-cli \
  -m Humanlike-Chat-2.0-Uncensored-AutoRound-Q3_K_M.gguf \
  -ngl 99 \
  -c 8192 \
  --temp 1.0 \
  --top-p 0.95 \
  --top-k 20 \
  -p "<|im_start|>user\nHello! How can you assist me today?<|im_end|>\n<|im_start|>assistant\n"

Note on Thinking: Like the upstream Humanlike-Chat 2.0 model, thinking tokens are enabled by default in the chat template.

Related Sakura models

webmp3/Sakura-Humanlike-Chat-2.0-JEV-AutoRound-GGUF: JEV-enabled variant with calibrated System-One decision capabilities.

License & Attribution

Upstream Model: LessThanThreeAI/Qwen3.8-27B-Humanlike-Chat-2.0 by LessThanThreeAI.
Abliterated Ancestor: huihui-ai/Huihui-Qwen3.8-27B-abliterated by huihui-ai.
Base Architecture: Qwen3.8 by Alibaba Cloud.
License: Released under the Apache-2.0 License.
Quantization: Sakura Quantization Pipeline with Intel AutoRound & llama.cpp.

中文说明 · 樱花 (Simplified Chinese)

English above. 本节为上文的中文翻译(Sakura = 樱花 yīnghuā);完整的独立中文版见 README_zh.md。

Sakura Qwen3.8-27B Humanlike Chat 2.0 — Uncensored AutoRound Q3_K_M GGUF

本仓库提供量化的 GGUF 发布版本,体现了三个关键要素的协同: Uncensored + Humanlike 2.0 + AutoRound Q3_K_M。

推荐 / 胜出: Humanlike-Chat-2.0-Uncensored-AutoRound-Q3_K_M.gguf(AutoRound GPU Q3_K_M)
对照基线: Humanlike-Chat-2.0-Uncensored-imatrix-Q3_K_M.gguf(标准 llama.cpp Imatrix Q3_K_M)

关键结果: AutoRound Q3_K_M 在预先设定的 Q3 保真度关卡中胜过了相同大小的 Imatrix Q3_K_M 基线。

模型谱系

Qwen/Qwen3.8-27B
  ↓
huihui-ai/Huihui-Qwen3.8-27B-abliterated
  ↓
LessThanThreeAI/Qwen3.8-27B-Humanlike-Chat-2.0
  ↓
Sakura AutoRound Q3_K_M

三者组合:Uncensored • Humanlike 2.0 • AutoRound

Uncensored(去除拒绝的基础模型): 基础来自 huihui-ai/Huihui-Qwen3.8-27B-abliterated。拒绝向量通过有针对性的逐层 abliteration 被移除,具体作用于第 18 至 51 层(保留最初的表示层 0 至 17 以及视觉/MTP 不变),在保留模型核心推理能力的同时有效去除了拒绝机制。
Humanlike 2.0(对话微调): 由 LessThanThreeAI 在 LessThanThreeAI/Qwen3.8-27B-Humanlike-Chat-2.0 中通过 on-policy 蒸馏微调。该模型专门调校为像人一样自然地发消息和交谈,具有细腻的语气、个性和角色扮演能力,而无需复杂的系统提示指令。
AutoRound(先进的 3 比特量化): 使用 Intel 的 AutoRound 优化,针对 BF16 参照 logits 对全部 64 层进行量化。AutoRound 显著降低了低比特表示中的逐层舍入误差,在留出的各个领域上胜过了相同大小的重要性矩阵量化。

模型详情

直接上游基础模型: LessThanThreeAI/Qwen3.8-27B-Humanlike-Chat-2.0
abliterated 的祖先模型: huihui-ai/Huihui-Qwen3.8-27B-abliterated
基础架构: Qwen3.8-27B 混合架构,包含线性注意力 / SSM 和全注意力(64 个 Transformer 层 + 1 个 MTP 层)
上下文长度: 262,144 个 token(llama.cpp 原生支持的上下文)
许可证: Apache-2.0
预期推理方式: 使用 llama.cpp 在本地 CPU / GPU / Vulkan 上运行

为什么选这个量化?

把 27B 混合模型量化到 3 比特,通常会对输出保真度造成严重挑战。在本项目中,AutoRound 优化针对全精度(BF16)参照 logits 在模型的 64 层上运行,以最小化逐层舍入误差。

为确保发布前的严格验证,我们把 AutoRound Q3_K_M 与相同大小的标准重要性矩阵量化(Q3_K_M)进行了正面对比,使用相同的 BF16 基线,在三个留出的测试领域(英文对话文本、开发者代码和维基百科文本)上测量。

预先设定的发布标准要求:

文件大小不超过基线(在 1% 以内)
平均 KL 散度至少改善 10%
在 3 个评测文本中至少 2 个上取得更好的 KL 散度

AutoRound GPU Q3 满足全部三项标准,通过了发布关卡。

量化对比

评测协议:llama-perplexity,上下文长度 512,每个文本 12 个块,批大小 512。基础 logits 以 BF16 精度评估;量化候选以相同设置评估。

指标	Imatrix Q3_K_M	AutoRound GPU Q3
Size bytes	13,500,737,152	13,500,736,832
KL EN	0.0421	0.0328
KL dev/code	0.0470	0.0332
KL Wiki	0.0499	0.0510
Mean KL	0.0463	0.0390
Top-1 EN	96.0%	96.4%
Top-1 dev/code	94.5%	95.6%
Top-1 Wiki	91.1%	90.0%

重要的基准测试说明

为保证完全透明:

平均 KL 散度改善约 15.8%(从 0.0463 降到 0.0390 的相对降幅)。
英文对话文本有所改善(KL 0.0328 对 0.0421;Top-1 96.4% 对 96.0%)。
开发者 / 代码文本明显改善(KL 0.0332 对 0.0470;Top-1 95.6% 对 94.5%)。
维基百科文本略有退步(KL 0.0510 对 0.0499;Top-1 90.0% 对 91.1%)。
AutoRound 在全部三个评测集上的 困惑度(PPL)略差(EN: 1.734 对 1.728;dev: 1.841 对 1.840;Wiki: 7.680 对 7.615)。PPL 不是预先设定的关卡指标,因为 KL 散度和 top-1 一致性直接衡量对原始模型输出 logits 的保真度。
大小差异: AutoRound 比 Imatrix 文件小 320 字节(13,500,736,832 对 13,500,737,152 字节),大小基本相同;320 字节并不构成有意义的大小优势。
这些结果仅适用于所测试的评测领域。

本仓库中的文件

文件	类型	角色	大小	SHA256
`Humanlike-Chat-2.0-Uncensored-AutoRound-Q3_K_M.gguf`	Q3_K_M	推荐 / 胜出	12.57 GiB (13,500,736,832 bytes)	`0fb6192dc16005f2dbeea449c6e6befc9732909d1541cb0197d492688088ed52`
`Humanlike-Chat-2.0-Uncensored-imatrix-Q3_K_M.gguf`	Q3_K_M	对照基线	12.57 GiB (13,500,737,152 bytes)	`444ae92cbb266df95d1acd1598a7b1107d639ecee4b2e55e5b506bf54fbe60e1`

下载

推荐:AutoRound Q3_K_M(胜出)

使用 Hugging Face CLI:

hf download webmp3/Sakura-Humanlike-Chat-2.0-AutoRound-GGUF Humanlike-Chat-2.0-Uncensored-AutoRound-Q3_K_M.gguf --local-dir .

或通过 curl:

curl -L -o Humanlike-Chat-2.0-Uncensored-AutoRound-Q3_K_M.gguf \
  https://huggingface.co/webmp3/Sakura-Humanlike-Chat-2.0-AutoRound-GGUF/resolve/main/Humanlike-Chat-2.0-Uncensored-AutoRound-Q3_K_M.gguf

基线:Imatrix Q3_K_M(用于对比)

hf download webmp3/Sakura-Humanlike-Chat-2.0-AutoRound-GGUF Humanlike-Chat-2.0-Uncensored-imatrix-Q3_K_M.gguf --local-dir .

llama.cpp 示例

使用 llama-cli 或 llama-server 运行:

llama-cli \
  -m Humanlike-Chat-2.0-Uncensored-AutoRound-Q3_K_M.gguf \
  -ngl 99 \
  -c 8192 \
  --temp 1.0 \
  --top-p 0.95 \
  --top-k 20 \
  -p "<|im_start|>user\nHello! How can you assist me today?<|im_end|>\n<|im_start|>assistant\n"

关于思考的说明: 与上游的 Humanlike-Chat 2.0 模型一样,聊天模板中默认启用思考 token。

许可证与归属

上游模型: LessThanThreeAI 的 LessThanThreeAI/Qwen3.8-27B-Humanlike-Chat-2.0。
abliterated 的祖先模型: huihui-ai 的 huihui-ai/Huihui-Qwen3.8-27B-abliterated。
基础架构: 阿里云的 Qwen3.8。
许可证: 以 Apache-2.0 许可证发布。
量化: 使用 Intel AutoRound 和 llama.cpp 的 Sakura 量化流水线。

Downloads last month: 411

GGUF

Model size

27B params

Architecture

qwen35

Hardware compatibility

3-bit

Model tree for webmp3/Sakura-Humanlike-Chat-2.0-AutoRound-GGUF

Base model

Qwen/Qwen3.8-27B

Finetuned

huihui-ai/Huihui-Qwen3.8-27B-abliterated

Quantized

LessThanThreeAI/Qwen3.8-27B-Humanlike-Chat-GGUF

Quantized

(3)

this model

webmp3
/

Sakura-Humanlike-Chat-2.0-AutoRound-GGUF

Sakura Qwen3.8-27B Humanlike Chat 2.0 — Uncensored AutoRound Q3_K_M GGUF

Model Provenance

The Trio: Uncensored • Humanlike 2.0 • AutoRound

Model Details

Why this Quant?

Quantization Comparison

Important Benchmark Notes

Files in this Repository

Download

Recommended: AutoRound Q3_K_M (Winner)

Baseline: Imatrix Q3_K_M (For Comparison)

llama.cpp Example

Related Sakura models

License & Attribution

Sakura Qwen3.8-27B Humanlike Chat 2.0 — Uncensored AutoRound Q3_K_M GGUF

模型谱系

三者组合:Uncensored • Humanlike 2.0 • AutoRound

模型详情

为什么选这个量化?

量化对比

重要的基准测试说明

本仓库中的文件

下载

推荐:AutoRound Q3_K_M(胜出)

基线:Imatrix Q3_K_M(用于对比)

llama.cpp 示例

相关的 Sakura 模型

许可证与归属

Model tree for webmp3/Sakura-Humanlike-Chat-2.0-AutoRound-GGUF