Instructions to use vinpix/Ternary-Bonsai-27B-Stock-MTP-GGUF with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use vinpix/Ternary-Bonsai-27B-Stock-MTP-GGUF with llama-cpp-python:

# !pip install llama-cpp-python

from llama_cpp import Llama

llm = Llama.from_pretrained(
	repo_id="vinpix/Ternary-Bonsai-27B-Stock-MTP-GGUF",
	filename="Ternary-Bonsai-27B-MTP-TQ2_0.gguf",
)

llm.create_chat_completion(
	messages = [
		{
			"role": "user",
			"content": "What is the capital of France?"
		}
	]
)

Notebooks
Google Colab
Kaggle
Local Apps Settings

llama.cpp

How to use vinpix/Ternary-Bonsai-27B-Stock-MTP-GGUF with llama.cpp:

Install (macOS, Linux)

curl -LsSf https://llama.app/install.sh | sh
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf vinpix/Ternary-Bonsai-27B-Stock-MTP-GGUF:TQ2_0
# Run inference directly in the terminal:
llama cli -hf vinpix/Ternary-Bonsai-27B-Stock-MTP-GGUF:TQ2_0

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf vinpix/Ternary-Bonsai-27B-Stock-MTP-GGUF:TQ2_0
# Run inference directly in the terminal:
llama cli -hf vinpix/Ternary-Bonsai-27B-Stock-MTP-GGUF:TQ2_0

Use pre-built binary

# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases
# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf vinpix/Ternary-Bonsai-27B-Stock-MTP-GGUF:TQ2_0
# Run inference directly in the terminal:
./llama-cli -hf vinpix/Ternary-Bonsai-27B-Stock-MTP-GGUF:TQ2_0

Build from source code

git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf vinpix/Ternary-Bonsai-27B-Stock-MTP-GGUF:TQ2_0
# Run inference directly in the terminal:
./build/bin/llama-cli -hf vinpix/Ternary-Bonsai-27B-Stock-MTP-GGUF:TQ2_0

Use Docker

docker model run hf.co/vinpix/Ternary-Bonsai-27B-Stock-MTP-GGUF:TQ2_0

LM Studio
Jan

vLLM

How to use vinpix/Ternary-Bonsai-27B-Stock-MTP-GGUF with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "vinpix/Ternary-Bonsai-27B-Stock-MTP-GGUF"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "vinpix/Ternary-Bonsai-27B-Stock-MTP-GGUF",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/vinpix/Ternary-Bonsai-27B-Stock-MTP-GGUF:TQ2_0

Ollama
How to use vinpix/Ternary-Bonsai-27B-Stock-MTP-GGUF with Ollama:
```
ollama run hf.co/vinpix/Ternary-Bonsai-27B-Stock-MTP-GGUF:TQ2_0
```

Unsloth Studio

How to use vinpix/Ternary-Bonsai-27B-Stock-MTP-GGUF with Unsloth Studio:

Install Unsloth Studio (macOS, Linux, WSL)

curl -fsSL https://unsloth.ai/install.sh | sh
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for vinpix/Ternary-Bonsai-27B-Stock-MTP-GGUF to start chatting

Install Unsloth Studio (Windows)

irm https://unsloth.ai/install.ps1 | iex
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for vinpix/Ternary-Bonsai-27B-Stock-MTP-GGUF to start chatting

Using HuggingFace Spaces for Unsloth

# No setup required
# Open https://huggingface.co/spaces/unsloth/studio in your browser
# Search for vinpix/Ternary-Bonsai-27B-Stock-MTP-GGUF to start chatting

How to use vinpix/Ternary-Bonsai-27B-Stock-MTP-GGUF with Pi:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf vinpix/Ternary-Bonsai-27B-Stock-MTP-GGUF:TQ2_0

Configure the model in Pi

# Install Pi:
npm install -g @mariozechner/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "llama-cpp": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "vinpix/Ternary-Bonsai-27B-Stock-MTP-GGUF:TQ2_0"
        }
      ]
    }
  }
}

Run Pi

# Start Pi in your project directory:
pi

Hermes Agent new

How to use vinpix/Ternary-Bonsai-27B-Stock-MTP-GGUF with Hermes Agent:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf vinpix/Ternary-Bonsai-27B-Stock-MTP-GGUF:TQ2_0

Configure Hermes

# Install Hermes:
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
# Point Hermes at the local server:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default vinpix/Ternary-Bonsai-27B-Stock-MTP-GGUF:TQ2_0

Run Hermes

hermes

Atomic Chat new

OpenClaw new

How to use vinpix/Ternary-Bonsai-27B-Stock-MTP-GGUF with OpenClaw:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf vinpix/Ternary-Bonsai-27B-Stock-MTP-GGUF:TQ2_0

Configure OpenClaw

# Install OpenClaw:
npm install -g openclaw@latest
# Register the local server and set it as the default model:
openclaw onboard --non-interactive --mode local \
  --auth-choice custom-api-key \
  --custom-base-url http://127.0.0.1:8080/v1 \
  --custom-model-id "vinpix/Ternary-Bonsai-27B-Stock-MTP-GGUF:TQ2_0" \
  --custom-provider-id llama-cpp \
  --custom-compatibility openai \
  --custom-text-input \
  --accept-risk \
  --skip-health

Run OpenClaw

openclaw agent --local --agent main --message "Hello from Hugging Face"

Docker Model Runner
How to use vinpix/Ternary-Bonsai-27B-Stock-MTP-GGUF with Docker Model Runner:
```
docker model run hf.co/vinpix/Ternary-Bonsai-27B-Stock-MTP-GGUF:TQ2_0
```

Lemonade

How to use vinpix/Ternary-Bonsai-27B-Stock-MTP-GGUF with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull vinpix/Ternary-Bonsai-27B-Stock-MTP-GGUF:TQ2_0

Run and chat with the model

lemonade run user.Ternary-Bonsai-27B-Stock-MTP-GGUF-TQ2_0

List all available models

lemonade list

Ternary Bonsai 27B / Embedded MTP

One immutable GGUF. A ternary 27B-class target. An embedded Qwen-style NextN predictor. Stock `llama.cpp`.

8.18 GiB · GGUF V3 · 866 tensors · 64 target blocks + 1 NextN block

ordinary decode · draft-mtp · no sidecar drafter · no private runtime patch

The contribution is the artifact boundary.
The Ternary Bonsai target and the predictor consumed by llama.cpp's draft-mtp path live in the same content-addressed GGUF.

Architecture

Ternary-Bonsai-27B-MTP-TQ2_0.gguf
│
├── qwen35 target
│   └── 64 ternary transformer blocks
│
└── embedded speculative path
    ├── dedicated MTP token embedding
    ├── 1 Qwen-style NextN predictor block
    └── MTP output path

The ordinary and speculative execution paths resolve from the same file and the same target identity.

Deployment property	External-drafter stack	This release
Target artifact	separate	embedded
Predictor artifact	separate	embedded
Runtime model identities	2	1
Target/drafter version skew	possible	structurally eliminated
Draft-model argument	required	not required
Stock `qwen35` / `draft-mtp` path	not intrinsic	yes

This is not a replacement for a purpose-trained external drafter architecture. It is a different systems trade-off: an atomic deployment object with a single checksum, a single provenance chain and no sidecar lifecycle.

Artifact fingerprint

Field	Value
File	`Ternary-Bonsai-27B-MTP-TQ2_0.gguf`
Size	8,785,215,776 bytes / 8.18 GiB
SHA-256	`480c695b0de5b78b720d7f500eb3e1650359d0ac8e18c29b231a14b2d9c9bf20`
GGUF	V3
Architecture	`qwen35`
Tensor count	866
Target depth	64 blocks
Embedded NextN depth	1 block
Dominant weight type	`TQ2_0`
Runtime contract	stock `llama.cpp` with `qwen35` and `draft-mtp` support
License	Apache-2.0

TQ2_0  480
F32    359
Q2_K    17
Q5_K     6
Q6_K     3
Q4_K     1
-------------
total   866

TQ2_0 is the dominant target representation. Selected high-sensitivity and auxiliary tensors remain in K-quant or floating-point formats; the filename does not imply uniform tensor typing.

Execution

Ordinary target decode:

llama-cli \
  -m Ternary-Bonsai-27B-MTP-TQ2_0.gguf \
  -p "Derive the latency model for speculative decoding." \
  -n 256

Embedded MTP:

llama-cli \
  -m Ternary-Bonsai-27B-MTP-TQ2_0.gguf \
  --spec-type draft-mtp \
  --spec-draft-n-max 2 \
  -p "Derive the latency model for speculative decoding." \
  -n 256

There is deliberately no -md / --model-draft argument.

Speculative-decoding model

MTP is profitable only when accepted speculative work amortizes predictor and verification cost:

(Cdraft + Cverify) / E[committed draft tokens] < Ctarget-step

Acceptance rate is an intermediate statistic, not the objective. The relevant quantity is end-to-end committed-token throughput under an identical prompt distribution, sampler, context and device-placement policy.

A disciplined sweep is small:

baseline
draft-mtp, n_max = 1
draft-mtp, n_max = 2

Retain MTP only where the complete pipeline wins. Larger draft depth is not intrinsically better, and no hardware-independent token/s claim is made here.

Construction

This is a model-engineering artifact, not a new pretraining run and not an instruction fine-tune.

The final graph combines:

a pinned Ternary Bonsai 27B trunk;
a Qwen3.6-derived NextN/MTP donor;
a dedicated MTP embedding and output path;
a stock-compatible qwen35 GGUF layout;
a mixed-precision tensor policy centered on TQ2_0;
candidate selection gated by structural integrity, language-model quality and stock-runtime behavior.

The predictor was grafted from a compatible donor. It was not jointly trained with the final ternary trunk. The exact claim is therefore functional embedded speculation in one stock-compatible artifact, not co-trained target/predictor optimality.

Validation envelope

The release was qualified on properties that travel with the artifact:

Check	Result
Exact public byte length and SHA-256	pass
GGUF V3 parse	pass
`qwen35` architecture	pass
Exact 866-tensor inventory	pass
64 target blocks + 1 NextN block	pass
Ordinary generation from the released file	pass
`draft-mtp` generation from the same file	pass
Requested generation budget completed	pass
Stock-runtime compatibility matrix	6 / 6
Unmodified `llama.cpp` revisions exercised	2
Full public binary privacy scan	pass
Remote Hugging Face size and LFS SHA-256 match	pass

Stock compatibility points:

f955e394bf94e01e5e36186d13c985727e5ef5b5
ae735b13148db250911ce9d07663d1b95474e7b0

These are tested points, not a statement about every historical or future revision.

Provenance

Role	Immutable source	Revision	Source SHA-256
Ternary target trunk	`prism-ml/Ternary-Bonsai-27B-gguf`	`3f8cc399dde45ac0475d023634974407af34907c`	`f659ca3dd7e28ada5d8b5f3637862d0d51ef433bde032ec4c8990ed27c91a385`
MTP donor package	`lym00/Qwen3.6-27B-MTP-ONLY-GGUF`	`03b35db648da71e23ecfb239f70661e069b16054`	`97697fc5278d4bfd0afaa733ba28c4338cb0396b98d5f5acb781e515f72b52c0`

The donor package identifies Qwen3.6-derived MTP material but does not pin a separate underlying Qwen source revision. This card does not invent one.

See NOTICE for attribution and LICENSE for terms.

Public artifact boundary

Before publication, two path-valued quantization metadata fields were sanitized:

quantize.imatrix.file
quantize.imatrix.dataset

Only those metadata strings changed. Tensor payloads, dimensions, offsets, topology and file length did not.

The checksum below is the authoritative identity of the public artifact.

Technical boundaries

The predictor is grafted, not jointly trained with the final target.
Aggressive quantization can alter accuracy, calibration and long-context behavior.
Upstream Bonsai benchmark results are not asserted as measurements of this file.
MTP may improve, match or reduce throughput depending on backend and workload.
The repository contains a text-generation GGUF, not a multimodal projector.
The release is independent and is not endorsed by Prism ML, Qwen, Alibaba Cloud, lym00 or llama.cpp.

Verify

hf download vinpix/Ternary-Bonsai-27B-Stock-MTP-GGUF \
  Ternary-Bonsai-27B-MTP-TQ2_0.gguf \
  SHA256SUMS \
  --local-dir .

sha256sum -c SHA256SUMS

Expected digest:

480c695b0de5b78b720d7f500eb3e1650359d0ac8e18c29b231a14b2d9c9bf20

Citation

@misc{vinpix2026ternarybonsai27bmtp,
  title  = {Ternary Bonsai 27B with Embedded MTP: A One-File GGUF for llama.cpp},
  author = {vinpix},
  year   = {2026},
  url    = {https://huggingface.co/vinpix/Ternary-Bonsai-27B-Stock-MTP-GGUF}
}

Ternary target · Embedded predictor · One artifact boundary

Downloads last month: 641

GGUF

Model size

29B params

Architecture

qwen35

Hardware compatibility

2-bit

Model tree for vinpix/Ternary-Bonsai-27B-Stock-MTP-GGUF

Base model

lym00/Qwen3.6-27B-MTP-ONLY-GGUF

Quantized

(1)

this model

Ternary Bonsai 27B / Embedded MTP

One immutable GGUF. A ternary 27B-class target. An embedded Qwen-style NextN predictor. Stock llama.cpp.

Architecture

Artifact fingerprint

Execution

Speculative-decoding model

Construction

Validation envelope

Provenance

Public artifact boundary

Technical boundaries

Verify

Citation

Model tree for vinpix/Ternary-Bonsai-27B-Stock-MTP-GGUF

One immutable GGUF. A ternary 27B-class target. An embedded Qwen-style NextN predictor. Stock `llama.cpp`.