Instructions to use Abiray/SenseNova-6.8-Code2Video-MTP-GGUF with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Notebooks
Google Colab
Kaggle
Local Apps Settings

How to use Abiray/SenseNova-6.8-Code2Video-MTP-GGUF with llama.cpp:

Install (macOS, Linux)

curl -LsSf https://llama.app/install.sh | sh
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf Abiray/SenseNova-6.8-Code2Video-MTP-GGUF:Q4_K_M
# Run inference directly in the terminal:
llama cli -hf Abiray/SenseNova-6.8-Code2Video-MTP-GGUF:Q4_K_M

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf Abiray/SenseNova-6.8-Code2Video-MTP-GGUF:Q4_K_M
# Run inference directly in the terminal:
llama cli -hf Abiray/SenseNova-6.8-Code2Video-MTP-GGUF:Q4_K_M

Use pre-built binary

# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases
# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf Abiray/SenseNova-6.8-Code2Video-MTP-GGUF:Q4_K_M
# Run inference directly in the terminal:
./llama-cli -hf Abiray/SenseNova-6.8-Code2Video-MTP-GGUF:Q4_K_M

Build from source code

git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf Abiray/SenseNova-6.8-Code2Video-MTP-GGUF:Q4_K_M
# Run inference directly in the terminal:
./build/bin/llama-cli -hf Abiray/SenseNova-6.8-Code2Video-MTP-GGUF:Q4_K_M

Use Docker

docker model run hf.co/Abiray/SenseNova-6.8-Code2Video-MTP-GGUF:Q4_K_M

Llama new
LM Studio
Jan

vLLM

How to use Abiray/SenseNova-6.8-Code2Video-MTP-GGUF with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "Abiray/SenseNova-6.8-Code2Video-MTP-GGUF"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Abiray/SenseNova-6.8-Code2Video-MTP-GGUF",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker

docker model run hf.co/Abiray/SenseNova-6.8-Code2Video-MTP-GGUF:Q4_K_M

Ollama
How to use Abiray/SenseNova-6.8-Code2Video-MTP-GGUF with Ollama:
```
ollama run hf.co/Abiray/SenseNova-6.8-Code2Video-MTP-GGUF:Q4_K_M
```
Unsloth Desktop

How to use Abiray/SenseNova-6.8-Code2Video-MTP-GGUF with Pi:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf Abiray/SenseNova-6.8-Code2Video-MTP-GGUF:Q4_K_M

Configure the model in Pi

# Install Pi:
npm install -g @earendil-works/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "llama-cpp": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "Abiray/SenseNova-6.8-Code2Video-MTP-GGUF:Q4_K_M"
        }
      ]
    }
  }
}

Run Pi

# Start Pi in your project directory:
pi

Docker Model Runner
How to use Abiray/SenseNova-6.8-Code2Video-MTP-GGUF with Docker Model Runner:
```
docker model run hf.co/Abiray/SenseNova-6.8-Code2Video-MTP-GGUF:Q4_K_M
```

Lemonade

How to use Abiray/SenseNova-6.8-Code2Video-MTP-GGUF with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull Abiray/SenseNova-6.8-Code2Video-MTP-GGUF:Q4_K_M

Run and chat with the model

lemonade run user.SenseNova-6.8-Code2Video-MTP-GGUF-Q4_K_M

List all available models

lemonade list

Hermes Agent

How to use Abiray/SenseNova-6.8-Code2Video-MTP-GGUF with Hermes Agent:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf Abiray/SenseNova-6.8-Code2Video-MTP-GGUF:Q4_K_M

Configure Hermes

# Install Hermes:
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
# Point Hermes at the local server:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default Abiray/SenseNova-6.8-Code2Video-MTP-GGUF:Q4_K_M

Run Hermes

hermes

Atomic Chat

OpenClaw

How to use Abiray/SenseNova-6.8-Code2Video-MTP-GGUF with OpenClaw:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf Abiray/SenseNova-6.8-Code2Video-MTP-GGUF:Q4_K_M

Configure OpenClaw

# Install OpenClaw:
npm install -g openclaw@latest
# Register the local server and set it as the default model:
openclaw onboard --non-interactive --mode local \
  --auth-choice custom-api-key \
  --custom-base-url http://127.0.0.1:8080/v1 \
  --custom-model-id "Abiray/SenseNova-6.8-Code2Video-MTP-GGUF:Q4_K_M" \
  --custom-provider-id llama-cpp \
  --custom-compatibility openai \
  --custom-text-input \
  --accept-risk \
  --skip-health

Run OpenClaw

openclaw agent --local --agent main --message "Hello from Hugging Face"

SenseNova-6.8-Code2Video — GGUF

GGUF quantizations of sensenova/SenseNova-6.8-Code2Video (27B hybrid qwen35 architecture), complete with the F16 Multimodal Vision Projector (mmproj-F16.gguf) and a grafted 1-layer Multi-Token Prediction (blk.64 / NextN) speculative decoding head preserved at Q8_0.

Embedded Q8_0 Multi-Token Prediction (MTP) Head (blk.64.*):
Hybrid Gated DeltaNet (ssm_*) & Attention Gate (attn_gate) Protection:
100% Unique Multi-Domain imatrix Calibration:
- Calibrated over 20 chunks (n_ctx=512) of 100% unique, non-repeating multi-domain data covering:
  - Bilingual (EN/ZH) Claude Code agent tool-calling trajectories & multimodal keyframe critiques
  - Remotion / React TypeScript multi-scene kinetic typography & data visualization code
  - Python procedural audio synthesis (numpy / scipy FM bells, Karplus-Strong pizzicato, Schroeder stereo reverb, FFmpeg filtergraphs)
  - Manim 3D differential geometry & Three.js custom GLSL shaders
Full Multimodal Vision Support (mmproj-F16.gguf):

📦 Provided Files, Sizes & Verified Perplexity (PPL)

Filename	Quant Type	File Size (HF / GiB)	Base `ffn_gate`	Protected `Q8_0` / `Q6_K` / `F32` Tensors	4-Chunk PPL (`c=512`)	Recommended VRAM / Use Case
`SenseNova-6.8-Code2Video-mmproj-F16.gguf`	`F16` / `F32`	928 MB (0.86 GiB)	`F16`	`F16: 112`, `F32: 222` (334 total)	—	Required for Vision / Keyframe Critique
`SenseNova-6.8-Code2Video-IQ3_M-MTP.gguf`	`IQ3_M` (imatrix)	15.1 GB (14.05 GiB)	`IQ3_S`	`Q8_0: 152`, `Q6_K: 50`, `Q4_K: 88`, `F32: 360`	`4.3738 ± 0.305` (+2.93%)	Best for 16 GB VRAM (with context + `mmproj`)
`SenseNova-6.8-Code2Video-Q3_K_M-MTP.gguf`	`Q3_K_M` (imatrix)	15.8 GB (14.72 GiB)	`Q3_K`	`Q8_0: 152`, `Q6_K: 50`, `Q4_K: 138`, `F32: 360`	—	Fast 3-bit K-quant for 16 GB VRAM
`SenseNova-6.8-Code2Video-IQ4_XS-MTP.gguf`	`IQ4_XS` (imatrix)	17.2 GB (15.97 GiB)	`IQ4_XS`	`Q8_0: 152`, `Q6_K: 50`, `Q5_K: 64`, `F32: 360`	`4.2311 ± 0.298` (Matches `Q8_0`!)	Sweet-Spot 4-bit I-Quant (16–20 GB VRAM)
`SenseNova-6.8-Code2Video-Q4_K_S-MTP.gguf`	`Q4_K_S` (imatrix)	17.5 GB (16.32 GiB)	`Q4_K`	`Q8_0: 152`, `Q6_K: 50`, `Q5_K: 12`, `F32: 360`	—	Compact 4-bit K-quant
`SenseNova-6.8-Code2Video-Q4_K_M-MTP.gguf`	`Q4_K_M` (imatrix)	18.5 GB (17.21 GiB)	`Q4_K`	`Q8_0: 152`, `Q6_K: 114`, `F32: 360`	`4.2709 ± 0.302` (+0.51%)	Default Daily Driver (20–24 GB VRAM)
`SenseNova-6.8-Code2Video-Q5_K_M-MTP.gguf`	`Q5_K_M` (imatrix)	20.6 GB (19.21 GiB)	`Q5_K`	`Q8_0: 152`, `Q6_K: 114`, `F32: 360`	—	High-precision 24 GB VRAM quant
`SenseNova-6.8-Code2Video-Q6_K-MTP.gguf`	`Q6_K` (imatrix)	22.9 GB (21.33 GiB)	`Q6_K`	`Q8_0: 152`, `Q6_K: 354`, `F32: 360`	—	Near-lossless 24–32 GB VRAM quant
`SenseNova-6.8-Code2Video-Q8_0-MTP.gguf`	`Q8_0`	29.0 GB (27.05 GiB)	`Q8_0`	`Q8_0: 506`, `F32: 360`	`4.2491 ± 0.300` (Baseline)	Archival / Reference 8-bit precision (32 GB+ RAM/VRAM)

💡 Which file should I pick?

16 GB VRAM GPUs (RTX 4080 / 4070 Ti Super / 5070 Ti): Grab IQ3_M-MTP.gguf (15.1 GB) if running alongside mmproj-F16.gguf, or IQ4_XS-MTP.gguf (17.2 GB) for text/code generation with partial offload.

24 GB VRAM GPUs (RTX 3090 / 4090 / 5090): Grab Q4_K_M-MTP.gguf (18.5 GB) or Q5_K_M-MTP.gguf (20.6 GB) + mmproj-F16.gguf (928 MB) for full GPU offload with long context.

32 GB+ Unified Memory / Workstation GPUs: Grab Q6_K-MTP.gguf (22.9 GB) or Q8_0-MTP.gguf (29.0 GB).

🚀 Quickstart & Usage (`llama.cpp`)

Make sure your llama.cpp build is up to date (supports the qwen35 architecture and llama-mtmd-cli).

1. Fast Download with `huggingface-cli`

# Download the recommended Q4_K_M quant + Vision Projector
hf download <your-username>/SenseNova-6.8-Code2Video-GGUF \
  SenseNova-6.8-Code2Video-Q4_K_M-MTP.gguf \
  SenseNova-6.8-Code2Video-mmproj-F16.gguf \
  --local-dir ./models

2. Text & Code2Video Generation (`llama-cli`)

./llama.cpp/build/bin/llama-cli \
  -m ./models/SenseNova-6.8-Code2Video-Q4_K_M-MTP.gguf \
  -ngl 99 \
  -c 32768 \
  --temp 0.6 \
  --top-k 20 \
  --top-p 0.95 \
  -p "Write a complete Python script using numpy and scipy to synthesize a stereo Karplus-Strong plucked string chord progression and save it to chord.wav."

3. Multimodal Vision & Keyframe Critique (`llama-mtmd-cli`)

./llama.cpp/build/bin/llama-mtmd-cli \
  -m ./models/SenseNova-6.8-Code2Video-Q4_K_M-MTP.gguf \
  --mmproj ./models/SenseNova-6.8-Code2Video-mmproj-F16.gguf \
  -ngl 99 \
  --image ./rendered_keyframe_0120.png \
  -p "Inspect this rendered animation keyframe for typography overlap, contrast issues, or layout clipping, and provide the exact CSS/Canvas fix."

4. OpenAI-Compatible Multimodal Server (`llama-server` for Claude Code / Agent Harnesses)

./llama.cpp/build/bin/llama-server \
  -m ./models/SenseNova-6.8-Code2Video-Q4_K_M-MTP.gguf \
  --mmproj ./models/SenseNova-6.8-Code2Video-mmproj-F16.gguf \
  -ngl 99 \
  -c 65536 \
  --port 8080 \
  --jinja

🔬 Architecture & Metadata Summary

Architecture: qwen35 (Qwen3_5ForConditionalGeneration)
Total Blocks: 65 (64 hybrid backbone layers + 1 grafted blk.64 NextN/MTP speculative prediction layer)
Recurrent vs. Full Attention Pattern: Every 4th layer (blk.3, 7, 11, ..., 63) is Full Grouped-Query Attention (24 Q heads / 4 KV heads); the remaining 48 layers are Gated DeltaNet recurrent state-space layers (ssm.inner_size = 6144, ssm.state_size = 128, ssm.group_count = 16, ssm.time_step_rank = 48).
Context Length: 262,144 tokens (YaRN RoPE scaling, rope.freq_base = 10,000,000)
Vocabulary Size: 248,320 tokens

📜 License & Credits

Original Model: sensenova/SenseNova-6.8-Code2Video by SenseTime Research (apache-2.0).
MTP Donor Weights: Qwen/Qwen3.8-27B (apache-2.0).

Downloads last month: -

GGUF

Model size

27B params

Architecture

qwen35

Hardware compatibility

3-bit

4-bit

5-bit

6-bit

8-bit

Model tree for Abiray/SenseNova-6.8-Code2Video-MTP-GGUF

Base model

sensenova/SenseNova-6.8-Code2Video

Quantized

(2)

this model

SenseNova-6.8-Code2Video — GGUF

📦 Provided Files, Sizes & Verified Perplexity (PPL)

🚀 Quickstart & Usage (llama.cpp)

1. Fast Download with huggingface-cli

2. Text & Code2Video Generation (llama-cli)

3. Multimodal Vision & Keyframe Critique (llama-mtmd-cli)

4. OpenAI-Compatible Multimodal Server (llama-server for Claude Code / Agent Harnesses)

🔬 Architecture & Metadata Summary

📜 License & Credits

Model tree for Abiray/SenseNova-6.8-Code2Video-MTP-GGUF

🚀 Quickstart & Usage (`llama.cpp`)

1. Fast Download with `huggingface-cli`

2. Text & Code2Video Generation (`llama-cli`)

3. Multimodal Vision & Keyframe Critique (`llama-mtmd-cli`)

4. OpenAI-Compatible Multimodal Server (`llama-server` for Claude Code / Agent Harnesses)