Holo4-27B-MTP-Repacked

Holo4-27B is H Company's 27B dense vision-language model for computer use, built on Qwen3.8-27B with a 262,144-token context. Paired with the hai-agents harness, it processes screenshots and tool results to return clicks, typing, code, and tool calls across web, desktop, and mobile environments. Available in FP8, NVFP4, and Q4 GGUF builds, it is the largest in the Holo4 family alongside Holo4-35B-A3B (Qwen3.6 MoE) and Holotron4-30B-A3B (NemotronH Nano Omni). While the full benchmark table and training pipeline are image-only, the model card notes strong gains over Qwen base models, reports results on Agentic Task Factory, and lists key scores: 85.2% on OSWorld ($0.08/task), 61.7% on OSWorld 2.0 ($1.22/task, roughly double the 35B-A3B's 30.9% at $0.61), and 45.4% on AutomationBench ($0.05/task vs. 34.5% at $0.02). A demo video shows it building an Eiffel Tower replica in FreeCAD, and all trajectories are published in the open Hcompany/trajectories dataset.

MTP — Speculative Decoding: Yes | GGUF — prithivMLmods/Holo4-27B-MTP-Repacked(GGUF)

Multi-Token Prediction (MTP) GGUF is a specialized GGUF model file format extension that integrates speculative decoding directly into the model weights to significantly accelerate local inference. Unlike traditional speculative decoding which requires a separate, smaller "draft" model, MTP GGUF files include additional output heads within the main model architecture that predict multiple future tokens in a single forward pass.

Model Files [GGUF]

File Name	Quant Type	File Size	File Link	Description
Holo4-27B-MTP.BF16.gguf	BF16	54.7 GB	Link	Full BF16 weights. Highest quality, largest file size.
Holo4-27B-MTP.Q3_K_L.gguf	Q3_K_L	14.6 GB	Link	Lower quality but usable, good for low RAM availability.
Holo4-27B-MTP.Q4_K_M.gguf	Q4_K_M	16.8 GB	Link	Good quality, default size for most use cases, recommended.
Holo4-27B-MTP.Q5_K_M.gguf	Q5_K_M	19.5 GB	Link	High quality, recommended.
Holo4-27B-MTP.Q8_0.gguf	Q8_0	29 GB	Link	Extremely high quality, generally unneeded but max available quant.
Holo4-27B-MTP.mmproj-bf16.gguf	mmproj-bf16	931 MB	Link	Multimodal projection file in BF16 format. Used for vision/video/language models.

MTP-Repacked [Transformers, BF16]

File Name	File Type	File Size	File Link	Description
chat_template.jinja	Jinja Template	8.95 kB	Link	Chat template for formatting conversations and prompts.
config.json	JSON	3.72 kB	Link	Core model architecture and configuration parameters.
generation_config.json	JSON	213 Bytes	Link	Default text generation parameters (e.g., temperature, top_p).
merges.txt	Text	3.35 MB	Link	Byte-Pair Encoding (BPE) merge rules for the tokenizer.
model-00001-of-00006.safetensors	Safetensors	10.9 GB	Link	Model weight shard 1/6. Required to load the model.
model-00002-of-00006.safetensors	Safetensors	11 GB	Link	Model weight shard 2/6. Required to load the model.
model-00003-of-00006.safetensors	Safetensors	10.9 GB	Link	Model weight shard 3/6. Required to load the model.
model-00004-of-00006.safetensors	Safetensors	10.8 GB	Link	Model weight shard 4/6. Required to load the model.
model-00005-of-00006.safetensors	Safetensors	8.54 GB	Link	Model weight shard 5/6. Required to load the model.
model-00006-of-00006.safetensors	Safetensors	3.39 GB	Link	Model weight shard 6/6. Required to load the model.
model.safetensors.index.json	JSON	112 kB	Link	Index file mapping model weights to their respective shards.
preprocessor_config.json	JSON	390 Bytes	Link	Configuration for preprocessing vision/image inputs.
tokenizer.json	JSON	12.8 MB	Link	Complete tokenizer vocabulary and encoding rules.
tokenizer_config.json	JSON	17.9 kB	Link	Tokenizer-specific configuration parameters.
video_preprocessor_config.json	JSON	385 Bytes	Link	Configuration for preprocessing video inputs.
vocab.json	JSON	6.72 MB	Link	Tokenizer vocabulary mapping tokens to IDs.

BF16 Test Run

Tested Holo4-27B-MTP.BF16.gguf (with Holo4-27B-MTP.mmproj-bf16.gguf) on an NVIDIA RTX PRO 6000 Blackwell Server Edition GPU with 96 GB of VRAM, using llama-cli with full GPU offloading and MTP speculative decoding.

Performance Metrics

Metric	Value
Average Prompt Processing Speed	1,056.22 tokens/s
Average Generation Speed	59.36 tokens/s
Total Wall Time	59.3 seconds
Context Length	16,384 tokens
KV Cache (K)	q8_0
GPU	NVIDIA RTX PRO 6000 Blackwell Server Edition
GPU VRAM Capacity	96 GB
Peak GPU VRAM Usage	52.17 GiB
Peak GPU Utilization	98%
GPU Layers Offloaded	99 (-ngl 99)
Random Seed	42
Speculative Decoding	MTP (draft_n_max=2)
Test Runs	3

Test Shots

Shot	Test
Shot 1	Vision — Image 1
Shot 2	Vision — Image 2
Shot 3	Vision — Image 3

Run Inference

./build/bin/llama-cli \
  -m /path/to/models/Holo4-27B-MTP.BF16.gguf \
  --mmproj /path/to/models/Holo4-27B-MTP.mmproj-bf16.gguf \
  --image /path/to/image.jpeg \
  --spec-type draft-mtp \
  --spec-draft-n-max 2 \
  -ngl 99 \
  -c 16384 \
  --cache-type-k q8_0 \
  --seed 42 \
  -n 8192 \
  -st \
  -p "OCR and detect the paragraphs."

Performance results reflect this specific hardware and inference configuration. Actual throughput may vary depending on the prompt, model, software build, and runtime settings.

Snapshot(3)

llama.cpp

LLM inference in C/C++ — https://github.com/ggml-org/llama.cpp

MTP-repackaged and quantized using llama.cpp by — prithivMLmods.

This model follows the same license and terms as the base model, Hcompany/Holo4-27B.