Xylolabs/GLM-4.6V-Flash-Text-GGUF

GLM-4.6V-Flash text GGUF reference

gguf · llama-cpp · quantized · glm4 · text-only

A preserved text-decoder GGUF quantization of GLM-4.6V-Flash, converted and quantized by Xylolabs with llama.cpp using the existing importance matrix. The vision encoder and projector are omitted; the evaluated path accepts text.

This export needs about 5.25 GB of device memory for its GGUF text-decoder weights alone, plus room for the KV cache and buffers.

Formats

The file contains 523 tensors from the 40 text-decoder layers, embedding and output head, plus the tokenizer and original chat template. It does not include image-encoder or projector weights.

File	Download GB	Average bits/weight
GLM-4.6V-Flash-Text-IQ4_XS-imat.gguf	5.262	4.470

Actual mixed tensor types: F32: 281, IQ4_NL: 40, IQ4_XS: 161, Q5_K: 40, Q6_K: 1. Average bits count every stored tensor, including embedding, output head, biases and float32 parameters.

The file preserves the existing importance-matrix result. Historical relative calibration filenames are metadata labels; no dataset is included. Original GGUF metadata includes the upstream image-text task tag; this derivative contains only the text-decoder weights. No new training, calibration or quantization was performed for publication.

Quickstart

With a llama.cpp build, download the file and run:

llama-cli -m GLM-4.6V-Flash-Text-IQ4_XS-imat.gguf -c 2048 -n 32 --temp 0 --jinja \
  --chat-template-kwargs '{"enable_thinking":false}' \
  --single-turn -p "Name the capital of France. Reply with only the city."

This command is checked with llama.cpp v0.9.0. The example limits context to 2,048 tokens; original context metadata is unchanged. KV and working buffers need additional memory.

Evaluation

The file passes the relative reference check with one counted lost answer and no broken answers. Stored BF16 answers 26/33 plus 6/7; current Q4_K_M answers 27/33 plus 7/7; this IQ4 file answers 25/33 plus 7/7. The counted loss is ko-fact-hangul. The strict zero-loss check fails; the relative gate permits one more counted loss/broken answer than Q4_K_M, which has zero.

Mean KL against the original stored BF16 pass is 0.029230, compared with current Q4_K_M's 0.024092: 1.213×, below the 1.5× bar. Sixteen matched 1,024-token sequences cover 16,248 scored positions across English/Korean plain/chat groups. Every original sequence ID, prompt token sequence, stop ID and 33+7 generation budget matches. Current checks use context 8,192.

Counted losses exclude BF16 near ties (top-two gap below 1.0 at the first divergent token) and reasoning items recovered with thinking on. Broken answers count where BF16 is correct. No loss is pardoned for this file. Raw correctness remains separate from these exclusions.

The exact local CLI loads this file and completes one text query. These checks cover stored texts and short prompts; an integration needs its own task evaluation. Speed is not reported here.

License

MIT, as declared by the original GLM-4.6V-Flash card. Full MIT terms are in LICENSE; NOTICE identifies the GLM Team / Z.ai origin and this conversion/quantization. Provided as is, without warranty or upstream endorsement.

Citation

The original model card recommends the GLM-V family paper:

@misc{glm_v_2025,
  title={GLM-4.5V and GLM-4.1V-Thinking: Towards Versatile Multimodal Reasoning with Scalable Reinforcement Learning},
  author={V Team and others},
  year={2025},
  eprint={2507.01006},
  archivePrefix={arXiv},
  url={https://arxiv.org/abs/2507.01006}
}