Xylolabs/GLM-4.7-Flash-GGUF

GLM-4.7-Flash GGUF references

gguf · llama-cpp · quantized · glm4

Two preserved GGUF quantizations of GLM-4.7-Flash, converted and quantized by Xylolabs with llama.cpp. Choose plain IQ4_XS or IQ4_XS with the existing importance matrix. Both retain the main decoder and MTP weights; generation checks use the main decoder.

This export needs about 16.87 GB of device memory for its largest GGUF weight choice alone, plus room for the KV cache and buffers.

Formats

Each alternative contains 868 tensors from 47 main text layers and one MTP module, plus the tokenizer and original chat template. Average bits include embedding, output head, router, MTP and float32 parameters.

File	Download GB	Average bits/weight
GLM-4.7-Flash-IQ4_XS-imat.gguf	16.713	4.280
GLM-4.7-Flash-IQ4_XS.gguf	16.876	4.322

Actual mixed tensor types:

Choice	Tensor types and counts
IQ4_XS-imat	F32: 290, IQ4_NL: 48, IQ4_XS: 529, Q6_K: 1
IQ4_XS	F32: 290, IQ4_NL: 48, IQ4_XS: 518, Q5_K: 11, Q6_K: 1

The imatrix file retains the existing importance-matrix result. Historical relative calibration filenames are metadata labels; datasets are not included. No new training, calibration or quantization was performed for publication. An MTP drafter requires its own compatible implementation and evaluation.

Quickstart

With a llama.cpp build, download one file and run:

llama-cli -m GLM-4.7-Flash-IQ4_XS-imat.gguf -c 2048 -n 32 --temp 0 --jinja \
  --chat-template-kwargs '{"enable_thinking":false}' \
  --single-turn -p "Name the capital of France. Reply with only the city."

This command is checked with llama.cpp v0.9.0. Select one alternative. The example explicitly limits context to 2,048 tokens; original context metadata is unchanged. KV and working buffers need additional memory.

Evaluation

Both alternatives pass the relative reference check with zero counted losses or broken answers. The stored BF16 pass answers 31/33 plus 7/7; current Q4_K_M answers 32/33 plus 7/7.

Choice	Mean KL	KL / Q4_K_M	Right	Counted lost/broken
IQ4_XS-imat	0.089672	1.133×	33/33 + 7/7	0
IQ4_XS	0.092851	1.173×	32/33 + 7/7	0

The 16 matched 1,024-token sequences cover 16,288 scored positions across English/Korean plain/chat groups. KL is measured against the original stored BF16 pass. Current Q4_K_M gives 0.079141. Both ratios are below 1.5×; the answer gate permits at most one more counted lost/broken answer than Q4_K_M. Every original sequence ID, prompt token sequence, stop ID and 33+7 generation budget is verified. Current runs use context 8,192 and the same reference contracts.

Counted losses exclude BF16 near ties (top-two gap below 1.0 at the first divergent token) and reasoning items recovered with thinking on. Broken answers count where BF16 is correct. Raw short-prompt correctness remains separate from these exclusions.

The exact local imatrix CLI example loads the released file and completes one query. All MTP weights are retained, while generation checks use the main decoder. These checks concern stored texts and short prompts; an integration needs its own task evaluation. Speed is not reported here.

License

MIT, as declared by the original GLM-4.7-Flash card. The full MIT terms are retained in LICENSE; NOTICE identifies the GLM Team / Z.ai origin and our conversion/quantization. Provided as is, without warranty or upstream endorsement.

Citation

Cite the original GLM-4.7-Flash model card and its linked GLM technical report.