qwen3.5-4B-Code-GGUF

GGUF files for a Qwen3.5 4B coding-oriented release, published by reaperdoesntknow. The original export card states that conversion used Unsloth.

Available files

File	Role
Qwen3.5-4B.Q4_0.gguf	Q4_0 language-model quantization
Qwen3.5-4B.Q4_K_M.gguf	Q4_K_M language-model quantization
Qwen3.5-4B.Q8_0.gguf	Q8_0 language-model quantization
Qwen3.5-4B.BF16-mmproj.gguf	Multimodal projector; not a standalone language model

Choose one language-model file for your runtime. For image inputs, a compatible multimodal runtime also needs the matching projector. The quantization labels above follow the published filenames; this card does not report a tensor-level audit or measured quality comparison.

Usage

Download your selected file from the Files tab, then load it in a current GGUF runtime that explicitly supports Qwen3.5. For llama.cpp, consult the official usage documentation and multimodal documentation.

Select the desired filename explicitly rather than relying on automatic selection among the three quantizations. For multimodal use, pair it with Qwen3.5-4B.BF16-mmproj.gguf. The projector cannot replace the main model file.

The original card supplied llama-cli and llama-mtmd-cli examples using this repository ID and --jinja. Exact command support depends on the installed runtime revision. No inference smoke test or hardware-specific memory requirement is documented for these exports.

Provenance

The included config.json identifies Qwen3_5ForConditionalGeneration with text and vision components. Its text configuration specifies bfloat16; that does not describe the storage precision of the quantized GGUF files.

A related Transformers repository is Qwen3.5-4B-Coder. The original GGUF card does not identify an exact source checkpoint revision, conversion command, or quantization tool revision. An exact conversion relationship is therefore not asserted here.

The existing GGUF metadata does not declare a license or base model. Those fields have not been inferred or copied from a similarly named repository. Confirm applicable source-model terms before redistribution or commercial use.

Evaluation and limitations

The published files reviewed for this card do not include training-data documentation, a coding benchmark, a comparison between quantizations, or a safety evaluation. A coding-oriented name and successful conversion do not establish improved coding ability or tested multimodal performance.

Use for local experimentation and evaluation. Generated code may contain errors or vulnerabilities; review and test it before execution. Do not expose secrets or production systems to unreviewed generated code. Quantization can change outputs, so evaluate the exact file and runtime you intend to use.