---
license: apache-2.0
base_model: Qwen/Qwen3-Coder-Next
tags:
- qwen
- qwen3
- coder
- cq
- cq_quant
- quantization
- 4bit
- vllm
- text-generation
pipeline_tag: text-generation
---

# Qwen3-Coder-Next-CQ-4bit

> **80B-parameter Mixture-of-Experts coding model · 3B activated parameters · Coded Quantization (CQ) 4-bit**
>
> **Note on Hugging Face's model-size display:** Hugging Face may report a smaller stored parameter count for this repository because CQ stores the weights in a packed 4-bit representation. The underlying model is **Qwen3-Coder-Next**, with approximately **80B total parameters and 3B activated parameters per token**. The smaller number reflects the packed storage tensors counted by the Hub, not the parameter count of the original model architecture.

**Qwen3-Coder-Next quantized to 4-bit with Coded Quantization (CQ).**

**ThousandML:** https://thousandml.com

CQ is a **calibration-free 4-bit quantization method** designed to preserve the behavior of the original model without requiring a calibration dataset. Because CQ does not optimize the quantization around a fixed calibration corpus, it is designed to generalize beyond calibration-specific data and workloads.

This repository contains **Qwen3-Coder-Next** quantized with **Coded Quantization (CQ) 4-bit**.

## Why CQ?

**Calibration-free.** CQ does not require a representative calibration dataset before quantization.

This avoids tying the quantization procedure to a particular calibration corpus and is intended to provide better generalization beyond the data that would otherwise have been used for calibration.

**4-bit inference.** Model weights are quantized to 4-bit for substantially lower weight memory requirements than BF16/FP16 deployment.

**Native vLLM integration.** CQ models run through the CQ vLLM quantization plugin and custom CUDA kernels included in the Thousandml CQ Docker image.

## Model

| | |
|---|---|
| **Base model** | `Qwen/Qwen3-Coder-Next` |
| **Original model parameters** | **~80B total / ~3B activated** |
| **Model type** | Mixture-of-Experts coding model |
| **Quantization** | **Coded Quantization (CQ) 4-bit** |
| **Calibration** | **None — calibration-free** |
| **Runtime** | vLLM + CQ custom kernels |
| **Quantization name** | `cq` |
| **Context length** | **262,144 tokens** |

## Run with Docker

The recommended way to run this model is with the CQ Docker image, which contains the compatible vLLM installation, CQ quantization plugin, and CQ CUDA kernels.

### 1. Make sure NVIDIA Docker support works

```bash
docker run --rm --gpus all \
  nvidia/cuda:13.0.2-base-ubuntu22.04 \
  nvidia-smi
```

Your GPU should appear in the output.

### 2. Start Qwen3-Coder-Next-CQ-4bit

```bash
docker run --rm --gpus all \
  -p 8000:8000 \
  docker.io/thousandml/cq:4bit-main \
  vllm serve thousandml/Qwen3-Coder-Next-CQ-4bit \
  --quantization cq
```

The model is then available through vLLM's OpenAI-compatible API on port `8000`.

### WSL2

If you are running Docker through WSL2 and vLLM reports that UVA is unavailable, enable vLLM's WSL2 pinned-memory support:

```bash
docker run --rm --gpus all \
  -e VLLM_WSL2_ENABLE_PIN_MEMORY=1 \
  -p 8000:8000 \
  docker.io/thousandml/cq:4bit-main \
  vllm serve thousandml/Qwen3-Coder-Next-CQ-4bit \
  --quantization cq
```

## Send a request

Once the server is ready:

```bash
curl http://localhost:8000/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "thousandml/Qwen3-Coder-Next-CQ-4bit",
    "messages": [
      {
        "role": "user",
        "content": "Write a Python function that performs binary search on a sorted list."
      }
    ]
  }'
```

## Tool calling

Qwen3-Coder-Next is designed for coding and agentic workflows and can be served with vLLM's automatic tool-choice support.

Example:

```bash
docker run --rm --gpus all \
  -p 8000:8000 \
  docker.io/thousandml/cq:4bit-main \
  vllm serve thousandml/Qwen3-Coder-Next-CQ-4bit \
  --quantization cq \
  --enable-auto-tool-choice \
  --tool-call-parser qwen3_coder
```

## Hugging Face authentication

For higher Hugging Face Hub rate limits, pass your Hugging Face token into the container:

```bash
docker run --rm --gpus all \
  -e HF_TOKEN="$HF_TOKEN" \
  -p 8000:8000 \
  docker.io/thousandml/cq:4bit-main \
  vllm serve thousandml/Qwen3-Coder-Next-CQ-4bit \
  --quantization cq
```

On WSL2, combine it with:

```bash
-e VLLM_WSL2_ENABLE_PIN_MEMORY=1
```

## About Coded Quantization (CQ)

Learn more at **https://thousandml.com**.

**Coded Quantization (CQ)** is a calibration-free quantization approach for efficient LLM inference.

Traditional post-training quantization methods can depend on a calibration dataset to estimate quantization parameters. The resulting quantization can therefore be influenced by the distribution and composition of that calibration data.

CQ removes that calibration-data requirement. The goal is to retain strong fidelity to the original model while allowing the quantized representation to generalize beyond any particular calibration corpus.

## Base model

This checkpoint is derived from **Qwen/Qwen3-Coder-Next**, a Mixture-of-Experts coding model designed for software development, agentic coding, tool use, and long-context workflows.

Please refer to the original Qwen model card for architecture details, capabilities, usage guidance, limitations, and upstream licensing information.

## License

The base model is released under the **Apache 2.0 License**. See the repository license and the original Qwen model card for applicable terms.