husamelfadil commited on
Commit
bee4f79
·
verified ·
1 Parent(s): 15328d0

Update CQ branding to Coded Quantization and ThousandML

Browse files
Files changed (1) hide show
  1. README.md +7 -7
README.md CHANGED
@@ -16,17 +16,17 @@ pipeline_tag: text-generation
16
 
17
  # Qwen3-Coder-Next-CQ-4bit
18
 
19
- > **80B-parameter Mixture-of-Experts coding model · 3B activated parameters · Thousandml Quant (CQ) 4-bit**
20
  >
21
  > **Note on Hugging Face's model-size display:** Hugging Face may report a smaller stored parameter count for this repository because CQ stores the weights in a packed 4-bit representation. The underlying model is **Qwen3-Coder-Next**, with approximately **80B total parameters and 3B activated parameters per token**. The smaller number reflects the packed storage tensors counted by the Hub, not the parameter count of the original model architecture.
22
 
23
- **Qwen3-Coder-Next quantized to 4-bit with Thousandml Quant (CQ).**
24
 
25
- **Thousandml Quant:** https://thousandml.com
26
 
27
  CQ is a **calibration-free 4-bit quantization method** designed to preserve the behavior of the original model without requiring a calibration dataset. Because CQ does not optimize the quantization around a fixed calibration corpus, it is designed to generalize beyond calibration-specific data and workloads.
28
 
29
- This repository contains **Qwen3-Coder-Next** quantized with **Thousandml Quant (CQ) 4-bit**.
30
 
31
  ## Why CQ?
32
 
@@ -45,7 +45,7 @@ This avoids tying the quantization procedure to a particular calibration corpus
45
  | **Base model** | `Qwen/Qwen3-Coder-Next` |
46
  | **Original model parameters** | **~80B total / ~3B activated** |
47
  | **Model type** | Mixture-of-Experts coding model |
48
- | **Quantization** | **Thousandml Quant (CQ) 4-bit** |
49
  | **Calibration** | **None — calibration-free** |
50
  | **Runtime** | vLLM + CQ custom kernels |
51
  | **Quantization name** | `cq` |
@@ -143,11 +143,11 @@ On WSL2, combine it with:
143
  -e VLLM_WSL2_ENABLE_PIN_MEMORY=1
144
  ```
145
 
146
- ## About Thousandml Quant (CQ)
147
 
148
  Learn more at **https://thousandml.com**.
149
 
150
- **Thousandml Quant (CQ)** is a calibration-free quantization approach for efficient LLM inference.
151
 
152
  Traditional post-training quantization methods can depend on a calibration dataset to estimate quantization parameters. The resulting quantization can therefore be influenced by the distribution and composition of that calibration data.
153
 
 
16
 
17
  # Qwen3-Coder-Next-CQ-4bit
18
 
19
+ > **80B-parameter Mixture-of-Experts coding model · 3B activated parameters · Coded Quantization (CQ) 4-bit**
20
  >
21
  > **Note on Hugging Face's model-size display:** Hugging Face may report a smaller stored parameter count for this repository because CQ stores the weights in a packed 4-bit representation. The underlying model is **Qwen3-Coder-Next**, with approximately **80B total parameters and 3B activated parameters per token**. The smaller number reflects the packed storage tensors counted by the Hub, not the parameter count of the original model architecture.
22
 
23
+ **Qwen3-Coder-Next quantized to 4-bit with Coded Quantization (CQ).**
24
 
25
+ **ThousandML:** https://thousandml.com
26
 
27
  CQ is a **calibration-free 4-bit quantization method** designed to preserve the behavior of the original model without requiring a calibration dataset. Because CQ does not optimize the quantization around a fixed calibration corpus, it is designed to generalize beyond calibration-specific data and workloads.
28
 
29
+ This repository contains **Qwen3-Coder-Next** quantized with **Coded Quantization (CQ) 4-bit**.
30
 
31
  ## Why CQ?
32
 
 
45
  | **Base model** | `Qwen/Qwen3-Coder-Next` |
46
  | **Original model parameters** | **~80B total / ~3B activated** |
47
  | **Model type** | Mixture-of-Experts coding model |
48
+ | **Quantization** | **Coded Quantization (CQ) 4-bit** |
49
  | **Calibration** | **None — calibration-free** |
50
  | **Runtime** | vLLM + CQ custom kernels |
51
  | **Quantization name** | `cq` |
 
143
  -e VLLM_WSL2_ENABLE_PIN_MEMORY=1
144
  ```
145
 
146
+ ## About Coded Quantization (CQ)
147
 
148
  Learn more at **https://thousandml.com**.
149
 
150
+ **Coded Quantization (CQ)** is a calibration-free quantization approach for efficient LLM inference.
151
 
152
  Traditional post-training quantization methods can depend on a calibration dataset to estimate quantization parameters. The resulting quantization can therefore be influenced by the distribution and composition of that calibration data.
153