---
license: mit
library_name: transformers
tags:
- vector-quantization
- vq2
- moe
- qwen
- compression
- triton
- paged-streaming
pipeline_tag: text-generation
---

# ⚡ VQ2 & LEGION AI: Model Hub

Official repository for compiled binary containers of **VQ2** and **LEGION AI**.

* **GitHub Code & Docs**: [https://github.com/Unoalss/CompressAI](https://github.com/Unoalss/CompressAI)
* **Architecture**: Vector Quantization with Incoherence (RHT), Hessian LDLQ compensation, and Paged MoE Streaming.

---

## 📦 Hosted Models

### 1. 🏆 `qwen35b_mixed_b2.vq2m` (Production Candidate MoE - 10.11 GB)
* **Base Model**: `Qwen3.8-35B-A3B-Distill` (35.5B parameters, 40 layers, 256 experts per layer).
* **Compression**: 86.0% smaller than BF16 (67 GB -> 10.11 GB / 9.41 GiB).
* **Quality**: **0.9974 cosine similarity** and **90.6% top-1 agreement** vs dense BF16.
* **Memory**: Runs in **< 1.25 GB active VRAM** and **~3.0 GB RAM** via LEGION Paged Streaming.
* **SHA-256**: `2ADB36E37F387842A521D2F3E07A29B77A7D9240997411372E0975586AEB8EAE`

### 2. ⚡ `qwen3.5_2b_vq2_560mb.vq2r` (Ultra-compact 2B RDO - 567.9 MB)
* **Base Model**: `Qwen3.5-2B` (2 Billion parameters).
* **Compression**: 53.5% smaller than GGUF Q4_K_M (1221 MB -> 567 MB) and 84.2% smaller than BF16.
* **Memory**: Loads into GPU VRAM in **0.61 seconds**, occupying only **582 MiB** of active VRAM.

### 3. 🔑 `codebooks.pt` (4.47 MB)
* Shared 8-dimensional spherical codebooks for on-the-fly decompression.

---

## 📥 How to Download & Run

Clone the engine from GitHub:
```bash
git clone https://github.com/Unoalss/CompressAI.git
cd CompressAI
pip install -r requirements.txt
pip install -e .
```

Download the models:
```bash
# MoE 35B B2 Model (10 GB)
huggingface-cli download ArthurLss/CompressAI qwen35b_mixed_b2.vq2m --local-dir vq2_35b_out

# 2B Model (567 MB)
huggingface-cli download ArthurLss/CompressAI qwen3.5_2b_vq2_560mb.vq2r --local-dir VQ2_MARCO_FINAL

# Shared Codebooks (4.47 MB)
huggingface-cli download ArthurLss/CompressAI codebooks.pt --local-dir vq2_out
```

Launch the LEGION chatbot interface:
```bash
# Windows
INICIAR_LEGION.bat

# Linux / Mac
python legion/server.py
```
Open `http://127.0.0.1:7777` in your browser!