File size: 2,225 Bytes
9a50a54 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 | ---
license: mit
library_name: transformers
tags:
- vector-quantization
- vq2
- moe
- qwen
- compression
- triton
- paged-streaming
pipeline_tag: text-generation
---
# ⚡ VQ2 & LEGION AI: Model Hub
Official repository for compiled binary containers of **VQ2** and **LEGION AI**.
* **GitHub Code & Docs**: [https://github.com/Unoalss/CompressAI](https://github.com/Unoalss/CompressAI)
* **Architecture**: Vector Quantization with Incoherence (RHT), Hessian LDLQ compensation, and Paged MoE Streaming.
---
## 📦 Hosted Models
### 1. 🏆 `qwen35b_mixed_b2.vq2m` (Production Candidate MoE - 10.11 GB)
* **Base Model**: `Qwen3.8-35B-A3B-Distill` (35.5B parameters, 40 layers, 256 experts per layer).
* **Compression**: 86.0% smaller than BF16 (67 GB -> 10.11 GB / 9.41 GiB).
* **Quality**: **0.9974 cosine similarity** and **90.6% top-1 agreement** vs dense BF16.
* **Memory**: Runs in **< 1.25 GB active VRAM** and **~3.0 GB RAM** via LEGION Paged Streaming.
* **SHA-256**: `2ADB36E37F387842A521D2F3E07A29B77A7D9240997411372E0975586AEB8EAE`
### 2. ⚡ `qwen3.5_2b_vq2_560mb.vq2r` (Ultra-compact 2B RDO - 567.9 MB)
* **Base Model**: `Qwen3.5-2B` (2 Billion parameters).
* **Compression**: 53.5% smaller than GGUF Q4_K_M (1221 MB -> 567 MB) and 84.2% smaller than BF16.
* **Memory**: Loads into GPU VRAM in **0.61 seconds**, occupying only **582 MiB** of active VRAM.
### 3. 🔑 `codebooks.pt` (4.47 MB)
* Shared 8-dimensional spherical codebooks for on-the-fly decompression.
---
## 📥 How to Download & Run
Clone the engine from GitHub:
```bash
git clone https://github.com/Unoalss/CompressAI.git
cd CompressAI
pip install -r requirements.txt
pip install -e .
```
Download the models:
```bash
# MoE 35B B2 Model (10 GB)
huggingface-cli download ArthurLss/CompressAI qwen35b_mixed_b2.vq2m --local-dir vq2_35b_out
# 2B Model (567 MB)
huggingface-cli download ArthurLss/CompressAI qwen3.5_2b_vq2_560mb.vq2r --local-dir VQ2_MARCO_FINAL
# Shared Codebooks (4.47 MB)
huggingface-cli download ArthurLss/CompressAI codebooks.pt --local-dir vq2_out
```
Launch the LEGION chatbot interface:
```bash
# Windows
INICIAR_LEGION.bat
# Linux / Mac
python legion/server.py
```
Open `http://127.0.0.1:7777` in your browser!
|