⚡ VQ2 & LEGION AI: Model Hub
Official repository for compiled binary containers of VQ2 and LEGION AI.
- GitHub Code & Docs: https://github.com/Unoalss/CompressAI
- Architecture: Vector Quantization with Incoherence (RHT), Hessian LDLQ compensation, and Paged MoE Streaming.
📦 Hosted Models
1. 🏆 qwen35b_mixed_b2.vq2m (Production Candidate MoE - 10.11 GB)
- Base Model:
Qwen3.8-35B-A3B-Distill(35.5B parameters, 40 layers, 256 experts per layer). - Compression: 86.0% smaller than BF16 (67 GB -> 10.11 GB / 9.41 GiB).
- Quality: 0.9974 cosine similarity and 90.6% top-1 agreement vs dense BF16.
- Memory: Runs in < 1.25 GB active VRAM and ~3.0 GB RAM via LEGION Paged Streaming.
- SHA-256:
2ADB36E37F387842A521D2F3E07A29B77A7D9240997411372E0975586AEB8EAE
2. ⚡ qwen3.5_2b_vq2_560mb.vq2r (Ultra-compact 2B RDO - 567.9 MB)
- Base Model:
Qwen3.5-2B(2 Billion parameters). - Compression: 53.5% smaller than GGUF Q4_K_M (1221 MB -> 567 MB) and 84.2% smaller than BF16.
- Memory: Loads into GPU VRAM in 0.61 seconds, occupying only 582 MiB of active VRAM.
3. 🔑 codebooks.pt (4.47 MB)
- Shared 8-dimensional spherical codebooks for on-the-fly decompression.
📥 How to Download & Run
Clone the engine from GitHub:
git clone https://github.com/Unoalss/CompressAI.git
cd CompressAI
pip install -r requirements.txt
pip install -e .
Download the models:
# MoE 35B B2 Model (10 GB)
huggingface-cli download ArthurLss/CompressAI qwen35b_mixed_b2.vq2m --local-dir vq2_35b_out
# 2B Model (567 MB)
huggingface-cli download ArthurLss/CompressAI qwen3.5_2b_vq2_560mb.vq2r --local-dir VQ2_MARCO_FINAL
# Shared Codebooks (4.47 MB)
huggingface-cli download ArthurLss/CompressAI codebooks.pt --local-dir vq2_out
Launch the LEGION chatbot interface:
# Windows
INICIAR_LEGION.bat
# Linux / Mac
python legion/server.py
Open http://127.0.0.1:7777 in your browser!