fp8 quant for qwen3.5 models, nearly half memory decrease, speedup 30%, vllm serve can run
🤝 Open to Collab
HyperAI
Hyper-AI
AI & ML interests
lightvl is a lightweight Vision-Language Model (VLM) quantization toolkit supporting FP8, INT8, FP8-Block. It integrates with vLLM for high-throughput inference and supports Qwen3-VL, Qwen3.5, InternVL-Chat, and Gemma-4 models.
fast quant your model step by step:
1、 pip3 install lightvl
2、 lightvl YOUR_HF_MODEL_PATH
Recent Activity
liked a model 26 days ago
Hyper-AI/Qwen3.5-9B-fp8 updated a model about 2 months ago
Hyper-AI/Qwen3-VL-Embedding-8B-fp8 updated a model about 2 months ago
Hyper-AI/gemma-4-31B-it-fp8Organizations
None yet
models 5
Hyper-AI/Qwen3-VL-Embedding-8B-fp8
Feature Extraction • 8B • Updated • 2.27k • 2
Hyper-AI/gemma-4-31B-it-fp8
Image-Text-to-Text • 31B • Updated • 179 • 2
Hyper-AI/gemma-4-E4B-it-fp8
Any-to-Any • Updated • 470
Hyper-AI/Qwen3.5-9B-fp8
Image-Text-to-Text • 10B • Updated • 3.71k • 3
Hyper-AI/Qwen3-VL-2B-Instruct-fp8
Image-Text-to-Text • 2B • Updated • 5 • 1
datasets 0
None public yet