Edit Models filters

Model Tree

Apps

Docker Model Runner

Unsloth Desktop

Inference Providers

OVHcloud AI Endpoints

HF Inference API

One-click Deployment

Inference Endpoints

Microsoft Foundry

Amazon SageMaker AI

Misc

text-generation-inference

Eval Results (legacy)

text-embeddings-inference

4-bit precision

8-bit precision

Mixture of Experts

Carbon Emissions

Models

3,305

Base only

Active filters: quantization

zhangsq-nju/MobileLLM-350M-EdgeRazor-2.79bit

Text Generation • 0.4B • Updated Jun 11 • 60 • 1

zhangsq-nju/MobileLLM-350M-EdgeRazor-1.88bit

Text Generation • 0.4B • Updated Jun 11 • 17 • 1

zhangsq-nju/MobileLLM-350M-EdgeRazor-1.58bit

Text Generation • 0.4B • Updated Jun 11 • 59 • 1

zhangsq-nju/Qwen2.5-Omni-7B-EdgeRazor-4bit

Text Generation • 11B • Updated May 11 • 30 • 1

zhangsq-nju/Qwen3-0.6B-EdgeRazor-GGUF

Text Generation • 0.8B • Updated May 11 • 265 • 8

zhangsq-nju/Qwen3-1.7B-EdgeRazor-GGUF

Text Generation • 2B • Updated May 11 • 271 • 12

NICKO/Qwen2.5-3B-BonfyreFPQ-v12

NICKO/SmolLM2-1.7B-BonfyreFPQ-v12

NICKO/TinyLlama-1.1B-BonfyreFPQ-v12

Venugopalan2610/quant-olmoe-2bit

Text Generation • Updated Apr 18

AEON-7/Gemma-4-E4B-DECKARD-HERETIC-NVFP4

Text Generation • 6B • Updated Jun 21 • 6.12k • 1

navanchauhan/Huihui-Qwopus3.5-9B-v3-abliterated-TQ3_4S

Text Generation • 9B • Updated Apr 14 • 25

srijayadav/llama3-8b-qlora-alpaca

Text Generation • Updated Apr 13 • 5

KeinNiemand/Qwen3.5-122B-A10B-abliterix-IK_GGUF

Text Generation • 122B • Updated Apr 14 • 228 • 3

MLliu6/Qwen3-VL-2B-Instruct-AWQ-W4A16

Image-Text-to-Text • 2B • Updated Apr 17 • 70

dangvansam/chandra-ocr-2-FP8-dynamic

Image-Text-to-Text • 5B • Updated May 26 • 2.61k • 2

majentik/Voxtral-Mini-3B-2507-TurboQuant-MLX-8bit

Automatic Speech Recognition • 5B • Updated Jul 20 • 31

majentik/Voxtral-Mini-3B-2507-TurboQuant-MLX-4bit

Automatic Speech Recognition • 5B • Updated Jul 20 • 40

majentik/Voxtral-Mini-3B-2507-TurboQuant-MLX-2bit

Automatic Speech Recognition • 5B • Updated Jul 20 • 22

majentik/Voxtral-Mini-4B-Realtime-2602-TurboQuant-MLX-8bit

Automatic Speech Recognition • 4B • Updated Jul 20 • 22

majentik/Voxtral-Mini-4B-Realtime-2602-TurboQuant-MLX-4bit

Automatic Speech Recognition • 4B • Updated Jul 20 • 21

majentik/Voxtral-Mini-4B-Realtime-2602-TurboQuant-MLX-2bit

Automatic Speech Recognition • 4B • Updated Jul 20 • 23

majentik/Voxtral-4B-TTS-2603-TurboQuant-MLX-8bit

Text-to-Speech • Updated Jul 20 • 40

majentik/Voxtral-4B-TTS-2603-TurboQuant-MLX-4bit

Text-to-Speech • Updated Jul 20 • 33

majentik/Voxtral-4B-TTS-2603-TurboQuant-MLX-2bit

Text-to-Speech • Updated Jul 20 • 21

AsadIsmail/Qwen2.5-VL-7B-Instruct-ternary

Image-Text-to-Text • Updated Apr 27 • 1

AsadIsmail/gemma-4-E4B-it-ternary

Image-Text-to-Text • Updated Apr 27 • 1

AsadIsmail/SmolVLM2-2.2B-Instruct-ternary

Image-Text-to-Text • Updated Apr 15

AsadIsmail/whisper-large-v3-ternary

Automatic Speech Recognition • Updated Apr 27 • 1

YTan2000/Gemma4-26b-Super-Abliterated-TQ3_4S

Image-Text-to-Text • 25B • Updated Apr 14 • 332 • 8