Edit Models filters

Model Tree

Apps

Docker Model Runner

Unsloth Desktop

Inference Providers

OVHcloud AI Endpoints

HF Inference API

One-click Deployment

Inference Endpoints

Microsoft Foundry

Amazon SageMaker AI

Misc

text-generation-inference

Eval Results (legacy)

text-embeddings-inference

4-bit precision

8-bit precision

Mixture of Experts

Carbon Emissions

Models

44

Base only

Active filters: smoothquant

bot-lab-21/GLM-5.3-EXL3-3.2bpw-Pollard-Smooth

Text Generation • 157B • Updated 28 days ago • 270 • 2

adriabama06/DeepCoder-1.5B-Preview-FP8-W8A8

Text Generation • Updated Apr 13, 2025 • 26 • 1

neonconverse/gemma-3-27b-abliterated-w8a8-8bit

Updated Oct 8, 2025

ArslanRobo/llama-3.1-8b-instruct-smoothquant-Pruned30Tailor-fp16

Text Generation • 6B • Updated Dec 16, 2025 • 19

JEILDLWLRMA/Qwen3VL-8B-Instruct-FP8

Image-to-Text • 9B • Updated Jan 20 • 16 • 1

JEILDLWLRMA/Qwen3-VL-8B-Instruct-NVFP4

Image-to-Text • 6B • Updated Feb 2 • 51.6k • 4

JEILDLWLRMA/Qwen3-VL-4B-Instruct-FP8-Dynamic

Image-to-Text • 5B • Updated Feb 7 • 68

JEILDLWLRMA/Qwen3-VL-4B-Instruct-NVFP4

Image-to-Text • 3B • Updated Feb 7 • 22.9k • 2

cstr/Octen-Embedding-0.6B-ONNX-INT8

Sentence Similarity • Updated Aug 2 • 17 • 2

cstr/F2LLM-v2-0.6B-ONNX-INT8

Sentence Similarity • Updated Aug 2 • 20

jsyeom/llama-2-13b-hf-smooth

13B • Updated Mar 24 • 14

emreyigitozturk/llama-3.2-3b-smoothquant-gptq-hybrid

Text Generation • 4B • Updated Mar 24 • 19

rungalileo/llama-3.1-8b-instruct-trtllm-ckpt-wq_w8a8sq-kv_int8

Text Generation • Updated Apr 2 • 18

rungalileo/mistral-7b-instruct-v0.3-trtllm-ckpt-wq_w8a8sq-kv_int8

Text Generation • Updated Apr 2 • 16

vrfai/Qwen3-ASR-1.7B-int8

Automatic Speech Recognition • 2B • Updated Jun 6 • 3

Sensen02/Fast_dVLM_3B_W8A8_FP8

4B • Updated Apr 19 • 19 • 2

cstr/jina-embeddings-v5-text-small-retrieval-onnx-int8

Sentence Similarity • Updated Aug 2 • 35

pantinor/parakeet-tdt-0.6b-v3-smoothquant

stellars/mmBERT-base-openvino-int8

Feature Extraction • Updated Jun 17 • 12

Freaksterz/Qwen3.8-27B-SmoothQuant-W8A8-INT8

22B • Updated Sep 4 • 15.6k • 22

spectator2026/Qwen3.8-27B-int8-prefill-sidecar

Updated 1 day ago

akumaburn/Qwen3.8-27B-heretic-SmoothQuant-W8A8-INT8

Image-Text-to-Text • 27B • Updated Aug 31 • 2.61k • 1

guanwenyu1995/MiniMAX-M2.7-MXFP4-CANN

Text Generation • 115B • Updated 21 days ago • 68

JoaoZaokk/Gemma-3-12B-it-Heretic-W4A8

Updated 12 days ago

florianvoss/Qwen3.5-0.8B-Autoround-a16w4-Modalix

Updated 23 days ago

florianvoss/Qwen3.5-2B-Autoround-a16w4-Modalix

Updated 23 days ago

florianvoss/Qwen3.5-9B-Autoround-a16w4-Modalix

Updated 23 days ago

florianvoss/Qwen3.5-4B-Autoround-a16w4-Modalix

Updated 23 days ago

CompressedMichael/Qwen3-VL-30B-A3B-Instruct-SmoothQuant-W8A8

Image-Text-to-Text • 31B • Updated 23 days ago • 64

akumaburn/Swift-Qwen3.8-27b-heretic-SmoothQuant-W8A8-INT8

Image-Text-to-Text • 27B • Updated 18 days ago • 153