Edit Models filters

Model Tree

Apps

Docker Model Runner

Inference Providers

OVHcloud AI Endpoints

HF Inference API

One-click Deployment

Inference Endpoints

Microsoft Foundry

Amazon SageMaker AI

Misc

text-generation-inference

Eval Results (legacy)

text-embeddings-inference

4-bit precision

8-bit precision

Mixture of Experts

Carbon Emissions

Models

13,608

Base only

Active filters: quantized

Makatia/TinyLlama_TinyLlama-1.1B-Chat-v1.0_onnx

Text Generation • Updated Mar 25 • 2

ramblingpolymath/Qwen3-32B-W8A8

Text Generation • 33B • Updated Aug 2, 2025 • 66

steampunque/Deepseek-R1-Distill-Llama-8B-MP-GGUF

8B • Updated Feb 18 • 4

ramblingpolymath/Qwen3-14B-W8A8

Text Generation • 15B • Updated Aug 3, 2025 • 47

ramblingpolymath/Qwen3-8B-W8A8

Text Generation • 8B • Updated Aug 3, 2025 • 48

ramblingpolymath/Qwen3-4B-W8A8

Text Generation • 4B • Updated Aug 2, 2025 • 50

RedHatAI/Kimi-K2-Instruct-quantized.w4a16

Text Generation • 1T • Updated Apr 28 • 248 • 12

adamrb/mpt-30b-chat-w4a16-gptq

30B • Updated Jul 15, 2025 • 4

adamrb/mpt-30b-chat-w8a8-gptq

30B • Updated Jul 15, 2025 • 8

ramblingpolymath/qwen3-30B-A3B-w8a8

Text Generation • 31B • Updated Aug 2, 2025 • 17

ramblingpolymath/Qwen3-0.6B-W8A8

Text Generation • 0.8B • Updated Aug 3, 2025 • 21

tachyphylaxis/DeepSeek-R1-0528-FP4

Text Generation • 397B • Updated Jul 18, 2025 • 11

SandLogicTechnologies/MedGemma-4B-IT-GGUF

4B • Updated Jul 29, 2025 • 42 • 4

PJEDeveloper/Mistral_Nemo_Instruct_2407-F16.gguf-Q4_K_M

12B • Updated Jul 23, 2025 • 6

sdurgi/bert_emotion_response_classifier_quantized

Text Classification • Updated Jul 19, 2025 • 3

Hariprasath28/orpheus-3b-4bit-AWQ

Text Generation • 3B • Updated Jul 19, 2025 • 150

mirekphd/whisper-large-v3-onnx-fp16

Automatic Speech Recognition • Updated Jul 19, 2025 • 3

mirekphd/whisper-large-v3-onnx-w8a16-dynamic

Automatic Speech Recognition • Updated Jul 19, 2025 • 4

mirekphd/whisper-large-v3-onnx-w4a16-dynamic

Automatic Speech Recognition • Updated Jul 19, 2025 • 2

steampunque/Qwen2.5-VL-32B-Instruct-MP-GGUF

33B • Updated Feb 18 • 12

theprint/Zeth-Gemma3-4B-GGUF

Text Generation • 5B • Updated Aug 16, 2025 • 18

HackNetAyush/smollm2-135M-instruct-gguf-q8

Text Generation • 0.1B • Updated Jul 20, 2025 • 94 • 3

ericrisco/medical-gemma-3n-4b-gguf

7B • Updated Jul 28, 2025 • 18

nvidia/DeepSeek-R1-0528-NVFP4-v2

Text Generation • 394B • Updated Sep 2, 2025 • 313k • 23

steampunque/GLM-Z1-9B-0414-MP-GGUF

9B • Updated Feb 18 • 82 • 2

nvidia/DeepSeek-R1-NVFP4-v2

Text Generation • 394B • Updated Jul 22, 2025 • 399 • 8

YongdongWang/llama-3.2-1b-lora-qlora-dart-llm-gguf

Text Generation • 1B • Updated 24 days ago • 577 • 3

sugiv/cardvaultplus-500m-gguf

Image-to-Text • 0.4B • Updated Jul 22, 2025 • 30 • 2

PJEDeveloper/mistralai_Mistral-7B-Instruct-v0.3-F16.gguf-Q5_K_M

7B • Updated Jul 23, 2025 • 15

nufikq/tinyllama-php-finetuned-test1

Updated Jul 23, 2025