Edit Models filters

Model Tree

Apps

Docker Model Runner

Inference Providers

OVHcloud AI Endpoints

HF Inference API

One-click Deployment

Inference Endpoints

Microsoft Foundry

Amazon SageMaker AI

Misc

text-generation-inference

Eval Results (legacy)

text-embeddings-inference

4-bit precision

8-bit precision

Mixture of Experts

Carbon Emissions

Models

13,762

Base only

Active filters: quantized

MaziyarPanahi/Meta-Llama-3.1-405B-Instruct-GGUF

Text Generation • 410B • Updated Jul 29, 2024 • 113k • 15

pek111/Meta-Llama-3.1-8B-Instruct-GGUF

Text Generation • 8B • Updated Jul 26, 2024 • 299 • 7

legraphista/Mistral-Large-Instruct-2407-IMat-GGUF

Text Generation • 123B • Updated Jul 25, 2024 • 264 • 29

MaziyarPanahi/Mistral-Large-Instruct-2407-GGUF

Text Generation • 123B • Updated Jul 26, 2024 • 2.18k • 21

akshathmangudi/llama3.1-8b-quantized

Text Generation • 8B • Updated Jul 25, 2024 • 6

akshathmangudi/llama3.1-8b-gguf

Updated Jul 26, 2024

legraphista/Meta-Llama-3.1-8B-Instruct-abliterated-IMat-GGUF

Text Generation • 8B • Updated Jul 28, 2024 • 359 • 1

MaziyarPanahi/calme-2.1-llama3.1-70b-GGUF

Text Generation • 71B • Updated Jul 31, 2024 • 191 • 2

RedHatAI/Meta-Llama-3.1-8B-quantized.w8a8

Text Generation • 8B • Updated 5 days ago • 493 • 5

legraphista/gemma-2-2b-it-IMat-GGUF

Text Generation • 3B • Updated Jul 31, 2024 • 908 • 1

legraphista/gemma-2-2b-IMat-GGUF

Text Generation • 3B • Updated Jul 31, 2024 • 1.04k • 1

MaziyarPanahi/gemma-2-2b-it-GGUF

Text Generation • 3B • Updated Aug 1, 2024 • 115k • 14

thesven/Mistral-7B-Instruct-v0.3-GPTQ-4bit

Text Generation • 7B • Updated Aug 2, 2024 • 4 • 1

MaziyarPanahi/calme-2.2-qwen2-72b-GGUF

Text Generation • 73B • Updated Aug 6, 2024 • 72 • 2

legraphista/Palmyra-Fin-70B-32K-IMat-GGUF

Text Generation • 71B • Updated Aug 2, 2024 • 1.5k • 10

liemo/open-lilm-v2-q4

Text Generation • 6B • Updated Sep 29, 2024 • 4

legraphista/internlm2_5-1_8b-chat-IMat-GGUF

Text Generation • 2B • Updated Aug 5, 2024 • 525

legraphista/internlm2_5-20b-chat-IMat-GGUF

Text Generation • 20B • Updated Aug 5, 2024 • 169

legraphista/shieldgemma-2b-IMat-GGUF

Text Generation • 3B • Updated Aug 5, 2024 • 611

legraphista/shieldgemma-9b-IMat-GGUF

Text Generation • 9B • Updated Aug 5, 2024 • 235

legraphista/shieldgemma-27b-IMat-GGUF

Text Generation • 27B • Updated Aug 5, 2024 • 1.07k • 1

MaziyarPanahi/RYS-XLarge-GGUF

Text Generation • 78B • Updated Aug 6, 2024 • 68 • 7

MaziyarPanahi/calme-2.3-legalkit-8b-GGUF

Text Generation • 8B • Updated Aug 7, 2024 • 955 • 8

MaziyarPanahi/Qwen2-Math-1.5B-Instruct-GGUF

Text Generation • 2B • Updated Aug 8, 2024 • 226 • 2

MaziyarPanahi/Qwen2-Math-7B-Instruct-GGUF

Text Generation • 8B • Updated Aug 8, 2024 • 284 • 1

MaziyarPanahi/Qwen2-Math-72B-Instruct-GGUF

Text Generation • 73B • Updated Aug 20, 2024 • 455 • 2

legraphista/Qwen2-Math-1.5B-Instruct-IMat-GGUF

Text Generation • 2B • Updated Aug 8, 2024 • 1.49k

legraphista/Qwen2-Math-7B-Instruct-IMat-GGUF

Text Generation • 8B • Updated Aug 8, 2024 • 272 • 1

legraphista/Qwen2-Math-72B-Instruct-IMat-GGUF

Text Generation • 73B • Updated Aug 8, 2024 • 133

juliensimon/Arcee-Nova-fp8

Text Generation • 73B • Updated Mar 22 • 8