Edit Models filters

Model Tree

Apps

Docker Model Runner

Inference Providers

OVHcloud AI Endpoints

HF Inference API

One-click Deployment

Inference Endpoints

Microsoft Foundry

Amazon SageMaker AI

Misc

text-generation-inference

Eval Results (legacy)

text-embeddings-inference

4-bit precision

8-bit precision

Mixture of Experts

Carbon Emissions

Models

7,868

Base only

Active filters: llama.cpp

google/gemma-1.1-2b-it-GGUF

3B • Updated Jun 27, 2024 • 1 • 21

webbigdata/C3TR-Adapter_gguf

Translation • 9B • Updated Aug 14, 2024 • 214 • 26

google/codegemma-7b-GGUF

Text Generation • 9B • Updated Jun 27, 2024 • 20 • 28

pacozaa/bonito-gguf

7B • Updated Apr 14, 2024 • 29

pmking27/PrathameshLLM-2B-GGUF

3B • Updated Apr 9, 2024 • 790 • 1

teleprint-me/cyberpunk-valerie-v0.1

Text Generation • 90.1M • Updated Apr 18, 2024 • 104 • 1

qwp4w3hyb/Meta-Llama-3-8B-Instruct-iMat-GGUF

Text Generation • 8B • Updated Apr 29, 2024 • 1.82k • 6

mgonzs13/Mistroll-7B-v2.2-GGUF

Text Generation • 7B • Updated Apr 29, 2024 • 74

mgonzs13/ladybird-base-7B-v8-GGUF

Text Generation • 7B • Updated Apr 29, 2024 • 27

google/codegemma-1.1-2b-GGUF

Text Generation • 3B • Updated Jun 27, 2024 • 7 • 5

google/codegemma-1.1-7b-it-GGUF

Text Generation • 9B • Updated Jun 27, 2024 • 14

openbmb/MiniCPM-Llama3-V-2_5-gguf

Updated Feb 27, 2025 • 1.83k • 216

QuantFactory/Ahma-3B-GGUF

Text Generation • 4B • Updated Jul 2, 2025 • 443 • 2

mgonzs13/TextBase-7B-v0.1-GGUF

Text Generation • 7B • Updated Jun 11, 2024 • 24

QuantFactory/TextBase-7B-v0.1-GGUF

Text Generation • 7B • Updated Jun 18, 2024 • 193

njwright92/ComicBot_v.2-gguf

Text Generation • 7B • Updated Aug 30, 2024 • 67

Irathernotsay/qwen2-1.5B-medical_qa-Finetune

Text Generation • 2B • Updated Jul 17, 2024 • 11

palusi/Qwen2-0.5B-Instruct-GGUF

0.5B • Updated Jun 27, 2024 • 79

XavierSpycy/Meta-Llama-3-8B-Instruct-zh-10k

Text Generation • 8B • Updated Jul 9, 2024 • 15 •

ruslanmv/Medical-Llama3-v2-Q4_K_M-GGUF

8B • Updated Jun 30, 2024 • 22

XavierSpycy/Meta-Llama-3-8B-Instruct-zh-10k-GGUF

Text Generation • 8B • Updated Jul 9, 2024 • 45

XavierSpycy/Meta-Llama-3-8B-Instruct-zh-10k-GPTQ

Text Generation • 8B • Updated Jul 9, 2024 • 7

zhhan/Phi-3-mini-4k-instruct_gguf_derived

Summarization • 4B • Updated Jul 2, 2024 • 9

XavierSpycy/Meta-Llama-3-8B-Instruct-zh-10k-AWQ

Text Generation • 8B • Updated Jul 9, 2024 • 5

mgonzs13/stablelm-zephyr-3B-localmentor-GGUF

Text Generation • 3B • Updated Jul 3, 2024 • 214

google/gemma-2-2b-it-GGUF

3B • Updated Aug 27, 2024 • 92 • 93

google/gemma-2-2b-GGUF

3B • Updated Aug 2, 2024 • 28 • 22

chatpdflocal/llama3.1-8b-gguf

8B • Updated Mar 23 • 235 • 29

akshathmangudi/llama3.1-8b-gguf

Updated Jul 26, 2024

dahara1/llama-translate-gguf

8B • Updated Aug 14, 2024 • 99 • 16