Edit Models filters

Model Tree

Apps

Docker Model Runner

Inference Providers

OVHcloud AI Endpoints

HF Inference API

One-click Deployment

Inference Endpoints

Microsoft Foundry

Amazon SageMaker AI

Misc

text-generation-inference

Eval Results (legacy)

text-embeddings-inference

4-bit precision

8-bit precision

Mixture of Experts

Carbon Emissions

Models

24,697

Base only

Active filters: llama-cpp

Ayyystin/Llama-3-8B-Lexi-Uncensored-Q4_0-GGUF

8B • Updated Jun 7, 2024 • 84 • 2

jeiku/AverageAdmixture-Q4_K_M-GGUF

8B • Updated Jun 7, 2024 • 8

drmcbride/Qwen2-0.5B-Q2_K-GGUF

Text Generation • 0.5B • Updated Jun 7, 2024 • 12

drmcbride/Qwen2-0.5B-Q8_0-GGUF

Text Generation • 0.5B • Updated Jun 7, 2024 • 19

drmcbride/Qwen2-1.5B-Q2_K-GGUF

Text Generation • 2B • Updated Jun 7, 2024 • 25

drmcbride/Qwen2-1.5B-Q8_0-GGUF

Text Generation • 2B • Updated Jun 7, 2024 • 22

drmcbride/Qwen2-7B-Q2_K-GGUF

Text Generation • 8B • Updated Jun 7, 2024 • 13

drmcbride/Qwen2-7B-Q8_0-GGUF

Text Generation • 8B • Updated Jun 7, 2024 • 8

v8karlo/v8-Qwen2-7B-Q5_K_M-GGUF

Text Generation • 8B • Updated Jun 7, 2024

v8karlo/MindChat-Qwen2-4B-Q5_K_M-GGUF

4B • Updated Jun 7, 2024 • 37

CarrotAI/Llama3-Ko-Carrot-8B-it-gguf

Text Generation • 8B • Updated Jun 20, 2024 • 19 • 1

v8karlo/Qwen2-7B-Q5_K_M-GGUF

Text Generation • 8B • Updated Jun 8, 2024 • 28

rzabounidis/falcon-11B-Q6_K-GGUF

11B • Updated Jun 8, 2024 • 4

farpluto/Qwen2-0.5B-Instruct-Q4_K_M-GGUF

Text Generation • 0.5B • Updated Jun 8, 2024 • 39

farpluto/internlm2-chat-1_8b-Q4_K_M-GGUF

Text Generation • 2B • Updated Jun 8, 2024 • 5

itod/Meta-Llama-3-8B-Instruct-Q8_0-GGUF

Text Generation • 8B • Updated Jun 8, 2024 • 5

itod/UAE-Large-V1-Q8_0-GGUF

0.3B • Updated Jun 8, 2024 • 9

TheVirusMaster/Meta-Llama-3-8B-Q4_K_M-GGUF

Text Generation • 8B • Updated Jun 8, 2024 • 9

calculatortamer/Llama-3-5B-Sheard-Q4_K_M-GGUF

Text Generation • 6B • Updated Jun 8, 2024 • 11

kpkiosk66/LLaMa-3-Stheno-v3.2-15B-Q4_K_S-GGUF

15B • Updated Jun 8, 2024 • 84 • 4

clop51/merlinite-7b-lab-Q4_K_M-lurn_slurm-Q4_K_M-GGUF

7B • Updated Jun 8, 2024 • 2

ClaudioItaly/TopEvolutionWiz-Q8_0-GGUF

7B • Updated Aug 13, 2024 • 12

AlirezaF138/Dorna-Llama3-8B-Instruct-GGUF

8B • Updated Jun 8, 2024 • 28 • 1

tihuang/Qwen2-7B-Instruct-Q5_K_M-GGUF

Text Generation • 8B • Updated Jun 8, 2024 • 20

NikolayKozloff/Llama-3-8B-Swedish-Norwegian-Danish-checkpoint-14375-08_06_2024-Q8_0-GGUF

Text Generation • 8B • Updated Jun 9, 2024 • 22 • 1

2sy1227/gemma-7b-ko-sum-Q4_K_M-GGUF

9B • Updated Jun 9, 2024 • 8

zachaman/phi3-goodwater5-merged-Q4_0-GGUF

4B • Updated Jun 9, 2024 • 7

rhudock/Meta-Llama-3-8B-Q4_K_M-GGUF

Text Generation • 8B • Updated Jun 9, 2024 • 8

NikolayKozloff/GeM2-Llamion-14B-LongChat-Q4_0-GGUF

14B • Updated Jun 9, 2024 • 18 • 1

NikolayKozloff/Irbis-7b-v0.1-Kazakh-Q8_0-GGUF

Text Generation • 7B • Updated Jun 9, 2024 • 15 • 2