Edit Models filters

Model Tree

Apps

Docker Model Runner

Unsloth Desktop

Inference Providers

OVHcloud AI Endpoints

HF Inference API

One-click Deployment

Inference Endpoints

Microsoft Foundry

Amazon SageMaker AI

Misc

kv-cache-quantization

text-generation-inference

Eval Results (legacy)

text-embeddings-inference

4-bit precision

8-bit precision

Mixture of Experts

Carbon Emissions

Models

144

Base only

Active filters: kv-cache-quantization

majentik/gpt-oss-20b-RotorQuant-GGUF-IQ4_XS

Text Generation • 21B • Updated 20 days ago • 509

majentik/Nemotron-3-Super-120B-A12B-TurboQuant-GGUF-Q8_0

Text Generation • 121B • Updated 20 days ago • 114

majentik/MiniMax-M2.7-TurboQuant-MLX-3bit

Text Generation • 229B • Updated Aug 20 • 153

majentik/MiniMax-M2.7-TurboQuant-MLX-4bit

Text Generation • 229B • Updated Aug 20 • 74

majentik/MiniMax-M2.7-TurboQuant-MLX-5bit

Text Generation • 229B • Updated Aug 20 • 89 • 1

haifeng1976/gemma-4-E2B-it-RotorQuant-GGUF-Q3_K_M

Image-Text-to-Text • 5B • Updated Apr 16 • 45

haifeng1976/gemma-4-E2B-it-RotorQuant-GGUF-Q2_K

Image-Text-to-Text • 5B • Updated Apr 16 • 57

majentik/garden

Text Generation • Updated 1 day ago

majentik/Qwen3.6-35B-A3B-TurboQuant-MLX-4bit

Text Generation • 35B • Updated Aug 20 • 514 • 16

majentik/Qwen3.6-35B-A3B-RotorQuant-MLX-2bit

Text Generation • 35B • Updated Aug 20 • 803 • 10

majentik/Qwen3.6-35B-A3B-RotorQuant-MLX-8bit

Text Generation • 35B • Updated Aug 20 • 199

majentik/Qwen3.6-35B-A3B-RotorQuant-GGUF-Q8_0

Image-Text-to-Text • 35B • Updated 20 days ago • 159

majentik/Qwen3.6-35B-A3B-RotorQuant-GGUF-Q3_K_M

Image-Text-to-Text • 35B • Updated 20 days ago • 259 • 2

majentik/Qwen3.6-35B-A3B-RotorQuant-GGUF-Q2_K

Image-Text-to-Text • 35B • Updated 20 days ago • 181

majentik/Qwen3.6-35B-A3B-RotorQuant-GGUF-Q5_K_M

Image-Text-to-Text • 35B • Updated 20 days ago • 149 • 1

majentik/Qwen3.6-35B-A3B-RotorQuant-GGUF-IQ4_XS

Image-Text-to-Text • 35B • Updated 20 days ago • 425 • 5

majentik/Qwen3.6-35B-A3B-RotorQuant-GGUF-Q4_K_M

Image-Text-to-Text • 35B • Updated 20 days ago • 190 • 3

majentik/Qwen3.6-35B-A3B-RotorQuant-MLX-6bit

Text Generation • 35B • Updated Aug 20 • 198

majentik/Qwen3.6-35B-A3B-RotorQuant-MLX-5bit

Text Generation • 35B • Updated Aug 20 • 337 • 1

majentik/Qwen3.6-35B-A3B-TurboQuant-MLX-3bit

Text Generation • 35B • Updated Aug 20 • 356 • 6

BananaMind/BananaMind-KV1-8M-2Bit-Experimental

Text Generation • 8.13M • Updated Jul 3 • 30 • 3

Ddavidich/LOMONOSOV-ZENIT-27B-1M-INDEV

Text Generation • 16B • Updated Jul 30 • 66 • 1

VAREQON/Nemotron-3-Nano-4B-RotorQuant-GGUF-Q2_K

Text Generation • 4B • Updated Jul 30 • 30 • 1

ajh-code/Qwen3.8-27B-Escha-W2-16GB-120Kctx-KVarN

Text Generation • Updated Aug 24 • 72 • 2