Edit Models filters

Model Tree

Apps

Docker Model Runner

Unsloth Desktop

Inference Providers

OVHcloud AI Endpoints

HF Inference API

One-click Deployment

Inference Endpoints

Microsoft Foundry

Amazon SageMaker AI

Misc

text-generation-inference

Eval Results (legacy)

text-embeddings-inference

4-bit precision

8-bit precision

Mixture of Experts

Carbon Emissions

Models

3,304

Base only

Active filters: quantization

forkjoin-ai/deepseek-r1-7b-rknot

AxisQuant/Qwen3.5-4B-INT8

Text Generation • 4B • Updated May 16 • 101 • 2

forkjoin-ai/llama-70b-rknot

forkjoin-ai/llama-70b-partial-rknot

Reinforce-ai/Qwen3.6-35B-A3B-Spiral

Text Generation • 35B • Updated May 12 • 56

embedl/mobilevit-small-quantized

Image Classification • Updated Jul 27

RthItalia/nano_compact_3b_qkvfp16

Text Generation • 2B • Updated May 4 • 10 • 1

toxzak/Qwen3.6-27B-FABQ-RC

Text Generation • Updated Jun 9 • 12 • 1

madhurithika22/mistral-compressed

Text Generation • 7B • Updated May 7

Entrit/tritllm-codec

Entrit/tritllm-kernel

Entrit/Entrit

Entrit/tritllm-paper

Updated May 4 • 2

plawanrath/qwen2.5-7b-instruct-bf16-mlx-cba

Text Generation • 8B • Updated May 19 • 14

plawanrath/qwen2.5-7b-instruct-q8-mlx-cba

Text Generation • 8B • Updated May 19 • 20

plawanrath/qwen2.5-7b-instruct-q6-mlx-cba

Text Generation • 8B • Updated May 19 • 21

plawanrath/qwen2.5-7b-instruct-q4-mlx-cba

Text Generation • 8B • Updated May 19 • 37

plawanrath/qwen2.5-7b-instruct-q3-mlx-cba

Text Generation • 8B • Updated May 19 • 39

plawanrath/mistral-7b-instruct-v0.3-bf16-mlx-cba

Text Generation • 7B • Updated May 19 • 20

plawanrath/mistral-7b-instruct-v0.3-q8-mlx-cba

Text Generation • 7B • Updated May 19 • 21

plawanrath/mistral-7b-instruct-v0.3-q6-mlx-cba

Text Generation • 7B • Updated May 19 • 24

plawanrath/mistral-7b-instruct-v0.3-q4-mlx-cba

Text Generation • 7B • Updated May 19 • 43

plawanrath/mistral-7b-instruct-v0.3-q3-mlx-cba

Text Generation • 7B • Updated May 19 • 27

plawanrath/phi-3.5-mini-instruct-bf16-mlx-cba

Text Generation • 4B • Updated May 19 • 26

plawanrath/phi-3.5-mini-instruct-q8-mlx-cba

Text Generation • 4B • Updated May 19 • 39

plawanrath/phi-3.5-mini-instruct-q6-mlx-cba

Text Generation • 4B • Updated May 19 • 27

plawanrath/phi-3.5-mini-instruct-q4-mlx-cba

Text Generation • 4B • Updated May 19 • 36

plawanrath/phi-3.5-mini-instruct-q3-mlx-cba

Text Generation • 4B • Updated May 19 • 31

jpcurada/SmolLM2-1.7B-Scheduled-QAT-Linear-INT4

Text Generation • 2B • Updated May 5 • 21

jpcurada/SmolLM2-1.7B-Scheduled-QAT-Linear-GGUF

Text Generation • 2B • Updated May 5 • 21 • 1