OpenVINO/phi-2-int4-ov
Text Generation • Updated • 26 • 1
int4 llm
-
-
nvidia/Gemma-2b-it-ONNX-INT4
Updated • 9 -
nvidia/Meta-Llama-3.1-8B-Instruct-ONNX-INT4
Updated • 125 • 9 -
nvidia/Meta-Llama-3.2-3B-Instruct-ONNX-INT4
Updated • 8 -
nvidia/Phi-3.5-mini-Instruct-ONNX-INT4
Updated • 2 -
nvidia/Mistral-Nemo-12B-Instruct-ONNX-INT4
Updated • 4 -
nvidia/Nemotron-Mini-4B-Instruct-ONNX-INT4
Updated • 9 -
meta-llama/Llama-3.2-1B-Instruct-SpinQuant_INT4_EO8
Text Generation • Updated • 173 • 41 -
hugging-quants/gemma-2-9b-it-AWQ-INT4
Text Generation • 9B • Updated • 5.94k • 10 -
Qwen/Qwen2-7B-Instruct-GPTQ-Int4
Text Generation • 8B • Updated • 405 • 28 -
hugging-quants/Meta-Llama-3.1-8B-Instruct-AWQ-INT4
Text Generation • 8B • Updated • 137k • 93 -
RedHatAI/Meta-Llama-3.1-8B-Instruct-quantized.w4a16
Text Generation • 8B • Updated • 42.9k • 30 -
ModelCloud/Meta-Llama-3.1-8B-gptq-4bit
Text Generation • 8B • Updated • 62 -
hugging-quants/Llama-3.2-3B-Instruct-Q4_K_M-GGUF
Text Generation • 3B • Updated • 37.3k • 48 -
hugging-quants/Meta-Llama-3.1-70B-Instruct-AWQ-INT4
Text Generation • 71B • Updated • 161k • 111 -
hugging-quants/Llama-3.2-1B-Instruct-Q4_K_M-GGUF
Text Generation • 1B • Updated • 55.8k • 33 -
hugging-quants/Meta-Llama-3.1-70B-Instruct-GPTQ-INT4
Text Generation • 71B • Updated • 334 • 23 -
hugging-quants/Meta-Llama-3.1-8B-Instruct-GPTQ-INT4
Text Generation • 8B • Updated • 8.64k • 42 -
meta-llama/Llama-Guard-3-1B-INT4
Text Generation • Updated • 13 • 31 -
meta-llama/Llama-3.2-3B-Instruct-QLORA_INT4_EO8
Text Generation • Updated • 124 • 76 -
meta-llama/Llama-3.2-3B-Instruct-SpinQuant_INT4_EO8
Text Generation • Updated • 154 • 40 -
meta-llama/Llama-3.2-1B-Instruct-QLORA_INT4_EO8
Text Generation • Updated • 148 • 49 -
RedHatAI/Mistral-7B-Instruct-v0.3-GPTQ-4bit
Text Generation • 7B • Updated • 1.03k • 25 -
RedHatAI/Mistral-7B-Instruct-v0.3-quantized.w4a16
Text Generation • 7B • Updated • 224 • 2 -
RedHatAI/Llama-2-7b-chat-quantized.w4a16
Text Generation • 7B • Updated • 32 -
RedHatAI/Meta-Llama-3-8B-Instruct-quantized.w4a16
Text Generation • 8B • Updated • 96 • 2 -
RedHatAI/Meta-Llama-3-70B-Instruct-quantized.w4a16
Text Generation • 71B • Updated • 32 • 2 -
RedHatAI/gemma-2-2b-it-quantized.w4a16
Text Generation • 3B • Updated • 510 • 1 -
RedHatAI/gemma-2-9b-it-quantized.w4a16
Text Generation • 10B • Updated • 116 • 2 -
RedHatAI/Mistral-Nemo-Instruct-2407-quantized.w4a16
Text Generation • 12B • Updated • 1.18k • 4 -
RedHatAI/Meta-Llama-3.1-70B-Instruct-quantized.w4a16
Text Generation • 71B • Updated • 8.7k • 33 -
nvidia/Mistral-7B-Instruct-v0.3-ONNX-INT4
Updated • 8 -
OpenVINO/mistral-7b-instruct-v0.1-int4-ov
Text Generation • Updated • 82 • 1 -
OpenVINO/Mistral-7B-Instruct-v0.2-int4-ov
Text Generation • Updated • 212 • 1 -
Qwen/Qwen-72B-Chat-Int4
Text Generation • 72B • Updated • 247 • 47 -
Qwen/Qwen-14B-Chat-Int4
Text Generation • 14B • Updated • 371 • 101 -
Qwen/Qwen-7B-Chat-Int4
Text Generation • 8B • Updated • 418 • 75 -
Qwen/Qwen-1_8B-Chat-Int4
Text Generation • 2B • Updated • 359 • 36 -
Qwen/Qwen1.5-110B-Chat-GPTQ-Int4
Text Generation • 111B • Updated • 137 • 19 -
Qwen/Qwen1.5-1.8B-Chat-GPTQ-Int4
Text Generation • 2B • Updated • 1.19k • 7 -
Qwen/Qwen1.5-MoE-A2.7B-Chat-GPTQ-Int4
Text Generation • 14B • Updated • 1.38k • 50 -
Qwen/Qwen1.5-4B-Chat-GPTQ-Int4
Text Generation • 4B • Updated • 1.72k • 7 -
Qwen/Qwen1.5-72B-Chat-GPTQ-Int4
Text Generation • 72B • Updated • 4.11k • 37 -
Qwen/Qwen1.5-4B-Chat-GGUF
Text Generation • 4B • Updated • 926 • 16 -
Qwen/Qwen1.5-0.5B-Chat-GGUF
Text Generation • 0.6B • Updated • 10.3k • 37 -
Qwen/Qwen1.5-7B-Chat-GGUF
Text Generation • 8B • Updated • 12.5k • 71 -
Qwen/CodeQwen1.5-7B-Chat-GGUF
Text Generation • 7B • Updated • 1.14k • 111 -
Qwen/Qwen2.5-1.5B-Instruct-GPTQ-Int4
Text Generation • 2B • Updated • 3.46k • 3 -
Qwen/Qwen2.5-0.5B-Instruct-GPTQ-Int4
Text Generation • 0.5B • Updated • 1.08k • 9 -
Qwen/Qwen2.5-0.5B-Instruct-GGUF
Text Generation • 0.6B • Updated • 238k • 142 -
Qwen/Qwen2-1.5B-Instruct-GGUF
Text Generation • 2B • Updated • 21.9k • 33 -
Qwen/Qwen2-0.5B-Instruct-GGUF
Text Generation • 0.5B • Updated • 12.7k • 77 -
Qwen/Qwen2-7B-Instruct-GGUF
Text Generation • 8B • Updated • 3.98k • 180 -
Qwen/Qwen2-0.5B-Instruct-GPTQ-Int4
Text Generation • 0.6B • Updated • 444 • 16 -
Qwen/Qwen2-1.5B-Instruct-GPTQ-Int4
Text Generation • 2B • Updated • 3.55k • 6 -
Qwen/Qwen2-72B-Instruct-GPTQ-Int4
Text Generation • 73B • Updated • 154 • 33 -
Qwen/Qwen2-57B-A14B-Instruct-GPTQ-Int4
Text Generation • 57B • Updated • 284 • 23