AI & ML interests
None defined yet.
Recent Activity
View all activity
Not evaluated yet....
vLLM compatible. Tested with v0.26.
Verified models. Compatible with vLLM v0.15.1.
Quickly tested with vLLM. Not fully compatible yet.
Verified models.
Verified models.
Verified models. All compatible with vLLM for very fast inference. Use the 3.1 models as they are more recent.
-
kaitchup/Qwen2.5-72B-Instruct-autoround-2bit-32g-4096-gptq
73B • Updated • 3 -
kaitchup/Qwen2.5-72B-Instruct-autoround-2bit-128g-4096-gptq
73B • Updated • 5 -
kaitchup/Qwen2.5-72B-Instruct-autoround-2bit-128g-2048-gptq
73B • Updated • 6 -
kaitchup/Qwen2.5-72B-Instruct-autoround-2bit-64g-4096-gptq
73B • Updated • 3
-
kaitchup/Phi-4-mini-instruct-AutoRoundGPTQ-8bit
Text Generation • 4B • Updated • 11 -
kaitchup/Phi-4-mini-instruct-AutoRoundGPTQ-3bit
Text Generation • 4B • Updated • 14 -
kaitchup/Phi-4-mini-instruct-AutoRoundGPTQ-4bit
Text Generation • 4B • Updated • 19 -
kaitchup/Phi-4-mini-instruct-AutoRoundGPTQ-2bit
Text Generation • 4B • Updated • 15
-
kaitchup/Falcon3-10B-Instruct-AutoRound-GPTQ-4bit
Text Generation • 10B • Updated • 23 -
kaitchup/Falcon3-10B-Base-AutoRound-GPTQ-4bit
Text Generation • 10B • Updated • 6 -
kaitchup/Falcon3-7B-Base-AutoRound-GPTQ-4bit
Text Generation • 7B • Updated • 13 -
kaitchup/Falcon3-7B-Instruct-AutoRound-GPTQ-4bit
Text Generation • 7B • Updated • 8
-
kaitchup/Qwen2.5-1.5B-AutoRound-GPTQ-asym-4bit
Text Generation • 2B • Updated • 16 -
kaitchup/Qwen2.5-7B-AutoRound-GPTQ-asym-4bit
Text Generation • 8B • Updated • 8 -
kaitchup/Qwen2.5-1.5B-Instruct-AutoRound-GPTQ-asym-4bit
Text Generation • 2B • Updated • 5 -
kaitchup/Qwen2.5-7B-Instruct-AutoRound-GPTQ-asym-4bit
Text Generation • 8B • Updated • 15
Machine translation adapters for Llama 2 7B.
Quantized and fine-tuned versions of the Yi models
Contaminated Mistral 7B and TinyLlama adapters, and the datasets used for contamination.
-
kaitchup/Phi-3-mini-4k-instruct-gptq-4bit
Text Generation • 4B • Updated • 2.26k • 2 -
kaitchup/Phi-3-medium-128k-instruct-awq-4bit
Text Generation • 14B • Updated -
kaitchup/Phi-3-mini-4k-instruct-bnb-4bit
Text Generation • 4B • Updated • 5 -
kaitchup/Phi-3-medium-4k-instruct-awq-4bit
Text Generation • 14B • Updated
-
kaitchup/Meta-Llama-3.1-8B-Instruct-autoround-gptq-4bit-sym
Text Generation • 8B • Updated • 25 • 1 -
kaitchup/Meta-Llama-3.1-8B-Instruct-awq-4bit
Text Generation • 8B • Updated • 11 • 1 -
kaitchup/Meta-Llama-3.1-8B-awq-4bit
Text Generation • 8B • Updated • 7 -
kaitchup/Meta-Llama-3.1-8B-Instruct-gptq-4bit
Text Generation • 8B • Updated • 15
-
kaitchup/Mistral-NeMo-Minitron-8B-Base-Minivoc-32k-v0.1a
Text Generation • 8B • Updated -
kaitchup/Llama-3.1-8B-Minivoc-32k-v0.1a
Text Generation • 7B • Updated -
kaitchup/Qwen2-1.5B-Minivoc-32k-v0.1a
Text Generation • 1B • Updated • 768 • 2 -
kaitchup/Qwen2.5-1.5B-Minivoc-32k-v0.1a-AutoRound-GPTQ-asym-4bit
Text Generation • 1B • Updated • 14
Not evaluated yet....
Compatible with vLLM 0.27+
Verified models. Compatible with Transformers v5.3 and vLLM v0.16.1rc1 (nightly). Under evaluation.
Verified models.
Verified models. Compatible with vLLM.
Verified models.
Verified models. Tested with vLLM.
LLMs quantized with LLM Compressor to NVFP4. Check the JSON files in the model directories for more information.
-
kaitchup/OLMo-2-1124-7B-Instruct-AutoRound-GPTQ-4bit
Text Generation • 7B • Updated • 81 • 1 -
kaitchup/Llama-3.1-Tulu-3-70B-AutoRound-GPTQ-4bit
Text Generation • 71B • Updated • 33 -
kaitchup/Llama-3.1-Tulu-3-8B-AutoRound-GPTQ-4bit
Text Generation • 8B • Updated • 8 -
kaitchup/OLMo-2-1124-13B-Instruct-AutoRound-GPTQ-4bit
Text Generation • 14B • Updated • 35
Some language pairs of OPUS formatted to have source and target sentences as single sequences. Intended to facilitate fine-tuning of causal LLMs.
Llama 2 7B, 13B, Llama 3 8B, and Mistral 7B quantized with GPTQ in 2-bit, 3-bit, 4-bit and 8-bit with GPTQ.
A collection of 7B models made with mergekit.
-
kaitchup/Mistral-NeMo-Minitron-8B-Base-AutoRound-GPTQ-sym-4bit
Text Generation • 8B • Updated • 13 -
kaitchup/Mistral-NeMo-Minitron-8B-Base-AutoRound-GPTQ-asym-4bit
Text Generation • 8B • Updated • 5 -
kaitchup/Mistral-Nemo-Base-2407-AutoRound-GPTQ-asym-4bit
Text Generation • 12B • Updated • 4 -
kaitchup/Llama-3.1-Minitron-4B-Width-Base-AutoRound-GPTQ-asym-4bit
Text Generation • 5B • Updated • 6
Not evaluated yet....
Not evaluated yet....
Compatible with vLLM 0.27+
vLLM compatible. Tested with v0.26.
Verified models. Compatible with Transformers v5.3 and vLLM v0.16.1rc1 (nightly). Under evaluation.
Verified models. Compatible with vLLM v0.15.1.
Verified models.
Quickly tested with vLLM. Not fully compatible yet.
Verified models. Compatible with vLLM.
Verified models.
Verified models.
Verified models.
Verified models. Tested with vLLM.
Verified models. All compatible with vLLM for very fast inference. Use the 3.1 models as they are more recent.
LLMs quantized with LLM Compressor to NVFP4. Check the JSON files in the model directories for more information.
-
kaitchup/Qwen2.5-72B-Instruct-autoround-2bit-32g-4096-gptq
73B • Updated • 3 -
kaitchup/Qwen2.5-72B-Instruct-autoround-2bit-128g-4096-gptq
73B • Updated • 5 -
kaitchup/Qwen2.5-72B-Instruct-autoround-2bit-128g-2048-gptq
73B • Updated • 6 -
kaitchup/Qwen2.5-72B-Instruct-autoround-2bit-64g-4096-gptq
73B • Updated • 3
-
kaitchup/Phi-4-mini-instruct-AutoRoundGPTQ-8bit
Text Generation • 4B • Updated • 11 -
kaitchup/Phi-4-mini-instruct-AutoRoundGPTQ-3bit
Text Generation • 4B • Updated • 14 -
kaitchup/Phi-4-mini-instruct-AutoRoundGPTQ-4bit
Text Generation • 4B • Updated • 19 -
kaitchup/Phi-4-mini-instruct-AutoRoundGPTQ-2bit
Text Generation • 4B • Updated • 15
-
kaitchup/Falcon3-10B-Instruct-AutoRound-GPTQ-4bit
Text Generation • 10B • Updated • 23 -
kaitchup/Falcon3-10B-Base-AutoRound-GPTQ-4bit
Text Generation • 10B • Updated • 6 -
kaitchup/Falcon3-7B-Base-AutoRound-GPTQ-4bit
Text Generation • 7B • Updated • 13 -
kaitchup/Falcon3-7B-Instruct-AutoRound-GPTQ-4bit
Text Generation • 7B • Updated • 8
-
kaitchup/OLMo-2-1124-7B-Instruct-AutoRound-GPTQ-4bit
Text Generation • 7B • Updated • 81 • 1 -
kaitchup/Llama-3.1-Tulu-3-70B-AutoRound-GPTQ-4bit
Text Generation • 71B • Updated • 33 -
kaitchup/Llama-3.1-Tulu-3-8B-AutoRound-GPTQ-4bit
Text Generation • 8B • Updated • 8 -
kaitchup/OLMo-2-1124-13B-Instruct-AutoRound-GPTQ-4bit
Text Generation • 14B • Updated • 35
-
kaitchup/Qwen2.5-1.5B-AutoRound-GPTQ-asym-4bit
Text Generation • 2B • Updated • 16 -
kaitchup/Qwen2.5-7B-AutoRound-GPTQ-asym-4bit
Text Generation • 8B • Updated • 8 -
kaitchup/Qwen2.5-1.5B-Instruct-AutoRound-GPTQ-asym-4bit
Text Generation • 2B • Updated • 5 -
kaitchup/Qwen2.5-7B-Instruct-AutoRound-GPTQ-asym-4bit
Text Generation • 8B • Updated • 15
Some language pairs of OPUS formatted to have source and target sentences as single sequences. Intended to facilitate fine-tuning of causal LLMs.
Machine translation adapters for Llama 2 7B.
Llama 2 7B, 13B, Llama 3 8B, and Mistral 7B quantized with GPTQ in 2-bit, 3-bit, 4-bit and 8-bit with GPTQ.
Quantized and fine-tuned versions of the Yi models
A collection of 7B models made with mergekit.
Contaminated Mistral 7B and TinyLlama adapters, and the datasets used for contamination.
-
kaitchup/Phi-3-mini-4k-instruct-gptq-4bit
Text Generation • 4B • Updated • 2.26k • 2 -
kaitchup/Phi-3-medium-128k-instruct-awq-4bit
Text Generation • 14B • Updated -
kaitchup/Phi-3-mini-4k-instruct-bnb-4bit
Text Generation • 4B • Updated • 5 -
kaitchup/Phi-3-medium-4k-instruct-awq-4bit
Text Generation • 14B • Updated
-
kaitchup/Meta-Llama-3.1-8B-Instruct-autoround-gptq-4bit-sym
Text Generation • 8B • Updated • 25 • 1 -
kaitchup/Meta-Llama-3.1-8B-Instruct-awq-4bit
Text Generation • 8B • Updated • 11 • 1 -
kaitchup/Meta-Llama-3.1-8B-awq-4bit
Text Generation • 8B • Updated • 7 -
kaitchup/Meta-Llama-3.1-8B-Instruct-gptq-4bit
Text Generation • 8B • Updated • 15
-
kaitchup/Mistral-NeMo-Minitron-8B-Base-AutoRound-GPTQ-sym-4bit
Text Generation • 8B • Updated • 13 -
kaitchup/Mistral-NeMo-Minitron-8B-Base-AutoRound-GPTQ-asym-4bit
Text Generation • 8B • Updated • 5 -
kaitchup/Mistral-Nemo-Base-2407-AutoRound-GPTQ-asym-4bit
Text Generation • 12B • Updated • 4 -
kaitchup/Llama-3.1-Minitron-4B-Width-Base-AutoRound-GPTQ-asym-4bit
Text Generation • 5B • Updated • 6
-
kaitchup/Mistral-NeMo-Minitron-8B-Base-Minivoc-32k-v0.1a
Text Generation • 8B • Updated -
kaitchup/Llama-3.1-8B-Minivoc-32k-v0.1a
Text Generation • 7B • Updated -
kaitchup/Qwen2-1.5B-Minivoc-32k-v0.1a
Text Generation • 1B • Updated • 768 • 2 -
kaitchup/Qwen2.5-1.5B-Minivoc-32k-v0.1a-AutoRound-GPTQ-asym-4bit
Text Generation • 1B • Updated • 14