Instructions to use mozilla-ai/rocket-3B-llamafile with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use mozilla-ai/rocket-3B-llamafile with Transformers:
# pip install -U transformers accelerate # Load model directly from transformers import AutoModel model = AutoModel.from_pretrained("mozilla-ai/rocket-3B-llamafile", device_map="auto") - Notebooks
- Google Colab
- Kaggle
Commit History
-
Quantize rocket-3B with llamafile-0.8.12 Q4_0
Verified signaturee811bd9 -
Quantize rocket-3B with llamafile-0.8.12 Q6_K
Verified signatureb819cfb -
Quantize rocket-3B with llamafile-0.8.12 Q5_K_S
Verified signatureb72340f -
Quantize rocket-3B with llamafile-0.8.12 Q4_K_M
Verified signature3d31fb7 -
Quantize rocket-3B with llamafile-0.8.12 Q4_K_S
Verified signatured2d5004 -
Quantize rocket-3B with llamafile-0.8.12 Q3_K_L
Verified signaturec3d997e -
Quantize rocket-3B with llamafile-0.8.12 Q3_K_M
Verified signature8c95771 -
Quantize rocket-3B with llamafile-0.8.12 Q3_K_S
Verified signature7416bae -
Quantize rocket-3B with llamafile-0.8.12 Q2_K
Verified signaturedfa124f -
Quantize rocket-3B with llamafile-0.8.12 Q5_1
Verified signature4cc1f40 -
Quantize rocket-3B with llamafile-0.8.12 Q5_0
Verified signature7df8456 -
Quantize rocket-3B with llamafile-0.8.12 Q4_1
Verified signature39eb1fc -
Quantize rocket-3B with llamafile-0.8.12 BF16
Verified signaturedf36a8b -
Quantize rocket-3B with llamafile-0.8.12 F16
Verified signatured92b2e4 -
Quantize rocket-3B with llamafile-0.8.12 Q8_0
Verified signaturecfad3af -
Quantize rocket-3B with llamafile-0.8.12 Q5_K_M
Verified signature65cd08c
-
Quantize rocket-3B with llamafile-0.8.9 Q4_0
Verified signature46c6566 -
Quantize rocket-3B with llamafile-0.8.9 Q6_K
Verified signature295ea57 -
Quantize rocket-3B with llamafile-0.8.9 Q5_K_S
Verified signature1d5d032 -
Quantize rocket-3B with llamafile-0.8.9 Q4_K_M
Verified signature7db94d7 -
Quantize rocket-3B with llamafile-0.8.9 Q4_K_S
Verified signature7c18b86 -
Quantize rocket-3B with llamafile-0.8.9 Q3_K_L
Verified signatureb4c59e2 -
Quantize rocket-3B with llamafile-0.8.9 Q3_K_M
Verified signaturecc2c1b0 -
Quantize rocket-3B with llamafile-0.8.9 Q3_K_S
Verified signatureb18afee -
Quantize rocket-3B with llamafile-0.8.9 Q2_K
Verified signaturebba7d5a -
Quantize rocket-3B with llamafile-0.8.9 Q5_1
Verified signature790e337 -
Quantize rocket-3B with llamafile-0.8.9 Q5_0
Verified signature8aefee9 -
Quantize rocket-3B with llamafile-0.8.9 Q4_1
Verified signature19e71b3 -
Quantize rocket-3B with llamafile-0.8.9 BF16
Verified signaturebace141 -
Quantize rocket-3B with llamafile-0.8.9 F16
Verified signatureb91186b -
Quantize rocket-3B with llamafile-0.8.9 Q8_0
Verified signatureb00056c -
Quantize rocket-3B with llamafile-0.8.9 Q5_K_M
Verified signaturebcdb43c
-
Quantize rocket-3B with llamafile-0.8.8 Q4_0
Verified signature2d5e6ec -
Quantize rocket-3B with llamafile-0.8.8 Q6_K
Verified signaturec7cbd09 -
Quantize rocket-3B with llamafile-0.8.8 Q5_K_S
Verified signature17739fc -
Quantize rocket-3B with llamafile-0.8.8 Q4_K_M
Verified signature7cb587e -
Quantize rocket-3B with llamafile-0.8.8 Q4_K_S
Verified signature442da01 -
Quantize rocket-3B with llamafile-0.8.8 Q3_K_L
Verified signature1da7a7e -
Quantize rocket-3B with llamafile-0.8.8 Q3_K_M
Verified signatureb790eb1 -
Quantize rocket-3B with llamafile-0.8.8 Q3_K_S
Verified signaturea2bd834 -
Quantize rocket-3B with llamafile-0.8.8 Q2_K
Verified signature3ec3ac1 -
Quantize rocket-3B with llamafile-0.8.8 Q5_1
Verified signature2f05aab -
Quantize rocket-3B with llamafile-0.8.8 Q5_0
Verified signature8d36675 -
Quantize rocket-3B with llamafile-0.8.8 Q4_1
Verified signatureb5cb050 -
Quantize rocket-3B with llamafile-0.8.8 BF16
Verified signaturedd7ab40 -
Quantize rocket-3B with llamafile-0.8.8 F16
Verified signature94dda2a -
Quantize rocket-3B with llamafile-0.8.8 Q8_0
Verified signatured5c4424 -
Quantize rocket-3B with llamafile-0.8.8 Q5_K_M
Verified signature10269c8 -
Quantize rocket-3B with llamafile-0.8.8 Q8_0
Verified signaturebf3e456 -
Quantize rocket-3B with llamafile-0.8.8 Q5_K_M
Verified signature1fa49ab