Instructions to use tensorblock/amazon_MistralLite-GGUF with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- llama.cpp
How to use tensorblock/amazon_MistralLite-GGUF with llama.cpp:
Install (macOS, Linux)
curl -LsSf https://llama.app/install.sh | sh # Start a local OpenAI-compatible server with a web UI: llama serve -hf tensorblock/amazon_MistralLite-GGUF:Q2_K # Run inference directly in the terminal: llama cli -hf tensorblock/amazon_MistralLite-GGUF:Q2_K
Install from WinGet (Windows)
winget install llama.cpp # Start a local OpenAI-compatible server with a web UI: llama serve -hf tensorblock/amazon_MistralLite-GGUF:Q2_K # Run inference directly in the terminal: llama cli -hf tensorblock/amazon_MistralLite-GGUF:Q2_K
Build from source code
git clone https://github.com/ggml-org/llama.cpp.git cd llama.cpp cmake -B build cmake --build build -j --target llama-app # Start a local OpenAI-compatible server with a web UI: ./build/bin/llama serve -hf tensorblock/amazon_MistralLite-GGUF:Q2_K # Run inference directly in the terminal: ./build/bin/llama cli -hf tensorblock/amazon_MistralLite-GGUF:Q2_K
Use Docker
docker model run hf.co/tensorblock/amazon_MistralLite-GGUF:Q2_K
- Llama new
- LM Studio
- Jan
- Ollama
How to use tensorblock/amazon_MistralLite-GGUF with Ollama:
ollama run hf.co/tensorblock/amazon_MistralLite-GGUF:Q2_K
- Unsloth Desktop
- Docker Model Runner
How to use tensorblock/amazon_MistralLite-GGUF with Docker Model Runner:
docker model run hf.co/tensorblock/amazon_MistralLite-GGUF:Q2_K
- Lemonade
How to use tensorblock/amazon_MistralLite-GGUF with Lemonade:
Pull the model
# Download Lemonade from https://lemonade-server.ai/ lemonade pull tensorblock/amazon_MistralLite-GGUF:Q2_K
Run and chat with the model
lemonade run user.amazon_MistralLite-GGUF-Q2_K
List all available models
lemonade list
- Atomic Chat
Remove .gguf files (keep Q2_K.gguf)
Browse files- MistralLite-Q3_K_L.gguf +0 -3
- MistralLite-Q3_K_M.gguf +0 -3
- MistralLite-Q3_K_S.gguf +0 -3
- MistralLite-Q4_0.gguf +0 -3
- MistralLite-Q4_K_M.gguf +0 -3
- MistralLite-Q4_K_S.gguf +0 -3
- MistralLite-Q5_0.gguf +0 -3
- MistralLite-Q5_K_M.gguf +0 -3
- MistralLite-Q5_K_S.gguf +0 -3
- MistralLite-Q6_K.gguf +0 -3
- MistralLite-Q8_0.gguf +0 -3
MistralLite-Q3_K_L.gguf
DELETED
|
@@ -1,3 +0,0 @@
|
|
| 1 |
-
version https://git-lfs.github.com/spec/v1
|
| 2 |
-
oid sha256:bebad3b4bdb2aaaa9e3f73ebfdd296b4c3222a656ae54b0b28ddd84408ced764
|
| 3 |
-
size 3822040064
|
|
|
|
|
|
|
|
|
|
|
|
MistralLite-Q3_K_M.gguf
DELETED
|
@@ -1,3 +0,0 @@
|
|
| 1 |
-
version https://git-lfs.github.com/spec/v1
|
| 2 |
-
oid sha256:72fb63a5b8eec6980f6b52f90f19d821fa9cd5c68b4c6916fe0b02ce86918fa3
|
| 3 |
-
size 3519001600
|
|
|
|
|
|
|
|
|
|
|
|
MistralLite-Q3_K_S.gguf
DELETED
|
@@ -1,3 +0,0 @@
|
|
| 1 |
-
version https://git-lfs.github.com/spec/v1
|
| 2 |
-
oid sha256:b8e33c283d0783aef3ead6c5d2de0b4a6d5f584dd06e45b3259ab6c9fee98b6f
|
| 3 |
-
size 3164582912
|
|
|
|
|
|
|
|
|
|
|
|
MistralLite-Q4_0.gguf
DELETED
|
@@ -1,3 +0,0 @@
|
|
| 1 |
-
version https://git-lfs.github.com/spec/v1
|
| 2 |
-
oid sha256:9ad36e5757bdb6723feed9ce725f00cabc337dd441e72dd72928b4a45cc12e11
|
| 3 |
-
size 4108933728
|
|
|
|
|
|
|
|
|
|
|
|
MistralLite-Q4_K_M.gguf
DELETED
|
@@ -1,3 +0,0 @@
|
|
| 1 |
-
version https://git-lfs.github.com/spec/v1
|
| 2 |
-
oid sha256:0f2a30e6b271ec6f6ff757b2e9dc19f8e0df36cbe34e2aa2a06abac5de51245f
|
| 3 |
-
size 4368456288
|
|
|
|
|
|
|
|
|
|
|
|
MistralLite-Q4_K_S.gguf
DELETED
|
@@ -1,3 +0,0 @@
|
|
| 1 |
-
version https://git-lfs.github.com/spec/v1
|
| 2 |
-
oid sha256:9c4ddec14196d23b870ac0380c98b52b04f398757bca4a50b46b393aa1778458
|
| 3 |
-
size 4140391008
|
|
|
|
|
|
|
|
|
|
|
|
MistralLite-Q5_0.gguf
DELETED
|
@@ -1,3 +0,0 @@
|
|
| 1 |
-
version https://git-lfs.github.com/spec/v1
|
| 2 |
-
oid sha256:c27f2bc40d50d2d4a76fe6b1940e06c03b8b3065f76f95f1718b23e780852b32
|
| 3 |
-
size 4997734496
|
|
|
|
|
|
|
|
|
|
|
|
MistralLite-Q5_K_M.gguf
DELETED
|
@@ -1,3 +0,0 @@
|
|
| 1 |
-
version https://git-lfs.github.com/spec/v1
|
| 2 |
-
oid sha256:be4b3a6bb2b16e6a6033fb9eed48e8fdb1424f9c93933330dafbff7523ce198c
|
| 3 |
-
size 5131427936
|
|
|
|
|
|
|
|
|
|
|
|
MistralLite-Q5_K_S.gguf
DELETED
|
@@ -1,3 +0,0 @@
|
|
| 1 |
-
version https://git-lfs.github.com/spec/v1
|
| 2 |
-
oid sha256:2f92b78973fa46b3bc2983c1eb1fbb54281446cfaaeed28d0fa527c0a0076b38
|
| 3 |
-
size 4997734496
|
|
|
|
|
|
|
|
|
|
|
|
MistralLite-Q6_K.gguf
DELETED
|
@@ -1,3 +0,0 @@
|
|
| 1 |
-
version https://git-lfs.github.com/spec/v1
|
| 2 |
-
oid sha256:4695516fe3af4f641c3525b43ea3fdb2307223946db3e75b892d5fac5afe4fc9
|
| 3 |
-
size 5942085312
|
|
|
|
|
|
|
|
|
|
|
|
MistralLite-Q8_0.gguf
DELETED
|
@@ -1,3 +0,0 @@
|
|
| 1 |
-
version https://git-lfs.github.com/spec/v1
|
| 2 |
-
oid sha256:2a1a6d737fcdf181edc2b47cdfb113225830d594a52ca978c773e8f83b0dba14
|
| 3 |
-
size 7695883776
|
|
|
|
|
|
|
|
|
|
|
|