Instructions to use knifeayumu/LLM_Collection with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- llama.cpp
How to use knifeayumu/LLM_Collection with llama.cpp:
Install (macOS, Linux)
curl -LsSf https://llama.app/install.sh | sh # Start a local OpenAI-compatible server with a web UI: llama serve -hf knifeayumu/LLM_Collection:Q4_K_M # Run inference directly in the terminal: llama cli -hf knifeayumu/LLM_Collection:Q4_K_M
Install from WinGet (Windows)
winget install llama.cpp # Start a local OpenAI-compatible server with a web UI: llama serve -hf knifeayumu/LLM_Collection:Q4_K_M # Run inference directly in the terminal: llama cli -hf knifeayumu/LLM_Collection:Q4_K_M
Build from source code
git clone https://github.com/ggml-org/llama.cpp.git cd llama.cpp cmake -B build cmake --build build -j --target llama-app # Start a local OpenAI-compatible server with a web UI: ./build/bin/llama serve -hf knifeayumu/LLM_Collection:Q4_K_M # Run inference directly in the terminal: ./build/bin/llama cli -hf knifeayumu/LLM_Collection:Q4_K_M
Use Docker
docker model run hf.co/knifeayumu/LLM_Collection:Q4_K_M
- Llama new
- LM Studio
- Jan
- Ollama
How to use knifeayumu/LLM_Collection with Ollama:
ollama run hf.co/knifeayumu/LLM_Collection:Q4_K_M
- Unsloth Desktop
- Docker Model Runner
How to use knifeayumu/LLM_Collection with Docker Model Runner:
docker model run hf.co/knifeayumu/LLM_Collection:Q4_K_M
- Lemonade
How to use knifeayumu/LLM_Collection with Lemonade:
Pull the model
# Download Lemonade from https://lemonade-server.ai/ lemonade pull knifeayumu/LLM_Collection:Q4_K_M
Run and chat with the model
lemonade run user.LLM_Collection-Q4_K_M
List all available models
lemonade list
- Atomic Chat
Commit History
-
Rename 2024b/kunoichi-dpo-v2-7b.Q8_0.gguf to Mistral-7B/kunoichi-dpo-v2-7b.Q8_0.gguf
Verified signature8f4ff74 -
Rename 2024b/WizardLM-2-7B.Q8_0.gguf to Mistral-7B/WizardLM-2-7B.Q8_0.gguf
Verified signaturedfd18f4 -
Rename 2024b/LemonadeRP-4.5.3.Q8_0.gguf to Mistral-7B/LemonadeRP-4.5.3.Q8_0.gguf
Verified signature9f00411 -
Rename 2024b/LemonWizardv3.Q8_0.gguf to Mistral-7B/LemonWizardv3.Q8_0.gguf
Verified signature6e5bed2 -
Rename 2024b/LemonKunoichiWizardV3.Q8_0.gguf to Mistral-7B/LemonKunoichiWizardV3.Q8_0.gguf
Verified signature3380fc1 -
Rename Unused/silicon-maid-7b.Q6_K.gguf to Mistral-7B/silicon-maid-7b.Q6_K.gguf
Verified signature352211e -
Rename Unused/openhermes-2.5-neural-chat-7b-v3-2-7b.Q6_K.gguf to Mistral-7B/openhermes-2.5-neural-chat-7b-v3-2-7b.Q6_K.gguf
Verified signature5428831 -
Rename Unused/openhermes-2.5-neural-chat-7b-v3-1-7b.Q6_K.gguf to Mistral-7B/openhermes-2.5-neural-chat-7b-v3-1-7b.Q6_K.gguf
Verified signature46dc746 -
Rename Unused/openchat-3.5-0106.Q6_K.gguf to Mistral-7B/openchat-3.5-0106.Q6_K.gguf
Verified signaturec3c94a2 -
Rename Unused/neuralhermes-2.5-mistral-7b.Q6_K.gguf to Mistral-7B/neuralhermes-2.5-mistral-7b.Q6_K.gguf
Verified signature38e1192 -
Rename Unused/mistral-7b-v0.2-layla-v4-Q6_K.gguf to Mistral-7B/mistral-7b-v0.2-layla-v4-Q6_K.gguf
Verified signature399e691 -
Rename Unused/mistral-7b-v0.1-layla-v4-chatml-Q6_K.gguf to Mistral-7B/mistral-7b-v0.1-layla-v4-chatml-Q6_K.gguf
Verified signatured88580c -
Rename Unused/mistral-7b-instruct-v0.2.Q6_K.gguf to Mistral-7B/mistral-7b-instruct-v0.2.Q6_K.gguf
Verified signatured0bb631 -
Rename Unused/mistral-7b-instruct-v0.1.Q6_K.gguf to Mistral-7B/mistral-7b-instruct-v0.1.Q6_K.gguf
Verified signaturef61b9a6 -
Rename Unused/kunoichi-7b.Q6_K.gguf to Mistral-7B/kunoichi-7b.Q6_K.gguf
Verified signaturedb54251 -
Rename Unused/WestLake-7B-v2-Q6_K.gguf to Mistral-7B/WestLake-7B-v2-Q6_K.gguf
Verified signature736f753 -
Rename Unused/Toppy-M-7B.q6_k.gguf to Mistral-7B/Toppy-M-7B.q6_k.gguf
Verified signature1354d5f -
Rename Unused/Starling-LM-7B-beta-Q6_K.gguf to Mistral-7B/Starling-LM-7B-beta-Q6_K.gguf
Verified signature24ed0b1 -
Rename Unused/ShoriRP.v077.q6_k.gguf to Mistral-7B/ShoriRP.v077.q6_k.gguf
Verified signature3d34398 -
Rename Unused/Prodigy_7B-Q6_K-imatrix.gguf to Mistral-7B/Prodigy_7B-Q6_K-imatrix.gguf
Verified signature218e6c8 -
Rename Unused/Nous-Hermes-2-Mistral-7B-DPO.Q6_K.gguf to Mistral-7B/Nous-Hermes-2-Mistral-7B-DPO.Q6_K.gguf
Verified signaturea3d3f9b -
Rename Unused/neural-chat-7b-v3-16k-q8_0.gguf to Mistral-7B/neural-chat-7b-v3-16k-q8_0.gguf
Verified signature7be7896 -
Rename Unused/neural-chat-7b-v3-1-16k-q8_0.gguf to Mistral-7B/neural-chat-7b-v3-1-16k-q8_0.gguf
Verified signature2251194 -
Rename Unused/Fett-uccine-7B-Q6_K.gguf to Mistral-7B/Fett-uccine-7B-Q6_K.gguf
Verified signature9f011db -
Rename Unused/Erosumika-7B.q8_0.gguf to Mistral-7B/Erosumika-7B.q8_0.gguf
Verified signaturecfc7301 -
Rename Unused/Eris_7B-Q6_K-imatrix.gguf to Mistral-7B/Eris_7B-Q6_K-imatrix.gguf
Verified signaturef02ee0b