Instructions to use PrunaAI/gpt2-GGUF-smashed with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Pruna AI
How to use PrunaAI/gpt2-GGUF-smashed with Pruna AI:
from pruna import PrunaModel model = PrunaModel.from_pretrained("PrunaAI/gpt2-GGUF-smashed") - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- llama.cpp
How to use PrunaAI/gpt2-GGUF-smashed with llama.cpp:
Install (macOS, Linux)
curl -LsSf https://llama.app/install.sh | sh # Start a local OpenAI-compatible server with a web UI: llama serve -hf PrunaAI/gpt2-GGUF-smashed:Q4_K_M # Run inference directly in the terminal: llama cli -hf PrunaAI/gpt2-GGUF-smashed:Q4_K_M
Install from WinGet (Windows)
winget install llama.cpp # Start a local OpenAI-compatible server with a web UI: llama serve -hf PrunaAI/gpt2-GGUF-smashed:Q4_K_M # Run inference directly in the terminal: llama cli -hf PrunaAI/gpt2-GGUF-smashed:Q4_K_M
Use pre-built binary
# Download pre-built binary from: # https://github.com/ggerganov/llama.cpp/releases # Start a local OpenAI-compatible server with a web UI: ./llama-server -hf PrunaAI/gpt2-GGUF-smashed:Q4_K_M # Run inference directly in the terminal: ./llama-cli -hf PrunaAI/gpt2-GGUF-smashed:Q4_K_M
Build from source code
git clone https://github.com/ggerganov/llama.cpp.git cd llama.cpp cmake -B build cmake --build build -j --target llama-server llama-cli # Start a local OpenAI-compatible server with a web UI: ./build/bin/llama-server -hf PrunaAI/gpt2-GGUF-smashed:Q4_K_M # Run inference directly in the terminal: ./build/bin/llama-cli -hf PrunaAI/gpt2-GGUF-smashed:Q4_K_M
Use Docker
docker model run hf.co/PrunaAI/gpt2-GGUF-smashed:Q4_K_M
- LM Studio
- Jan
- Ollama
How to use PrunaAI/gpt2-GGUF-smashed with Ollama:
ollama run hf.co/PrunaAI/gpt2-GGUF-smashed:Q4_K_M
- Unsloth Desktop
- Docker Model Runner
How to use PrunaAI/gpt2-GGUF-smashed with Docker Model Runner:
docker model run hf.co/PrunaAI/gpt2-GGUF-smashed:Q4_K_M
- Lemonade
How to use PrunaAI/gpt2-GGUF-smashed with Lemonade:
Pull the model
# Download Lemonade from https://lemonade-server.ai/ lemonade pull PrunaAI/gpt2-GGUF-smashed:Q4_K_M
Run and chat with the model
lemonade run user.gpt2-GGUF-smashed-Q4_K_M
List all available models
lemonade list
- Atomic Chat
Commit History
-
Update additional links
Verified signature6614934 -
Update additional links
Verified signature0970a16 -
Update CTA badges
Verified signature2bd0cc3
-
Replace correct UTM link
Verified signaturef1346ac -
Add CTA to model card
Verified signaturec7daaef
-
Upload README.md with huggingface_hub
Verified signature74903be -
Upload gpt2.Q5_K_S.gguf with huggingface_hub
Verified signature0c51d0a -
Upload gpt2.Q5_K_M.gguf with huggingface_hub
Verified signature4a31ce0 -
Upload gpt2.Q5_1.gguf with huggingface_hub
Verified signature3fb4eab -
Upload gpt2.Q5_0.gguf with huggingface_hub
Verified signature98cecf2 -
Upload gpt2.Q4_K_S.gguf with huggingface_hub
Verified signature9bf6146 -
Upload gpt2.Q4_K_M.gguf with huggingface_hub
Verified signature5fc26ca -
Upload gpt2.Q4_1.gguf with huggingface_hub
Verified signaturef908fbc -
Upload gpt2.Q4_0.gguf with huggingface_hub
Verified signatureb8dff90 -
Upload gpt2.Q3_K_S.gguf with huggingface_hub
Verified signature6d7c160 -
Upload gpt2.Q3_K_M.gguf with huggingface_hub
Verified signature88438d4 -
Upload gpt2.Q3_K_L.gguf with huggingface_hub
Verified signaturee004aa8 -
Upload gpt2.Q2_K.gguf with huggingface_hub
Verified signature8c27800 -
Upload gpt2.fp16.bin with huggingface_hub
Verified signature709fff9
-
Update README.md
Verified signaturef06476e
-
Update README.md
Verified signature948c270 -
Update README.md
Verified signaturea6ad050
-
Upload gpt2.fp16.bin with huggingface_hub
Verified signature08f9f04 -
Upload gpt2.Q2_K.gguf with huggingface_hub
Verified signature328d530 -
Upload gpt2.Q3_K_L.gguf with huggingface_hub
Verified signatureb5c46ce -
Upload gpt2.Q5_1.gguf with huggingface_hub
Verified signature472f24c -
Upload gpt2.Q6_K.gguf with huggingface_hub
Verified signature67b4ad0 -
Upload gpt2.Q4_K_M.gguf with huggingface_hub
Verified signature9a4c001 -
Upload gpt2.Q4_0.gguf with huggingface_hub
Verified signature79f77bf -
Upload gpt2.Q3_K_M.gguf with huggingface_hub
Verified signature7ccafab -
Upload gpt2.Q5_0.gguf with huggingface_hub
Verified signature2393e12 -
Upload gpt2.Q4_K_S.gguf with huggingface_hub
Verified signature591d796 -
Upload gpt2.Q4_1.gguf with huggingface_hub
Verified signature9658e0f -
Upload gpt2.Q8_0.gguf with huggingface_hub
Verified signatureb52c97b -
Upload gpt2.Q5_K_M.gguf with huggingface_hub
Verified signature06f9f74 -
Upload gpt2.Q3_K_S.gguf with huggingface_hub
Verified signatured4a9534 -
Upload gpt2.Q5_K_S.gguf with huggingface_hub
Verified signature5c571be -
Upload README.md with huggingface_hub
Verified signaturedd66f2e -
initial commit
Verified signature9d8220a