Hugging Face's logo

ggml-org-dev
/
stories15M

GGUF
llama.cpp
test-model
Model card Files
xet
 Community
How to use from
llama.cpp
Install (macOS, Linux) 
curl -LsSf https://llama.app/install.sh | sh
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf ggml-org-dev/stories15M:Q8_0
# Run inference directly in the terminal:
llama cli -hf ggml-org-dev/stories15M:Q8_0
Install from WinGet (Windows) 
winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf ggml-org-dev/stories15M:Q8_0
# Run inference directly in the terminal:
llama cli -hf ggml-org-dev/stories15M:Q8_0
Build from source code 
git clone https://github.com/ggml-org/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-app
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama serve -hf ggml-org-dev/stories15M:Q8_0
# Run inference directly in the terminal:
./build/bin/llama cli -hf ggml-org-dev/stories15M:Q8_0
Use Docker 
docker model run hf.co/ggml-org-dev/stories15M:Q8_0
Quick Links

stories15M

Tiny model used by the llama.cpp test suite and CI. It is not meant for real-world use.

Originally hosted at ggml-org/models/tinyllamas/.

  • stories15M*.gguf: karpathy/tinyllamas stories15M (F32, Q4_0, Q8_0)
  • stories15M-be*.gguf: big-endian variants (s390x CI)
  • split/: the same model split into 3 shards (F32 and Q8_0), for split-loading tests
Downloads last month
5,230
GGUF
Model size
24.4M params
Architecture
llama
Hardware compatibility
Log In to add your hardware

4-bit

8-bit

View +3 variants
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support