README.md

metadata

title: Lightweight LLM
emoji: 🪶
colorFrom: blue
colorTo: pink
sdk: docker
app_port: 7860
pinned: false

Lightweight LLM Server

Self-hosted small LLM (Phi-3.5-mini, GGUF/Q4) served via llama.cpp for CPU inference.

• Endpoint: POST /generate with {"prompt": "..."}