--- title: MyVillage Activity Progression API emoji: 🧭 colorFrom: indigo colorTo: blue sdk: docker app_port: 7860 pinned: false --- # MyVillage Activity Progression API FastAPI inference service that runs the Progression GGUF model **in-process** with `llama-cpp-python`. ```text Uvicorn ↓ FastAPI ↓ llama-cpp-python ↓ Q8_0 GGUF ``` `llama-server` is **no longer used**. There is no subprocess, no internal inference port, and no second server to start. Uvicorn is the only process. ## Model | | | |---|---| | Hugging Face repo | `mjpsm/activity-progression-model-600-qwen3.5-0.8b-gguf` | | GGUF file | `activity-progression-model-600-qwen3.5-0.8b-Q8_0.gguf` | | Quantization | `Q8_0` | | Runtime | `llama-cpp-python` | On startup the API downloads the GGUF with `huggingface_hub.hf_hub_download()` (cached after the first run) and loads it into memory once. Requests are serialized through a single inference lock, and the blocking generation call runs in a worker thread so the event loop stays responsive. ## Local installation (macOS / Linux) ```bash python3 -m venv .venv source .venv/bin/activate pip install -r requirements.txt ``` ### Apple Silicon notes `llama-cpp-python` compiles llama.cpp during `pip install`, so you need the Xcode Command Line Tools (`xcode-select --install`). `cmake` is fetched automatically by the build. - Use a native **arm64** Python (`python3 -c "import platform; print(platform.machine())"` should print `arm64`). An x86_64/Rosetta Python produces a much slower build. - The default macOS build already includes Metal support. The API still runs on **CPU by default** (`LLAMA_GPU_LAYERS=0`). To use Metal: ```bash LLAMA_GPU_LAYERS=-1 python -m uvicorn app.main:app --host 0.0.0.0 --port 8000 ``` - If your install was built without Metal and you want it, rebuild: ```bash CMAKE_ARGS="-DGGML_METAL=on" pip install --force-reinstall --no-cache-dir llama-cpp-python ``` ## Hugging Face authentication The model repo is currently public, so no authentication is needed. If it is made private again, either log in once: ```bash hf auth login ``` or export a token with read access to the repo: ```bash export HF_TOKEN=hf_... ``` No token is stored in this repository. Do not commit `.env`. ## Run ```bash python -m uvicorn app.main:app --host 0.0.0.0 --port 8000 ``` That is the only command. Check readiness: ```bash curl -s http://localhost:8000/health | python3 -m json.tool ``` ```json { "status": "ok", "model_ready": true } ``` If the model failed to download or load, the API still starts, logs the error, and reports `"status": "degraded", "model_ready": false`; `/v1/progression/generate` then returns `503`. ## Local curl test ```bash curl -s -X POST "http://localhost:8000/v1/progression/generate" -H "Content-Type: application/json" -d '{ "version": "v1", "knowledge_submission": "I created the video and showed the finished result first. It received 1,800 views and more engagement than my previous videos. I noticed viewers stayed longer when the final result appeared before the process.", "wisdom": { "content": "When I presented in class, showing the most interesting result before explaining the process got peoples attention.", "chapterTitle": "Lead With the Result", "BookTypeName": "Testimony", "BookType": "Personal Record" }, "village_goal": "Learn how to create engaging social media content", "previous_activity_title": "Test a stronger opening hook" }' | python3 -m json.tool ``` The same request is in `scripts/local_curl.sh`. ## Routes - `GET /` - `GET /health` - `GET /models` - `GET /docs` - `POST /v1/progression/generate` ## Request schema ```json { "version": "v1", "knowledge_submission": "string", "wisdom": { "content": "string", "chapterTitle": "string", "BookTypeName": "string", "BookType": "string" }, "village_goal": "string", "previous_activity_title": "string" } ``` - The `wisdom` field names and capitalization are exact. - Unknown fields are rejected with `422`. - An unknown `version` returns `404`. ### Generation parameters are internal Generation settings are **not** part of the public API; a request containing a `generation` field is rejected. They are fixed in `app/config.py`: | Setting | Value | |---|---| | `max_new_tokens` | `450` | | `temperature` | `0.0` | | `top_p` | `1.0` | | `repeat_penalty` | `1.08` | | `seed` | `42` | ## Response schema ```json { "progression": { "knowledge_state": "COMPLETED | PARTIAL | UNCLEAR | BLOCKED | ADVANCED", "demonstrated_state": ["string"], "next_gap": "string", "activity_plan": { "action": "string", "target": "string", "avoid_repeating": ["string"] } }, "usage": { "input_tokens": 312, "output_tokens": 147 } } ``` `usage` is counted with the loaded model's own tokenizer: - `input_tokens` — tokens in the final raw SFT prompt sent to the model - `output_tokens` — tokens in the generated completion Model metadata is not returned here; use `GET /models`. If the model output contains no parseable JSON object, or the JSON does not match the Progression schema, the API returns `502` with the raw output for debugging. Malformed output is never silently repaired. ## Prompt format The model was fine-tuned on a raw completion prompt, so the API uses raw completion (never a chat template or chat-completion API). `previous_activity_title` is converted to the structure used for SFT: ```json { "village_goal": "...", "previous_activity": { "title": "..." }, "knowledge_submission": "...", "wisdom": { "content": "...", "chapterTitle": "...", "BookTypeName": "...", "BookType": "..." } } ``` and serialized as compact JSON into: ```text INPUT: OUTPUT: ``` Do not change this format unless the model is retrained and re-evaluated with the new one. ## Configuration Optional environment variables (see `.env.example`): | Variable | Default | Purpose | |---|---|---| | `HF_TOKEN` | unset | Read token, only needed if the model repo is private | | `LLAMA_CONTEXT_SIZE` | `2048` | Context window | | `LLAMA_THREADS` | `2` | CPU threads for generation | | `LLAMA_GPU_LAYERS` | `0` | Layers to offload (`-1` = all) | ## Tests ```bash pip install -r requirements-dev.txt python -m pytest ``` The tests use a fake runtime and do not download or load the model. ## Docker / Hugging Face Space deployment The image contains only Uvicorn + FastAPI + `llama-cpp-python` and listens on port `7860`. Local Docker run: ```bash docker build -t progression-api . docker run --rm -p 7860:7860 progression-api ``` ### Space setup 1. Create a **Docker Space**. 2. Only if the model repo is private, add this secret under **Space → Settings → Variables and secrets → New secret**: ```text HF_TOKEN= ``` 3. Push this repo to the Space: ```bash git clone https://huggingface.co/spaces/mjpsm/YOUR-SPACE-NAME cd YOUR-SPACE-NAME # copy this repo's files in, then: git add . git commit -m "Deploy Progression API (llama-cpp-python)" git push ``` Hugging Face builds the Docker image and exposes port `7860`. ```bash curl -s "https://YOUR-SPACE-HOST/health" | python3 -m json.tool ``` ## CPU behavior One model instance is loaded and generation is serialized through one inference lock, so concurrent requests queue rather than compete for memory/compute on a small Space. Benchmark real Progression requests before raising `LLAMA_THREADS`.