--- title: MyVillage Activity Instruction API emoji: 🧭 colorFrom: blue colorTo: indigo sdk: docker app_port: 7860 models: - mjpsm/activity-instruction-model-500-qwen3.5-2b-gguf --- # MyVillage Activity Instruction API — llama-cpp-python A FastAPI Docker Space that loads the **Q4_K_M GGUF** directly via `llama_cpp.Llama`. **There is no internal `llama-server` process.** Only Uvicorn serves port 7860. - Merged model: `mjpsm/activity-instruction-model-500-qwen3.5-2b` - Evaluated merged revision: `9ee80b07eb127e22c2008ea95fa75c43230b5c0e` - Public GGUF repo: [`mjpsm/activity-instruction-model-500-qwen3.5-2b-gguf`](https://huggingface.co/mjpsm/activity-instruction-model-500-qwen3.5-2b-gguf) (no token needed) - Default GGUF filename: `activity-instruction-model-500-qwen3.5-2b-Q4_K_M.gguf` (1.27 GB); `...-Q8_0.gguf` (2.01 GB) is also in the repo - Inference: 4096 context, 192 tokens maximum, temperature 0, repetition penalty 1.0, seed 42, thinking disabled ## Run locally with Uvicorn Requires Python 3.11+. On macOS you also need CMake and the Xcode command line tools (`brew install cmake`, `xcode-select --install`), because `llama-cpp-python` is compiled from source. ```bash python3.11 -m venv .venv source .venv/bin/activate pip install --upgrade pip # macOS (Apple Silicon): build llama-cpp-python from source with Metal. Takes several minutes. CMAKE_ARGS="-DGGML_METAL=on" pip install --no-cache-dir --no-binary llama-cpp-python -r requirements-dev.txt # Linux x86_64: use the prebuilt CPU wheel instead, as the Dockerfile does. # pip install --prefer-binary --extra-index-url https://abetlen.github.io/llama-cpp-python/whl/cpu -r requirements-dev.txt ``` Start the API: ```bash uvicorn main:app --host 127.0.0.1 --port 7860 --workers 1 ``` The first start downloads the 1.27 GB Q4_K_M GGUF into the Hugging Face cache (`~/.cache/huggingface`); later starts load it from disk. The API is ready when `GET /health` returns 200. Then, in a second terminal: ```bash source .venv/bin/activate python test_api.py # hits /health, /model and /generate # or open http://127.0.0.1:7860/docs ``` Keep `--workers 1`: each worker loads its own copy of the model. Verified on macOS arm64 with Python 3.11 and `llama-cpp-python` 0.3.36; `/generate` takes roughly 15–20 seconds for the example request with the default `N_THREADS=2`. ## Endpoints - `GET /` — service metadata - `GET /health` — 200 once the model is loaded, otherwise 503 - `GET /model` — source, GGUF and inference settings - `POST /generate` — generate validated 1–5 step instructions - `GET /docs` — interactive Swagger UI - `POST /predict` — backwards-compatible alias ## Input Send the **actual Progression output object** and **Composition `output` object**. Do not include `success_evidence`. ```json { "progression": { "knowledge_state": "PARTIAL", "demonstrated_state": ["Created the base animation", "Imported the finalized imagery"], "next_gap": "Complete the remaining animation and composition work.", "activity_plan": { "action": "complete the remaining animation work", "target": "the year-end review video", "avoid_repeating": ["creating the base animation", "reimporting finalized imagery"] } }, "composition": { "activity_title": "Complete the Animation", "activity_description": "A continuation of the remaining animation and composition work so the year-end review is ready as a complete video." } } ``` ## Output Example shape (generated wording may vary): ```json {"output":{"instructions":["Complete the remaining animation work for the year-end review video.","Finish the remaining composition work using the existing imagery."]}} ``` ## Space settings 1. Create a Docker Space named `mjpsm/activity-instruction-api`. 2. Upload **all root-level files** from this package to the Space root. Do not upload the ZIP as the app. 3. The GGUF repository is public, so no `HF_TOKEN` is needed. If you set `API_KEY` as a secret, clients must supply `X-API-Key` for `/generate`. 4. The default `MODEL_FILENAME` (Q4_K_M) is present in the repo, so no variable is required. To use the larger quantization, set `MODEL_FILENAME=activity-instruction-model-500-qwen3.5-2b-Q8_0.gguf`. 5. Start the Space; the GGUF downloads and loads at startup. The Space is ready when `/health` returns 200. 6. Run `test_api.py` or try `POST /generate` in `/docs`. Expected URL after Space creation: `https://mjpsm-activity-instruction-api.hf.space/docs`. ## Configuration | Variable | Default | Description | | --- | --- | --- | | `HF_TOKEN` | unset | Not needed while the GGUF repository is public; only set it if the repo is made private again | | `MODEL_REPO` | `mjpsm/activity-instruction-model-500-qwen3.5-2b-gguf` | Model repository | | `MODEL_FILENAME` | `activity-instruction-model-500-qwen3.5-2b-Q4_K_M.gguf` | Exact quantized filename | | `MODEL_REVISION` | unset | Optional GGUF commit hash pin | | `N_CTX` | `4096` | Context size | | `N_THREADS` | `2` | CPU threads | | `MAX_NEW_TOKENS` | `192` | Maximum output tokens | | `REPEAT_PENALTY` | `1.0` | Repetition penalty | | `SEED` | `42` | Deterministic seed | | `QUEUE_TIMEOUT` | `90` | Wait for concurrent inference (single model lock) | | `API_KEY` | unset | Optional inbound authentication; strongly recommended on public Spaces | | `ALLOWED_ORIGINS` | `*` | Optional CORS restriction | ### Why the prompt differs from Composition The Composition Model was evaluated with a plain `INPUT:` / `OUTPUT:` format. The Instruction Model was evaluated as a **system + user chat**, with `enable_thinking=False`. `llama-cpp-python`'s raw `create_completion` is used for simplicity, but its prompt contains the Qwen3.5 chat role markers and an **empty `` prefix** to represent disabled thinking. The system prompt is copied exactly from the final evaluation. This requires live comparison against the saved GGUF regression outputs because it uses a different inference binding and prompt renderer than the original `llama-server` benchmark. The API **rejects** malformed model JSON with HTTP 502; it never silently substitutes a generic fallback activity or forwards invalid output to the Activity Generation wrapper. Test important diagnostic examples before deploying beyond a reviewed beta. ## Local Docker ```bash docker build -t myvillage-instruction . docker run --rm -p 7860:7860 myvillage-instruction ``` If you add an `API_KEY`, do not commit it into source control. For Space deployment, use Settings → Variables and secrets instead of placing secrets in code.