Spaces:
Sleeping
Sleeping
|
Download README.md from mjpsm/activity-progression-api: direct link, hf CLI and curl.
- Browser
- Download file 7.67 kB
-
https://huggingface.co/spaces/mjpsm/activity-progression-api/resolve/main/README.md
- Command line
-
hf download hf://spaces/mjpsm/activity-progression-api/README.md
-
curl -L -o README.md https://huggingface.co/spaces/mjpsm/activity-progression-api/resolve/main/README.md
7.67 kB
| title: MyVillage Activity Progression API | |
| emoji: 🧭 | |
| colorFrom: indigo | |
| colorTo: blue | |
| sdk: docker | |
| app_port: 7860 | |
| pinned: false | |
| # MyVillage Activity Progression API | |
| FastAPI inference service that runs the Progression GGUF model **in-process** | |
| with `llama-cpp-python`. | |
| ```text | |
| Uvicorn | |
| ↓ | |
| FastAPI | |
| ↓ | |
| llama-cpp-python | |
| ↓ | |
| Q8_0 GGUF | |
| ``` | |
| `llama-server` is **no longer used**. There is no subprocess, no internal | |
| inference port, and no second server to start. Uvicorn is the only process. | |
| ## Model | |
| | | | | |
| |---|---| | |
| | Hugging Face repo | `mjpsm/activity-progression-model-600-qwen3.5-0.8b-gguf` | | |
| | GGUF file | `activity-progression-model-600-qwen3.5-0.8b-Q8_0.gguf` | | |
| | Quantization | `Q8_0` | | |
| | Runtime | `llama-cpp-python` | | |
| On startup the API downloads the GGUF with `huggingface_hub.hf_hub_download()` | |
| (cached after the first run) and loads it into memory once. Requests are | |
| serialized through a single inference lock, and the blocking generation call | |
| runs in a worker thread so the event loop stays responsive. | |
| ## Local installation (macOS / Linux) | |
| ```bash | |
| python3 -m venv .venv | |
| source .venv/bin/activate | |
| pip install -r requirements.txt | |
| ``` | |
| ### Apple Silicon notes | |
| `llama-cpp-python` compiles llama.cpp during `pip install`, so you need the | |
| Xcode Command Line Tools (`xcode-select --install`). `cmake` is fetched | |
| automatically by the build. | |
| - Use a native **arm64** Python (`python3 -c "import platform; print(platform.machine())"` | |
| should print `arm64`). An x86_64/Rosetta Python produces a much slower build. | |
| - The default macOS build already includes Metal support. The API still runs | |
| on **CPU by default** (`LLAMA_GPU_LAYERS=0`). To use Metal: | |
| ```bash | |
| LLAMA_GPU_LAYERS=-1 python -m uvicorn app.main:app --host 0.0.0.0 --port 8000 | |
| ``` | |
| - If your install was built without Metal and you want it, rebuild: | |
| ```bash | |
| CMAKE_ARGS="-DGGML_METAL=on" pip install --force-reinstall --no-cache-dir llama-cpp-python | |
| ``` | |
| ## Hugging Face authentication | |
| The model repo is currently public, so no authentication is needed. If it is | |
| made private again, either log in once: | |
| ```bash | |
| hf auth login | |
| ``` | |
| or export a token with read access to the repo: | |
| ```bash | |
| export HF_TOKEN=hf_... | |
| ``` | |
| No token is stored in this repository. Do not commit `.env`. | |
| ## Run | |
| ```bash | |
| python -m uvicorn app.main:app --host 0.0.0.0 --port 8000 | |
| ``` | |
| That is the only command. Check readiness: | |
| ```bash | |
| curl -s http://localhost:8000/health | python3 -m json.tool | |
| ``` | |
| ```json | |
| { | |
| "status": "ok", | |
| "model_ready": true | |
| } | |
| ``` | |
| If the model failed to download or load, the API still starts, logs the | |
| error, and reports `"status": "degraded", "model_ready": false`; | |
| `/v1/progression/generate` then returns `503`. | |
| ## Local curl test | |
| ```bash | |
| curl -s -X POST "http://localhost:8000/v1/progression/generate" -H "Content-Type: application/json" -d '{ | |
| "version": "v1", | |
| "knowledge_submission": "I created the video and showed the finished result first. It received 1,800 views and more engagement than my previous videos. I noticed viewers stayed longer when the final result appeared before the process.", | |
| "wisdom": { | |
| "content": "When I presented in class, showing the most interesting result before explaining the process got peoples attention.", | |
| "chapterTitle": "Lead With the Result", | |
| "BookTypeName": "Testimony", | |
| "BookType": "Personal Record" | |
| }, | |
| "village_goal": "Learn how to create engaging social media content", | |
| "previous_activity_title": "Test a stronger opening hook" | |
| }' | python3 -m json.tool | |
| ``` | |
| The same request is in `scripts/local_curl.sh`. | |
| ## Routes | |
| - `GET /` | |
| - `GET /health` | |
| - `GET /models` | |
| - `GET /docs` | |
| - `POST /v1/progression/generate` | |
| ## Request schema | |
| ```json | |
| { | |
| "version": "v1", | |
| "knowledge_submission": "string", | |
| "wisdom": { | |
| "content": "string", | |
| "chapterTitle": "string", | |
| "BookTypeName": "string", | |
| "BookType": "string" | |
| }, | |
| "village_goal": "string", | |
| "previous_activity_title": "string" | |
| } | |
| ``` | |
| - The `wisdom` field names and capitalization are exact. | |
| - Unknown fields are rejected with `422`. | |
| - An unknown `version` returns `404`. | |
| ### Generation parameters are internal | |
| Generation settings are **not** part of the public API; a request containing | |
| a `generation` field is rejected. They are fixed in `app/config.py`: | |
| | Setting | Value | | |
| |---|---| | |
| | `max_new_tokens` | `450` | | |
| | `temperature` | `0.0` | | |
| | `top_p` | `1.0` | | |
| | `repeat_penalty` | `1.08` | | |
| | `seed` | `42` | | |
| ## Response schema | |
| ```json | |
| { | |
| "progression": { | |
| "knowledge_state": "COMPLETED | PARTIAL | UNCLEAR | BLOCKED | ADVANCED", | |
| "demonstrated_state": ["string"], | |
| "next_gap": "string", | |
| "activity_plan": { | |
| "action": "string", | |
| "target": "string", | |
| "avoid_repeating": ["string"] | |
| } | |
| }, | |
| "usage": { | |
| "input_tokens": 312, | |
| "output_tokens": 147 | |
| } | |
| } | |
| ``` | |
| `usage` is counted with the loaded model's own tokenizer: | |
| - `input_tokens` — tokens in the final raw SFT prompt sent to the model | |
| - `output_tokens` — tokens in the generated completion | |
| Model metadata is not returned here; use `GET /models`. | |
| If the model output contains no parseable JSON object, or the JSON does not | |
| match the Progression schema, the API returns `502` with the raw output for | |
| debugging. Malformed output is never silently repaired. | |
| ## Prompt format | |
| The model was fine-tuned on a raw completion prompt, so the API uses raw | |
| completion (never a chat template or chat-completion API). | |
| `previous_activity_title` is converted to the structure used for SFT: | |
| ```json | |
| { | |
| "village_goal": "...", | |
| "previous_activity": { | |
| "title": "..." | |
| }, | |
| "knowledge_submission": "...", | |
| "wisdom": { | |
| "content": "...", | |
| "chapterTitle": "...", | |
| "BookTypeName": "...", | |
| "BookType": "..." | |
| } | |
| } | |
| ``` | |
| and serialized as compact JSON into: | |
| ```text | |
| <SYSTEM_INSTRUCTION> | |
| INPUT: | |
| <compact JSON> | |
| OUTPUT: | |
| ``` | |
| Do not change this format unless the model is retrained and re-evaluated | |
| with the new one. | |
| ## Configuration | |
| Optional environment variables (see `.env.example`): | |
| | Variable | Default | Purpose | | |
| |---|---|---| | |
| | `HF_TOKEN` | unset | Read token, only needed if the model repo is private | | |
| | `LLAMA_CONTEXT_SIZE` | `2048` | Context window | | |
| | `LLAMA_THREADS` | `2` | CPU threads for generation | | |
| | `LLAMA_GPU_LAYERS` | `0` | Layers to offload (`-1` = all) | | |
| ## Tests | |
| ```bash | |
| pip install -r requirements-dev.txt | |
| python -m pytest | |
| ``` | |
| The tests use a fake runtime and do not download or load the model. | |
| ## Docker / Hugging Face Space deployment | |
| The image contains only Uvicorn + FastAPI + `llama-cpp-python` and listens | |
| on port `7860`. | |
| Local Docker run: | |
| ```bash | |
| docker build -t progression-api . | |
| docker run --rm -p 7860:7860 progression-api | |
| ``` | |
| ### Space setup | |
| 1. Create a **Docker Space**. | |
| 2. Only if the model repo is private, add this secret under | |
| **Space → Settings → Variables and secrets → New secret**: | |
| ```text | |
| HF_TOKEN=<Hugging Face token with read access to the private model repo> | |
| ``` | |
| 3. Push this repo to the Space: | |
| ```bash | |
| git clone https://huggingface.co/spaces/mjpsm/YOUR-SPACE-NAME | |
| cd YOUR-SPACE-NAME | |
| # copy this repo's files in, then: | |
| git add . | |
| git commit -m "Deploy Progression API (llama-cpp-python)" | |
| git push | |
| ``` | |
| Hugging Face builds the Docker image and exposes port `7860`. | |
| ```bash | |
| curl -s "https://YOUR-SPACE-HOST/health" | python3 -m json.tool | |
| ``` | |
| ## CPU behavior | |
| One model instance is loaded and generation is serialized through one | |
| inference lock, so concurrent requests queue rather than compete for | |
| memory/compute on a small Space. Benchmark real Progression requests before | |
| raising `LLAMA_THREADS`. | |