Spaces:
Sleeping
Sleeping
|
Download README.md from mjpsm/activity-instruction-api: direct link, hf CLI and curl.
- Browser
- Download file 6.63 kB
-
https://huggingface.co/spaces/mjpsm/activity-instruction-api/resolve/main/README.md
- Command line
-
hf download hf://spaces/mjpsm/activity-instruction-api/README.md
-
curl -L -o README.md https://huggingface.co/spaces/mjpsm/activity-instruction-api/resolve/main/README.md
6.63 kB
| title: MyVillage Activity Instruction API | |
| emoji: π§ | |
| colorFrom: blue | |
| colorTo: indigo | |
| sdk: docker | |
| app_port: 7860 | |
| models: | |
| - mjpsm/activity-instruction-model-500-qwen3.5-2b-gguf | |
| # MyVillage Activity Instruction API β llama-cpp-python | |
| A FastAPI Docker Space that loads the **Q4_K_M GGUF** directly via `llama_cpp.Llama`. **There is no internal `llama-server` process.** Only Uvicorn serves port 7860. | |
| - Merged model: `mjpsm/activity-instruction-model-500-qwen3.5-2b` | |
| - Evaluated merged revision: `9ee80b07eb127e22c2008ea95fa75c43230b5c0e` | |
| - Public GGUF repo: [`mjpsm/activity-instruction-model-500-qwen3.5-2b-gguf`](https://huggingface.co/mjpsm/activity-instruction-model-500-qwen3.5-2b-gguf) (no token needed) | |
| - Default GGUF filename: `activity-instruction-model-500-qwen3.5-2b-Q4_K_M.gguf` (1.27 GB); `...-Q8_0.gguf` (2.01 GB) is also in the repo | |
| - Inference: 4096 context, 192 tokens maximum, temperature 0, repetition penalty 1.0, seed 42, thinking disabled | |
| ## Run locally with Uvicorn | |
| Requires Python 3.11+. On macOS you also need CMake and the Xcode command line tools (`brew install cmake`, `xcode-select --install`), because `llama-cpp-python` is compiled from source. | |
| ```bash | |
| python3.11 -m venv .venv | |
| source .venv/bin/activate | |
| pip install --upgrade pip | |
| # macOS (Apple Silicon): build llama-cpp-python from source with Metal. Takes several minutes. | |
| CMAKE_ARGS="-DGGML_METAL=on" pip install --no-cache-dir --no-binary llama-cpp-python -r requirements-dev.txt | |
| # Linux x86_64: use the prebuilt CPU wheel instead, as the Dockerfile does. | |
| # pip install --prefer-binary --extra-index-url https://abetlen.github.io/llama-cpp-python/whl/cpu -r requirements-dev.txt | |
| ``` | |
| Start the API: | |
| ```bash | |
| uvicorn main:app --host 127.0.0.1 --port 7860 --workers 1 | |
| ``` | |
| The first start downloads the 1.27 GB Q4_K_M GGUF into the Hugging Face cache (`~/.cache/huggingface`); later starts load it from disk. The API is ready when `GET /health` returns 200. Then, in a second terminal: | |
| ```bash | |
| source .venv/bin/activate | |
| python test_api.py # hits /health, /model and /generate | |
| # or open http://127.0.0.1:7860/docs | |
| ``` | |
| Keep `--workers 1`: each worker loads its own copy of the model. Verified on macOS arm64 with Python 3.11 and `llama-cpp-python` 0.3.36; `/generate` takes roughly 15β20 seconds for the example request with the default `N_THREADS=2`. | |
| ## Endpoints | |
| - `GET /` β service metadata | |
| - `GET /health` β 200 once the model is loaded, otherwise 503 | |
| - `GET /model` β source, GGUF and inference settings | |
| - `POST /generate` β generate validated 1β5 step instructions | |
| - `GET /docs` β interactive Swagger UI | |
| - `POST /predict` β backwards-compatible alias | |
| ## Input | |
| Send the **actual Progression output object** and **Composition `output` object**. Do not include `success_evidence`. | |
| ```json | |
| { | |
| "progression": { | |
| "knowledge_state": "PARTIAL", | |
| "demonstrated_state": ["Created the base animation", "Imported the finalized imagery"], | |
| "next_gap": "Complete the remaining animation and composition work.", | |
| "activity_plan": { | |
| "action": "complete the remaining animation work", | |
| "target": "the year-end review video", | |
| "avoid_repeating": ["creating the base animation", "reimporting finalized imagery"] | |
| } | |
| }, | |
| "composition": { | |
| "activity_title": "Complete the Animation", | |
| "activity_description": "A continuation of the remaining animation and composition work so the year-end review is ready as a complete video." | |
| } | |
| } | |
| ``` | |
| ## Output | |
| Example shape (generated wording may vary): | |
| ```json | |
| {"output":{"instructions":["Complete the remaining animation work for the year-end review video.","Finish the remaining composition work using the existing imagery."]}} | |
| ``` | |
| ## Space settings | |
| 1. Create a Docker Space named `mjpsm/activity-instruction-api`. | |
| 2. Upload **all root-level files** from this package to the Space root. Do not upload the ZIP as the app. | |
| 3. The GGUF repository is public, so no `HF_TOKEN` is needed. If you set `API_KEY` as a secret, clients must supply `X-API-Key` for `/generate`. | |
| 4. The default `MODEL_FILENAME` (Q4_K_M) is present in the repo, so no variable is required. To use the larger quantization, set `MODEL_FILENAME=activity-instruction-model-500-qwen3.5-2b-Q8_0.gguf`. | |
| 5. Start the Space; the GGUF downloads and loads at startup. The Space is ready when `/health` returns 200. | |
| 6. Run `test_api.py` or try `POST /generate` in `/docs`. | |
| Expected URL after Space creation: `https://mjpsm-activity-instruction-api.hf.space/docs`. | |
| ## Configuration | |
| | Variable | Default | Description | | |
| | --- | --- | --- | | |
| | `HF_TOKEN` | unset | Not needed while the GGUF repository is public; only set it if the repo is made private again | | |
| | `MODEL_REPO` | `mjpsm/activity-instruction-model-500-qwen3.5-2b-gguf` | Model repository | | |
| | `MODEL_FILENAME` | `activity-instruction-model-500-qwen3.5-2b-Q4_K_M.gguf` | Exact quantized filename | | |
| | `MODEL_REVISION` | unset | Optional GGUF commit hash pin | | |
| | `N_CTX` | `4096` | Context size | | |
| | `N_THREADS` | `2` | CPU threads | | |
| | `MAX_NEW_TOKENS` | `192` | Maximum output tokens | | |
| | `REPEAT_PENALTY` | `1.0` | Repetition penalty | | |
| | `SEED` | `42` | Deterministic seed | | |
| | `QUEUE_TIMEOUT` | `90` | Wait for concurrent inference (single model lock) | | |
| | `API_KEY` | unset | Optional inbound authentication; strongly recommended on public Spaces | | |
| | `ALLOWED_ORIGINS` | `*` | Optional CORS restriction | | |
| ### Why the prompt differs from Composition | |
| The Composition Model was evaluated with a plain `INPUT:` / `OUTPUT:` format. The Instruction Model was evaluated as a **system + user chat**, with `enable_thinking=False`. `llama-cpp-python`'s raw `create_completion` is used for simplicity, but its prompt contains the Qwen3.5 chat role markers and an **empty `<think></think>` prefix** to represent disabled thinking. The system prompt is copied exactly from the final evaluation. This requires live comparison against the saved GGUF regression outputs because it uses a different inference binding and prompt renderer than the original `llama-server` benchmark. | |
| The API **rejects** malformed model JSON with HTTP 502; it never silently substitutes a generic fallback activity or forwards invalid output to the Activity Generation wrapper. Test important diagnostic examples before deploying beyond a reviewed beta. | |
| ## Local Docker | |
| ```bash | |
| docker build -t myvillage-instruction . | |
| docker run --rm -p 7860:7860 myvillage-instruction | |
| ``` | |
| If you add an `API_KEY`, do not commit it into source control. For Space deployment, use Settings β Variables and secrets instead of placing secrets in code. | |