Spaces:
Sleeping
Download README.md from mjpsm/activity-instruction-api: direct link, hf CLI and curl.
- Browser
- Download file 6.63 kB
-
https://huggingface.co/spaces/mjpsm/activity-instruction-api/resolve/main/README.md
- Command line
-
hf download hf://spaces/mjpsm/activity-instruction-api/README.md
-
curl -L -o README.md https://huggingface.co/spaces/mjpsm/activity-instruction-api/resolve/main/README.md
title: MyVillage Activity Instruction API
emoji: π§
colorFrom: blue
colorTo: indigo
sdk: docker
app_port: 7860
models:
- mjpsm/activity-instruction-model-500-qwen3.5-2b-gguf
MyVillage Activity Instruction API β llama-cpp-python
A FastAPI Docker Space that loads the Q4_K_M GGUF directly via llama_cpp.Llama. There is no internal llama-server process. Only Uvicorn serves port 7860.
- Merged model:
mjpsm/activity-instruction-model-500-qwen3.5-2b - Evaluated merged revision:
9ee80b07eb127e22c2008ea95fa75c43230b5c0e - Public GGUF repo:
mjpsm/activity-instruction-model-500-qwen3.5-2b-gguf(no token needed) - Default GGUF filename:
activity-instruction-model-500-qwen3.5-2b-Q4_K_M.gguf(1.27 GB);...-Q8_0.gguf(2.01 GB) is also in the repo - Inference: 4096 context, 192 tokens maximum, temperature 0, repetition penalty 1.0, seed 42, thinking disabled
Run locally with Uvicorn
Requires Python 3.11+. On macOS you also need CMake and the Xcode command line tools (brew install cmake, xcode-select --install), because llama-cpp-python is compiled from source.
python3.11 -m venv .venv
source .venv/bin/activate
pip install --upgrade pip
# macOS (Apple Silicon): build llama-cpp-python from source with Metal. Takes several minutes.
CMAKE_ARGS="-DGGML_METAL=on" pip install --no-cache-dir --no-binary llama-cpp-python -r requirements-dev.txt
# Linux x86_64: use the prebuilt CPU wheel instead, as the Dockerfile does.
# pip install --prefer-binary --extra-index-url https://abetlen.github.io/llama-cpp-python/whl/cpu -r requirements-dev.txt
Start the API:
uvicorn main:app --host 127.0.0.1 --port 7860 --workers 1
The first start downloads the 1.27 GB Q4_K_M GGUF into the Hugging Face cache (~/.cache/huggingface); later starts load it from disk. The API is ready when GET /health returns 200. Then, in a second terminal:
source .venv/bin/activate
python test_api.py # hits /health, /model and /generate
# or open http://127.0.0.1:7860/docs
Keep --workers 1: each worker loads its own copy of the model. Verified on macOS arm64 with Python 3.11 and llama-cpp-python 0.3.36; /generate takes roughly 15β20 seconds for the example request with the default N_THREADS=2.
Endpoints
GET /β service metadataGET /healthβ 200 once the model is loaded, otherwise 503GET /modelβ source, GGUF and inference settingsPOST /generateβ generate validated 1β5 step instructionsGET /docsβ interactive Swagger UIPOST /predictβ backwards-compatible alias
Input
Send the actual Progression output object and Composition output object. Do not include success_evidence.
{
"progression": {
"knowledge_state": "PARTIAL",
"demonstrated_state": ["Created the base animation", "Imported the finalized imagery"],
"next_gap": "Complete the remaining animation and composition work.",
"activity_plan": {
"action": "complete the remaining animation work",
"target": "the year-end review video",
"avoid_repeating": ["creating the base animation", "reimporting finalized imagery"]
}
},
"composition": {
"activity_title": "Complete the Animation",
"activity_description": "A continuation of the remaining animation and composition work so the year-end review is ready as a complete video."
}
}
Output
Example shape (generated wording may vary):
{"output":{"instructions":["Complete the remaining animation work for the year-end review video.","Finish the remaining composition work using the existing imagery."]}}
Space settings
- Create a Docker Space named
mjpsm/activity-instruction-api. - Upload all root-level files from this package to the Space root. Do not upload the ZIP as the app.
- The GGUF repository is public, so no
HF_TOKENis needed. If you setAPI_KEYas a secret, clients must supplyX-API-Keyfor/generate. - The default
MODEL_FILENAME(Q4_K_M) is present in the repo, so no variable is required. To use the larger quantization, setMODEL_FILENAME=activity-instruction-model-500-qwen3.5-2b-Q8_0.gguf. - Start the Space; the GGUF downloads and loads at startup. The Space is ready when
/healthreturns 200. - Run
test_api.pyor tryPOST /generatein/docs.
Expected URL after Space creation: https://mjpsm-activity-instruction-api.hf.space/docs.
Configuration
| Variable | Default | Description |
|---|---|---|
HF_TOKEN |
unset | Not needed while the GGUF repository is public; only set it if the repo is made private again |
MODEL_REPO |
mjpsm/activity-instruction-model-500-qwen3.5-2b-gguf |
Model repository |
MODEL_FILENAME |
activity-instruction-model-500-qwen3.5-2b-Q4_K_M.gguf |
Exact quantized filename |
MODEL_REVISION |
unset | Optional GGUF commit hash pin |
N_CTX |
4096 |
Context size |
N_THREADS |
2 |
CPU threads |
MAX_NEW_TOKENS |
192 |
Maximum output tokens |
REPEAT_PENALTY |
1.0 |
Repetition penalty |
SEED |
42 |
Deterministic seed |
QUEUE_TIMEOUT |
90 |
Wait for concurrent inference (single model lock) |
API_KEY |
unset | Optional inbound authentication; strongly recommended on public Spaces |
ALLOWED_ORIGINS |
* |
Optional CORS restriction |
Why the prompt differs from Composition
The Composition Model was evaluated with a plain INPUT: / OUTPUT: format. The Instruction Model was evaluated as a system + user chat, with enable_thinking=False. llama-cpp-python's raw create_completion is used for simplicity, but its prompt contains the Qwen3.5 chat role markers and an empty <think></think> prefix to represent disabled thinking. The system prompt is copied exactly from the final evaluation. This requires live comparison against the saved GGUF regression outputs because it uses a different inference binding and prompt renderer than the original llama-server benchmark.
The API rejects malformed model JSON with HTTP 502; it never silently substitutes a generic fallback activity or forwards invalid output to the Activity Generation wrapper. Test important diagnostic examples before deploying beyond a reviewed beta.
Local Docker
docker build -t myvillage-instruction .
docker run --rm -p 7860:7860 myvillage-instruction
If you add an API_KEY, do not commit it into source control. For Space deployment, use Settings β Variables and secrets instead of placing secrets in code.