| --- |
| title: MEPLIFEOS FUGTH AI Gateway |
| emoji: 🤖 |
| colorFrom: blue |
| colorTo: green |
| sdk: docker |
| app_port: 7860 |
| pinned: false |
| --- |
| |
| # MEPLIFEOS — FUGTH AI Gateway |
|
|
| A lightweight, protected GGUF inference API for FUGTH Management OS. |
|
|
| - Models: Qwen2.5-0.5B-Instruct and Qwen2.5-1.5B-Instruct (Q4_K_M GGUF). |
| - Both load into memory; only one inference is processed at a time (shared bounded FIFO queue). |
| - `POST /v1/chat` — protected generation with caller-provided RAG context. |
| - `GET /health` — public readiness, queue depth, and loaded models. |
| - `POST /admin/default-model` — change the default model until restart (admin key required). |
| - `/` — manual smoke-test page; keys are entered in the browser and are not stored. |
| - `/docs` — API documentation. |
|
|
| ## Configuration in Space Settings → Variables and secrets |
|
|
| Required **Secret**: `FUGTH_API_KEY` — a strong random key (32+ characters). |
|
|
| Optional **Secret**: `FUGTH_ADMIN_KEY` — a separate strong random key for admin configuration. |
|
|
| Optional variables: |
|
|
| | Name | Default | Meaning | |
| |---|---|---| |
| | `FUGTH_DEFAULT_MODEL` | `1.5b` | Default on startup (`1.5b` or `0.5b`) | |
| | `MAX_QUEUE_SIZE` | `8` | Number of waiting requests (not counting the active one) | |
| | `REQUEST_TIMEOUT_SECONDS` | `110` | HTTP request wait time before returning 504 | |
| | `N_CTX` | `2048` | Context window for each model | |
| | `N_THREADS` | `2` | CPU inference threads | |
|
|
| Models download on startup and live on the Space's **ephemeral** disk. Restarts can require a download. Do not upload model weights to the repo. |
|
|
| ## API example |
|
|
| ```bash |
| curl -X POST 'https://fugthchat-meplifeos.hf.space/v1/chat' \ |
| -H 'Content-Type: application/json' \ |
| -H 'X-API-Key: YOUR_SECRET_VALUE' \ |
| -d '{ |
| "model": "0.5b", |
| "business_name": "Example Hair Studio", |
| "message": "What time do you close on Saturday?", |
| "context": "Hours: Monday-Friday 9AM-6PM. Saturday 10AM-3PM. Sunday closed.", |
| "max_tokens": 120 |
| }' |
| ``` |
|
|
| For the 1.5B model, set `"model": "1.5b"`. Omit `model` to use the default. |
|
|
| **Important:** RAG retrieval, authentication of end-users, customer isolation, rate limits, and account authorization belong in the **FUGTH backend**. This service receives only context selected by the backend; it does not implement vector retrieval, durable conversations, or fine-tuning. Keep `FUGTH_API_KEY` on your backend, never in public website source code. Do not pass sensitive data unless your architecture and policies permit it. |
|
|
| ## What if it fails to build? |
|
|
| - Check the Space's **Build logs** for missing compiler or network errors. |
| - Check **Container logs** for downloading/initializing the models. |
| - Visit `/health` and wait for `"ready": true`. |
| - If Hugging Face blocks restarting older Docker Spaces, changing source code cannot bypass the account/hardware restriction. |
| - To limit downloads for troubleshooting, set `LOAD_MODELS=0.5b` as a Space variable, then return to `LOAD_MODELS=0.5b,1.5b` after validation. |
|
|
| For production-scale usage, add a persistent job queue, tenant rate limits, billing controls, observability, and another always-on inference provider. Free Spaces can sleep. |
|
|
