File size: 3,147 Bytes
d85712e | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 | ---
title: MEPLIFEOS FUGTH AI Gateway
emoji: 🤖
colorFrom: blue
colorTo: green
sdk: docker
app_port: 7860
pinned: false
---
# MEPLIFEOS — FUGTH AI Gateway
A lightweight, protected GGUF inference API for FUGTH Management OS.
- Models: Qwen2.5-0.5B-Instruct and Qwen2.5-1.5B-Instruct (Q4_K_M GGUF).
- Both load into memory; only one inference is processed at a time (shared bounded FIFO queue).
- `POST /v1/chat` — protected generation with caller-provided RAG context.
- `GET /health` — public readiness, queue depth, and loaded models.
- `POST /admin/default-model` — change the default model until restart (admin key required).
- `/` — manual smoke-test page; keys are entered in the browser and are not stored.
- `/docs` — API documentation.
## Configuration in Space Settings → Variables and secrets
Required **Secret**: `FUGTH_API_KEY` — a strong random key (32+ characters).
Optional **Secret**: `FUGTH_ADMIN_KEY` — a separate strong random key for admin configuration.
Optional variables:
| Name | Default | Meaning |
|---|---|---|
| `FUGTH_DEFAULT_MODEL` | `1.5b` | Default on startup (`1.5b` or `0.5b`) |
| `MAX_QUEUE_SIZE` | `8` | Number of waiting requests (not counting the active one) |
| `REQUEST_TIMEOUT_SECONDS` | `110` | HTTP request wait time before returning 504 |
| `N_CTX` | `2048` | Context window for each model |
| `N_THREADS` | `2` | CPU inference threads |
Models download on startup and live on the Space's **ephemeral** disk. Restarts can require a download. Do not upload model weights to the repo.
## API example
```bash
curl -X POST 'https://fugthchat-meplifeos.hf.space/v1/chat' \
-H 'Content-Type: application/json' \
-H 'X-API-Key: YOUR_SECRET_VALUE' \
-d '{
"model": "0.5b",
"business_name": "Example Hair Studio",
"message": "What time do you close on Saturday?",
"context": "Hours: Monday-Friday 9AM-6PM. Saturday 10AM-3PM. Sunday closed.",
"max_tokens": 120
}'
```
For the 1.5B model, set `"model": "1.5b"`. Omit `model` to use the default.
**Important:** RAG retrieval, authentication of end-users, customer isolation, rate limits, and account authorization belong in the **FUGTH backend**. This service receives only context selected by the backend; it does not implement vector retrieval, durable conversations, or fine-tuning. Keep `FUGTH_API_KEY` on your backend, never in public website source code. Do not pass sensitive data unless your architecture and policies permit it.
## What if it fails to build?
- Check the Space's **Build logs** for missing compiler or network errors.
- Check **Container logs** for downloading/initializing the models.
- Visit `/health` and wait for `"ready": true`.
- If Hugging Face blocks restarting older Docker Spaces, changing source code cannot bypass the account/hardware restriction.
- To limit downloads for troubleshooting, set `LOAD_MODELS=0.5b` as a Space variable, then return to `LOAD_MODELS=0.5b,1.5b` after validation.
For production-scale usage, add a persistent job queue, tenant rate limits, billing controls, observability, and another always-on inference provider. Free Spaces can sleep.
|