|
Download README.md from bolikoto/laya-multilingual-cpu-basic: direct link, hf CLI and curl.
- Browser
- Download file 5.17 kB
-
https://huggingface.co/spaces/bolikoto/laya-multilingual-cpu-basic/resolve/main/README.md
- Command line
-
hf download hf://spaces/bolikoto/laya-multilingual-cpu-basic/README.md
-
curl -L -o README.md https://huggingface.co/spaces/bolikoto/laya-multilingual-cpu-basic/resolve/main/README.md
5.17 kB
| title: Laya Multilingual CPU API | |
| sdk: gradio | |
| sdk_version: 6.24.0 | |
| app_file: app.py | |
| python_version: "3.10" | |
| license: apache-2.0 | |
| # Laya multilingual CPU API | |
| Only `convaiinnovations/laya-multilingual` is loaded, pinned to the revision in | |
| `app.py`. CPU is forced even on ZeroGPU. The unused GPU declaration is never | |
| called and is not exposed as an API. HF runs two server workers, each admitting | |
| at most eight inference requests (including its bounded inference queue), for a | |
| total ceiling of 16. InstaBox and Evroc each also run two workers admitting eight | |
| requests per worker, retaining a uniform per-host total ceiling of 16. | |
| Excess requests receive HTTP 503 and `Retry-After: 1`; a full worker can reject | |
| even when another worker has a free slot. Actual CPU inference is serialized | |
| within each worker by Laya's native server to avoid oversubscription and memory spikes. | |
| The CPU-host launcher allows 30 seconds for worker healthchecks so slow checkpoint | |
| initialization is not mistaken for a dead process; request timeouts are unchanged. | |
| Only on ZeroGPU, ASGI startup with Spaces SDK 0.51.3 explicitly runs the SDK's startup-report | |
| callback (normally called by Gradio.launch), not its GPU scheduling API. The | |
| dummy remains uncalled; recheck this callback when the platform upgrades Spaces. | |
| `POST /v1/systemone` accepts the existing Jev `model`, `state`, `questions` JSON | |
| and returns only `model`, `answers`, `usage`. `LAYA_API_KEY` is mandatory and must | |
| be sent as a Bearer token. `/health` is public liveness only; an authenticated | |
| request includes the loaded checkpoint, revision and actual CPU device. | |
| Jev confidence is calculated with Laya's compatibility helper; truncated input | |
| has confidence zero, preserving Opportunity's existing fail-open gate. | |
| ## InstaBox (persistent installation) | |
| Install `app.py`, `requirements.txt`, `.venv`, and the model cache below | |
| `/data/workspace/laya`. Keep `LAYA_API_KEY` in the mode-0600 `.env` there, never | |
| in Git or command-line arguments. Link `laya.service` below the persistent | |
| `~/.config/systemd/user/` and enable it with `systemctl --user`. Its default-target | |
| link is persistent too; InstaBox enables user linger during boot. Publish | |
| port 7860 with InstaBox's HTTPS port forwarding; never expose an unauthenticated | |
| inference listener. Do not restart unrelated services to test boot persistence. | |
| On InstaBox, pre-install `torch==2.8.0+cpu` from PyTorch's CPU wheel index before | |
| installing `requirements.txt`; it satisfies `torch==2.8.0` without CUDA packages. | |
| The Space uses HF's supported Torch declaration but never places a tensor on GPU. | |
| Evroc uses the same unit, CPU wheel, threads and admission limit. Enable user | |
| linger, bind Uvicorn to `127.0.0.1:7860` with `--workers 2` via the unit's `ExecStart`, and publish | |
| through the dedicated `laya-evroc` Cloudflare Tunnel, keeping its token in a | |
| mode-0600 file, not service arguments. `laya-tunnel.service` uses HTTP/2 outbound | |
| connections and restarts on failure; no inbound security-group change is needed. | |
| Keep the existing VM services intact. | |
| ## Hugging Face | |
| Create a new Gradio Space on `cpu-basic`; upload only these three Space files | |
| (`app.py`, `requirements.txt`, `README.md`) and set its `LAYA_API_KEY` secret. | |
| Do not upload `.env`, caches, units, or virtual environments. This runs on CPU, | |
| without registering ZeroGPU startup or invoking the dummy GPU function. Preserve | |
| the existing Space variables (`LAYA_THREADS=4`, `TOKENIZERS_PARALLELISM=false`). | |
| Never silently substitute paid hardware. CPU Basic sleeps after 48 hours of inactivity. | |
| ## Opportunity | |
| Set `LAYA_PROVIDERS_JSON` in the restricted runtime secret group to an array: | |
| ```json | |
| [ | |
| {"endpoint": "https://instabox.example/v1/systemone", "api_key": "<secret>", "max_concurrent": 16}, | |
| {"endpoint": "https://hf-1.example/v1/systemone", "api_key": "<secret>", "max_concurrent": 16}, | |
| {"endpoint": "https://hf-2.example/v1/systemone", "api_key": "<secret>", "max_concurrent": 16}, | |
| {"endpoint": "https://evroc.example/v1/systemone", "api_key": "<secret>", "max_concurrent": 16} | |
| ] | |
| ``` | |
| Each request chooses its first endpoint randomly with weight `max_concurrent`: | |
| 16:16:16:16 gives 25% each over time. Failures try remaining | |
| endpoints with the same weighted selection, without replacement. Add future | |
| providers to this list with their actual admission limits; no new code branch | |
| is needed. This is not a distributed in-flight counter: limits are enforced by | |
| each API, include queued work, and do not imply 64 simultaneous inferences. | |
| Safe endpoint/attempt logs expose routing and fallback without keys or input. | |
| The four legacy `LAYA_INSTABOX_ENDPOINT`, `LAYA_INSTABOX_API_KEY`, | |
| `LAYA_HF_ENDPOINT`, `LAYA_HF_API_KEY` variables remain supported with weights | |
| 16:32 when the JSON variable is absent. Invalid or partial configuration is | |
| refused, never silently routed to external Jev. No model weights or ML | |
| dependencies are installed in Opportunity. To restore the old Jev chain, remove | |
| `LAYA_PROVIDERS_JSON` and all four legacy variables; YAML questions, thresholds | |
| and legacy credentials are unchanged. Verify a new CPU Space and the production | |
| switch before pausing the old ZeroGPU Space. | |