--- title: Laya Multilingual CPU API sdk: gradio sdk_version: 6.24.0 app_file: app.py python_version: "3.10" license: apache-2.0 --- # Laya multilingual CPU API Only `convaiinnovations/laya-multilingual` is loaded, pinned to the revision in `app.py`. CPU is forced even on ZeroGPU. The unused GPU declaration is never called and is not exposed as an API. HF runs two server workers, each admitting at most eight inference requests (including its bounded inference queue), for a total ceiling of 16. InstaBox and Evroc each also run two workers admitting eight requests per worker, retaining a uniform per-host total ceiling of 16. Excess requests receive HTTP 503 and `Retry-After: 1`; a full worker can reject even when another worker has a free slot. Actual CPU inference is serialized within each worker by Laya's native server to avoid oversubscription and memory spikes. The CPU-host launcher allows 30 seconds for worker healthchecks so slow checkpoint initialization is not mistaken for a dead process; request timeouts are unchanged. Only on ZeroGPU, ASGI startup with Spaces SDK 0.51.3 explicitly runs the SDK's startup-report callback (normally called by Gradio.launch), not its GPU scheduling API. The dummy remains uncalled; recheck this callback when the platform upgrades Spaces. `POST /v1/systemone` accepts the existing Jev `model`, `state`, `questions` JSON and returns only `model`, `answers`, `usage`. `LAYA_API_KEY` is mandatory and must be sent as a Bearer token. `/health` is public liveness only; an authenticated request includes the loaded checkpoint, revision and actual CPU device. Jev confidence is calculated with Laya's compatibility helper; truncated input has confidence zero, preserving Opportunity's existing fail-open gate. ## InstaBox (persistent installation) Install `app.py`, `requirements.txt`, `.venv`, and the model cache below `/data/workspace/laya`. Keep `LAYA_API_KEY` in the mode-0600 `.env` there, never in Git or command-line arguments. Link `laya.service` below the persistent `~/.config/systemd/user/` and enable it with `systemctl --user`. Its default-target link is persistent too; InstaBox enables user linger during boot. Publish port 7860 with InstaBox's HTTPS port forwarding; never expose an unauthenticated inference listener. Do not restart unrelated services to test boot persistence. On InstaBox, pre-install `torch==2.8.0+cpu` from PyTorch's CPU wheel index before installing `requirements.txt`; it satisfies `torch==2.8.0` without CUDA packages. The Space uses HF's supported Torch declaration but never places a tensor on GPU. Evroc uses the same unit, CPU wheel, threads and admission limit. Enable user linger, bind Uvicorn to `127.0.0.1:7860` with `--workers 2` via the unit's `ExecStart`, and publish through the dedicated `laya-evroc` Cloudflare Tunnel, keeping its token in a mode-0600 file, not service arguments. `laya-tunnel.service` uses HTTP/2 outbound connections and restarts on failure; no inbound security-group change is needed. Keep the existing VM services intact. ## Hugging Face Create a new Gradio Space on `cpu-basic`; upload only these three Space files (`app.py`, `requirements.txt`, `README.md`) and set its `LAYA_API_KEY` secret. Do not upload `.env`, caches, units, or virtual environments. This runs on CPU, without registering ZeroGPU startup or invoking the dummy GPU function. Preserve the existing Space variables (`LAYA_THREADS=4`, `TOKENIZERS_PARALLELISM=false`). Never silently substitute paid hardware. CPU Basic sleeps after 48 hours of inactivity. ## Opportunity Set `LAYA_PROVIDERS_JSON` in the restricted runtime secret group to an array: ```json [ {"endpoint": "https://instabox.example/v1/systemone", "api_key": "", "max_concurrent": 16}, {"endpoint": "https://hf-1.example/v1/systemone", "api_key": "", "max_concurrent": 16}, {"endpoint": "https://hf-2.example/v1/systemone", "api_key": "", "max_concurrent": 16}, {"endpoint": "https://evroc.example/v1/systemone", "api_key": "", "max_concurrent": 16} ] ``` Each request chooses its first endpoint randomly with weight `max_concurrent`: 16:16:16:16 gives 25% each over time. Failures try remaining endpoints with the same weighted selection, without replacement. Add future providers to this list with their actual admission limits; no new code branch is needed. This is not a distributed in-flight counter: limits are enforced by each API, include queued work, and do not imply 64 simultaneous inferences. Safe endpoint/attempt logs expose routing and fallback without keys or input. The four legacy `LAYA_INSTABOX_ENDPOINT`, `LAYA_INSTABOX_API_KEY`, `LAYA_HF_ENDPOINT`, `LAYA_HF_API_KEY` variables remain supported with weights 16:32 when the JSON variable is absent. Invalid or partial configuration is refused, never silently routed to external Jev. No model weights or ML dependencies are installed in Opportunity. To restore the old Jev chain, remove `LAYA_PROVIDERS_JSON` and all four legacy variables; YAML questions, thresholds and legacy credentials are unchanged. Verify a new CPU Space and the production switch before pausing the old ZeroGPU Space.