| --- |
| license: apache-2.0 |
| base_model: convaiinnovations/laya |
| pipeline_tag: text-classification |
| tags: |
| - verdict |
| - onnx |
| - system-one |
| - decision-models |
| - laya |
| - serving |
| --- |
| |
| # Laya (Verdict-ready) |
|
|
| The [Laya](https://huggingface.co/convaiinnovations/laya) System One decision model, exported to ONNX fp32 and packaged for [Verdict](https://github.com/sgyaqing/verdict) -- a high-performance serving engine for System One decision models. Serve it with one command: |
|
|
| ```bash |
| # CPU host |
| docker run --rm -p 8080:8080 \ |
| -v ~/.cache/verdict:/root/.cache/verdict \ |
| sgyaqing/verdict:latest-cpu serve verdict-models/laya --host 0.0.0.0 |
| |
| # GPU host |
| docker run --rm --gpus all -p 8080:8080 \ |
| -v ~/.cache/verdict:/root/.cache/verdict \ |
| sgyaqing/verdict:latest-cuda serve verdict-models/laya --host 0.0.0.0 |
| |
| # ModelScope: |
| docker run --rm -p 8080:8080 \ |
| -e VERDICT_USE_MODELSCOPE=1 \ |
| -v ~/.cache/verdict:/root/.cache/verdict \ |
| sgyaqing/verdict:latest-cpu serve sgyaqing/laya --host 0.0.0.0 |
| ``` |
|
|
| Endpoints: the standard decision protocols `POST /v1/systemone` and `POST /v1/decisions`, native `POST /v1/choice | /v1/score | /v1/noul`, plus an OpenAI-compatible layer (`/v1/chat/completions`, `/v1/models`). |
|
|
| ## Performance |
|
|
| Measured on one RTX 4090, end to end over HTTP, against the model author's own server (`laya-serve`). A ratio above 1x means Verdict is ahead. |
|
|
| | concurrency | Verdict rps | `laya-serve` rps | rps ratio | Verdict P50 / P99 (ms) | `laya-serve` P50 / P99 (ms) | P50 / P99 ratio | |
| |---|---:|---:|---:|---:|---:|---:| |
| | 1 | 121.4 | 34.4 | 3.53x | 8.37 / 11.07 | 28.58 / 32.73 | 3.42 / 2.96x | |
| | 8 | 501.4 | 35.4 | 14.2x | 15.99 / 22.42 | 227.96 / 250.60 | 14.3 / 11.2x | |
| | 64 | 1353.0 | refused requests | — | 46.71 / 65.19 | refused requests | — | |
|
|
| ## Files |
|
|
| | File | Purpose | |
| |---|---| |
| | `model.json` | manifest: graph file name, budgets, calibration temperatures | |
| | `laya.onnx` + `laya.onnx.data` | the ONNX graph (fp32, external weights) | |
| | `tokenizer/tokenizer.json` (+ `tokenizer_config.json`) | the tokenizer | |
| | `SHA256SUMS` | checksums for every file above | |
|
|
| ## Provenance & fidelity |
|
|
| Weights are unchanged from `convaiinnovations/laya`; this repo adds only the ONNX export and the serving manifest. Conversion and serving are verified against the golden standard (the official Laya SDK, CPU fp32, 40-case corpus): identical answers on CPU and on CUDA (RTX 4090). |
|
|
| Accuracy PK — against `laya-serve` 0.4.7; both servers on one RTX 4090 (CUDA), graded against the same golden standard: |
|
|
| | | Verdict | `laya-serve` | result | |
| |---|---:|---:|:--| |
| | worst deviation | 0.0 | 0.943 | Verdict ahead | |
|
|
| ## License |
|
|
| Apache-2.0, inherited from the base model. Credit for the model itself belongs to the [original authors](https://huggingface.co/convaiinnovations/laya); this repository only repackages it for serving. |
|
|
