Instructions to use verdict-models/laya with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Laya
How to use verdict-models/laya with Laya:
# No code snippets available yet for this library. # To use this model, check the repository files and the library's documentation. # Want to help? PRs adding snippets are welcome at: # https://github.com/huggingface/huggingface.js
- Notebooks
- Google Colab
- Kaggle
Upload README.md with huggingface_hub
Browse files
README.md
CHANGED
|
@@ -32,7 +32,7 @@ One RTX 3080, one session, one fixed request (full methodology and CPU numbers i
|
|
| 32 |
| concurrency 32 | Throughput | P50 |
|
| 33 |
|---|---:|---:|
|
| 34 |
| FastAPI + torch (cuda) | 37.8 rps | 844 ms |
|
| 35 |
-
| Verdict (CUDA
|
| 36 |
|
| 37 |
## Files
|
| 38 |
|
|
@@ -50,7 +50,7 @@ Weights are unchanged from `convaiinnovations/laya`; this repo adds the ONNX exp
|
|
| 50 |
- export parity: max logits difference 6.6e-6 against the official forward
|
| 51 |
- token ids: id-for-id identical to the official pipeline
|
| 52 |
- responses: round-4 probabilities identical (golden vectors, 40 cases per task)
|
| 53 |
-
- serving on
|
| 54 |
|
| 55 |
## License
|
| 56 |
|
|
|
|
| 32 |
| concurrency 32 | Throughput | P50 |
|
| 33 |
|---|---:|---:|
|
| 34 |
| FastAPI + torch (cuda) | 37.8 rps | 844 ms |
|
| 35 |
+
| Verdict (CUDA) | 408 rps | 78 ms |
|
| 36 |
|
| 37 |
## Files
|
| 38 |
|
|
|
|
| 50 |
- export parity: max logits difference 6.6e-6 against the official forward
|
| 51 |
- token ids: id-for-id identical to the official pipeline
|
| 52 |
- responses: round-4 probabilities identical (golden vectors, 40 cases per task)
|
| 53 |
+
- serving on CPU and CUDA, against a 2e-4 tolerance: CPU worst logits difference 1.1e-4, CUDA 5e-5; response layer bitwise on both
|
| 54 |
|
| 55 |
## License
|
| 56 |
|