File size: 2,434 Bytes
d39b40d 4ec320f 2282a89 d39b40d b273c46 d39b40d 02f1755 4ec320f | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 | ---
license: cc-by-nc-4.0
base_model: openjev/openjev
tags:
- decision-model
- zero-shot-classification
- agents
- fp8
- vllm
---
# OpenJev, FP8 checkpoint
The FP8 file of [OpenJev](https://huggingface.co/openjev/openjev): the same weights stored as FP8 (e4m3, 128×128 weight blocks, dynamic activations), about 29 GB instead of 54 GB. Everything about the model, the API and the results is in the main card; this page only says how this file was checked and how to run it.
**
## Measured, not assumed
A one-pass readout lives on small score differences, so the file was re-run on the same frozen tests as the 16-bit model, with the same helper and the same fixed calibration:
| test | 16-bit model as served | this FP8 file |
|---|---|---|
| 10,000 text questions, 34 public sources | 84.0% (8,403) | 84.2% (8,420); +0.17 points, 95% interval −0.13 to +0.45 |
| 2,000 desktop screenshots, next action | 88.0% (1,759) | 88.0% (1,760); +0.05 points |
About 2% of individual answers change between the two files (219 of 10,000 text answers, 40 of 2,000 screenshot answers), in both directions. The rule fixed before the run was "no more than 0.5 points below the served model"; both tests pass it.
## Run it
```bash
vllm serve openjev/openjev-FP8 --served-model-name qwen --port 8000 --enable-prefix-caching \
--max-model-len 16384 --gpu-memory-utilization 0.90 --limit-mm-per-prompt '{"image":1}' --trust-remote-code \
--max-num-seqs 16 --max-logprobs 64 --gdn-prefill-backend triton --quantization fp8
```
Then start the helper from the main repository (`helper/shim.py`) exactly as its card describes; the calibration settings are the same as for the 16-bit model. The conversion receipt (`FP8_CONVERSION_RECEIPT.json`) records the source checkpoint and the conversion tool.
Serverless: Runpod has a step-by-step guide for serving this build on Runpod Serverless with one 80 GB H100: [How to deploy OpenJev on Runpod Serverless](https://www.runpod.io/blog/how-to-deploy-openjev-on-runpod-serverless).
## Licence
Weights: CC BY-NC 4.0 (research and non-commercial use), the same as the main repository. For a commercial licence, email [support@loopai.com](mailto:support@loopai.com). Helper and serving files: Apache 2.0. The licence texts and the base model's attribution are in `LICENSE`, `LICENSE-APACHE-2.0` and `NOTICE`.
OpenJev is an independent project, not affiliated with TypeSafe; Jev is their product.
|