cloudyu commited on
Commit
4f4bfd2
·
verified ·
1 Parent(s): 12e60c0

Model card: NVFP4 version autotrust/JEV-35B-NVFP4

Browse files
Files changed (1) hide show
  1. README.md +21 -0
README.md CHANGED
@@ -55,6 +55,17 @@ model-index:
55
  **72.08** (our rebuild of the vision benchmarks; JEV-27B-VL 71.67 on the same rebuild, a tie within noise). Median single-request latency
56
  241 ms on one B200 (JEV-27B-VL 271 ms).
57
 
 
 
 
 
 
 
 
 
 
 
 
58
  > **Two models, two organisations.** **TypeSafe Jev 1.13** is the hosted, closed model made by TypeSafe AI.
59
  > **autotrust/JEV-35B** is an independent open-weights model built by AutoTrust AI; it is not affiliated with,
60
  > endorsed by, or a product of TypeSafe AI.
@@ -243,6 +254,14 @@ hf download autotrust/JEV-35B --local-dir JEV-35B
243
  bash JEV-35B/serve.sh # vLLM on :8000; one GPU with 80 GB or more
244
  ```
245
 
 
 
 
 
 
 
 
 
246
  `serve.sh` runs `serve_decide.py`: the standard vLLM OpenAI server (same flags as `vllm serve`) with a `POST /v1/decide`
247
  route. Plain requests are System 2; requests for the LoRA module `jev-decision` (`adapter_vllm/`: backbone LoRA + the
248
  decision head as an `lm_head` LoRA) are System 1. Tested with a vLLM development build from September 2026.
@@ -328,6 +347,8 @@ videos/ computer-use and robot-arm episodes
328
  reports/demos/ per-episode computer-use, robot-arm and game results
329
  ```
330
 
 
 
331
  ## License
332
 
333
  Apache-2.0. This repository contains the weights of [Qwen/Qwen3.5-35B-A3B](https://huggingface.co/Qwen/Qwen3.5-35B-A3B)
 
55
  **72.08** (our rebuild of the vision benchmarks; JEV-27B-VL 71.67 on the same rebuild, a tie within noise). Median single-request latency
56
  241 ms on one B200 (JEV-27B-VL 271 ms).
57
 
58
+ > [!TIP]
59
+ > **NVFP4 version: [autotrust/JEV-35B-NVFP4](https://huggingface.co/autotrust/JEV-35B-NVFP4)**, the same model with its
60
+ > routed experts in NVFP4: **25.6 GB download, 23 GiB of GPU memory** (bf16: 72 GB, 66 GiB), so it runs on one 32–48 GB
61
+ > GPU. Decision Index 0.3 public 59.18 (bf16 59.75), vision rebuild 71.23 (bf16 72.08), 96.6 % of the Decision Index
62
+ > answers identical to bf16, same latency, same `serve.sh` and API.
63
+
64
+ | version | download | GPU memory (weights) | Decision Index 0.3 public | vision (rebuild) |
65
+ |---|---:|---:|---:|---:|
66
+ | [autotrust/JEV-35B](https://huggingface.co/autotrust/JEV-35B) (bf16, this repo) | 72 GB | 66.5 GiB | 59.75 | 72.08 |
67
+ | [**autotrust/JEV-35B-NVFP4**](https://huggingface.co/autotrust/JEV-35B-NVFP4) | **25.6 GB** | **23.3 GiB** | 59.18 | 71.23 |
68
+
69
  > **Two models, two organisations.** **TypeSafe Jev 1.13** is the hosted, closed model made by TypeSafe AI.
70
  > **autotrust/JEV-35B** is an independent open-weights model built by AutoTrust AI; it is not affiliated with,
71
  > endorsed by, or a product of TypeSafe AI.
 
254
  bash JEV-35B/serve.sh # vLLM on :8000; one GPU with 80 GB or more
255
  ```
256
 
257
+ On a smaller GPU, use [autotrust/JEV-35B-NVFP4](https://huggingface.co/autotrust/JEV-35B-NVFP4) (one GPU with 32 GB or
258
+ more; same `serve.sh` and API):
259
+
260
+ ```bash
261
+ hf download autotrust/JEV-35B-NVFP4 --local-dir JEV-35B-NVFP4
262
+ bash JEV-35B-NVFP4/serve.sh
263
+ ```
264
+
265
  `serve.sh` runs `serve_decide.py`: the standard vLLM OpenAI server (same flags as `vllm serve`) with a `POST /v1/decide`
266
  route. Plain requests are System 2; requests for the LoRA module `jev-decision` (`adapter_vllm/`: backbone LoRA + the
267
  decision head as an `lm_head` LoRA) are System 1. Tested with a vLLM development build from September 2026.
 
347
  reports/demos/ per-episode computer-use, robot-arm and game results
348
  ```
349
 
350
+ NVFP4 weights (routed experts in NVFP4, everything else identical): [autotrust/JEV-35B-NVFP4](https://huggingface.co/autotrust/JEV-35B-NVFP4).
351
+
352
  ## License
353
 
354
  Apache-2.0. This repository contains the weights of [Qwen/Qwen3.5-35B-A3B](https://huggingface.co/Qwen/Qwen3.5-35B-A3B)