Update README.md
#1
by MackenzieNVIDIA - opened
README.md
CHANGED
|
@@ -112,7 +112,7 @@ The model version is FP4 QAD 1.0 and is quantized with nvidia-modelopt **v0.47.0
|
|
| 112 |
**Datasets:** GPQA Diamond, ToolEvalBench <br>
|
| 113 |
**Data Collection Method by dataset:** Hybrid: Automated, Manually-Collected <br>
|
| 114 |
**Labeling Method by dataset:** Hybrid: Manually-Labeled, Automated <br>
|
| 115 |
-
**Properties:** GPQA Diamond contains
|
| 116 |
|
| 117 |
The accuracy benchmark results are presented in the table below:
|
| 118 |
|
|
@@ -227,7 +227,7 @@ vllm serve nvidia/Qwen3.8-Flash-Next-FP4-Local \
|
|
| 227 |
|
| 228 |
### NVIDIA RTX PRO 6000 Blackwell
|
| 229 |
|
| 230 |
-
vLLM container: [`ghcr.io/local-inference-lab/vllm:
|
| 231 |
|
| 232 |
Start the container. Set `HF_TOKEN` in the host environment first.
|
| 233 |
|
|
@@ -242,7 +242,7 @@ docker run --rm -it \
|
|
| 242 |
-e HF_TOKEN=$HF_TOKEN \
|
| 243 |
-v ~/.cache/huggingface:/root/.cache/huggingface \
|
| 244 |
--entrypoint /bin/bash \
|
| 245 |
-
ghcr.io/local-inference-lab/vllm:
|
| 246 |
```
|
| 247 |
|
| 248 |
Then launch vLLM inside the container:
|
|
|
|
| 112 |
**Datasets:** GPQA Diamond, ToolEvalBench <br>
|
| 113 |
**Data Collection Method by dataset:** Hybrid: Automated, Manually-Collected <br>
|
| 114 |
**Labeling Method by dataset:** Hybrid: Manually-Labeled, Automated <br>
|
| 115 |
+
**Properties:** GPQA Diamond contains 198 graduate-level multiple-choice questions written by domain experts in biology, physics, and chemistry. ToolEvalBench is a tool-calling quality benchmark that runs deterministic multi-turn agent scenarios against mock tools, scoring tool selection, parameter precision, multi-step chains, error recovery, and structured output. <br>
|
| 116 |
|
| 117 |
The accuracy benchmark results are presented in the table below:
|
| 118 |
|
|
|
|
| 227 |
|
| 228 |
### NVIDIA RTX PRO 6000 Blackwell
|
| 229 |
|
| 230 |
+
vLLM container: [`ghcr.io/local-inference-lab/vllm:release-10142026`](https://github.com/local-inference-lab/blackwell-llm-docker/pkgs/container/vllm)
|
| 231 |
|
| 232 |
Start the container. Set `HF_TOKEN` in the host environment first.
|
| 233 |
|
|
|
|
| 242 |
-e HF_TOKEN=$HF_TOKEN \
|
| 243 |
-v ~/.cache/huggingface:/root/.cache/huggingface \
|
| 244 |
--entrypoint /bin/bash \
|
| 245 |
+
ghcr.io/local-inference-lab/vllm:release-10142026
|
| 246 |
```
|
| 247 |
|
| 248 |
Then launch vLLM inside the container:
|