Commit ·
264d77e
1
Parent(s): 740b40c
Update README.md (#4)
Browse files- Update README.md (0653b55f11d8e1a4ae142f16c06cb335b273ef1b)
Co-authored-by: Mackenzie <MackenzieNVIDIA@users.noreply.huggingface.co>
README.md
CHANGED
|
@@ -95,7 +95,7 @@ The model version is FP4 QAD 1.0 and is quantized with nvidia-modelopt **v0.47.0
|
|
| 95 |
**Datasets:** GPQA Diamond, ToolEvalBench <br>
|
| 96 |
**Data Collection Method by dataset:** Hybrid: Automated, Manually-Collected <br>
|
| 97 |
**Labeling Method by dataset:** Hybrid: Manually-Labeled, Automated <br>
|
| 98 |
-
**Properties:** GPQA Diamond contains
|
| 99 |
|
| 100 |
The accuracy benchmark results are presented in the table below:
|
| 101 |
|
|
@@ -274,7 +274,7 @@ vllm serve nvidia/GLM-5.3-Flash-FP4-Local \
|
|
| 274 |
|
| 275 |
### 2 x NVIDIA RTX PRO 6000 Blackwell
|
| 276 |
|
| 277 |
-
vLLM container: [`ghcr.io/local-inference-lab/vllm:
|
| 278 |
|
| 279 |
Start the container. Set `HF_TOKEN` in the host environment first.
|
| 280 |
|
|
@@ -289,7 +289,7 @@ docker run --rm -it \
|
|
| 289 |
-e HF_TOKEN=$HF_TOKEN \
|
| 290 |
-v ~/.cache/huggingface:/root/.cache/huggingface \
|
| 291 |
--entrypoint /bin/bash \
|
| 292 |
-
ghcr.io/local-inference-lab/vllm:
|
| 293 |
```
|
| 294 |
|
| 295 |
Then launch vLLM inside the container:
|
|
|
|
| 95 |
**Datasets:** GPQA Diamond, ToolEvalBench <br>
|
| 96 |
**Data Collection Method by dataset:** Hybrid: Automated, Manually-Collected <br>
|
| 97 |
**Labeling Method by dataset:** Hybrid: Manually-Labeled, Automated <br>
|
| 98 |
+
**Properties:** GPQA Diamond contains 198 graduate-level multiple-choice questions written by domain experts in biology, physics, and chemistry. ToolEvalBench is a tool-calling quality benchmark that runs deterministic multi-turn agent scenarios against mock tools, scoring tool selection, parameter precision, multi-step chains, error recovery, and structured output. <br>
|
| 99 |
|
| 100 |
The accuracy benchmark results are presented in the table below:
|
| 101 |
|
|
|
|
| 274 |
|
| 275 |
### 2 x NVIDIA RTX PRO 6000 Blackwell
|
| 276 |
|
| 277 |
+
vLLM container: [`ghcr.io/local-inference-lab/vllm:release-10142026`](https://github.com/local-inference-lab/blackwell-llm-docker/pkgs/container/vllm)
|
| 278 |
|
| 279 |
Start the container. Set `HF_TOKEN` in the host environment first.
|
| 280 |
|
|
|
|
| 289 |
-e HF_TOKEN=$HF_TOKEN \
|
| 290 |
-v ~/.cache/huggingface:/root/.cache/huggingface \
|
| 291 |
--entrypoint /bin/bash \
|
| 292 |
+
ghcr.io/local-inference-lab/vllm:release-10142026
|
| 293 |
```
|
| 294 |
|
| 295 |
Then launch vLLM inside the container:
|