Update README.md
Browse files
README.md
CHANGED
|
@@ -50,7 +50,9 @@ python3 quantize_quark.py \
|
|
| 50 |
This model can be deployed efficiently using the [vLLM](https://docs.vllm.ai/en/latest/) backend.
|
| 51 |
|
| 52 |
Docker image:
|
|
|
|
| 53 |
docker pull rocm/vllm-dev:nightly_main_20260616
|
|
|
|
| 54 |
|
| 55 |
Note: this model has 64 KV heads, which is incompatible with the AITER MLA
|
| 56 |
kernel (supports 16 or 128 only). Disable AITER MLA when serving on ROCm:
|
|
|
|
| 50 |
This model can be deployed efficiently using the [vLLM](https://docs.vllm.ai/en/latest/) backend.
|
| 51 |
|
| 52 |
Docker image:
|
| 53 |
+
```bash
|
| 54 |
docker pull rocm/vllm-dev:nightly_main_20260616
|
| 55 |
+
```
|
| 56 |
|
| 57 |
Note: this model has 64 KV heads, which is incompatible with the AITER MLA
|
| 58 |
kernel (supports 16 or 128 only). Disable AITER MLA when serving on ROCm:
|