Update README.md
Browse files
README.md
CHANGED
|
@@ -49,6 +49,9 @@ python3 quantize_quark.py \
|
|
| 49 |
|
| 50 |
This model can be deployed efficiently using the [vLLM](https://docs.vllm.ai/en/latest/) backend.
|
| 51 |
|
|
|
|
|
|
|
|
|
|
| 52 |
Note: this model has 64 KV heads, which is incompatible with the AITER MLA
|
| 53 |
kernel (supports 16 or 128 only). Disable AITER MLA when serving on ROCm:
|
| 54 |
|
|
|
|
| 49 |
|
| 50 |
This model can be deployed efficiently using the [vLLM](https://docs.vllm.ai/en/latest/) backend.
|
| 51 |
|
| 52 |
+
Docker image:
|
| 53 |
+
docker pull rocm/vllm-dev:nightly_main_20260616
|
| 54 |
+
|
| 55 |
Note: this model has 64 KV heads, which is incompatible with the AITER MLA
|
| 56 |
kernel (supports 16 or 128 only). Disable AITER MLA when serving on ROCm:
|
| 57 |
|