krishnateja95 commited on
Commit
337923a
·
verified ·
1 Parent(s): 7ad3574

Add note about PR #48876 for FP8 compressed tensors support

Browse files
Files changed (1) hide show
  1. README.md +2 -0
README.md CHANGED
@@ -40,6 +40,8 @@ Weights are quantized statically using per-channel FP8 scaling, and activations
40
  This model can be deployed using [vLLM](https://docs.vllm.ai/en/latest/).
41
 
42
  > **Note:** Inkling model support in vLLM is being added in [vllm-project/vllm#48768](https://github.com/vllm-project/vllm/pull/48768). This PR is currently a draft and has not yet been merged, so you will need to install vLLM from that pull request's branch (`tml-inkling`) to run this model until it lands on `main`.
 
 
43
 
44
  1. Start the vLLM server:
45
  ```
 
40
  This model can be deployed using [vLLM](https://docs.vllm.ai/en/latest/).
41
 
42
  > **Note:** Inkling model support in vLLM is being added in [vllm-project/vllm#48768](https://github.com/vllm-project/vllm/pull/48768). This PR is currently a draft and has not yet been merged, so you will need to install vLLM from that pull request's branch (`tml-inkling`) to run this model until it lands on `main`.
43
+ >
44
+ > Additionally, FP8 quantization support for Inkling requires [vllm-project/vllm#48876](https://github.com/vllm-project/vllm/pull/48876).
45
 
46
  1. Start the vLLM server:
47
  ```