Roy2144 commited on
Commit
fa8a50f
·
verified ·
1 Parent(s): 0930d73

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +5 -3
README.md CHANGED
@@ -48,9 +48,11 @@ print(tokenizer.decode(outputs[0], skip_special_tokens=True))
48
 
49
  ## Benchmark Results
50
 
51
- Benchmarked on a single NVIDIA A100 40GB GPU using:
52
- - Transformers backend (FP16 and INT8 variants). However, The current implementation of transformers does not include optimized kernels to support int8 computation directly.
53
- - vLLM engine for INT8 inference acceleration
 
 
54
  ---
55
 
56
  | Metric | Transformers (INT8) | Transformers (FP16) | vLLM (INT8) |
 
48
 
49
  ## Benchmark Results
50
 
51
+ - Benchmarked on a single NVIDIA A100 40GB GPU using:
52
+ - Transformers backend (FP16 and INT8 variants). Note: The current implementation of `transformers` does not include optimized kernels for direct INT8 computation.
53
+ - vLLM engine with INT8 inference acceleration for improved performance.
54
+
55
+
56
  ---
57
 
58
  | Metric | Transformers (INT8) | Transformers (FP16) | vLLM (INT8) |