Rhett1 commited on
Commit
f681697
·
verified ·
1 Parent(s): 649bf5e

Refresh benchmark tables: base-model arms complete (SVD-LLM / V2 / ASVD at 7B, SVD-LLM / V2 at 13B)

Browse files
Files changed (1) hide show
  1. README.md +5 -5
README.md CHANGED
@@ -78,8 +78,8 @@ Perplexity is **token-level** over fixed non-overlapping 2048-token windows, fol
78
  | | SVD-LLM V2 ‡ | 4.023 | 9.73 | -- |
79
  | | ASVD | 4.023 | 10.04 | -- |
80
  | **LLaMA-2-13B (base)** | *dense FP16* | 13.016 | 4.88 | 6.74 |
81
- | | SVD-LLM | 7.738 | -- | -- |
82
- | | SVD-LLM V2 ‡ | 7.740 | -- | -- |
83
  | **Qwen2.5-3B** | *dense FP16* | 3.086 | 8.56 | -- |
84
  | | TR-3 (ours) | 2.078 | 11.75 | -- |
85
  | | TT-3 (open chain) | 2.077 | -- | -- |
@@ -106,10 +106,10 @@ Perplexity is **token-level** over fixed non-overlapping 2048-token windows, fol
106
  | | TR-3 (ours) | 70.95 | 39.16 | 65.24 | 68.02 | 75.35 | 64.48 | 25.66 | 58.41 |
107
  | | SVD-LLM | 68.62 | 33.19 | 58.71 | 55.58 | 71.16 | 61.33 | 24.19 | 53.25 |
108
  | | SVD-LLM V2 ‡ | 68.53 | 33.96 | 60.06 | 56.49 | 72.31 | 62.83 | 25.59 | 54.25 |
109
- | | ASVD | 59.54 | -- | -- | -- | -- | -- | -- | -- |
110
  | **LLaMA-2-13B (base)** | *dense FP16* | 82.11 | 48.81 | 76.47 | 76.54 | 80.25 | 72.45 | 31.46 | 66.87 |
111
- | | SVD-LLM | -- | -- | -- | -- | -- | -- | -- | -- |
112
- | | SVD-LLM V2 ‡ | -- | -- | -- | -- | -- | -- | -- | -- |
113
  | **Qwen2.5-3B** | *dense FP16* | 80.09 | 48.04 | 72.90 | 66.12 | 78.29 | 69.38 | 35.61 | 64.35 |
114
  | | TR-3 (ours) | 76.39 | 43.17 | 69.36 | 60.51 | 75.24 | 64.09 | 34.17 | 60.42 |
115
  | | TT-3 (open chain) | 66.88 | 31.14 | 56.19 | 49.16 | 70.40 | 59.12 | 27.64 | 51.50 |
 
78
  | | SVD-LLM V2 ‡ | 4.023 | 9.73 | -- |
79
  | | ASVD | 4.023 | 10.04 | -- |
80
  | **LLaMA-2-13B (base)** | *dense FP16* | 13.016 | 4.88 | 6.74 |
81
+ | | SVD-LLM | 7.738 | 9.14 | -- |
82
+ | | SVD-LLM V2 ‡ | 7.740 | 8.74 | -- |
83
  | **Qwen2.5-3B** | *dense FP16* | 3.086 | 8.56 | -- |
84
  | | TR-3 (ours) | 2.078 | 11.75 | -- |
85
  | | TT-3 (open chain) | 2.077 | -- | -- |
 
106
  | | TR-3 (ours) | 70.95 | 39.16 | 65.24 | 68.02 | 75.35 | 64.48 | 25.66 | 58.41 |
107
  | | SVD-LLM | 68.62 | 33.19 | 58.71 | 55.58 | 71.16 | 61.33 | 24.19 | 53.25 |
108
  | | SVD-LLM V2 ‡ | 68.53 | 33.96 | 60.06 | 56.49 | 72.31 | 62.83 | 25.59 | 54.25 |
109
+ | | ASVD | 59.54 | 32.17 | 57.83 | 57.83 | 71.49 | 60.14 | 23.92 | 51.85 |
110
  | **LLaMA-2-13B (base)** | *dense FP16* | 82.11 | 48.81 | 76.47 | 76.54 | 80.25 | 72.45 | 31.46 | 66.87 |
111
+ | | SVD-LLM | 70.34 | 36.01 | 64.94 | 60.94 | 73.78 | 65.82 | 24.69 | 56.64 |
112
+ | | SVD-LLM V2 ‡ | 71.13 | 38.14 | 67.30 | 61.67 | 73.83 | 65.67 | 25.66 | 57.63 |
113
  | **Qwen2.5-3B** | *dense FP16* | 80.09 | 48.04 | 72.90 | 66.12 | 78.29 | 69.38 | 35.61 | 64.35 |
114
  | | TR-3 (ours) | 76.39 | 43.17 | 69.36 | 60.51 | 75.24 | 64.09 | 34.17 | 60.42 |
115
  | | TT-3 (open chain) | 66.88 | 31.14 | 56.19 | 49.16 | 70.40 | 59.12 | 27.64 | 51.50 |