hiaripc commited on
Commit
5fc17bf
·
1 Parent(s): f3f23b0
Files changed (1) hide show
  1. index.html +37 -36
index.html CHANGED
@@ -64,42 +64,6 @@
64
  SweetSpot v2 — Energy-per-Token demo
65
  </h2>
66
 
67
- <p>
68
- This interactive demo visualizes our analytical model for predicting the
69
- <b>energy consumption per generated token</b> as a function of the number of
70
- output tokens (<i>n</i><sub>out</sub>).
71
- Select one of the validated models used in the paper or load the architecture
72
- of any <b>dense decoder-only Transformer</b> (up to approximately 20B
73
- parameters) from Hugging Face to explore how the different terms contribute
74
- to the prediction.
75
- </p>
76
-
77
- <p>
78
- The model was fitted using empirical measurements collected on an
79
- <b>NVIDIA H100 GPU</b> with <b>NVIDIA TensorRT-LLM</b> under the following
80
- experimental configuration:
81
- </p>
82
-
83
- <ul>
84
- <li><code>max_batch_size = 1024</code></li>
85
- <li><code>max_num_tokens = 8192</code></li>
86
- <li><code>num_requests = 1000</code></li>
87
- </ul>
88
-
89
- <p>
90
- Future releases will extend the analysis to additional hardware platforms
91
- and inference frameworks.
92
- </p>
93
-
94
- <p>
95
- For the complete methodology, experimental validation, and citation
96
- information, please see the paper:
97
- <a href="YOUR_PAPER_URL" target="_blank">
98
- SweetSpot v2: An Architecture-Aware Analytical Model for Predicting Energy
99
- Efficiency of LLM Inference
100
- </a>.
101
- </p>
102
-
103
  <h4>Validated models:</h4>
104
 
105
  <select id="preset"></select>
@@ -196,6 +160,43 @@ placeholder="tiiuae/falcon-rw-7b"
196
  }
197
  </script>
198
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
199
  <script src="app.js"></script>
200
 
201
  </body>
 
64
  SweetSpot v2 — Energy-per-Token demo
65
  </h2>
66
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
67
  <h4>Validated models:</h4>
68
 
69
  <select id="preset"></select>
 
160
  }
161
  </script>
162
 
163
+
164
+ <p>
165
+ This interactive demo visualizes our analytical model for predicting the
166
+ <b>energy consumption per generated token</b> as a function of the number of
167
+ output tokens (<i>n</i><sub>out</sub>).
168
+ Select one of the validated models used in the paper or load the architecture
169
+ of any <b>dense decoder-only Transformer</b> (up to approximately 20B
170
+ parameters) from Hugging Face to explore how the different terms contribute
171
+ to the prediction.
172
+ </p>
173
+
174
+ <p>
175
+ The model was fitted using empirical measurements collected on an
176
+ <b>NVIDIA H100 GPU</b> with <b>NVIDIA TensorRT-LLM</b> under the following
177
+ experimental configuration:
178
+ </p>
179
+
180
+ <ul>
181
+ <li><code>max_batch_size = 1024</code></li>
182
+ <li><code>max_num_tokens = 8192</code></li>
183
+ <li><code>num_requests = 1000</code></li>
184
+ </ul>
185
+
186
+ <p>
187
+ Future releases will extend the analysis to additional hardware platforms
188
+ and inference frameworks.
189
+ </p>
190
+
191
+ <p>
192
+ For the complete methodology, experimental validation, and citation
193
+ information, please see the paper:
194
+ <a href="YOUR_PAPER_URL" target="_blank">
195
+ SweetSpot v2: An Architecture-Aware Analytical Model for Predicting Energy
196
+ Efficiency of LLM Inference
197
+ </a>.
198
+ </p>
199
+
200
  <script src="app.js"></script>
201
 
202
  </body>