hiaripc commited on
Commit ·
5fc17bf
1
Parent(s): f3f23b0
style
Browse files- index.html +37 -36
index.html
CHANGED
|
@@ -64,42 +64,6 @@
|
|
| 64 |
SweetSpot v2 — Energy-per-Token demo
|
| 65 |
</h2>
|
| 66 |
|
| 67 |
-
<p>
|
| 68 |
-
This interactive demo visualizes our analytical model for predicting the
|
| 69 |
-
<b>energy consumption per generated token</b> as a function of the number of
|
| 70 |
-
output tokens (<i>n</i><sub>out</sub>).
|
| 71 |
-
Select one of the validated models used in the paper or load the architecture
|
| 72 |
-
of any <b>dense decoder-only Transformer</b> (up to approximately 20B
|
| 73 |
-
parameters) from Hugging Face to explore how the different terms contribute
|
| 74 |
-
to the prediction.
|
| 75 |
-
</p>
|
| 76 |
-
|
| 77 |
-
<p>
|
| 78 |
-
The model was fitted using empirical measurements collected on an
|
| 79 |
-
<b>NVIDIA H100 GPU</b> with <b>NVIDIA TensorRT-LLM</b> under the following
|
| 80 |
-
experimental configuration:
|
| 81 |
-
</p>
|
| 82 |
-
|
| 83 |
-
<ul>
|
| 84 |
-
<li><code>max_batch_size = 1024</code></li>
|
| 85 |
-
<li><code>max_num_tokens = 8192</code></li>
|
| 86 |
-
<li><code>num_requests = 1000</code></li>
|
| 87 |
-
</ul>
|
| 88 |
-
|
| 89 |
-
<p>
|
| 90 |
-
Future releases will extend the analysis to additional hardware platforms
|
| 91 |
-
and inference frameworks.
|
| 92 |
-
</p>
|
| 93 |
-
|
| 94 |
-
<p>
|
| 95 |
-
For the complete methodology, experimental validation, and citation
|
| 96 |
-
information, please see the paper:
|
| 97 |
-
<a href="YOUR_PAPER_URL" target="_blank">
|
| 98 |
-
SweetSpot v2: An Architecture-Aware Analytical Model for Predicting Energy
|
| 99 |
-
Efficiency of LLM Inference
|
| 100 |
-
</a>.
|
| 101 |
-
</p>
|
| 102 |
-
|
| 103 |
<h4>Validated models:</h4>
|
| 104 |
|
| 105 |
<select id="preset"></select>
|
|
@@ -196,6 +160,43 @@ placeholder="tiiuae/falcon-rw-7b"
|
|
| 196 |
}
|
| 197 |
</script>
|
| 198 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 199 |
<script src="app.js"></script>
|
| 200 |
|
| 201 |
</body>
|
|
|
|
| 64 |
SweetSpot v2 — Energy-per-Token demo
|
| 65 |
</h2>
|
| 66 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 67 |
<h4>Validated models:</h4>
|
| 68 |
|
| 69 |
<select id="preset"></select>
|
|
|
|
| 160 |
}
|
| 161 |
</script>
|
| 162 |
|
| 163 |
+
|
| 164 |
+
<p>
|
| 165 |
+
This interactive demo visualizes our analytical model for predicting the
|
| 166 |
+
<b>energy consumption per generated token</b> as a function of the number of
|
| 167 |
+
output tokens (<i>n</i><sub>out</sub>).
|
| 168 |
+
Select one of the validated models used in the paper or load the architecture
|
| 169 |
+
of any <b>dense decoder-only Transformer</b> (up to approximately 20B
|
| 170 |
+
parameters) from Hugging Face to explore how the different terms contribute
|
| 171 |
+
to the prediction.
|
| 172 |
+
</p>
|
| 173 |
+
|
| 174 |
+
<p>
|
| 175 |
+
The model was fitted using empirical measurements collected on an
|
| 176 |
+
<b>NVIDIA H100 GPU</b> with <b>NVIDIA TensorRT-LLM</b> under the following
|
| 177 |
+
experimental configuration:
|
| 178 |
+
</p>
|
| 179 |
+
|
| 180 |
+
<ul>
|
| 181 |
+
<li><code>max_batch_size = 1024</code></li>
|
| 182 |
+
<li><code>max_num_tokens = 8192</code></li>
|
| 183 |
+
<li><code>num_requests = 1000</code></li>
|
| 184 |
+
</ul>
|
| 185 |
+
|
| 186 |
+
<p>
|
| 187 |
+
Future releases will extend the analysis to additional hardware platforms
|
| 188 |
+
and inference frameworks.
|
| 189 |
+
</p>
|
| 190 |
+
|
| 191 |
+
<p>
|
| 192 |
+
For the complete methodology, experimental validation, and citation
|
| 193 |
+
information, please see the paper:
|
| 194 |
+
<a href="YOUR_PAPER_URL" target="_blank">
|
| 195 |
+
SweetSpot v2: An Architecture-Aware Analytical Model for Predicting Energy
|
| 196 |
+
Efficiency of LLM Inference
|
| 197 |
+
</a>.
|
| 198 |
+
</p>
|
| 199 |
+
|
| 200 |
<script src="app.js"></script>
|
| 201 |
|
| 202 |
</body>
|