org card: requests-per-GPU multiple to ~3.7x (nearly 4x), T=2 cached
Browse filesAlign the org profile throughput claim with the featured updated economics (was ~3x). Tokens (~49%), accuracy (+7.4pp), and storage (~5x) unchanged.
README.md
CHANGED
|
@@ -24,7 +24,7 @@ model to read the compressed representation. On ScienceQA the recipe holds or
|
|
| 24 |
improves accuracy while shrinking the visual sequence:
|
| 25 |
|
| 26 |
- **+8.4pp accuracy** over the un-spliced baseline at the accuracy tier (T=3, n=200)
|
| 27 |
-
- At the compression tier (T=2, +7.4pp accuracy): **~49% fewer visual tokens** and **~
|
| 28 |
- **~5x smaller** stored visual representation
|
| 29 |
|
| 30 |
The vision encoder and LLM weights are otherwise untouched; the compression sits
|
|
|
|
| 24 |
improves accuracy while shrinking the visual sequence:
|
| 25 |
|
| 26 |
- **+8.4pp accuracy** over the un-spliced baseline at the accuracy tier (T=3, n=200)
|
| 27 |
+
- At the compression tier (T=2, +7.4pp accuracy): **~49% fewer visual tokens** and **~3.7x (nearly 4x) more requests per GPU** in cached serving
|
| 28 |
- **~5x smaller** stored visual representation
|
| 29 |
|
| 30 |
The vision encoder and LLM weights are otherwise untouched; the compression sits
|