Card: size and speed (measured, incl. Bev-Pro-C on one H200)

.gitattributes

@@ -34,3 +34,4 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
 assets/automation.png filter=lfs diff=lfs merge=lfs -text
+assets/speed.png filter=lfs diff=lfs merge=lfs -text

README.md

@@ -123,10 +123,11 @@ model = bev.load("sathiskumarjothi0/Bev-Large-C", device="mps")    # Apple Silic
 
 Command line: `bev "I was charged twice, please refund" --noul "Does the customer ask for a refund?" --device cpu`.
 
-| Device (measured on an Apple M4 Pro laptop) | First load | One document with 3 questions |
+| Device | First load | One document with 3 questions |
 |---|---|---|
-| CPU only | about 25 s | about 0.18 s |
-| Apple GPU (`mps`) | about 22 s | about 0.05 s |
+| Apple M4 Pro laptop, CPU only | about 25 s | about 0.18 s |
+| Apple M4 Pro laptop, GPU (`mps`) | about 22 s | about 0.05 s |
+| One NVIDIA H200 GPU (`cuda`) | about 32 s | about 0.015 s |
 
 The first load includes reading the 1.6 GB weights; after that, use `decide_batch` for many documents.
 
@@ -240,6 +241,25 @@ best in the row; – = not run. * = an area Bev was not built for.
 | Legal case holdings* | 42.0 | 71.2 | **77.6** | 69.2 | 72.2 | 61.2 | 71.6 | 55.0 |
 
 
+### Size and speed
+
+![Accuracy against model size](assets/size.png)
+
+Accuracy on the same 10,238 answers against total parameters (log scale), for every system with a published size:
+Bev-Large-C (0.4B) and Bev-Pro-C (25.8B) are more accurate than LLMs of 31B to 675B.
+
+![Time per decision](assets/speed.png)
+
+| Bev, one document with 3 questions (package from PyPI) | Per decision | In batches of 64 |
+|---|---|---|
+| Bev-Large-C, one NVIDIA H200 GPU | 4.9 ms | 3.4 ms |
+| Bev-Large-C, Apple M4 Pro laptop GPU | 17 ms | |
+| Bev-Large-C, Apple M4 Pro laptop CPU | 60 ms | |
+| Bev-Pro-C, one NVIDIA H200 GPU | 22 ms | 7.8 ms |
+
+The LLMs and Jev were measured through their cloud APIs (median response time per decision, including network and
+queueing): Jev 98 ms, the five LLMs 4.1 to 7.0 s.
+
 ### How much of the work can you automate?
 
 ![Share of decisions automated at a target accuracy](assets/automation.png)