card: held-out text numbers aligned with the site data
Browse files
README.md
CHANGED
|
@@ -20,7 +20,7 @@ This card describes the models and publishes the benchmark numbers. **Weights ar
|
|
| 20 |
## Numbers (measured by us, same candidates, same day; details and caveats at https://llav.app/benchmarks)
|
| 21 |
| Benchmark | llav | Jev 1.14 | Gemini Decision 1 |
|
| 22 |
|---|---|---|---|
|
| 23 |
-
| Text judge, MuSiQue held-out (AUC
|
| 24 |
| Calibration, round 0 (Brier / ECE, lower is better) | **0.0095 / 0.004** | 0.0113 / 0.015 | 0.0118 / 0.031 |
|
| 25 |
| Image judge, 2,458 items (AUC) | 0.950 | **0.955** | 0.942 |
|
| 26 |
| Audio judge, 1,200 clips (AUC) | **0.952** | 0.533 | 0.842 |
|
|
|
|
| 20 |
## Numbers (measured by us, same candidates, same day; details and caveats at https://llav.app/benchmarks)
|
| 21 |
| Benchmark | llav | Jev 1.14 | Gemini Decision 1 |
|
| 22 |
|---|---|---|---|
|
| 23 |
+
| Text judge, MuSiQue held-out q50–200 (AUC) | **0.970** single 4B · 0.977 five-model ensemble | 0.969 | 0.966 |
|
| 24 |
| Calibration, round 0 (Brier / ECE, lower is better) | **0.0095 / 0.004** | 0.0113 / 0.015 | 0.0118 / 0.031 |
|
| 25 |
| Image judge, 2,458 items (AUC) | 0.950 | **0.955** | 0.942 |
|
| 26 |
| Audio judge, 1,200 clips (AUC) | **0.952** | 0.533 | 0.842 |
|