celestwong commited on
Commit
f35b691
·
verified ·
1 Parent(s): e1395aa

card: held-out text numbers aligned with the site data

Browse files
Files changed (1) hide show
  1. README.md +1 -1
README.md CHANGED
@@ -20,7 +20,7 @@ This card describes the models and publishes the benchmark numbers. **Weights ar
20
  ## Numbers (measured by us, same candidates, same day; details and caveats at https://llav.app/benchmarks)
21
  | Benchmark | llav | Jev 1.14 | Gemini Decision 1 |
22
  |---|---|---|---|
23
- | Text judge, MuSiQue held-out (AUC / R@10) | **0.977 / 0.942** | 0.969 / 0.924 | 0.941 / 0.845 |
24
  | Calibration, round 0 (Brier / ECE, lower is better) | **0.0095 / 0.004** | 0.0113 / 0.015 | 0.0118 / 0.031 |
25
  | Image judge, 2,458 items (AUC) | 0.950 | **0.955** | 0.942 |
26
  | Audio judge, 1,200 clips (AUC) | **0.952** | 0.533 | 0.842 |
 
20
  ## Numbers (measured by us, same candidates, same day; details and caveats at https://llav.app/benchmarks)
21
  | Benchmark | llav | Jev 1.14 | Gemini Decision 1 |
22
  |---|---|---|---|
23
+ | Text judge, MuSiQue held-out q50–200 (AUC) | **0.970** single 4B · 0.977 five-model ensemble | 0.969 | 0.966 |
24
  | Calibration, round 0 (Brier / ECE, lower is better) | **0.0095 / 0.004** | 0.0113 / 0.015 | 0.0118 / 0.031 |
25
  | Image judge, 2,458 items (AUC) | 0.950 | **0.955** | 0.942 |
26
  | Audio judge, 1,200 clips (AUC) | **0.952** | 0.533 | 0.842 |