lionel-mp commited on
Commit
79f4f3e
·
verified ·
1 Parent(s): 2a08aa6

Card: board of 12 November 2026, highest Full score of 300 entries; new leaderboard screenshot

Browse files
.gitattributes CHANGED
@@ -35,3 +35,4 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
35
  *tfevents* filter=lfs diff=lfs merge=lfs -text
36
  tokenizer.json filter=lfs diff=lfs merge=lfs -text
37
  assets/decision-index-full-2026-11-11.jpg filter=lfs diff=lfs merge=lfs -text
 
 
35
  *tfevents* filter=lfs diff=lfs merge=lfs -text
36
  tokenizer.json filter=lfs diff=lfs merge=lfs -text
37
  assets/decision-index-full-2026-11-11.jpg filter=lfs diff=lfs merge=lfs -text
38
+ assets/decision-index-full-2026-11-12.png filter=lfs diff=lfs merge=lfs -text
README.md CHANGED
@@ -23,7 +23,7 @@ model-index:
23
  metrics:
24
  - type: decision_index_full
25
  value: 73.86
26
- name: "Decision Index 0.4.1, Full score (leaderboard, 11 November 2026; rank #1, tied)"
27
  source:
28
  name: Decision Index leaderboard
29
  url: https://huggingface.co/spaces/multimodalart/jev-decision-index
@@ -46,18 +46,18 @@ model-index:
46
 
47
  <p>
48
  <a href="https://github.com/machinepulse-ai/karpo-decision"><img alt="GitHub" src="https://img.shields.io/badge/GitHub-machinepulse--ai%2Fkarpo--decision-181717?style=flat-square&logo=github&logoColor=white"></a>
49
- <a href="https://huggingface.co/spaces/multimodalart/jev-decision-index"><img alt="Decision Index: #1 (tied), Full 73.9" src="https://img.shields.io/badge/Decision_Index-%231_(tied)_%C2%B7_Full_73.9-FFD21E?style=flat-square"></a>
50
  <a href="https://huggingface.co/datasets/MachinePulseAI/karpo-decision-31B-decision-index"><img alt="Results" src="https://img.shields.io/badge/Results-Decision_Index_run-0A84FF?style=flat-square&logo=huggingface&logoColor=white"></a>
51
  <a href="https://huggingface.co/MachinePulseAI/karpo-decision-31B/blob/main/LICENSE"><img alt="License: MachinePulse Community License" src="https://img.shields.io/badge/License-MachinePulse_Community-4b5563?style=flat-square"></a>
52
  </p>
53
 
54
- **Ranked #1 (tied) on the [Decision Index](https://huggingface.co/spaces/multimodalart/jev-decision-index)**, with a
55
- Full score of **73.9** on the maintainers' private tests (edition 0.4.1, board of 11 November 2026, 297 entries).
 
56
 
57
- <img src="assets/decision-index-full-2026-11-11.jpg" alt="Decision Index 0.4.1 leaderboard, Full score, 11 November 2026: karpo-decision-31B ranked #1 (tied) with 73.9" width="100%">
58
 
59
- *Decision Index leaderboard, Full score, 11 November 2026, with the board's "inference technique" entries hidden; one of
60
- them, REAX 1.3, is also in the #1 group.*
61
 
62
  A decision model: given a state (a web page, a document, a conversation: any text or JSON) and typed questions about
63
  it, it returns a probability for every option of every question in one forward pass, without generating text.
@@ -81,17 +81,17 @@ The [board](https://huggingface.co/spaces/multimodalart/jev-decision-index) rank
81
  maintainers measure on private tests only: 60% tests of the same skills as the public suite and 40% tasks from new
82
  domains.
83
 
84
- | Decision Index 0.4.1, board of 11 November 2026 | karpo-decision-31B |
85
  |---|---|
86
- | **Full score** | **73.9**, rank **#1 (tied)** |
87
  | Private tests, same skills (on the public scale) | 70.7 |
88
  | Private tasks, new domains (on the public scale) | 78.6 |
89
  | Public index (our run, adopted by the board) | 75.0 |
90
  | Median latency (one GPU, one request at a time) | 94 ms per request |
91
  | Calibration error | 4.8 points |
92
 
93
- The board gives one rank to models within 0.9 points of the first model in a group. Three entries share #1: GLiDE 27B,
94
- karpo-decision-31B and REAX 1.3 (an inference technique, hidden in the chart above).
95
 
96
  ### Decision Index public suite
97
 
 
23
  metrics:
24
  - type: decision_index_full
25
  value: 73.86
26
+ name: "Decision Index 0.4.1, Full score (leaderboard, 12 November 2026; rank #1, the highest score)"
27
  source:
28
  name: Decision Index leaderboard
29
  url: https://huggingface.co/spaces/multimodalart/jev-decision-index
 
46
 
47
  <p>
48
  <a href="https://github.com/machinepulse-ai/karpo-decision"><img alt="GitHub" src="https://img.shields.io/badge/GitHub-machinepulse--ai%2Fkarpo--decision-181717?style=flat-square&logo=github&logoColor=white"></a>
49
+ <a href="https://huggingface.co/spaces/multimodalart/jev-decision-index"><img alt="Decision Index: #1, Full 73.9" src="https://img.shields.io/badge/Decision_Index-%231_%C2%B7_Full_73.9-FFD21E?style=flat-square"></a>
50
  <a href="https://huggingface.co/datasets/MachinePulseAI/karpo-decision-31B-decision-index"><img alt="Results" src="https://img.shields.io/badge/Results-Decision_Index_run-0A84FF?style=flat-square&logo=huggingface&logoColor=white"></a>
51
  <a href="https://huggingface.co/MachinePulseAI/karpo-decision-31B/blob/main/LICENSE"><img alt="License: MachinePulse Community License" src="https://img.shields.io/badge/License-MachinePulse_Community-4b5563?style=flat-square"></a>
52
  </p>
53
 
54
+ **Ranked #1 on the [Decision Index](https://huggingface.co/spaces/multimodalart/jev-decision-index)**, with the
55
+ highest Full score, **73.9**, on the maintainers' private tests (edition 0.4.1, board of 12 November 2026, 300
56
+ entries).
57
 
58
+ <img src="assets/decision-index-full-2026-11-12.png" alt="Decision Index 0.4.1 leaderboard, Full score, 12 November 2026: karpo-decision-31B first with 73.9" width="100%">
59
 
60
+ *Decision Index leaderboard, Full score, 12 November 2026.*
 
61
 
62
  A decision model: given a state (a web page, a document, a conversation: any text or JSON) and typed questions about
63
  it, it returns a probability for every option of every question in one forward pass, without generating text.
 
81
  maintainers measure on private tests only: 60% tests of the same skills as the public suite and 40% tasks from new
82
  domains.
83
 
84
+ | Decision Index 0.4.1, board of 12 November 2026 | karpo-decision-31B |
85
  |---|---|
86
+ | **Full score** | **73.9**, rank **#1**, the highest of 300 entries |
87
  | Private tests, same skills (on the public scale) | 70.7 |
88
  | Private tasks, new domains (on the public scale) | 78.6 |
89
  | Public index (our run, adopted by the board) | 75.0 |
90
  | Median latency (one GPU, one request at a time) | 94 ms per request |
91
  | Calibration error | 4.8 points |
92
 
93
+ The board gives one rank to models within 0.9 points of the first model in a group: karpo-decision-31B (73.9) and
94
+ GLiDE 27B (73.4) share #1.
95
 
96
  ### Decision Index public suite
97
 
assets/{decision-index-full-2026-11-11.jpg → decision-index-full-2026-11-12.png} RENAMED
File without changes