mkirkby commited on
Commit
d304174
·
verified ·
1 Parent(s): 06f37ef

Update model card

Browse files
Files changed (1) hide show
  1. README.md +48 -2
README.md CHANGED
@@ -16,11 +16,20 @@ tags:
16
  - baseten
17
  ---
18
 
 
 
 
 
 
 
19
  # bison-1
20
 
21
  bison-1 is a 31B parameter **decision model** post-trained from [Gemma-4-31B](https://huggingface.co/google/gemma-4-31B-it) by Base Labs at Baseten.
22
  Provide the model a state (text, optionally with images) and a set of questions and it will return a calibrated probability for every option,
23
- answering all questions from **one shared-prefix prefill**, without generating text.
 
 
 
24
 
25
  For faster, cheaper decisions, see the mixture-of-experts sibling [groundhog-1](https://huggingface.co/baselabs/groundhog-1).
26
 
@@ -39,6 +48,12 @@ For faster, cheaper decisions, see the mixture-of-experts sibling [groundhog-1](
39
 
40
  ## 🏃 How to use
41
 
 
 
 
 
 
 
42
  A request carries one `state` and any number of `questions`. Every question is answered in the same call.
43
 
44
  ```python
@@ -68,9 +83,40 @@ request = {"state": "I was charged twice this month, please refund one of them."
68
 
69
  Requests longer than the trained length limits are refused, not truncated.
70
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
71
  ## 🔧 Training
72
 
73
  - **Data**: 6.71M decision examples (Mix 12): core decision skills, safety and abstention, judging and expert domains, document and long-context tasks, and public decision data, including public benchmark sets.
74
- - **Objective**: cross-entropy plus a Brier calibration term (λ = 1), with a trained "I don't know" option.
75
  - **Recipe**: 1 epoch (3,747 steps), peak LR 5e-6, no weight decay, 3% warm-up, bf16, sequences up to 131,072 tokens.
76
  - **Hardware**: 24 nodes × 8 B200 (192 GPUs).
 
16
  - baseten
17
  ---
18
 
19
+ <div align="center">
20
+ <div style="display: flex; justify-content: center; gap: 0.5em; margin-bottom: 1em;">
21
+ <a href="https://www.baseten.co/library/bison-1/"><strong>Try on Baseten</strong></a>
22
+ </div>
23
+ </div>
24
+
25
  # bison-1
26
 
27
  bison-1 is a 31B parameter **decision model** post-trained from [Gemma-4-31B](https://huggingface.co/google/gemma-4-31B-it) by Base Labs at Baseten.
28
  Provide the model a state (text, optionally with images) and a set of questions and it will return a calibrated probability for every option,
29
+ answering all questions from **one shared-prefix prefill**, without generating text. On tested benchmarks:
30
+
31
+ - **#1 of 19 decision models on the Artificial Analysis Decision Index v2.1**: 76, ahead of d1.2-ultra (75) and mammoth-1 (71).
32
+ - **Best-calibrated model on that index**: Brier score 0.14.
33
 
34
  For faster, cheaper decisions, see the mixture-of-experts sibling [groundhog-1](https://huggingface.co/baselabs/groundhog-1).
35
 
 
48
 
49
  ## 🏃 How to use
50
 
51
+ ### Baseten Model APIs
52
+
53
+ bison-1 is available on [Baseten Model APIs](https://www.baseten.co/library/bison-1/) as `decision-models/baseten-bison-latest`, at $0.10 per 1M tokens. It uses the same API as mammoth-1 and elephant-1, and the endpoint accepts text and image inputs.
54
+
55
+ ### Request format
56
+
57
  A request carries one `state` and any number of `questions`. Every question is answered in the same call.
58
 
59
  ```python
 
83
 
84
  Requests longer than the trained length limits are refused, not truncated.
85
 
86
+ ## 📊 Performance
87
+
88
+ ### Artificial Analysis Decision Index v2.1
89
+
90
+ Scored by Artificial Analysis, November 19, 2026.
91
+
92
+ | Model | Decision Index |
93
+ |---|---:|
94
+ | **bison-1** | **76** |
95
+ | d1.2-ultra | 75 |
96
+ | Drex Max v1.1 | 73 |
97
+ | SubQ Snap 1.2 | 73 |
98
+ | GLiDE v1.2 | 73 |
99
+ | mammoth-1 | 71 |
100
+
101
+ | Benchmark | bison-1 | Rank of 19 |
102
+ |---|---:|---:|
103
+ | AA-Paperclip | 73 | 2 |
104
+ | EvalSafe-O*NET-AA | 93 | 1 |
105
+ | When2Call | 98 | 1 |
106
+ | AA-Omniscience-Decision v1.1 | 60 | 11 |
107
+ | HLE-MultiChoice v1.1 | 51 | 1 |
108
+ | AA-Symmetry v1.1 | 89 | 1 |
109
+
110
+ ### Long context (our run)
111
+
112
+ | Benchmark | bison-1 |
113
+ |---|---:|
114
+ | LongMemEval M, 128k tokens | 84.8 |
115
+ | LongMemEval S, 128k tokens | 87.1 |
116
+
117
  ## 🔧 Training
118
 
119
  - **Data**: 6.71M decision examples (Mix 12): core decision skills, safety and abstention, judging and expert domains, document and long-context tasks, and public decision data, including public benchmark sets.
120
+ - **Objective**: cross-entropy plus a Brier calibration term (λ = 1), with a trained "I don't know" option. About one example in seven carries a probability target instead of a single right answer.
121
  - **Recipe**: 1 epoch (3,747 steps), peak LR 5e-6, no weight decay, 3% warm-up, bf16, sequences up to 131,072 tokens.
122
  - **Hardware**: 24 nodes × 8 B200 (192 GPUs).