ironbcc commited on
Commit
3d03a11
·
verified ·
1 Parent(s): f109955

add blog/post.md

Browse files
Files changed (1) hide show
  1. blog/post.md +3 -3
blog/post.md CHANGED
@@ -1,14 +1,14 @@
1
  # Seb-9B-v2: beats hosted Jev 1.13 on the Decision Index, at 22 ms, on your own hardware
2
 
3
- Seb-9B-v2 scores **59.16** on the Decision Index public index, ahead of hosted Jev 1.13 (57.96). It takes about **22 ms** a decision on one GPU or **88 ms** on a Mac, and nothing leaves your machine. On public decision rows it is level with Claude Haiku and gpt-6-luna. Put in front of Claude Opus as a first tier, it answered about half the requests itself and kept Opus-level accuracy overall, at an estimated **51% lower cost**.
4
 
5
- It is a decision model: you give it a situation, a question and a few options, and it returns a probability for every option in one forward pass. No text is generated, so there is nothing to parse. It reads images, takes inputs up to 32k tokens, and speaks both a regular chat API and the System One API.
6
 
7
  ![Seb-9B-v2](https://huggingface.co/ironbcc/Seb-9B-v2/resolve/main/blog/hero.png)
8
 
9
  ## TL;DR
10
 
11
- - **Quality.** 0.801 mean accuracy over 17 public suites (our v1 0.793, hosted Jev 1.13 0.786). BANKING77 0.973, jailbreak detection 0.940, model routing 0.980. On 420 public decision rows it is level with gpt-6-luna and Claude Haiku (medium effort), and behind Claude Opus.
12
  - **Speed.** About 22 ms per decision on one RTX PRO 6000 and about 88 ms on an Apple M5 Max, up to roughly 87 decisions a second per GPU.
13
  - **Decision Index.** 59.16 on the 0.3 public index, against 57.96 for hosted Jev 1.13 and 60.70 for Jev 1.14. Ahead of Jev 1.13 on 22 of 37 benchmarks and in 4 of 5 areas.
14
 
 
1
  # Seb-9B-v2: beats hosted Jev 1.13 on the Decision Index, at 22 ms, on your own hardware
2
 
3
+ Seb-9B-v2 is a 9B decision model that scores **59.16** on the Decision Index, ahead of hosted Jev 1.13 (57.96). It runs in about **22 ms** a decision on one GPU or **88 ms** on a Mac, and your data never leaves your machine. On public decision rows it edges out Claude Haiku and gpt-6-luna (0.798 vs 0.795 and 0.779, within noise). Put in front of Claude Opus as a first tier, it answered about half the requests itself and kept Opus-level accuracy overall, at an estimated **51% lower cost**.
4
 
5
+ You give it a situation, a question and a few options, and it returns a probability for every option in one forward pass. No text is generated, so there is nothing to parse. It reads images, takes inputs up to 32k tokens, and speaks both a regular chat API and the System One API.
6
 
7
  ![Seb-9B-v2](https://huggingface.co/ironbcc/Seb-9B-v2/resolve/main/blog/hero.png)
8
 
9
  ## TL;DR
10
 
11
+ - **Quality.** 0.801 mean accuracy over 17 public suites (our v1 0.793, hosted Jev 1.13 0.786). BANKING77 0.973, jailbreak detection 0.940, model routing 0.980. On 420 public decision rows it edges out gpt-6-luna and Claude Haiku (medium effort), within noise, and trails Claude Opus.
12
  - **Speed.** About 22 ms per decision on one RTX PRO 6000 and about 88 ms on an Apple M5 Max, up to roughly 87 decisions a second per GPU.
13
  - **Decision Index.** 59.16 on the 0.3 public index, against 57.96 for hosted Jev 1.13 and 60.70 for Jev 1.14. Ahead of Jev 1.13 on 22 of 37 benchmarks and in 4 of 5 areas.
14