deepanshupillm commited on
Commit
c5a27bf
·
verified ·
1 Parent(s): 189fbed

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +24 -15
README.md CHANGED
@@ -91,6 +91,15 @@ Alpie Seed uses a two-stage post-training pipeline.
91
 
92
  Training used eight NVIDIA H100 GPUs, with 3,034 GRPO steps across five phases. The reward combines seven signals: answer correctness, hallucination penalty, self-consistency, educational quality, structure preservation, repetition penalty, and length penalty.
93
 
 
 
 
 
 
 
 
 
 
94
  <br>
95
 
96
  <div align="center">
@@ -107,37 +116,37 @@ Training used eight NVIDIA H100 GPUs, with 3,034 GRPO steps across five phases.
107
 
108
  </div>
109
 
110
- > Alpie Seed scores are from the FP16 model and do not apply to the 2-bit, 3-bit, or 4-bit variants. 🏆 gold badge = best score in the row; purple badge = Alpie Seed score. Bold 🏆 = best score in the row. Comparison-model scores are taken from published model cards and reports (including the [Qwen3.5-9B model card](https://huggingface.co/Qwen/Qwen3.5-9B)).
111
 
112
  ### 🧮 Language
113
 
114
  | | Qwen3.5-9B | Gemma 4 12B | Ministral 3 8B | GPT-OSS-20B | Sarvam-30B | Nemotron 3.5 Lightning | **Alpie Seed 9B** |
115
  |:---|:---:|:---:|:---:|:---:|:---:|:---:|:---:|
116
  | **Knowledge & STEM** | | | | | | | |
117
- | MMLU | -- | -- | -- | 85.3 | 85.1 | -- | ![87.30](https://img.shields.io/badge/87.30-F59E0B?style=flat-square) 🏆 |
118
- | MMLU-Pro | 82.5 | 77.2 | 75.00 | 77.2 | 80.0 | 81.94 | ![82.54](https://img.shields.io/badge/82.54-F59E0B?style=flat-square) 🏆 |
119
- | GPQA Diamond | 81.7 | 78.8 | 38.6 | 71.5 | 66.5 | 75.44 | ![84.32](https://img.shields.io/badge/84.32-F59E0B?style=flat-square) 🏆 |
120
  | **Mathematics** | | | | | | | |
121
  | GSM8K | -- | -- | -- | -- | -- | -- | ![98.00](https://img.shields.io/badge/98.00-7C3AED?style=flat-square) |
122
- | Math-500 | -- | -- | -- | 94.2 | **97.0** 🏆 | -- | ![91.00](https://img.shields.io/badge/91.00-7C3AED?style=flat-square) |
123
- | AIME 2026 | 69.4 | 77.5 | -- | -- | -- | -- | ![83.33](https://img.shields.io/badge/83.33-F59E0B?style=flat-square) 🏆 |
124
  | **Coding & Software Engineering** | | | | | | | |
125
- | LiveCodeBench v6 | 65.6 | 72.0 | 61.6 | 61.0 | 70.0 | -- | ![72.99](https://img.shields.io/badge/72.99-F59E0B?style=flat-square) 🏆 |
126
- | SWE-Bench Verified | 60.0 | 52.0 | -- | 60.7 | 34.0 | 51.56 | ![61.20](https://img.shields.io/badge/61.20-F59E0B?style=flat-square) 🏆 |
127
 
128
  ### 👁️ Vision Language
129
 
130
  | | GPT-5-Nano-2025-08-07 | Gemini-2.5-Flash-Lite | Qwen3-VL-30B-A3B | Qwen3.5-9B | Gemma 4 12B | **Alpie Seed 9B** |
131
  |:---|:---:|:---:|:---:|:---:|:---:|:---:|
132
  | **STEM and Puzzle** | | | | | | |
133
- | MMMU-Pro | 57.2 | 59.7 | 63.0 | **70.1** 🏆 | 69.1 | ![68.70](https://img.shields.io/badge/68.70-7C3AED?style=flat-square) |
134
- | MathVision | 62.2 | 52.1 | 65.7 | 78.9 | **79.7** 🏆 | ![76.76](https://img.shields.io/badge/76.76-7C3AED?style=flat-square) |
135
  | **General VQA** | | | | | | |
136
- | RealWorldQA | 71.8 | 72.2 | 77.4 | 80.3 | -- | ![80.39](https://img.shields.io/badge/80.39-F59E0B?style=flat-square) 🏆 |
137
- | MMStar | 68.6 | 69.1 | 75.5 | **79.7** 🏆 | -- | ![78.87](https://img.shields.io/badge/78.87-7C3AED?style=flat-square) |
138
- | HallusionBench | 58.4 | 64.5 | 66.0 | 69.3 | -- | ![79.88](https://img.shields.io/badge/79.88-F59E0B?style=flat-square) 🏆 |
139
  | **Text Recognition and Document Understanding** | | | | | | |
140
- | OCRBench | 75.3 | 82.5 | 83.9 | 89.2 | -- | ![90.30](https://img.shields.io/badge/90.30-F59E0B?style=flat-square) 🏆 |
141
 
142
  * SWE-Bench Verified: resolved rate using Mini-SWE-agent, with retries disabled.
143
  * Comparison-model scores were not re-run by 169Pi, and harnesses, prompts, and benchmark versions differ, so cross-model comparisons are indicative rather than strictly like-for-like. Scores vary with dataset versions, prompts, evaluation harnesses, and inference settings.
@@ -155,7 +164,7 @@ vllm serve 169Pi/Alpie_seed --port 8000 --max-model-len 262144 --reasoning-parse
155
  Run locally with Ollama:
156
 
157
  ```shell
158
- ollama run 169pi/alpie-core
159
  ```
160
 
161
  Quantized 2-bit, 3-bit, and 4-bit builds are in the [GGUF repository](https://huggingface.co/169Pi/Alpie_Seed_GGUF). Thinking and non-thinking behavior is controlled through the chat template.
 
91
 
92
  Training used eight NVIDIA H100 GPUs, with 3,034 GRPO steps across five phases. The reward combines seven signals: answer correctness, hallucination penalty, self-consistency, educational quality, structure preservation, repetition penalty, and length penalty.
93
 
94
+ ## Links
95
+
96
+ - GitHub: https://github.com/169Pi
97
+ - LinkedIn: https://www.linkedin.com/company/169pi
98
+ - Instagram: https://www.instagram.com/169pi.ai/
99
+ - YouTube: https://www.youtube.com/@169pi
100
+ - X: https://x.com/169Pi_ai
101
+ - Ollama: https://ollama.com/169pi
102
+
103
  <br>
104
 
105
  <div align="center">
 
116
 
117
  </div>
118
 
119
+ > Alpie Seed scores are from the FP16 model and do not apply to the 2-bit, 3-bit, or 4-bit variants. Gold badge = best score in the row; purple badge = Alpie Seed score. Bold = best score in the row. Comparison-model scores are taken from published model cards and reports (including the [Qwen3.5-9B model card](https://huggingface.co/Qwen/Qwen3.5-9B)).
120
 
121
  ### 🧮 Language
122
 
123
  | | Qwen3.5-9B | Gemma 4 12B | Ministral 3 8B | GPT-OSS-20B | Sarvam-30B | Nemotron 3.5 Lightning | **Alpie Seed 9B** |
124
  |:---|:---:|:---:|:---:|:---:|:---:|:---:|:---:|
125
  | **Knowledge & STEM** | | | | | | | |
126
+ | MMLU | -- | -- | -- | 85.3 | 85.1 | -- | ![87.30](https://img.shields.io/badge/87.30-F59E0B?style=flat-square) |
127
+ | MMLU-Pro | 82.5 | 77.2 | 75.00 | 77.2 | 80.0 | 81.94 | ![82.54](https://img.shields.io/badge/82.54-F59E0B?style=flat-square) |
128
+ | GPQA Diamond | 81.7 | 78.8 | 38.6 | 71.5 | 66.5 | 75.44 | ![84.32](https://img.shields.io/badge/84.32-F59E0B?style=flat-square) |
129
  | **Mathematics** | | | | | | | |
130
  | GSM8K | -- | -- | -- | -- | -- | -- | ![98.00](https://img.shields.io/badge/98.00-7C3AED?style=flat-square) |
131
+ | Math-500 | -- | -- | -- | 94.2 | **97.0** | -- | ![91.00](https://img.shields.io/badge/91.00-7C3AED?style=flat-square) |
132
+ | AIME 2026 | 69.4 | 77.5 | -- | -- | -- | -- | ![83.33](https://img.shields.io/badge/83.33-F59E0B?style=flat-square) |
133
  | **Coding & Software Engineering** | | | | | | | |
134
+ | LiveCodeBench v6 | 65.6 | 72.0 | 61.6 | 61.0 | 70.0 | -- | ![72.99](https://img.shields.io/badge/72.99-F59E0B?style=flat-square) |
135
+ | SWE-Bench Verified | 60.0 | 52.0 | -- | 60.7 | 34.0 | 51.56 | ![61.20](https://img.shields.io/badge/61.20-F59E0B?style=flat-square) |
136
 
137
  ### 👁️ Vision Language
138
 
139
  | | GPT-5-Nano-2025-08-07 | Gemini-2.5-Flash-Lite | Qwen3-VL-30B-A3B | Qwen3.5-9B | Gemma 4 12B | **Alpie Seed 9B** |
140
  |:---|:---:|:---:|:---:|:---:|:---:|:---:|
141
  | **STEM and Puzzle** | | | | | | |
142
+ | MMMU-Pro | 57.2 | 59.7 | 63.0 | **70.1** | 69.1 | ![68.70](https://img.shields.io/badge/68.70-7C3AED?style=flat-square) |
143
+ | MathVision | 62.2 | 52.1 | 65.7 | 78.9 | **79.7** | ![76.76](https://img.shields.io/badge/76.76-7C3AED?style=flat-square) |
144
  | **General VQA** | | | | | | |
145
+ | RealWorldQA | 71.8 | 72.2 | 77.4 | 80.3 | -- | ![80.39](https://img.shields.io/badge/80.39-F59E0B?style=flat-square) |
146
+ | MMStar | 68.6 | 69.1 | 75.5 | **79.7** | -- | ![78.87](https://img.shields.io/badge/78.87-7C3AED?style=flat-square) |
147
+ | HallusionBench | 58.4 | 64.5 | 66.0 | 69.3 | -- | ![79.88](https://img.shields.io/badge/79.88-F59E0B?style=flat-square) |
148
  | **Text Recognition and Document Understanding** | | | | | | |
149
+ | OCRBench | 75.3 | 82.5 | 83.9 | 89.2 | -- | ![90.30](https://img.shields.io/badge/90.30-F59E0B?style=flat-square) |
150
 
151
  * SWE-Bench Verified: resolved rate using Mini-SWE-agent, with retries disabled.
152
  * Comparison-model scores were not re-run by 169Pi, and harnesses, prompts, and benchmark versions differ, so cross-model comparisons are indicative rather than strictly like-for-like. Scores vary with dataset versions, prompts, evaluation harnesses, and inference settings.
 
164
  Run locally with Ollama:
165
 
166
  ```shell
167
+ ollama run 169pi/alpie_seed
168
  ```
169
 
170
  Quantized 2-bit, 3-bit, and 4-bit builds are in the [GGUF repository](https://huggingface.co/169Pi/Alpie_Seed_GGUF). Thinking and non-thinking behavior is controlled through the chat template.