Maxwell-V1-Flash / README.md
sunzhongkai588's picture
Fix model card table backgrounds for dark mode
ca536c4 verified
|
Raw History Blame Contribute Delete
172 kB
--- license: apache-2.0 pipeline_tag: text-generation tags: - moe - text-generation --- <div align="center" style="line-height: 0;"> <img src="assets/logo-en.png" alt="Maxwell-V1.0-Flash" width="380"/> </div> <div style="width: 100%; height: 0.5px; background-color: #e5e7eb;"></div> <div style="height: 56px;"></div> <div align="center" style="line-height: 0;"> <span style="display: inline-flex; align-items: center; gap: 6px;"> <a href="https://maxwell-llm.ai/"><img src="https://img.shields.io/badge/%F0%9F%8C%90%20Homepage-Maxwell-4f62d8" alt="Homepage" height="18"></a> <a href="https://maxwell-llm.ai/playground"><img src="https://img.shields.io/badge/%F0%9F%A4%96%20Chat-Playground-18a058" alt="Chat" height="18"></a> </span> </div> <div align="center" style="line-height: 0; padding-top: 3px;"> <span style="display: inline-flex; align-items: center; gap: 6px;"> <a href="https://huggingface.co/maxwell-model/Maxwell-V1-Flash"><img src="https://img.shields.io/badge/%F0%9F%A4%97%20Hugging%20Face-Maxwell-ffc107?color=ffc107&logoColor=white" alt="Hugging Face" height="18"></a> <a href="#6-license"><img src="https://img.shields.io/badge/License-Apache%202.0-blue" alt="License" height="18"></a> </span> </div> <div style="height: 20px;"></div> <div align="center" style="font-size: 15px;"> 📄&nbsp;<a href="https://github.com/maxwell-model/Maxwell-V1-Flash/blob/main/Maxwell_v1_flash_tech_report.pdf" style="color: inherit; text-decoration: underline;">Technical Report</a>&nbsp;&nbsp;|&nbsp;&nbsp; 💬&nbsp;<a href="https://github.com/maxwell-model/Maxwell-V1-Flash" style="color: inherit; text-decoration: underline;">GitHub</a> </div> <br> ## 1. Model Introduction **Maxwell-V1.0-Flash** is a next-generation, high-performance Mixture-of-Experts (MoE) language model developed by the Maxwell Team, with **35B total parameters and 3B active parameters per token** (both excluding embeddings) and support for context lengths up to 256K tokens. It is designed for strong reasoning, agentic, and long-context capabilities at a practical inference cost. <h5 style="margin: 0; font-weight: 700;">Key Features</h5> - **Maxwell Block Sparse Attention (MBSA).** Each KV group keeps fixed coverage of 8 prefix and 8 local 128-token blocks and adds up to 16 remote blocks chosen by a learned indexer, bounding main attention at 4,096 tokens per KV group while the full-history KV cache stays available. The MBSA kernel is 6.7x faster than dense GQA at 256K. - **Manifold-Constrained Hyper-Connections.** Multiple residual streams with learned, input-dependent aggregation and constrained mixing, widening information flow without extra attention or feed-forward evaluations. - **Fine-Grained Mixture of Experts.** A mostly-MoE backbone pairing fine-grained routed experts with always-active shared experts, using a dynamic routing correction bias alongside a lightweight sequence-wise load-balancing loss. **Pre-training.** Approximately **26T tokens** of multilingual data, with the Muon optimizer on most hidden weight matrices. Training starts dense; block-sparse attention enters at 128K mid-training, once the indexer is warmed up against dense full-history attention, then trains jointly with the backbone. **Tokenizer and Long Context.** A script-aware multilingual tokenizer stays compact across languages, especially low-resource Southeast Asian ones such as Thai, Lao, Myanmar, Khmer, and Shan. Long-context training then extends the sequence length in stages from 4K to 256K. **Post-training.** Two-stage all-domain SFT builds general, reasoning, and agentic capabilities, then targets harder reasoning problems and agent trajectories. Independent RL branches cover general tasks, verifiable reasoning (mathematics, science, logic, code), and agentic tasks (search, terminal use, function calling, software engineering). Multi-teacher on-policy distillation then merges them into one model by supervising student-generated responses. The results below summarize Maxwell-V1.0-Flash's performance across reasoning, agentic search, and other capabilities alongside the evaluated comparison models. <div align="center"> <img src="assets/performance-overview.svg" alt="Selected Maxwell-V1.0-Flash benchmark results across reasoning, agentic search, and other capabilities" width="100%"> </div> ## 2. Model Specifications <table align="center" style="margin-left: auto; margin-right: auto; border-collapse: collapse; border: 1px solid #e4eaf1; border-radius: 4px;"> <thead> <tr> <th style="background-color: rgba(128, 128, 128, 0.08); padding: 8px 14px; border-bottom: 1px solid #e4eaf1; text-align: left; vertical-align: top; white-space: nowrap;">Component</th> <th style="background-color: rgba(128, 128, 128, 0.08); padding: 8px 14px; border-bottom: 1px solid #e4eaf1; text-align: left; vertical-align: top;">Configuration</th> <th style="background-color: rgba(128, 128, 128, 0.08); padding: 8px 14px; border-bottom: 1px solid #e4eaf1; text-align: right; vertical-align: top;">Value</th> </tr> </thead> <tbody> <tr> <th rowspan="3" scope="rowgroup" style="padding: 8px 14px; border-right: 1px solid #e4eaf1; text-align: left; vertical-align: top; white-space: nowrap; font-weight: 600; background-color: transparent;">Model</th> <td style="padding: 8px 14px; text-align: left; vertical-align: top;">Architecture</td> <td style="padding: 8px 14px; text-align: right; vertical-align: top; font-variant-numeric: tabular-nums; white-space: nowrap;">Mixture-of-Experts (MoE)</td> </tr> <tr> <td style="padding: 8px 14px; text-align: left; vertical-align: top;">Model type</td> <td style="padding: 8px 14px; text-align: right; vertical-align: top; font-variant-numeric: tabular-nums; white-space: nowrap;">Text-only causal language model</td> </tr> <tr> <td style="padding: 8px 14px; text-align: left; vertical-align: top;">Total / active parameters</td> <td style="padding: 8px 14px; text-align: right; vertical-align: top; font-variant-numeric: tabular-nums; white-space: nowrap;">35B / 3B</td> </tr> <tr> <th rowspan="1" scope="rowgroup" style="padding: 8px 14px; border-right: 1px solid #e4eaf1; border-top: 1px solid #e4eaf1; text-align: left; vertical-align: top; white-space: nowrap; font-weight: 600; background-color: transparent;">Tokenizer</th> <td style="padding: 8px 14px; border-top: 1px solid #e4eaf1; text-align: left; vertical-align: top;">Vocabulary size</td> <td style="padding: 8px 14px; border-top: 1px solid #e4eaf1; text-align: right; vertical-align: top; font-variant-numeric: tabular-nums; white-space: nowrap;">200,704</td> </tr> <tr> <th rowspan="3" scope="rowgroup" style="padding: 8px 14px; border-right: 1px solid #e4eaf1; border-top: 1px solid #e4eaf1; text-align: left; vertical-align: top; white-space: nowrap; font-weight: 600; background-color: transparent;">Backbone</th> <td style="padding: 8px 14px; border-top: 1px solid #e4eaf1; text-align: left; vertical-align: top;">Transformer layers</td> <td style="padding: 8px 14px; border-top: 1px solid #e4eaf1; text-align: right; vertical-align: top; font-variant-numeric: tabular-nums; white-space: nowrap;">45</td> </tr> <tr> <td style="padding: 8px 14px; text-align: left; vertical-align: top;">Hidden size</td> <td style="padding: 8px 14px; text-align: right; vertical-align: top; font-variant-numeric: tabular-nums; white-space: nowrap;">2,048</td> </tr> <tr> <td style="padding: 8px 14px; text-align: left; vertical-align: top;">Configured sequence length</td> <td style="padding: 8px 14px; text-align: right; vertical-align: top; font-variant-numeric: tabular-nums; white-space: nowrap;">262,144</td> </tr> <tr> <th rowspan="8" scope="rowgroup" style="padding: 8px 14px; border-right: 1px solid #e4eaf1; border-top: 1px solid #e4eaf1; text-align: left; vertical-align: top; white-space: nowrap; font-weight: 600; background-color: transparent;">Attention</th> <td style="padding: 8px 14px; border-top: 1px solid #e4eaf1; text-align: left; vertical-align: top;">Query heads / KV heads</td> <td style="padding: 8px 14px; border-top: 1px solid #e4eaf1; text-align: right; vertical-align: top; font-variant-numeric: tabular-nums; white-space: nowrap;">32 / 4</td> </tr> <tr> <td style="padding: 8px 14px; text-align: left; vertical-align: top;">Head dimension</td> <td style="padding: 8px 14px; text-align: right; vertical-align: top; font-variant-numeric: tabular-nums; white-space: nowrap;">128</td> </tr> <tr> <td style="padding: 8px 14px; text-align: left; vertical-align: top;">RoPE base</td> <td style="padding: 8px 14px; text-align: right; vertical-align: top; font-variant-numeric: tabular-nums; white-space: nowrap;">10,000,000</td> </tr> <tr> <td style="padding: 8px 14px; text-align: left; vertical-align: top;">Dense-GQA query-position prefix</td> <td style="padding: 8px 14px; text-align: right; vertical-align: top; font-variant-numeric: tabular-nums; white-space: nowrap;">16K tokens</td> </tr> <tr> <td style="padding: 8px 14px; text-align: left; vertical-align: top;">Attention beyond the dense prefix</td> <td style="padding: 8px 14px; text-align: right; vertical-align: top; font-variant-numeric: tabular-nums; white-space: nowrap;">BSA</td> </tr> <tr> <td style="padding: 8px 14px; text-align: left; vertical-align: top;">Block size</td> <td style="padding: 8px 14px; text-align: right; vertical-align: top; font-variant-numeric: tabular-nums; white-space: nowrap;">128 tokens</td> </tr> <tr> <td style="padding: 8px 14px; text-align: left; vertical-align: top;">Maximum block budget per KV group</td> <td style="padding: 8px 14px; text-align: right; vertical-align: top; font-variant-numeric: tabular-nums; white-space: nowrap;">32</td> </tr> <tr> <td style="padding: 8px 14px; text-align: left; vertical-align: top;">Prefix / local / remote block budgets</td> <td style="padding: 8px 14px; text-align: right; vertical-align: top; font-variant-numeric: tabular-nums; white-space: nowrap;">8 / 8 / 16</td> </tr> <tr> <th rowspan="2" scope="rowgroup" style="padding: 8px 14px; border-right: 1px solid #e4eaf1; border-top: 1px solid #e4eaf1; text-align: left; vertical-align: top; white-space: nowrap; font-weight: 600; background-color: transparent;">Indexer</th> <td style="padding: 8px 14px; border-top: 1px solid #e4eaf1; text-align: left; vertical-align: top;">Query heads / key heads</td> <td style="padding: 8px 14px; border-top: 1px solid #e4eaf1; text-align: right; vertical-align: top; font-variant-numeric: tabular-nums; white-space: nowrap;">4 / 1</td> </tr> <tr> <td style="padding: 8px 14px; text-align: left; vertical-align: top;">Head dimension</td> <td style="padding: 8px 14px; text-align: right; vertical-align: top; font-variant-numeric: tabular-nums; white-space: nowrap;">128</td> </tr> <tr> <th rowspan="2" scope="rowgroup" style="padding: 8px 14px; border-right: 1px solid #e4eaf1; border-top: 1px solid #e4eaf1; text-align: left; vertical-align: top; white-space: nowrap; font-weight: 600; background-color: transparent;">Dense FFN</th> <td style="padding: 8px 14px; border-top: 1px solid #e4eaf1; text-align: left; vertical-align: top;">Layers</td> <td style="padding: 8px 14px; border-top: 1px solid #e4eaf1; text-align: right; vertical-align: top; font-variant-numeric: tabular-nums; white-space: nowrap;">3</td> </tr> <tr> <td style="padding: 8px 14px; text-align: left; vertical-align: top;">Intermediate size</td> <td style="padding: 8px 14px; text-align: right; vertical-align: top; font-variant-numeric: tabular-nums; white-space: nowrap;">6,144</td> </tr> <tr> <th rowspan="4" scope="rowgroup" style="padding: 8px 14px; border-right: 1px solid #e4eaf1; border-top: 1px solid #e4eaf1; text-align: left; vertical-align: top; white-space: nowrap; font-weight: 600; background-color: transparent;">MoE</th> <td style="padding: 8px 14px; border-top: 1px solid #e4eaf1; text-align: left; vertical-align: top;">Layers</td> <td style="padding: 8px 14px; border-top: 1px solid #e4eaf1; text-align: right; vertical-align: top; font-variant-numeric: tabular-nums; white-space: nowrap;">42</td> </tr> <tr> <td style="padding: 8px 14px; text-align: left; vertical-align: top;">Routed experts (total / active per token)</td> <td style="padding: 8px 14px; text-align: right; vertical-align: top; font-variant-numeric: tabular-nums; white-space: nowrap;">256 / 12</td> </tr> <tr> <td style="padding: 8px 14px; text-align: left; vertical-align: top;">Shared experts per layer</td> <td style="padding: 8px 14px; text-align: right; vertical-align: top; font-variant-numeric: tabular-nums; white-space: nowrap;">2</td> </tr> <tr> <td style="padding: 8px 14px; text-align: left; vertical-align: top;">Intermediate size per expert</td> <td style="padding: 8px 14px; text-align: right; vertical-align: top; font-variant-numeric: tabular-nums; white-space: nowrap;">512</td> </tr> <tr> <th rowspan="3" scope="rowgroup" style="padding: 8px 14px; border-right: 1px solid #e4eaf1; border-top: 1px solid #e4eaf1; text-align: left; vertical-align: top; white-space: nowrap; font-weight: 600; background-color: transparent;">mHC</th> <td style="padding: 8px 14px; border-top: 1px solid #e4eaf1; text-align: left; vertical-align: top;">Residual streams</td> <td style="padding: 8px 14px; border-top: 1px solid #e4eaf1; text-align: right; vertical-align: top; font-variant-numeric: tabular-nums; white-space: nowrap;">4</td> </tr> <tr> <td style="padding: 8px 14px; text-align: left; vertical-align: top;">Sinkhorn–Knopp iterations</td> <td style="padding: 8px 14px; text-align: right; vertical-align: top; font-variant-numeric: tabular-nums; white-space: nowrap;">20</td> </tr> <tr> <td style="padding: 8px 14px; text-align: left; vertical-align: top;">Initial scale of input-dependent mappings</td> <td style="padding: 8px 14px; text-align: right; vertical-align: top; font-variant-numeric: tabular-nums; white-space: nowrap;">0.01</td> </tr> <tr> <th rowspan="2" scope="rowgroup" style="padding: 8px 14px; border-right: 1px solid #e4eaf1; border-top: 1px solid #e4eaf1; text-align: left; vertical-align: top; white-space: nowrap; font-weight: 600; background-color: transparent;">MTP</th> <td style="padding: 8px 14px; border-top: 1px solid #e4eaf1; text-align: left; vertical-align: top;">Auxiliary layers</td> <td style="padding: 8px 14px; border-top: 1px solid #e4eaf1; text-align: right; vertical-align: top; font-variant-numeric: tabular-nums; white-space: nowrap;">1</td> </tr> <tr> <td style="padding: 8px 14px; text-align: left; vertical-align: top;">Attention type</td> <td style="padding: 8px 14px; text-align: right; vertical-align: top; font-variant-numeric: tabular-nums; white-space: nowrap;">Dense GQA</td> </tr> </tbody> </table> *The block budget applies in sparse-attention mode. The MTP layer is auxiliary and is not included in the backbone layer count.* ## 3. Evaluation Results **Base model.** Maxwell-V1.0-Flash (Base) is compared with two Qwen base models of comparable scale under a unified internal evaluation framework. All scores are percentages; the best score in each row is shown in bold. GSM8K is evaluated using pass@8. <table style="border-collapse: collapse; width: 100%; border: 1px solid #e4eaf1; border-radius: 4px; table-layout: fixed;"> <thead> <tr> <th style="background-color: rgba(128, 128, 128, 0.08); padding: 10px 12px; border-bottom: 1px solid #e4eaf1; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere; width: 20%; border-top: 0;">Category</th> <th style="background-color: rgba(128, 128, 128, 0.08); padding: 10px 12px; border-bottom: 1px solid #e4eaf1; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere; width: 20%; border-top: 0;">Benchmark</th> <th style="background-color: rgba(128, 128, 128, 0.08); padding: 10px 12px; border-bottom: 1px solid #e4eaf1; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere; width: 20%; border-top: 0;">Qwen3.0-w30a3-base</th> <th style="background-color: rgba(128, 128, 128, 0.08); padding: 10px 12px; border-bottom: 1px solid #e4eaf1; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere; width: 20%; border-top: 0;">Qwen3.5-w35a3-base</th> <th style="background-color: rgba(59, 130, 246, 0.10); padding: 10px 12px; border-bottom: 1px solid #e4eaf1; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere; width: 20%; border-top: 0;">Maxwell-V1.0-Flash (Base)</th> </tr> </thead> <tbody style="border-top: 0; border-bottom: 0;"> <tr> <td style="background-color: transparent; padding: 10px 12px; border-bottom: 1px solid #e4eaf1; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">Math</td> <td style="background-color: transparent; padding: 10px 12px; border-bottom: 1px solid #e4eaf1; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">GSM8K (pass@8)</td> <td style="background-color: transparent; padding: 10px 12px; border-bottom: 1px solid #e4eaf1; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">97.95</td> <td style="background-color: transparent; padding: 10px 12px; border-bottom: 1px solid #e4eaf1; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;"><strong>98.41</strong></td> <td style="background-color: rgba(59, 130, 246, 0.10); padding: 10px 12px; border-bottom: 1px solid #e4eaf1; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">97.19</td> </tr> <tr> <td style="background-color: transparent; padding: 10px 12px; border-bottom: 1px solid #e4eaf1; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;"></td> <td style="background-color: transparent; padding: 10px 12px; border-bottom: 1px solid #e4eaf1; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">MATH-500</td> <td style="background-color: transparent; padding: 10px 12px; border-bottom: 1px solid #e4eaf1; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">59.00</td> <td style="background-color: transparent; padding: 10px 12px; border-bottom: 1px solid #e4eaf1; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">57.50</td> <td style="background-color: rgba(59, 130, 246, 0.10); padding: 10px 12px; border-bottom: 1px solid #e4eaf1; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;"><strong>71.40</strong></td> </tr> <tr> <td style="background-color: transparent; padding: 10px 12px; border-bottom: 1px solid #e4eaf1; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;"></td> <td style="background-color: transparent; padding: 10px 12px; border-bottom: 1px solid #e4eaf1; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">MinervaMath (MATH)</td> <td style="background-color: transparent; padding: 10px 12px; border-bottom: 1px solid #e4eaf1; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">62.14</td> <td style="background-color: transparent; padding: 10px 12px; border-bottom: 1px solid #e4eaf1; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">58.62</td> <td style="background-color: rgba(59, 130, 246, 0.10); padding: 10px 12px; border-bottom: 1px solid #e4eaf1; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;"><strong>74.56</strong></td> </tr> <tr> <td style="background-color: transparent; padding: 10px 12px; border-bottom: 1px solid #e4eaf1; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;"></td> <td style="background-color: transparent; padding: 10px 12px; border-bottom: 1px solid #e4eaf1; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">MathQA</td> <td style="background-color: transparent; padding: 10px 12px; border-bottom: 1px solid #e4eaf1; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">60.40</td> <td style="background-color: transparent; padding: 10px 12px; border-bottom: 1px solid #e4eaf1; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">63.69</td> <td style="background-color: rgba(59, 130, 246, 0.10); padding: 10px 12px; border-bottom: 1px solid #e4eaf1; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;"><strong>66.10</strong></td> </tr> <tr> <td style="background-color: transparent; padding: 10px 12px; border-bottom: 1px solid #e4eaf1; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">Code</td> <td style="background-color: transparent; padding: 10px 12px; border-bottom: 1px solid #e4eaf1; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">HumanEval</td> <td style="background-color: transparent; padding: 10px 12px; border-bottom: 1px solid #e4eaf1; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">70.73</td> <td style="background-color: transparent; padding: 10px 12px; border-bottom: 1px solid #e4eaf1; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">67.68</td> <td style="background-color: rgba(59, 130, 246, 0.10); padding: 10px 12px; border-bottom: 1px solid #e4eaf1; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;"><strong>78.05</strong></td> </tr> <tr> <td style="background-color: transparent; padding: 10px 12px; border-bottom: 1px solid #e4eaf1; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;"></td> <td style="background-color: transparent; padding: 10px 12px; border-bottom: 1px solid #e4eaf1; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">MBPP</td> <td style="background-color: transparent; padding: 10px 12px; border-bottom: 1px solid #e4eaf1; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;"><strong>73.93</strong></td> <td style="background-color: transparent; padding: 10px 12px; border-bottom: 1px solid #e4eaf1; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">61.09</td> <td style="background-color: rgba(59, 130, 246, 0.10); padding: 10px 12px; border-bottom: 1px solid #e4eaf1; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">65.37</td> </tr> <tr> <td style="background-color: transparent; padding: 10px 12px; border-bottom: 1px solid #e4eaf1; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;"></td> <td style="background-color: transparent; padding: 10px 12px; border-bottom: 1px solid #e4eaf1; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">HumanEval+</td> <td style="background-color: transparent; padding: 10px 12px; border-bottom: 1px solid #e4eaf1; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">63.41</td> <td style="background-color: transparent; padding: 10px 12px; border-bottom: 1px solid #e4eaf1; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">56.10</td> <td style="background-color: rgba(59, 130, 246, 0.10); padding: 10px 12px; border-bottom: 1px solid #e4eaf1; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;"><strong>71.95</strong></td> </tr> <tr> <td style="background-color: transparent; padding: 10px 12px; border-bottom: 1px solid #e4eaf1; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;"></td> <td style="background-color: transparent; padding: 10px 12px; border-bottom: 1px solid #e4eaf1; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">MBPP+</td> <td style="background-color: transparent; padding: 10px 12px; border-bottom: 1px solid #e4eaf1; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;"><strong>75.40</strong></td> <td style="background-color: transparent; padding: 10px 12px; border-bottom: 1px solid #e4eaf1; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">70.90</td> <td style="background-color: rgba(59, 130, 246, 0.10); padding: 10px 12px; border-bottom: 1px solid #e4eaf1; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">75.13</td> </tr> <tr> <td style="background-color: transparent; padding: 10px 12px; border-bottom: 1px solid #e4eaf1; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">STEM</td> <td style="background-color: transparent; padding: 10px 12px; border-bottom: 1px solid #e4eaf1; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">MMLU</td> <td style="background-color: transparent; padding: 10px 12px; border-bottom: 1px solid #e4eaf1; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">80.98</td> <td style="background-color: transparent; padding: 10px 12px; border-bottom: 1px solid #e4eaf1; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;"><strong>82.85</strong></td> <td style="background-color: rgba(59, 130, 246, 0.10); padding: 10px 12px; border-bottom: 1px solid #e4eaf1; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">79.85</td> </tr> <tr> <td style="background-color: transparent; padding: 10px 12px; border-bottom: 1px solid #e4eaf1; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;"></td> <td style="background-color: transparent; padding: 10px 12px; border-bottom: 1px solid #e4eaf1; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">MMLU-Redux</td> <td style="background-color: transparent; padding: 10px 12px; border-bottom: 1px solid #e4eaf1; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;"><strong>78.17</strong></td> <td style="background-color: transparent; padding: 10px 12px; border-bottom: 1px solid #e4eaf1; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;"><strong>78.17</strong></td> <td style="background-color: rgba(59, 130, 246, 0.10); padding: 10px 12px; border-bottom: 1px solid #e4eaf1; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">74.90</td> </tr> <tr> <td style="background-color: transparent; padding: 10px 12px; border-bottom: 1px solid #e4eaf1; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;"></td> <td style="background-color: transparent; padding: 10px 12px; border-bottom: 1px solid #e4eaf1; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">MMLU-Pro</td> <td style="background-color: transparent; padding: 10px 12px; border-bottom: 1px solid #e4eaf1; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">61.64</td> <td style="background-color: transparent; padding: 10px 12px; border-bottom: 1px solid #e4eaf1; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">64.68</td> <td style="background-color: rgba(59, 130, 246, 0.10); padding: 10px 12px; border-bottom: 1px solid #e4eaf1; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;"><strong>65.25</strong></td> </tr> <tr> <td style="background-color: transparent; padding: 10px 12px; border-bottom: 1px solid #e4eaf1; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;"></td> <td style="background-color: transparent; padding: 10px 12px; border-bottom: 1px solid #e4eaf1; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">ARC-Easy</td> <td style="background-color: transparent; padding: 10px 12px; border-bottom: 1px solid #e4eaf1; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">80.01</td> <td style="background-color: transparent; padding: 10px 12px; border-bottom: 1px solid #e4eaf1; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">82.15</td> <td style="background-color: rgba(59, 130, 246, 0.10); padding: 10px 12px; border-bottom: 1px solid #e4eaf1; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;"><strong>86.78</strong></td> </tr> <tr> <td style="background-color: transparent; padding: 10px 12px; border-bottom: 1px solid #e4eaf1; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;"></td> <td style="background-color: transparent; padding: 10px 12px; border-bottom: 1px solid #e4eaf1; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">ARC-Challenge</td> <td style="background-color: transparent; padding: 10px 12px; border-bottom: 1px solid #e4eaf1; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">57.59</td> <td style="background-color: transparent; padding: 10px 12px; border-bottom: 1px solid #e4eaf1; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">62.12</td> <td style="background-color: rgba(59, 130, 246, 0.10); padding: 10px 12px; border-bottom: 1px solid #e4eaf1; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;"><strong>65.78</strong></td> </tr> <tr> <td style="background-color: transparent; padding: 10px 12px; border-bottom: 1px solid #e4eaf1; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;"></td> <td style="background-color: transparent; padding: 10px 12px; border-bottom: 1px solid #e4eaf1; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">BBH</td> <td style="background-color: transparent; padding: 10px 12px; border-bottom: 1px solid #e4eaf1; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">28.08</td> <td style="background-color: transparent; padding: 10px 12px; border-bottom: 1px solid #e4eaf1; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">52.17</td> <td style="background-color: rgba(59, 130, 246, 0.10); padding: 10px 12px; border-bottom: 1px solid #e4eaf1; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;"><strong>57.01</strong></td> </tr> <tr> <td style="background-color: transparent; padding: 10px 12px; border-bottom: 1px solid #e4eaf1; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;"></td> <td style="background-color: transparent; padding: 10px 12px; border-bottom: 1px solid #e4eaf1; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">GPQA-main</td> <td style="background-color: transparent; padding: 10px 12px; border-bottom: 1px solid #e4eaf1; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">40.62</td> <td style="background-color: transparent; padding: 10px 12px; border-bottom: 1px solid #e4eaf1; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;"><strong>54.69</strong></td> <td style="background-color: rgba(59, 130, 246, 0.10); padding: 10px 12px; border-bottom: 1px solid #e4eaf1; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">52.68</td> </tr> <tr> <td style="background-color: transparent; padding: 10px 12px; border-bottom: 1px solid #e4eaf1; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;"></td> <td style="background-color: transparent; padding: 10px 12px; border-bottom: 1px solid #e4eaf1; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">GPQA-diamond</td> <td style="background-color: transparent; padding: 10px 12px; border-bottom: 1px solid #e4eaf1; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">42.42</td> <td style="background-color: transparent; padding: 10px 12px; border-bottom: 1px solid #e4eaf1; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;"><strong>54.04</strong></td> <td style="background-color: rgba(59, 130, 246, 0.10); padding: 10px 12px; border-bottom: 1px solid #e4eaf1; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">51.52</td> </tr> <tr> <td style="background-color: transparent; padding: 10px 12px; border-bottom: 1px solid #e4eaf1; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">Multilingual</td> <td style="background-color: transparent; padding: 10px 12px; border-bottom: 1px solid #e4eaf1; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">MMLU (SEA)</td> <td style="background-color: transparent; padding: 10px 12px; border-bottom: 1px solid #e4eaf1; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">73.89</td> <td style="background-color: transparent; padding: 10px 12px; border-bottom: 1px solid #e4eaf1; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;"><strong>77.37</strong></td> <td style="background-color: rgba(59, 130, 246, 0.10); padding: 10px 12px; border-bottom: 1px solid #e4eaf1; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">75.06</td> </tr> <tr> <td style="background-color: transparent; padding: 10px 12px; border-bottom: 1px solid #e4eaf1; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;"></td> <td style="background-color: transparent; padding: 10px 12px; border-bottom: 1px solid #e4eaf1; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">XCOPA (SEA)</td> <td style="background-color: transparent; padding: 10px 12px; border-bottom: 1px solid #e4eaf1; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">79.63</td> <td style="background-color: transparent; padding: 10px 12px; border-bottom: 1px solid #e4eaf1; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">78.93</td> <td style="background-color: rgba(59, 130, 246, 0.10); padding: 10px 12px; border-bottom: 1px solid #e4eaf1; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;"><strong>79.80</strong></td> </tr> <tr> <td style="background-color: transparent; padding: 10px 12px; border-bottom: 1px solid #e4eaf1; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;"></td> <td style="background-color: transparent; padding: 10px 12px; border-bottom: 1px solid #e4eaf1; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">OpenBookQA (SEA)</td> <td style="background-color: transparent; padding: 10px 12px; border-bottom: 1px solid #e4eaf1; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">43.20</td> <td style="background-color: transparent; padding: 10px 12px; border-bottom: 1px solid #e4eaf1; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">41.30</td> <td style="background-color: rgba(59, 130, 246, 0.10); padding: 10px 12px; border-bottom: 1px solid #e4eaf1; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;"><strong>43.30</strong></td> </tr> <tr> <td style="background-color: transparent; padding: 10px 12px; border-bottom: 1px solid #e4eaf1; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;"></td> <td style="background-color: transparent; padding: 10px 12px; border-bottom: 1px solid #e4eaf1; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">ARC-Challenge (SEA)</td> <td style="background-color: transparent; padding: 10px 12px; border-bottom: 1px solid #e4eaf1; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">51.83</td> <td style="background-color: transparent; padding: 10px 12px; border-bottom: 1px solid #e4eaf1; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">54.67</td> <td style="background-color: rgba(59, 130, 246, 0.10); padding: 10px 12px; border-bottom: 1px solid #e4eaf1; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;"><strong>58.13</strong></td> </tr> <tr> <td style="background-color: transparent; padding: 10px 12px; border-bottom: 1px solid #e4eaf1; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;"></td> <td style="background-color: transparent; padding: 10px 12px; border-bottom: 1px solid #e4eaf1; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">MGSM</td> <td style="background-color: transparent; padding: 10px 12px; border-bottom: 1px solid #e4eaf1; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;"><strong>85.20</strong></td> <td style="background-color: transparent; padding: 10px 12px; border-bottom: 1px solid #e4eaf1; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">84.13</td> <td style="background-color: rgba(59, 130, 246, 0.10); padding: 10px 12px; border-bottom: 1px solid #e4eaf1; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">83.87</td> </tr> <tr> <td style="background-color: transparent; padding: 10px 12px; border-bottom: 1px solid #e4eaf1; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">Knowledge</td> <td style="background-color: transparent; padding: 10px 12px; border-bottom: 1px solid #e4eaf1; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">HellaSwag</td> <td style="background-color: transparent; padding: 10px 12px; border-bottom: 1px solid #e4eaf1; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">81.38</td> <td style="background-color: transparent; padding: 10px 12px; border-bottom: 1px solid #e4eaf1; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;"><strong>83.64</strong></td> <td style="background-color: rgba(59, 130, 246, 0.10); padding: 10px 12px; border-bottom: 1px solid #e4eaf1; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">79.84</td> </tr> <tr> <td style="background-color: transparent; padding: 10px 12px; border-bottom: 1px solid #e4eaf1; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;"></td> <td style="background-color: transparent; padding: 10px 12px; border-bottom: 1px solid #e4eaf1; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">PIQA</td> <td style="background-color: transparent; padding: 10px 12px; border-bottom: 1px solid #e4eaf1; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">81.12</td> <td style="background-color: transparent; padding: 10px 12px; border-bottom: 1px solid #e4eaf1; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;"><strong>82.54</strong></td> <td style="background-color: rgba(59, 130, 246, 0.10); padding: 10px 12px; border-bottom: 1px solid #e4eaf1; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;"><strong>82.54</strong></td> </tr> <tr> <td style="background-color: transparent; padding: 10px 12px; border-bottom: 1px solid #e4eaf1; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;"></td> <td style="background-color: transparent; padding: 10px 12px; border-bottom: 1px solid #e4eaf1; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">WinoGrande</td> <td style="background-color: transparent; padding: 10px 12px; border-bottom: 1px solid #e4eaf1; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;"><strong>77.74</strong></td> <td style="background-color: transparent; padding: 10px 12px; border-bottom: 1px solid #e4eaf1; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">74.11</td> <td style="background-color: rgba(59, 130, 246, 0.10); padding: 10px 12px; border-bottom: 1px solid #e4eaf1; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">72.30</td> </tr> <tr> <td style="background-color: transparent; padding: 10px 12px; border-bottom: 1px solid #e4eaf1; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;"></td> <td style="background-color: transparent; padding: 10px 12px; border-bottom: 1px solid #e4eaf1; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">TriviaQA</td> <td style="background-color: transparent; padding: 10px 12px; border-bottom: 1px solid #e4eaf1; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">66.66</td> <td style="background-color: transparent; padding: 10px 12px; border-bottom: 1px solid #e4eaf1; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">71.01</td> <td style="background-color: rgba(59, 130, 246, 0.10); padding: 10px 12px; border-bottom: 1px solid #e4eaf1; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;"><strong>75.02</strong></td> </tr> <tr> <td style="background-color: transparent; padding: 10px 12px; border-bottom: 1px solid #e4eaf1; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;"></td> <td style="background-color: transparent; padding: 10px 12px; border-bottom: 1px solid #e4eaf1; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">RACE</td> <td style="background-color: transparent; padding: 10px 12px; border-bottom: 1px solid #e4eaf1; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;"><strong>86.89</strong></td> <td style="background-color: transparent; padding: 10px 12px; border-bottom: 1px solid #e4eaf1; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">81.15</td> <td style="background-color: rgba(59, 130, 246, 0.10); padding: 10px 12px; border-bottom: 1px solid #e4eaf1; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">84.88</td> </tr> <tr> <td style="background-color: transparent; padding: 10px 12px; border-bottom: 1px solid #e4eaf1; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;"></td> <td style="background-color: transparent; padding: 10px 12px; border-bottom: 1px solid #e4eaf1; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">OpenBookQA</td> <td style="background-color: transparent; padding: 10px 12px; border-bottom: 1px solid #e4eaf1; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;"><strong>90.00</strong></td> <td style="background-color: transparent; padding: 10px 12px; border-bottom: 1px solid #e4eaf1; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">88.60</td> <td style="background-color: rgba(59, 130, 246, 0.10); padding: 10px 12px; border-bottom: 1px solid #e4eaf1; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">89.60</td> </tr> <tr> <td style="background-color: transparent; padding: 10px 12px; border-bottom: 1px solid #e4eaf1; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">Chinese</td> <td style="background-color: transparent; padding: 10px 12px; border-bottom: 1px solid #e4eaf1; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">CMMLU</td> <td style="background-color: transparent; padding: 10px 12px; border-bottom: 1px solid #e4eaf1; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;"><strong>87.41</strong></td> <td style="background-color: transparent; padding: 10px 12px; border-bottom: 1px solid #e4eaf1; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">86.17</td> <td style="background-color: rgba(59, 130, 246, 0.10); padding: 10px 12px; border-bottom: 1px solid #e4eaf1; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">79.41</td> </tr> <tr> <td style="background-color: transparent; padding: 10px 12px; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;"></td> <td style="background-color: transparent; padding: 10px 12px; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">C-Eval</td> <td style="background-color: transparent; padding: 10px 12px; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;"><strong>88.78</strong></td> <td style="background-color: transparent; padding: 10px 12px; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">86.33</td> <td style="background-color: rgba(59, 130, 246, 0.10); padding: 10px 12px; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">77.41</td> </tr> </tbody> </table> **Base model, long context.** Scores are on a 0–100 scale. All results use block-sparse attention in every layer. <table style="border-collapse: collapse; width: 100%; border: 1px solid #e4eaf1; border-radius: 4px; table-layout: fixed;"> <thead> <tr> <th style="background-color: rgba(128, 128, 128, 0.08); padding: 10px 12px; border-bottom: 1px solid #e4eaf1; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere; width: 25%; border-top: 0;">Benchmark</th> <th style="background-color: rgba(128, 128, 128, 0.08); padding: 10px 12px; border-bottom: 1px solid #e4eaf1; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere; width: 25%; border-top: 0;">Context length</th> <th style="background-color: rgba(128, 128, 128, 0.08); padding: 10px 12px; border-bottom: 1px solid #e4eaf1; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere; width: 25%; border-top: 0;">Qwen3.5-w35a3-base</th> <th style="background-color: rgba(59, 130, 246, 0.10); padding: 10px 12px; border-bottom: 1px solid #e4eaf1; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere; width: 25%; border-top: 0;">Maxwell-V1.0-Flash (Base)</th> </tr> </thead> <tbody style="border-top: 0; border-bottom: 0;"> <tr> <td style="background-color: transparent; padding: 10px 12px; border-bottom: 1px solid #e4eaf1; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">RULER</td> <td style="background-color: transparent; padding: 10px 12px; border-bottom: 1px solid #e4eaf1; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">8K</td> <td style="background-color: transparent; padding: 10px 12px; border-bottom: 1px solid #e4eaf1; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">91.98</td> <td style="background-color: rgba(59, 130, 246, 0.10); padding: 10px 12px; border-bottom: 1px solid #e4eaf1; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;"><strong>94.80</strong></td> </tr> <tr> <td style="background-color: transparent; padding: 10px 12px; border-bottom: 1px solid #e4eaf1; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;"></td> <td style="background-color: transparent; padding: 10px 12px; border-bottom: 1px solid #e4eaf1; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">16K</td> <td style="background-color: transparent; padding: 10px 12px; border-bottom: 1px solid #e4eaf1; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">91.44</td> <td style="background-color: rgba(59, 130, 246, 0.10); padding: 10px 12px; border-bottom: 1px solid #e4eaf1; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;"><strong>94.72</strong></td> </tr> <tr> <td style="background-color: transparent; padding: 10px 12px; border-bottom: 1px solid #e4eaf1; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;"></td> <td style="background-color: transparent; padding: 10px 12px; border-bottom: 1px solid #e4eaf1; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">32K</td> <td style="background-color: transparent; padding: 10px 12px; border-bottom: 1px solid #e4eaf1; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">90.91</td> <td style="background-color: rgba(59, 130, 246, 0.10); padding: 10px 12px; border-bottom: 1px solid #e4eaf1; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;"><strong>94.63</strong></td> </tr> <tr> <td style="background-color: transparent; padding: 10px 12px; border-bottom: 1px solid #e4eaf1; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;"></td> <td style="background-color: transparent; padding: 10px 12px; border-bottom: 1px solid #e4eaf1; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">64K</td> <td style="background-color: transparent; padding: 10px 12px; border-bottom: 1px solid #e4eaf1; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">89.65</td> <td style="background-color: rgba(59, 130, 246, 0.10); padding: 10px 12px; border-bottom: 1px solid #e4eaf1; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;"><strong>93.77</strong></td> </tr> <tr> <td style="background-color: transparent; padding: 10px 12px; border-bottom: 1px solid #e4eaf1; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;"></td> <td style="background-color: transparent; padding: 10px 12px; border-bottom: 1px solid #e4eaf1; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">128K</td> <td style="background-color: transparent; padding: 10px 12px; border-bottom: 1px solid #e4eaf1; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">88.39</td> <td style="background-color: rgba(59, 130, 246, 0.10); padding: 10px 12px; border-bottom: 1px solid #e4eaf1; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;"><strong>94.57</strong></td> </tr> <tr> <td style="background-color: transparent; padding: 10px 12px; border-bottom: 1px solid #e4eaf1; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;"></td> <td style="background-color: transparent; padding: 10px 12px; border-bottom: 1px solid #e4eaf1; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">256K</td> <td style="background-color: transparent; padding: 10px 12px; border-bottom: 1px solid #e4eaf1; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">85.75</td> <td style="background-color: rgba(59, 130, 246, 0.10); padding: 10px 12px; border-bottom: 1px solid #e4eaf1; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;"><strong>93.35</strong></td> </tr> <tr> <td style="background-color: transparent; padding: 10px 12px; border-bottom: 1px solid #e4eaf1; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">HELMET</td> <td style="background-color: transparent; padding: 10px 12px; border-bottom: 1px solid #e4eaf1; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">8K</td> <td style="background-color: transparent; padding: 10px 12px; border-bottom: 1px solid #e4eaf1; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">56.62</td> <td style="background-color: rgba(59, 130, 246, 0.10); padding: 10px 12px; border-bottom: 1px solid #e4eaf1; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;"><strong>62.61</strong></td> </tr> <tr> <td style="background-color: transparent; padding: 10px 12px; border-bottom: 1px solid #e4eaf1; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;"></td> <td style="background-color: transparent; padding: 10px 12px; border-bottom: 1px solid #e4eaf1; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">16K</td> <td style="background-color: transparent; padding: 10px 12px; border-bottom: 1px solid #e4eaf1; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">54.93</td> <td style="background-color: rgba(59, 130, 246, 0.10); padding: 10px 12px; border-bottom: 1px solid #e4eaf1; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;"><strong>61.43</strong></td> </tr> <tr> <td style="background-color: transparent; padding: 10px 12px; border-bottom: 1px solid #e4eaf1; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;"></td> <td style="background-color: transparent; padding: 10px 12px; border-bottom: 1px solid #e4eaf1; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">32K</td> <td style="background-color: transparent; padding: 10px 12px; border-bottom: 1px solid #e4eaf1; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">56.87</td> <td style="background-color: rgba(59, 130, 246, 0.10); padding: 10px 12px; border-bottom: 1px solid #e4eaf1; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;"><strong>60.28</strong></td> </tr> <tr> <td style="background-color: transparent; padding: 10px 12px; border-bottom: 1px solid #e4eaf1; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;"></td> <td style="background-color: transparent; padding: 10px 12px; border-bottom: 1px solid #e4eaf1; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">64K</td> <td style="background-color: transparent; padding: 10px 12px; border-bottom: 1px solid #e4eaf1; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">54.61</td> <td style="background-color: rgba(59, 130, 246, 0.10); padding: 10px 12px; border-bottom: 1px solid #e4eaf1; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;"><strong>58.74</strong></td> </tr> <tr> <td style="background-color: transparent; padding: 10px 12px; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;"></td> <td style="background-color: transparent; padding: 10px 12px; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">128K</td> <td style="background-color: transparent; padding: 10px 12px; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">52.35</td> <td style="background-color: rgba(59, 130, 246, 0.10); padding: 10px 12px; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;"><strong>56.55</strong></td> </tr> </tbody> </table> **Post-trained model.** Maxwell-V1.0-Flash is compared with four models of comparable scale: Qwen3.6-35B-A3B, which has the same nominal 35B total / 3B active parameter scale, together with Gemma-4-26B-A4B-it, Nemotron-3.5-Lightning, and Muse-Glimmer-30B. All scores are percentages; the highest score for each metric is shown in bold. Multilingual results cover only market languages. <table style="border-collapse: collapse; width: 100%; border: 1px solid #e4eaf1; border-radius: 4px; table-layout: fixed;"> <thead> <tr> <th style="background-color: rgba(128, 128, 128, 0.08); padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: middle; text-align: left; white-space: normal; overflow-wrap: anywhere; width: 28%; border-top: 0;">Benchmark (Metric)</th> <th style="background-color: rgba(128, 128, 128, 0.08); padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: middle; text-align: left; white-space: normal; overflow-wrap: anywhere; width: 14.4%; border-top: 0;">Qwen3.6<br/>35B-A3B</th> <th style="background-color: rgba(128, 128, 128, 0.08); padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: middle; text-align: left; white-space: normal; overflow-wrap: anywhere; width: 14.4%; border-top: 0;">Gemma-4<br/>26B-A4B-it</th> <th style="background-color: rgba(128, 128, 128, 0.08); padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: middle; text-align: left; white-space: normal; overflow-wrap: anywhere; width: 14.4%; border-top: 0;">Nemotron-3.5<br/>Lightning</th> <th style="background-color: rgba(128, 128, 128, 0.08); padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: middle; text-align: left; white-space: normal; overflow-wrap: anywhere; width: 14.4%; border-top: 0;">Muse<br/>Glimmer-30B</th> <th style="background-color: rgba(59, 130, 246, 0.10); padding: 8px 9px; border-bottom: 1px solid #e4eaf1; vertical-align: middle; text-align: left; white-space: normal; overflow-wrap: anywhere; width: 14.4%; border-top: 0;">Maxwell<br/>V1.0-Flash</th> </tr> </thead> <tbody style="border-top: 0; border-bottom: 0;"> <tr><th colspan="6" style="background-color: rgba(128, 128, 128, 0.14); padding: 8px 9px; border-bottom: 1px solid #e4eaf1; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere; border-top: 0;">General</th></tr> <tr> <th scope="rowgroup" style="background-color: transparent; padding: 10px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere; border-top: 0;">Instruction following</th> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;"></td> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;"></td> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;"></td> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;"></td> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;"></td> </tr> <tr> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">AdvancedIF <span>(Score)</span></td> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">72.50</td> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;"><strong>76.10</strong></td> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">65.60</td> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">73.80</td> <td style="background-color: rgba(59, 130, 246, 0.10); padding: 8px 9px; border-bottom: 1px solid #e4eaf1; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">71.30</td> </tr> <tr> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">IFBench <span>(Score)</span></td> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">59.60</td> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">79.10</td> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">70.80</td> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;"><strong>83.20</strong></td> <td style="background-color: rgba(59, 130, 246, 0.10); padding: 8px 9px; border-bottom: 1px solid #e4eaf1; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">64.80</td> </tr> <tr> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">IFEval <span>(Score)</span></td> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">93.80</td> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">94.00</td> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">91.50</td> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;"><strong>96.80</strong></td> <td style="background-color: rgba(59, 130, 246, 0.10); padding: 8px 9px; border-bottom: 1px solid #e4eaf1; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">93.00</td> </tr> <tr> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;"><em>Instruction-following average</em> <span>(Average)</span></td> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">75.30</td> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">83.07</td> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">75.97</td> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;"><strong>84.60</strong></td> <td style="background-color: rgba(59, 130, 246, 0.10); padding: 8px 9px; border-bottom: 1px solid #e4eaf1; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">76.37</td> </tr> <tr> <th scope="rowgroup" style="background-color: transparent; padding: 10px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere; border-top: 0;">Multi-turn</th> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;"></td> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;"></td> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;"></td> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;"></td> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;"></td> </tr> <tr> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">MultiChallenge <span>(pass@1)</span></td> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">62.59</td> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">65.76</td> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">61.07</td> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">70.45</td> <td style="background-color: rgba(59, 130, 246, 0.10); padding: 8px 9px; border-bottom: 1px solid #e4eaf1; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;"><strong>72.20</strong></td> </tr> <tr> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">MT-Bench-101 <span>(Score)</span></td> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">94.20</td> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">93.90</td> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;"><strong>94.50</strong></td> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">88.20</td> <td style="background-color: rgba(59, 130, 246, 0.10); padding: 8px 9px; border-bottom: 1px solid #e4eaf1; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">93.30</td> </tr> <tr> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">MT-RAG <span>(Average)</span></td> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">65.03</td> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">70.54</td> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">67.00</td> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;"><strong>72.70</strong></td> <td style="background-color: rgba(59, 130, 246, 0.10); padding: 8px 9px; border-bottom: 1px solid #e4eaf1; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">65.74</td> </tr> <tr> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">LongMemEval <span>(Score)</span></td> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">79.52</td> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">85.89</td> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">83.44</td> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;"><strong>94.48</strong></td> <td style="background-color: rgba(59, 130, 246, 0.10); padding: 8px 9px; border-bottom: 1px solid #e4eaf1; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">86.18</td> </tr> <tr> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;"><em>Multi-turn average</em> <span>(Average)</span></td> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">75.34</td> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">79.02</td> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">76.50</td> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;"><strong>81.46</strong></td> <td style="background-color: rgba(59, 130, 246, 0.10); padding: 8px 9px; border-bottom: 1px solid #e4eaf1; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">79.36</td> </tr> <tr> <th scope="rowgroup" style="background-color: transparent; padding: 10px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere; border-top: 0;">Long context</th> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;"></td> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;"></td> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;"></td> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;"></td> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;"></td> </tr> <tr> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">AA-LCR <span>(mean@5)</span></td> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">71.70</td> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">65.70</td> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">60.30</td> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;"><strong>79.60</strong></td> <td style="background-color: rgba(59, 130, 246, 0.10); padding: 8px 9px; border-bottom: 1px solid #e4eaf1; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">75.60</td> </tr> <tr> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">LongBench V2 <span>(Score)</span></td> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">58.80</td> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">53.30</td> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">46.30</td> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">38.60</td> <td style="background-color: rgba(59, 130, 246, 0.10); padding: 8px 9px; border-bottom: 1px solid #e4eaf1; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;"><strong>59.40</strong></td> </tr> <tr> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">RULER <span>(Score)</span></td> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;"><strong>97.08</strong></td> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">94.74</td> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">96.75</td> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">94.84</td> <td style="background-color: rgba(59, 130, 246, 0.10); padding: 8px 9px; border-bottom: 1px solid #e4eaf1; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">95.81</td> </tr> <tr> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;"><em>Long-context average</em> <span>(Average)</span></td> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">75.86</td> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">71.25</td> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">67.78</td> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">71.01</td> <td style="background-color: rgba(59, 130, 246, 0.10); padding: 8px 9px; border-bottom: 1px solid #e4eaf1; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;"><strong>76.94</strong></td> </tr> <tr> <th scope="rowgroup" style="background-color: transparent; padding: 10px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere; border-top: 0;">Multilingual</th> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;"></td> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;"></td> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;"></td> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;"></td> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;"></td> </tr> <tr> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">MLogiQA <span>(Score)</span></td> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">77.8</td> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">77.8</td> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">71.3</td> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">80.0</td> <td style="background-color: rgba(59, 130, 246, 0.10); padding: 8px 9px; border-bottom: 1px solid #e4eaf1; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;"><strong>81.5</strong></td> </tr> <tr> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">CEval <span>(Score)</span></td> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">88.3</td> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">83.7</td> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">76.2</td> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">85.4</td> <td style="background-color: rgba(59, 130, 246, 0.10); padding: 8px 9px; border-bottom: 1px solid #e4eaf1; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;"><strong>91.0</strong></td> </tr> <tr> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">MMMLU <span>(Score)</span></td> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">89.1</td> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">90.9</td> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">87.8</td> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">90.7</td> <td style="background-color: rgba(59, 130, 246, 0.10); padding: 8px 9px; border-bottom: 1px solid #e4eaf1; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;"><strong>92.0</strong></td> </tr> <tr> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">INCLUDE <span>(Score)</span></td> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">84.7</td> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;"><strong>85.8</strong></td> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">71.5</td> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">79.4</td> <td style="background-color: rgba(59, 130, 246, 0.10); padding: 8px 9px; border-bottom: 1px solid #e4eaf1; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">84.7</td> </tr> <tr> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">Belebele <span>(Score)</span></td> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">93.8</td> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;"><strong>94.8</strong></td> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">91.8</td> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">94.3</td> <td style="background-color: rgba(59, 130, 246, 0.10); padding: 8px 9px; border-bottom: 1px solid #e4eaf1; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">92.8</td> </tr> <tr> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">Global-PIQA <span>(Score)</span></td> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">91.7</td> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;"><strong>93.9</strong></td> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">90.9</td> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">92.4</td> <td style="background-color: rgba(59, 130, 246, 0.10); padding: 8px 9px; border-bottom: 1px solid #e4eaf1; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">92.3</td> </tr> <tr> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">MGSM <span>(Score)</span></td> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">90.3</td> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">91.1</td> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">91.5</td> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;"><strong>93.2</strong></td> <td style="background-color: rgba(59, 130, 246, 0.10); padding: 8px 9px; border-bottom: 1px solid #e4eaf1; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">93.0</td> </tr> <tr> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;"><em>Multilingual average</em> <span>(Average)</span></td> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">88.0</td> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">88.3</td> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">83.0</td> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">87.9</td> <td style="background-color: rgba(59, 130, 246, 0.10); padding: 8px 9px; border-bottom: 1px solid #e4eaf1; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;"><strong>89.6</strong></td> </tr> <tr> <th scope="rowgroup" style="background-color: transparent; padding: 10px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere; border-top: 0;">Role playing</th> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;"></td> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;"></td> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;"></td> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;"></td> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;"></td> </tr> <tr> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">CoSER <span>(pass@1)</span></td> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">47.79</td> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">56.19</td> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">43.78</td> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;"><strong>60.32</strong></td> <td style="background-color: rgba(59, 130, 246, 0.10); padding: 8px 9px; border-bottom: 1px solid #e4eaf1; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">53.54</td> </tr> <tr> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">RoleBench <span>(pass@1)</span></td> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">89.21</td> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">84.05</td> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">81.10</td> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">73.40</td> <td style="background-color: rgba(59, 130, 246, 0.10); padding: 8px 9px; border-bottom: 1px solid #e4eaf1; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;"><strong>89.92</strong></td> </tr> <tr> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;"><em>Role-playing average</em> <span>(Average)</span></td> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">68.50</td> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">70.12</td> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">62.44</td> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">66.86</td> <td style="background-color: rgba(59, 130, 246, 0.10); padding: 8px 9px; border-bottom: 1px solid #e4eaf1; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;"><strong>71.73</strong></td> </tr> <tr><th colspan="6" style="background-color: rgba(128, 128, 128, 0.14); padding: 8px 9px; border-bottom: 1px solid #e4eaf1; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere; border-top: 0;">Reasoning</th></tr> <tr> <th scope="rowgroup" style="background-color: transparent; padding: 10px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere; border-top: 0;">Mathematics</th> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;"></td> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;"></td> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;"></td> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;"></td> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;"></td> </tr> <tr> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">AIME 2025 <span>(pass@8)</span></td> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">93.33</td> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">93.33</td> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">96.67</td> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;"><strong>100.00</strong></td> <td style="background-color: rgba(59, 130, 246, 0.10); padding: 8px 9px; border-bottom: 1px solid #e4eaf1; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;"><strong>100.00</strong></td> </tr> <tr> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">AIME 2025 <span>(mean@8)</span></td> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">90.00</td> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">85.83</td> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">87.92</td> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">92.08</td> <td style="background-color: rgba(59, 130, 246, 0.10); padding: 8px 9px; border-bottom: 1px solid #e4eaf1; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;"><strong>96.25</strong></td> </tr> <tr> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">AIME 2026 <span>(pass@8)</span></td> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;"><strong>96.67</strong></td> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;"><strong>96.67</strong></td> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;"><strong>96.67</strong></td> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;"><strong>96.67</strong></td> <td style="background-color: rgba(59, 130, 246, 0.10); padding: 8px 9px; border-bottom: 1px solid #e4eaf1; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;"><strong>96.67</strong></td> </tr> <tr> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">AIME 2026 <span>(mean@8)</span></td> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">91.25</td> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">87.50</td> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">87.08</td> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">94.17</td> <td style="background-color: rgba(59, 130, 246, 0.10); padding: 8px 9px; border-bottom: 1px solid #e4eaf1; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;"><strong>95.00</strong></td> </tr> <tr> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">HMMT February 2025 <span>(pass@8)</span></td> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;"><strong>100.00</strong></td> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;"><strong>100.00</strong></td> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">96.67</td> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">96.67</td> <td style="background-color: rgba(59, 130, 246, 0.10); padding: 8px 9px; border-bottom: 1px solid #e4eaf1; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;"><strong>100.00</strong></td> </tr> <tr> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">HMMT February 2025 <span>(mean@8)</span></td> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">89.58</td> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">82.50</td> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">85.83</td> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">91.25</td> <td style="background-color: rgba(59, 130, 246, 0.10); padding: 8px 9px; border-bottom: 1px solid #e4eaf1; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;"><strong>96.25</strong></td> </tr> <tr> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">HMMT November 2025 <span>(pass@8)</span></td> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">93.33</td> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">96.67</td> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">93.33</td> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">93.33</td> <td style="background-color: rgba(59, 130, 246, 0.10); padding: 8px 9px; border-bottom: 1px solid #e4eaf1; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;"><strong>100.00</strong></td> </tr> <tr> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">HMMT November 2025 <span>(mean@8)</span></td> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">88.75</td> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">80.83</td> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">86.67</td> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">87.92</td> <td style="background-color: rgba(59, 130, 246, 0.10); padding: 8px 9px; border-bottom: 1px solid #e4eaf1; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;"><strong>90.83</strong></td> </tr> <tr> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">HMMT February 2026 <span>(pass@8)</span></td> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">93.94</td> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">87.88</td> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">90.91</td> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">93.94</td> <td style="background-color: rgba(59, 130, 246, 0.10); padding: 8px 9px; border-bottom: 1px solid #e4eaf1; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;"><strong>100.00</strong></td> </tr> <tr> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">HMMT February 2026 <span>(mean@8)</span></td> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">83.33</td> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">71.21</td> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">75.76</td> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">83.33</td> <td style="background-color: rgba(59, 130, 246, 0.10); padding: 8px 9px; border-bottom: 1px solid #e4eaf1; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;"><strong>91.29</strong></td> </tr> <tr> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">IMO-AnswerBench <span>(pass@8)</span></td> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">92.25</td> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">88.50</td> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">88.00</td> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">91.75</td> <td style="background-color: rgba(59, 130, 246, 0.10); padding: 8px 9px; border-bottom: 1px solid #e4eaf1; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;"><strong>95.50</strong></td> </tr> <tr> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">IMO-AnswerBench <span>(mean@8)</span></td> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">81.25</td> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">70.00</td> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">73.25</td> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">78.62</td> <td style="background-color: rgba(59, 130, 246, 0.10); padding: 8px 9px; border-bottom: 1px solid #e4eaf1; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;"><strong>86.03</strong></td> </tr> <tr> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;"><em>Mathematical average</em> <span>(pass@8)</span></td> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">94.92</td> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">93.84</td> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">93.71</td> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">95.39</td> <td style="background-color: rgba(59, 130, 246, 0.10); padding: 8px 9px; border-bottom: 1px solid #e4eaf1; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;"><strong>98.70</strong></td> </tr> <tr> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;"><em>Mathematical average</em> <span>(mean@8)</span></td> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">87.36</td> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">79.65</td> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">82.75</td> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">87.90</td> <td style="background-color: rgba(59, 130, 246, 0.10); padding: 8px 9px; border-bottom: 1px solid #e4eaf1; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;"><strong>92.61</strong></td> </tr> <tr> <th scope="rowgroup" style="background-color: transparent; padding: 10px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere; border-top: 0;">Science</th> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;"></td> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;"></td> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;"></td> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;"></td> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;"></td> </tr> <tr> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">GPQA-Diamond <span>(pass@1)</span></td> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">86.00</td> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">82.30</td> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">75.44</td> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">83.50</td> <td style="background-color: rgba(59, 130, 246, 0.10); padding: 8px 9px; border-bottom: 1px solid #e4eaf1; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;"><strong>86.52</strong></td> </tr> <tr> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">MMLU-Pro <span>(Accuracy)</span></td> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">85.20</td> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">82.60</td> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">81.94</td> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">83.16</td> <td style="background-color: rgba(59, 130, 246, 0.10); padding: 8px 9px; border-bottom: 1px solid #e4eaf1; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;"><strong>85.46</strong></td> </tr> <tr> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">MMLU-Redux <span>(Accuracy)</span></td> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">93.30</td> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">93.10</td> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">91.91</td> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">90.72</td> <td style="background-color: rgba(59, 130, 246, 0.10); padding: 8px 9px; border-bottom: 1px solid #e4eaf1; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;"><strong>93.59</strong></td> </tr> <tr> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;"><em>Scientific average</em> <span>(Average)</span></td> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">88.17</td> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">86.00</td> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">83.10</td> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">85.79</td> <td style="background-color: rgba(59, 130, 246, 0.10); padding: 8px 9px; border-bottom: 1px solid #e4eaf1; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;"><strong>88.52</strong></td> </tr> <tr> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">HLE (128K context) <span>(Score)</span></td> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">21.40</td> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">20.60</td> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">11.40</td> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">21.83</td> <td style="background-color: rgba(59, 130, 246, 0.10); padding: 8px 9px; border-bottom: 1px solid #e4eaf1; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;"><strong>24.79</strong></td> </tr> <tr> <th scope="rowgroup" style="background-color: transparent; padding: 10px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere; border-top: 0;">Logic</th> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;"></td> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;"></td> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;"></td> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;"></td> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;"></td> </tr> <tr> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">BBH <span>(pass@1)</span></td> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">90.87</td> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">86.50</td> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">84.03</td> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">89.33</td> <td style="background-color: rgba(59, 130, 246, 0.10); padding: 8px 9px; border-bottom: 1px solid #e4eaf1; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;"><strong>91.02</strong></td> </tr> <tr> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">BBEH <span>(pass@1)</span></td> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">48.05</td> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">55.18</td> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">25.86</td> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;"><strong>64.69</strong></td> <td style="background-color: rgba(59, 130, 246, 0.10); padding: 8px 9px; border-bottom: 1px solid #e4eaf1; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">52.51</td> </tr> <tr> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">KOR-Bench <span>(pass@1)</span></td> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">74.48</td> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">77.36</td> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">72.32</td> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;"><strong>80.08</strong></td> <td style="background-color: rgba(59, 130, 246, 0.10); padding: 8px 9px; border-bottom: 1px solid #e4eaf1; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">77.20</td> </tr> <tr> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;"><em>Logical macro average</em> <span>(Macro average)</span></td> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">71.13</td> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">73.01</td> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">60.74</td> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;"><strong>78.03</strong></td> <td style="background-color: rgba(59, 130, 246, 0.10); padding: 8px 9px; border-bottom: 1px solid #e4eaf1; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">73.58</td> </tr> <tr> <th scope="rowgroup" style="background-color: transparent; padding: 10px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere; border-top: 0;">Code</th> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;"></td> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;"></td> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;"></td> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;"></td> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;"></td> </tr> <tr> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">LiveCodeBench v6 <span>(pass@1)</span></td> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">85.32</td> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;"><strong>85.96</strong></td> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">78.30</td> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">76.80</td> <td style="background-color: rgba(59, 130, 246, 0.10); padding: 8px 9px; border-bottom: 1px solid #e4eaf1; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">85.66</td> </tr> <tr> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">LiveCodeBench v6 <span>(pass@8)</span></td> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">92.32</td> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">92.89</td> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">87.90</td> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">85.70</td> <td style="background-color: rgba(59, 130, 246, 0.10); padding: 8px 9px; border-bottom: 1px solid #e4eaf1; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;"><strong>93.74</strong></td> </tr> <tr><th colspan="6" style="background-color: rgba(128, 128, 128, 0.14); padding: 8px 9px; border-bottom: 1px solid #e4eaf1; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere; border-top: 0;">Agent</th></tr> <tr> <th scope="rowgroup" style="background-color: transparent; padding: 10px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere; border-top: 0;">Terminal</th> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;"></td> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;"></td> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;"></td> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;"></td> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;"></td> </tr> <tr> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">Terminal-Bench <span>(mean@5)</span></td> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">47.42</td> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">38.43</td> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">23.10</td> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">48.50</td> <td style="background-color: rgba(59, 130, 246, 0.10); padding: 8px 9px; border-bottom: 1px solid #e4eaf1; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;"><strong>58.65</strong></td> </tr> <tr> <th scope="rowgroup" style="background-color: transparent; padding: 10px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere; border-top: 0;">SWE</th> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;"></td> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;"></td> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;"></td> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;"></td> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;"></td> </tr> <tr> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">SWE-Bench-Verified (Claude Code) <span>(pass@1)</span></td> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">63.40</td> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">39.20</td> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">42.60</td> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">47.60</td> <td style="background-color: rgba(59, 130, 246, 0.10); padding: 8px 9px; border-bottom: 1px solid #e4eaf1; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;"><strong>65.40</strong></td> </tr> <tr> <th scope="rowgroup" style="background-color: transparent; padding: 10px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere; border-top: 0;">Tool use</th> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;"></td> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;"></td> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;"></td> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;"></td> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;"></td> </tr> <tr> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">MCP-Mark <span>(mean@8)</span></td> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">36.70</td> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">14.50</td> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">14.47</td> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">27.56</td> <td style="background-color: rgba(59, 130, 246, 0.10); padding: 8px 9px; border-bottom: 1px solid #e4eaf1; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;"><strong>37.30</strong></td> </tr> <tr> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">Tau3-Bench <span>(pass@1)</span></td> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">64.00</td> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">58.60</td> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">58.63</td> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">63.20</td> <td style="background-color: rgba(59, 130, 246, 0.10); padding: 8px 9px; border-bottom: 1px solid #e4eaf1; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;"><strong>67.37</strong></td> </tr> <tr> <th scope="rowgroup" style="background-color: transparent; padding: 10px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere; border-top: 0;">Search</th> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;"></td> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;"></td> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;"></td> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;"></td> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;"></td> </tr> <tr> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">BrowseComp <span>(pass@1)</span></td> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">47.00</td> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">29.20</td> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">24.20</td> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">55.60</td> <td style="background-color: rgba(59, 130, 246, 0.10); padding: 8px 9px; border-bottom: 1px solid #e4eaf1; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;"><strong>76.30</strong> / 82.20*</td> </tr> <tr> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">HLE w/Tools <span>(pass@1)</span></td> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">28.30</td> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">31.70</td> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">16.60</td> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">34.40</td> <td style="background-color: rgba(59, 130, 246, 0.10); padding: 8px 9px; border-bottom: 1px solid #e4eaf1; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;"><strong>46.10</strong> / 55.10*</td> </tr> <tr> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">WideSearch <span>(max@4)</span></td> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">70.40</td> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">54.50</td> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">60.70</td> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">70.70</td> <td style="background-color: rgba(59, 130, 246, 0.10); padding: 8px 9px; border-bottom: 1px solid #e4eaf1; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;"><strong>74.40</strong> / 75.80*</td> </tr> <tr> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">DeepSearchQA <span>(pass@1)</span></td> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">71.10</td> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">63.00</td> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">62.80</td> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">81.80</td> <td style="background-color: rgba(59, 130, 246, 0.10); padding: 8px 9px; border-bottom: 1px solid #e4eaf1; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;"><strong>83.30</strong> / 89.50*</td> </tr> <tr> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">Bamboogle <span>(mean@8)</span></td> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">90.90</td> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">91.30</td> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">90.60</td> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">86.00</td> <td style="background-color: rgba(59, 130, 246, 0.10); padding: 8px 9px; border-bottom: 1px solid #e4eaf1; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;"><strong>92.10</strong></td> </tr> <tr> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">WebWalkerQA-300 <span>(mean@8)</span></td> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">62.80</td> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">54.50</td> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">55.40</td> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">56.00</td> <td style="background-color: rgba(59, 130, 246, 0.10); padding: 8px 9px; border-bottom: 1px solid #e4eaf1; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;"><strong>70.50</strong></td> </tr> <tr> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">2WikiMultihopQA-300 <span>(mean@8)</span></td> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">87.75</td> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">89.20</td> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">86.20</td> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">89.40</td> <td style="background-color: rgba(59, 130, 246, 0.10); padding: 8px 9px; border-bottom: 1px solid #e4eaf1; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;"><strong>89.80</strong></td> </tr> <tr> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">MuSiQue-300 <span>(mean@8)</span></td> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">49.25</td> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">49.30</td> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">42.40</td> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">46.00</td> <td style="background-color: rgba(59, 130, 246, 0.10); padding: 8px 9px; border-bottom: 1px solid #e4eaf1; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;"><strong>63.00</strong></td> </tr> <tr> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">HotpotQA-300 <span>(mean@8)</span></td> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">80.80</td> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">79.50</td> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">73.20</td> <td style="background-color: transparent; padding: 8px 9px; border-bottom: 1px solid #e4eaf1; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">80.50</td> <td style="background-color: rgba(59, 130, 246, 0.10); padding: 8px 9px; border-bottom: 1px solid #e4eaf1; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;"><strong>84.70</strong></td> </tr> <tr> <td style="background-color: transparent; padding: 8px 9px; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">PopQA-300 <span>(mean@8)</span></td> <td style="background-color: transparent; padding: 8px 9px; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">68.29</td> <td style="background-color: transparent; padding: 8px 9px; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">64.70</td> <td style="background-color: transparent; padding: 8px 9px; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">58.90</td> <td style="background-color: transparent; padding: 8px 9px; border-right: 1px solid #eef2f6; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;">68.80</td> <td style="background-color: rgba(59, 130, 246, 0.10); padding: 8px 9px; vertical-align: top; text-align: left; white-space: normal; overflow-wrap: anywhere;"><strong>70.10</strong></td> </tr> </tbody> </table> \* Maxwell with context management enabled. Post-training evaluations mostly use temperature 0.6, top-p 0.95, top-k 20, and a 32K-token context. AA-LCR, mathematical benchmarks, and HLE use 128K; scientific benchmarks use 80K; LiveCodeBench v6 uses 64K; multilingual evaluation uses a maximum generation length of 16K tokens. SWE-Bench Verified is evaluated with the Claude Code harness (version 2.1.212) in a network-isolated environment, with a 256K-token context window, at most 240 interaction turns, and a four-hour wall-clock limit per task. Terminal-Bench 2.1 is evaluated in the Harbor framework with the Terminus-2 harness, using a two-hour timeout per task, a 256K-token context window, 32K max tokens, temperature 1.0, top-p 0.95, top-k 20, and interleaved thinking. The scientific average excludes HLE. ## 4. Deployment Requires NVIDIA driver >= 580 (CUDA 13.0). On older drivers, add `-e VLLM_ENABLE_CUDA_COMPATIBILITY=1` to `docker run`. Download the weights to a local directory first with the Hugging Face CLI (install it with `pip install -U huggingface_hub` if `hf` is not available): ```bash export MODEL_DIR=$PWD/Maxwell-V1-Flash hf download maxwell-model/Maxwell-V1-Flash --local-dir "$MODEL_DIR" ``` Start the vLLM OpenAI-compatible server: ```bash docker run --rm --init \ --name vllm-maxwell \ --gpus all \ --shm-size 8g \ --network host \ -v "$MODEL_DIR":/models/Maxwell-V1-Flash:ro \ --entrypoint vllm \ maxwellmodel/vllm-openai:v0.30.0 \ serve /models/Maxwell-V1-Flash \ --trust-remote-code \ --block-size 128 \ --served-model-name Maxwell-V1.0-Flash \ --port 8000 \ --tensor-parallel-size 1 \ --reasoning-parser qwen3 \ --enable-auto-tool-choice \ --tool-call-parser qwen3_coder ``` <blockquote style="margin: 16px 0; padding: 14px 16px; color: #047857; background-color: #ecfdf5; border-left: 4px solid #10b981; font-style: normal;"> You can also try Maxwell-V1.0-Flash at <a href="https://maxwell-llm.ai/playground" style="color: inherit; text-decoration: underline;">https://maxwell-llm.ai/playground</a>. </blockquote> ## 5. Model Usage Maxwell-V1.0-Flash uses the bundled chat template for system, user, assistant, and tool messages, with thinking enabled by default. For multi-turn conversations and tool calls, pass the complete assistant message back into `messages`, including `reasoning_content` and `tool_calls` when present. OpenAI-compatible tool definitions are accepted through the `tools` field. ```python import openai def chat_completion( client: openai.OpenAI, model_name: str, ) -> str: messages = [ { "role": "user", "content": ( "Why is it harder to keep a room cool when the door is open " "on a hot day? Explain in 3-4 simple sentences." ), }, ] response = client.chat.completions.create( model=model_name, messages=messages, stream=False, max_tokens=4096, temperature=0.6, top_p=0.95, extra_body={ "top_k": 20, "chat_template_kwargs": { "enable_thinking": True, }, }, ) return response.choices[0].message.content ``` For full guides and examples, see the <u><a href="https://maxwell-llm.ai/docs/quickstart"><strong>Maxwell-V1.0-Flash Quickstart</strong></a></u>. ## 6. License Maxwell-V1.0-Flash is released under the **Apache License 2.0**. See [LICENSE](./LICENSE) for details. ## 7. Contact Us If you have any questions or suggestions, feel free to reach out to our R&D and product teams via email: contact@maxwell-llm.ai --- <div align="center"> <i>Maxwell-V1.0-Flash is developed by the Maxwell team.</i> </div>
---
license: apache-2.0
pipeline_tag: text-generation
tags:
- moe
- text-generation
---
<div align="center" style="line-height: 0;">
<img src="assets/logo-en.png" alt="Maxwell-V1.0-Flash" width="380"/>
</div>
<div style="width: 100%; height: 0.5px; background-color: #e5e7eb;"></div>
<div style="height: 56px;"></div>
<div align="center" style="line-height: 0;">
<span style="display: inline-flex; align-items: center; gap: 6px;">
<a href="https://maxwell-llm.ai/"><img src="https://img.shields.io/badge/%F0%9F%8C%90%20Homepage-Maxwell-4f62d8" alt="Homepage" height="18"></a>
<a href="https://maxwell-llm.ai/playground"><img src="https://img.shields.io/badge/%F0%9F%A4%96%20Chat-Playground-18a058" alt="Chat" height="18"></a>
</span>
</div>
<div align="center" style="line-height: 0; padding-top: 3px;">
<span style="display: inline-flex; align-items: center; gap: 6px;">
<a href="https://huggingface.co/maxwell-model/Maxwell-V1-Flash"><img src="https://img.shields.io/badge/%F0%9F%A4%97%20Hugging%20Face-Maxwell-ffc107?color=ffc107&logoColor=white" alt="Hugging Face" height="18"></a>
<a href="#6-license"><img src="https://img.shields.io/badge/License-Apache%202.0-blue" alt="License" height="18"></a>
</span>
</div>
<div style="height: 20px;"></div>
<div align="center" style="font-size: 15px;">
📄&nbsp;<a href="https://github.com/maxwell-model/Maxwell-V1-Flash/blob/main/Maxwell_v1_flash_tech_report.pdf" style="color: inherit; text-decoration: underline;">Technical Report</a>&nbsp;&nbsp;|&nbsp;&nbsp;
💬&nbsp;<a href="https://github.com/maxwell-model/Maxwell-V1-Flash" style="color: inherit; text-decoration: underline;">GitHub</a>
</div>
<br>
## 1. Model Introduction
**Maxwell-V1.0-Flash** is a next-generation, high-performance Mixture-of-Experts
(MoE) language model developed by the Maxwell Team, with **35B total
parameters and 3B active parameters per token** (both excluding embeddings)
and support for context lengths up to 256K tokens. It is designed for strong
reasoning, agentic, and long-context capabilities at a practical inference cost.
<h5 style="margin: 0; font-weight: 700;">Key Features</h5>
- **Maxwell Block Sparse Attention (MBSA).** Each KV group keeps fixed coverage
of 8 prefix and 8 local 128-token blocks and adds up to 16 remote blocks
chosen by a learned indexer, bounding main attention at 4,096 tokens per KV
group while the full-history KV cache stays available. The MBSA kernel is
6.7x faster than dense GQA at 256K.
- **Manifold-Constrained Hyper-Connections.** Multiple residual streams with
learned, input-dependent aggregation and constrained mixing, widening
information flow without extra attention or feed-forward evaluations.
- **Fine-Grained Mixture of Experts.** A mostly-MoE backbone pairing
fine-grained routed experts with always-active shared experts, using a dynamic
routing correction bias alongside a lightweight sequence-wise load-balancing
loss.
**Pre-training.** Approximately **26T tokens** of multilingual data, with the
Muon optimizer on most hidden weight matrices. Training starts dense;
block-sparse attention enters at 128K mid-training, once the indexer is warmed
up against dense full-history attention, then trains jointly with the backbone.
**Tokenizer and Long Context.** A script-aware multilingual tokenizer stays
compact across languages, especially low-resource Southeast Asian ones such as
Thai, Lao, Myanmar, Khmer, and Shan. Long-context training then extends the
sequence length in stages from 4K to 256K.
**Post-training.** Two-stage all-domain SFT builds general, reasoning, and
agentic capabilities, then targets harder reasoning problems and agent
trajectories. Independent RL branches cover general tasks, verifiable reasoning
(mathematics, science, logic, code), and agentic tasks (search, terminal use,
function calling, software engineering). Multi-teacher on-policy distillation
then merges them into one model by supervising student-generated responses.
The results below summarize Maxwell-V1.0-Flash's performance across reasoning,
agentic search, and other capabilities alongside the evaluated comparison
models.
<div align="center">
<img src="assets/performance-overview.svg" alt="Selected Maxwell-V1.0-Flash benchmark results across reasoning, agentic search, and other capabilities" width="100%">
</div>
## 2. Model Specifications
<table align="center" style="margin-left: auto; margin-right: auto; border-collapse: collapse; border: 1px solid #e4eaf1; border-radius: 4px;">
<thead>
<tr>
<th style="background-color: rgba(128, 128, 128, 0.08); padding: 8px 14px; border-bottom: 1px solid #e4eaf1; text-align: left; vertical-align: top; white-space: nowrap;">Component</th>
<th style="background-color: rgba(128, 128, 128, 0.08); padding: 8px 14px; border-bottom: 1px solid #e4eaf1; text-align: left; vertical-align: top;">Configuration</th>
<th style="background-color: rgba(128, 128, 128, 0.08); padding: 8px 14px; border-bottom: 1px solid #e4eaf1; text-align: right; vertical-align: top;">Value</th>
</tr>
</thead>
<tbody>