DedeProGames commited on
Commit
07ba04e
·
verified ·
1 Parent(s): 2419e16

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +30 -26
README.md CHANGED
@@ -22,7 +22,9 @@ We are proud to introduce **OxCoder-9B**, our lightweight coding model for long-
22
 
23
  ## Benchmark
24
 
25
- <table>
 
 
26
  <tr>
27
  <th style="background: rgba(128,128,128,0.1); text-align: center; border: hidden;"> </th>
28
  <th style="background: rgba(128,128,128,0.1); text-align: center; border: hidden;">OxCoder-9B</th>
@@ -32,103 +34,105 @@ We are proud to introduce **OxCoder-9B**, our lightweight coding model for long-
32
  <th style="background: rgba(128,128,128,0.1); text-align: center; border: hidden;">Gemma-4-31B</th>
33
  </tr>
34
  <tr>
35
- <td align="center" colspan="6" style="background: linear-gradient(90deg, rgba(124,58,237,0.45) 0%, rgba(99,102,241,0.42) 50%, rgba(59,130,246,0.45) 100%); font-weight: bold; height:32px; padding-top:2px; padding-bottom:2px;"><i>Coding</i></td>
36
  </tr>
37
  <tr>
38
- <td style="text-align:left; padding:9px 10px 9px 18px;"><div style="font-size:15px; font-weight:600; line-height:1.22;">Agentic terminal coding</div><div style="margin-top:4px; font-size:11px; font-weight:400; line-height:1.2; opacity:0.62;">Terminal-Bench 2.1 (Terminus-2)</div></td>
39
- <td align="center" style="background: linear-gradient(135deg, rgba(124,58,237,0.16) 0%, rgba(118,65,238,0.16) 100%); vertical-align:middle;"><b>49.6</b></td>
40
  <td align="center" style="vertical-align:middle;">46.2</td>
41
  <td align="center" style="vertical-align:middle;">43.1</td>
42
  <td align="center" style="vertical-align:middle;">21.3</td>
43
  <td align="center" style="vertical-align:middle;">42.1</td>
44
  </tr>
45
  <tr>
46
- <td style="text-align:left; padding:9px 10px 9px 18px;"><div style="font-size:15px; font-weight:600; line-height:1.22;">Agentic terminal coding</div><div style="margin-top:4px; font-size:11px; font-weight:400; line-height:1.2; opacity:0.62;">Terminal-Bench 2.1 (Claude Code)</div></td>
47
- <td align="center" style="background: linear-gradient(135deg, rgba(118,65,238,0.16) 0%, rgba(112,71,239,0.16) 100%); vertical-align:middle;"><b>50.8</b></td>
48
  <td align="center" style="vertical-align:middle;">47.0</td>
49
  <td align="center" style="vertical-align:middle;">40.6</td>
50
  <td align="center" style="vertical-align:middle;">18.9</td>
51
  <td align="center" style="vertical-align:middle;">—</td>
52
  </tr>
53
  <tr>
54
- <td style="text-align:left; padding:9px 10px 9px 18px;"><div style="font-size:15px; font-weight:600; line-height:1.22;">Agentic coding</div><div style="margin-top:4px; font-size:11px; font-weight:400; line-height:1.2; opacity:0.62;">SWE-bench Verified</div></td>
55
- <td align="center" style="background: linear-gradient(135deg, rgba(112,71,239,0.16) 0%, rgba(106,78,239,0.16) 100%); vertical-align:middle;"><b>73.5</b></td>
56
  <td align="center" style="vertical-align:middle;">70.6</td>
57
  <td align="center" style="vertical-align:middle;">69.4</td>
58
  <td align="center" style="vertical-align:middle;">53.2</td>
59
  <td align="center" style="vertical-align:middle;">52.0</td>
60
  </tr>
61
  <tr>
62
- <td style="text-align:left; padding:9px 10px 9px 18px;"><div style="font-size:15px; font-weight:600; line-height:1.22;">Real-world software engineering</div><div style="margin-top:4px; font-size:11px; font-weight:400; line-height:1.2; opacity:0.62;">SWE-bench Pro</div></td>
63
- <td align="center" style="background: linear-gradient(135deg, rgba(106,78,239,0.16) 0%, rgba(100,84,240,0.16) 100%); vertical-align:middle;"><b>49.1</b></td>
64
  <td align="center" style="vertical-align:middle;">47.5</td>
65
  <td align="center" style="vertical-align:middle;">42.9</td>
66
  <td align="center" style="vertical-align:middle;">31.3</td>
67
  <td align="center" style="vertical-align:middle;">35.7</td>
68
  </tr>
69
  <tr>
70
- <td style="text-align:left; padding:9px 10px 9px 18px;"><div style="font-size:15px; font-weight:600; line-height:1.22;">Repo-level code generation</div><div style="margin-top:4px; font-size:11px; font-weight:400; line-height:1.2; opacity:0.62;">NL2Repo</div></td>
71
- <td align="center" style="background: linear-gradient(135deg, rgba(100,84,240,0.16) 0%, rgba(94,91,241,0.16) 100%); vertical-align:middle;"><b>36.2</b></td>
72
  <td align="center" style="vertical-align:middle;">32.4</td>
73
  <td align="center" style="vertical-align:middle;">27.2</td>
74
  <td align="center" style="vertical-align:middle;">16.2</td>
75
  <td align="center" style="vertical-align:middle;">15.5</td>
76
  </tr>
77
  <tr>
78
- <td align="center" colspan="6" style="background: linear-gradient(90deg, rgba(124,58,237,0.45) 0%, rgba(99,102,241,0.42) 50%, rgba(59,130,246,0.45) 100%); font-weight: bold; height:32px; padding-top:2px; padding-bottom:2px;"><i>Reasoning</i></td>
79
  </tr>
80
  <tr>
81
- <td style="text-align:left; padding:9px 10px 9px 18px;"><div style="font-size:15px; font-weight:600; line-height:1.22;">Expert-level reasoning</div><div style="margin-top:4px; font-size:11px; font-weight:400; line-height:1.2; opacity:0.62;">HLE (no tools)</div></td>
82
- <td align="center" style="background: linear-gradient(135deg, rgba(94,91,241,0.16) 0%, rgba(89,97,242,0.16) 100%); vertical-align:middle;"><b>21.2</b></td>
83
  <td align="center" style="vertical-align:middle;">20.2</td>
84
  <td align="center" style="vertical-align:middle;">16.8</td>
85
  <td align="center" style="vertical-align:middle;">14.7</td>
86
  <td align="center" style="vertical-align:middle;">19.5</td>
87
  </tr>
88
  <tr>
89
- <td style="text-align:left; padding:9px 10px 9px 18px;"><div style="font-size:15px; font-weight:600; line-height:1.22;">Tool-augmented reasoning</div><div style="margin-top:4px; font-size:11px; font-weight:400; line-height:1.2; opacity:0.62;">HLE (with tools)</div></td>
90
- <td align="center" style="background: linear-gradient(135deg, rgba(89,97,242,0.16) 0%, rgba(83,104,243,0.16) 100%); vertical-align:middle;"><b>32.8</b></td>
91
  <td align="center" style="vertical-align:middle;">30.5</td>
92
  <td align="center" style="vertical-align:middle;">26.4</td>
93
  <td align="center" style="vertical-align:middle;">24.5</td>
94
  <td align="center" style="vertical-align:middle;">26.5</td>
95
  </tr>
96
  <tr>
97
- <td style="text-align:left; padding:9px 10px 9px 18px;"><div style="font-size:15px; font-weight:600; line-height:1.22;">Scientific reasoning</div><div style="margin-top:4px; font-size:11px; font-weight:400; line-height:1.2; opacity:0.62;">GPQA Diamond</div></td>
98
- <td align="center" style="background: linear-gradient(135deg, rgba(83,104,243,0.16) 0%, rgba(77,110,244,0.16) 100%); vertical-align:middle;"><b>86.9</b></td>
99
  <td align="center" style="vertical-align:middle;">86.4</td>
100
  <td align="center" style="vertical-align:middle;">82.5</td>
101
  <td align="center" style="vertical-align:middle;">81.7</td>
102
  <td align="center" style="vertical-align:middle;">84.3</td>
103
  </tr>
104
  <tr>
105
- <td align="center" colspan="6" style="background: linear-gradient(90deg, rgba(124,58,237,0.45) 0%, rgba(99,102,241,0.42) 50%, rgba(59,130,246,0.45) 100%); font-weight: bold; height:32px; padding-top:2px; padding-bottom:2px;"><i>Agentic</i></td>
106
  </tr>
107
  <tr>
108
- <td style="text-align:left; padding:9px 10px 9px 18px;"><div style="font-size:15px; font-weight:600; line-height:1.22;">Multi-tool orchestration</div><div style="margin-top:4px; font-size:11px; font-weight:400; line-height:1.2; opacity:0.62;">MCP-Atlas</div></td>
109
- <td align="center" style="background: linear-gradient(135deg, rgba(77,110,244,0.16) 0%, rgba(71,117,244,0.16) 100%); vertical-align:middle;"><b>56.7</b></td>
110
  <td align="center" style="vertical-align:middle;">54.2</td>
111
  <td align="center" style="vertical-align:middle;">49.4</td>
112
  <td align="center" style="vertical-align:middle;">46.8</td>
113
  <td align="center" style="vertical-align:middle;">55.0</td>
114
  </tr>
115
  <tr>
116
- <td style="text-align:left; padding:9px 10px 9px 18px;"><div style="font-size:15px; font-weight:600; line-height:1.22;">Web browsing and research</div><div style="margin-top:4px; font-size:11px; font-weight:400; line-height:1.2; opacity:0.62;">BrowseComp</div></td>
117
- <td align="center" style="background: linear-gradient(135deg, rgba(71,117,244,0.16) 0%, rgba(65,123,245,0.16) 100%); vertical-align:middle;"><b>57.4</b></td>
118
  <td align="center" style="vertical-align:middle;">56.4</td>
119
  <td align="center" style="vertical-align:middle;">44.8</td>
120
  <td align="center" style="vertical-align:middle;">41.5</td>
121
  <td align="center" style="vertical-align:middle;">—</td>
122
  </tr>
123
  <tr>
124
- <td style="text-align:left; padding:9px 10px 9px 18px;"><div style="font-size:15px; font-weight:600; line-height:1.22;">Real-user agentic coding</div><div style="margin-top:4px; font-size:11px; font-weight:400; line-height:1.2; opacity:0.62;">ClawEval</div></td>
125
- <td align="center" style="background: linear-gradient(135deg, rgba(65,123,245,0.16) 0%, rgba(59,130,246,0.16) 100%); vertical-align:middle;"><b>67.8</b></td>
126
  <td align="center" style="vertical-align:middle;">66.5</td>
127
  <td align="center" style="vertical-align:middle;">63.1</td>
128
  <td align="center" style="vertical-align:middle;">53.2</td>
129
  <td align="center" style="vertical-align:middle;">48.5</td>
130
  </tr>
131
  </table>
 
 
132
 
133
  <p style="margin-top:12px;font-size:10px;opacity:0.7">
134
  * All results reported for OxCoder-9B are averaged over five independent runs. A dash (—) means the score was not reported for that model.<br/>
 
22
 
23
  ## Benchmark
24
 
25
+ <div align="center">
26
+ <div style="display:inline-block; max-width:100%; border-radius:20px; overflow:hidden;">
27
+ <table style="margin:0; border-collapse:collapse; border-radius:20px;">
28
  <tr>
29
  <th style="background: rgba(128,128,128,0.1); text-align: center; border: hidden;"> </th>
30
  <th style="background: rgba(128,128,128,0.1); text-align: center; border: hidden;">OxCoder-9B</th>
 
34
  <th style="background: rgba(128,128,128,0.1); text-align: center; border: hidden;">Gemma-4-31B</th>
35
  </tr>
36
  <tr>
37
+ <td align="center" colspan="6" style="background: linear-gradient(90deg, rgba(0,120,205,0.45) 0px, rgba(0,125,198,0.45) 4px, rgba(0,120,205,0.45) 4px, rgba(23,158,114,0.42) 50%, rgba(90,158,5,0.45) 100%); border: hidden; font-weight: bold; height:32px; padding-top:2px; padding-bottom:2px;"><i>Coding</i></td>
38
  </tr>
39
  <tr>
40
+ <td style="text-align:left; padding:9px 10px 9px 20px; background-image: linear-gradient(180deg, rgba(0,125,198,0.45) 0%, rgba(0,134,188,0.45) 100%); background-repeat: no-repeat; background-size: 4px 100%; background-position: left center; border: hidden;"><div style="font-size:15px; font-weight:600; line-height:1.22;">Agentic terminal coding</div><div style="margin-top:4px; font-size:11px; font-weight:400; line-height:1.2; opacity:0.62;">Terminal-Bench 2.1 (Terminus-2)</div></td>
41
+ <td align="center" style="background: linear-gradient(135deg, rgba(0,125,198,0.16) 0%, rgba(0,134,188,0.16) 100%); vertical-align:middle;"><b>49.6</b></td>
42
  <td align="center" style="vertical-align:middle;">46.2</td>
43
  <td align="center" style="vertical-align:middle;">43.1</td>
44
  <td align="center" style="vertical-align:middle;">21.3</td>
45
  <td align="center" style="vertical-align:middle;">42.1</td>
46
  </tr>
47
  <tr>
48
+ <td style="text-align:left; padding:9px 10px 9px 20px; background-image: linear-gradient(180deg, rgba(0,134,188,0.45) 0%, rgba(1,141,178,0.45) 100%); background-repeat: no-repeat; background-size: 4px 100%; background-position: left center; border: hidden;"><div style="font-size:15px; font-weight:600; line-height:1.22;">Agentic terminal coding</div><div style="margin-top:4px; font-size:11px; font-weight:400; line-height:1.2; opacity:0.62;">Terminal-Bench 2.1 (Claude Code)</div></td>
49
+ <td align="center" style="background: linear-gradient(135deg, rgba(0,134,188,0.16) 0%, rgba(1,141,178,0.16) 100%); vertical-align:middle;"><b>50.8</b></td>
50
  <td align="center" style="vertical-align:middle;">47.0</td>
51
  <td align="center" style="vertical-align:middle;">40.6</td>
52
  <td align="center" style="vertical-align:middle;">18.9</td>
53
  <td align="center" style="vertical-align:middle;">—</td>
54
  </tr>
55
  <tr>
56
+ <td style="text-align:left; padding:9px 10px 9px 20px; background-image: linear-gradient(180deg, rgba(1,141,178,0.45) 0%, rgba(4,148,165,0.45) 100%); background-repeat: no-repeat; background-size: 4px 100%; background-position: left center; border: hidden;"><div style="font-size:15px; font-weight:600; line-height:1.22;">Agentic coding</div><div style="margin-top:4px; font-size:11px; font-weight:400; line-height:1.2; opacity:0.62;">SWE-bench Verified</div></td>
57
+ <td align="center" style="background: linear-gradient(135deg, rgba(1,141,178,0.16) 0%, rgba(4,148,165,0.16) 100%); vertical-align:middle;"><b>73.5</b></td>
58
  <td align="center" style="vertical-align:middle;">70.6</td>
59
  <td align="center" style="vertical-align:middle;">69.4</td>
60
  <td align="center" style="vertical-align:middle;">53.2</td>
61
  <td align="center" style="vertical-align:middle;">52.0</td>
62
  </tr>
63
  <tr>
64
+ <td style="text-align:left; padding:9px 10px 9px 20px; background-image: linear-gradient(180deg, rgba(4,148,165,0.45) 0%, rgba(11,152,146,0.45) 100%); background-repeat: no-repeat; background-size: 4px 100%; background-position: left center; border: hidden;"><div style="font-size:15px; font-weight:600; line-height:1.22;">Real-world software engineering</div><div style="margin-top:4px; font-size:11px; font-weight:400; line-height:1.2; opacity:0.62;">SWE-bench Pro</div></td>
65
+ <td align="center" style="background: linear-gradient(135deg, rgba(4,148,165,0.16) 0%, rgba(11,152,146,0.16) 100%); vertical-align:middle;"><b>49.1</b></td>
66
  <td align="center" style="vertical-align:middle;">47.5</td>
67
  <td align="center" style="vertical-align:middle;">42.9</td>
68
  <td align="center" style="vertical-align:middle;">31.3</td>
69
  <td align="center" style="vertical-align:middle;">35.7</td>
70
  </tr>
71
  <tr>
72
+ <td style="text-align:left; padding:9px 10px 9px 20px; background-image: linear-gradient(180deg, rgba(11,152,146,0.45) 0%, rgba(17,155,128,0.45) 100%); background-repeat: no-repeat; background-size: 4px 100%; background-position: left center; border: hidden;"><div style="font-size:15px; font-weight:600; line-height:1.22;">Repo-level code generation</div><div style="margin-top:4px; font-size:11px; font-weight:400; line-height:1.2; opacity:0.62;">NL2Repo</div></td>
73
+ <td align="center" style="background: linear-gradient(135deg, rgba(11,152,146,0.16) 0%, rgba(17,155,128,0.16) 100%); vertical-align:middle;"><b>36.2</b></td>
74
  <td align="center" style="vertical-align:middle;">32.4</td>
75
  <td align="center" style="vertical-align:middle;">27.2</td>
76
  <td align="center" style="vertical-align:middle;">16.2</td>
77
  <td align="center" style="vertical-align:middle;">15.5</td>
78
  </tr>
79
  <tr>
80
+ <td align="center" colspan="6" style="background: linear-gradient(90deg, rgba(17,155,128,0.45) 0px, rgba(22,158,117,0.45) 4px, rgba(0,120,205,0.45) 4px, rgba(23,158,114,0.42) 50%, rgba(90,158,5,0.45) 100%); border: hidden; font-weight: bold; height:32px; padding-top:2px; padding-bottom:2px;"><i>Reasoning</i></td>
81
  </tr>
82
  <tr>
83
+ <td style="text-align:left; padding:9px 10px 9px 20px; background-image: linear-gradient(180deg, rgba(22,158,117,0.45) 0%, rgba(33,159,98,0.45) 100%); background-repeat: no-repeat; background-size: 4px 100%; background-position: left center; border: hidden;"><div style="font-size:15px; font-weight:600; line-height:1.22;">Expert-level reasoning</div><div style="margin-top:4px; font-size:11px; font-weight:400; line-height:1.2; opacity:0.62;">HLE (no tools)</div></td>
84
+ <td align="center" style="background: linear-gradient(135deg, rgba(22,158,117,0.16) 0%, rgba(33,159,98,0.16) 100%); vertical-align:middle;"><b>21.2</b></td>
85
  <td align="center" style="vertical-align:middle;">20.2</td>
86
  <td align="center" style="vertical-align:middle;">16.8</td>
87
  <td align="center" style="vertical-align:middle;">14.7</td>
88
  <td align="center" style="vertical-align:middle;">19.5</td>
89
  </tr>
90
  <tr>
91
+ <td style="text-align:left; padding:9px 10px 9px 20px; background-image: linear-gradient(180deg, rgba(33,159,98,0.45) 0%, rgba(45,159,79,0.45) 100%); background-repeat: no-repeat; background-size: 4px 100%; background-position: left center; border: hidden;"><div style="font-size:15px; font-weight:600; line-height:1.22;">Tool-augmented reasoning</div><div style="margin-top:4px; font-size:11px; font-weight:400; line-height:1.2; opacity:0.62;">HLE (with tools)</div></td>
92
+ <td align="center" style="background: linear-gradient(135deg, rgba(33,159,98,0.16) 0%, rgba(45,159,79,0.16) 100%); vertical-align:middle;"><b>32.8</b></td>
93
  <td align="center" style="vertical-align:middle;">30.5</td>
94
  <td align="center" style="vertical-align:middle;">26.4</td>
95
  <td align="center" style="vertical-align:middle;">24.5</td>
96
  <td align="center" style="vertical-align:middle;">26.5</td>
97
  </tr>
98
  <tr>
99
+ <td style="text-align:left; padding:9px 10px 9px 20px; background-image: linear-gradient(180deg, rgba(45,159,79,0.45) 0%, rgba(56,160,61,0.45) 100%); background-repeat: no-repeat; background-size: 4px 100%; background-position: left center; border: hidden;"><div style="font-size:15px; font-weight:600; line-height:1.22;">Scientific reasoning</div><div style="margin-top:4px; font-size:11px; font-weight:400; line-height:1.2; opacity:0.62;">GPQA Diamond</div></td>
100
+ <td align="center" style="background: linear-gradient(135deg, rgba(45,159,79,0.16) 0%, rgba(56,160,61,0.16) 100%); vertical-align:middle;"><b>86.9</b></td>
101
  <td align="center" style="vertical-align:middle;">86.4</td>
102
  <td align="center" style="vertical-align:middle;">82.5</td>
103
  <td align="center" style="vertical-align:middle;">81.7</td>
104
  <td align="center" style="vertical-align:middle;">84.3</td>
105
  </tr>
106
  <tr>
107
+ <td align="center" colspan="6" style="background: linear-gradient(90deg, rgba(56,160,61,0.45) 0px, rgba(63,159,50,0.45) 4px, rgba(0,120,205,0.45) 4px, rgba(23,158,114,0.42) 50%, rgba(90,158,5,0.45) 100%); border: hidden; font-weight: bold; height:32px; padding-top:2px; padding-bottom:2px;"><i>Agentic</i></td>
108
  </tr>
109
  <tr>
110
+ <td style="text-align:left; padding:9px 10px 9px 20px; background-image: linear-gradient(180deg, rgba(63,159,50,0.45) 0%, rgba(72,159,35,0.45) 100%); background-repeat: no-repeat; background-size: 4px 100%; background-position: left center; border: hidden;"><div style="font-size:15px; font-weight:600; line-height:1.22;">Multi-tool orchestration</div><div style="margin-top:4px; font-size:11px; font-weight:400; line-height:1.2; opacity:0.62;">MCP-Atlas</div></td>
111
+ <td align="center" style="background: linear-gradient(135deg, rgba(63,159,50,0.16) 0%, rgba(72,159,35,0.16) 100%); vertical-align:middle;"><b>56.7</b></td>
112
  <td align="center" style="vertical-align:middle;">54.2</td>
113
  <td align="center" style="vertical-align:middle;">49.4</td>
114
  <td align="center" style="vertical-align:middle;">46.8</td>
115
  <td align="center" style="vertical-align:middle;">55.0</td>
116
  </tr>
117
  <tr>
118
+ <td style="text-align:left; padding:9px 10px 9px 20px; background-image: linear-gradient(180deg, rgba(72,159,35,0.45) 0%, rgba(81,158,20,0.45) 100%); background-repeat: no-repeat; background-size: 4px 100%; background-position: left center; border: hidden;"><div style="font-size:15px; font-weight:600; line-height:1.22;">Web browsing and research</div><div style="margin-top:4px; font-size:11px; font-weight:400; line-height:1.2; opacity:0.62;">BrowseComp</div></td>
119
+ <td align="center" style="background: linear-gradient(135deg, rgba(72,159,35,0.16) 0%, rgba(81,158,20,0.16) 100%); vertical-align:middle;"><b>57.4</b></td>
120
  <td align="center" style="vertical-align:middle;">56.4</td>
121
  <td align="center" style="vertical-align:middle;">44.8</td>
122
  <td align="center" style="vertical-align:middle;">41.5</td>
123
  <td align="center" style="vertical-align:middle;">—</td>
124
  </tr>
125
  <tr>
126
+ <td style="text-align:left; padding:9px 10px 9px 20px; background-image: linear-gradient(180deg, rgba(81,158,20,0.45) 0%, rgba(90,158,5,0.45) 100%); background-repeat: no-repeat; background-size: 4px 100%; background-position: left center; border: hidden;"><div style="font-size:15px; font-weight:600; line-height:1.22;">Real-user agentic coding</div><div style="margin-top:4px; font-size:11px; font-weight:400; line-height:1.2; opacity:0.62;">ClawEval</div></td>
127
+ <td align="center" style="background: linear-gradient(135deg, rgba(81,158,20,0.16) 0%, rgba(90,158,5,0.16) 100%); vertical-align:middle;"><b>67.8</b></td>
128
  <td align="center" style="vertical-align:middle;">66.5</td>
129
  <td align="center" style="vertical-align:middle;">63.1</td>
130
  <td align="center" style="vertical-align:middle;">53.2</td>
131
  <td align="center" style="vertical-align:middle;">48.5</td>
132
  </tr>
133
  </table>
134
+ </div>
135
+ </div>
136
 
137
  <p style="margin-top:12px;font-size:10px;opacity:0.7">
138
  * All results reported for OxCoder-9B are averaged over five independent runs. A dash (—) means the score was not reported for that model.<br/>