Image-Text-to-Text
Transformers
Safetensors
inkling2
conversational
Mixture of Experts
Eval Results
8-bit precision
modelopt
Instructions to use thinkingmachines/Inkling-2-Small with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use thinkingmachines/Inkling-2-Small with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("image-text-to-text", model="thinkingmachines/Inkling-2-Small") messages = [ { "role": "user", "content": [ {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"}, {"type": "text", "text": "What animal is on the candy?"} ] }, ] pipe(text=messages)# pip install -U transformers accelerate # Load model directly from transformers import AutoProcessor, AutoModelForMultimodalLM processor = AutoProcessor.from_pretrained("thinkingmachines/Inkling-2-Small") model = AutoModelForMultimodalLM.from_pretrained("thinkingmachines/Inkling-2-Small", device_map="auto") messages = [ { "role": "user", "content": [ {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"}, {"type": "text", "text": "What animal is on the candy?"} ] }, ] inputs = processor.apply_chat_template( messages, add_generation_prompt=True, tokenize=True, return_dict=True, return_tensors="pt", ).to(model.device) outputs = model.generate(**inputs, max_new_tokens=256) print(processor.decode(outputs[0][inputs["input_ids"].shape[-1]:])) - Inference
- HuggingChat
- Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use thinkingmachines/Inkling-2-Small with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "thinkingmachines/Inkling-2-Small" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "thinkingmachines/Inkling-2-Small", "messages": [ { "role": "user", "content": [ { "type": "text", "text": "Describe this image in one sentence." }, { "type": "image_url", "image_url": { "url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg" } } ] } ] }'Use Docker
docker model run hf.co/thinkingmachines/Inkling-2-Small
- SGLang
How to use thinkingmachines/Inkling-2-Small with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "thinkingmachines/Inkling-2-Small" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "thinkingmachines/Inkling-2-Small", "messages": [ { "role": "user", "content": [ { "type": "text", "text": "Describe this image in one sentence." }, { "type": "image_url", "image_url": { "url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg" } } ] } ] }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "thinkingmachines/Inkling-2-Small" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "thinkingmachines/Inkling-2-Small", "messages": [ { "role": "user", "content": [ { "type": "text", "text": "Describe this image in one sentence." }, { "type": "image_url", "image_url": { "url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg" } } ] } ] }' - Docker Model Runner
How to use thinkingmachines/Inkling-2-Small with Docker Model Runner:
docker model run hf.co/thinkingmachines/Inkling-2-Small
[marstin-dev] update DAYJOB (Finance) and DAYJOB (Healthcare) for Inkling 2 Small
Browse filesUpdates the evaluation table to match the latest Inkling 2 Small results (https://thinkingmachines.ai/model-card/inkling-2-small/):
- DAYJOB (Finance): Inkling 2 Small 14.3% -> 13.8%.
- DAYJOB (Healthcare): Inkling 2 Small 6.4% -> 11.2%.
Best and second-best highlights are unchanged. No other changes to the README.
README.md
CHANGED
|
@@ -139,7 +139,7 @@ The training data curation process includes cleaning, processing, and modifying
|
|
| 139 |
<tr style="white-space:nowrap"><td style="border-right-color:rgba(128,128,128,0.45)">AA-Briefcase v1.1</td><td style="text-align:center;border-right-color:rgba(128,128,128,0.45)">1527</td><td style="text-align:center">1451</td><td style="text-align:center">1487</td><td style="text-align:center">1476</td><td style="text-align:center;border-right-color:rgba(128,128,128,0.45)"><span title="Second-best reported score in this row" style="display:block;margin:0 -4px;padding:2px 4px;border-radius:3px;background-color:rgba(255,206,82,0.28);color:inherit">1570</span></td><td style="text-align:center">1279</td><td style="text-align:center">1503</td><td style="text-align:center">1512</td><td style="text-align:center">1348</td><td style="text-align:center;border-right-color:rgba(128,128,128,0.45)">1497</td><td style="text-align:center">1337</td><td style="text-align:center">1203</td><td style="text-align:center"><span title="Best reported score in this row" style="display:block;margin:0 -4px;padding:2px 4px;border-radius:3px;background-color:rgba(255,206,82,0.75);color:#282828;font-weight:600">1574</span></td></tr>
|
| 140 |
<tr style="white-space:nowrap"><td style="border-right-color:rgba(128,128,128,0.45)">GDPVal-AA v2.1</td><td style="text-align:center;border-right-color:rgba(128,128,128,0.45)">1630</td><td style="text-align:center">1649</td><td style="text-align:center">1614</td><td style="text-align:center">1465</td><td style="text-align:center;border-right-color:rgba(128,128,128,0.45)"><span title="Best reported score in this row" style="display:block;margin:0 -4px;padding:2px 4px;border-radius:3px;background-color:rgba(255,206,82,0.75);color:#282828;font-weight:600">1746</span></td><td style="text-align:center">1401</td><td style="text-align:center">1657</td><td style="text-align:center"><span title="Second-best reported score in this row" style="display:block;margin:0 -4px;padding:2px 4px;border-radius:3px;background-color:rgba(255,206,82,0.28);color:inherit">1689</span></td><td style="text-align:center">1384</td><td style="text-align:center;border-right-color:rgba(128,128,128,0.45)">1543</td><td style="text-align:center">1444</td><td style="text-align:center">1445</td><td style="text-align:center">1631</td></tr>
|
| 141 |
<tr style="white-space:nowrap"><td style="border-right-color:rgba(128,128,128,0.45)">JobBench</td><td style="text-align:center;border-right-color:rgba(128,128,128,0.45)"><span title="Best reported score in this row" style="display:block;margin:0 -4px;padding:2px 4px;border-radius:3px;background-color:rgba(255,206,82,0.75);color:#282828;font-weight:600">70.5%</span></td><td style="text-align:center">60.7%</td><td style="text-align:center">61.2%</td><td style="text-align:center">67.5%</td><td style="text-align:center;border-right-color:rgba(128,128,128,0.45)"><span title="Second-best reported score in this row" style="display:block;margin:0 -4px;padding:2px 4px;border-radius:3px;background-color:rgba(255,206,82,0.28);color:inherit">68.5%</span></td><td style="text-align:center">61.8%</td><td style="text-align:center">61.4%</td><td style="text-align:center">62.0%</td><td style="text-align:center">55.2%</td><td style="text-align:center;border-right-color:rgba(128,128,128,0.45)">54.3%</td><td style="text-align:center">49.0%</td><td style="text-align:center">63.4%</td><td style="text-align:center">39.2%</td></tr>
|
| 142 |
-
<tr style="white-space:nowrap"><td style="border-right-color:rgba(128,128,128,0.45)">DAYJOB (Healthcare)</td><td style="text-align:center;border-right-color:rgba(128,128,128,0.45)"><span title="Best reported score in this row" style="display:block;margin:0 -4px;padding:2px 4px;border-radius:3px;background-color:rgba(255,206,82,0.75);color:#282828;font-weight:600">
|
| 143 |
<tr><td style="background-color:rgba(128,128,128,0.1);font-weight:600;border-right-color:rgba(128,128,128,0.45)">Enterprise Operations</td><td style="background-color:rgba(128,128,128,0.1);border-right-color:rgba(128,128,128,0.45)"></td><td style="background-color:rgba(128,128,128,0.1)"></td><td style="background-color:rgba(128,128,128,0.1)"></td><td style="background-color:rgba(128,128,128,0.1)"></td><td style="background-color:rgba(128,128,128,0.1);border-right-color:rgba(128,128,128,0.45)"></td><td style="background-color:rgba(128,128,128,0.1)"></td><td style="background-color:rgba(128,128,128,0.1)"></td><td style="background-color:rgba(128,128,128,0.1)"></td><td style="background-color:rgba(128,128,128,0.1)"></td><td style="background-color:rgba(128,128,128,0.1);border-right-color:rgba(128,128,128,0.45)"></td><td style="background-color:rgba(128,128,128,0.1)"></td><td style="background-color:rgba(128,128,128,0.1)"></td><td style="background-color:rgba(128,128,128,0.1)"></td></tr>
|
| 144 |
<tr style="white-space:nowrap"><td style="border-right-color:rgba(128,128,128,0.45)">EnterpriseOps-Gym</td><td style="text-align:center;border-right-color:rgba(128,128,128,0.45)"><span title="Second-best reported score in this row" style="display:block;margin:0 -4px;padding:2px 4px;border-radius:3px;background-color:rgba(255,206,82,0.28);color:inherit">46.8%</span></td><td style="text-align:center">33.2%</td><td style="text-align:center">29.0%</td><td style="text-align:center"><span title="Best reported score in this row" style="display:block;margin:0 -4px;padding:2px 4px;border-radius:3px;background-color:rgba(255,206,82,0.75);color:#282828;font-weight:600">48.9%</span></td><td style="text-align:center;border-right-color:rgba(128,128,128,0.45)">43.6%</td><td style="text-align:center">43.1%</td><td style="text-align:center">36.4%</td><td style="text-align:center">38.0%</td><td style="text-align:center">44.9%</td><td style="text-align:center;border-right-color:rgba(128,128,128,0.45)">45.3%</td><td style="text-align:center">24.9%</td><td style="text-align:center">45.7%</td><td style="text-align:center">25.1%</td></tr>
|
| 145 |
<tr style="white-space:nowrap"><td style="border-right-color:rgba(128,128,128,0.45)">EnterpriseBench-CoreCraft</td><td style="text-align:center;border-right-color:rgba(128,128,128,0.45)"><span title="Best reported score in this row" style="display:block;margin:0 -4px;padding:2px 4px;border-radius:3px;background-color:rgba(255,206,82,0.75);color:#282828;font-weight:600">71.8%</span></td><td style="text-align:center">46.7%</td><td style="text-align:center">–</td><td style="text-align:center">–</td><td style="text-align:center;border-right-color:rgba(128,128,128,0.45)"><span title="Second-best reported score in this row" style="display:block;margin:0 -4px;padding:2px 4px;border-radius:3px;background-color:rgba(255,206,82,0.28);color:inherit">58.5%</span></td><td style="text-align:center">–</td><td style="text-align:center">51.8%</td><td style="text-align:center">–</td><td style="text-align:center">47.7%</td><td style="text-align:center;border-right-color:rgba(128,128,128,0.45)">43.1%</td><td style="text-align:center">43.1%</td><td style="text-align:center"><span title="Second-best reported score in this row" style="display:block;margin:0 -4px;padding:2px 4px;border-radius:3px;background-color:rgba(255,206,82,0.28);color:inherit">58.5%</span></td><td style="text-align:center">–</td></tr>
|
|
@@ -148,7 +148,7 @@ The training data curation process includes cleaning, processing, and modifying
|
|
| 148 |
<tr style="white-space:nowrap"><td style="border-right-color:rgba(128,128,128,0.45)">Excel Modeling Benchmark v1.1</td><td style="text-align:center;border-right-color:rgba(128,128,128,0.45)">69.5%</td><td style="text-align:center">66.3%</td><td style="text-align:center">68.9%</td><td style="text-align:center">67.8%</td><td style="text-align:center;border-right-color:rgba(128,128,128,0.45)">69.2%</td><td style="text-align:center">62.0%</td><td style="text-align:center">69.1%</td><td style="text-align:center">67.4%</td><td style="text-align:center">68.7%</td><td style="text-align:center;border-right-color:rgba(128,128,128,0.45)"><span title="Second-best reported score in this row" style="display:block;margin:0 -4px;padding:2px 4px;border-radius:3px;background-color:rgba(255,206,82,0.28);color:inherit">71.3%</span></td><td style="text-align:center">70.0%</td><td style="text-align:center"><span title="Best reported score in this row" style="display:block;margin:0 -4px;padding:2px 4px;border-radius:3px;background-color:rgba(255,206,82,0.75);color:#282828;font-weight:600">72.9%</span></td><td style="text-align:center">65.5%</td></tr>
|
| 149 |
<tr style="white-space:nowrap"><td style="border-right-color:rgba(128,128,128,0.45)">Tax Agent Bench</td><td style="text-align:center;border-right-color:rgba(128,128,128,0.45)">38.9%</td><td style="text-align:center">31.6%</td><td style="text-align:center">28.5%</td><td style="text-align:center">32.1%</td><td style="text-align:center;border-right-color:rgba(128,128,128,0.45)"><span title="Best reported score in this row" style="display:block;margin:0 -4px;padding:2px 4px;border-radius:3px;background-color:rgba(255,206,82,0.75);color:#282828;font-weight:600">44.4%</span></td><td style="text-align:center">26.3%</td><td style="text-align:center"><span title="Second-best reported score in this row" style="display:block;margin:0 -4px;padding:2px 4px;border-radius:3px;background-color:rgba(255,206,82,0.28);color:inherit">39.7%</span></td><td style="text-align:center">32.4%</td><td style="text-align:center">30.7%</td><td style="text-align:center;border-right-color:rgba(128,128,128,0.45)">33.4%</td><td style="text-align:center">19.9%</td><td style="text-align:center">32.1%</td><td style="text-align:center">31.9%</td></tr>
|
| 150 |
<tr style="white-space:nowrap"><td style="border-right-color:rgba(128,128,128,0.45);vertical-align:middle">Apex-Accounting v1.1<br><span style="font-size:11px;opacity:0.7">pass@1</span></td><td style="text-align:center;vertical-align:middle;border-right-color:rgba(128,128,128,0.45)">14.5%</td><td style="text-align:center;vertical-align:middle">12.3%</td><td style="text-align:center;vertical-align:middle">–</td><td style="text-align:center;vertical-align:middle">13.5%</td><td style="text-align:center;vertical-align:middle;border-right-color:rgba(128,128,128,0.45)">12.5%</td><td style="text-align:center;vertical-align:middle">13.8%</td><td style="text-align:center;vertical-align:middle">13.9%</td><td style="text-align:center;vertical-align:middle">–</td><td style="text-align:center;vertical-align:middle">12.0%</td><td style="text-align:center;vertical-align:middle;border-right-color:rgba(128,128,128,0.45)">9.5%</td><td style="text-align:center;vertical-align:middle">10.8%</td><td style="text-align:center;vertical-align:middle"><span title="Best reported score in this row" style="display:block;margin:0 -4px;padding:2px 4px;border-radius:3px;background-color:rgba(255,206,82,0.75);color:#282828;font-weight:600">15.5%</span></td><td style="text-align:center;vertical-align:middle"><span title="Second-best reported score in this row" style="display:block;margin:0 -4px;padding:2px 4px;border-radius:3px;background-color:rgba(255,206,82,0.28);color:inherit">15.3%</span></td></tr>
|
| 151 |
-
<tr style="white-space:nowrap"><td style="border-right-color:rgba(128,128,128,0.45)">DAYJOB (Finance)</td><td style="text-align:center;border-right-color:rgba(128,128,128,0.45)"><span title="Best reported score in this row" style="display:block;margin:0 -4px;padding:2px 4px;border-radius:3px;background-color:rgba(255,206,82,0.75);color:#282828;font-weight:600">
|
| 152 |
<tr style="white-space:nowrap"><td style="border-right-color:rgba(128,128,128,0.45)">DeepFinance</td><td style="text-align:center;border-right-color:rgba(128,128,128,0.45)"><span title="Second-best reported score in this row" style="display:block;margin:0 -4px;padding:2px 4px;border-radius:3px;background-color:rgba(255,206,82,0.28);color:inherit">29.8%</span></td><td style="text-align:center">–</td><td style="text-align:center">–</td><td style="text-align:center">–</td><td style="text-align:center;border-right-color:rgba(128,128,128,0.45)">–</td><td style="text-align:center">–</td><td style="text-align:center">27.5%</td><td style="text-align:center">–</td><td style="text-align:center">21.5%</td><td style="text-align:center;border-right-color:rgba(128,128,128,0.45)">–</td><td style="text-align:center">26.4%</td><td style="text-align:center">–</td><td style="text-align:center"><span title="Best reported score in this row" style="display:block;margin:0 -4px;padding:2px 4px;border-radius:3px;background-color:rgba(255,206,82,0.75);color:#282828;font-weight:600">40.0%</span></td></tr>
|
| 153 |
<tr style="white-space:nowrap"><td style="border-right-color:rgba(128,128,128,0.45);vertical-align:middle">Diligence Bench<br><span style="font-size:11px;opacity:0.7">Best harness</span></td><td style="text-align:center;vertical-align:middle;border-right-color:rgba(128,128,128,0.45)"><span title="Second-best reported score in this row" style="display:block;margin:0 -4px;padding:2px 4px;border-radius:3px;background-color:rgba(255,206,82,0.28);color:inherit">58.1%</span></td><td style="text-align:center;vertical-align:middle">48.3%</td><td style="text-align:center;vertical-align:middle">–</td><td style="text-align:center;vertical-align:middle">51.6%</td><td style="text-align:center;vertical-align:middle;border-right-color:rgba(128,128,128,0.45)">50.4%</td><td style="text-align:center;vertical-align:middle"><span title="Best reported score in this row" style="display:block;margin:0 -4px;padding:2px 4px;border-radius:3px;background-color:rgba(255,206,82,0.75);color:#282828;font-weight:600">59.0%</span></td><td style="text-align:center;vertical-align:middle">49.4%</td><td style="text-align:center;vertical-align:middle">51.8%</td><td style="text-align:center;vertical-align:middle">51.5%</td><td style="text-align:center;vertical-align:middle;border-right-color:rgba(128,128,128,0.45)">50.9%</td><td style="text-align:center;vertical-align:middle">42.1%</td><td style="text-align:center;vertical-align:middle">51.1%</td><td style="text-align:center;vertical-align:middle">40.8%</td></tr>
|
| 154 |
<tr style="white-space:nowrap"><td style="border-right-color:rgba(128,128,128,0.45)">AA-AnalystAgent</td><td style="text-align:center;border-right-color:rgba(128,128,128,0.45)"><span title="Second-best reported score in this row" style="display:block;margin:0 -4px;padding:2px 4px;border-radius:3px;background-color:rgba(255,206,82,0.28);color:inherit">51.2%</span></td><td style="text-align:center">25.0%</td><td style="text-align:center">–</td><td style="text-align:center">–</td><td style="text-align:center;border-right-color:rgba(128,128,128,0.45)">–</td><td style="text-align:center">–</td><td style="text-align:center">38.8%</td><td style="text-align:center">37.5%</td><td style="text-align:center">–</td><td style="text-align:center;border-right-color:rgba(128,128,128,0.45)">38.8%</td><td style="text-align:center">43.8%</td><td style="text-align:center">46.3%</td><td style="text-align:center"><span title="Best reported score in this row" style="display:block;margin:0 -4px;padding:2px 4px;border-radius:3px;background-color:rgba(255,206,82,0.75);color:#282828;font-weight:600">53.8%</span></td></tr>
|
|
|
|
| 139 |
<tr style="white-space:nowrap"><td style="border-right-color:rgba(128,128,128,0.45)">AA-Briefcase v1.1</td><td style="text-align:center;border-right-color:rgba(128,128,128,0.45)">1527</td><td style="text-align:center">1451</td><td style="text-align:center">1487</td><td style="text-align:center">1476</td><td style="text-align:center;border-right-color:rgba(128,128,128,0.45)"><span title="Second-best reported score in this row" style="display:block;margin:0 -4px;padding:2px 4px;border-radius:3px;background-color:rgba(255,206,82,0.28);color:inherit">1570</span></td><td style="text-align:center">1279</td><td style="text-align:center">1503</td><td style="text-align:center">1512</td><td style="text-align:center">1348</td><td style="text-align:center;border-right-color:rgba(128,128,128,0.45)">1497</td><td style="text-align:center">1337</td><td style="text-align:center">1203</td><td style="text-align:center"><span title="Best reported score in this row" style="display:block;margin:0 -4px;padding:2px 4px;border-radius:3px;background-color:rgba(255,206,82,0.75);color:#282828;font-weight:600">1574</span></td></tr>
|
| 140 |
<tr style="white-space:nowrap"><td style="border-right-color:rgba(128,128,128,0.45)">GDPVal-AA v2.1</td><td style="text-align:center;border-right-color:rgba(128,128,128,0.45)">1630</td><td style="text-align:center">1649</td><td style="text-align:center">1614</td><td style="text-align:center">1465</td><td style="text-align:center;border-right-color:rgba(128,128,128,0.45)"><span title="Best reported score in this row" style="display:block;margin:0 -4px;padding:2px 4px;border-radius:3px;background-color:rgba(255,206,82,0.75);color:#282828;font-weight:600">1746</span></td><td style="text-align:center">1401</td><td style="text-align:center">1657</td><td style="text-align:center"><span title="Second-best reported score in this row" style="display:block;margin:0 -4px;padding:2px 4px;border-radius:3px;background-color:rgba(255,206,82,0.28);color:inherit">1689</span></td><td style="text-align:center">1384</td><td style="text-align:center;border-right-color:rgba(128,128,128,0.45)">1543</td><td style="text-align:center">1444</td><td style="text-align:center">1445</td><td style="text-align:center">1631</td></tr>
|
| 141 |
<tr style="white-space:nowrap"><td style="border-right-color:rgba(128,128,128,0.45)">JobBench</td><td style="text-align:center;border-right-color:rgba(128,128,128,0.45)"><span title="Best reported score in this row" style="display:block;margin:0 -4px;padding:2px 4px;border-radius:3px;background-color:rgba(255,206,82,0.75);color:#282828;font-weight:600">70.5%</span></td><td style="text-align:center">60.7%</td><td style="text-align:center">61.2%</td><td style="text-align:center">67.5%</td><td style="text-align:center;border-right-color:rgba(128,128,128,0.45)"><span title="Second-best reported score in this row" style="display:block;margin:0 -4px;padding:2px 4px;border-radius:3px;background-color:rgba(255,206,82,0.28);color:inherit">68.5%</span></td><td style="text-align:center">61.8%</td><td style="text-align:center">61.4%</td><td style="text-align:center">62.0%</td><td style="text-align:center">55.2%</td><td style="text-align:center;border-right-color:rgba(128,128,128,0.45)">54.3%</td><td style="text-align:center">49.0%</td><td style="text-align:center">63.4%</td><td style="text-align:center">39.2%</td></tr>
|
| 142 |
+
<tr style="white-space:nowrap"><td style="border-right-color:rgba(128,128,128,0.45)">DAYJOB (Healthcare)</td><td style="text-align:center;border-right-color:rgba(128,128,128,0.45)"><span title="Best reported score in this row" style="display:block;margin:0 -4px;padding:2px 4px;border-radius:3px;background-color:rgba(255,206,82,0.75);color:#282828;font-weight:600">11.2%</span></td><td style="text-align:center">1.2%</td><td style="text-align:center">–</td><td style="text-align:center">–</td><td style="text-align:center;border-right-color:rgba(128,128,128,0.45)"><span title="Second-best reported score in this row" style="display:block;margin:0 -4px;padding:2px 4px;border-radius:3px;background-color:rgba(255,206,82,0.28);color:inherit">6.0%</span></td><td style="text-align:center">–</td><td style="text-align:center">4.4%</td><td style="text-align:center">–</td><td style="text-align:center">3.6%</td><td style="text-align:center;border-right-color:rgba(128,128,128,0.45)">4.0%</td><td style="text-align:center">1.2%</td><td style="text-align:center">2.0%</td><td style="text-align:center">–</td></tr>
|
| 143 |
<tr><td style="background-color:rgba(128,128,128,0.1);font-weight:600;border-right-color:rgba(128,128,128,0.45)">Enterprise Operations</td><td style="background-color:rgba(128,128,128,0.1);border-right-color:rgba(128,128,128,0.45)"></td><td style="background-color:rgba(128,128,128,0.1)"></td><td style="background-color:rgba(128,128,128,0.1)"></td><td style="background-color:rgba(128,128,128,0.1)"></td><td style="background-color:rgba(128,128,128,0.1);border-right-color:rgba(128,128,128,0.45)"></td><td style="background-color:rgba(128,128,128,0.1)"></td><td style="background-color:rgba(128,128,128,0.1)"></td><td style="background-color:rgba(128,128,128,0.1)"></td><td style="background-color:rgba(128,128,128,0.1)"></td><td style="background-color:rgba(128,128,128,0.1);border-right-color:rgba(128,128,128,0.45)"></td><td style="background-color:rgba(128,128,128,0.1)"></td><td style="background-color:rgba(128,128,128,0.1)"></td><td style="background-color:rgba(128,128,128,0.1)"></td></tr>
|
| 144 |
<tr style="white-space:nowrap"><td style="border-right-color:rgba(128,128,128,0.45)">EnterpriseOps-Gym</td><td style="text-align:center;border-right-color:rgba(128,128,128,0.45)"><span title="Second-best reported score in this row" style="display:block;margin:0 -4px;padding:2px 4px;border-radius:3px;background-color:rgba(255,206,82,0.28);color:inherit">46.8%</span></td><td style="text-align:center">33.2%</td><td style="text-align:center">29.0%</td><td style="text-align:center"><span title="Best reported score in this row" style="display:block;margin:0 -4px;padding:2px 4px;border-radius:3px;background-color:rgba(255,206,82,0.75);color:#282828;font-weight:600">48.9%</span></td><td style="text-align:center;border-right-color:rgba(128,128,128,0.45)">43.6%</td><td style="text-align:center">43.1%</td><td style="text-align:center">36.4%</td><td style="text-align:center">38.0%</td><td style="text-align:center">44.9%</td><td style="text-align:center;border-right-color:rgba(128,128,128,0.45)">45.3%</td><td style="text-align:center">24.9%</td><td style="text-align:center">45.7%</td><td style="text-align:center">25.1%</td></tr>
|
| 145 |
<tr style="white-space:nowrap"><td style="border-right-color:rgba(128,128,128,0.45)">EnterpriseBench-CoreCraft</td><td style="text-align:center;border-right-color:rgba(128,128,128,0.45)"><span title="Best reported score in this row" style="display:block;margin:0 -4px;padding:2px 4px;border-radius:3px;background-color:rgba(255,206,82,0.75);color:#282828;font-weight:600">71.8%</span></td><td style="text-align:center">46.7%</td><td style="text-align:center">–</td><td style="text-align:center">–</td><td style="text-align:center;border-right-color:rgba(128,128,128,0.45)"><span title="Second-best reported score in this row" style="display:block;margin:0 -4px;padding:2px 4px;border-radius:3px;background-color:rgba(255,206,82,0.28);color:inherit">58.5%</span></td><td style="text-align:center">–</td><td style="text-align:center">51.8%</td><td style="text-align:center">–</td><td style="text-align:center">47.7%</td><td style="text-align:center;border-right-color:rgba(128,128,128,0.45)">43.1%</td><td style="text-align:center">43.1%</td><td style="text-align:center"><span title="Second-best reported score in this row" style="display:block;margin:0 -4px;padding:2px 4px;border-radius:3px;background-color:rgba(255,206,82,0.28);color:inherit">58.5%</span></td><td style="text-align:center">–</td></tr>
|
|
|
|
| 148 |
<tr style="white-space:nowrap"><td style="border-right-color:rgba(128,128,128,0.45)">Excel Modeling Benchmark v1.1</td><td style="text-align:center;border-right-color:rgba(128,128,128,0.45)">69.5%</td><td style="text-align:center">66.3%</td><td style="text-align:center">68.9%</td><td style="text-align:center">67.8%</td><td style="text-align:center;border-right-color:rgba(128,128,128,0.45)">69.2%</td><td style="text-align:center">62.0%</td><td style="text-align:center">69.1%</td><td style="text-align:center">67.4%</td><td style="text-align:center">68.7%</td><td style="text-align:center;border-right-color:rgba(128,128,128,0.45)"><span title="Second-best reported score in this row" style="display:block;margin:0 -4px;padding:2px 4px;border-radius:3px;background-color:rgba(255,206,82,0.28);color:inherit">71.3%</span></td><td style="text-align:center">70.0%</td><td style="text-align:center"><span title="Best reported score in this row" style="display:block;margin:0 -4px;padding:2px 4px;border-radius:3px;background-color:rgba(255,206,82,0.75);color:#282828;font-weight:600">72.9%</span></td><td style="text-align:center">65.5%</td></tr>
|
| 149 |
<tr style="white-space:nowrap"><td style="border-right-color:rgba(128,128,128,0.45)">Tax Agent Bench</td><td style="text-align:center;border-right-color:rgba(128,128,128,0.45)">38.9%</td><td style="text-align:center">31.6%</td><td style="text-align:center">28.5%</td><td style="text-align:center">32.1%</td><td style="text-align:center;border-right-color:rgba(128,128,128,0.45)"><span title="Best reported score in this row" style="display:block;margin:0 -4px;padding:2px 4px;border-radius:3px;background-color:rgba(255,206,82,0.75);color:#282828;font-weight:600">44.4%</span></td><td style="text-align:center">26.3%</td><td style="text-align:center"><span title="Second-best reported score in this row" style="display:block;margin:0 -4px;padding:2px 4px;border-radius:3px;background-color:rgba(255,206,82,0.28);color:inherit">39.7%</span></td><td style="text-align:center">32.4%</td><td style="text-align:center">30.7%</td><td style="text-align:center;border-right-color:rgba(128,128,128,0.45)">33.4%</td><td style="text-align:center">19.9%</td><td style="text-align:center">32.1%</td><td style="text-align:center">31.9%</td></tr>
|
| 150 |
<tr style="white-space:nowrap"><td style="border-right-color:rgba(128,128,128,0.45);vertical-align:middle">Apex-Accounting v1.1<br><span style="font-size:11px;opacity:0.7">pass@1</span></td><td style="text-align:center;vertical-align:middle;border-right-color:rgba(128,128,128,0.45)">14.5%</td><td style="text-align:center;vertical-align:middle">12.3%</td><td style="text-align:center;vertical-align:middle">–</td><td style="text-align:center;vertical-align:middle">13.5%</td><td style="text-align:center;vertical-align:middle;border-right-color:rgba(128,128,128,0.45)">12.5%</td><td style="text-align:center;vertical-align:middle">13.8%</td><td style="text-align:center;vertical-align:middle">13.9%</td><td style="text-align:center;vertical-align:middle">–</td><td style="text-align:center;vertical-align:middle">12.0%</td><td style="text-align:center;vertical-align:middle;border-right-color:rgba(128,128,128,0.45)">9.5%</td><td style="text-align:center;vertical-align:middle">10.8%</td><td style="text-align:center;vertical-align:middle"><span title="Best reported score in this row" style="display:block;margin:0 -4px;padding:2px 4px;border-radius:3px;background-color:rgba(255,206,82,0.75);color:#282828;font-weight:600">15.5%</span></td><td style="text-align:center;vertical-align:middle"><span title="Second-best reported score in this row" style="display:block;margin:0 -4px;padding:2px 4px;border-radius:3px;background-color:rgba(255,206,82,0.28);color:inherit">15.3%</span></td></tr>
|
| 151 |
+
<tr style="white-space:nowrap"><td style="border-right-color:rgba(128,128,128,0.45)">DAYJOB (Finance)</td><td style="text-align:center;border-right-color:rgba(128,128,128,0.45)"><span title="Best reported score in this row" style="display:block;margin:0 -4px;padding:2px 4px;border-radius:3px;background-color:rgba(255,206,82,0.75);color:#282828;font-weight:600">13.8%</span></td><td style="text-align:center">7.3%</td><td style="text-align:center">–</td><td style="text-align:center">–</td><td style="text-align:center;border-right-color:rgba(128,128,128,0.45)"><span title="Second-best reported score in this row" style="display:block;margin:0 -4px;padding:2px 4px;border-radius:3px;background-color:rgba(255,206,82,0.28);color:inherit">12.5%</span></td><td style="text-align:center">–</td><td style="text-align:center">11.0%</td><td style="text-align:center">–</td><td style="text-align:center">4.8%</td><td style="text-align:center;border-right-color:rgba(128,128,128,0.45)">5.5%</td><td style="text-align:center">6.8%</td><td style="text-align:center">4.5%</td><td style="text-align:center">–</td></tr>
|
| 152 |
<tr style="white-space:nowrap"><td style="border-right-color:rgba(128,128,128,0.45)">DeepFinance</td><td style="text-align:center;border-right-color:rgba(128,128,128,0.45)"><span title="Second-best reported score in this row" style="display:block;margin:0 -4px;padding:2px 4px;border-radius:3px;background-color:rgba(255,206,82,0.28);color:inherit">29.8%</span></td><td style="text-align:center">–</td><td style="text-align:center">–</td><td style="text-align:center">–</td><td style="text-align:center;border-right-color:rgba(128,128,128,0.45)">–</td><td style="text-align:center">–</td><td style="text-align:center">27.5%</td><td style="text-align:center">–</td><td style="text-align:center">21.5%</td><td style="text-align:center;border-right-color:rgba(128,128,128,0.45)">–</td><td style="text-align:center">26.4%</td><td style="text-align:center">–</td><td style="text-align:center"><span title="Best reported score in this row" style="display:block;margin:0 -4px;padding:2px 4px;border-radius:3px;background-color:rgba(255,206,82,0.75);color:#282828;font-weight:600">40.0%</span></td></tr>
|
| 153 |
<tr style="white-space:nowrap"><td style="border-right-color:rgba(128,128,128,0.45);vertical-align:middle">Diligence Bench<br><span style="font-size:11px;opacity:0.7">Best harness</span></td><td style="text-align:center;vertical-align:middle;border-right-color:rgba(128,128,128,0.45)"><span title="Second-best reported score in this row" style="display:block;margin:0 -4px;padding:2px 4px;border-radius:3px;background-color:rgba(255,206,82,0.28);color:inherit">58.1%</span></td><td style="text-align:center;vertical-align:middle">48.3%</td><td style="text-align:center;vertical-align:middle">–</td><td style="text-align:center;vertical-align:middle">51.6%</td><td style="text-align:center;vertical-align:middle;border-right-color:rgba(128,128,128,0.45)">50.4%</td><td style="text-align:center;vertical-align:middle"><span title="Best reported score in this row" style="display:block;margin:0 -4px;padding:2px 4px;border-radius:3px;background-color:rgba(255,206,82,0.75);color:#282828;font-weight:600">59.0%</span></td><td style="text-align:center;vertical-align:middle">49.4%</td><td style="text-align:center;vertical-align:middle">51.8%</td><td style="text-align:center;vertical-align:middle">51.5%</td><td style="text-align:center;vertical-align:middle;border-right-color:rgba(128,128,128,0.45)">50.9%</td><td style="text-align:center;vertical-align:middle">42.1%</td><td style="text-align:center;vertical-align:middle">51.1%</td><td style="text-align:center;vertical-align:middle">40.8%</td></tr>
|
| 154 |
<tr style="white-space:nowrap"><td style="border-right-color:rgba(128,128,128,0.45)">AA-AnalystAgent</td><td style="text-align:center;border-right-color:rgba(128,128,128,0.45)"><span title="Second-best reported score in this row" style="display:block;margin:0 -4px;padding:2px 4px;border-radius:3px;background-color:rgba(255,206,82,0.28);color:inherit">51.2%</span></td><td style="text-align:center">25.0%</td><td style="text-align:center">–</td><td style="text-align:center">–</td><td style="text-align:center;border-right-color:rgba(128,128,128,0.45)">–</td><td style="text-align:center">–</td><td style="text-align:center">38.8%</td><td style="text-align:center">37.5%</td><td style="text-align:center">–</td><td style="text-align:center;border-right-color:rgba(128,128,128,0.45)">38.8%</td><td style="text-align:center">43.8%</td><td style="text-align:center">46.3%</td><td style="text-align:center"><span title="Best reported score in this row" style="display:block;margin:0 -4px;padding:2px 4px;border-radius:3px;background-color:rgba(255,206,82,0.75);color:#282828;font-weight:600">53.8%</span></td></tr>
|