arex-2-p300

A tt-model v6 thin bundle that serves BAAI/AREX-2 (revision d4e3502f92d9e889031c2d04e387ab2eb3520268) on 2 Tenstorrent blackhole chips (mesh P150x2). The tt-orchard harness staged it from a run of that model.

License

The weights this bundle points to are licensed Apache 2.0 (https://www.apache.org/licenses/LICENSE-2.0). The bundle ships no weights. tt-model downloads them from BAAI/AREX-2 with your own Hugging Face account.

What it runs

• Weights: BAAI/AREX-2 at revision d4e3502f92d9e889031c2d04e387ab2eb3520268.
• Model code: the Tenstorrent implementation of Qwen/Qwen3.8-27B, taken from the bundle qwen3.8-27b-dflash2-p300. BAAI/AREX-2 has the same text architecture, so only the weights differ.
• At each start the launcher builds model-dir/ from the configuration files of Qwen/Qwen3.8-27B (config.json, generation_config.json, preprocessor_config.json, video_preprocessor_config.json, shipped in base_config/) and the tokenizer and weights of BAAI/AREX-2. It sets MODEL_WEIGHTS_DIR and HF_MODEL to that directory, so the chips load the weights of BAAI/AREX-2.
• Context length 262144 tokens; up to 4 sequences at a time.
• BAAI/AREX-2 ships no generation_config.json, so the stop tokens and the sampling defaults the server applies when a request sets none are those of Qwen/Qwen3.8-27B.
• Speculative decoding uses the drafter incoai/Qwen3.8-27B-DFlash2, which was trained on Qwen/Qwen3.8-27B. Its acceptance rate on this model is listed under Not measured.
• BAAI/AREX-2 ships no mtp.* tensors, so the drafter has no MTP head of its own. The launcher reads the 15 mtp.* tensors of Qwen/Qwen3.8-27B from that model's pinned snapshot (revision 1d4bf0f2ff6012fd82039f2fa52739d0dd7c60c0, file model-00018-of-00018.safetensors, downloaded alone on the first start). They are added at each start to model-dir/ as one more weight file, model-mtp-from-base.safetensors (graft.json names them). The head was trained on Qwen/Qwen3.8-27B, so how well it suits BAAI/AREX-2 is shown only by the rows below.

Intended use

BAAI/AREX-2 is a reasoning model for long, multi-step work. Its authors describe it as built for iterative problem solving: it proposes a solution, measures it, reflects on the result and revises it over several rounds. They name long-horizon agents, machine-learning engineering, algorithmic coding and tool-augmented deep research as the intended uses. This package serves it on 2 Blackhole chips for those uses, through the OpenAI-compatible chat endpoint.

It is not a quick-answer chat model. What that means in practice:

• It thinks before it answers, and the thinking is long. The chat template turns thinking on by default at its highest reasoning effort. The server separates the thinking into a reasoning_content field and puts the answer in content. Set max_tokens high. In the benchmark runs below, many replies that hit the lower limit came back empty, because the whole budget went on thinking.
• It is meant to call tools. The server parses tool calls into the standard tool_calls field. In the speed runs, the two tool-use prompts each returned a well-formed call (run_tests, read_file) in place of a text answer, which is what an agent loop expects.
• It works best where feedback exists. The authors trained it on tasks with checkable results (test scores, logs, timings) and its strengths show when a loop feeds those results back to it.
• Text only. The model is multimodal upstream, but this package skips the vision tower. It does not accept images or video.

Reasoning effort and thinking can be set per request through chat_template_kwargs (the model's template accepts reasoning_effort of xhigh, medium or low, and enable_thinking: false). Only the default, xhigh with thinking on, was measured here. The other settings are taken from the chat template and were not tested on this server.

The server names the model by the path of the directory it serves and does not accept BAAI/AREX-2 as a name, so ask it for its name first:

curl -s http://localhost:8000/v1/models

Put the id it returns in "model" below. Example request with a tool:

curl http://localhost:8000/v1/chat/completions -H 'content-type: application/json' -d '{
  "model": "<the id from /v1/models>",
  "messages": [{"role": "user", "content": "Run the tests under tests/utils and tell me which fail."}],
  "tools": [{"type": "function", "function": {"name": "run_tests", "description": "Run pytest on a path",
             "parameters": {"type": "object", "properties": {"path": {"type": "string"}}, "required": ["path"]}}}],
  "max_tokens": 32768
}'

The port is vLLM's default (8000) unless you pass --port after the bundle id. Long-running agent use, long contexts and several requests at once were not measured here.

What it looks like

A Gradio page, tt-arex2-agent, runs this package's server in a research loop and shows every step. The screenshots below come from the real model served by this package on 2 chips. Each is one run. They show what the page does with the model and are not benchmarks.

In the improvement lab the model wrote an edit-distance function, tested it several times with a Python tool, then submitted it. The lab ran hidden tests and a large input and gave it a full score. Plain chat on the same task ran out of the page's per-reply token limit while thinking and wrote no code. That limit is a page setting and says nothing about the model.

In a research mission the model searched, read two pages, used the calculator and answered with sources, and the page shows the text it read for each. Plain chat reached the same years and gave its own dates, which differ from the dates in the sources the page shows. The page's claim check matches words and numbers literally, so it can mark a correct answer "not backed" when the number is not written out in the retrieved text. The page's README shows an example.

Expected performance

Every number is labelled. A measured number names its evidence files; the first file holds the value. The files are in this repository under evidence/, at the paths shown, with the run's directory, the home directory and the host name replaced by <RUN_DIR>, <HOME> and <HOST>. TODO means not measured.

Number	Value	Label	Evidence
top1 agreement with the CPU reference, stage 2 (2 chips, bundle qwen3.8-27b-dflash2-p300)	0.96875 fraction	measured	stages/2/result.json, stages/2/evidence/swap-check.json, stages/2/evidence/server.log
server ready after start, stage 2 (empty tensor cache)	160.1 s	measured	stages/2/result.json, stages/2/evidence/swap-check.json, stages/2/evidence/server.log
top1 agreement with the CPU reference, this package (stage 7)	0.96875 fraction	measured	stages/7/verify/evidence/verify.json, stages/7/verify/evidence/server.log
server ready after start, this package (stage 7, fresh install, empty tensor cache)	334.2 s	measured	stages/7/verify/evidence/verify.json, stages/7/verify/evidence/server.log
GSM8K score, this model	0.9378 fraction	measured	stages/7/benchmark/benchmark.json
GSM8K replies cut off at the token limit, this model	0 items	measured	stages/7/benchmark/benchmark.json
GSM8K score, nearest model, same conditions	0.956 fraction	measured	stages/7/benchmark/benchmark.json
GSM8K paired sign test p, this model against the nearest model	0.00631 p-value	measured	stages/7/benchmark/benchmark.json
IFEval score, this model	0.8928 fraction	measured	stages/7/benchmark/benchmark.json
IFEval replies cut off at the token limit, this model	18 items	measured	stages/7/benchmark/benchmark.json
IFEval score, nearest model, same conditions	0.8595 fraction	measured	stages/7/benchmark/benchmark.json
IFEval paired sign test p, this model against the nearest model	0.06297 p-value	measured	stages/7/benchmark/benchmark.json
decode speed, this package (drafter on, grafted head), one request at a time, median of four prompts	65.79 tokens/s	measured	stages/7/benchmark/speed.json
decode speed, same model with the drafter off and sampling on the host, same prompts	19.88 tokens/s	measured	stages/7/benchmark/speed.json
decode speed, nearest model served the plain way, same prompts	16.73 tokens/s	measured	stages/7/benchmark/speed.json
time to first token, this package, median of four prompts	0.19 s	measured	stages/7/benchmark/speed.json
time to first token, same model with the drafter off, same prompts	0.32 s	measured	stages/7/benchmark/speed.json
GSM8K score, same model with the drafter off	0.9356 fraction	measured	stages/7/benchmark/graft-comparison.json
GSM8K paired sign test p, this package against the drafter-off run	0.7709 p-value	measured	stages/7/benchmark/graft-comparison.json
IFEval score, same model with the drafter off (8192-token first pass, empty replies asked again at 32768)	0.8484 fraction	measured	stages/7/benchmark/graft-comparison.json
IFEval paired sign test p, this package against the drafter-off run	0.009683 p-value	measured	stages/7/benchmark/graft-comparison.json

Speed and benchmark conditions

The speed rows come from one request at a time, greedy decoding, three repetitions of each of four short prompts, on the same 2-chip bundle. The same prompts ran against BAAI/AREX-2 with the drafter off and sampling on the host (an earlier configuration of this package) and against Qwen/Qwen3.8-27B served that way. The per-prompt values are in evidence/stages/7/benchmark/speed.json. The prompts are short. Longer contexts and several requests at once were not measured.

The accuracy rows have different conditions, and the differences change what they show:

• Both models ran with thinking on, as the chat template sets it, at temperature 0.
• GSM8K covers all 1,319 items, 5-shot, with the flexible-extract filter and a 32,768-token limit. The harness's default stop string Question: cuts a reasoning model off when it repeats that word while thinking, so these runs stop only on </s>. The nearest model and the drafter-off run used the same items and settings.
• IFEval is the prompt-level strict score over all 541 prompts. This package ran every prompt at a 32,768-token limit. The nearest model and the drafter-off run used an 8,192-token limit, and only their empty replies were asked again at 32,768. About 20 more of their replies were cut off at the lower limit and stayed scored as they were. The IFEval difference between this package and the drafter-off run may come from that limit and not from the drafter.
• On GSM8K this package and the drafter-off run cannot be told apart. The nearest model scores higher than this package on GSM8K (paired sign test in the table). On IFEval the paired difference against the nearest model is not clear.
• None of the authors' reported benchmarks (research, agent and coding tasks) was run here. Tool calling and long-running agent use were not tested.

Reported by the authors

These scores come from the BAAI/AREX-2 model card on Hugging Face (revision d4e3502f92d9e889031c2d04e387ab2eb3520268), which says the results follow the protocols in the AREX-2 paper. They were not measured here: the authors' protocol, tools and judge differ from the checks in the table above, and nothing in this repository reproduces them.

Benchmark	Reported	Note
Frontier-CS (188-task Agent Track)	70.7
MLE-Lite	81.8	Any Medal, averaged over three seeds
BrowseComp	84.0
HLE	52.6	text-only subset
GAIA	92.2
DeepSearchQA	93.8

Sampling

This package serves requests with the DFlash2 drafter on, and the drafter's decoder supports greedy decoding only. A request that sets temperature above 0 is accepted, but only the first generated token is sampled. Every later token is greedy. top_p, top_k and seed act on that first token only, min_p and the penalties have no effect, and logprobs is refused with HTTP 400. Repeated requests with different seeds return the same text apart from the first token.

Measured on the base bundle episod/qwen3.8-27b-dflash2-p300 with Qwen/Qwen3.8-27B weights, on one 2-chip board in one session, with 96-token requests and three probes of each kind. This package runs the same decoder code and was not measured separately for sampling.

	Drafter on (as shipped here)	Drafter off, host sampling
Decode speed, greedy request	about 60 tokens/s	about 18 tokens/s
Decode speed, temperature 1.0 and top-p 0.9	about 60 tokens/s	about 17 tokens/s
Sampled text equal to the greedy text	6 of 6 requests	0 of 6 requests
Five seeds at temperature 1.0	1 distinct text	5 distinct texts
logprobs	refused	accepted

If your use needs sampled output, serve with the drafter off. In run.sh, set export QWEN36_DRAFTER="", delete the DFLASH_* exports, and remove "sample_on_device_mode": "decode_only" from --additional-config. The Clef, RAG-Gate and CJK-Trim packages ship in that configuration. It decodes about 3.3 times slower on 2 chips. The measurements and the code reading are in the tt-orchard repository: issue 107 and pull request 109.

Limitations

• Tested on 2 chips (mesh P150x2) only; other configurations are not claimed by this bundle.
• Accuracy was compared with the CPU reference on one short fixed prompt only.
• The drafter's MTP head is the nearest model's (Qwen/Qwen3.8-27B), added to BAAI/AREX-2 at start. How often the drafter's guesses are accepted on this model was not measured.
• The stage 2 and stage 4 numbers were measured without the graft (the drafter off and sampling on the host). The stage 7 rows were measured with this package's settings.
• Default sampling settings are those of Qwen/Qwen3.8-27B, because BAAI/AREX-2 ships no generation_config.json. With the drafter on, as this package ships, the settings of a request affect only its first token (see Sampling).

Risks and safety considerations

The check above does not cover long outputs, tool calling or safety behaviour. Output can differ from BAAI/AREX-2 run on a CPU or GPU in ways it does not show.

Not measured

• the drafter's acceptance rate on this model
• a download of the weights through tt-model pull, and a boot of the package from the Hub

Boot check

Stage 7 of the run installed this bundle, served it on a leased board and compared its tokens with the CPU reference. The Expected performance table has the result.

How to serve

tt-model serve episod/arex-2-p300