Instructions to use BAAI/AREX-2 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use BAAI/AREX-2 with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("image-text-to-text", model="BAAI/AREX-2") messages = [ { "role": "user", "content": [ {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"}, {"type": "text", "text": "What animal is on the candy?"} ] }, ] pipe(text=messages)# pip install -U transformers accelerate # Load model directly from transformers import AutoProcessor, AutoModelForMultimodalLM processor = AutoProcessor.from_pretrained("BAAI/AREX-2") model = AutoModelForMultimodalLM.from_pretrained("BAAI/AREX-2", device_map="auto") messages = [ { "role": "user", "content": [ {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"}, {"type": "text", "text": "What animal is on the candy?"} ] }, ] inputs = processor.apply_chat_template( messages, add_generation_prompt=True, tokenize=True, return_dict=True, return_tensors="pt", ).to(model.device) outputs = model.generate(**inputs, max_new_tokens=256) print(processor.decode(outputs[0][inputs["input_ids"].shape[-1]:])) - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use BAAI/AREX-2 with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "BAAI/AREX-2" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "BAAI/AREX-2", "messages": [ { "role": "user", "content": [ { "type": "text", "text": "Describe this image in one sentence." }, { "type": "image_url", "image_url": { "url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg" } } ] } ] }'Use Docker
docker model run hf.co/BAAI/AREX-2
- SGLang
How to use BAAI/AREX-2 with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "BAAI/AREX-2" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "BAAI/AREX-2", "messages": [ { "role": "user", "content": [ { "type": "text", "text": "Describe this image in one sentence." }, { "type": "image_url", "image_url": { "url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg" } } ] } ] }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "BAAI/AREX-2" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "BAAI/AREX-2", "messages": [ { "role": "user", "content": [ { "type": "text", "text": "Describe this image in one sentence." }, { "type": "image_url", "image_url": { "url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg" } } ] } ] }' - Docker Model Runner
How to use BAAI/AREX-2 with Docker Model Runner:
docker model run hf.co/BAAI/AREX-2
ThinkingCap vs Arex-2 Bench
ThinkingCap vs AREX-2 β standard benchmark report
Date: 2026-11-17/18. Both models are Qwen3.8-27B-based, INT4 (W4A16), served by HyperQwen (batch mode), one RTX 3090 each.
- ThinkingCap: ThinkingCap-Qwen3.8-27B-W4A16-Merlin (GPU 3, 64 seqs, 150k context)
- AREX-2: BAAI/AREX-2 27B INT4-W4A16 (GPU 1, 16 seqs, 40k context)
Tools: EvalPlus 0.3.1 (HumanEval+), lm-eval 0.4.13 (IFEval), bfcl-eval 2026.3.23 (BFCL v4, non multi-turn).
Setup: greedy decoding (temperature 0 / 0.001), max 16,000 output tokens, 16 parallel requests (EvalPlus: 4 shards), one run.
1. Quality
| Benchmark | Metric | ThinkingCap | AREX-2 |
|---|---|---|---|
| HumanEval+ | pass@1 (base tests) | 97.0% | 97.0% |
| HumanEval+ | pass@1 (base + extra tests) | 91.5% | 91.5% |
| IFEval | prompt-level strict | 86.5% | 75.4% |
| IFEval | prompt-level loose | 89.1% | 76.3% |
| IFEval | instruction-level strict | 89.0% | 78.5% |
| IFEval | instruction-level loose | 90.6% | 79.1% |
| IFEval | empty answers (token cap hit) | 27 / 541 (5.0%) | 94 / 541 (17.4%) |
| BFCL v4 | non-live overall (AST) | 84.67% | 87.67% |
| BFCL v4 | live overall (AST) | 82.68% | 82.53% |
BFCL non-live detail
| Category | ThinkingCap | AREX-2 |
|---|---|---|
| Simple (avg of Python/Java/JS) | 77.17% | 75.67% |
| Python simple | 96.50% | 95.00% |
| Java simple | 63.00% | 62.00% |
| JavaScript simple | 72.00% | 70.00% |
| Multiple | 95.50% | 97.00% |
| Parallel | 81.00% | 92.50% |
| Parallel multiple | 85.00% | 85.50% |
| Irrelevance detection | 75.00% | 70.83% |
BFCL live detail
| Category | ThinkingCap | AREX-2 |
|---|---|---|
| Python simple | 87.60% | 87.21% |
| Python multiple | 81.58% | 81.67% |
| Python parallel | 87.50% | 75.00% |
| Python parallel multiple | 75.00% | 75.00% |
| Irrelevance detection | 75.90% | 49.10% |
| Relevance detection | 81.25% | 100.00% |
2. Speed
| Measure | ThinkingCap | AREX-2 |
|---|---|---|
| IFEval wall time (541 prompts, 16 parallel) | 29.6 min (1,776 s) | 88.5 min (5,307 s) |
| BFCL wall time (3,641 requests, 16 threads) | ~34 min | ~35 min |
| Pagoda single-file 3D prompt, total time | 1,581 s (26.4 min) | 2,071 s (34.5 min) |
| Pagoda output tokens (reasoning + answer) | 74,084 (63,758 reasoning) | 87,145 (78,746 reasoning) |
| Pagoda single-stream speed | 46.8 tok/s | 42.1 tok/s |
| Pagoda time to first token | 0.17 s | 0.15 s |
| Aggregate throughput, IFEval, 16 parallel | not logged | ~545 tok/s |
Pagoda: both finished with finish_reason=stop, ThinkingCap first (about 490 s earlier). Sampling there was temperature 0.6, top_p 0.95, max 32,000 tokens requested (streams ran to 74-87k, so see caveat below).
HumanEval+ wall time is not reported: the run was relaunched several times (timeouts, dataset download race), so the timings are not comparable.
3. Reading the results
- Code generation: identical (HumanEval+ 97.0 / 91.5).
- Instruction following: ThinkingCap leads by ~11-13 points. Part of the gap is AREX-2 reasoning past the 16k-token cap (17% empty answers vs 5%), so the real gap at a higher cap would likely be smaller.
- Tool calling: AREX-2 is ahead on non-live (+3.0, mostly parallel calls: 92.5 vs 81.0). Live is a tie overall, but AREX-2 is much weaker at irrelevance detection (49.1% vs 75.9%: it calls a tool when it should not) and perfect on relevance (100%).
- Speed: per-stream decoding is similar (47 vs 42 tok/s); AREX-2 is slower end-to-end because it produces ~18% more reasoning tokens, and ThinkingCap ran with a larger batch budget.
4. Limits and fairness notes
- One run per benchmark, greedy decoding: no variance estimate. Differences of 1-2 points are noise (IFEval stderr about 1.8 points).
- 16,000-token cap: empty answers (reasoning not finished) count as failures. This penalizes the model with longer reasoning.
- Different server settings: ThinkingCap 64 seqs / 150k context, AREX-2 16 seqs / 40k context.
- Local patches to EvalPlus:
max_new_tokens768 β 16,000, request alarm 900 s,Nonecontent treated as empty string. - BFCL ran through the
Qwen/Qwen3-32B-FChandler (hence the label "Qwen3-32B (FC)" in BFCL tables); the models actually tested are the ones above. Only non multi-turn categories were run (13 categories). - All GPUs report PCIe Gen1 x1; this affects load time, not generation speed measured here.