Instructions to use Spreadsheet-RL/Spreadsheet-RL-8B with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use Spreadsheet-RL/Spreadsheet-RL-8B with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="Spreadsheet-RL/Spreadsheet-RL-8B")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("Spreadsheet-RL/Spreadsheet-RL-8B")
model = AutoModelForCausalLM.from_pretrained("Spreadsheet-RL/Spreadsheet-RL-8B", device_map="auto")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use Spreadsheet-RL/Spreadsheet-RL-8B with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "Spreadsheet-RL/Spreadsheet-RL-8B"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Spreadsheet-RL/Spreadsheet-RL-8B",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/Spreadsheet-RL/Spreadsheet-RL-8B

SGLang

How to use Spreadsheet-RL/Spreadsheet-RL-8B with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "Spreadsheet-RL/Spreadsheet-RL-8B" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Spreadsheet-RL/Spreadsheet-RL-8B",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "Spreadsheet-RL/Spreadsheet-RL-8B" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Spreadsheet-RL/Spreadsheet-RL-8B",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use Spreadsheet-RL/Spreadsheet-RL-8B with Docker Model Runner:
```
docker model run hf.co/Spreadsheet-RL/Spreadsheet-RL-8B
```

Spreadsheet-RL-8B

Spreadsheet-RL logo

Project Page | Paper | Dataset | Code

Spreadsheet-RL-8B is the RL-trained 8B spreadsheet agent checkpoint from Spreadsheet-RL: Advancing Large Language Model Agents on Realistic Spreadsheet Tasks via Reinforcement Learning. It starts from Qwen/Qwen3-8B and is post-trained with outcome-based reinforcement learning in Spreadsheet Gym, a multi-turn Microsoft Excel environment with spreadsheet-native tools, sandboxed code execution, and Excel-based recalculation rewards.

This checkpoint is intended to be used with the Spreadsheet-RL agent harness and tool environment. Loading it as a plain chat model can be useful for inspection, but it will not reproduce the reported results without Spreadsheet Gym, the tool set, and the reward/evaluation pipeline.

News

2026-08-01: Released the Spreadsheet-RL-8B model checkpoint on Hugging Face at Spreadsheet-RL/Spreadsheet-RL-8B.

Model Details

Field	Value
Base model	`Qwen/Qwen3-8B`
Released checkpoint	Step 35, the best observed checkpoint
Training method	GRPO with outcome-based rewards
Environment	Spreadsheet Gym with Microsoft Excel 365, spreadsheet-native tools, SandboxFusion code execution, and async Excel recalculation/reward service
Training data	Spreadsheet-RL training split: 5,928 filtered ExcelForum tasks
Evaluation	SpreadsheetBench
License	Apache-2.0, following the base model license

Training Configuration

For full details, please see the paper. The released 8B checkpoint uses:

Hyperparameter	Value
Algorithm	GRPO; KL-regularized against a frozen reference model; 1 PPO epoch
Released checkpoint	35 RL steps
Prompt/response limits	8,192 / 24,576 tokens
Rollout sampling	temperature 0.6; top-p 0.95; top-k 20
Batching	128 prompts/step; 8 rollouts/prompt; 1,024 rollouts/step
Training sampler	Difficulty-aware mixture with difficulty level 5 excluded
Optimizer	AdamW; learning rate 2e-6; weight decay 0.01; constant schedule
KL loss	low-var KL; coefficient 0.001
Actor update batching	mini-batch 64; dynamic batch sizing enabled
Hardware	2 nodes x 4 GPUs

Results

Spreadsheet-RL improves Qwen3-8B through spreadsheet-native interaction design, comprehensive tool access, and RL post-training.

Benchmark	Base	+ Agent Harness & Full Tools	Spreadsheet-RL-8B (step 35)
SpreadsheetBench Pass@1	15.9	16.7	22.3

The released step-35 checkpoint is the best observed checkpoint and improves the full-harness pre-RL result by 5.6 percentage points.

The corresponding 4B release starts from Qwen/Qwen3-4B-Thinking-2507. Its 2507 checkpoint postdates the original Qwen3 series, including Qwen/Qwen3-8B, so model size and base-checkpoint vintage are confounded. Under the full Spreadsheet-RL harness, the 8B model trails the 4B model both before RL (16.7 versus 19.3) and after RL (22.3 versus 23.4); this should not be interpreted as a clean size-only comparison. Instead, the 8B gain from 16.7 to 22.3 shows that Spreadsheet-RL remains effective at a larger model scale, and we release the post-RL checkpoint to make that scaling result reproducible.

Usage

Install the standard Transformers stack and load the checkpoint:

from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "Spreadsheet-RL/Spreadsheet-RL-8B"

tokenizer = AutoTokenizer.from_pretrained(model_id, trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(
    model_id,
    torch_dtype="auto",
    device_map="auto",
    trust_remote_code=True,
)

For task evaluation and agent rollouts, use the full Spreadsheet-RL codebase with the released dataset and Spreadsheet Gym:

hf download Spreadsheet-RL/Spreadsheet-RL --repo-type dataset --local-dir data
git clone https://github.com/Spreadsheet-RL/Spreadsheet-RL.git

The default training/evaluation harness is maintained in the code repository under configs/, scripts/, reward/, and verl/.

Citation

@misc{chi2026spreadsheetrl,
  title         = {Spreadsheet-RL: Advancing Large Language Model Agents on Realistic Spreadsheet Tasks via Reinforcement Learning},
  author        = {Banghao Chi and Yining Xie and Mingyuan Wu and Jingcheng Yang and Jize Jiang and Zhaoheng Li and Shengyi Qian and Minjia Zhang and Klara Nahrstedt and Rui Hou and Xiangjun Fan and Hanchao Yu},
  year          = {2026},
  eprint        = {2605.22642},
  archivePrefix = {arXiv},
  primaryClass  = {cs.AI},
  doi           = {10.48550/arXiv.2605.22642},
  url           = {https://arxiv.org/abs/2605.22642}
}