gpt-oss-20b-p150 on Tenstorrent p150 and p150x2 and p150x4

Serves openai/gpt-oss-20b by OpenAI at revision 6cee5e81. See the original model card for training and evaluation details.

gpt-oss-20b, OpenAI's 21B-parameter mixture-of-experts reasoning model (3.6B active parameters), served with vLLM for chat, reasoning and function calling.

Runs on Tenstorrent p150 or p150x2 or p150x4 — see the serving profiles below.

Packaged and published with tt-model-manager 0.1.0 (manifest schema 5.1).

At a glance

Hardware	p150, p150x2, p150x4
Context	131,072 tokens
License	apache-2.0

Intended use

Chat, reasoning and function calling in English through an OpenAI-compatible API.

Prerequisites

• The Tenstorrent CLI, tt: uv tool install tenstorrent (or use tt-model alone, see the Quickstart)
• Docker
• Tenstorrent hardware: p150 or p150x2 or p150x4

Quickstart

tt serve tt-hous/gpt-oss-20b-p150

tt serve downloads the Docker image and the openai/gpt-oss-20b at 6cee5e81ee83917806bbde320786a8fb61efebee weights (into your HF cache; they are not in the image), then starts an OpenAI-compatible server on port 20000 (or the next free port, if that one is busy). The first start compiles kernels for your device, which takes several minutes; the server is ready when it logs Application startup complete.

Without tt-cli: tt-model serve tt-hous/gpt-oss-20b-p150.

Pick a profile for the traffic. The -throughput profiles prefill in 32,768-token steps and pause decode for every request while a long prompt prefills: use them for batch work. The p150, p150x2 and p150x4 profiles prefill in 8,192-token steps with 8 decode steps between them: use them for chat, agents and mixed prompt lengths. Pick the largest mesh your box has. The p150x2 profiles need devices 0 and 1 to be the two ASICs of one P300C board.

Serving profiles

Select a different serving profile using --profile.

profile	hardware	mesh	max_num_seqs	max_model_len	description
p150	p150	P150	32	131072	Interactive profile; full 131072-token context, up to 32 sequences.
p150-throughput	p150	P150	32	131072	Throughput profile; full 131072-token context, up to 32 sequences.
p150x2	p150x2	P150x2	32	131072	Interactive profile; full 131072-token context, up to 32 sequences.
p150x2-throughput (default)	p150x2	P150x2	32	131072	Throughput profile; full 131072-token context, up to 32 sequences.
p150x4	p150x4	P150x4	32	131072	Interactive profile; full 131072-token context, up to 32 sequences.
p150x4-throughput	p150x4	P150x4	32	131072	Throughput profile; full 131072-token context, up to 32 sequences.

Capabilities

The server speaks the OpenAI API at http://127.0.0.1:20000/v1 (or whichever port your serve command reported; chat completions, completions, and /v1/models). Pass "model": "openai/gpt-oss-20b" — the weights id, not this package's name.

Tool calling is enabled (--tool-call-parser openai): a request that passes tools comes back with tool_calls and finish_reason: tool_calls.

Reasoning comes back separately from the answer, in message.reasoning. max_tokens covers both. For a chat completion with max_tokens of 256 or more, the server keeps clamp(max_tokens // 8, 128, 1024) tokens for the answer, so a request that runs out of budget while reasoning still returns an answer (finish_reason: "length" with non-null content). reasoning_effort (low, medium, high) sets how much the model reasons, and usage.completion_tokens_details.reasoning_tokens reports what it used. Set VLLM_HARMONY_FINAL_RESERVE_TOKENS=-1 in the container to turn the reserve off.

Greedy requests, and sampled requests with top_k of 32 or lower, are sampled on the device.

Expected performance

eval	this package, P150	this package, P150x2	this package, P150x4	reference	prompts
IFEval, prompt-level strict	81.1%	80.6%	81.0%	60.4%	541
IFEval, prompt-level loose	82.6%	83.0%	83.4%	63.8%	541
IFEval, instruction-level strict / loose	86.0% / 87.1%	85.9% / 87.8%	86.1% / 87.6%	68.5% / 71.8%	834 instructions
GSM8K, exact match, flexible extract	86.4%	86.8%	86.1%	88.9%	1,319

Evals use lm-evaluation-harness 0.4.10.dev0 through the chat API: reasoning_effort medium, greedy, a 4,096-token output budget, seed 42, 32 concurrent requests, chat template applied, on the throughput profile of each mesh. GSM8K is zero-shot. Scores grade the final answer only, and every request returned a final answer on every mesh. The OpenAI model card publishes neither benchmark. The reference column is the frozebench run of openai/gpt-oss-20b with lm-eval, the chat template and a 4,096-token budget on 2025 Oct 17 (IFEval, GSM8K, the lm-eval default gsm8k task).

profile	ISL	OSL	concurrency	N	TTFT (ms)	prefill tok/s/u	decode tok/s/u	E2EL (s)
p150-throughput	128	128	1	1	209	613	48.5	2.83
p150-throughput	128	128	8	1	500	256	22.9	6.05
p150-throughput	128	128	32	1	1,085	118	16.8	8.62
p150-throughput	1,024	256	1	1	308	3,320	47.7	5.65
p150-throughput	1,024	256	8	1	1,962	522	20.2	14.59
p150-throughput	1,024	256	32	1	6,751	152	14.6	24.25
p150-throughput	4,096	256	1	1	816	5,019	47.2	6.22
p150-throughput	16,384	256	1	1	3,518	4,657	45.1	9.18
p150-throughput	32,768	256	1	1	7,803	4,200	42.5	13.80
p150-throughput	65,536	256	1	1	18,648	3,514	38.2	25.32
p150x2-throughput	128	128	1	1	126	1,019	73.7	1.85
p150x2-throughput	128	128	8	1	423	303	40.8	3.53
p150x2-throughput	128	128	32	1	883	145	31.5	4.92
p150x2-throughput	1,024	256	1	1	347	2,954	72.9	3.85
p150x2-throughput	1,024	256	8	1	1,567	653	37.5	8.37
p150x2-throughput	1,024	256	32	1	4,942	207	27.8	14.10
p150x2-throughput	4,096	256	1	1	824	4,969	71.9	4.37
p150x2-throughput	16,384	256	1	1	2,835	5,779	70.3	6.46
p150x2-throughput	32,768	256	1	1	5,551	5,904	68.3	9.28
p150x2-throughput	65,536	256	1	1	12,043	5,442	64.6	15.99
p150x4-throughput	128	128	1	1	87	1,477	88.8	1.52
p150x4-throughput	128	128	8	1	310	413	57.1	2.54
p150x4-throughput	128	128	32	1	589	217	46.2	3.34
p150x4-throughput	1,024	256	1	1	275	3,717	87.8	3.18
p150x4-throughput	1,024	256	8	1	1,064	962	52.7	5.91
p150x4-throughput	1,024	256	32	1	3,140	326	41.6	9.27
p150x4-throughput	4,096	256	1	1	580	7,068	86.8	3.52
p150x4-throughput	16,384	256	1	1	2,416	6,781	84.7	5.43
p150x4-throughput	32,768	256	1	1	3,698	8,860	82.1	6.80
p150x4-throughput	65,536	256	1	1	7,357	8,908	77.3	10.65

Means over one run per row on a QuietBox 2 (two P300C boards), against this package's image, with a closed-loop latency client: random-token prompts of exactly ISL tokens sent to /v1/completions and streamed, output pinned to OSL tokens, greedy, 4 requests per user up to 4,096 input tokens, 2 per user up to 16,384 and 1 per user above, one unmeasured warmup request per input length. The rows are selected from a sweep at 1 to 32 concurrent users. The P150 and P150x2 rows were measured at the same time on separate boards. Random prompts never hit the prefix cache. Prefill tok/s/u is ISL divided by TTFT, so above concurrency 1 it includes queueing time.

Limitations

• Speed was measured on the three throughput profiles only; the p150, p150x2 and p150x4 profiles were not benchmarked. Tested on one QuietBox 2 only: four separate P150 boards were not tested, and eight-chip meshes are not offered.
• Not for workloads that need OpenAI's built-in Python or browser tools: none is provided. Function calling uses the OpenAI tool format. The model sometimes emits malformed tool-call JSON, so agent scaffolds need a retry.
• Capacity across all requests is 264,192 tokens of context on P150, 526,336 on P150x2 and 1,054,720 on P150x4. Further requests wait for admission, and the wait is part of TTFT.
• max_tokens covers reasoning and answer. Below 256 tokens, or for a request served without the answer reserve (n above 1, logprobs, ignore_eos, a required tool call), a response can end with finish_reason: "length" and null content.
• One request with min_p, logit_bias, bad_words, allowed_token_ids, min_tokens, or a non-zero temperature with top_k above 32 or unset, moves the whole batch to host sampling and slows decode for every user until it finishes.
• No run of the same prompts against a BF16 reference of the checkpoint was measured.

Feedback

Questions or problems with this package: open a discussion at https://huggingface.co/tt-hous/gpt-oss-20b-p150/discussions — that is what reaches its author. A problem with the tt tooling itself: tt report issue (collects your environment and opens a prefilled issue against tenstorrent/tt-cli). Product feedback: support@tenstorrent.com.

Provenance

The exact sources the image was built from — code/ in this repo is byte-identical to the model code inside the image:

component	built from
tt-metal	a local checkout — commit not published (dirty tree — the image includes uncommitted changes)
vLLM	a local checkout — commit not published
vllm-tt-plugin	a local checkout — commit not published
code/ digest	26d8a7e6b40424a5 (sha256, first 16 hex digits)
built	2026-10-09T16:52:14.188884+00:00 by tt-model 0.1.0