Lernex/Metis-1.6-Praxis-Base

Technical report  ·  Metis-1.6  ·  Lernex  ·  X

Metis-1.6 Praxis Base

Praxis Base is the pretrained base of Metis-1.6: a 2.94B-parameter mixture of experts that interleaves Mamba-2 and attention, runs every token through its stack twice, and touches 0.46B parameters on each pass. I trained it from scratch on 0.82 trillion tokens, none of them question-answer, instruction or chat data, and it reads 131,072 tokens of context. The base models it is measured against below saw 1 to 36 trillion, where their labs say. It continues text; for a model that follows instructions, use Metis-1.6 Praxis.

Benchmarks

Praxis beside 16 base models from other labs, every one run on this server through the same harness, prompts and scorers, and ordered by Average, highest first. The picker above the table narrows it to the models you choose; bold is always the best score among the models in view. … is queued and - does not apply. Updated October 21, 2026, 00:47 UTC; the table fills in as runs finish.

Metis-1.6 Praxis Basevsall 16 modelsOuro 2.6BQwen3.5 4BQwen3 4BNanbeige4.2 3BMinistral 3 3BGranite 4.1 3BFalcon-H1 3BOuro 1.4BGemma 4 E4BSmolLM3 3BLFM2.5 2.6BGemma 3 4BQwen3.5 2BLlama 3.2 3BGemma 4 E2BQwen3.5 0.8B

Pick any to compare with Praxis; pick none to see every model. Best first by Average.

Ouro 2.6B2.67B · avg 43.15Qwen3.5 4B4.21B · avg 42.93Qwen3 4B4.02B · avg 42.88Nanbeige4.2 3B4.17B · avg 41.23Ministral 3 3B3.43B · avg 40.59Granite 4.1 3B3.40B · avg 39.99Falcon-H1 3B3.15B · avg 39.81Ouro 1.4B1.43B · avg 38.78Gemma 4 E4B7.52B · avg 38.55SmolLM3 3B3.08B · avg 37.17LFM2.5 2.6B2.70B · avg 35.93Gemma 3 4B3.88B · avg 34.37Qwen3.5 2B1.88B · avg 34.17Llama 3.2 3B3.21B · avg 30.92Gemma 4 E2B4.65B · avg 30.53Qwen3.5 0.8B0.752B · avg 27.98

	Metis-1.6 Praxis Base	Ouro 2.6B	Qwen3.5 4B	Qwen3 4B	Nanbeige4.2 3B	Ministral 3 3B	Granite 4.1 3B	Falcon-H1 3B	Ouro 1.4B	Gemma 4 E4B	SmolLM3 3B	LFM2.5 2.6B	Gemma 3 4B	Qwen3.5 2B	Llama 3.2 3B	Gemma 4 E2B	Qwen3.5 0.8B
# Params	2.94B	2.67B	4.21B	4.02B	4.17B	3.43B	3.40B	3.15B	1.43B	7.52B	3.08B	2.70B	3.88B	1.88B	3.21B	4.65B	0.752B
# Activated params, per pass	0.464B	2.47B	3.57B	3.63B	3.15B	3.03B	3.15B	2.81B	1.23B	4.03B	2.81B	2.44B	3.21B	1.37B	2.82B	1.90B	0.498B
# Activated params, total	0.94B	9.87B	3.57B	3.63B	6.30B	3.03B	3.15B	2.81B	4.93B	4.03B	2.81B	2.44B	3.21B	1.37B	2.82B	1.90B	0.498B
Context	128K	64K	256K	32K	128K	256K	128K	128K	64K	128K	64K	125K	128K	256K	128K	128K	256K
Training tokens	0.82T	7.7T	undisclosed	36T	28T	1-3T	14.9T	2.5T	7.7T	undisclosed	11.2T	34T	4T	undisclosed	9T	undisclosed	undisclosed
Q&A or instruction data in pretraining	none	yes, in ChatML	undisclosed	yes	yes	undisclosed	yes	yes	yes, in ChatML	undisclosed	yes	undisclosed	undisclosed	undisclosed	undisclosed	undisclosed	undisclosed
Obeys a bare instruction	5.5%	5.5%	36.0%	81.5%	86.0%	28.5%	72.0%	14.5%	2.0%	4.0%	1.0%	34.5%	9.0%	31.0%	8.0%	3.0%	20.5%
Overall
Average	24.66	43.15	42.93	42.88	41.23	40.59	39.99	39.81	38.78	38.55	37.17	35.93	34.37	34.17	30.92	30.53	27.98
Average, with long context	27.72	-	45.57	-	-	-	-	42.42	-	41.54	-	-	37.20	38.72	-	33.57	32.64
Frontier reasoning and knowledge
Humanity's Last Exam	3.89	4.23	3.34	3.99	3.48	4.17	4.36	3.94	4.41	4.22	4.40	5.56	4.36	3.80	4.49	3.80	3.99
HLE-Diamond	3.43	3.32	2.88	5.08	1.65	4.12	4.81	3.98	4.00	3.71	3.98	5.08	4.67	3.30	3.71	4.12	3.57
HLE-Verified (Gold)	4.00	4.87	4.00	4.35	4.70	4.87	5.22	3.65	4.70	5.04	4.00	6.26	5.74	3.65	4.17	5.22	4.17
GPQA	27.90	26.34	25.67	22.99	25.89	28.57	24.78	23.44	25.89	26.12	25.45	22.32	26.12	24.33	25.45	26.56	27.01
GPQA Diamond	24.24	28.79	24.24	25.25	28.28	29.29	25.76	20.20	27.27	27.27	29.80	25.25	28.28	23.74	26.26	29.80	25.76
SuperGPQA	11.12	25.81	27.11	28.07	32.58	25.15	21.37	24.49	22.32	24.71	23.40	22.85	17.41	21.43	19.25	16.49	15.98
MMLU-Pro	14.19	48.84	48.62	45.74	45.84	40.92	33.36	40.63	41.43	38.19	33.18	37.01	27.92	36.61	24.30	24.74	25.05
BIG-Bench Extra Hard	1.66	7.01	2.17	7.92	12.06	7.48	7.85	4.18	3.89	2.19	6.28	7.63	3.21	4.09	2.85	3.69	1.84
ARC-AGI-2	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.00
SimpleQA Verified	2.70	3.00	4.80	3.20	4.30	3.00	2.40	2.50	1.50	4.00	4.00	3.60	3.10	2.60	4.20	2.30	1.80
Math
AIME 2026	0.00	0.00	0.00	3.33	0.00	0.00	0.00	0.00	6.67	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.00
HMMT Feb 2026	0.00	3.03	0.00	3.03	3.03	0.00	3.03	3.03	3.03	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.00
AIME 2025	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.00	3.33	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.00
MATH-500	16.20	40.80	44.00	48.20	20.60	36.00	39.00	43.40	1.80	20.60	38.80	41.20	24.00	32.80	6.00	9.20	20.40
Omni-MATH	5.83	15.27	14.79	13.30	12.08	11.68	10.48	12.83	21.41	6.14	10.34	12.40	6.50	9.24	3.50	3.75	7.25
GSM8K	22.59	82.34	80.89	72.33	69.14	71.11	72.48	68.99	78.47	56.41	67.93	73.16	36.85	59.06	26.38	21.83	38.59
GSM8K-Platinum	23.57	85.28	81.97	75.52	79.57	71.63	72.87	71.88	79.40	57.24	69.48	77.50	39.62	63.03	27.05	23.24	39.70
MGSM	6.25	37.67	50.98	51.13	45.35	44.44	40.84	40.73	38.22	43.20	35.31	44.11	29.89	35.71	16.04	16.80	22.95
Coding
LiveCodeBench	0.00	21.59	16.59	15.44	24.73	16.04	16.32	16.43	17.42	13.68	12.97	2.09	7.42	4.56	7.86	4.73	1.15
HumanEval+	16.46	34.76	44.51	51.22	44.51	35.98	43.29	55.49	33.54	25.00	31.10	31.10	28.66	29.88	25.00	21.34	19.51
MBPP+	35.98	82.80	72.49	80.42	78.04	70.63	76.19	75.66	76.98	65.34	66.14	47.88	62.43	49.47	55.29	50.79	36.24
CRUXEval-O	12.49	33.74	30.32	28.66	31.52	25.42	27.06	27.97	28.09	24.29	22.04	18.14	21.24	20.45	18.52	16.05	13.01
CRUXEval-I	20.84	50.59	48.96	44.00	50.94	40.55	44.01	40.09	41.57	37.13	35.73	27.24	32.82	36.23	28.91	26.85	30.91
BigCodeBench	7.28	40.44	35.70	41.84	9.12	33.95	43.16	32.72	28.77	31.84	28.77	22.19	24.39	17.54	19.39	14.56	7.63
MultiPL-E	8.80	30.41	32.02	41.82	44.61	27.66	42.39	39.75	25.22	18.62	21.72	12.82	21.52	17.48	17.11	13.17	8.99
SciCode	0.00	0.00	0.00	0.00	0.00	0.00	1.54	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.00
Long context
RULER (4K)	75.54	91.69	87.17	92.41	92.91	88.51	86.55	91.24	86.68	88.99	87.92	84.61	86.78	91.16	87.60	89.68	86.49
RULER (8K)	66.86	-	62.09	75.79	75.87	85.94	82.55	83.05	72.84	76.74	80.96	79.35	74.51	87.45	81.84	66.55	81.81
RULER (16K)	61.53	-	65.65	70.17	85.91	82.05	76.95	75.47	-	74.40	78.29	72.59	69.07	83.84	78.85	68.25	76.20
RULER (32K)	56.23	-	59.92	61.31	-	76.78	75.35	72.00	-	70.90	71.80	63.58	68.16	80.63	75.73	66.54	72.65
RULER (64K)	50.76	-	89.25	-	-	-	69.53	58.59	-	70.38	73.27	54.37	57.16	80.51	74.24	59.73	75.45
RULER (128K)	42.38	-	80.58	-	-	-	-	53.33	-	60.99	-	-	51.83	69.15	-	47.69	68.14
LongBench v2	23.26	28.83	32.01	31.61	37.18	25.65	26.24	27.24	23.26	27.24	28.63	28.43	26.84	30.82	26.84	26.24	26.84
Knowledge and languages
TriviaQA	38.29	53.06	50.58	50.62	27.89	56.24	50.39	40.73	43.92	64.93	58.13	43.40	60.59	32.71	59.54	53.46	19.54
MMMLU	28.57	48.33	59.91	57.26	54.66	55.59	46.68	46.76	40.99	56.91	45.87	48.54	47.26	46.93	40.93	41.91	40.17
TruthfulQA MC2	37.49	49.40	52.30	53.38	50.36	44.96	47.68	41.85	46.72	44.98	45.98	43.65	39.67	45.86	39.30	38.18	44.93
AGIEval (English)	18.46	42.89	44.83	45.92	43.74	35.72	35.36	40.60	34.87	36.45	29.60	33.80	31.67	33.13	24.12	28.84	23.73
General
MMLU	39.62	73.74	74.21	71.19	75.76	68.67	61.56	65.55	68.31	66.54	59.71	64.09	57.89	61.52	54.94	53.95	52.90
MMLU (5-shot)	43.26	74.62	72.97	73.07	75.45	71.19	64.40	68.20	67.42	70.65	61.44	63.81	59.61	53.88	56.59	57.66	48.11
MMLU (cloze)	38.80	48.82	48.41	49.32	51.99	49.26	45.39	44.84	46.67	48.72	44.64	38.34	46.35	38.22	42.54	44.08	35.28
MMLU-Redux	41.24	76.29	78.48	75.85	78.99	72.57	65.44	69.79	71.09	69.83	62.95	66.47	61.54	66.32	57.69	57.04	56.59
MMLU-Redux (5-shot)	44.41	77.47	76.44	77.47	78.80	74.86	68.01	72.27	70.38	73.38	64.09	66.10	62.96	56.85	59.19	60.83	50.49
BIG-Bench Hard	31.18	80.34	80.45	71.37	80.03	71.17	63.95	70.86	71.08	62.74	59.75	51.82	51.60	63.43	47.29	42.88	46.81
DROP	29.91	62.55	67.88	60.14	26.16	59.11	65.62	54.06	51.05	56.80	44.52	47.10	45.64	54.22	43.91	41.16	39.77
HellaSwag	66.53	76.43	75.23	73.75	75.88	73.76	74.67	71.22	71.61	78.51	75.13	66.18	75.81	65.94	74.22	73.71	54.92
PIQA	76.33	80.03	79.22	78.07	80.14	79.27	79.33	77.69	78.51	80.74	78.67	75.84	80.09	74.81	77.91	78.78	71.87
ARC-Easy	65.78	80.60	76.22	76.18	79.46	83.00	79.25	77.06	76.89	85.19	76.85	80.89	81.57	71.42	72.14	80.05	67.42
ARC-Challenge	38.23	58.70	55.29	51.96	55.63	58.02	49.83	54.10	52.56	59.73	52.13	51.62	54.52	45.82	46.25	52.30	40.44
WinoGrande	61.64	68.75	71.03	70.96	70.09	69.69	69.06	66.30	66.54	70.24	68.27	65.98	69.69	64.64	70.09	65.43	60.06
OpenBookQA	40.40	43.80	42.40	41.00	44.00	41.60	43.80	42.40	41.60	46.00	44.80	43.20	43.00	37.80	40.60	44.40	36.00

Reading the table

Columns. Every column is a base model, never an instruct one, run on this server through the same harness, prompts, shot counts, windows and scorers as Praxis, in BF16 with greedy decoding wherever Praxis decodes greedily. Praxis stays in the first column, boxed; the rest are ordered by their Average, highest first, and a column still running joins the order when its Average lands. Picking models in the menu above the table narrows it to Praxis and those models, and the bold moves to the best of what is in view; unpicking them all brings every column back. Each header links to that model's weights. … is queued; - does not apply, such as a RULER length past a model's context window or past what fits the 16 GB evaluation GPU.

Average. The unweighted mean over the 44 benchmarks every finished column has a score for, so each average covers the same tests. The long-context tests are left out: RULER past 4K, since not every model can run that long, and LongBench v2, whose documents are cut to each model's own window, so its score measures how far a model can read as much as what it knows.

Average, with long context. The same 44 plus LongBench v2 and RULER at 8K, 16K, 32K, 64K and 128K, for the models that can run at every length.

# Params. The language model alone. Vision and audio encoders and multi-token-prediction heads are not counted.

# Activated params. The weights one token runs through, counted one way for every column: token embeddings, the readout and per-layer embedding tables are lookups rather than compute and are left out. Per pass is one trip through the stack; total is a token's whole path. Praxis makes two passes (every non-expert weight plus 4 of 128 routed experts per layer, each pass), Ouro four recurrent steps, Nanbeige 4.2 two loops, every other model one.

Training tokens. As each lab publishes it for the base model. Praxis: 804.76B of pretraining and 18B of context extension. Ministral 3's 1-3T is the range its paper gives for the whole family; Qwen3.5 and Gemma 4 publish no figure. Gemma 3, Llama 3.2 and Ministral 3 were distilled from larger models' output distributions, which carry more per token than text alone.

Q&A or instruction data in pretraining. What each lab says went into its base model's pretraining, in its own report or card. Metis-1.6 Praxis: no question-answer, instruction or chat data; 87.9B of its 804.76B pretraining tokens are synthetic teaching prose. Ouro: its final 300B-token stage is drawn from 20+ supervised fine-tuning datasets converted to the ChatML chat format, and SFT data is 9.6% of its second stage (report). Granite 4.1: chain-of-thought and synthetic instruction data, about a quarter of its third phase (blog). Qwen3: trillions of synthesized tokens including question answering and instructions (report). Nanbeige 4.2: up-weighted synthetic QA and some agentic trajectories (report). SmolLM3: instruction and reasoning datasets in its last pretraining stage (blog). Falcon-H1: question-answer pairs, chain-of-thought solutions and synthetic QA (report). Qwen3.5, Gemma 4, Gemma 3, LFM2.5, Ministral 3 and Llama 3.2 do not say; Qwen3.5's card notes its chat-template tokens were trained in the base model, and Gemma 3, Ministral 3 and Llama 3.2 learned from larger models' outputs without saying whether that teacher was a base or an instruct model. A base model that has seen these formats starts closer to an instruct model, which helps most on zero- and few-shot question answering.

Obeys a bare instruction. The same 200 lines for every column: one-line instructions whose answer takes no knowledge or skill (say a word, copy a sentence, change its case, answer yes or no), sent as the instruction and a newline, with no chat template and no examples. The figure is the share answered exactly on the first line, greedy. A model trained on documents alone tends to carry on the page instead, repeating the instruction or starting a post; one that has trained on instructions answers them.

Not on this card. 13 benchmarks that need instruction following, a judge model, tools or an agent loop join the table with the post-trained Metis-1.6.

How each benchmark is run

Praxis's operating point. Native BF16, forced depth 2, four routed experts per token, deterministic routing, depth-memory gates at 1.0, no n-gram memory and the full 131,072-token window. Prompts longer than 4,096 tokens run as 4,096-token blocks that each continue the model's cache: one block is bit-identical to the ordinary forward, and splitting moves the score by 0.035 nats per token with 98% of next-token choices agreeing, no more than batching already does to this model. These are scores for that fixed setting, not for every depth or routing policy the architecture allows.

Every other column runs in BF16 through vLLM (Nanbeige 4.2 through Transformers), fed the harness's exact token ids.

Harness. lm-evaluation-harness 0.4.13, except where a benchmark ships its own grader (LiveCodeBench, BigCodeBench, MultiPL-E, SciCode, BIG-Bench Extra Hard, ARC-AGI-2), which is used unchanged. Each benchmark runs at its standard shot count for base models. The exceptions are deliberate: TriviaQA at 5 shots, SuperGPQA and SimpleQA Verified in base-model form, and the competition math rows on MATH-500's 4-shot prompt so they compare with it directly. benchmark_suite.json holds every row with its protocol and every column's score.

Multiple choice, 0-shot, scored by answer likelihood. Length-normalized for HellaSwag, ARC, OpenBookQA and PIQA. MMMLU questions too long for the window count wrong.
GPQA, GPQA Diamond, MMMLU, TruthfulQA MC2, AGIEval (English), MMLU, MMLU-Redux, HellaSwag, PIQA, ARC-Easy, ARC-Challenge, WinoGrande and OpenBookQA

Multiple choice, 5-shot, scored by answer likelihood. SuperGPQA uses five fixed demonstrations in place of its generative official protocol, and counts the questions too long for the window wrong.
SuperGPQA, MMLU-Pro, MMLU (5-shot) and MMLU-Redux (5-shot)

Multiple choice with no options shown. Each answer's own text is scored after the question, 0-shot, normalized by its length in bytes. The other MMLU rows show lettered options and score the letter, which also tests whether a small model can tie a letter to its option; this one does not.
MMLU (cloze)

Text only, no judge model. Multiple choice scored by answer likelihood; short answers generated greedily and matched exactly or as an equivalent. Items too long for the window count wrong. Not comparable with official HLE scores, which are judged and include images.
Humanity's Last Exam, HLE-Diamond and HLE-Verified (Gold)

Greedy generation, answer matched. GSM8K and GSM8K-Platinum 5-shot, strict match. SimpleQA Verified 5-shot, normalized match, no grader model. TriviaQA 5-shot. MGSM 8-shot native chain of thought in all 11 languages, items that outgrow the window counted wrong. BIG-Bench Hard 3-shot chain of thought. DROP 3-shot F1.
SimpleQA Verified, GSM8K, GSM8K-Platinum, MGSM, TriviaQA, BIG-Bench Hard and DROP

Minerva 4-shot prompt, greedy. MATH-500 by exact match with a 256-token cap; AIME, HMMT and Omni-MATH checked by math_verify, Omni-MATH without its judge model.
AIME 2026, HMMT Feb 2026, AIME 2025, MATH-500 and Omni-MATH

Code run against the tests, greedy. HumanEval+ 0-shot, MBPP+ 3-shot, and BigCodeBench's complete split, all 1,140 tasks with 4 GB of memory each (the official 30 GB is sized for a larger machine; 99.5% of reference solutions pass here). Prompts are right-stripped, as the reference harnesses do.
HumanEval+, MBPP+ and BigCodeBench

Code, pass@1 from 10 samples at temperature 0.2. LiveCodeBench with its official base-model prompt on the January to April 2025 problems, CRUXEval 2-shot, MultiPL-E as the mean of 18 languages.
LiveCodeBench, CRUXEval-O, CRUXEval-I and MultiPL-E

The benchmark's own prompt and scorer. BIG-Bench Extra Hard 0-shot, items too long for the window counted wrong. ARC-AGI-2 as text grids, two attempts. SciCode without background, the reply starting at the step's function header.
BIG-Bench Extra Hard, ARC-AGI-2 and SciCode

RULER, 13 tasks at each length. Haystacks built with each model's own tokenizer: 500 items per task up to 16K, the first 200 at 32K, the first 100 at 64K and 128K. No model is run past its context window, or past what fits the 16 GB evaluation GPU at that length.
RULER

LongBench v2, 0-shot. All 503 questions in the official prompt, scored by the answer letter's likelihood. A prompt longer than the window is cut in the middle, as the official protocol does. Chance is 25.
LongBench v2

Praxis, row by row

• Humanity's Last Exam: 2,158 text-only items; multiple choice 15.40, short answer 0.30.
• HLE-Diamond: 728 text-only items; multiple choice 13.73, short answer 0.70.
• HLE-Verified (Gold): 575 text-only items; multiple choice 11.56, short answer 0.75.
• MATH-500 by math_verify: 16.20.
• GSM8K flexible extraction: 22.90.
• MGSM by language, strict match: bn 0.0, de 6.0, en 26.8, es 7.2, fr 12.4, ja 1.2, ru 4.8, sw 1.6, te 0.0, th 2.0, zh 6.8. With flexible answer extraction the 11-language average is 8.11.
• MultiPL-E by language: js 16.5, ts 14.9, cpp 14.7, java 11.8, php 10.9, lua 10.1, jl 9.7, scala 9.2, go 8.5, pl 8.0, cs 7.5, rs 7.1, d 7.1, rb 7.0, r 6.5, swift 4.9, rkt 2.9, sh 1.4.
• SciCode: 0 of 65 problems solved end to end, 0 of 288 steps (0.00).
• LongBench v2 by document length: short 21.67, medium 25.12, long 22.22. Below chance because of a letter habit, not the documents: the model answers A 42% of the time, and A is correct 19% of the time.
• TruthfulQA MC1: 22.52.
• MMLU cloze without length normalization: 36.38.

Contamination screening

Praxis's pretraining data was screened against a pinned registry of 63 benchmark sources (203 evaluation splits). A document matching any held-out item exactly, after normalization, or on long shared n-grams (13 tokens for text, 12 for code) was removed whole. That covers most of the suite above, including LiveCodeBench through its sixth release, which spans the January to April 2025 window scored here. AIME 2026 and HMMT February 2026 were written after the training data.

Screening lowers the risk; it does not prove absence. Paraphrased or translated copies can pass n-gram filters, and benchmarks outside the registry (SuperGPQA, BIG-Bench Extra Hard, AGIEval, PIQA, BigCodeBench, ARC-AGI-2) were not screened.

From Metis-1.5

Benchmark	Metric	Praxis Base	1.5-base	1.5-think	Gain over 1.5-think
MMLU	acc	39.62	23.6	23.3	+16.32
HellaSwag	acc_norm	66.53	30.4	31.0	+35.53
ARC-Easy	acc_norm	65.78	41.3	41.7	+24.08
ARC-Challenge	acc_norm	38.23	25.9	28.2	+10.03
OpenBookQA	acc_norm	40.40	29.6	28.6	+11.80
PIQA	acc_norm	76.33	54.7	54.6	+21.73
WinoGrande	acc	61.64	51.5	51.8	+9.84
Average		55.51	36.71	37.03	+18.48

Seven zero-shot tasks, full test sets. Metis-1.5 had 898M parameters (about 340M active) and saw 50B tokens. Its scores are the ones published on the Metis-1.5-think card, measured by its own JAX likelihood harness, so a point either way is noise and ten are not. The two 1.5 columns sit within a point of each other, so the gap is pretraining, not post-training.

Long context

Context extension trained on 18B tokens of sequences up to 163,840 tokens. On 384 held-out records at 131,072 tokens, a planted needle's top-1 recall rose from 18% to 66%, and the far context now lowers loss by 0.22 nats per token where before it raised it by 3.9. RULER at every length from 4K to 128K is in the table above.

The extension gate

Metric	Before extension	This checkpoint
Full-context tail NLL	5.735	1.614
Short-context tail NLL	1.861	1.833
Long-context gain (short minus full)	-3.875	+0.220
Base-window NLL	1.703	1.665
Needle top-1 accuracy	0.180	0.656
Needle target log-probability	-5.918	-1.871

Measured before training and at this checkpoint; lower NLL is better. Loss on short windows improved too, but that held-out text came from the extension's own mix.

What extension cost on short benchmarks

Benchmark	4K, before extension	This checkpoint	Change
Average of these 44	25.52	24.66	-0.86
Humanity's Last Exam	5.14	3.89	-1.25
HLE-Diamond	3.59	3.43	-0.16
HLE-Verified (Gold)	5.24	4.00	-1.24
GPQA	29.69	27.90	-1.79
GPQA Diamond	27.78	24.24	-3.54
SuperGPQA	10.96	11.12	+0.16
MMLU-Pro	13.60	14.19	+0.59
BIG-Bench Extra Hard	1.22	1.66	+0.44
ARC-AGI-2	0.00	0.00	0.00
SimpleQA Verified	2.50	2.70	+0.20
AIME 2026	0.00	0.00	0.00
HMMT Feb 2026	0.00	0.00	0.00
AIME 2025	0.00	0.00	0.00
MATH-500	19.20	16.20	-3.00
Omni-MATH	6.17	5.83	-0.34
GSM8K	24.72	22.59	-2.13
GSM8K-Platinum	25.31	23.57	-1.74
MGSM	8.15	6.25	-1.90
LiveCodeBench	0.44	0.00	-0.44
HumanEval+	17.68	16.46	-1.22
MBPP+	42.33	35.98	-6.35
CRUXEval-O	14.04	12.49	-1.55
CRUXEval-I	23.04	20.84	-2.20
BigCodeBench	10.09	7.28	-2.81
MultiPL-E	10.47	8.80	-1.67
SciCode	0.00	0.00	0.00
RULER (4K)	68.17	75.54	+7.37
TriviaQA	40.72	38.29	-2.43
MMMLU	28.09	28.57	+0.48
TruthfulQA MC2	36.71	37.49	+0.78
AGIEval (English)	18.25	18.46	+0.21
MMLU	39.65	39.62	-0.03
MMLU (5-shot)	43.58	43.26	-0.32
MMLU (cloze)	39.30	38.80	-0.50
MMLU-Redux	41.39	41.24	-0.15
MMLU-Redux (5-shot)	45.31	44.41	-0.90
BIG-Bench Hard	33.44	31.18	-2.26
DROP	29.36	29.91	+0.55
HellaSwag	67.50	66.53	-0.97
PIQA	77.42	76.33	-1.09
ARC-Easy	69.74	65.78	-3.96
ARC-Challenge	39.76	38.23	-1.53
WinoGrande	62.43	61.64	-0.79
OpenBookQA	40.60	40.40	-0.20

The 4K column is the checkpoint before extension as this repository first held it (see the earlier revision, below). MMLU 5-shot, GSM8K, GPQA, GPQA Diamond, MMLU-Pro, MATH-500 and Humanity's Last Exam in it are the training cluster's measurements of the same file; every other score is from this server. Extension cost under a point on average, most of it in code and math (MBPP+ fell 6.4 points, MATH-500 3.0) and ARC-Easy (4.0), while RULER at 4K gained 7.4.

Quickstart

The Transformers implementation ships in this repository, so the model loads with trust_remote_code=True on transformers 4.57 or newer. It needs about 6.2 GB of memory in BF16. It is a base model, so give it the beginning of a document, not a question.

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "Lernex/Metis-1.6-Praxis-Base"
tokenizer = AutoTokenizer.from_pretrained(model_id, trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(model_id, dtype="auto", device_map="auto", trust_remote_code=True)

prompt = "The three laws of thermodynamics state that"
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
output = model.generate(**inputs, max_new_tokens=200, do_sample=True, temperature=0.7, top_p=0.95)
print(tokenizer.decode(output[0], skip_special_tokens=True))

This implementation computes exactly what the native model computes, and it is the same code as the post-trained model's: the two checkpoints share every tensor name and shape. The vLLM and SGLang plugins and the llama.cpp converter in the Metis-1.6 Praxis card read the same configuration; they were validated on the post-trained weights.

Architecture

Parameters	2.94B; 0.46B per pass, two passes per token
Architecture	12 layers of Mamba-2 and attention, width 2,048; 128 routed experts, 4 per token
Context	131,072 tokens; attention has no positional encoding
Training	822.76B tokens: 804.76B of pretraining, then 18B of sequences up to 163,840 tokens
Vocabulary	65,536 tokens
Weights	BF16 with FP32 routers and state-space parameters, one file for Transformers and the native implementation

Metis-1.6 does not stack more layers. It runs the same 12 twice.

• Two passes, one set of weights. Every token goes through 24 sequence-mixing and expert stages while only 12 are stored. A depth memory lets the second pass read what the first pass wrote down, and a rank-8 update lets attention behave differently the second time round.
• Mostly Mamba-2. Attention sits at layers 4 and 8 only, and it uses no positional encoding at all: the Mamba-2 layers already know where each token is. So only 2 of 12 layers keep a KV cache, memory grows slowly with context, and the context stretched from 4,096 to 131,072 tokens without touching position handling.
• A latent mixture-of-experts. Every layer ends in 128 routed experts and one shared expert that work in a 1,024-dimensional latent space, four routed experts per token. That is where the efficiency comes from: most of the 2.94B parameters sit idle for any given token.
• Four residual streams of width 2,048, read, written and mixed at every sub-layer by a doubly stochastic 4x4 matrix that knows which pass it is on.
• A memory the model switched off. Pretraining included a 0.60B-parameter hashed n-gram memory feeding layers 2 and 4. The model learned to close its gates, so I removed it before release: it moves held-out bits per byte from 0.78110 to 0.78123.

A quick note

Run it, build on it, and try to break it. Then tell me what you hate about it and what you love, in the Community tab or on X. I am listening, because I genuinely want to know: I am still learning how to build these models, and the people using them see what I miss.

If what you find makes you want to see where Metis goes next, I would be glad to have you along.

Giulianno Vollmer, founder of Lernex

Files

The same model.safetensors loads in Transformers and in the native Metis-1.6 implementation; for the native one, build the model inside ngram_free.without_ngram_memory(), since the native code always builds an n-gram memory and this archive has none. Every score on this page was produced from these weights.

File	What it holds
model.safetensors	The weights, routed experts in global expert order
native_config.json	The Metis-1.6 family configuration
eval_native_config.json	The single-device configuration behind every score on this page
tokenizer.json	The tokenizer, 65,536 tokens
NGRAM_CANONICAL_IDS.json, .uint16	The canonical ID sidecar the native forward takes as an input
ngram_free.py	Builds the model without its n-gram memory
NGRAM_MEMORY_REMOVED.json	What was removed, and from which file
conversion_receipt.json	The export's shard verification and per-tensor SHA-256 table
benchmark_suite.json	Every benchmark: its protocol, Praxis's score and every lab column's
config.json, configuration_metis16.py, modeling_metis16.py	Hugging Face AutoConfig / AutoModelForCausalLM (trust_remote_code=True)
tokenizer_config.json, special_tokens_map.json, generation_config.json	Tokenizer and generation settings for Transformers

Without its n-gram memory

Metis-1.6 was trained with a hashed n-gram memory: 0.60B parameters of tables, looked up from the preceding tokens and added to the residual stream through a learned gate at layers 2 and 4. Praxis learned to keep that gate shut. Measured on 300,000 characters of prose at this checkpoint:

Injection	Mean gate (0 shut, 1 open)	Update, relative to the stream
Layer 2, pass 1	0.0006	0.15%
Layer 2, pass 2	0.0008	0.09%
Layer 4, pass 1	0.0003	0.09%
Layer 4, pass 2	0.0060	0.29%

Removing it moves bits per byte from 0.78110 to 0.78123, and the seven zero-shot tasks from a 55.36 average to 55.51, every task within the half-point a rerun moves it. So this archive has none of it: no tables, no projection, no gates, 602,180,520 parameters fewer. It is exactly the trained model with its n-gram gates at zero, which is how every score on this page was measured, since the memory's only path into the model is that gated addition. A fix for how the memory trains is part of the next generation.

The earlier revision

Until October 5, 2026 this repository held the 4,096-token checkpoint from before context extension. Its n-gram tables were written with their rows in expert-parallel shard order rather than the global order the model reads them in, so every n-gram lookup on that archive returned an unrelated row. The training cluster's evaluation read the same file, so its scores, and every score first published here, measured the model with its n-gram memory scrambled. Restored to global order (each repaired table then matches the extended checkpoint's row for row, median cosine 1.000 against 0.000), the same checkpoint scored 56.75 on the seven zero-shot tasks against 56.73 as first archived: a memory the model keeps shut cannot be broken by scrambling or fixed by repair. Its scores stay on this page, under long context, for what they are. Its weights, and this checkpoint's own with the n-gram memory still in, were removed from this repository, history included, to keep private storage for current weights.

Checkpoint identity

	SHA-256
Checkpoint	0ecb0918cd31527aced61d8d6325150c94cd5657d8352b4010580394755c546e
Pretraining checkpoint it was extended from	91aec83eac8f4cc90c90a29ad56cbcb309ea4c710e1a799e6ff9a2268661ffb1
Context-extension stage configuration	501a745878846542e4a0ea1535880d3ab0d938b4f4c65d51f4cf757c972261b1
Exposure plan	4f9e04f1a327878cca9cd584504e768309d861b6c911e2a2871b77c519917f19
model.safetensors	4a29eba773aa80c059b8dfde514429e5b3f202bbb0faf93fb91ac711e491c3bc
The same checkpoint with its n-gram memory	dfb8873baa676ab4dfc9cbd98ed2f9b3785d72bc7c7781334483266c84c96040

model.safetensors keeps every name, dtype, shape and byte of the evaluated state dict except the n-gram memory's 20 tensors: 418 tensors and 3,077,520,828 serialized elements, a count that includes the tied embedding and readout twice, plus buffers, so it is larger than the parameter count. conversion_receipt.json describes the export with the n-gram memory: the source shard verification, its file hashes and a per-tensor dtype, shape and SHA-256 table. Every tensor kept here is byte-identical to its entry there.

Citation

If Metis helps your work, please cite the technical report, and the model if you used it directly:

@misc{vollmer2026thinktwice,
  title         = {Think Twice: The Metis-1.6 Technical Report},
  author        = {Vollmer, Giulianno},
  year          = {2026},
  eprint        = {2610.37903},
  archivePrefix = {arXiv},
  primaryClass  = {cs.LG},
  url           = {https://arxiv.org/abs/2610.37903}
}

@misc{vollmer2026praxisbase,
  title     = {{Metis-1.6 Praxis Base}},
  author    = {Vollmer, Giulianno},
  year      = {2026},
  publisher = {Hugging Face},
  url       = {https://huggingface.co/Lernex/Metis-1.6-Praxis-Base}
}

License

Metis-1.6 Praxis Base is released under the Apache 2.0 license.