Instructions to use CortexLM/Teutonic-1-Chat-Preview with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use CortexLM/Teutonic-1-Chat-Preview with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="CortexLM/Teutonic-1-Chat-Preview")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("CortexLM/Teutonic-1-Chat-Preview")
model = AutoModelForCausalLM.from_pretrained("CortexLM/Teutonic-1-Chat-Preview", device_map="auto")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=256)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use CortexLM/Teutonic-1-Chat-Preview with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "CortexLM/Teutonic-1-Chat-Preview"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "CortexLM/Teutonic-1-Chat-Preview",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/CortexLM/Teutonic-1-Chat-Preview

SGLang

How to use CortexLM/Teutonic-1-Chat-Preview with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "CortexLM/Teutonic-1-Chat-Preview" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "CortexLM/Teutonic-1-Chat-Preview",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "CortexLM/Teutonic-1-Chat-Preview" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "CortexLM/Teutonic-1-Chat-Preview",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use CortexLM/Teutonic-1-Chat-Preview with Docker Model Runner:
```
docker model run hf.co/CortexLM/Teutonic-1-Chat-Preview
```

Teutonic-1-Chat-Preview / README.md

echoctx

Clarify default checkpoint already has strong needle @128k /256k; LC math-kill co-train not shipped

05e3460 verified about 2 months ago

preview code

Raw History Blame Contribute Delete

9.91 kB

---
license: apache-2.0
language:
- en
pipeline_tag: text-generation
library_name: transformers
tags:
- teutonic
- cortexlm
- chat
- reasoning
- hybrid-attention
- fp8
- long-context
base_model:
- Qwen/Qwen3.5-9B
---
> ⚠️ **Benchmarks:** Benchmarks on this page may differ from other evals; they are **not fully verified** and may change.
>
> ⚠️ **Preview only:** **Training is in progress**; uploaded checkpoints are **preview only**, **not for production**.

# Teutonic-1-Chat-Preview

**CortexLM Teutonic-1-Chat-Preview** is a hybrid linear+softmax ~9B-class chat model with thinking-style reasoning, trained for long-context conversation and verifiable math/code problem solving.

> **Preview — still under active training.** Expect weight updates, better alignment, and longer reliable generations. This card is a snapshot, not a final release.

## Highlights

- Hybrid architecture (linear attention + softmax attention) in the Qwen3.5 text lineage
- Default weights: **FP8** (block-scaled `e4m3`, Qwen-style `quantization_config`) for efficient inference
- RoPE context capacity up to **256k** input tokens
- **Default checkpoint already shows strong long-context retrieval** in internal needle smokes (**1.0** single-needle @**128k** and @**256k**), alongside strong verifiable-math behavior
- Chat template with optional `<think>…</think>` reasoning blocks and tool-calling tags
- Trained with continued long-context mid-training, supervised chat+thinking+tools, and RL on verifiable rewards

## What’s done vs what’s still cooking

| Done in this preview | Still cooking |
| --- | --- |
| Long-context continued pretraining (up to 64k packs) | More preference / alignment optimization |
| Supervised chat, thinking, and tools | Broader RL beyond math/code verifiers |
| RL on verifiable math (and code-oriented) rewards | Reliable **64k** free-form generation (in progress) |
| Long-output continuation (16k→32k response budgets) | Multimodal later |
| **256k RoPE** on default weights + strong single-needle @128k/256k (internal smokes) | Harder multikey / multi-hop long-ctx retrieval |
| Chat + math quality retained on the shipped snapshot | Further alignment / preference passes |

**Practical limits today:** the **default FP8 weights** are the chat + math snapshot (strong internal math + single-needle @128k/256k). A later **pure long-context co-train** that pushed harder on 256k packs improved needle-style retrieval but **collapsed thinking/math** — that experiment is **not shipped** here and will not be promoted without math recovery. Generation defaults target long answers (up to 64k in `generation_config`), but **stable 64k-out training is still underway**.

## How it was trained (high level)

1. **Mid-training** — continued pretraining with a long-context curriculum (8k → 32k → 64k packs) to extend useful context under FP8 training.
2. **Supervised chat fine-tune** — chat, thinking traces, and tool-use style data.
3. **Reinforcement learning** — policy optimization against verifiable math/code rewards (boxed-answer style grading).
4. **Long-context / long-output continuation** — further supervised + RL passes to keep needle-style long-context behavior and raise response-length budgets.

### Training charts (real logged metrics)

![Training overview](assets/training_overview.png)

*Overview of mid-training, supervised chat fine-tune, RL verifiable rewards, and long-output continuation.*

![Mid-training loss](assets/midtrain_loss.png)

*Mid-training train loss (smoothed) with held-out val points across the long-context curriculum.*

![Supervised fine-tune loss](assets/sft_loss.png)

*Supervised chat+thinking fine-tune loss, plus a later long-context SFT continuation.*

![RL verifiable reward](assets/rlvr_reward.png)

*RL mean verifiable reward and response clip ratio; lower panel shows long-output continuation runs.*

## Quick start

### Transformers

```python
from transformers import AutoModelForCausalLM, AutoTokenizer
repo = "CortexLM/Teutonic-1-Chat-Preview"
tokenizer = AutoTokenizer.from_pretrained(repo, trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(
    repo,
    trust_remote_code=True,
    device_map="auto",
    torch_dtype="auto",  # loads FP8 weights via quantization_config
)
messages = [
    {"role": "system", "content": "You are Teutonic, a helpful reasoning assistant."},
    {"role": "user", "content": "Solve: 17 * 19. Put the final answer in \\boxed{}."},
]
prompt = tokenizer.apply_chat_template(
    messages,
    tokenize=False,
    add_generation_prompt=True,
    enable_thinking=True,  # if supported by the template path you use
)
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
out = model.generate(
    **inputs,
    max_new_tokens=4096,
    temperature=0.6,
    top_p=0.95,
    do_sample=True,
)
print(tokenizer.decode(out[0][inputs["input_ids"].shape[-1]:], skip_special_tokens=False))
```

Thinking content appears inside `<think>…</think>` when the model opts to reason before the final answer. Turns end with `<|im_end|>`.

### vLLM

```bash
vllm serve CortexLM/Teutonic-1-Chat-Preview \
  --trust-remote-code \
  --dtype auto \
  --max-model-len 32768
```

```python
from openai import OpenAI
client = OpenAI(base_url="http://localhost:8000/v1", api_key="EMPTY")
resp = client.chat.completions.create(
    model="CortexLM/Teutonic-1-Chat-Preview",
    messages=[{"role": "user", "content": "Write a short haiku about glaciers."}],
    temperature=0.7,
    top_p=0.9,
    max_tokens=512,
)
print(resp.choices[0].message.content)
```

### Recommended sampling

| Mode | temperature | top_p | notes |
| --- | ---: | ---: | --- |
| Reasoning / math | 0.5–0.7 | 0.90–0.95 | Prefer `\boxed{}` for graded answers |
| Chat / creative | 0.7–0.9 | 0.90–0.95 | — |
| Deterministic smoke | 0.0 | — | `do_sample=False` |

## Context & generation

- **Input capacity (config):** up to **256k** tokens (RoPE `max_position_embeddings=262144`)
- **Long-context retrieval (default weights, internal smokes):** single-needle pass rate **1.0** at **128k** and **256k**
- **Generation target:** moving toward **64k** free-form outputs; this preview is strongest at moderate-to-long answers (multi-k tokens). Treat 64k-out as aspirational until a later drop.

## Long-context experiments (not shipped)

We also ran a **pure 256k-input continued-training** co-train after the chat/RL path (128k → 256k packs). That path kept strong needle retrieval but **broke thinking/math** (internal private math collapsed; model often never closed `</think>` even at large think budgets). **Those weights are not published** and are **not** candidates for promotion without a clear math recovery (≥ strong chat/math gate).

| Capability | Default weights (this repo) | Pure LC co-train (internal only) |
| --- | --- | --- |
| Single-needle @128k / @256k | **1.0** (internal smokes) | Strong (also ~1.0) |
| Multi-key needle @128k+ | Still cooking / uneven | Weak–mixed |
| Verifiable math (internal private probe) | **Strong** (~chat/RL quality) | **Regressed hard** — do not ship |
| 64k free-form **output** training | Still **in progress** | Orthogonal track |

**Default weights on this repo** remain the stronger chat + math + long-ctx retrieval snapshot (FP8). We will only publish a dedicated “harder long-ctx” weight drop if math stays ≥ the promotion gate after any further long-context work.

## Evaluation honesty

Internal math/reasoning probes improved substantially after verifiable-reward RL relative to the supervised merge. We **do not** claim contaminated public-bench SOTA (e.g. treat GSM8K-style numbers with caution). Prefer private or carefully decontaminated suites when comparing. Behavioural chat smokes (format, tools, basic instruction following) pass on this snapshot; broader alignment and preference quality are still in progress.

## Limitations

- Preview checkpoint — APIs and weights will change
- Long free-form generation beyond ~32k tokens is not fully hardened
- Not fully preference-aligned; may over-think or under-refuse
- Hybrid kernels need recent `transformers` / `vLLM` with `trust_remote_code`
- FP8 inference quality tracks the Qwen-style block FP8 path; if your stack lacks FP8 support, request a BF16 variant or dequantize

## License & acknowledgements

Released under **Apache 2.0**.

This model continues the **Qwen3.5** text architecture lineage (hybrid Teutonic conversion). Please respect the Qwen / Alibaba base model terms and citations where applicable:

- [Qwen](https://huggingface.co/Qwen)
- CortexLM / Dendrite training stack

## Citation

```bibtex
@misc{teutonic1chatpreview2026,
  title        = {Teutonic-1-Chat-Preview},
  author       = {CortexLM},
  year         = {2026},
  howpublished = {\url{https://huggingface.co/CortexLM/Teutonic-1-Chat-Preview}},
  note         = {Preview checkpoint; under active training}
}
```




## Benchmarks

See [`benchmarks.md`](./benchmarks.md) for the **corrected** same-harness comparison vs Qwen3.5-9B (chat + thinking on, fixed 2026-09-01). The first published Qwen column (MMLU-Pro ~21 / MATH ~20) was an **invalid harness artifact** (thinking ON with 1k–8k caps → mid-think truncation).

| Benchmark | Teutonic | Qwen3.5-9B (fixed harness) | Δ |
|---|---:|---:|---:|
| MMLU-Pro (n=2000) | 69.8 | **73.2** | −3.4 |
| MMLU-Redux (generative) | 87.0 | **91.3** | −4.3 |
| MATH-500 | **88.6** | 68.2 | **+20.4** |
| HumanEval+ | **82.9** | 81.1 | **+1.8** |
| MBPP+ | **75.1** | 64.6 | **+10.5** |
| IFEval | **68.6** | 65.4 | **+3.2** |
| GPQA-Diamond (fair boxed) | 46.0 | **55.1** | −9.1 |
| LiveCodeBench (v5+v6) | 26.9 | 26.9 | 0.0 |
| RULER (4k–64k) | **89.5** | 88.4 | **+1.1** |

Protocol: chat · thinking on · raised think budgets · seed 20260830. GPQA = generative boxed-letter (not card shuffle). See benchmarks.md.

	---
	license: apache-2.0
	language:
	- en
	pipeline_tag: text-generation
	library_name: transformers
	tags:
	- teutonic
	- cortexlm
	- chat
	- reasoning
	- hybrid-attention
	- fp8
	- long-context
	base_model:
	- Qwen/Qwen3.5-9B
	---

	> ⚠️ Benchmarks: Benchmarks on this page may differ from other evals; they are not fully verified and may change.
	>
	> ⚠️ Preview only: Training is in progress; uploaded checkpoints are preview only, not for production.

	# Teutonic-1-Chat-Preview

	CortexLM Teutonic-1-Chat-Preview is a hybrid linear+softmax ~9B-class chat model with thinking-style reasoning, trained for long-context conversation and verifiable math/code problem solving.

	> Preview — still under active training. Expect weight updates, better alignment, and longer reliable generations. This card is a snapshot, not a final release.

	## Highlights

	- Hybrid architecture (linear attention + softmax attention) in the Qwen3.5 text lineage
	- Default weights: FP8 (block-scaled `e4m3`, Qwen-style `quantization_config`) for efficient inference
	- RoPE context capacity up to 256k input tokens
	- Default checkpoint already shows strong long-context retrieval in internal needle smokes (1.0 single-needle @128k and @256k), alongside strong verifiable-math behavior
	- Chat template with optional `<think>…</think>` reasoning blocks and tool-calling tags
	- Trained with continued long-context mid-training, supervised chat+thinking+tools, and RL on verifiable rewards

	## What’s done vs what’s still cooking

	\| Done in this preview \| Still cooking \|
	\| --- \| --- \|
	\| Long-context continued pretraining (up to 64k packs) \| More preference / alignment optimization \|
	\| Supervised chat, thinking, and tools \| Broader RL beyond math/code verifiers \|
	\| RL on verifiable math (and code-oriented) rewards \| Reliable 64k free-form generation (in progress) \|
	\| Long-output continuation (16k→32k response budgets) \| Multimodal later \|
	\| 256k RoPE on default weights + strong single-needle @128k/256k (internal smokes) \| Harder multikey / multi-hop long-ctx retrieval \|
	\| Chat + math quality retained on the shipped snapshot \| Further alignment / preference passes \|

	Practical limits today: the default FP8 weights are the chat + math snapshot (strong internal math + single-needle @128k/256k). A later pure long-context co-train that pushed harder on 256k packs improved needle-style retrieval but collapsed thinking/math — that experiment is not shipped here and will not be promoted without math recovery. Generation defaults target long answers (up to 64k in `generation_config`), but stable 64k-out training is still underway.

	## How it was trained (high level)

	1. Mid-training — continued pretraining with a long-context curriculum (8k → 32k → 64k packs) to extend useful context under FP8 training.
	2. Supervised chat fine-tune — chat, thinking traces, and tool-use style data.
	3. Reinforcement learning — policy optimization against verifiable math/code rewards (boxed-answer style grading).
	4. Long-context / long-output continuation — further supervised + RL passes to keep needle-style long-context behavior and raise response-length budgets.

	### Training charts (real logged metrics)

	![Training overview](assets/training_overview.png)

	Overview of mid-training, supervised chat fine-tune, RL verifiable rewards, and long-output continuation.

	![Mid-training loss](assets/midtrain_loss.png)

	Mid-training train loss (smoothed) with held-out val points across the long-context curriculum.

	![Supervised fine-tune loss](assets/sft_loss.png)

	Supervised chat+thinking fine-tune loss, plus a later long-context SFT continuation.

	![RL verifiable reward](assets/rlvr_reward.png)

	RL mean verifiable reward and response clip ratio; lower panel shows long-output continuation runs.

	## Quick start

	### Transformers

	```python
	from transformers import AutoModelForCausalLM, AutoTokenizer

	repo = "CortexLM/Teutonic-1-Chat-Preview"
	tokenizer = AutoTokenizer.from_pretrained(repo, trust_remote_code=True)
	model = AutoModelForCausalLM.from_pretrained(
	repo,
	trust_remote_code=True,
	device_map="auto",
	torch_dtype="auto", # loads FP8 weights via quantization_config
	)

	messages = [
	{"role": "system", "content": "You are Teutonic, a helpful reasoning assistant."},
	{"role": "user", "content": "Solve: 17 * 19. Put the final answer in \\boxed{}."},
	]
	prompt = tokenizer.apply_chat_template(
	messages,
	tokenize=False,
	add_generation_prompt=True,
	enable_thinking=True, # if supported by the template path you use
	)
	inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
	out = model.generate(
	**inputs,
	max_new_tokens=4096,
	temperature=0.6,
	top_p=0.95,
	do_sample=True,
	)
	print(tokenizer.decode(out[0][inputs["input_ids"].shape[-1]:], skip_special_tokens=False))
	```

	Thinking content appears inside `<think>…</think>` when the model opts to reason before the final answer. Turns end with `<\|im_end\|>`.

	### vLLM

	```bash
	vllm serve CortexLM/Teutonic-1-Chat-Preview \
	--trust-remote-code \
	--dtype auto \
	--max-model-len 32768
	```

	```python
	from openai import OpenAI
	client = OpenAI(base_url="http://localhost:8000/v1", api_key="EMPTY")
	resp = client.chat.completions.create(
	model="CortexLM/Teutonic-1-Chat-Preview",
	messages=[{"role": "user", "content": "Write a short haiku about glaciers."}],
	temperature=0.7,
	top_p=0.9,
	max_tokens=512,
	)
	print(resp.choices[0].message.content)
	```

	### Recommended sampling

	\| Mode \| temperature \| top_p \| notes \|
	\| --- \| ---: \| ---: \| --- \|
	\| Reasoning / math \| 0.5–0.7 \| 0.90–0.95 \| Prefer `\boxed{}` for graded answers \|
	\| Chat / creative \| 0.7–0.9 \| 0.90–0.95 \| — \|
	\| Deterministic smoke \| 0.0 \| — \| `do_sample=False` \|

	## Context & generation

	- Input capacity (config): up to 256k tokens (RoPE `max_position_embeddings=262144`)
	- Long-context retrieval (default weights, internal smokes): single-needle pass rate 1.0 at 128k and 256k
	- Generation target: moving toward 64k free-form outputs; this preview is strongest at moderate-to-long answers (multi-k tokens). Treat 64k-out as aspirational until a later drop.

	## Long-context experiments (not shipped)

	We also ran a pure 256k-input continued-training co-train after the chat/RL path (128k → 256k packs). That path kept strong needle retrieval but broke thinking/math (internal private math collapsed; model often never closed `</think>` even at large think budgets). Those weights are not published and are not candidates for promotion without a clear math recovery (≥ strong chat/math gate).

	\| Capability \| Default weights (this repo) \| Pure LC co-train (internal only) \|
	\| --- \| --- \| --- \|
	\| Single-needle @128k / @256k \| 1.0 (internal smokes) \| Strong (also ~1.0) \|
	\| Multi-key needle @128k+ \| Still cooking / uneven \| Weak–mixed \|
	\| Verifiable math (internal private probe) \| Strong (~chat/RL quality) \| Regressed hard — do not ship \|
	\| 64k free-form output training \| Still in progress \| Orthogonal track \|

	Default weights on this repo remain the stronger chat + math + long-ctx retrieval snapshot (FP8). We will only publish a dedicated “harder long-ctx” weight drop if math stays ≥ the promotion gate after any further long-context work.

	## Evaluation honesty

	Internal math/reasoning probes improved substantially after verifiable-reward RL relative to the supervised merge. We do not claim contaminated public-bench SOTA (e.g. treat GSM8K-style numbers with caution). Prefer private or carefully decontaminated suites when comparing. Behavioural chat smokes (format, tools, basic instruction following) pass on this snapshot; broader alignment and preference quality are still in progress.

	## Limitations

	- Preview checkpoint — APIs and weights will change
	- Long free-form generation beyond ~32k tokens is not fully hardened
	- Not fully preference-aligned; may over-think or under-refuse
	- Hybrid kernels need recent `transformers` / `vLLM` with `trust_remote_code`
	- FP8 inference quality tracks the Qwen-style block FP8 path; if your stack lacks FP8 support, request a BF16 variant or dequantize

	## License & acknowledgements

	Released under Apache 2.0.

	This model continues the Qwen3.5 text architecture lineage (hybrid Teutonic conversion). Please respect the Qwen / Alibaba base model terms and citations where applicable:

	- [Qwen](https://huggingface.co/Qwen)
	- CortexLM / Dendrite training stack

	## Citation

	```bibtex
	@misc{teutonic1chatpreview2026,
	title = {Teutonic-1-Chat-Preview},
	author = {CortexLM},
	year = {2026},
	howpublished = {\url{https://huggingface.co/CortexLM/Teutonic-1-Chat-Preview}},
	note = {Preview checkpoint; under active training}
	}
	```




	## Benchmarks

	See [`benchmarks.md`](./benchmarks.md) for the corrected same-harness comparison vs Qwen3.5-9B (chat + thinking on, fixed 2026-09-01). The first published Qwen column (MMLU-Pro ~21 / MATH ~20) was an invalid harness artifact (thinking ON with 1k–8k caps → mid-think truncation).

	\| Benchmark \| Teutonic \| Qwen3.5-9B (fixed harness) \| Δ \|
	\|---\|---:\|---:\|---:\|
	\| MMLU-Pro (n=2000) \| 69.8 \| 73.2 \| −3.4 \|
	\| MMLU-Redux (generative) \| 87.0 \| 91.3 \| −4.3 \|
	\| MATH-500 \| 88.6 \| 68.2 \| +20.4 \|
	\| HumanEval+ \| 82.9 \| 81.1 \| +1.8 \|
	\| MBPP+ \| 75.1 \| 64.6 \| +10.5 \|
	\| IFEval \| 68.6 \| 65.4 \| +3.2 \|
	\| GPQA-Diamond (fair boxed) \| 46.0 \| 55.1 \| −9.1 \|
	\| LiveCodeBench (v5+v6) \| 26.9 \| 26.9 \| 0.0 \|
	\| RULER (4k–64k) \| 89.5 \| 88.4 \| +1.1 \|

	Protocol: chat · thinking on · raised think budgets · seed 20260830. GPQA = generative boxed-letter (not card shuffle). See benchmarks.md.