README.md · kerzgrr/Senryu at main

Senryu / README.md

Upload Senryu (apo_zero apo-v2 EMA @ step 704, from SFT step 12800)

d5df530 verified 2 days ago

9.51 kB

---
license: apache-2.0
language:
  - en
tags:
  - text-generation
  - causal-lm
  - pytorch
  - sft
  - preference-optimization
  - apo
  - on-policy
  - instruction-tuned
  - chat
  - reasoning
  - tool-use
  - hybrid
  - gated-deltanet-2
  - gated-mla
  - senryu
pipeline_tag: text-generation
library_name: tiny_gdn
datasets:
  - HuggingFaceFW/fineweb-edu
  - HuggingFaceTB/smoltalk2
  - HuggingFaceTB/smoltalk
  - nvidia/Nemotron-Post-Training-Dataset-v1
  - nvidia/Nemotron-Agentic-v1
  - nvidia/Nemotron-SFT-Agentic-v2
  - nvidia/Nemotron-SFT-Instruction-Following-Chat-v2
  - nvidia/Nemotron-Instruction-Following-Chat-v1
  - nvidia/OpenMathReasoning
  - nvidia/OpenCodeReasoning
  - nvidia/When2Call
  - Agent-Ark/Toucan-1.5M
  - interstellarninja/hermes_reasoning_tool_use
  - interstellarninja/tool-use-multiturn-reasoning
  - NousResearch/Hermes-3-Dataset
  - glaiveai/glaive-function-calling-v2
  - allenai/WildChat-4.8M
  - QuixiAI/SystemChat-2.0
  - Open-Orca/SlimOrca
  - HuggingFaceH4/ultrachat_200k
  - ise-uiuc/Magicoder-Evol-Instruct-110K
  - bigcode/self-oss-instruct-sc2-exec-filter-50k
  - OpenAssistant/oasst2
  - allenai/tulu-3-sft-personas-instruction-following
  - HuggingFaceH4/no_robots
---
<div align="center">

# Senryu

### Reasoning + tool-call chat model (803M), preference-tuned on its own samples

[![Model](https://img.shields.io/badge/Model-803M_params-blue)](.)
[![Stage](https://img.shields.io/badge/Stage-SFT_+_on--policy_APO-green.svg)](.)
[![License](https://img.shields.io/badge/License-Apache_2.0-green.svg)](LICENSE)
[![Architecture](https://img.shields.io/badge/Arch-Gated_DeltaNet--2_+_Gated_MLA-purple.svg)](.)
[![Demo](https://img.shields.io/badge/Space-senryu--demo-indigo.svg)](https://huggingface.co/spaces/kerzgrr/senryu-demo)

*An 803M hybrid Gated DeltaNet-2 + gated MLA model: 32k-context SFT on reasoning, tool and chat data, then APO-zero on pairs sampled from the model itself*

</div>

---

## What this is

**Senryu** is the 803M successor to [Haiku](https://huggingface.co/kerzgrr/Haiku): eight more layers, Gated DeltaNet-2 in place of Kimi Delta Attention, and a full-rank MLA query.

- Pretraining on **1.15B** FineWeb-Edu tokens
- Supervised fine-tuning at a **32,768-token** sequence length on 6.7B tokens (5.39B assistant tokens) of reasoning, tool-call, instruction-following and chat data
- Then **APO-zero** on **22,517 on-policy preference pairs**: four samples per prompt from the SFT model itself, judged by [Skywork-Reward-V2-Llama-3.1-8B](https://huggingface.co/Skywork/Skywork-Reward-V2-Llama-3.1-8B) and rule checks for tool calls, answers and loops
- Live demo: [`kerzgrr/senryu-demo`](https://huggingface.co/spaces/kerzgrr/senryu-demo)
- Hub weights are the **EMA** snapshot in **bfloat16**

This upload is the final checkpoint of the preference run (optimizer step **704**, one full epoch of 22,517 pairs). It is the only released checkpoint of the Senryu generation; the pretrain and SFT checkpoints behind it are not published.

---

## Chat contract

### Thinking

Each assistant turn is prefixed with a **zero-loss control token**:

| Mode | Prefix | Typical body |
|------|--------|----------------|
| think | `<\|think\|>\n` | `<think>…</think>` then the answer |
| no-think | `<\|no_think\|>\n` | answer only |

`inference.py` streams the `<think>` region live (dim yellow) and hides the control tokens.

### Tool calls (Hermes JSON)

Tool schemas go in a Hermes-style `<tools>` block in the system turn. The model calls them with:

```
<tool_call>
{"name": "web-search", "arguments": {"query": "…"}}
</tool_call>
```

Senryu learned schema-following calls from Glaive function calling v2, When2Call, Hermes reasoning tool-use, SmolTalk and the stage-3 tool mix (Toucan 1.5M, Nemotron PTD v1, Nemotron Agentic v1/v2). `web-search` is the only built-in it saw by name, in the Nemotron Agentic conversations; to Senryu, `stateful_python_code_exec` and `calculator` are ordinary schemas it reads from the `<tools>` block. On prompts that offer tools, the preference pairs reward calling and answering directly equally often (6,794 pairs each). `inference.py` auto-runs:

| Built-in | Tool name | Observation |
|----------|-----------|-------------|
| `--tools web-search` | `web-search` | Tavily-shaped JSON (Tavily if `TAVILY_API_KEY` is set, else DuckDuckGo + Wikipedia) |
| `--tools python` | `stateful_python_code_exec` | Jupyter-style stdout / last value from a restricted interpreter |
| `--tools calculator` | `calculator` | Numeric result of a math expression |

### Tool results

Each observation is a `tool` (or `user`) turn prefixed with:

```
<|tool_response|>
{observation}
```

---

## Install & run

```bash
pip install torch safetensors tokenizers huggingface_hub
hf download kerzgrr/Senryu inference.py --local-dir .
python inference.py
```

```bash
python inference.py --prompt "What is the capital of France?"
python inference.py --tools web-search,python,calculator
python inference.py --no-think --prompt "Reply in one sentence."
```

`inference.py` auto-downloads weights / tokenizer / `tiny_gdn/` and auto-installs pinned `flash-linear-attention` (plus `transformers`, which its decode cache imports). Git is required on `PATH`. A CUDA GPU is required: the Gated DeltaNet-2 layers run on Triton kernels.

| Flag | Default | Description |
|------|---------|-------------|
| `--prompt` | — | One-shot user message |
| `--system` | — | System prompt, used verbatim |
| `--think` / `--no-think` | think | Assistant control prefix |
| `--tools` | — | Built-ins: `web-search`, `python`, `calculator` (comma-separated) |
| `--temperature` | `0.6` | Sampling temperature (0.3 suits tool-calling turns) |
| `--top-p` | `0.95` | Nucleus sampling |
| `--repetition-penalty` | `1.08` | Over the last 256 tokens |
| `--max-new-tokens` | `4096` | Max generation length |
| `--context-length` | `32768` | Conversation tokens kept (oldest dropped first) |
| `--device` | `cuda` | CUDA device, e.g. `cuda:1` |

Interactive commands: `/think` `/no_think` `/system …` `/reset` `/exit`. Ctrl+C stops the current reply and keeps the session.

---

## Model architecture

803,000,968 parameters:

| | |
|--|--|
| **Layers** | 44 (Gated DeltaNet-2 ×3 + gated MLA every 4th) |
| **Hidden** | 1,024 |
| **MLP** | SiTU-GLU 3,392 |
| **Attention** | Gated MLA (NoPE), 8 heads, full-rank query, KV LoRA rank 512 |
| **Linear** | Gated DeltaNet-2, 8 heads × 128, channel-wise erase and write gates |
| **Residuals** | Block attention residuals |
| **Vocab** | 65,536 BPE (the Haiku tokenizer) |
| **Context** | 32,768 (SFT sequence length) |

---

## Training

| Stage | Details |
|-------|---------|
| **Pretrain** | [FineWeb-Edu](https://huggingface.co/datasets/HuggingFaceFW/fineweb-edu), seq 2,048, 2,200 steps = 1.15B tokens, hybrid Muon + AdamW at peak 5×10⁻⁴, 10.4 hours. Val loss (EMA) 3.06 |
| **SFT** | Natural-proportion mixture with a full shuffle: [SmolTalk2](https://huggingface.co/datasets/HuggingFaceTB/smoltalk2) Mid + SFT, the stage-3 tool/agentic mix (Nemotron PTD v1, Agentic v1/v2, IF-Chat v1/v2, Toucan 1.5M), OpenMathReasoning, OpenCodeReasoning, WildChat-4.8M, SmolTalk, Hermes-3, Hermes reasoning tool-use, SystemChat 2.0, SlimOrca, UltraChat 200k, Magicoder Evol-Instruct, Self-OSS-Instruct, Glaive function calling v2, When2Call, OASST2, Tulu 3 IF personas, No Robots. Seq 32,768, hybrid Muon + AdamW 4×10⁻⁴, 12,800 steps = 6.7B tokens (5.39B assistant tokens), 11.8 hours. Val loss (EMA) 0.827 (ppl 2.29) |
| **Preference (APO)** | APO-zero, β = 0.05, on on-policy pairs. 20,000 prompts drawn from the SFT mixture, four samples each from the SFT EMA (temperature 0.8, top-p 0.95). The chosen reply is the best-judged sample, or the dataset's reference reply when no sample passed; pairs need a reward margin of at least 3. Prompts that offer tools have 6,794 call pairs and 6,794 answer-directly pairs. 3,697 loop-stop pairs prefer ending the turn exactly where a repeat begins over one more repeat. 22,517 pairs, one epoch, 32 pairs per step, 704 steps, AdamW 1×10⁻⁶ cosine to 1×10⁻⁷, 10% warmup, clip 0.2, max length 16,512. Reference model: the SFT EMA. 1.8 hours |
| **Weights** | EMA (this repo's `model.safetensors`) |
| **Preference val (EMA)** | 903 held-out pairs: reward accuracy 79.0%, reward margin 1.33, chosen NLL 0.56 per token |

`preference_config.json`, `validation.json`, `sft_config.json` and `sft_validation.json` hold the exact settings and held-out results. `preference_data.json` describes the pair pack: prompt selection, sampling, judging, and the composition by source and label.

---

## Limitations

- **Scale**: 803M is a research / edge model, not a frontier system
---

## Model family

| Model | Stage | Hub |
|-------|-------|-----|
| **Senryu** | **SFT + on-policy APO (chat, reasoning, tools)** | **this repo** |
| Demo | ZeroGPU Space (chat) | [`kerzgrr/senryu-demo`](https://huggingface.co/spaces/kerzgrr/senryu-demo) |
| Haiku | Previous generation (~655M) | [`kerzgrr/Haiku`](https://huggingface.co/kerzgrr/Haiku) |
| Haiku-base | Previous generation, pretrain | [`kerzgrr/Haiku-base`](https://huggingface.co/kerzgrr/Haiku-base) |

---

## Citation

```bibtex
@misc{senryu2026,
  title={Senryu: An 803M Hybrid Gated DeltaNet-2 + Gated-MLA Reasoning Model},
  author={kerzgrr},
  year={2026},
  url={https://huggingface.co/kerzgrr/Senryu}
}
```

---

<div align="center">

*Five, seven, five — about people this time.*

</div>

	---
	license: apache-2.0
	language:
	- en
	tags:
	- text-generation
	- causal-lm
	- pytorch
	- sft
	- preference-optimization
	- apo
	- on-policy
	- instruction-tuned
	- chat
	- reasoning
	- tool-use
	- hybrid
	- gated-deltanet-2
	- gated-mla
	- senryu
	pipeline_tag: text-generation
	library_name: tiny_gdn
	datasets:
	- HuggingFaceFW/fineweb-edu
	- HuggingFaceTB/smoltalk2
	- HuggingFaceTB/smoltalk
	- nvidia/Nemotron-Post-Training-Dataset-v1
	- nvidia/Nemotron-Agentic-v1
	- nvidia/Nemotron-SFT-Agentic-v2
	- nvidia/Nemotron-SFT-Instruction-Following-Chat-v2
	- nvidia/Nemotron-Instruction-Following-Chat-v1
	- nvidia/OpenMathReasoning
	- nvidia/OpenCodeReasoning
	- nvidia/When2Call
	- Agent-Ark/Toucan-1.5M
	- interstellarninja/hermes_reasoning_tool_use
	- interstellarninja/tool-use-multiturn-reasoning
	- NousResearch/Hermes-3-Dataset
	- glaiveai/glaive-function-calling-v2
	- allenai/WildChat-4.8M
	- QuixiAI/SystemChat-2.0
	- Open-Orca/SlimOrca
	- HuggingFaceH4/ultrachat_200k
	- ise-uiuc/Magicoder-Evol-Instruct-110K
	- bigcode/self-oss-instruct-sc2-exec-filter-50k
	- OpenAssistant/oasst2
	- allenai/tulu-3-sft-personas-instruction-following
	- HuggingFaceH4/no_robots
	---

	<div align="center">

	# Senryu

	### Reasoning + tool-call chat model (803M), preference-tuned on its own samples

	[![Model](https://img.shields.io/badge/Model-803M_params-blue)](.)
	[![Stage](https://img.shields.io/badge/Stage-SFT_+_on--policy_APO-green.svg)](.)
	[![License](https://img.shields.io/badge/License-Apache_2.0-green.svg)](LICENSE)
	[![Architecture](https://img.shields.io/badge/Arch-Gated_DeltaNet--2_+_Gated_MLA-purple.svg)](.)
	[![Demo](https://img.shields.io/badge/Space-senryu--demo-indigo.svg)](https://huggingface.co/spaces/kerzgrr/senryu-demo)

	An 803M hybrid Gated DeltaNet-2 + gated MLA model: 32k-context SFT on reasoning, tool and chat data, then APO-zero on pairs sampled from the model itself

	</div>

	---

	## What this is

	Senryu is the 803M successor to [Haiku](https://huggingface.co/kerzgrr/Haiku): eight more layers, Gated DeltaNet-2 in place of Kimi Delta Attention, and a full-rank MLA query.

	- Pretraining on 1.15B FineWeb-Edu tokens
	- Supervised fine-tuning at a 32,768-token sequence length on 6.7B tokens (5.39B assistant tokens) of reasoning, tool-call, instruction-following and chat data
	- Then APO-zero on 22,517 on-policy preference pairs: four samples per prompt from the SFT model itself, judged by [Skywork-Reward-V2-Llama-3.1-8B](https://huggingface.co/Skywork/Skywork-Reward-V2-Llama-3.1-8B) and rule checks for tool calls, answers and loops
	- Live demo: [`kerzgrr/senryu-demo`](https://huggingface.co/spaces/kerzgrr/senryu-demo)
	- Hub weights are the EMA snapshot in bfloat16

	This upload is the final checkpoint of the preference run (optimizer step 704, one full epoch of 22,517 pairs). It is the only released checkpoint of the Senryu generation; the pretrain and SFT checkpoints behind it are not published.

	---

	## Chat contract

	### Thinking

	Each assistant turn is prefixed with a zero-loss control token:

	\| Mode \| Prefix \| Typical body \|
	\|------\|--------\|----------------\|
	\| think \| `<\\|think\\|>\n` \| `<think>…</think>` then the answer \|
	\| no-think \| `<\\|no_think\\|>\n` \| answer only \|

	`inference.py` streams the `<think>` region live (dim yellow) and hides the control tokens.

	### Tool calls (Hermes JSON)

	Tool schemas go in a Hermes-style `<tools>` block in the system turn. The model calls them with:

	```
	<tool_call>
	{"name": "web-search", "arguments": {"query": "…"}}
	</tool_call>
	```

	Senryu learned schema-following calls from Glaive function calling v2, When2Call, Hermes reasoning tool-use, SmolTalk and the stage-3 tool mix (Toucan 1.5M, Nemotron PTD v1, Nemotron Agentic v1/v2). `web-search` is the only built-in it saw by name, in the Nemotron Agentic conversations; to Senryu, `stateful_python_code_exec` and `calculator` are ordinary schemas it reads from the `<tools>` block. On prompts that offer tools, the preference pairs reward calling and answering directly equally often (6,794 pairs each). `inference.py` auto-runs:

	\| Built-in \| Tool name \| Observation \|
	\|----------\|-----------\|-------------\|
	\| `--tools web-search` \| `web-search` \| Tavily-shaped JSON (Tavily if `TAVILY_API_KEY` is set, else DuckDuckGo + Wikipedia) \|
	\| `--tools python` \| `stateful_python_code_exec` \| Jupyter-style stdout / last value from a restricted interpreter \|
	\| `--tools calculator` \| `calculator` \| Numeric result of a math expression \|

	### Tool results

	Each observation is a `tool` (or `user`) turn prefixed with:

	```
	<\|tool_response\|>
	{observation}
	```

	---

	## Install & run

	```bash
	pip install torch safetensors tokenizers huggingface_hub
	hf download kerzgrr/Senryu inference.py --local-dir .
	python inference.py
	```

	```bash
	python inference.py --prompt "What is the capital of France?"
	python inference.py --tools web-search,python,calculator
	python inference.py --no-think --prompt "Reply in one sentence."
	```

	`inference.py` auto-downloads weights / tokenizer / `tiny_gdn/` and auto-installs pinned `flash-linear-attention` (plus `transformers`, which its decode cache imports). Git is required on `PATH`. A CUDA GPU is required: the Gated DeltaNet-2 layers run on Triton kernels.

	\| Flag \| Default \| Description \|
	\|------\|---------\|-------------\|
	\| `--prompt` \| — \| One-shot user message \|
	\| `--system` \| — \| System prompt, used verbatim \|
	\| `--think` / `--no-think` \| think \| Assistant control prefix \|
	\| `--tools` \| — \| Built-ins: `web-search`, `python`, `calculator` (comma-separated) \|
	\| `--temperature` \| `0.6` \| Sampling temperature (0.3 suits tool-calling turns) \|
	\| `--top-p` \| `0.95` \| Nucleus sampling \|
	\| `--repetition-penalty` \| `1.08` \| Over the last 256 tokens \|
	\| `--max-new-tokens` \| `4096` \| Max generation length \|
	\| `--context-length` \| `32768` \| Conversation tokens kept (oldest dropped first) \|
	\| `--device` \| `cuda` \| CUDA device, e.g. `cuda:1` \|

	Interactive commands: `/think` `/no_think` `/system …` `/reset` `/exit`. Ctrl+C stops the current reply and keeps the session.

	---

	## Model architecture

	803,000,968 parameters:

	\| \| \|
	\|--\|--\|
	\| Layers \| 44 (Gated DeltaNet-2 ×3 + gated MLA every 4th) \|
	\| Hidden \| 1,024 \|
	\| MLP \| SiTU-GLU 3,392 \|
	\| Attention \| Gated MLA (NoPE), 8 heads, full-rank query, KV LoRA rank 512 \|
	\| Linear \| Gated DeltaNet-2, 8 heads × 128, channel-wise erase and write gates \|
	\| Residuals \| Block attention residuals \|
	\| Vocab \| 65,536 BPE (the Haiku tokenizer) \|
	\| Context \| 32,768 (SFT sequence length) \|

	---

	## Training

	\| Stage \| Details \|
	\|-------\|---------\|
	\| Pretrain \| [FineWeb-Edu](https://huggingface.co/datasets/HuggingFaceFW/fineweb-edu), seq 2,048, 2,200 steps = 1.15B tokens, hybrid Muon + AdamW at peak 5×10⁻⁴, 10.4 hours. Val loss (EMA) 3.06 \|
	\| SFT \| Natural-proportion mixture with a full shuffle: [SmolTalk2](https://huggingface.co/datasets/HuggingFaceTB/smoltalk2) Mid + SFT, the stage-3 tool/agentic mix (Nemotron PTD v1, Agentic v1/v2, IF-Chat v1/v2, Toucan 1.5M), OpenMathReasoning, OpenCodeReasoning, WildChat-4.8M, SmolTalk, Hermes-3, Hermes reasoning tool-use, SystemChat 2.0, SlimOrca, UltraChat 200k, Magicoder Evol-Instruct, Self-OSS-Instruct, Glaive function calling v2, When2Call, OASST2, Tulu 3 IF personas, No Robots. Seq 32,768, hybrid Muon + AdamW 4×10⁻⁴, 12,800 steps = 6.7B tokens (5.39B assistant tokens), 11.8 hours. Val loss (EMA) 0.827 (ppl 2.29) \|
	\| Preference (APO) \| APO-zero, β = 0.05, on on-policy pairs. 20,000 prompts drawn from the SFT mixture, four samples each from the SFT EMA (temperature 0.8, top-p 0.95). The chosen reply is the best-judged sample, or the dataset's reference reply when no sample passed; pairs need a reward margin of at least 3. Prompts that offer tools have 6,794 call pairs and 6,794 answer-directly pairs. 3,697 loop-stop pairs prefer ending the turn exactly where a repeat begins over one more repeat. 22,517 pairs, one epoch, 32 pairs per step, 704 steps, AdamW 1×10⁻⁶ cosine to 1×10⁻⁷, 10% warmup, clip 0.2, max length 16,512. Reference model: the SFT EMA. 1.8 hours \|
	\| Weights \| EMA (this repo's `model.safetensors`) \|
	\| Preference val (EMA) \| 903 held-out pairs: reward accuracy 79.0%, reward margin 1.33, chosen NLL 0.56 per token \|

	`preference_config.json`, `validation.json`, `sft_config.json` and `sft_validation.json` hold the exact settings and held-out results. `preference_data.json` describes the pair pack: prompt selection, sampling, judging, and the composition by source and label.

	---

	## Limitations

	- Scale: 803M is a research / edge model, not a frontier system
	---

	## Model family

	\| Model \| Stage \| Hub \|
	\|-------\|-------\|-----\|
	\| Senryu \| SFT + on-policy APO (chat, reasoning, tools) \| this repo \|
	\| Demo \| ZeroGPU Space (chat) \| [`kerzgrr/senryu-demo`](https://huggingface.co/spaces/kerzgrr/senryu-demo) \|
	\| Haiku \| Previous generation (~655M) \| [`kerzgrr/Haiku`](https://huggingface.co/kerzgrr/Haiku) \|
	\| Haiku-base \| Previous generation, pretrain \| [`kerzgrr/Haiku-base`](https://huggingface.co/kerzgrr/Haiku-base) \|

	---

	## Citation

	```bibtex
	@misc{senryu2026,
	title={Senryu: An 803M Hybrid Gated DeltaNet-2 + Gated-MLA Reasoning Model},
	author={kerzgrr},
	year={2026},
	url={https://huggingface.co/kerzgrr/Senryu}
	}
	```

	---

	<div align="center">

	Five, seven, five — about people this time.

	</div>