README.md · kerzgrr/Senryu at main

File size: 9,511 Bytes

d5df530

---
license: apache-2.0
language:
  - en
tags:
  - text-generation
  - causal-lm
  - pytorch
  - sft
  - preference-optimization
  - apo
  - on-policy
  - instruction-tuned
  - chat
  - reasoning
  - tool-use
  - hybrid
  - gated-deltanet-2
  - gated-mla
  - senryu
pipeline_tag: text-generation
library_name: tiny_gdn
datasets:
  - HuggingFaceFW/fineweb-edu
  - HuggingFaceTB/smoltalk2
  - HuggingFaceTB/smoltalk
  - nvidia/Nemotron-Post-Training-Dataset-v1
  - nvidia/Nemotron-Agentic-v1
  - nvidia/Nemotron-SFT-Agentic-v2
  - nvidia/Nemotron-SFT-Instruction-Following-Chat-v2
  - nvidia/Nemotron-Instruction-Following-Chat-v1
  - nvidia/OpenMathReasoning
  - nvidia/OpenCodeReasoning
  - nvidia/When2Call
  - Agent-Ark/Toucan-1.5M
  - interstellarninja/hermes_reasoning_tool_use
  - interstellarninja/tool-use-multiturn-reasoning
  - NousResearch/Hermes-3-Dataset
  - glaiveai/glaive-function-calling-v2
  - allenai/WildChat-4.8M
  - QuixiAI/SystemChat-2.0
  - Open-Orca/SlimOrca
  - HuggingFaceH4/ultrachat_200k
  - ise-uiuc/Magicoder-Evol-Instruct-110K
  - bigcode/self-oss-instruct-sc2-exec-filter-50k
  - OpenAssistant/oasst2
  - allenai/tulu-3-sft-personas-instruction-following
  - HuggingFaceH4/no_robots
---

<div align="center">

# Senryu

### Reasoning + tool-call chat model (803M), preference-tuned on its own samples

[![Model](https://img.shields.io/badge/Model-803M_params-blue)](.)
[![Stage](https://img.shields.io/badge/Stage-SFT_+_on--policy_APO-green.svg)](.)
[![License](https://img.shields.io/badge/License-Apache_2.0-green.svg)](LICENSE)
[![Architecture](https://img.shields.io/badge/Arch-Gated_DeltaNet--2_+_Gated_MLA-purple.svg)](.)
[![Demo](https://img.shields.io/badge/Space-senryu--demo-indigo.svg)](https://huggingface.co/spaces/kerzgrr/senryu-demo)

*An 803M hybrid Gated DeltaNet-2 + gated MLA model: 32k-context SFT on reasoning, tool and chat data, then APO-zero on pairs sampled from the model itself*

</div>

---

## What this is

**Senryu** is the 803M successor to [Haiku](https://huggingface.co/kerzgrr/Haiku): eight more layers, Gated DeltaNet-2 in place of Kimi Delta Attention, and a full-rank MLA query.

- Pretraining on **1.15B** FineWeb-Edu tokens
- Supervised fine-tuning at a **32,768-token** sequence length on 6.7B tokens (5.39B assistant tokens) of reasoning, tool-call, instruction-following and chat data
- Then **APO-zero** on **22,517 on-policy preference pairs**: four samples per prompt from the SFT model itself, judged by [Skywork-Reward-V2-Llama-3.1-8B](https://huggingface.co/Skywork/Skywork-Reward-V2-Llama-3.1-8B) and rule checks for tool calls, answers and loops
- Live demo: [`kerzgrr/senryu-demo`](https://huggingface.co/spaces/kerzgrr/senryu-demo)
- Hub weights are the **EMA** snapshot in **bfloat16**

This upload is the final checkpoint of the preference run (optimizer step **704**, one full epoch of 22,517 pairs). It is the only released checkpoint of the Senryu generation; the pretrain and SFT checkpoints behind it are not published.

---

## Chat contract

### Thinking

Each assistant turn is prefixed with a **zero-loss control token**:

| Mode | Prefix | Typical body |
|------|--------|----------------|
| think | `<\|think\|>\n` | `<think>…</think>` then the answer |
| no-think | `<\|no_think\|>\n` | answer only |

`inference.py` streams the `<think>` region live (dim yellow) and hides the control tokens.

### Tool calls (Hermes JSON)

Tool schemas go in a Hermes-style `<tools>` block in the system turn. The model calls them with:

```
<tool_call>
{"name": "web-search", "arguments": {"query": "…"}}
</tool_call>
```

Senryu learned schema-following calls from Glaive function calling v2, When2Call, Hermes reasoning tool-use, SmolTalk and the stage-3 tool mix (Toucan 1.5M, Nemotron PTD v1, Nemotron Agentic v1/v2). `web-search` is the only built-in it saw by name, in the Nemotron Agentic conversations; to Senryu, `stateful_python_code_exec` and `calculator` are ordinary schemas it reads from the `<tools>` block. On prompts that offer tools, the preference pairs reward calling and answering directly equally often (6,794 pairs each). `inference.py` auto-runs:

| Built-in | Tool name | Observation |
|----------|-----------|-------------|
| `--tools web-search` | `web-search` | Tavily-shaped JSON (Tavily if `TAVILY_API_KEY` is set, else DuckDuckGo + Wikipedia) |
| `--tools python` | `stateful_python_code_exec` | Jupyter-style stdout / last value from a restricted interpreter |
| `--tools calculator` | `calculator` | Numeric result of a math expression |

### Tool results

Each observation is a `tool` (or `user`) turn prefixed with:

```
<|tool_response|>
{observation}
```

---

## Install & run

```bash
pip install torch safetensors tokenizers huggingface_hub
hf download kerzgrr/Senryu inference.py --local-dir .
python inference.py
```

```bash
python inference.py --prompt "What is the capital of France?"
python inference.py --tools web-search,python,calculator
python inference.py --no-think --prompt "Reply in one sentence."
```

`inference.py` auto-downloads weights / tokenizer / `tiny_gdn/` and auto-installs pinned `flash-linear-attention` (plus `transformers`, which its decode cache imports). Git is required on `PATH`. A CUDA GPU is required: the Gated DeltaNet-2 layers run on Triton kernels.

| Flag | Default | Description |
|------|---------|-------------|
| `--prompt` | — | One-shot user message |
| `--system` | — | System prompt, used verbatim |
| `--think` / `--no-think` | think | Assistant control prefix |
| `--tools` | — | Built-ins: `web-search`, `python`, `calculator` (comma-separated) |
| `--temperature` | `0.6` | Sampling temperature (0.3 suits tool-calling turns) |
| `--top-p` | `0.95` | Nucleus sampling |
| `--repetition-penalty` | `1.08` | Over the last 256 tokens |
| `--max-new-tokens` | `4096` | Max generation length |
| `--context-length` | `32768` | Conversation tokens kept (oldest dropped first) |
| `--device` | `cuda` | CUDA device, e.g. `cuda:1` |

Interactive commands: `/think` `/no_think` `/system …` `/reset` `/exit`. Ctrl+C stops the current reply and keeps the session.

---

## Model architecture

803,000,968 parameters:

| | |
|--|--|
| **Layers** | 44 (Gated DeltaNet-2 ×3 + gated MLA every 4th) |
| **Hidden** | 1,024 |
| **MLP** | SiTU-GLU 3,392 |
| **Attention** | Gated MLA (NoPE), 8 heads, full-rank query, KV LoRA rank 512 |
| **Linear** | Gated DeltaNet-2, 8 heads × 128, channel-wise erase and write gates |
| **Residuals** | Block attention residuals |
| **Vocab** | 65,536 BPE (the Haiku tokenizer) |
| **Context** | 32,768 (SFT sequence length) |

---

## Training

| Stage | Details |
|-------|---------|
| **Pretrain** | [FineWeb-Edu](https://huggingface.co/datasets/HuggingFaceFW/fineweb-edu), seq 2,048, 2,200 steps = 1.15B tokens, hybrid Muon + AdamW at peak 5×10⁻⁴, 10.4 hours. Val loss (EMA) 3.06 |
| **SFT** | Natural-proportion mixture with a full shuffle: [SmolTalk2](https://huggingface.co/datasets/HuggingFaceTB/smoltalk2) Mid + SFT, the stage-3 tool/agentic mix (Nemotron PTD v1, Agentic v1/v2, IF-Chat v1/v2, Toucan 1.5M), OpenMathReasoning, OpenCodeReasoning, WildChat-4.8M, SmolTalk, Hermes-3, Hermes reasoning tool-use, SystemChat 2.0, SlimOrca, UltraChat 200k, Magicoder Evol-Instruct, Self-OSS-Instruct, Glaive function calling v2, When2Call, OASST2, Tulu 3 IF personas, No Robots. Seq 32,768, hybrid Muon + AdamW 4×10⁻⁴, 12,800 steps = 6.7B tokens (5.39B assistant tokens), 11.8 hours. Val loss (EMA) 0.827 (ppl 2.29) |
| **Preference (APO)** | APO-zero, β = 0.05, on on-policy pairs. 20,000 prompts drawn from the SFT mixture, four samples each from the SFT EMA (temperature 0.8, top-p 0.95). The chosen reply is the best-judged sample, or the dataset's reference reply when no sample passed; pairs need a reward margin of at least 3. Prompts that offer tools have 6,794 call pairs and 6,794 answer-directly pairs. 3,697 loop-stop pairs prefer ending the turn exactly where a repeat begins over one more repeat. 22,517 pairs, one epoch, 32 pairs per step, 704 steps, AdamW 1×10⁻⁶ cosine to 1×10⁻⁷, 10% warmup, clip 0.2, max length 16,512. Reference model: the SFT EMA. 1.8 hours |
| **Weights** | EMA (this repo's `model.safetensors`) |
| **Preference val (EMA)** | 903 held-out pairs: reward accuracy 79.0%, reward margin 1.33, chosen NLL 0.56 per token |

`preference_config.json`, `validation.json`, `sft_config.json` and `sft_validation.json` hold the exact settings and held-out results. `preference_data.json` describes the pair pack: prompt selection, sampling, judging, and the composition by source and label.

---

## Limitations

- **Scale**: 803M is a research / edge model, not a frontier system
---

## Model family

| Model | Stage | Hub |
|-------|-------|-----|
| **Senryu** | **SFT + on-policy APO (chat, reasoning, tools)** | **this repo** |
| Demo | ZeroGPU Space (chat) | [`kerzgrr/senryu-demo`](https://huggingface.co/spaces/kerzgrr/senryu-demo) |
| Haiku | Previous generation (~655M) | [`kerzgrr/Haiku`](https://huggingface.co/kerzgrr/Haiku) |
| Haiku-base | Previous generation, pretrain | [`kerzgrr/Haiku-base`](https://huggingface.co/kerzgrr/Haiku-base) |

---

## Citation

```bibtex
@misc{senryu2026,
  title={Senryu: An 803M Hybrid Gated DeltaNet-2 + Gated-MLA Reasoning Model},
  author={kerzgrr},
  year={2026},
  url={https://huggingface.co/kerzgrr/Senryu}
}
```

---

<div align="center">

*Five, seven, five — about people this time.*

</div>