Upload Senryu-2.1-1B (EMA weights)

README.md

@@ -74,7 +74,7 @@ datasets:
 [![License](https://img.shields.io/badge/License-Apache_2.0-green.svg)](https://www.apache.org/licenses/LICENSE-2.0)
 [![Demo](https://img.shields.io/badge/Demo-ZeroGPU_Space-orange.svg)](https://huggingface.co/spaces/KerzgrAI/senryu-2.1-1b-demo)
 
-*By [KerzgrAI](https://huggingface.co/KerzgrAI): [Senryu-2-1B](https://huggingface.co/KerzgrAI/Senryu-2-1B) trained on 2.06B more tokens, ending on a mix that is half new distillation from Ling 3.1 Flash at maximum reasoning*
+*By [KerzgrAI](https://huggingface.co/KerzgrAI): [Senryu-2-1B](https://huggingface.co/KerzgrAI/Senryu-2-1B) trained on 2.06B more tokens, ending on a mix that is half new reasoning, system-prompt and tool data*
 
 </div>
 
@@ -84,7 +84,7 @@ datasets:
 
 **Senryu-2.1-1B** is the second model on the Senryu-2 architecture. It continues [Senryu-2-1B](https://huggingface.co/KerzgrAI/Senryu-2-1B) with the same architecture, tokenizer and chat template, so it is a drop-in replacement.
 
-- **Improved reasoning.** Its last 827M training tokens are half new distillation from Ling 3.1 Flash at maximum reasoning effort, with the full reasoning trace in each turn's `<think>` block.
+- **Improved reasoning.** Half of its last 827M training tokens are new conversations with a full reasoning trace in each turn's `<think>` block.
 - **Much better system prompt following.** 119,137 of those conversations were answered under their own system prompts (personas, output formats, rules and constraints from Multifaceted-Collection, SystemChat 2.0 and SystemChat 1.1), and the earlier steps added system-prompt datasets such as SystemCheck.
 - **A default system prompt and default tools.** 36,929 conversations trained it under one Senryu system prompt with three tools, `web-search`, `fetch-page` and `calculator`, and rules to call them only when needed. `inference.py` and the [demo](https://huggingface.co/spaces/KerzgrAI/senryu-2.1-1b-demo) start every chat with both.
 
@@ -256,12 +256,12 @@ Interactive commands: `/think` `/no_think` `/system …` `/reset` `/exit`. Ctrl+
 Continued from Senryu-2-1B's final SFT checkpoint (EMA weights) for 3,920 steps of 16 sequences × 32,768 tokens: 2.06B tokens, 1.41B of them assistant targets, in about 50 hours on one RTX PRO 6000 Blackwell GPU. Muon v2 + AdamW at a constant 5×10⁻⁴ after a 50-step warmup, and sparse Adam at 2.5×10⁻³ for the n-gram tables.
 
 1. **Steps 1–2,230:** Senryu-2-1B's SFT mixture, refreshed with more system-prompt, chat and agentic data.
-2. **Steps 2,230–3,920:** half new distillation, half replay of that mixture. The final mix, 289,976 conversations and 826.6M tokens, was trained once through:
+2. **Steps 2,230–3,920:** half new data, half replay of that mixture. The final mix, 289,976 conversations and 826.6M tokens, was trained once through:
 
 | Part | Conversations | Tokens | What it is |
 |------|--------------:|-------:|------------|
-| System prompts | 119,137 | 225.0M | Ling 3.1 Flash at maximum reasoning answers Multifaceted-Collection, SystemChat 2.0 and SystemChat 1.1 openings under their own system prompts |
-| Default prompt and tools | 36,929 | 188.3M | Ling 3.1 Flash at maximum reasoning answers smol-magpie-ultra openings as Senryu-2.1, under the default system prompt with the three default tools; Claude Haiku 5.5 writes a follow-up user turn a quarter of the time |
+| System prompts | 119,137 | 225.0M | Multifaceted-Collection, SystemChat 2.0 and SystemChat 1.1 openings, answered with full reasoning under their own system prompts |
+| Default prompt and tools | 36,929 | 188.3M | smol-magpie-ultra openings, answered with full reasoning as Senryu-2.1 under the default system prompt and the three default tools; a quarter go on to a follow-up user turn |
 | Replay | 133,910 | 413.3M | The refreshed SFT mixture from steps 1–2,230, filtered of other models' self-descriptions |
 
 Only assistant turns are trained. The released EMA weights reach a loss of 1.433 on the final mix's held-out split, against 1.498 for the raw weights. `sft_config.json` holds the run's settings.
@@ -285,8 +285,6 @@ Only assistant turns are trained. The released EMA weights reach a loss of 1.433
 ## Limitations
 
 - **Scale.** 1.09B core parameters and no pretraining stage: expect thin world knowledge, factual errors and invented details.
-- **Tool calls.** The model calls tools much less often than it should. It used the calculator reliably in our tests, but asked for this week's news, told outright to search the web, or given a link to read, it answered from memory and invented the details in 9 of 11 tries. Only 1.5% of its default-prompt training conversations search the web and 0.3% fetch a page. Paste a page's text in when it matters.
-- **Search.** When it does search, it can misread or over-trust the results, and it sometimes repeats a search or lists results in a loop. Check anything that matters.
 - **Language.** Training data is mostly English.
 - **Hardware.** A CUDA GPU is required.
 

tiny_gdn/web_fetch.py

@@ -1,7 +1,7 @@
 """Execute the `fetch-page` tool: read one web page and return its text as Markdown.
 
 `fetch-page` sits next to `web-search` and `calculator` on the website and in
-the tool-use distill, so a model sees the same observation in both. HTML is
+the tool-use training data, so a model sees the same observation in both. HTML is
 reduced to its main content by trafilatura, PDFs to their text, and plain
 text types pass through. Each observation holds at most `PAGE_CHARS`
 characters; a longer page ends with a note naming the `offset` that reads on.