README: sumeng-summarizer-qwen3.5-4b

README.md

@@ -19,6 +19,7 @@ Small models that run on an ordinary CPU and take frequent, repetitive work away
 
 | Folder | What it does | Size |
 |---|---|---|
+| [`sumeng-summarizer-qwen3.5-4b/`](sumeng-summarizer-qwen3.5-4b/) | Writes SumEng's summaries of passages and messages: one plain paragraph within a length limit. Fine-tuned Qwen3.5-4B, run by llama.cpp. For non-commercial use (see its training data). | 4B parameters (2.8 GB, Q4_K_M) |
 | [`same-subject-r3/`](same-subject-r3/) | The same task as r2, trained on more kinds of text: textbooks, novels, chats, and pairs of summaries. Better than r2 on every kind it was tested on. For non-commercial use (see its training data). | 150M parameters |
 | [`same-subject-r2/`](same-subject-r2/) | Says whether a new passage or message continues the subject of the text before it. Used to cut a long stream (a book, a chat, meeting notes) into topics for a summary tree. | 150M parameters |
 | [`entity-link-r1/`](entity-link-r1/) | Says whether a name in a sentence refers to a given known entity (a person, place, thing). Used to link each mention in a stream of text to the right entity in a growing graph. | 150M parameters |
@@ -31,6 +32,35 @@ Made by **Carl Free**, an AI engineer who trains small, specialized models that
 
 ---
 
+# sumeng-summarizer-qwen3.5-4b
+
+[Qwen/Qwen3.5-4B](https://huggingface.co/Qwen/Qwen3.5-4B) fine-tuned to write the summaries SumEng asks for: given SumEng's prompt (a run of passages or messages, a length limit, and plain rules), one paragraph that says what the stretch is about, names the people and things, and keeps within the limit. It replaces Claude Haiku at SumEng's `basic` level; Claude stays as the fallback.
+
+## Results
+
+On a whole textbook it never saw (OpenStax Psychology 2e, 921 passages), it wrote all 246 passage summaries, each accepted on the first try (Claude Haiku: 53 of 299 replies too long and asked again). Searching that tree found the evidence for questions as well as searching the same tree summarized by Claude: 59% against 63% of detail questions with the tree walk alone, and 100% for both with SumEng's keyword index; 68% against 64% of the book's review questions. Judged side by side by Claude Sonnet on 30 prompts, its summaries scored 3.6 of 5 and Claude's about 4.1. About 10 seconds per summary on an M1 Pro (Metal).
+
+## Use
+
+```sh
+uvx --from huggingface_hub hf download freeideas/UsefulLocalModels --include "sumeng-summarizer-qwen3.5-4b/*" --local-dir models
+llama-server -m models/sumeng-summarizer-qwen3.5-4b/sumeng-summarizer-qwen3.5-4b.Q4_K_M.gguf --jinja --reasoning-budget 0 -c 16384
+```
+
+Send SumEng's prompt as the user message, with thinking off (`chat_template_kwargs: {"enable_thinking": false}`), temperature 0.3. SumEng's `sumeng.local.LocalModel` does this.
+
+## Training
+
+754 summaries that Claude (Haiku for passages, Sonnet for summaries of summaries) wrote and SumEng accepted while building trees on training texts only: OpenStax Introduction to Sociology 3e and U.S. History (CC BY-NC-SA 4.0), four public-domain novels and eight Sherlock Holmes stories, and six LoCoMo chats (CC BY-NC 4.0); 83 from other books and a chat for validation. LoRA rank 32 on the language layers with Unsloth, loss on the reply only, one epoch (validation loss 0.91), merged and quantized to Q4_K_M.
+
+**Licence:** the base model is Apache 2.0, but part of the training text is licensed for non-commercial use only, so use this model for non-commercial purposes.
+
+## Limits
+
+English only. Trained on SumEng's own prompt wording; other prompts may work less well. Not yet tried for summaries of summaries in a whole tree.
+
+---
+
 # same-subject-r3
 
 The successor to `same-subject-r2` below: the same model, input, files and use, retrained on more kinds of text. It answers **is the new text about the same subject as the earlier text?** and returns a probability.