---
license: apache-2.0
language:
  - en
tags:
  - text-generation
  - causal-lm
  - pytorch
  - prompter
  - user-simulator
  - synthetic-data
  - chat
  - hybrid
  - gated-deltanet-2
  - parallax-attention
  - n-gram-memory
  - senryu-2
pipeline_tag: text-generation
library_name: tiny_gdn
datasets:
  - allenai/WildChat-4.8M
  - HuggingFaceTB/smoltalk
  - HuggingFaceTB/smoltalk2
  - HuggingFaceH4/ultrachat_200k
  - NousResearch/Hermes-3-Dataset
  - QuixiAI/SystemChat-2.0
  - Open-Orca/SlimOrca
  - glaiveai/glaive-function-calling-v2
  - OpenAssistant/oasst2
  - ise-uiuc/Magicoder-Evol-Instruct-110K
  - bigcode/self-oss-instruct-sc2-exec-filter-50k
  - nvidia/OpenCodeReasoning
  - nvidia/When2Call
  - interstellarninja/hermes_reasoning_tool_use
  - interstellarninja/tool-use-multiturn-reasoning
  - allenai/tulu-3-sft-personas-instruction-following
  - HuggingFaceH4/no_robots
---

<div align="center">

# PromptTercet-0.5B

### A 501M model that writes the user's side of a chat

[![Model](https://img.shields.io/badge/Model-501M_core_params-blue)](.)
[![Memory](https://img.shields.io/badge/N--gram_memory-1.09B_params-blue)](.)
[![License](https://img.shields.io/badge/License-Apache_2.0-green.svg)](https://www.apache.org/licenses/LICENSE-2.0)
[![Demo](https://img.shields.io/badge/Demo-ZeroGPU_Space-orange.svg)](https://huggingface.co/spaces/KerzgrAI/prompttercet-0.5b-demo)

*By [KerzgrAI](https://huggingface.co/KerzgrAI)*

</div>

---

## What this is

Chat models write replies. **PromptTercet-0.5B** writes the questions: give it a conversation and it asks the next user turn, give it nothing and it opens a new one. A system prompt, the one the assistant would see, steers what it asks about.

## Uses

- **Synthetic chat data.** Pair it with any assistant model: PromptTercet asks, the assistant answers, turn after turn. `inference.py` does this with any OpenAI-compatible API.
- **Prompt pools.** Sample thousands of distinct, realistic user requests to seed SFT, preference or RL data.
- **User simulation.** Drive a chatbot through multi-turn conversations to test, demo or evaluate it.
- **Follow-up suggestions.** Propose what a user might ask next in a chat app.

Try it in the [demo Space](https://huggingface.co/spaces/KerzgrAI/prompttercet-0.5b-demo).

---

## Run

```bash
pip install torch safetensors tokenizers huggingface_hub
hf download KerzgrAI/PromptTercet-0.5B inference.py --local-dir .
python inference.py
```

| Command | What it does |
|---|---|
| `python inference.py` | It asks, you answer as the assistant |
| `python inference.py --system "You are a travel agent."` | The same, steered by a system prompt |
| `python inference.py --conversation chat.json` | Prints the next question for a saved chat |
| `python inference.py --count 1000 --output prompts.jsonl` | 1,000 distinct opening prompts |
| `python inference.py --count 200 --turns 3 --output chats.jsonl --answer-url https://openrouter.ai/api/v1 --answer-model openai/gpt-4o-mini` | 200 three-turn chats, answered by the API (key from `OPENAI_API_KEY`) |

Batch mode resamples questions that are cut off or repeat an opening, and redoes answers that hit `--answer-max-tokens`. Each output line is `{"messages": [...]}` with ordinary roles. `python inference.py --help` lists every flag.

`inference.py` downloads the weights, n-gram tables, tokenizer and `tiny_gdn/`, and installs `flash-linear-attention` from git at a pinned commit, so Git must be on `PATH`. A CUDA GPU is required.

## Prompt format

ChatML with the Senryu-2 tokenizer. The chat is rendered as usual (system, user, assistant) and generation continues from an open `<|im_start|>user\n`. Assistant turns it reads start with `<|no_think|>\n`. In code, `tiny_gdn.chatml.encode_prompter_messages` builds the prompt; it takes roles from PromptTercet's side, its own turns as `assistant` and the replies as `user`.

---

## Model

| | |
|---|---|
| **Core parameters** | 500,915,856 (embeddings included) |
| **N-gram memory** | 1.09B table parameters: hashed 2- and 3-grams |
| **Layers** | 24: 6 × (3 Gated DeltaNet-2 + 1 Parallax attention) |
| **Hidden / MLP** | 1,024 / SiTU-GLU 2,816 |
| **Attention** | 8 query and 4 KV heads of 128 |
| **Vocab** | 65,536 BPE, tied embeddings |
| **Context** | 32,768 |

The [Senryu-2-1B](https://huggingface.co/KerzgrAI/Senryu-2-1B) architecture at width 1,024. Weights are the EMA snapshot in bfloat16.

## Training

From random weights on the user turns of the chat datasets in this card's metadata. Only user turns carry loss; system prompts, assistant replies and tool results are context, with assistant reasoning removed. Duplicate conversations were dropped and single-question ones thinned, so it learns follow-ups and not only openers.

## Limitations

- **Small model.** Questions can drift, repeat earlier ones, or lose the thread in long chats.
- **Real user data.** It learned from real user prompts (WildChat among them), so it can write rude, unsafe or personal requests. Filter its output before training on it.
- **English** mostly, and a **CUDA GPU** is required.

Weights are Apache-2.0. Each training dataset keeps its own license; some carry attribution or non-commercial terms.