PromptTercet-0.5B

A 501M model that writes the user's side of a chat

By KerzgrAI

---

What this is

Chat models write replies. PromptTercet-0.5B writes the questions: give it a conversation and it asks the next user turn, give it nothing and it opens a new one. A system prompt, the one the assistant would see, steers what it asks about.

Uses

• Synthetic chat data. Pair it with any assistant model: PromptTercet asks, the assistant answers, turn after turn. inference.py does this with any OpenAI-compatible API.
• Prompt pools. Sample thousands of distinct, realistic user requests to seed SFT, preference or RL data.
• User simulation. Drive a chatbot through multi-turn conversations to test, demo or evaluate it.
• Follow-up suggestions. Propose what a user might ask next in a chat app.

Try it in the demo Space.

---

Run

pip install torch safetensors tokenizers huggingface_hub
hf download KerzgrAI/PromptTercet-0.5B inference.py --local-dir .
python inference.py

Command	What it does
python inference.py	It asks, you answer as the assistant
python inference.py --system "You are a travel agent."	The same, steered by a system prompt
python inference.py --conversation chat.json	Prints the next question for a saved chat
python inference.py --count 1000 --output prompts.jsonl	1,000 distinct opening prompts
python inference.py --count 200 --turns 3 --output chats.jsonl --answer-url https://openrouter.ai/api/v1 --answer-model openai/gpt-4o-mini	200 three-turn chats, answered by the API (key from OPENAI_API_KEY)

Batch mode resamples questions that are cut off or repeat an opening, and redoes answers that hit --answer-max-tokens. Each output line is {"messages": [...]} with ordinary roles. python inference.py --help lists every flag.

inference.py downloads the weights, n-gram tables, tokenizer and tiny_gdn/, and installs flash-linear-attention from git at a pinned commit, so Git must be on PATH. A CUDA GPU is required.

Prompt format

ChatML with the Senryu-2 tokenizer. The chat is rendered as usual (system, user, assistant) and generation continues from an open <|im_start|>user\n. Assistant turns it reads start with <|no_think|>\n. In code, tiny_gdn.chatml.encode_prompter_messages builds the prompt; it takes roles from PromptTercet's side, its own turns as assistant and the replies as user.

---

Model

Core parameters	500,915,856 (embeddings included)
N-gram memory	1.09B table parameters: hashed 2- and 3-grams
Layers	24: 6 × (3 Gated DeltaNet-2 + 1 Parallax attention)
Hidden / MLP	1,024 / SiTU-GLU 2,816
Attention	8 query and 4 KV heads of 128
Vocab	65,536 BPE, tied embeddings
Context	32,768

The Senryu-2-1B architecture at width 1,024. Weights are the EMA snapshot in bfloat16.

Training

From random weights on the user turns of the chat datasets in this card's metadata. Only user turns carry loss; system prompts, assistant replies and tool results are context, with assistant reasoning removed. Duplicate conversations were dropped and single-question ones thinned, so it learns follow-ups and not only openers.

Limitations

• Small model. Questions can drift, repeat earlier ones, or lose the thread in long chats.
• Real user data. It learned from real user prompts (WildChat among them), so it can write rude, unsafe or personal requests. Filter its output before training on it.
• English mostly, and a CUDA GPU is required.

Weights are Apache-2.0. Each training dataset keeps its own license; some carry attribution or non-commercial terms.