Instructions to use Ayaka157/RetNet-0.5B-v21.3-chat with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use Ayaka157/RetNet-0.5B-v21.3-chat with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="Ayaka157/RetNet-0.5B-v21.3-chat", trust_remote_code=True)# pip install -U transformers accelerate # Load model directly from transformers import AutoModelForCausalLM model = AutoModelForCausalLM.from_pretrained("Ayaka157/RetNet-0.5B-v21.3-chat", trust_remote_code=True, device_map="auto") - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use Ayaka157/RetNet-0.5B-v21.3-chat with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "Ayaka157/RetNet-0.5B-v21.3-chat" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "Ayaka157/RetNet-0.5B-v21.3-chat", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker
docker model run hf.co/Ayaka157/RetNet-0.5B-v21.3-chat
- SGLang
How to use Ayaka157/RetNet-0.5B-v21.3-chat with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "Ayaka157/RetNet-0.5B-v21.3-chat" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "Ayaka157/RetNet-0.5B-v21.3-chat", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "Ayaka157/RetNet-0.5B-v21.3-chat" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "Ayaka157/RetNet-0.5B-v21.3-chat", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }' - Docker Model Runner
How to use Ayaka157/RetNet-0.5B-v21.3-chat with Docker Model Runner:
docker model run hf.co/Ayaka157/RetNet-0.5B-v21.3-chat
RetNet-0.5B v21.3-chat — post-trained chat model
Status: chat gate PASS; held-out criterion 2.5914 (seed 2.5898; +0.0016).
This model is the result of the 2026-11-12 post-training round: 8,000 steps of concise/IF SFT on 117,712 prompt-aligned rows, then 2,000 steps of low-LR replay repair with 20% general pretrain replay. It is the served-demo candidate, not a general-purpose model.
| Parameters | 504,179,896 (100% live) |
| Architecture | RetNet, 20 layers, width 1408, 22 heads, value/FFN 2816 |
| Vocabulary / context | 20,000 ByteLevel BPE / 2048 training context |
| Decay | per-head learned γ; GLA branch present with gla_rank=0 |
| Precision | model.safetensors stored fp32; load bf16 for inference |
| SHA256 (model.safetensors) | 40a919bca43dfad776203af463598c79cd0f340b40c39a592995e518250fc54e |
All checkpoints
This repo is the latest release of the family. The full training line — epoch 1-3, the repair round (v21.1 / v21.1-chat), long-context v21.2, this chat release, and the 16k / 8k research exports — is collected here:
Usage
from transformers import AutoModelForCausalLM, AutoTokenizer
tok = AutoTokenizer.from_pretrained("Ayaka157/RetNet-0.5B-v21.3-chat", trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(
"Ayaka157/RetNet-0.5B-v21.3-chat", trust_remote_code=True, torch_dtype="bfloat16")
The play page sends this default system preset: 你是一个说话简短直接的助手:每次回答控制在两句话内,先给结论,不解释推理过程,不列清单。
Use an explicit short-answer system prompt; without one the model still stops, but answers are longer.
Measured post-training gates (2026-11-12, 2×RTX PRO 6000)
Behaviour suite, 18 prompts, concise condition:
| Metric | Seed v21.1-chat | v21.3-chat |
|---|---|---|
| cap-hit | 33.3% | 0% |
| unclosed think | 5.6% | 0% |
| median reply | 107 tok | 63 tok |
No-system condition also 0% cap, 0% unclosed, median 70 tok.
Held-out criterion (5 files, 4,454,400 tokens, skypile excluded): 2.5914 vs seed 2.5898 (+0.0016) and C-end 2.4885. Chat-val is not a gate.
Instruction-following diagnostic: 4/10 (10 fixed items, concise). This is a diagnostic, not a capability claim; arithmetic/world knowledge remain weak.
Limitations
- Knowledge, arithmetic, code accuracy, and factual reliability remain those of a 0.5B model; the post-training improves reply shape and stopping, not knowledge.
- This checkpoint is not a long-context/retrieval model. The 16k retrieval result is reported separately and must not be inferred from this card.
- The 18-prompt suite is directional. A pass here is not a general human-likeness or safety claim.
Files
config.json, model.safetensors, modeling_retnet.py, retention_fast.py, fused_ce.py,
bpe_tokenizer.json, and tokenizer files. Code and analysis: https://github.com/Ayaka157/RetNet
Retrieval round (negative)
The 2026-11-12 γ-frozen answer-only retrieval run reached needle/item accuracy 1.0 but collapsed the 2k criterion to 4.56 (8k) / 6.65 (16k) with negative two-arm marginal. It failed the registered gate and is not released; this chat checkpoint does not contain that retrieval training.
- Downloads last month
- 94