Instructions to use Lythri/Lythri-Wink-2B-pt with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use Lythri/Lythri-Wink-2B-pt with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="Lythri/Lythri-Wink-2B-pt") messages = [ { "role": "user", "content": [ {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"}, {"type": "text", "text": "What animal is on the candy?"} ] }, ] pipe(text=messages)# pip install -U transformers accelerate # Load model directly from transformers import AutoProcessor, AutoModelForMultimodalLM processor = AutoProcessor.from_pretrained("Lythri/Lythri-Wink-2B-pt") model = AutoModelForMultimodalLM.from_pretrained("Lythri/Lythri-Wink-2B-pt", device_map="auto") messages = [ { "role": "user", "content": [ {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"}, {"type": "text", "text": "What animal is on the candy?"} ] }, ] inputs = processor.apply_chat_template( messages, add_generation_prompt=True, tokenize=True, return_dict=True, return_tensors="pt", ).to(model.device) outputs = model.generate(**inputs, max_new_tokens=256) print(processor.decode(outputs[0][inputs["input_ids"].shape[-1]:])) - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use Lythri/Lythri-Wink-2B-pt with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "Lythri/Lythri-Wink-2B-pt" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "Lythri/Lythri-Wink-2B-pt", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker
docker model run hf.co/Lythri/Lythri-Wink-2B-pt
- SGLang
How to use Lythri/Lythri-Wink-2B-pt with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "Lythri/Lythri-Wink-2B-pt" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "Lythri/Lythri-Wink-2B-pt", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "Lythri/Lythri-Wink-2B-pt" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "Lythri/Lythri-Wink-2B-pt", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }' - Docker Model Runner
How to use Lythri/Lythri-Wink-2B-pt with Docker Model Runner:
docker model run hf.co/Lythri/Lythri-Wink-2B-pt
Hugging Face | ModelScope | Technical Report
Introduction
Lythri-Wink-2B-pt is the pre-trained base model behind Lythri-Wink-2B. It has 2.52B total parameters and 1.48B activated parameters.
Lythri-Wink-2B-pt is structurally sheared from google/gemma-4-E2B and recovered through teacher distillation on approximately 813M tokens at an 8K training context length. It is intended for continued pre-training, supervised fine-tuning and researchβnot direct assistant-style conversation.
Architecture
| Gemma4-E2B | Lythri-Wink-2B-pt | |
|---|---|---|
| Total parameters | 4.63B | 2.52B |
| Activated parameters | 2.28B | 1.48B |
| Decoder layers | 35 | 31 |
| PLE dimension | 256 | 128 |
The FFN width is reduced to approximately 60% of the original model. The language-model hidden size and attention-head structure are retained.
Distillation Recovery
At the final validation point before export (step 1500):
| Metric | Value |
|---|---|
| Validation KL | 0.3241 |
| Student CE | 2.552 |
| Teacher CE | 2.270 |
The released checkpoint was exported at step 1550, corresponding to approximately 812.6M training tokens.
Base-Model Benchmarks
| Benchmark | Gemma 3 1B PT | Lythri-Wink-2B-pt |
|---|---|---|
| Knowledge | ||
| MMLU | 26.50 | 38.50 |
| ARC-C | 38.40 | 40.10 |
| ARC-E | 73.00 | 65.99 |
| Reasoning | ||
| HellaSwag | 62.30 | 57.45 |
| Commonsense | ||
| PIQA | 73.80 | 71.60 |
| BoolQ | 63.20 | 64.98 |
| SocialIQA | 48.90 | 42.43 |
| WinoGrande | 58.20 | 58.33 |
Lythri-Wink-2B-pt is evaluated without a chat template and with a BOS token prepended, Lythri-Wink-2B-pt acheives similar intelligence compare Gemma3-1B-pt, while holding a larger knowlege scale and a more permissive license.
Quickstart
from transformers import AutoModelForCausalLM, AutoTokenizer
import torch
model_path = "Lythri/Lythri-Wink-2B-pt"
tokenizer = AutoTokenizer.from_pretrained(model_path)
model = AutoModelForCausalLM.from_pretrained(
model_path,
dtype=torch.bfloat16,
device_map="auto",
)
prompt = "Once upon a time"
inputs = tokenizer(
prompt,
return_tensors="pt",
add_special_tokens=True,
).to(model.device)
with torch.inference_mode():
output = model.generate(
**inputs,
max_new_tokens=128,
do_sample=True,
temperature=0.8,
top_p=0.95,
)
print(tokenizer.decode(output[0], skip_special_tokens=True))
Training Your Own Model
If You dislike our instruct model Lythri/Lythri-Wink-2B, you could train a friend from our pt model!
First, prepare a JSONL dataset with one text field per sample:
{"text": "Your training text goes here."}
Here's an example about the code:
from datasets import load_dataset
from transformers import (
AutoModelForCausalLM,
AutoTokenizer,
DataCollatorForLanguageModeling,
Trainer,
TrainingArguments,
)
import torch
model_id = "Lythri/Lythri-Wink-2B-pt"
tokenizer = AutoTokenizer.from_pretrained(model_id)
tokenizer.add_bos_token = True
tokenizer.pad_token = tokenizer.eos_token
model = AutoModelForCausalLM.from_pretrained(
model_id,
dtype=torch.bfloat16,
device_map="auto",
)
model.config.use_cache = False
dataset = load_dataset("json", data_files="train.jsonl", split="train")
dataset = dataset.map(
lambda x: tokenizer(
x["text"],
truncation=True,
max_length=2048,
add_special_tokens=True,
),
remove_columns=dataset.column_names,
)
trainer = Trainer(
model=model,
args=TrainingArguments(
output_dir="./wink-2b-output",
per_device_train_batch_size=1,
gradient_accumulation_steps=16,
learning_rate=2e-5,
num_train_epochs=1,
bf16=True,
gradient_checkpointing=True,
logging_steps=10,
save_steps=200,
),
train_dataset=dataset,
data_collator=DataCollatorForLanguageModeling(tokenizer, mlm=False),
)
trainer.train()
trainer.save_model("./wink-2b-output")
tokenizer.save_pretrained("./wink-2b-output")
Important: Keep the BOS token enabled when evaluating or fine-tuning this model.
Intended Use
- Continued pre-training
- Domain adaptation
- Supervised fine-tuning
- Research on structured pruning and distillation
For direct conversation, use Lythri/Lythri-Wink-2B.
Limitations
- This is a base model and is not instruction-tuned.
- The model inherits limitations and biases from its base model, teacher and training data.
- It can produce inaccurate or inappropriate content.
License
Lythri-Wink-2B-pt is built on Gemma 4 and released under the Apache License 2.0.
Citation
@techreport{li2026lythri,
title = {Lythri Technical Report},
author = {Li, Jiawen},
year = {2026},
institution = {Zenodo},
doi = {10.5281/zenodo.23179311},
url = {https://doi.org/10.5281/zenodo.23179311}
}
Support
The whole training process is self-funded. If you like our model, please click a free like β that means a lot to me as a student!
- Downloads last month
- -