hausmer/truha-gemma4

Truha — Ukrainian news satire LoRA (Gemma 4 E4B)

A LoRA adapter (r=16, alpha=32) for google/gemma-4-E4B-it that turns a seed topic into a short, meta-ironic satirical "news" item in Ukrainian — the truha register: punchline-driven fake-news headlines with a bitter aftertaste.

Trained (LoRA SFT) on a 46k-row Ukrainian corpus:

• 39,272 synthetic satirical news items and 1,722 Wikipedia-derived items — fully public in hausmer/truha-news-satire;
• 5,145 short audience comments from the public satirical news channel «Труха⚡️Україна» (comments were already public to the channel's subscribers). These carry no @handles or personal data, but because they are real user text they are not republished in the open dataset — so this model is not 100% reproducible from the public dataset alone.

Base model is Apache-2.0 licensed; this adapter is published under Apache-2.0 with attribution to Google DeepMind for the base model.

Usage

from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel

base = "google/gemma-4-E4B-it"
adapter = "hausmer/truha-gemma4"

tok = AutoTokenizer.from_pretrained(adapter)
model = AutoModelForCausalLM.from_pretrained(
    base, torch_dtype="bfloat16", device_map="cuda"
)
model = PeftModel.from_pretrained(model, adapter)

# NOTE: the trained system prompt is "Ти — TrueXAnеws." — the last "е"
# is CYRILLIC U+0435, not latin 'e'. The model was tuned on that exact
# string; using a pure-latin "TrueXAnеws" with 'e' U+0065 may degrade style.

messages = [
    {"role": "system", "content": "Ти — TrueXAnеws."},
    {"role": "user", "content": "Зроби новину в стилі трухи: <топик>"},
]
prompt = tok.apply_chat_template(messages, add_generation_prompt=True, tokenize=False)
inputs = tok(prompt, return_tensors="pt").to(model.device)

# Style-tuned sampling
out = model.generate(
    **inputs,
    do_sample=True, temperature=0.95, top_p=0.92,
    repetition_penalty=1.05, max_new_tokens=160,
)
print(tok.decode(out[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True))

Example

prompt: Зроби новину в стилі трухи: зафіксовано удар бпла по

output: 🔴 ЗАФІКСОВАНО удар БПЛА по рекламному щиту «Купуй квартиру в іпотеку». Реклама брехала — тепер правда.

Training

• Base: google/gemma-4-E4B-it (256K context, multimodal)
• PEFT 0.20 LoRA: r=16, alpha=32, dropout 0.05, target modules = all attention/MLP projections of the language model (q/k/v/o/gate/up/down_proj), 3 epochs, bf16
• Data: 46,139 rows — 39,272 synthetic truha_satire + 1,722 general_ua (public, in the open dataset) + 5,145 public-audience comment rows (not republished)
• Val: 2,066 held-out synthetic news items; eval loss ≈ 1.31

License

Apache-2.0. Weights derive from Google's Gemma 4 (Apache-2.0); attribution required per the base model license. Satirical content is fictional; the model writes parody, not journalism.