| --- |
| license: apache-2.0 |
| base_model: Qwen/Qwen2.5-1.5B-Instruct |
| tags: |
| - text-generation |
| - title-generation |
| - headline-generation |
| - lora |
| - qwen2 |
| - fine-tuned |
| language: |
| - en |
| pipeline_tag: text-generation |
| --- |
| |
| # HeadlineGPT |
|
|
| **HeadlineGPT** is a fine-tuned version of |
| [Qwen/Qwen2.5-1.5B-Instruct](https://huggingface.co/Qwen/Qwen2.5-1.5B-Instruct) |
| specialized for generating concise, engaging titles from source content. |
|
|
| It is designed for: |
|
|
| - News and article headlines |
| - Research and academic content |
| - Talks and presentations |
| - Social media posts |
| - Other short-form content that needs an attention-grabbing title |
|
|
| ## Model Details |
|
|
| | Property | Value | |
| |---|---| |
| | Base model | Qwen2.5-1.5B-Instruct | |
| | Fine-tuning | LoRA / PEFT | |
| | LoRA rank | 16 | |
| | LoRA alpha | 32 | |
| | LoRA dropout | 0.05 | |
| | Language | English | |
| | Training objective | Reward-weighted supervised fine-tuning | |
|
|
| ### Training |
|
|
| The model was trained on content–title pairs. |
|
|
| Instead of treating every training example equally, examples were weighted according to their associated engagement score. Higher-scoring examples therefore contribute more strongly to the training loss. |
|
|
| This approach is intended to encourage title characteristics associated with higher engagement while retaining the broader writing patterns present in the training data. |
|
|
| The model was trained for **2 epochs on a randomly sampled 25,000-example subset** of the larger dataset. |
|
|
| ## Usage |
|
|
| ```python |
| from transformers import AutoModelForCausalLM, AutoTokenizer |
| import torch |
| |
| model = AutoModelForCausalLM.from_pretrained( |
| "csankalp21/headlinegpt", |
| torch_dtype=torch.float16, |
| device_map="auto" |
| ) |
| |
| tokenizer = AutoTokenizer.from_pretrained( |
| "csankalp21/headlinegpt" |
| ) |
| |
| messages = [ |
| { |
| "role": "system", |
| "content": "You are an expert at writing highly engaging titles." |
| }, |
| { |
| "role": "user", |
| "content": ( |
| "Generate a high-engagement title for the following content:\n\n" |
| "<your content here>" |
| ) |
| } |
| ] |
| |
| text = tokenizer.apply_chat_template( |
| messages, |
| tokenize=False, |
| add_generation_prompt=True |
| ) |
| |
| inputs = tokenizer( |
| text, |
| return_tensors="pt" |
| ).to(model.device) |
| |
| with torch.no_grad(): |
| output = model.generate( |
| **inputs, |
| max_new_tokens=40, |
| temperature=0.7, |
| do_sample=True, |
| top_p=0.9, |
| repetition_penalty=1.1 |
| ) |
| |
| generated_tokens = output[0][inputs["input_ids"].shape[1]:] |
| |
| print( |
| tokenizer.decode( |
| generated_tokens, |
| skip_special_tokens=True |
| ) |
| ) |
