| --- |
| library_name: transformers |
| base_model: google/pegasus-xsum |
| pipeline_tag: summarization |
| tags: |
| - pegasus |
| - title-generation |
| - scientific-text |
| - summarization |
| - scihigh-2026 |
| - fire-2026 |
| --- |
| |
| # SciHigh 2026 Task 2 — PEGASUS-XSum Title Generation Model |
|
|
| This repository contains a fine-tuned **PEGASUS-XSum** model for **Task 2: Title Generation from Abstracts** of the SciHigh 2026 shared task at FIRE 2026. |
|
|
| The model takes a scientific paper abstract as input and generates a concise research-paper title. |
|
|
| ## Base Model |
|
|
| * Model: `google/pegasus-xsum` |
| * Architecture: PEGASUS encoder-decoder Transformer |
| * Parameters: approximately 570 million |
| * Framework: Hugging Face Transformers |
|
|
| ## Task |
|
|
| **Input:** Scientific paper abstract |
| **Output:** Generated scientific paper title |
|
|
| PEGASUS is designed for abstractive summarization, making it suitable for highly compressed generation tasks such as generating a paper title from an abstract. |
|
|
| ## Dataset |
|
|
| The model was fine-tuned on the **SpringerSSAT** dataset released for SciHigh 2026 Task 2. |
|
|
| Dataset split used: |
|
|
| * Training: 2,778 abstract-title pairs |
| * Validation: 347 abstract-title pairs |
| * Test: 348 abstracts with masked reference titles |
|
|
| Only the training split was used for gradient updates. The validation set was used for evaluation and best-checkpoint selection. |
|
|
| ## Preprocessing |
|
|
| * Maximum abstract length: 512 tokens |
| * Maximum target-title length: 64 tokens |
| * Input truncation: enabled |
| * Target truncation: enabled |
| * Dynamic batch padding using `DataCollatorForSeq2Seq` |
| * No task-specific instruction prefix was added |
| * Abstracts were provided directly to PEGASUS |
|
|
| ## Fine-Tuning Configuration |
|
|
| * Epochs: 3 |
| * Learning rate: `3e-5` |
| * Weight decay: `0.01` |
| * Per-device training batch size: 1 |
| * Per-device evaluation batch size: 1 |
| * Gradient accumulation steps: 8 |
| * Effective training batch size: 8 |
| * Mixed-precision training: FP16 |
| * Gradient checkpointing: enabled |
| * Evaluation strategy: once per epoch |
| * Checkpoint saving strategy: once per epoch |
| * Generation beam size during validation: 4 |
| * Maximum generation length: 64 |
| * Best-model selection metric: ROUGE-L |
| * Best model automatically restored after training |
|
|
| ## Validation Results |
|
|
| | Epoch | Validation Loss | ROUGE-1 | ROUGE-2 | ROUGE-L | ROUGE-Lsum | |
| | ----- | --------------: | ----------: | ------: | ----------: | ----------: | |
| | 1 | 2.632621 | 42.9633 | 20.7970 | 36.6800 | 36.5897 | |
| | 2 | 2.586069 | 43.4483 | 21.3048 | 36.9716 | 36.8742 | |
| | 3 | **2.579055** | **43.6198** | 21.2562 | **37.0629** | **36.9731** | |
|
|
| ### Best Checkpoint |
|
|
| The best model was obtained at **Epoch 3**. |
|
|
| * ROUGE-1: **43.6198** |
| * ROUGE-2: **21.2562** |
| * ROUGE-L: **37.0629** |
| * ROUGE-Lsum: **36.9731** |
| * Validation loss: **2.579055** |
|
|
| A separate evaluation over all 347 validation examples reproduced these results. |
|
|
| These are validation-set results and are not official hidden-test scores. |
|
|
| ## Example Usage |
|
|
| ```python |
| from transformers import AutoTokenizer, AutoModelForSeq2SeqLM |
| |
| model_name = "kckrish21/SciHigh2026-Task2-PEGASUS-XSum" |
| |
| tokenizer = AutoTokenizer.from_pretrained(model_name) |
| model = AutoModelForSeq2SeqLM.from_pretrained(model_name) |
| |
| abstract = """ |
| Insert a scientific paper abstract here. |
| """ |
| |
| inputs = tokenizer( |
| abstract, |
| return_tensors="pt", |
| max_length=512, |
| truncation=True |
| ) |
| |
| generated_ids = model.generate( |
| **inputs, |
| max_length=64, |
| num_beams=4, |
| early_stopping=True |
| ) |
| |
| title = tokenizer.decode( |
| generated_ids[0], |
| skip_special_tokens=True |
| ) |
| |
| print(title) |
| ``` |
|
|
| ## Test-Set Generation |
|
|
| For SciHigh 2026 test inference: |
|
|
| * Test examples: 348 |
| * Generation batch size: 4 |
| * Beam size: 4 |
| * Maximum generation length: 64 |
| * Early stopping: enabled |
| * Missing predictions: 0 |
| * Empty predictions: 0 |
|
|
| The reference test titles are masked by the task organizers, so no test-set metric is reported here. |
|
|
| ## Intended Use |
|
|
| This model is intended for: |
|
|
| * scientific paper title generation from abstracts |
| * research on scientific text generation |
| * participation and reproducibility for SciHigh 2026 Task 2 |
|
|
| Generated titles should be treated as model suggestions rather than authoritative paper titles. |
|
|
| ## Limitations |
|
|
| The model was fine-tuned on a relatively small dataset of social-science research abstracts. Performance may differ for scientific domains or writing styles that are substantially different from the SpringerSSAT training distribution. |
|
|
| Like other neural text-generation systems, the model may generate titles that are incomplete, overly generic, or not fully supported by the source abstract. |
|
|
| ## Shared Task |
|
|
| **SciHigh 2026 — Research Highlight Generation from Scientific Papers** |
| **FIRE 2026** |
| Subtask 2: **Title Generation from Abstracts** |
|
|
| Ranking for the shared task is primarily based on ROUGE-L F1. |
|
|
