File size: 4,802 Bytes
87b1da9 e662719 87b1da9 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 | ---
library_name: transformers
base_model: facebook/bart-large
pipeline_tag: summarization
tags:
- bart
- title-generation
- scientific-text
- summarization
- scihigh-2026
- fire-2026
---
# SciHigh 2026 Task 2 — BART-large Title Generation Model
This repository contains a fine-tuned **BART-large** model for **Task 2: Title Generation from Abstracts** of the SciHigh 2026 shared task at FIRE 2026.
The model takes a scientific paper abstract as input and generates a concise research-paper title.
## Base Model
* Model: `facebook/bart-large`
* Architecture: BART encoder-decoder Transformer
* Parameters: approximately 406 million
* Framework: Hugging Face Transformers
## Task
**Input:** Scientific paper abstract
**Output:** Generated scientific paper title
The model was fine-tuned for highly compressed abstractive generation, where the goal is to capture the central topic and contribution of an abstract in title form.
## Dataset
The model was fine-tuned on the **SpringerSSAT** dataset released for SciHigh 2026 Task 2.
Dataset split used:
* Training: 2,778 abstract-title pairs
* Validation: 347 abstract-title pairs
* Test: 348 abstracts with masked reference titles
Only the training split was used for gradient updates. The validation set was used for evaluation and best-checkpoint selection.
## Preprocessing
* Maximum abstract length: 512 tokens
* Maximum target-title length: 64 tokens
* Input truncation: enabled
* Target truncation: enabled
* Dynamic batch padding using `DataCollatorForSeq2Seq`
* No task-specific instruction prefix was added
## Fine-Tuning Configuration
* Epochs: 3
* Learning rate: `3e-5`
* Weight decay: `0.01`
* Per-device training batch size: 1
* Per-device evaluation batch size: 1
* Gradient accumulation steps: 8
* Effective training batch size: 8
* Mixed-precision training: FP16 AMP
* Gradient checkpointing: enabled
* Evaluation strategy: once per epoch
* Checkpoint saving strategy: once per epoch
* Generation beam size during validation: 4
* Maximum generation length: 64
* Best-model selection metric: ROUGE-L
* Best model automatically restored after training
## Validation Results
| Epoch | Validation Loss | ROUGE-1 | ROUGE-2 | ROUGE-L | ROUGE-Lsum |
| ----- | --------------: | ----------: | ----------: | ----------: | ----------: |
| 1 | 2.282688 | 44.9926 | 21.3081 | 36.6377 | 36.6539 |
| 2 | **2.217401** | **45.7446** | **22.2135** | **37.6433** | **37.6641** |
| 3 | 2.242662 | 45.2856 | 21.8391 | 37.2244 | 37.2773 |
### Best Checkpoint
The best model was obtained at **Epoch 2**.
* ROUGE-1: **45.7446**
* ROUGE-2: **22.2135**
* ROUGE-L: **37.6433**
* ROUGE-Lsum: **37.6641**
* Validation loss: **2.217401**
A separate evaluation over all 347 validation examples reproduced these results.
These are validation-set results and are not official hidden-test scores.
## Example Usage
```python
from transformers import AutoTokenizer, AutoModelForSeq2SeqLM
model_name = "kckrish21/SciHigh2026-Task2-BART-large"
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForSeq2SeqLM.from_pretrained(model_name)
abstract = """
Insert a scientific paper abstract here.
"""
inputs = tokenizer(
abstract,
return_tensors="pt",
max_length=512,
truncation=True
)
generated_ids = model.generate(
**inputs,
max_length=64,
num_beams=4,
early_stopping=True
)
title = tokenizer.decode(
generated_ids[0],
skip_special_tokens=True
)
print(title)
```
## Test-Set Generation
For SciHigh 2026 test inference:
* Test examples: 348
* Generation batch size: 4
* Beam size: 4
* Maximum generation length: 64
* Early stopping: enabled
* Missing predictions: 0
* Empty predictions: 0
The reference test titles are masked by the task organizers, so no test-set metric is reported here.
## Intended Use
This model is intended for:
* scientific paper title generation from abstracts
* research on scientific text generation
* participation and reproducibility for SciHigh 2026 Task 2
Generated titles should be treated as model suggestions rather than authoritative paper titles.
## Limitations
The model was fine-tuned on a relatively small dataset of social-science research abstracts. Performance may differ for scientific domains or writing styles that are substantially different from the SpringerSSAT training distribution.
Like other neural text-generation systems, the model may generate wording that is incomplete, overly generic, or not fully supported by the abstract.
## Shared Task
**SciHigh 2026 — Research Highlight Generation from Scientific Papers**
**FIRE 2026**
Subtask 2: **Title Generation from Abstracts**
Ranking for the shared task is primarily based on ROUGE-L F1.
|