File size: 1,803 Bytes
3b68fb8 a1d0daa 3b68fb8 9929fa4 3b68fb8 a1d0daa 3b68fb8 a1d0daa 3b68fb8 a1d0daa 3b68fb8 a1d0daa 3b68fb8 a1d0daa 3b68fb8 a1d0daa 3b68fb8 a1d0daa | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 | ---
language: en
license: apache-2.0
library_name: transformers
pipeline_tag: summarization
tags:
- scihigh-2026
- research-highlight-generation
- scientific-summarization
- text2text-generation
- seq2seq
- bart
base_model: facebook/bart-large-cnn
---
# Origin Task 1 — BART-large-CNN
This is a full-parameter fine-tuned checkpoint of `facebook/bart-large-cnn` for **SciHigh-2026 Task 1**. It generates concise research highlights from scientific paper abstracts.
## Model Details
- **Parameters:** 406M trainable
- **Framework:** Transformers 4.57.6, PyTorch 2.13.0+cu130
- **Training Data:** 10,000 examples (SciHigh Task 1 MixSub-SciHigh split)
- **Hyperparameters:** 3 epochs, lr `3e-5`, batch size 32, max input/target lengths 1024/320, bfloat16, SDPA attention.
## Validation Results (1,985 examples)
| Metric | Score |
|---|---|
| ROUGE-1 | 0.3808 |
| ROUGE-2 | 0.1403 |
| ROUGE-L / Lsum | 0.2503 / 0.2505 |
| METEOR | 0.3114 |
| BERTScore F1 | 0.8721 |
## Inference
```python
import torch
from transformers import AutoModelForSeq2SeqLM, AutoTokenizer
model_id = "uthayamurthy/origin-task1-bart-large"
tokenizer = AutoTokenizer.from_pretrained(model_id, use_fast=True)
model = AutoModelForSeq2SeqLM.from_pretrained(
model_id,
dtype=torch.bfloat16,
attn_implementation="sdpa",
).to("cuda").eval()
abstract = "..."
inputs = tokenizer(abstract, return_tensors="pt", truncation=True, max_length=1024)
inputs = {name: value.to("cuda") for name, value in inputs.items()}
with torch.inference_mode():
generated = model.generate(
**inputs,
max_length=128,
min_length=0,
num_beams=4,
length_penalty=1.0,
no_repeat_ngram_size=3,
early_stopping=True,
)
print(tokenizer.decode(generated[0], skip_special_tokens=True))
``` |