| --- |
| language: en |
| license: apache-2.0 |
| library_name: transformers |
| pipeline_tag: summarization |
| tags: |
| - scihigh-2026 |
| - research-highlight-generation |
| - scientific-summarization |
| - text2text-generation |
| - seq2seq |
| - bart |
| base_model: facebook/bart-large-cnn |
| --- |
| |
| # Origin Task 1 — BART-large-CNN |
|
|
| This is a full-parameter fine-tuned checkpoint of `facebook/bart-large-cnn` for **SciHigh-2026 Task 1**. It generates concise research highlights from scientific paper abstracts. |
|
|
| ## Model Details |
| - **Parameters:** 406M trainable |
| - **Framework:** Transformers 4.57.6, PyTorch 2.13.0+cu130 |
| - **Training Data:** 10,000 examples (SciHigh Task 1 MixSub-SciHigh split) |
| - **Hyperparameters:** 3 epochs, lr `3e-5`, batch size 32, max input/target lengths 1024/320, bfloat16, SDPA attention. |
|
|
| ## Validation Results (1,985 examples) |
| | Metric | Score | |
| |---|---| |
| | ROUGE-1 | 0.3808 | |
| | ROUGE-2 | 0.1403 | |
| | ROUGE-L / Lsum | 0.2503 / 0.2505 | |
| | METEOR | 0.3114 | |
| | BERTScore F1 | 0.8721 | |
|
|
| ## Inference |
|
|
| ```python |
| import torch |
| from transformers import AutoModelForSeq2SeqLM, AutoTokenizer |
| |
| model_id = "uthayamurthy/origin-task1-bart-large" |
| tokenizer = AutoTokenizer.from_pretrained(model_id, use_fast=True) |
| model = AutoModelForSeq2SeqLM.from_pretrained( |
| model_id, |
| dtype=torch.bfloat16, |
| attn_implementation="sdpa", |
| ).to("cuda").eval() |
| |
| abstract = "..." |
| inputs = tokenizer(abstract, return_tensors="pt", truncation=True, max_length=1024) |
| inputs = {name: value.to("cuda") for name, value in inputs.items()} |
| |
| with torch.inference_mode(): |
| generated = model.generate( |
| **inputs, |
| max_length=128, |
| min_length=0, |
| num_beams=4, |
| length_penalty=1.0, |
| no_repeat_ngram_size=3, |
| early_stopping=True, |
| ) |
| print(tokenizer.decode(generated[0], skip_special_tokens=True)) |
| ``` |
