Highlight_Generator / README.md
Akash212's picture
Upload 3 files
e945472 verified
|
Raw History Blame Contribute Delete
5.95 kB

A newer version of the Gradio SDK is available: 6.32.0

Upgrade
metadata
title: Highlight Generator
emoji: 🐠
colorFrom: red
colorTo: gray
sdk: gradio
sdk_version: 6.28.0
python_version: '3.12'
app_file: app.py
pinned: false
license: mit
short_description: Extractive highlight generator for FIRE 2026 SciHigh
suggested_hardware: cpu-basic
models:
  - sentence-transformers/all-MiniLM-L6-v2
tags:
  - summarization
  - extractive-summarization
  - scientific-highlights
  - fire-2026
  - scikit-learn

SciHigh 2026: Supervised Extractive Research-Highlight Generator

Submission to FIRE 2026 SciHigh, Subtask 1 (research highlight generation from MixSub abstracts). The system runs entirely on CPU. Open the App tab to try it on your own abstract or on a CSV file.

Output modes (App tab)

Mode What it does Can it invent facts? Validation ROUGE-L Avg. words
FIRE submission Top-ranked whole sentences (the submitted system) No 24.76 64
Concise The same sentences, each compressed to about 18 words using word importance No 24.28 50
LLM rewrite Qwen/Qwen2.5-1.5B-Instruct rewrites the abstract into short highlights, guided by the ranked sentences; fact-checked Checked 20.20 on a 20-abstract sample* about 50
Fine-tuned model BART fine-tuned on MixSub (SciHigh_2026_Abstractive_FineTune.ipynb, published as Akash212/scihigh-bart-base); fact-checked Checked see the notebook about 50

*On the same 20 random validation abstracts, FIRE mode scored 21.83 and Concise 21.24 ROUGE-L.

Author highlights in MixSub average 52 words. The fact check rejects a generated highlight that has a number, name or acronym absent from the abstract, or too many content words absent from it, and puts the Concise highlight in its place.

Method

Sentence ranking is shared by all modes; the task is treated as sentence ranking, not text generation.

Stage What happens
1. Pre-processing Clean the abstract and split it into sentences (splitter written for MixSub's spaced periods)
2. Hand-built features (31) Position, length, lexical density, discourse cue counts, TF-IDF centrality, TextRank
3. Embedding features (36) all-MiniLM-L6-v2: similarity to the abstract centroid, to a highlight-style prototype, to other sentences, a retrieval feature from the 10 nearest training abstracts, and a 32-dim SVD projection
4. Scoring Five classifiers: LinearSVC, Ridge, LogReg, MLP, ExtraTrees
5. Fusion Reciprocal Rank Fusion of the five rankings within each abstract (k = 60)
6. Decoding Top sentences under a 110-word budget, at most 3 sentences, MMR redundancy threshold 0.85; original order restored; reporting frames ("In this study, we found that ...") removed

Training labels come from a greedy ROUGE-1+2 F1 oracle over the 10,000 training abstracts (61,775 sentences).

Results (MixSub-SciHigh validation set, 1,985 abstracts)

System ROUGE-1 ROUGE-2 ROUGE-L
This model (5-model fusion) 38.12 14.13 24.76
FIRE 2025 winner (Pegasus), reported - - 23.45

Single members (decoded with budget 110, 3 sentences, MMR 0.55; fast stemmed ROUGE-L):

Model ROUGE-L
LinearSVC 23.47
Ridge 23.47
LogReg 23.45
MLP 23.40
ExtraTrees 23.25

Every output word is copied from the source abstract, so the system cannot hallucinate facts or numbers. The trade-off is that it cannot paraphrase the way authors often do in their own highlights.

Use it from Python

# pip install scikit-learn==1.8.0 "numpy>=2" pandas sentence-transformers huggingface_hub
from huggingface_hub import snapshot_download
import sys
path = snapshot_download("Akash212/Highlight_Generator", repo_type="space")
sys.path.insert(0, path)
from scihigh_model import SciHighExtractor

model = SciHighExtractor.from_pretrained(path)
print(model.predict(["<paste an abstract here>"])[0])
print(model.predict(["<abstract>"], budget=90, maxs=4))        # optional decoder overrides
print(model.concise(["<abstract>"], target=15)[0])             # Concise mode: compressed highlights

Files

File Content
app.py Gradio test UI (single abstract, batch CSV, per-sentence score table)
scihigh_model.py Pre-processing, features, decoder and the SciHighExtractor class
config.json Ensemble members, decoder settings, feature order, validation scores, library versions
clf_*.joblib The five fitted scikit-learn models
tfidf.joblib, svd.joblib TF-IDF vectoriser and 32-dim SVD fitted on training sentences
proto.npy, train_centroids*.npy, train_highlight_emb*.npy Embedding prototype and the retrieval index (training abstracts and their gold highlights)
concise.py Sentence compression (reporting-frame removal, word-importance-guided phrase dropping)
generators.py LLM rewriter, fine-tuned summariser loader and the fact check
highlight_lexicon.json Highlight log-odds lexicon (18,646 words) learned from the training data, used as word importance
train_export.py Re-trains the ranking model and writes these files
SciHigh_2026_Abstractive_FineTune.ipynb Kaggle notebook that fine-tunes BART on MixSub and publishes it for the Fine-tuned mode
SciHigh_2026_CPU_Hybrid.ipynb Full experiment notebook (16 models, ensemble selection, decoder tuning, error analysis)

Files larger than 9.5 MB are stored as byte chunks (name.part00.ext, name.part01.ext, ...); scihigh_model.py joins them automatically when loading.

Reproducibility

Trained with Python 3.11.15, scikit-learn 1.8.0, numpy 2.4.4, pandas 3.0.2. The .joblib files must be loaded with scikit-learn 1.8.0. They are pickle files, so Hugging Face shows a security notice for them; load them only from a source you trust.

The training data (MixSub-SciHigh, provided by the FIRE 2026 organisers) is not included in this repository.