Spaces:
Running on Zero
Running on Zero
File size: 5,947 Bytes
4742559 d7b7e82 4742559 d7b7e82 4d72966 e945472 4d72966 e945472 4d72966 d7b7e82 4d72966 d7b7e82 4d72966 d7b7e82 4d72966 d7b7e82 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 | ---
title: Highlight Generator
emoji: 🐠
colorFrom: red
colorTo: gray
sdk: gradio
sdk_version: 6.28.0
python_version: '3.12'
app_file: app.py
pinned: false
license: mit
short_description: Extractive highlight generator for FIRE 2026 SciHigh
suggested_hardware: cpu-basic
models:
- sentence-transformers/all-MiniLM-L6-v2
tags:
- summarization
- extractive-summarization
- scientific-highlights
- fire-2026
- scikit-learn
---
# SciHigh 2026: Supervised Extractive Research-Highlight Generator
Submission to **FIRE 2026 SciHigh, Subtask 1** (research highlight generation from MixSub abstracts).
The system runs entirely on CPU. Open the **App** tab to try it on your own abstract or on a CSV file.
## Output modes (App tab)
| Mode | What it does | Can it invent facts? | Validation ROUGE-L | Avg. words |
|---|---|---|---|---|
| FIRE submission | Top-ranked whole sentences (the submitted system) | No | 24.76 | 64 |
| Concise | The same sentences, each compressed to about 18 words using word importance | No | 24.28 | 50 |
| LLM rewrite | `Qwen/Qwen2.5-1.5B-Instruct` rewrites the abstract into short highlights, guided by the ranked sentences; fact-checked | Checked | 20.20 on a 20-abstract sample* | about 50 |
| Fine-tuned model | BART fine-tuned on MixSub (`SciHigh_2026_Abstractive_FineTune.ipynb`, published as `Akash212/scihigh-bart-base`); fact-checked | Checked | see the notebook | about 50 |
*On the same 20 random validation abstracts, FIRE mode scored 21.83 and Concise 21.24 ROUGE-L.
Author highlights in MixSub average 52 words. The fact check rejects a generated highlight that has a number, name or acronym
absent from the abstract, or too many content words absent from it, and puts the Concise highlight in its place.
## Method
Sentence ranking is shared by all modes; the task is treated as sentence ranking, not text generation.
| Stage | What happens |
|---|---|
| 1. Pre-processing | Clean the abstract and split it into sentences (splitter written for MixSub's spaced periods) |
| 2. Hand-built features (31) | Position, length, lexical density, discourse cue counts, TF-IDF centrality, TextRank |
| 3. Embedding features (36) | `all-MiniLM-L6-v2`: similarity to the abstract centroid, to a highlight-style prototype, to other sentences, a retrieval feature from the 10 nearest training abstracts, and a 32-dim SVD projection |
| 4. Scoring | Five classifiers: LinearSVC, Ridge, LogReg, MLP, ExtraTrees |
| 5. Fusion | Reciprocal Rank Fusion of the five rankings within each abstract (k = 60) |
| 6. Decoding | Top sentences under a 110-word budget, at most 3 sentences, MMR redundancy threshold 0.85; original order restored; reporting frames ("In this study, we found that ...") removed |
Training labels come from a greedy ROUGE-1+2 F1 oracle over the 10,000 training abstracts (61,775 sentences).
## Results (MixSub-SciHigh validation set, 1,985 abstracts)
| System | ROUGE-1 | ROUGE-2 | ROUGE-L |
|---|---|---|---|
| This model (5-model fusion) | 38.12 | 14.13 | 24.76 |
| FIRE 2025 winner (Pegasus), reported | - | - | 23.45 |
Single members (decoded with budget 110, 3 sentences, MMR 0.55; fast stemmed ROUGE-L):
| Model | ROUGE-L |
|---|---|
| LinearSVC | 23.47 |
| Ridge | 23.47 |
| LogReg | 23.45 |
| MLP | 23.40 |
| ExtraTrees | 23.25 |
Every output word is copied from the source abstract, so the system cannot hallucinate facts or numbers.
The trade-off is that it cannot paraphrase the way authors often do in their own highlights.
## Use it from Python
```python
# pip install scikit-learn==1.8.0 "numpy>=2" pandas sentence-transformers huggingface_hub
from huggingface_hub import snapshot_download
import sys
path = snapshot_download("Akash212/Highlight_Generator", repo_type="space")
sys.path.insert(0, path)
from scihigh_model import SciHighExtractor
model = SciHighExtractor.from_pretrained(path)
print(model.predict(["<paste an abstract here>"])[0])
print(model.predict(["<abstract>"], budget=90, maxs=4)) # optional decoder overrides
print(model.concise(["<abstract>"], target=15)[0]) # Concise mode: compressed highlights
```
## Files
| File | Content |
|---|---|
| `app.py` | Gradio test UI (single abstract, batch CSV, per-sentence score table) |
| `scihigh_model.py` | Pre-processing, features, decoder and the `SciHighExtractor` class |
| `config.json` | Ensemble members, decoder settings, feature order, validation scores, library versions |
| `clf_*.joblib` | The five fitted scikit-learn models |
| `tfidf.joblib`, `svd.joblib` | TF-IDF vectoriser and 32-dim SVD fitted on training sentences |
| `proto.npy`, `train_centroids*.npy`, `train_highlight_emb*.npy` | Embedding prototype and the retrieval index (training abstracts and their gold highlights) |
| `concise.py` | Sentence compression (reporting-frame removal, word-importance-guided phrase dropping) |
| `generators.py` | LLM rewriter, fine-tuned summariser loader and the fact check |
| `highlight_lexicon.json` | Highlight log-odds lexicon (18,646 words) learned from the training data, used as word importance |
| `train_export.py` | Re-trains the ranking model and writes these files |
| `SciHigh_2026_Abstractive_FineTune.ipynb` | Kaggle notebook that fine-tunes BART on MixSub and publishes it for the Fine-tuned mode |
| `SciHigh_2026_CPU_Hybrid.ipynb` | Full experiment notebook (16 models, ensemble selection, decoder tuning, error analysis) |
Files larger than 9.5 MB are stored as byte chunks (`name.part00.ext`, `name.part01.ext`, ...);
`scihigh_model.py` joins them automatically when loading.
## Reproducibility
Trained with Python 3.11.15, scikit-learn 1.8.0, numpy 2.4.4, pandas 3.0.2.
The `.joblib` files must be loaded with **scikit-learn 1.8.0**. They are pickle files, so Hugging Face
shows a security notice for them; load them only from a source you trust.
The training data (MixSub-SciHigh, provided by the FIRE 2026 organisers) is not included in this repository.
|