--- title: Highlight Generator emoji: 🐠 colorFrom: red colorTo: gray sdk: gradio sdk_version: 6.28.0 python_version: '3.12' app_file: app.py pinned: false license: mit short_description: Extractive highlight generator for FIRE 2026 SciHigh suggested_hardware: cpu-basic models: - sentence-transformers/all-MiniLM-L6-v2 tags: - summarization - extractive-summarization - scientific-highlights - fire-2026 - scikit-learn --- # SciHigh 2026: Supervised Extractive Research-Highlight Generator Submission to **FIRE 2026 SciHigh, Subtask 1** (research highlight generation from MixSub abstracts). The system runs entirely on CPU. Open the **App** tab to try it on your own abstract or on a CSV file. ## Output modes (App tab) | Mode | What it does | Can it invent facts? | Validation ROUGE-L | Avg. words | |---|---|---|---|---| | FIRE submission | Top-ranked whole sentences (the submitted system) | No | 24.76 | 64 | | Concise | The same sentences, each compressed to about 18 words using word importance | No | 24.28 | 50 | | LLM rewrite | `Qwen/Qwen2.5-1.5B-Instruct` rewrites the abstract into short highlights, guided by the ranked sentences; fact-checked | Checked | 20.20 on a 20-abstract sample* | about 50 | | Fine-tuned model | BART fine-tuned on MixSub (`SciHigh_2026_Abstractive_FineTune.ipynb`, published as `Akash212/scihigh-bart-base`); fact-checked | Checked | see the notebook | about 50 | *On the same 20 random validation abstracts, FIRE mode scored 21.83 and Concise 21.24 ROUGE-L. Author highlights in MixSub average 52 words. The fact check rejects a generated highlight that has a number, name or acronym absent from the abstract, or too many content words absent from it, and puts the Concise highlight in its place. ## Method Sentence ranking is shared by all modes; the task is treated as sentence ranking, not text generation. | Stage | What happens | |---|---| | 1. Pre-processing | Clean the abstract and split it into sentences (splitter written for MixSub's spaced periods) | | 2. Hand-built features (31) | Position, length, lexical density, discourse cue counts, TF-IDF centrality, TextRank | | 3. Embedding features (36) | `all-MiniLM-L6-v2`: similarity to the abstract centroid, to a highlight-style prototype, to other sentences, a retrieval feature from the 10 nearest training abstracts, and a 32-dim SVD projection | | 4. Scoring | Five classifiers: LinearSVC, Ridge, LogReg, MLP, ExtraTrees | | 5. Fusion | Reciprocal Rank Fusion of the five rankings within each abstract (k = 60) | | 6. Decoding | Top sentences under a 110-word budget, at most 3 sentences, MMR redundancy threshold 0.85; original order restored; reporting frames ("In this study, we found that ...") removed | Training labels come from a greedy ROUGE-1+2 F1 oracle over the 10,000 training abstracts (61,775 sentences). ## Results (MixSub-SciHigh validation set, 1,985 abstracts) | System | ROUGE-1 | ROUGE-2 | ROUGE-L | |---|---|---|---| | This model (5-model fusion) | 38.12 | 14.13 | 24.76 | | FIRE 2025 winner (Pegasus), reported | - | - | 23.45 | Single members (decoded with budget 110, 3 sentences, MMR 0.55; fast stemmed ROUGE-L): | Model | ROUGE-L | |---|---| | LinearSVC | 23.47 | | Ridge | 23.47 | | LogReg | 23.45 | | MLP | 23.40 | | ExtraTrees | 23.25 | Every output word is copied from the source abstract, so the system cannot hallucinate facts or numbers. The trade-off is that it cannot paraphrase the way authors often do in their own highlights. ## Use it from Python ```python # pip install scikit-learn==1.8.0 "numpy>=2" pandas sentence-transformers huggingface_hub from huggingface_hub import snapshot_download import sys path = snapshot_download("Akash212/Highlight_Generator", repo_type="space") sys.path.insert(0, path) from scihigh_model import SciHighExtractor model = SciHighExtractor.from_pretrained(path) print(model.predict([""])[0]) print(model.predict([""], budget=90, maxs=4)) # optional decoder overrides print(model.concise([""], target=15)[0]) # Concise mode: compressed highlights ``` ## Files | File | Content | |---|---| | `app.py` | Gradio test UI (single abstract, batch CSV, per-sentence score table) | | `scihigh_model.py` | Pre-processing, features, decoder and the `SciHighExtractor` class | | `config.json` | Ensemble members, decoder settings, feature order, validation scores, library versions | | `clf_*.joblib` | The five fitted scikit-learn models | | `tfidf.joblib`, `svd.joblib` | TF-IDF vectoriser and 32-dim SVD fitted on training sentences | | `proto.npy`, `train_centroids*.npy`, `train_highlight_emb*.npy` | Embedding prototype and the retrieval index (training abstracts and their gold highlights) | | `concise.py` | Sentence compression (reporting-frame removal, word-importance-guided phrase dropping) | | `generators.py` | LLM rewriter, fine-tuned summariser loader and the fact check | | `highlight_lexicon.json` | Highlight log-odds lexicon (18,646 words) learned from the training data, used as word importance | | `train_export.py` | Re-trains the ranking model and writes these files | | `SciHigh_2026_Abstractive_FineTune.ipynb` | Kaggle notebook that fine-tunes BART on MixSub and publishes it for the Fine-tuned mode | | `SciHigh_2026_CPU_Hybrid.ipynb` | Full experiment notebook (16 models, ensemble selection, decoder tuning, error analysis) | Files larger than 9.5 MB are stored as byte chunks (`name.part00.ext`, `name.part01.ext`, ...); `scihigh_model.py` joins them automatically when loading. ## Reproducibility Trained with Python 3.11.15, scikit-learn 1.8.0, numpy 2.4.4, pandas 3.0.2. The `.joblib` files must be loaded with **scikit-learn 1.8.0**. They are pickle files, so Hugging Face shows a security notice for them; load them only from a source you trust. The training data (MixSub-SciHigh, provided by the FIRE 2026 organisers) is not included in this repository.