Spaces:
Running on Zero
Running on Zero
|
Download README.md from Akash212/Highlight_Generator: direct link, hf CLI and curl.
- Browser
- Download file 5.95 kB
-
https://huggingface.co/spaces/Akash212/Highlight_Generator/resolve/main/README.md
- Command line
-
hf download hf://spaces/Akash212/Highlight_Generator/README.md
-
curl -L -o README.md https://huggingface.co/spaces/Akash212/Highlight_Generator/resolve/main/README.md
5.95 kB
| title: Highlight Generator | |
| emoji: 🐠 | |
| colorFrom: red | |
| colorTo: gray | |
| sdk: gradio | |
| sdk_version: 6.28.0 | |
| python_version: '3.12' | |
| app_file: app.py | |
| pinned: false | |
| license: mit | |
| short_description: Extractive highlight generator for FIRE 2026 SciHigh | |
| suggested_hardware: cpu-basic | |
| models: | |
| - sentence-transformers/all-MiniLM-L6-v2 | |
| tags: | |
| - summarization | |
| - extractive-summarization | |
| - scientific-highlights | |
| - fire-2026 | |
| - scikit-learn | |
| # SciHigh 2026: Supervised Extractive Research-Highlight Generator | |
| Submission to **FIRE 2026 SciHigh, Subtask 1** (research highlight generation from MixSub abstracts). | |
| The system runs entirely on CPU. Open the **App** tab to try it on your own abstract or on a CSV file. | |
| ## Output modes (App tab) | |
| | Mode | What it does | Can it invent facts? | Validation ROUGE-L | Avg. words | | |
| |---|---|---|---|---| | |
| | FIRE submission | Top-ranked whole sentences (the submitted system) | No | 24.76 | 64 | | |
| | Concise | The same sentences, each compressed to about 18 words using word importance | No | 24.28 | 50 | | |
| | LLM rewrite | `Qwen/Qwen2.5-1.5B-Instruct` rewrites the abstract into short highlights, guided by the ranked sentences; fact-checked | Checked | 20.20 on a 20-abstract sample* | about 50 | | |
| | Fine-tuned model | BART fine-tuned on MixSub (`SciHigh_2026_Abstractive_FineTune.ipynb`, published as `Akash212/scihigh-bart-base`); fact-checked | Checked | see the notebook | about 50 | | |
| *On the same 20 random validation abstracts, FIRE mode scored 21.83 and Concise 21.24 ROUGE-L. | |
| Author highlights in MixSub average 52 words. The fact check rejects a generated highlight that has a number, name or acronym | |
| absent from the abstract, or too many content words absent from it, and puts the Concise highlight in its place. | |
| ## Method | |
| Sentence ranking is shared by all modes; the task is treated as sentence ranking, not text generation. | |
| | Stage | What happens | | |
| |---|---| | |
| | 1. Pre-processing | Clean the abstract and split it into sentences (splitter written for MixSub's spaced periods) | | |
| | 2. Hand-built features (31) | Position, length, lexical density, discourse cue counts, TF-IDF centrality, TextRank | | |
| | 3. Embedding features (36) | `all-MiniLM-L6-v2`: similarity to the abstract centroid, to a highlight-style prototype, to other sentences, a retrieval feature from the 10 nearest training abstracts, and a 32-dim SVD projection | | |
| | 4. Scoring | Five classifiers: LinearSVC, Ridge, LogReg, MLP, ExtraTrees | | |
| | 5. Fusion | Reciprocal Rank Fusion of the five rankings within each abstract (k = 60) | | |
| | 6. Decoding | Top sentences under a 110-word budget, at most 3 sentences, MMR redundancy threshold 0.85; original order restored; reporting frames ("In this study, we found that ...") removed | | |
| Training labels come from a greedy ROUGE-1+2 F1 oracle over the 10,000 training abstracts (61,775 sentences). | |
| ## Results (MixSub-SciHigh validation set, 1,985 abstracts) | |
| | System | ROUGE-1 | ROUGE-2 | ROUGE-L | | |
| |---|---|---|---| | |
| | This model (5-model fusion) | 38.12 | 14.13 | 24.76 | | |
| | FIRE 2025 winner (Pegasus), reported | - | - | 23.45 | | |
| Single members (decoded with budget 110, 3 sentences, MMR 0.55; fast stemmed ROUGE-L): | |
| | Model | ROUGE-L | | |
| |---|---| | |
| | LinearSVC | 23.47 | | |
| | Ridge | 23.47 | | |
| | LogReg | 23.45 | | |
| | MLP | 23.40 | | |
| | ExtraTrees | 23.25 | | |
| Every output word is copied from the source abstract, so the system cannot hallucinate facts or numbers. | |
| The trade-off is that it cannot paraphrase the way authors often do in their own highlights. | |
| ## Use it from Python | |
| ```python | |
| # pip install scikit-learn==1.8.0 "numpy>=2" pandas sentence-transformers huggingface_hub | |
| from huggingface_hub import snapshot_download | |
| import sys | |
| path = snapshot_download("Akash212/Highlight_Generator", repo_type="space") | |
| sys.path.insert(0, path) | |
| from scihigh_model import SciHighExtractor | |
| model = SciHighExtractor.from_pretrained(path) | |
| print(model.predict(["<paste an abstract here>"])[0]) | |
| print(model.predict(["<abstract>"], budget=90, maxs=4)) # optional decoder overrides | |
| print(model.concise(["<abstract>"], target=15)[0]) # Concise mode: compressed highlights | |
| ``` | |
| ## Files | |
| | File | Content | | |
| |---|---| | |
| | `app.py` | Gradio test UI (single abstract, batch CSV, per-sentence score table) | | |
| | `scihigh_model.py` | Pre-processing, features, decoder and the `SciHighExtractor` class | | |
| | `config.json` | Ensemble members, decoder settings, feature order, validation scores, library versions | | |
| | `clf_*.joblib` | The five fitted scikit-learn models | | |
| | `tfidf.joblib`, `svd.joblib` | TF-IDF vectoriser and 32-dim SVD fitted on training sentences | | |
| | `proto.npy`, `train_centroids*.npy`, `train_highlight_emb*.npy` | Embedding prototype and the retrieval index (training abstracts and their gold highlights) | | |
| | `concise.py` | Sentence compression (reporting-frame removal, word-importance-guided phrase dropping) | | |
| | `generators.py` | LLM rewriter, fine-tuned summariser loader and the fact check | | |
| | `highlight_lexicon.json` | Highlight log-odds lexicon (18,646 words) learned from the training data, used as word importance | | |
| | `train_export.py` | Re-trains the ranking model and writes these files | | |
| | `SciHigh_2026_Abstractive_FineTune.ipynb` | Kaggle notebook that fine-tunes BART on MixSub and publishes it for the Fine-tuned mode | | |
| | `SciHigh_2026_CPU_Hybrid.ipynb` | Full experiment notebook (16 models, ensemble selection, decoder tuning, error analysis) | | |
| Files larger than 9.5 MB are stored as byte chunks (`name.part00.ext`, `name.part01.ext`, ...); | |
| `scihigh_model.py` joins them automatically when loading. | |
| ## Reproducibility | |
| Trained with Python 3.11.15, scikit-learn 1.8.0, numpy 2.4.4, pandas 3.0.2. | |
| The `.joblib` files must be loaded with **scikit-learn 1.8.0**. They are pickle files, so Hugging Face | |
| shows a security notice for them; load them only from a source you trust. | |
| The training data (MixSub-SciHigh, provided by the FIRE 2026 organisers) is not included in this repository. | |