Instructions to use Souvikbasur/roberta-base-news-ner with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use Souvikbasur/roberta-base-news-ner with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("token-classification", model="Souvikbasur/roberta-base-news-ner")# pip install -U transformers accelerate # Load model directly from transformers import AutoTokenizer, AutoModelForTokenClassification tokenizer = AutoTokenizer.from_pretrained("Souvikbasur/roberta-base-news-ner") model = AutoModelForTokenClassification.from_pretrained("Souvikbasur/roberta-base-news-ner", device_map="auto") - Notebooks
- Google Colab
- Kaggle
RoBERTa-base for News Domain Named Entity Recognition using Transformer Architecture
A FacebookAI/roberta-base model fine-tuned for Named Entity Recognition (NER) on political, business, defence, crime, and international news text, using a custom 17-tag entity schema.
This model was trained as part of a backbone comparison study (BERT vs. RoBERTa vs. ELECTRA vs. DeBERTa-v3) for an automated political news monitoring system. RoBERTa achieved the best overall micro F1-score (0.8788) among all four backbones evaluated.
Model Details
| Base model | FacebookAI/roberta-base |
| Task | Token Classification (Named Entity Recognition) |
| Language | English |
| Labels | 17 entity types (35 BIO tags + O) |
| Training framework | π€ Transformers Trainer (custom WeightedTrainer) |
| Max sequence length | 256 tokens |
| License | MIT |
Entity Schema
The model predicts the following 17 entity types in BIO format (B-TAG / I-TAG / O):
| Tag | Description | Tag | Description |
|---|---|---|---|
PER |
Person | ORG |
Organization |
LOC |
Location | FAC |
Facility |
DATE |
Date | TIME |
Time |
EVT |
Event | PROD |
Product |
MONEY |
Monetary value | PCT |
Percentage |
LAW |
Law / legal reference | NORP |
Nationality, religious, or political group |
LANG |
Language | WOA |
Work of Art |
QTY |
Quantity | ORD |
Ordinal Number |
CARD |
Cardinal Number |
Training Data
The model was trained on a manually annotated and audited dataset of political/business/defence/crime/international news sentences, originally collected from 40+ sources (Reuters, Al Jazeera, NPR, DW, BBC, Indian Express, and others) and annotated in Label Studio.
Key dataset characteristics:
- BIO-tagged spans across all 17 entity categories
- Cleaned via a multi-stage audit pipeline (overlap resolution, whitespace trimming, deduplication, majority-vote label correction)
- Split 80% train / 10% validation / 10% test (held-out test set never used during training, tuning, or checkpoint selection)
Evaluation Results
Evaluated on a held-out test set never seen during training, validation, or hyperparameter tuning.
Overall Test Set Performance
| Metric | Score |
|---|---|
| Precision | 0.8641 |
| Recall | 0.8941 |
| Micro F1-score | 0.8788 |
| Macro F1-score | 0.8134 |
| Weighted F1-score | 0.8799 |
The model achieved a micro F1-score of 87.88%, slightly exceeding the best validation F1-score of 87.39% β indicating strong generalization with no signs of overfitting.
Per-Entity F1-Scores
| High-Frequency | F1 | Medium-Frequency | F1 | Low-Frequency | F1 |
|---|---|---|---|---|---|
| PER | 0.9501 | ORG | 0.8404 | LAW | 0.7417 |
| LOC | 0.9425 | PROD | 0.8094 | EVT | 0.7407 |
| NORP | 0.9273 | TIME | 0.7831 | FAC | 0.6316 |
| PCT | 0.9055 | LANG | 0.7719 | WOA | 0.5732 |
| MONEY | 0.8869 | QTY | 0.7005 | ||
| ORD | 0.8816 | ||||
| CARD | 0.8791 | ||||
| DATE | 0.8627 |
Validation vs. Test Generalization
| Metric | Validation | Test |
|---|---|---|
| Precision | 0.8585 | 0.8641 |
| Recall | 0.8898 | 0.8941 |
| F1-score | 0.8739 | 0.8788 |
The test F1-score exceeds the validation F1-score by ~0.49 points, confirming the model generalizes well to unseen data.
Interpretation
Performance is strongest on high-frequency, well-bounded categories (PER, LOC, NORP, PCT, MONEY), all scoring near or above 0.90 F1. The weighted loss strategy (3Γ weight on rare tags) helped lift historically difficult classes like LAW, EVT, and FAC.
WOA (Work of Art) remains the most challenging category. This is attributed less to raw data sparsity and more to annotation noise and semantic boundary overlap with adjacent categories (e.g. titles, named works, and policy/product references being inconsistently distinguished) β a pattern that has held consistently across backbone comparisons on this dataset.
Intended Use
This model is intended for extracting structured entities from English-language political, business, defence, crime, and international news etc. text β e.g. for automated news monitoring, media analysis, or knowledge-base population pipelines.
It is not intended for:
- Non-English text
- Domains substantially different from news (e.g. legal contracts, medical records, social media slang)
- High-stakes automated decision-making without human review
How to Use
from transformers import pipeline
ner = pipeline(
"token-classification",
model="Souvikbasur/roberta-base-news-ner",
aggregation_strategy="simple",
)
result = ner("Narendra Modi met officials in New Delhi on Monday regarding the new defence policy.")
for entity in result:
print(f"{entity['entity_group']:6s} | {entity['word']:20s} | score={entity['score']:.3f}")
Or load the model and tokenizer directly:
from transformers import AutoTokenizer, AutoModelForTokenClassification
import torch
tokenizer = AutoTokenizer.from_pretrained("Souvikbasur/roberta-base-news-ner")
model = AutoModelForTokenClassification.from_pretrained("Souvikbasur/roberta-base-news-ner")
text = "The United Nations Security Council discussed the proposed law on climate change in Geneva."
inputs = tokenizer(text, return_tensors="pt")
with torch.no_grad():
outputs = model(**inputs)
predictions = torch.argmax(outputs.logits, dim=-1)
tokens = tokenizer.convert_ids_to_tokens(inputs["input_ids"][0])
for token, pred_id in zip(tokens, predictions[0]):
label = model.config.id2label[pred_id.item()]
if label != "O" and token not in tokenizer.all_special_tokens:
print(f"{token:15s} -> {label}")
Limitations
WOA(Work of Art) andFAC(Facility) categories have the lowest F1-scores (0.57 and 0.63 respectively) due to annotation boundary ambiguity in the training data, not just low support.- The model was trained and evaluated primarily on political/news-domain text; performance on other domains is not guaranteed.
- As with any NER model, performance may degrade on entity names, events, or terminology not well represented in the training distribution.
Citation
If you use this model, please reference the base model and this fine-tuned checkpoint:
Base model: FacebookAI/roberta-base (Liu et al., 2019 β RoBERTa: A Robustly Optimized BERT Pretraining Approach)
Fine-tuned checkpoint: Souvikbasur/roberta-base-news-ner
- Downloads last month
- 8
Model tree for Souvikbasur/roberta-base-news-ner
Base model
FacebookAI/roberta-base