Heroine2/lingala-news-topic-classifier

Lingala News Topic Classifier (fine-tuned AfroXLMR)

Classifies Lingala news articles into four topics: politics, health, sports and business.

It is Davlan/afro-xlmr-base (Alabi et al., 2022) fine-tuned on the Lingala part of MasakhaNEWS (Adelani et al., 2023). Built for the ALU NLP and Language Technologies summative project.

• Code, experiments and report: https://github.com/h-mutumwinka/Lingala_News_Classification
• Live demo: https://huggingface.co/spaces/Heroine2/lingala-news-topic-classifier

Results on the MasakhaNEWS Lingala test set (175 articles)

Model	Macro-F1	Weighted-F1	Accuracy
Majority class	0.182	0.416	0.571
TF-IDF (word + char) + linear SVM	0.822	0.852	0.857
BiLSTM + attention (Word2Vec init)	0.859 ± 0.026	0.888	0.893
Transformer trained from scratch	0.824 ± 0.041	0.876	0.880
This model (fine-tuned AfroXLMR)	0.911 ± 0.014	0.929	0.930

Mean ± standard deviation over 3 training seeds. Per-class F1: politics 0.94, health 0.93, sports 0.98, business 0.79.

How to use

from transformers import pipeline

classifier = pipeline("text-classification", model="Heroine2/lingala-news-topic-classifier")
classifier("Ba Léopards babetami lisusu fimbu na CAN 2019")

Input format used in training: headline + ". " + article text, truncated to the first 512 sub-word tokens.

Training

• Data: 608 training and 87 validation articles from VOA Lingala (MasakhaNEWS official splits).
• AdamW, learning rate 2e-5, weight decay 0.01, 10% linear warm-up, batch size 16, up to 10 epochs with early stopping (patience 3) on validation macro-F1, class-weighted cross-entropy, fp16 on a Colab T4 GPU.

Limitations

• Only four topics: articles about culture, religion or technology are still forced into one of them.
• Trained on one broadcaster (VOA Lingala), mostly 2018-2023 news (Ebola, COVID-19, DRC politics); other styles such as social media or Brazzaville spelling may be classified less reliably.
• Most errors are articles that mix politics with business or health, such as a parliament vote on a health emergency.
• The model is over-confident: it gives high probabilities even to many of its mistakes.

References

• Adelani, D. I., et al. (2023). MasakhaNEWS: News Topic Classification for African languages. IJCNLP-AACL 2023.
• Alabi, J. O., Adelani, D. I., Mosbach, M., & Klakow, D. (2022). Adapting Pre-trained Language Models to African Languages via Multilingual Adaptive Fine-Tuning. COLING 2022.