Instructions to use Ido-shraga/setswana-sentiment-afroxlmr-base with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use Ido-shraga/setswana-sentiment-afroxlmr-base with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-classification", model="Ido-shraga/setswana-sentiment-afroxlmr-base")# pip install -U transformers accelerate # Load model directly from transformers import AutoTokenizer, AutoModelForSequenceClassification tokenizer = AutoTokenizer.from_pretrained("Ido-shraga/setswana-sentiment-afroxlmr-base") model = AutoModelForSequenceClassification.from_pretrained("Ido-shraga/setswana-sentiment-afroxlmr-base", device_map="auto") - Notebooks
- Google Colab
- Kaggle
setswana-sentiment-afroxlmr-base
AfroXLMR-base fine-tuned for 3-class sentiment classification of Setswana (Tswana) tweets: Positive / Negative / Neutral, using the training recipe published by the dataset's authors.
Trained on the DSFSI Setswana Sentiment corpus — 3,555 Setswana Twitter posts (2021–2022) annotated by three native speakers (arXiv:2605.27239).
Results (held-out test split, 346 tweets, seed 42)
| Metric | Value |
|---|---|
| Accuracy | 0.500 |
| Macro-F1 | 0.486 |
Sibling models fine-tuned on the same data for comparison:
| Model | Test accuracy | Test macro-F1 |
|---|---|---|
| Ido-shraga/setswana-sentiment-mbert | 0.567 | 0.524 |
| Ido-shraga/setswana-sentiment-afriberta-large | 0.552 | 0.504 |
| Ido-shraga/setswana-sentiment-afroxlmr-large | 0.532 | 0.405 |
| this model (afro-xlmr-base) | 0.500 | 0.486 |
vs the paper's number: the DSFSI paper reports 53.6±1.2 macro-F1 (mean over 5 seeds) for afro-xlmr-base on this dataset — our single-seed run lands ~5 points below. Unlike the paper's protocol we did not average or tune over seeds; treat the gap as a reproduction finding, not a contradiction.
Training details
- Base model: Davlan/afro-xlmr-base
- Data:
dsfsi/setswana-sentimenttext→consensus_label(train 2,762 / validation 346 / test 346) - Recipe from the DSFSI paper (Appendix E): lr 5e-5, batch 32, up to 20 epochs with early stopping (patience 3 on dev macro-F1), max length 128, AdamW, linear decay, 10% warmup, seed 42
- Early stopping fired at epoch 10; best checkpoint by validation macro-F1; test split evaluated once at the end
- transformers 5.23.0 / torch 2.15.0
Usage
from transformers import pipeline
clf = pipeline("text-classification", model="Ido-shraga/setswana-sentiment-afroxlmr-base")
clf("setswana text here")
# [{'label': 'Negative', 'score': 0.87}]
Limitations
- Trained and evaluated on Twitter (X) posts from 2021–2022; may not transfer to other registers.
- Positive sentiment is under-represented in the corpus (~15%); the dominant confusion in the data is Negative vs Neutral, reflecting ironic phrasing common in Setswana commentary.
- Single-seed result; the paper's own protocol shows ±1.2 macro-F1 seed variance on this task.
Data license
The training data is released under the NOODL license (dsfsi/setswana-sentiment).
Citation
@misc{abdulmumin2026temporalsimultaneitypredictsannotation,
title={Temporal Simultaneity Predicts Annotation Quality in Sentiment Corpora},
author={Idris Abdulmumin and Mokgadi Penelope Matloga and Tadesse Destaw Belay and Botshelo Kondowe and Letlhogonolo Mohleleng and Hareaipha Nkopo Letsoalo and Shamsuddeen Hassan Muhammad and Vukosi Marivate},
year={2026},
eprint={2605.27239},
archivePrefix={arXiv},
url={https://arxiv.org/abs/2605.27239},
}
- Downloads last month
- 22
Model tree for Ido-shraga/setswana-sentiment-afroxlmr-base
Base model
Davlan/afro-xlmr-baseDataset used to train Ido-shraga/setswana-sentiment-afroxlmr-base
Paper for Ido-shraga/setswana-sentiment-afroxlmr-base
Evaluation results
- Test accuracy on DSFSI Setswana Sentiment (test split)test set self-reported0.500
- Test macro-F1 on DSFSI Setswana Sentiment (test split)test set self-reported0.486