README.md

---
language:
- fil
- eng
license: cc-by-4.0
library_name: transformers
tags:
- sentiment-analysis
- code-switching
- taglish
- filipino-nlp
- xlm-roberta
- lexiliksik
- thesis-model
---

# LexCAT: Lexicon-Enhanced Code-Switched Attention Transformer for Tagalog–English Sentiment Analysis

**Author**: Glenn Marcus D. Cinco  
**Institution**: Mapúa University  
**Thesis Title**: LexCAT: A Lexicon-Based Approach for Code-Switching Analysis with Transformers Using XLM-RoBERTa and LexiLiksik  
**Degree**: BS & MS in Computer Science  
**Date**: July 2025  

---

## 🎓 Abstract

LexCAT is a fine-tuned XLM-RoBERTa model enhanced with **LexiLiksik**, a cross-lingual, context-aware Tagalog–English sentiment lexicon developed to handle **intra-sentential sentiment shifts** in code-switched Taglish text. LexCAT achieves **84.31% validation accuracy** on the FiReCS dataset, significantly outperforming monolingual baselines (FilCon: 73.12%, SentiWordNet: 69.52%).

This model is the final output of a thesis that systematically addresses gaps in code-switched sentiment analysis through lexicon development, attention biasing, and real-world fine-tuning.

---

## 🔧 Model Architecture

- **Base Model**: `xlm-roberta-base`
- **Enhancement**: Integrated **LexiLiksik** via:
  - Soft constraints during fine-tuning
  - Attention weight adjustment for sentiment-relevant tokens
  - Metadata-aware pooling (POS, code-switching type)
- **Fine-tuning Dataset**: FiReCS (10,487 Filipino–English code-switched reviews)
- **Evaluation Dataset**: SentiTaglish Products and Services + FiReCS Test Set

---

## 📊 Performance

| Metric       | Value   |
|--------------|---------|
| Accuracy     | 84.31%  |
| F1-Score     | 0.8566  |
| Precision    | 0.8353  |
| Recall       | 0.8574  |
| Cohen’s κ    | 0.83 (from FiReCS annotators) |

> ✅ **Key Strength**: Correctly classifies contrastive phrases like *“Maganda pero expensive”* as **negative**.

---

## 📥 How to Use

```python
from transformers import AutoTokenizer, AutoModelForSequenceClassification
import torch

model_name = "your-hf-username/LexCAT-LexiLiksik-Final"
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForSequenceClassification.from_pretrained(model_name)

text = "sobrang lambot ng burger pero expensive tlga"
inputs = tokenizer(text, return_tensors="pt", truncation=True, padding=True)
outputs = model(**inputs)
predictions = torch.nn.functional.softmax(outputs.logits, dim=-1)
predicted_class = torch.argmax(predictions, dim=1).item()

# Class: 0 = Negative, 1 = Neutral, 2 = Positive
sentiment = ["Negative", "Neutral", "Positive"][predicted_class]
print(f"Predicted Sentiment: {sentiment}")
```
---

## 📚 Citation
```
@mastersthesis{cinco2025lexcat,
  author = {Cinco, Glenn Marcus D.},
  title = {LexCAT: A Lexicon-Based Approach for Code-Switching Analysis with Transformers Using XLM-RoBERTa and LexiLiksik},
  school = {Mapúa University},
  year = {2025},
  month = {July}
}
```