Upload README.md with huggingface_hub

README.md

@@ -1,62 +1,51 @@
----
-language:
-- sin
-- pli
-- san
-- en
-- ta
-- hi
-- bn
-- ar
-- fr
-- de
-license: cc-by-4.0
-tags:
-- language-identification
-- fasttext
-- continual-learning
-- leaf-surgery
-- sinhala
-- pali
-- sanskrit
-datasets:
-- script-langid/sinhala-pali-sanskrit-target
-metrics:
-- f1
-- accuracy
----
-
-# FastText Continual Leaf Surgery (11-Language Rehearsal)
-
-This model is a continual fine-tuning of Meta's `fastText LID-176` utilizing **Hierarchical Softmax Leaf Surgery**.
-
-## Architecture & Innovation
-- **Base Architecture**: Meta `fastText LID-176` (100-dim dense subword embeddings with Huffman tree hierarchical softmax).
-- **Leaf Surgery**: Instead of retraining the classification head from scratch (which shuffles the Huffman tree and causes catastrophic forgetting of 176 pre-trained languages), the original hierarchical softmax binary decision paths are preserved. The Sinhala (`si`) leaf node is surgically split into an internal decision node branching into modern Sinhala and canonical Pali, with Sanskrit adaptation.
-- **Continual Rehearsal**: Trained on the 11-language uniform rehearsal buffer (`train_11lang_uniform.csv`) balancing the 3 target languages in Sinhala script with 8 global and regional anchor languages (English, Tamil, Hindi, Bengali, Arabic, French, German, and Sanskrit in Devanagari).
-
-## Benchmark Performance
-
-| Benchmark | Macro F1 | Sinhala F1 | Pali F1 | Sanskrit F1 | Overall Accuracy |
-|---|:---:|:---:|:---:|:---:|:---:|
-| **WiLI-2018 (11-Lang)** | **0.9811** | 0.9766 | 0.9881 | 0.9895 | **98.00%** |
-| **CommonLID (11-Lang)** | **0.9486** | 0.9766 | 0.9881 | 0.9695 | **96.82%** |
-| **FLORES+ (11-Lang)** | **0.9395** | 0.9766 | 0.9881 | 0.9817 | **92.71%** |
-
-## How to Load in Python
-
-```python
-from data_pipeline.fasttext_continual.model import ContinualLID
-
-# Loads config.json, vocab.json, and weights.pt automatically from Hugging Face
-model = ContinualLID.from_pretrained("script-langid/fasttext-leaf-surgery-11lang")
-
-# Inference example:
-text = "නමෝ තස්ස භගවතෝ අරහතෝ සම්මා සම්බුද්ධස්ස"
-prediction = model.predict(text)
-print(prediction)  # 'pi' (Pali)
-```
-
-## Repository & Research
-- **GitHub**: [Sinhala-Script-Language-Identification-LangID-for-Sinhala-Pali-and-Sanskrit](https://github.com/Maleesha-K/Sinhala-Script-Language-Identification-LangID-for-Sinhala-Pali-and-Sanskrit)
-- **Research**: University of Moratuwa, Department of Computer Science & Engineering.
+---
+license: cc-by-nc-4.0
+language:
+- sin
+- pli
+- san
+- en
+- ta
+- hi
+- bn
+- ar
+- fr
+- de
+tags:
+- language-identification
+- fasttext
+- continual-learning
+- leaf-surgery
+- table-3-multilingual-rehearsal
+pipeline_tag: text-classification
+base_model: facebook/fasttext-language-identification
+---
+
+# fastText Continual Leaf Surgery: 11-Language Multilingual Rehearsal (Table 3)
+
+This model represents our **Continual Learning (Leaf Surgery)** architecture trained with the 11-language balanced rehearsal buffer (Table 3 in the research paper):
+> **"A Benchmark for Sinhala-Script Language Identification: Disambiguating Sinhala, Pali, and Sanskrit in Closed and Open World Contexts"**  
+> *University of Moratuwa, Department of Computer Science & Engineering*
+
+## 1. Architecture & Innovation
+- **Hierarchical Softmax Leaf Surgery**: The Sinhala (`si`) leaf node in the official Facebook `lid.176.bin` Huffman tree was split into Sinhala and Pali branches.
+- **Multilingual Rehearsal**: Fine-tuned on the balanced 11-language Aya dataset buffer (3 target languages in Sinhala script + 8 global/anchor languages: English, Tamil, Hindi, Bengali, Arabic, French, German, and Sanskrit Devanagari) to preserve cross-lingual representations.
+
+## 2. Benchmark Performance (11-Language Evaluation)
+- **WiLI-2018 (Macro-F1):** **0.9811** (Accuracy: 98.00%)
+- **CommonLID (Macro-F1):** **0.9486** (Accuracy: 96.82%)
+- **FLORES+ (Macro-F1):** **0.9395** (Accuracy: 92.71%)
+
+## 3. How to Load and Run Inference in Python
+```python
+from data_pipeline.fasttext_continual.model import ContinualLID
+
+# Loads weights.pt, config.json, and vocab.json automatically from Hugging Face
+model = ContinualLID.from_pretrained("script-langid/fasttext-leaf-surgery-11lang")
+
+# Inference example:
+text = "නමෝ තස්ස භගවතෝ අරහතෝ සම්මා සම්බුද්ධස්ස"
+prediction, score = model.predict(text)
+print(f"Language: {prediction}, Score: {score:.4f}")
+# Output: Language: pi, Score: 0.99...
+```