Replace with IndoBERT trained on the sanitised corpus (ICDEES 2026)

README.md

@@ -1,133 +1,48 @@
 ---
-title: IndoHoaxDetector
-emoji: 🌖
-colorFrom: pink
-colorTo: indigo
-sdk: gradio
-sdk_version: 5.49.1
-app_file: app.py
-pinned: false
+language: id
 license: mit
-short_description: Indonesian News Detector
-tags: ["nlp", "text-classification", "indonesian", "hoax-detection", "machine-learning", "fact-checking", "transformers", "tfidf"]
+base_model: indobenchmark/indobert-base-p1
+pipeline_tag: text-classification
+tags: [hoax-detection, indonesian, misinformation]
 ---
 
-# IndoHoaxDetector
+# IndoHoaxDetector: IndoBERT, sanitised mixed corpus
 
-A comprehensive machine learning system for detecting hoax-style news articles in Indonesian language. This project provides multiple models trained on linguistic features of Indonesian news to identify articles written in a style typical of hoaxes or fake news. The system analyzes writing patterns, sensationalism, and other stylistic indicators rather than checking factual accuracy.
+Binary hoax classifier (0 = factual, 1 = hoax) fine-tuned from `indobenchmark/indobert-base-p1`.
+It accompanies the ICDEES 2026 paper *IndoHoaxDetector: A Comparative Evaluation of Classical
+Machine Learning and IndoBERT for Indonesian Mixed Corpus Hoax Detection*.
+Code: https://github.com/theonegareth/IndoHoaxDetector
 
-## Features
+## Training data
 
-- **Multiple Models**: Logistic Regression, Linear SVM, Random Forest, Naive Bayes, and IndoBERT
-- **High Accuracy**: Best model (IndoBERT) achieves 99.89% accuracy on test data
-- **Web Interface**: Interactive Gradio app for easy prediction
-- **Lightweight Options**: TF-IDF based models for fast inference
-- **Transformer Support**: IndoBERT for state-of-the-art performance
+The training data is 62,934 Indonesian documents from news portals (CNN Indonesia, Detik.com,
+Kompas.com, Tempo.co), the TurnBackHoax fact-checker and the Komdigi hoax registry. Fact-checker
+verdict tags (`[SALAH]`, `[HOAKS]`, ...) and the fact-check template sections were removed
+before training. Left in place, they let a rule with no training reach 99.64% weighted F1.
+**Earlier versions of this repository were trained on the unsanitised text and reported 99.89%;
+those numbers reflected the verdict tags and are superseded.**
 
-## Model Comparison
+## Training
 
-All models were trained on 62,972 Indonesian news articles and evaluated on 12,595 held-out test samples:
+Input is the title and body, unstemmed, truncated to 128 tokens. Training ran for 3 epochs with
+batch size 16, AdamW at learning rate 1e-5, weight decay 0.01, 10% warm-up and fp16. The best
+epoch was chosen on a 10% validation split, with seeds 42, 1 and 2. Hardware was an NVIDIA RTX
+3050 Ti Laptop GPU (4 GB), and one training run took 2,190 s.
 
-| Model                  | Accuracy | Precision (Macro) | Recall (Macro) | F1 (Macro) | Type |
-|------------------------|----------|-------------------|----------------|------------|------|
-| **IndoBERT**          | **0.9989** | **0.9989**      | **0.9989**   | **0.9989** | Transformer |
-| Linear SVM            | 0.9819  | 0.9820          | 0.9817       | 0.9818    | TF-IDF + SVM |
-| Logistic Regression   | 0.9782  | 0.9787          | 0.9777       | 0.9781    | TF-IDF + LR |
-| Random Forest         | 0.9765  | 0.9768          | 0.9760       | 0.9764    | TF-IDF + RF |
-| Multinomial Naive Bayes| 0.9398 | 0.9414          | 0.9381       | 0.9393    | TF-IDF + NB |
+## Results (held-out 20%, stratified)
 
-## Model Details
-
-### IndoBERT (Recommended)
-- **Type**: Fine-tuned IndoBenchmark/indobert-base-p1
-- **Training**: 3 epochs, batch_size=16, learning_rate=2e-5, max_length=128
-- **Accuracy**: 99.89%
-- **Best For**: Highest accuracy, contextual understanding
-- **Requirements**: transformers, torch (GPU recommended)
-
-### TF-IDF Based Models
-- **Vectorizer**: TF-IDF (unigrams, max_features=5000, sublinear_tf=True)
-- **Preprocessing**: Lowercase, URL removal, punctuation, Indonesian stopwords, Sastrawi stemming
-- **Logistic Regression**: L2 regularization, max_iter=1000 (97.82% accuracy)
-- **Linear SVM**: C=1.0, max_iter=10000 (98.19% accuracy)
-- **Random Forest**: 100 trees, random_state=42 (97.65% accuracy)
-- **Naive Bayes**: Multinomial, alpha=1.0 (93.98% accuracy)
-
-## Usage
-
-### Web Interface
-Visit the Hugging Face Space to use the interactive app:
-1. Select a model from the dropdown
-2. Enter Indonesian news text
-3. Click "Detect"
-4. View the prediction and confidence
-
-### Local Usage
-To run locally:
-
-```bash
-pip install gradio scikit-learn transformers torch
-python app.py
-```
-
-### API Usage
-Load and use models programmatically:
-
-```python
-import pickle
-from transformers import AutoTokenizer, AutoModelForSequenceClassification
-import torch
-
-# For TF-IDF models (Logistic Regression, SVM, RF, NB)
-with open('logreg_model.pkl', 'rb') as f:
-    model = pickle.load(f)
-
-# Load vectorizer
-with open('tfidf_vectorizer.pkl', 'rb') as f:
-    vectorizer = pickle.load(f)
-
-text = "Your Indonesian news text here"
-X = vectorizer.transform([text])
-prediction = model.predict(X)  # 0 for FAKTA, 1 for HOAX
-
-# For IndoBERT
-tokenizer = AutoTokenizer.from_pretrained('indobert_model')
-model = AutoModelForSequenceClassification.from_pretrained('indobert_model')
-
-inputs = tokenizer(text, return_tensors="pt", truncation=True, max_length=128)
-with torch.no_grad():
-    outputs = model(**inputs)
-    prediction = torch.argmax(outputs.logits, dim=-1).item()
-```
-
-## Training Data
-
-- **Source**: TurnBackHoax fact-checks, Kompas/CekFakta.com articles
-- **Size**: 62,972 labeled articles (balanced HOAX/FAKTA)
-- **Preprocessing**: Indonesian text normalization, stemming, stopword removal
-- **Split**: 80% train, 20% test (stratified)
+| Metric | Value |
+|---|---|
+| Accuracy (3 seeds) | 99.71 ± 0.05% |
+| Macro-F1 | 99.71% |
+| Balanced accuracy | 99.71% |
+| ROC-AUC | 99.99% |
+| Weighted F1, unseen publishers (train CNN/Kompas/TurnBackHoax, test Detik/Tempo/Komdigi) | 84.40% |
 
 ## Limitations
 
-- **Stylistic Analysis Only**: Detects hoax-like writing style, not factual accuracy
-- **Language Specific**: Trained on Indonesian news patterns
-- **Domain Shift**: May perform differently on social media vs. formal news
-- **False Positives/Negatives**: Sensational legitimate news may be flagged as hoax
-- **Transformer Requirements**: IndoBERT needs GPU for training/inference
-
-## Ethical Considerations
-
-- **Responsible Use**: Augment human fact-checkers, not replace them
-- **Bias Awareness**: Training data may reflect source biases
-- **Transparency**: All models and code are open-source
-- **Misuse Prevention**: Clearly labeled as stylistic detector, not truth verifier
-
-## Contributing
-
-Contributions welcome! Open issues or PRs for improvements.
-
-## License
-
-MIT License - see LICENSE file for details
-
-Check out the configuration reference at https://huggingface.co/docs/hub/spaces-config-reference
+Labels come from each document's source rather than from per-document fact-checking, so the
+model partly learns publisher style. On unseen publishers weighted F1 falls by 15.3 points,
+below a linear SVM: the model transfers well to unseen news outlets but recognises only
+75.9% of hoaxes from an unseen hoax registry. The model was not trained on social-media posts and should not be applied to them
+without in-domain data.

indobert_model/config.json

@@ -1,33 +1,33 @@
-{
-  "_num_labels": 5,
-  "architectures": [
-    "BertForSequenceClassification"
-  ],
-  "attention_probs_dropout_prob": 0.1,
-  "classifier_dropout": null,
-  "directionality": "bidi",
-  "dtype": "float32",
-  "hidden_act": "gelu",
-  "hidden_dropout_prob": 0.1,
-  "hidden_size": 768,
-  "initializer_range": 0.02,
-  "intermediate_size": 3072,
-  "layer_norm_eps": 1e-12,
-  "max_position_embeddings": 512,
-  "model_type": "bert",
-  "num_attention_heads": 12,
-  "num_hidden_layers": 12,
-  "output_past": true,
-  "pad_token_id": 0,
-  "pooler_fc_size": 768,
-  "pooler_num_attention_heads": 12,
-  "pooler_num_fc_layers": 3,
-  "pooler_size_per_head": 128,
-  "pooler_type": "first_token_transform",
-  "position_embedding_type": "absolute",
-  "problem_type": "single_label_classification",
-  "transformers_version": "4.57.1",
-  "type_vocab_size": 2,
-  "use_cache": true,
-  "vocab_size": 50000
-}
+{
+  "_num_labels": 5,
+  "architectures": [
+    "BertForSequenceClassification"
+  ],
+  "attention_probs_dropout_prob": 0.1,
+  "classifier_dropout": null,
+  "directionality": "bidi",
+  "dtype": "float32",
+  "hidden_act": "gelu",
+  "hidden_dropout_prob": 0.1,
+  "hidden_size": 768,
+  "initializer_range": 0.02,
+  "intermediate_size": 3072,
+  "layer_norm_eps": 1e-12,
+  "max_position_embeddings": 512,
+  "model_type": "bert",
+  "num_attention_heads": 12,
+  "num_hidden_layers": 12,
+  "output_past": true,
+  "pad_token_id": 0,
+  "pooler_fc_size": 768,
+  "pooler_num_attention_heads": 12,
+  "pooler_num_fc_layers": 3,
+  "pooler_size_per_head": 128,
+  "pooler_type": "first_token_transform",
+  "position_embedding_type": "absolute",
+  "problem_type": "single_label_classification",
+  "transformers_version": "4.57.1",
+  "type_vocab_size": 2,
+  "use_cache": true,
+  "vocab_size": 50000
+}

indobert_model/logs/events.out.tfevents.1763960326.MSI.18903.0

@@ -1,3 +0,0 @@
-version https://git-lfs.github.com/spec/v1
-oid sha256:13d31c4f157768f65b29ea90da6f227967713bf6e63890456c16e99ed1b7b86a
-size 10393

indobert_model/model.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:307ae084e201e8867cd13ff4962636aa99a7604d25122fdf9d7c732e975b0452
+oid sha256:2f2b6b6077b2d32f7ab1d580224222cef02d9dff010d75c870a90dbcf7e51f3e
 size 497795072

indobert_model/special_tokens_map.json

@@ -1,7 +1,7 @@
-{
-  "cls_token": "[CLS]",
-  "mask_token": "[MASK]",
-  "pad_token": "[PAD]",
-  "sep_token": "[SEP]",
-  "unk_token": "[UNK]"
-}
+{
+  "cls_token": "[CLS]",
+  "mask_token": "[MASK]",
+  "pad_token": "[PAD]",
+  "sep_token": "[SEP]",
+  "unk_token": "[UNK]"
+}

indobert_model/tokenizer.json

@@ -6,16 +6,7 @@
     "strategy": "LongestFirst",
     "stride": 0
   },
-  "padding": {
-    "strategy": {
-      "Fixed": 128
-    },
-    "direction": "Right",
-    "pad_to_multiple_of": null,
-    "pad_id": 0,
-    "pad_type_id": 0,
-    "pad_token": "[PAD]"
-  },
+  "padding": null,
   "added_tokens": [
     {
       "id": 0,

indobert_model/tokenizer_config.json

@@ -1,58 +1,58 @@
-{
-  "added_tokens_decoder": {
-    "0": {
-      "content": "[PAD]",
-      "lstrip": false,
-      "normalized": false,
-      "rstrip": false,
-      "single_word": false,
-      "special": true
-    },
-    "1": {
-      "content": "[UNK]",
-      "lstrip": false,
-      "normalized": false,
-      "rstrip": false,
-      "single_word": false,
-      "special": true
-    },
-    "2": {
-      "content": "[CLS]",
-      "lstrip": false,
-      "normalized": false,
-      "rstrip": false,
-      "single_word": false,
-      "special": true
-    },
-    "3": {
-      "content": "[SEP]",
-      "lstrip": false,
-      "normalized": false,
-      "rstrip": false,
-      "single_word": false,
-      "special": true
-    },
-    "4": {
-      "content": "[MASK]",
-      "lstrip": false,
-      "normalized": false,
-      "rstrip": false,
-      "single_word": false,
-      "special": true
-    }
-  },
-  "clean_up_tokenization_spaces": true,
-  "cls_token": "[CLS]",
-  "do_basic_tokenize": true,
-  "do_lower_case": true,
-  "extra_special_tokens": {},
-  "mask_token": "[MASK]",
-  "model_max_length": 1000000000000000019884624838656,
-  "never_split": null,
-  "pad_token": "[PAD]",
-  "sep_token": "[SEP]",
-  "strip_accents": null,
-  "tokenize_chinese_chars": true,
-  "tokenizer_class": "BertTokenizer",
-  "unk_token": "[UNK]"
-}
+{
+  "added_tokens_decoder": {
+    "0": {
+      "content": "[PAD]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "1": {
+      "content": "[UNK]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "2": {
+      "content": "[CLS]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "3": {
+      "content": "[SEP]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "4": {
+      "content": "[MASK]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    }
+  },
+  "clean_up_tokenization_spaces": true,
+  "cls_token": "[CLS]",
+  "do_basic_tokenize": true,
+  "do_lower_case": true,
+  "extra_special_tokens": {},
+  "mask_token": "[MASK]",
+  "model_max_length": 1000000000000000019884624838656,
+  "never_split": null,
+  "pad_token": "[PAD]",
+  "sep_token": "[SEP]",
+  "strip_accents": null,
+  "tokenize_chinese_chars": true,
+  "tokenizer_class": "BertTokenizer",
+  "unk_token": "[UNK]"
+}

indobert_model/training/checkpoint-3149/config.json

@@ -1,33 +0,0 @@
-{
-  "_num_labels": 5,
-  "architectures": [
-    "BertForSequenceClassification"
-  ],
-  "attention_probs_dropout_prob": 0.1,
-  "classifier_dropout": null,
-  "directionality": "bidi",
-  "dtype": "float32",
-  "hidden_act": "gelu",
-  "hidden_dropout_prob": 0.1,
-  "hidden_size": 768,
-  "initializer_range": 0.02,
-  "intermediate_size": 3072,
-  "layer_norm_eps": 1e-12,
-  "max_position_embeddings": 512,
-  "model_type": "bert",
-  "num_attention_heads": 12,
-  "num_hidden_layers": 12,
-  "output_past": true,
-  "pad_token_id": 0,
-  "pooler_fc_size": 768,
-  "pooler_num_attention_heads": 12,
-  "pooler_num_fc_layers": 3,
-  "pooler_size_per_head": 128,
-  "pooler_type": "first_token_transform",
-  "position_embedding_type": "absolute",
-  "problem_type": "single_label_classification",
-  "transformers_version": "4.57.1",
-  "type_vocab_size": 2,
-  "use_cache": true,
-  "vocab_size": 50000
-}

indobert_model/training/checkpoint-3149/model.safetensors

@@ -1,3 +0,0 @@
-version https://git-lfs.github.com/spec/v1
-oid sha256:c13f7fba081ce22d68a7764710bf6a6c7a328e3e1adca74e35fec810bb86c067
-size 497795072

indobert_model/training/checkpoint-3149/optimizer.pt

@@ -1,3 +0,0 @@
-version https://git-lfs.github.com/spec/v1
-oid sha256:7ba43eaa26ab43815e4dca4d2095519467bf2bb47e29529f3c21c8faf9c82204
-size 995714443

indobert_model/training/checkpoint-3149/rng_state.pth

@@ -1,3 +0,0 @@
-version https://git-lfs.github.com/spec/v1
-oid sha256:15b6fd0ef06d2af26e25d6e95586490215b372cfac6bdba987fdcc8cca9f89d2
-size 14645

indobert_model/training/checkpoint-3149/scheduler.pt

@@ -1,3 +0,0 @@
-version https://git-lfs.github.com/spec/v1
-oid sha256:9a692fdba2188743a1f0b8150a1afef797a26e1c622d4b613bb30cdba4de6f4c
-size 1465

indobert_model/training/checkpoint-3149/special_tokens_map.json

@@ -1,7 +0,0 @@
-{
-  "cls_token": "[CLS]",
-  "mask_token": "[MASK]",
-  "pad_token": "[PAD]",
-  "sep_token": "[SEP]",
-  "unk_token": "[UNK]"
-}

indobert_model/training/checkpoint-3149/tokenizer_config.json

@@ -1,58 +0,0 @@
-{
-  "added_tokens_decoder": {
-    "0": {
-      "content": "[PAD]",
-      "lstrip": false,
-      "normalized": false,
-      "rstrip": false,
-      "single_word": false,
-      "special": true
-    },
-    "1": {
-      "content": "[UNK]",
-      "lstrip": false,
-      "normalized": false,
-      "rstrip": false,
-      "single_word": false,
-      "special": true
-    },
-    "2": {
-      "content": "[CLS]",
-      "lstrip": false,
-      "normalized": false,
-      "rstrip": false,
-      "single_word": false,
-      "special": true
-    },
-    "3": {
-      "content": "[SEP]",
-      "lstrip": false,
-      "normalized": false,
-      "rstrip": false,
-      "single_word": false,
-      "special": true
-    },
-    "4": {
-      "content": "[MASK]",
-      "lstrip": false,
-      "normalized": false,
-      "rstrip": false,
-      "single_word": false,
-      "special": true
-    }
-  },
-  "clean_up_tokenization_spaces": true,
-  "cls_token": "[CLS]",
-  "do_basic_tokenize": true,
-  "do_lower_case": true,
-  "extra_special_tokens": {},
-  "mask_token": "[MASK]",
-  "model_max_length": 1000000000000000019884624838656,
-  "never_split": null,
-  "pad_token": "[PAD]",
-  "sep_token": "[SEP]",
-  "strip_accents": null,
-  "tokenize_chinese_chars": true,
-  "tokenizer_class": "BertTokenizer",
-  "unk_token": "[UNK]"
-}

indobert_model/training/checkpoint-3149/trainer_state.json

@@ -1,85 +0,0 @@
-{
-  "best_global_step": 3149,
-  "best_metric": 0.9922191345772132,
-  "best_model_checkpoint": "/home/user/Machine Learning/IndoHoaxDetector/indobert_model/training/checkpoint-3149",
-  "epoch": 1.0,
-  "eval_steps": 500,
-  "global_step": 3149,
-  "is_hyper_param_search": false,
-  "is_local_process_zero": true,
-  "is_world_process_zero": true,
-  "log_history": [
-    {
-      "epoch": 0.15878056525881232,
-      "grad_norm": 6.066575527191162,
-      "learning_rate": 1.8943579972478037e-05,
-      "loss": 0.1413,
-      "step": 500
-    },
-    {
-      "epoch": 0.31756113051762463,
-      "grad_norm": 2.5585596561431885,
-      "learning_rate": 1.788504287075262e-05,
-      "loss": 0.079,
-      "step": 1000
-    },
-    {
-      "epoch": 0.476341695776437,
-      "grad_norm": 6.151716232299805,
-      "learning_rate": 1.6826505769027206e-05,
-      "loss": 0.0669,
-      "step": 1500
-    },
-    {
-      "epoch": 0.6351222610352493,
-      "grad_norm": 0.012287363409996033,
-      "learning_rate": 1.576796866730179e-05,
-      "loss": 0.0569,
-      "step": 2000
-    },
-    {
-      "epoch": 0.7939028262940616,
-      "grad_norm": 0.09763805568218231,
-      "learning_rate": 1.4709431565576376e-05,
-      "loss": 0.0528,
-      "step": 2500
-    },
-    {
-      "epoch": 0.952683391552874,
-      "grad_norm": 5.14717435836792,
-      "learning_rate": 1.3650894463850958e-05,
-      "loss": 0.0486,
-      "step": 3000
-    },
-    {
-      "epoch": 1.0,
-      "eval_accuracy": 0.9922191345772132,
-      "eval_loss": 0.03707018122076988,
-      "eval_runtime": 83.2394,
-      "eval_samples_per_second": 151.311,
-      "eval_steps_per_second": 9.467,
-      "step": 3149
-    }
-  ],
-  "logging_steps": 500,
-  "max_steps": 9447,
-  "num_input_tokens_seen": 0,
-  "num_train_epochs": 3,
-  "save_steps": 500,
-  "stateful_callbacks": {
-    "TrainerControl": {
-      "args": {
-        "should_epoch_stop": false,
-        "should_evaluate": false,
-        "should_log": false,
-        "should_save": true,
-        "should_training_stop": false
-      },
-      "attributes": {}
-    }
-  },
-  "total_flos": 3313686408967680.0,
-  "train_batch_size": 16,
-  "trial_name": null,
-  "trial_params": null
-}

indobert_model/training/checkpoint-3149/training_args.bin

@@ -1,3 +0,0 @@
-version https://git-lfs.github.com/spec/v1
-oid sha256:6bf0c0a22bf8643019429801930b8f01cdf82327a398b9e59c1324ef729e2b13
-size 5905

indobert_model/training/checkpoint-6298/config.json

@@ -1,33 +0,0 @@
-{
-  "_num_labels": 5,
-  "architectures": [
-    "BertForSequenceClassification"
-  ],
-  "attention_probs_dropout_prob": 0.1,
-  "classifier_dropout": null,
-  "directionality": "bidi",
-  "dtype": "float32",
-  "hidden_act": "gelu",
-  "hidden_dropout_prob": 0.1,
-  "hidden_size": 768,
-  "initializer_range": 0.02,
-  "intermediate_size": 3072,
-  "layer_norm_eps": 1e-12,
-  "max_position_embeddings": 512,
-  "model_type": "bert",
-  "num_attention_heads": 12,
-  "num_hidden_layers": 12,
-  "output_past": true,
-  "pad_token_id": 0,
-  "pooler_fc_size": 768,
-  "pooler_num_attention_heads": 12,
-  "pooler_num_fc_layers": 3,
-  "pooler_size_per_head": 128,
-  "pooler_type": "first_token_transform",
-  "position_embedding_type": "absolute",
-  "problem_type": "single_label_classification",
-  "transformers_version": "4.57.1",
-  "type_vocab_size": 2,
-  "use_cache": true,
-  "vocab_size": 50000
-}

indobert_model/training/checkpoint-6298/model.safetensors

@@ -1,3 +0,0 @@
-version https://git-lfs.github.com/spec/v1
-oid sha256:be74eb579e30894b49ed55edf4c95de3196873a9bfa5b36bcee44c9dbd70cff2
-size 497795072

indobert_model/training/checkpoint-6298/optimizer.pt

@@ -1,3 +0,0 @@
-version https://git-lfs.github.com/spec/v1
-oid sha256:f712c1deabd1dd3687723f696eb73afdb9ae230f38489c8ef20f1841a0aef42e
-size 995714443

indobert_model/training/checkpoint-6298/rng_state.pth

@@ -1,3 +0,0 @@
-version https://git-lfs.github.com/spec/v1
-oid sha256:89d203a89830c5846a8f2be717f1889093bb9690135b7469beb281351a8a4dfa
-size 14645

indobert_model/training/checkpoint-6298/scheduler.pt

@@ -1,3 +0,0 @@
-version https://git-lfs.github.com/spec/v1
-oid sha256:48c004f15fd5ee7a2922596c91957017c15a22e51ff1e3e5f451ee212dab0f2b
-size 1465

indobert_model/training/checkpoint-6298/special_tokens_map.json

@@ -1,7 +0,0 @@
-{
-  "cls_token": "[CLS]",
-  "mask_token": "[MASK]",
-  "pad_token": "[PAD]",
-  "sep_token": "[SEP]",
-  "unk_token": "[UNK]"
-}

indobert_model/training/checkpoint-6298/tokenizer_config.json

@@ -1,58 +0,0 @@
-{
-  "added_tokens_decoder": {
-    "0": {
-      "content": "[PAD]",
-      "lstrip": false,
-      "normalized": false,
-      "rstrip": false,
-      "single_word": false,
-      "special": true
-    },
-    "1": {
-      "content": "[UNK]",
-      "lstrip": false,
-      "normalized": false,
-      "rstrip": false,
-      "single_word": false,
-      "special": true
-    },
-    "2": {
-      "content": "[CLS]",
-      "lstrip": false,
-      "normalized": false,
-      "rstrip": false,
-      "single_word": false,
-      "special": true
-    },
-    "3": {
-      "content": "[SEP]",
-      "lstrip": false,
-      "normalized": false,
-      "rstrip": false,
-      "single_word": false,
-      "special": true
-    },
-    "4": {
-      "content": "[MASK]",
-      "lstrip": false,
-      "normalized": false,
-      "rstrip": false,
-      "single_word": false,
-      "special": true
-    }
-  },
-  "clean_up_tokenization_spaces": true,
-  "cls_token": "[CLS]",
-  "do_basic_tokenize": true,
-  "do_lower_case": true,
-  "extra_special_tokens": {},
-  "mask_token": "[MASK]",
-  "model_max_length": 1000000000000000019884624838656,
-  "never_split": null,
-  "pad_token": "[PAD]",
-  "sep_token": "[SEP]",
-  "strip_accents": null,
-  "tokenize_chinese_chars": true,
-  "tokenizer_class": "BertTokenizer",
-  "unk_token": "[UNK]"
-}

indobert_model/training/checkpoint-6298/trainer_state.json

@@ -1,136 +0,0 @@
-{
-  "best_global_step": 6298,
-  "best_metric": 0.993092497022628,
-  "best_model_checkpoint": "/home/user/Machine Learning/IndoHoaxDetector/indobert_model/training/checkpoint-6298",
-  "epoch": 2.0,
-  "eval_steps": 500,
-  "global_step": 6298,
-  "is_hyper_param_search": false,
-  "is_local_process_zero": true,
-  "is_world_process_zero": true,
-  "log_history": [
-    {
-      "epoch": 0.15878056525881232,
-      "grad_norm": 6.066575527191162,
-      "learning_rate": 1.8943579972478037e-05,
-      "loss": 0.1413,
-      "step": 500
-    },
-    {
-      "epoch": 0.31756113051762463,
-      "grad_norm": 2.5585596561431885,
-      "learning_rate": 1.788504287075262e-05,
-      "loss": 0.079,
-      "step": 1000
-    },
-    {
-      "epoch": 0.476341695776437,
-      "grad_norm": 6.151716232299805,
-      "learning_rate": 1.6826505769027206e-05,
-      "loss": 0.0669,
-      "step": 1500
-    },
-    {
-      "epoch": 0.6351222610352493,
-      "grad_norm": 0.012287363409996033,
-      "learning_rate": 1.576796866730179e-05,
-      "loss": 0.0569,
-      "step": 2000
-    },
-    {
-      "epoch": 0.7939028262940616,
-      "grad_norm": 0.09763805568218231,
-      "learning_rate": 1.4709431565576376e-05,
-      "loss": 0.0528,
-      "step": 2500
-    },
-    {
-      "epoch": 0.952683391552874,
-      "grad_norm": 5.14717435836792,
-      "learning_rate": 1.3650894463850958e-05,
-      "loss": 0.0486,
-      "step": 3000
-    },
-    {
-      "epoch": 1.0,
-      "eval_accuracy": 0.9922191345772132,
-      "eval_loss": 0.03707018122076988,
-      "eval_runtime": 83.2394,
-      "eval_samples_per_second": 151.311,
-      "eval_steps_per_second": 9.467,
-      "step": 3149
-    },
-    {
-      "epoch": 1.1114639568116862,
-      "grad_norm": 0.0047888318076729774,
-      "learning_rate": 1.2592357362125544e-05,
-      "loss": 0.0229,
-      "step": 3500
-    },
-    {
-      "epoch": 1.2702445220704985,
-      "grad_norm": 0.043374646455049515,
-      "learning_rate": 1.1533820260400128e-05,
-      "loss": 0.019,
-      "step": 4000
-    },
-    {
-      "epoch": 1.4290250873293109,
-      "grad_norm": 0.03199789300560951,
-      "learning_rate": 1.0475283158674712e-05,
-      "loss": 0.0191,
-      "step": 4500
-    },
-    {
-      "epoch": 1.5878056525881232,
-      "grad_norm": 0.016201680526137352,
-      "learning_rate": 9.416746056949296e-06,
-      "loss": 0.0247,
-      "step": 5000
-    },
-    {
-      "epoch": 1.7465862178469356,
-      "grad_norm": 0.003989357966929674,
-      "learning_rate": 8.35820895522388e-06,
-      "loss": 0.021,
-      "step": 5500
-    },
-    {
-      "epoch": 1.9053667831057477,
-      "grad_norm": 0.013209746219217777,
-      "learning_rate": 7.299671853498465e-06,
-      "loss": 0.0171,
-      "step": 6000
-    },
-    {
-      "epoch": 2.0,
-      "eval_accuracy": 0.993092497022628,
-      "eval_loss": 0.033595748245716095,
-      "eval_runtime": 83.3444,
-      "eval_samples_per_second": 151.12,
-      "eval_steps_per_second": 9.455,
-      "step": 6298
-    }
-  ],
-  "logging_steps": 500,
-  "max_steps": 9447,
-  "num_input_tokens_seen": 0,
-  "num_train_epochs": 3,
-  "save_steps": 500,
-  "stateful_callbacks": {
-    "TrainerControl": {
-      "args": {
-        "should_epoch_stop": false,
-        "should_evaluate": false,
-        "should_log": false,
-        "should_save": true,
-        "should_training_stop": false
-      },
-      "attributes": {}
-    }
-  },
-  "total_flos": 6627372817935360.0,
-  "train_batch_size": 16,
-  "trial_name": null,
-  "trial_params": null
-}

indobert_model/training/checkpoint-6298/training_args.bin

@@ -1,3 +0,0 @@
-version https://git-lfs.github.com/spec/v1
-oid sha256:6bf0c0a22bf8643019429801930b8f01cdf82327a398b9e59c1324ef729e2b13
-size 5905

indobert_model/training/checkpoint-9447/config.json

@@ -1,33 +0,0 @@
-{
-  "_num_labels": 5,
-  "architectures": [
-    "BertForSequenceClassification"
-  ],
-  "attention_probs_dropout_prob": 0.1,
-  "classifier_dropout": null,
-  "directionality": "bidi",
-  "dtype": "float32",
-  "hidden_act": "gelu",
-  "hidden_dropout_prob": 0.1,
-  "hidden_size": 768,
-  "initializer_range": 0.02,
-  "intermediate_size": 3072,
-  "layer_norm_eps": 1e-12,
-  "max_position_embeddings": 512,
-  "model_type": "bert",
-  "num_attention_heads": 12,
-  "num_hidden_layers": 12,
-  "output_past": true,
-  "pad_token_id": 0,
-  "pooler_fc_size": 768,
-  "pooler_num_attention_heads": 12,
-  "pooler_num_fc_layers": 3,
-  "pooler_size_per_head": 128,
-  "pooler_type": "first_token_transform",
-  "position_embedding_type": "absolute",
-  "problem_type": "single_label_classification",
-  "transformers_version": "4.57.1",
-  "type_vocab_size": 2,
-  "use_cache": true,
-  "vocab_size": 50000
-}

indobert_model/training/checkpoint-9447/model.safetensors

@@ -1,3 +0,0 @@
-version https://git-lfs.github.com/spec/v1
-oid sha256:307ae084e201e8867cd13ff4962636aa99a7604d25122fdf9d7c732e975b0452
-size 497795072

indobert_model/training/checkpoint-9447/optimizer.pt

@@ -1,3 +0,0 @@
-version https://git-lfs.github.com/spec/v1
-oid sha256:04e0c904d346a06476020de43f96c94bee971931520a69e5e4d2806d60817b09
-size 995714443

indobert_model/training/checkpoint-9447/rng_state.pth

@@ -1,3 +0,0 @@
-version https://git-lfs.github.com/spec/v1
-oid sha256:8ab24f3da118b50afdf186f60aa31ba4e4512fdcf26c8fd14c3eabf53aeec8b9
-size 14645

indobert_model/training/checkpoint-9447/scheduler.pt

@@ -1,3 +0,0 @@
-version https://git-lfs.github.com/spec/v1
-oid sha256:2792ca39051534f3ef5a5f1445de1c961ae11679dc2c461c0a424ad030d41e70
-size 1465

indobert_model/training/checkpoint-9447/special_tokens_map.json

@@ -1,7 +0,0 @@
-{
-  "cls_token": "[CLS]",
-  "mask_token": "[MASK]",
-  "pad_token": "[PAD]",
-  "sep_token": "[SEP]",
-  "unk_token": "[UNK]"
-}

indobert_model/training/checkpoint-9447/tokenizer_config.json

@@ -1,58 +0,0 @@
-{
-  "added_tokens_decoder": {
-    "0": {
-      "content": "[PAD]",
-      "lstrip": false,
-      "normalized": false,
-      "rstrip": false,
-      "single_word": false,
-      "special": true
-    },
-    "1": {
-      "content": "[UNK]",
-      "lstrip": false,
-      "normalized": false,
-      "rstrip": false,
-      "single_word": false,
-      "special": true
-    },
-    "2": {
-      "content": "[CLS]",
-      "lstrip": false,
-      "normalized": false,
-      "rstrip": false,
-      "single_word": false,
-      "special": true
-    },
-    "3": {
-      "content": "[SEP]",
-      "lstrip": false,
-      "normalized": false,
-      "rstrip": false,
-      "single_word": false,
-      "special": true
-    },
-    "4": {
-      "content": "[MASK]",
-      "lstrip": false,
-      "normalized": false,
-      "rstrip": false,
-      "single_word": false,
-      "special": true
-    }
-  },
-  "clean_up_tokenization_spaces": true,
-  "cls_token": "[CLS]",
-  "do_basic_tokenize": true,
-  "do_lower_case": true,
-  "extra_special_tokens": {},
-  "mask_token": "[MASK]",
-  "model_max_length": 1000000000000000019884624838656,
-  "never_split": null,
-  "pad_token": "[PAD]",
-  "sep_token": "[SEP]",
-  "strip_accents": null,
-  "tokenize_chinese_chars": true,
-  "tokenizer_class": "BertTokenizer",
-  "unk_token": "[UNK]"
-}

indobert_model/training/checkpoint-9447/trainer_state.json

@@ -1,187 +0,0 @@
-{
-  "best_global_step": 9447,
-  "best_metric": 0.9938070662961492,
-  "best_model_checkpoint": "/home/user/Machine Learning/IndoHoaxDetector/indobert_model/training/checkpoint-9447",
-  "epoch": 3.0,
-  "eval_steps": 500,
-  "global_step": 9447,
-  "is_hyper_param_search": false,
-  "is_local_process_zero": true,
-  "is_world_process_zero": true,
-  "log_history": [
-    {
-      "epoch": 0.15878056525881232,
-      "grad_norm": 6.066575527191162,
-      "learning_rate": 1.8943579972478037e-05,
-      "loss": 0.1413,
-      "step": 500
-    },
-    {
-      "epoch": 0.31756113051762463,
-      "grad_norm": 2.5585596561431885,
-      "learning_rate": 1.788504287075262e-05,
-      "loss": 0.079,
-      "step": 1000
-    },
-    {
-      "epoch": 0.476341695776437,
-      "grad_norm": 6.151716232299805,
-      "learning_rate": 1.6826505769027206e-05,
-      "loss": 0.0669,
-      "step": 1500
-    },
-    {
-      "epoch": 0.6351222610352493,
-      "grad_norm": 0.012287363409996033,
-      "learning_rate": 1.576796866730179e-05,
-      "loss": 0.0569,
-      "step": 2000
-    },
-    {
-      "epoch": 0.7939028262940616,
-      "grad_norm": 0.09763805568218231,
-      "learning_rate": 1.4709431565576376e-05,
-      "loss": 0.0528,
-      "step": 2500
-    },
-    {
-      "epoch": 0.952683391552874,
-      "grad_norm": 5.14717435836792,
-      "learning_rate": 1.3650894463850958e-05,
-      "loss": 0.0486,
-      "step": 3000
-    },
-    {
-      "epoch": 1.0,
-      "eval_accuracy": 0.9922191345772132,
-      "eval_loss": 0.03707018122076988,
-      "eval_runtime": 83.2394,
-      "eval_samples_per_second": 151.311,
-      "eval_steps_per_second": 9.467,
-      "step": 3149
-    },
-    {
-      "epoch": 1.1114639568116862,
-      "grad_norm": 0.0047888318076729774,
-      "learning_rate": 1.2592357362125544e-05,
-      "loss": 0.0229,
-      "step": 3500
-    },
-    {
-      "epoch": 1.2702445220704985,
-      "grad_norm": 0.043374646455049515,
-      "learning_rate": 1.1533820260400128e-05,
-      "loss": 0.019,
-      "step": 4000
-    },
-    {
-      "epoch": 1.4290250873293109,
-      "grad_norm": 0.03199789300560951,
-      "learning_rate": 1.0475283158674712e-05,
-      "loss": 0.0191,
-      "step": 4500
-    },
-    {
-      "epoch": 1.5878056525881232,
-      "grad_norm": 0.016201680526137352,
-      "learning_rate": 9.416746056949296e-06,
-      "loss": 0.0247,
-      "step": 5000
-    },
-    {
-      "epoch": 1.7465862178469356,
-      "grad_norm": 0.003989357966929674,
-      "learning_rate": 8.35820895522388e-06,
-      "loss": 0.021,
-      "step": 5500
-    },
-    {
-      "epoch": 1.9053667831057477,
-      "grad_norm": 0.013209746219217777,
-      "learning_rate": 7.299671853498465e-06,
-      "loss": 0.0171,
-      "step": 6000
-    },
-    {
-      "epoch": 2.0,
-      "eval_accuracy": 0.993092497022628,
-      "eval_loss": 0.033595748245716095,
-      "eval_runtime": 83.3444,
-      "eval_samples_per_second": 151.12,
-      "eval_steps_per_second": 9.455,
-      "step": 6298
-    },
-    {
-      "epoch": 2.06414734836456,
-      "grad_norm": 0.0014694147976115346,
-      "learning_rate": 6.24113475177305e-06,
-      "loss": 0.0149,
-      "step": 6500
-    },
-    {
-      "epoch": 2.2229279136233724,
-      "grad_norm": 0.0010297272820025682,
-      "learning_rate": 5.182597650047635e-06,
-      "loss": 0.0042,
-      "step": 7000
-    },
-    {
-      "epoch": 2.3817084788821847,
-      "grad_norm": 0.0013135488843545318,
-      "learning_rate": 4.124060548322219e-06,
-      "loss": 0.0052,
-      "step": 7500
-    },
-    {
-      "epoch": 2.540489044140997,
-      "grad_norm": 0.0010974216274917126,
-      "learning_rate": 3.0655234465968036e-06,
-      "loss": 0.0059,
-      "step": 8000
-    },
-    {
-      "epoch": 2.6992696093998094,
-      "grad_norm": 0.0037105802912265062,
-      "learning_rate": 2.0069863448713877e-06,
-      "loss": 0.003,
-      "step": 8500
-    },
-    {
-      "epoch": 2.8580501746586218,
-      "grad_norm": 0.002627098932862282,
-      "learning_rate": 9.484492431459723e-07,
-      "loss": 0.0033,
-      "step": 9000
-    },
-    {
-      "epoch": 3.0,
-      "eval_accuracy": 0.9938070662961492,
-      "eval_loss": 0.03967500478029251,
-      "eval_runtime": 83.2653,
-      "eval_samples_per_second": 151.264,
-      "eval_steps_per_second": 9.464,
-      "step": 9447
-    }
-  ],
-  "logging_steps": 500,
-  "max_steps": 9447,
-  "num_input_tokens_seen": 0,
-  "num_train_epochs": 3,
-  "save_steps": 500,
-  "stateful_callbacks": {
-    "TrainerControl": {
-      "args": {
-        "should_epoch_stop": false,
-        "should_evaluate": false,
-        "should_log": false,
-        "should_save": true,
-        "should_training_stop": true
-      },
-      "attributes": {}
-    }
-  },
-  "total_flos": 9941059226903040.0,
-  "train_batch_size": 16,
-  "trial_name": null,
-  "trial_params": null
-}

indobert_model/training/checkpoint-9447/training_args.bin

@@ -1,3 +0,0 @@
-version https://git-lfs.github.com/spec/v1
-oid sha256:6bf0c0a22bf8643019429801930b8f01cdf82327a398b9e59c1324ef729e2b13
-size 5905