File size: 4,298 Bytes
d190c88 4423ac6 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 | ---
language:
- bo
library_name: transformers
pipeline_tag: token-classification
tags:
- tibetan
- token-classification
- layer-detection
base_model: jhu-clsp/mmBERT-base
datasets:
- karma689/layer_detection
---
# mmbert_base_v1.2
Tibetan annotation-layer token classifier fine-tuned from
[`jhu-clsp/mmBERT-base`](https://huggingface.co/jhu-clsp/mmBERT-base).
## Validation metrics
The values below are from the **best** evaluation record.
The complete evaluation history and original Trainer state are included in
`training/metrics.json` and `training/trainer_state.json`.
| Metric | Value |
|---|---:|
| `epoch` | `8.0` |
| `eval_f1` | `0.078113428181641` |
| `eval_f1_author` | `0.6329113924050632` |
| `eval_f1_chapter` | `0.5358490566037736` |
| `eval_f1_quote` | `0.022925944252020453` |
| `eval_f1_sabche` | `0.19551934826883913` |
| `eval_f1_tsawa` | `0.03828125` |
| `eval_f1_yigchung` | `0.02306079664570231` |
| `eval_loss` | `0.15565519034862518` |
| `eval_precision` | `0.052703555649063746` |
| `eval_recall` | `0.15083546590395905` |
| `eval_runtime` | `165.0116` |
| `eval_samples_per_second` | `8.581` |
| `eval_soft_f1_tol0` | `0.078113428181641` |
| `eval_soft_f1_tol1` | `0.09284739816799845` |
| `eval_soft_precision_tol0` | `0.052703555649063746` |
| `eval_soft_precision_tol1` | `0.06264464548706081` |
| `eval_soft_recall_tol0` | `0.15083546590395905` |
| `eval_soft_recall_tol1` | `0.17928646695769984` |
| `eval_steps_per_second` | `4.291` |
| `step` | `10832` |
## Training parameters
Every value from the experiment's `config.yaml` is listed below. The original
file is also included as `training/config.yaml`.
| Parameter | Value |
|---|---|
| `experiment_name` | `mmbert_base_v1.2` |
| `dataset_repo` | `karma689/layer_detection` |
| `dataset_revision` | `null` |
| `scope_policy` | `confirmed` |
| `local_fallback.repos_dir` | `../../Data/repos` |
| `local_fallback.split_csv` | `../../Data/audit/split.csv` |
| `checkpoint` | `jhu-clsp/mmBERT-base` |
| `fallback_checkpoint` | `KoichiYasuoka/bert-base-tibetan` |
| `attn_implementation` | `flash_attention_2` |
| `attn_fallback_implementation` | `sdpa` |
| `max_length` | `8192` |
| `stride` | `5120` |
| `pad_to_multiple_of` | `8` |
| `window_cache_dir` | `window_cache` |
| `tokenize_num_workers` | `8` |
| `masked_layers` | `["BookTitle"]` |
| `learning_rate` | `1e-05` |
| `train_batch_size` | `2` |
| `eval_batch_size` | `2` |
| `gradient_accumulation_steps` | `4` |
| `num_train_epochs` | `10` |
| `weight_decay` | `0.01` |
| `warmup_ratio` | `0.2` |
| `max_grad_norm` | `0.3` |
| `class_weighting` | `True` |
| `class_weight_cap` | `5.0` |
| `o_class_weight` | `0.07` |
| `fp16` | `False` |
| `bf16` | `True` |
| `gradient_checkpointing` | `False` |
| `dataloader_num_workers` | `4` |
| `dataloader_pin_memory` | `True` |
| `tf32` | `True` |
| `seed` | `42` |
| `logging_steps` | `20` |
| `save_total_limit` | `3` |
| `resume_from_checkpoint` | `null` |
| `eval_strategy` | `epoch` |
| `save_strategy` | `epoch` |
| `load_best_model_at_end` | `True` |
| `metric_for_best_model` | `soft_f1_tol1` |
| `greater_is_better` | `True` |
| `early_stopping_patience` | `3` |
| `report_to` | `["wandb"]` |
| `wandb.project` | `layer-detection` |
| `wandb.mode` | `online` |
| `output_dir` | `outputs` |
## Labels
| ID | BIO label |
|---:|---|
| 0 | `O` |
| 1 | `B-QUOTE` |
| 2 | `I-QUOTE` |
| 3 | `B-SABCHE` |
| 4 | `I-SABCHE` |
| 5 | `B-TSAWA` |
| 6 | `I-TSAWA` |
| 7 | `B-YIGCHUNG` |
| 8 | `I-YIGCHUNG` |
| 9 | `B-CHAPTER` |
| 10 | `I-CHAPTER` |
| 11 | `B-AUTHOR` |
| 12 | `I-AUTHOR` |
| 13 | `B-BOOKTITLE` |
| 14 | `I-BOOKTITLE` |
## Usage
```python
from transformers import AutoModelForTokenClassification, AutoTokenizer
model_id = "karma689/mmbert-base-layer-detection-v1.2"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForTokenClassification.from_pretrained(model_id)
```
Long texts should use the same window settings shown in the training parameters.
## License and attribution
Base model license follows [`jhu-clsp/mmBERT-base`](https://huggingface.co/jhu-clsp/mmBERT-base) (Apache-2.0). Source texts were digitized and made available by the Buddhist Digital Resource Center (BDRC). Annotations were prepared through OpenPecha with support from the Tsadra Foundation.
|