BDRC/Bo-Multilayer-Detection

mmbert_base_v1.2

Tibetan annotation-layer token classifier fine-tuned from jhu-clsp/mmBERT-base.

Validation metrics

The values below are from the best evaluation record. The complete evaluation history and original Trainer state are included in training/metrics.json and training/trainer_state.json.

Metric	Value
epoch	8.0
eval_f1	0.078113428181641
eval_f1_author	0.6329113924050632
eval_f1_chapter	0.5358490566037736
eval_f1_quote	0.022925944252020453
eval_f1_sabche	0.19551934826883913
eval_f1_tsawa	0.03828125
eval_f1_yigchung	0.02306079664570231
eval_loss	0.15565519034862518
eval_precision	0.052703555649063746
eval_recall	0.15083546590395905
eval_runtime	165.0116
eval_samples_per_second	8.581
eval_soft_f1_tol0	0.078113428181641
eval_soft_f1_tol1	0.09284739816799845
eval_soft_precision_tol0	0.052703555649063746
eval_soft_precision_tol1	0.06264464548706081
eval_soft_recall_tol0	0.15083546590395905
eval_soft_recall_tol1	0.17928646695769984
eval_steps_per_second	4.291
step	10832

Training parameters

Every value from the experiment's config.yaml is listed below. The original file is also included as training/config.yaml.

Parameter	Value
experiment_name	mmbert_base_v1.2
dataset_repo	karma689/layer_detection
dataset_revision	null
scope_policy	confirmed
local_fallback.repos_dir	../../Data/repos
local_fallback.split_csv	../../Data/audit/split.csv
checkpoint	jhu-clsp/mmBERT-base
fallback_checkpoint	KoichiYasuoka/bert-base-tibetan
attn_implementation	flash_attention_2
attn_fallback_implementation	sdpa
max_length	8192
stride	5120
pad_to_multiple_of	8
window_cache_dir	window_cache
tokenize_num_workers	8
masked_layers	["BookTitle"]
learning_rate	1e-05
train_batch_size	2
eval_batch_size	2
gradient_accumulation_steps	4
num_train_epochs	10
weight_decay	0.01
warmup_ratio	0.2
max_grad_norm	0.3
class_weighting	True
class_weight_cap	5.0
o_class_weight	0.07
fp16	False
bf16	True
gradient_checkpointing	False
dataloader_num_workers	4
dataloader_pin_memory	True
tf32	True
seed	42
logging_steps	20
save_total_limit	3
resume_from_checkpoint	null
eval_strategy	epoch
save_strategy	epoch
load_best_model_at_end	True
metric_for_best_model	soft_f1_tol1
greater_is_better	True
early_stopping_patience	3
report_to	["wandb"]
wandb.project	layer-detection
wandb.mode	online
output_dir	outputs

Labels

ID	BIO label
0	O
1	B-QUOTE
2	I-QUOTE
3	B-SABCHE
4	I-SABCHE
5	B-TSAWA
6	I-TSAWA
7	B-YIGCHUNG
8	I-YIGCHUNG
9	B-CHAPTER
10	I-CHAPTER
11	B-AUTHOR
12	I-AUTHOR
13	B-BOOKTITLE
14	I-BOOKTITLE

Usage

from transformers import AutoModelForTokenClassification, AutoTokenizer

model_id = "karma689/mmbert-base-layer-detection-v1.2"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForTokenClassification.from_pretrained(model_id)

Long texts should use the same window settings shown in the training parameters.

License and attribution

Base model license follows jhu-clsp/mmBERT-base (Apache-2.0). Source texts were digitized and made available by the Buddhist Digital Resource Center (BDRC). Annotations were prepared through OpenPecha with support from the Tsadra Foundation.