failed09 commited on
Commit
1e7b82e
·
verified ·
1 Parent(s): 0549cad

Align BashkirRoBERTa model card with release template

Browse files
Files changed (2) hide show
  1. README.md +31 -22
  2. SHA256SUMS +1 -1
README.md CHANGED
@@ -17,21 +17,20 @@ tags:
17
 
18
  # BashkirRoBERTa
19
 
20
- > A masked language model for Bashkir, for fill-mask, spellchecking and
21
- > foundation fine-tuning.
22
 
23
  ## Overview
24
 
25
- A masked language model for Bashkir. Given a sentence with one `[MASK]` token, it
26
- predicts the most probable missing Bashkir token from context. The model is useful
27
- for fill-mask experiments, spellchecking and as a foundation for further
28
- fine-tuning. It preserves a custom Pre-LayerNorm architecture rather than the
29
- stock post-LayerNorm RoBERTa implementation.
30
 
31
  | At a glance | |
32
  | --- | --- |
33
  | Task | Masked language modelling / fill-mask |
34
- | Default artifact | `model.safetensors` (Transformers) or `onnx/model_int8.onnx` (ONNX) |
35
  | Source | A monolingual Bashkir-language dataset |
36
  | Version / license | v1 / Apache-2.0 |
37
 
@@ -41,9 +40,9 @@ stock post-LayerNorm RoBERTa implementation.
41
 
42
  | File | Purpose | Size |
43
  | --- | --- | ---: |
44
- | `model.safetensors` | PyTorch weights for Transformers | 200.2 MB |
45
  | `onnx/model_fp16.onnx` | FP16 ONNX model for GPU / DirectML | 121.2 MB |
46
- | `onnx/model_int8.onnx` | INT8 ONNX model for fast CPU / mobile | 60.9 MB |
47
  | `spm_bashkir_bert_16k.model` | SentencePiece tokenizer | — |
48
  | `config.json` | Model configuration (`auto_map` for custom code) | — |
49
  | `configuration_bashkir_roberta.py`, `modeling_bashkir_roberta.py`, `tokenization_bashkir_roberta.py` | Custom Pre-LayerNorm implementation | — |
@@ -82,11 +81,16 @@ Outputs from the INT8 ONNX model on CPU:
82
 
83
  ## Method
84
 
85
- The model was pretrained with dynamic masked-language modelling on a monolingual
86
- Bashkir-language dataset assembled from encyclopedic, periodical and literary
87
- sources. The source texts are not distributed in this repository.
88
-
89
- ### Evaluation
 
 
 
 
 
90
 
91
  On a held-out Bashkir encyclopedic evaluation set the project reports **24.7%
92
  top-1** and **54.0% top-5** accuracy for masked subword prediction. These are
@@ -95,19 +99,24 @@ may represent a whole word or a SentencePiece subword fragment.
95
 
96
  ## Quality and Use
97
 
98
- This is a research model, not a production language service. Fill-mask predictions
99
- are ranking suggestions that require context-appropriate review, especially for
100
- ambiguous or short contexts. The model weights and release code are available
101
- under Apache-2.0; source-text provenance remains documented separately.
102
 
103
  ### Limitations
104
 
105
  - Diagnostic MLM accuracy only; not fine-tuned for any downstream task.
106
  - A mask may correspond to a partial subword, not always a full word.
107
  - Predictions reflect the training corpus and may prefer frequent or encyclopedic phrasing.
108
- - No training texts are redistributed; provenance or removal requests go through the maintainer.
109
-
110
- ## Usage
 
 
 
 
 
 
111
 
112
  ```bash
113
  pip install transformers torch huggingface_hub
 
17
 
18
  # BashkirRoBERTa
19
 
20
+ > A Bashkir masked language model for fill-mask suggestions and further fine-tuning.
 
21
 
22
  ## Overview
23
 
24
+ BashkirRoBERTa predicts masked SentencePiece tokens from context. It can provide
25
+ fill-mask suggestions and serve as an encoder for further fine-tuning, including
26
+ the related person-name model. The release includes Transformers weights and FP16
27
+ and INT8 ONNX graphs. Its custom Pre-LayerNorm architecture requires the included
28
+ model code when loaded with Transformers.
29
 
30
  | At a glance | |
31
  | --- | --- |
32
  | Task | Masked language modelling / fill-mask |
33
+ | Default artifact | `model.safetensors` (Transformers); `onnx/model_int8.onnx` (CPU ONNX) |
34
  | Source | A monolingual Bashkir-language dataset |
35
  | Version / license | v1 / Apache-2.0 |
36
 
 
40
 
41
  | File | Purpose | Size |
42
  | --- | --- | ---: |
43
+ | **`model.safetensors`** | Transformers weights and fine-tuning starting point | 200.2 MB |
44
  | `onnx/model_fp16.onnx` | FP16 ONNX model for GPU / DirectML | 121.2 MB |
45
+ | **`onnx/model_int8.onnx`** | Compact INT8 ONNX graph for CPU use | 60.9 MB |
46
  | `spm_bashkir_bert_16k.model` | SentencePiece tokenizer | — |
47
  | `config.json` | Model configuration (`auto_map` for custom code) | — |
48
  | `configuration_bashkir_roberta.py`, `modeling_bashkir_roberta.py`, `tokenization_bashkir_roberta.py` | Custom Pre-LayerNorm implementation | — |
 
81
 
82
  ## Method
83
 
84
+ The model was pretrained with dynamic masked-language modelling on a monolingual
85
+ Bashkir-language dataset assembled from encyclopedic, periodical and literary
86
+ sources. The source texts are not distributed in this repository.
87
+
88
+ ```text
89
+ Bashkir text → SentencePiece tokens → masked-language pretraining
90
+ → Transformers checkpoint → FP16 and INT8 ONNX exports
91
+ ```
92
+
93
+ ### Benchmark
94
 
95
  On a held-out Bashkir encyclopedic evaluation set the project reports **24.7%
96
  top-1** and **54.0% top-5** accuracy for masked subword prediction. These are
 
99
 
100
  ## Quality and Use
101
 
102
+ Use the Transformers checkpoint for fill-mask experiments or fine-tuning and the
103
+ INT8 ONNX graph for compact CPU inference. Predictions are token suggestions that
104
+ need review in context, especially when the mask represents only part of a word.
 
105
 
106
  ### Limitations
107
 
108
  - Diagnostic MLM accuracy only; not fine-tuned for any downstream task.
109
  - A mask may correspond to a partial subword, not always a full word.
110
  - Predictions reflect the training corpus and may prefer frequent or encyclopedic phrasing.
111
+ - No training texts are redistributed; provenance or removal requests go through the maintainer.
112
+
113
+ ## Related Resources
114
+
115
+ - [BashkirRoBERTa NER](https://huggingface.co/failed09/bashkir-roberta-ner)
116
+ fine-tunes this encoder to find person names and return their text spans. Use it
117
+ when the task is person-name extraction rather than masked-token prediction.
118
+
119
+ ## Usage
120
 
121
  ```bash
122
  pip install transformers torch huggingface_hub
SHA256SUMS CHANGED
@@ -1,6 +1,6 @@
1
  cfc7749b96f63bd31c3c42b5c471bf756814053e847c10f3eb003417bc523d30 LICENSE
2
  f6d59748711ddcc45af84d09cf7970a276ee93820a83dda4fb5fda8209f878ca META.json
3
- 5da8ad2a9ea236fdfdc3f41b0518c3544f97fffc0fee2bc6deb7b7d9fb271ad4 README.md
4
  59b773953af5b2a69adb7e3c5d345a90f34f848f6a1543309d5acc7fb78b28c4 config.json
5
  e9f179aad7619153724920c86a90ecc3c161fd15ffc3332f200e9f2073a07f72 configuration_bashkir_roberta.py
6
  48e4d4e7b8af675d222e58e14776abac24b7e83a937d7884bd9d735ea9dd2173 model.safetensors
 
1
  cfc7749b96f63bd31c3c42b5c471bf756814053e847c10f3eb003417bc523d30 LICENSE
2
  f6d59748711ddcc45af84d09cf7970a276ee93820a83dda4fb5fda8209f878ca META.json
3
+ d11fdb4dffe9cc59684392c54e798f194493bb48862196291e96a067a8471efb README.md
4
  59b773953af5b2a69adb7e3c5d345a90f34f848f6a1543309d5acc7fb78b28c4 config.json
5
  e9f179aad7619153724920c86a90ecc3c161fd15ffc3332f200e9f2073a07f72 configuration_bashkir_roberta.py
6
  48e4d4e7b8af675d222e58e14776abac24b7e83a937d7884bd9d735ea9dd2173 model.safetensors