NagaYu commited on
Commit
30a638a
·
verified ·
1 Parent(s): 86fd05f

Scribe usage classifier (mBERT token-classification, POC on synthetic seeds)

Browse files
Files changed (6) hide show
  1. README.md +98 -0
  2. config.json +44 -0
  3. model.safetensors +3 -0
  4. tokenizer.json +0 -0
  5. tokenizer_config.json +15 -0
  6. training_args.bin +3 -0
README.md ADDED
@@ -0,0 +1,98 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ language:
3
+ - ja
4
+ license: apache-2.0
5
+ library_name: transformers
6
+ pipeline_tag: token-classification
7
+ base_model: google-bert/bert-base-multilingual-cased
8
+ tags:
9
+ - japanese
10
+ - koyobun
11
+ - orthography
12
+ - kanji-kana
13
+ - government-writing
14
+ - token-classification
15
+ metrics:
16
+ - accuracy
17
+ model-index:
18
+ - name: scribe-usage-classifier
19
+ results:
20
+ - task:
21
+ type: token-classification
22
+ name: Context-dependent kanji/kana usage judgment
23
+ metrics:
24
+ - type: accuracy
25
+ name: test span accuracy (held-out by document)
26
+ value: 0.818
27
+ - type: accuracy
28
+ name: test hard-set span accuracy
29
+ value: 0.800
30
+ ---
31
+
32
+ # scribe-usage-classifier
33
+
34
+ Token-classification model that judges **context-dependent kanji/kana usage** in Japanese official documents:
35
+ given each occurrence of a monitored word, it decides whether the norm writes it in **kana** or **kanji** based on
36
+ the word's **syntactic role** — a distinction that dictionary/regex replacement cannot make.
37
+
38
+ - Part of the **Scribe** project · 💻 [GitHub](https://github.com/NagaYu/scribe-koyobun) ·
39
+ 🤗 [Space](https://huggingface.co/spaces/NagaYu/scribe-koyobun) · 📚 [Dataset](https://huggingface.co/datasets/NagaYu/scribe-koyobun-usage)
40
+ - Base model: `google-bert/bert-base-multilingual-cased` (~178M params, within the 0.1–0.2B target).
41
+ Chosen because its **fast tokenizer** returns `offset_mapping`, which the span-labeling pipeline needs.
42
+
43
+ ## Labels
44
+
45
+ Per subtoken: `O` / `TARGET-KANA` / `TARGET-KANJI`. Subtokens overlapping a monitored word's span receive the
46
+ usage label; the input surface may be correct or wrong (the training data includes flipped errors), so the model
47
+ must learn from **context, not surface**.
48
+
49
+ ## How to use
50
+
51
+ The model is meant to be driven through the Scribe pipeline, which finds the monitored words and attaches the
52
+ normative clause as rationale:
53
+
54
+ ```python
55
+ from scribe import ScribeChecker
56
+ checker = ScribeChecker(model_dir="NagaYu/scribe-usage-classifier")
57
+ for f in checker.check("確認する事がある。重い物を運ぶ。"):
58
+ print(f.surface, "→", f.recommended_surface, "|", f.message)
59
+ ```
60
+
61
+ Raw transformers use is also possible (`AutoModelForTokenClassification`), but you then need to locate the target
62
+ spans yourself; the Scribe repo does this with a morphological analyzer.
63
+
64
+ ## Training data
65
+
66
+ Fine-tuned on the **Scribe usage-judgment dataset** ([NagaYu/scribe-koyobun-usage](https://huggingface.co/datasets/NagaYu/scribe-koyobun-usage)),
67
+ a **synthetic seed corpus (CC0)** of official-style sentences plus flip-generated errors, split by document.
68
+ The dataset separates a **hard** split (words appearing in both usages) as the headline diagnostic.
69
+
70
+ ## Evaluation
71
+
72
+ Span-level accuracy (predicted kind vs. gold kind):
73
+
74
+ | split | span accuracy | hard-set span accuracy |
75
+ |---|---|---|
76
+ | test (held out by document) | **0.818** | **0.800** |
77
+ | hard (diagnostic, overlaps train) | 0.947 | 0.947 |
78
+
79
+ Evaluate `test` for generalization; `hard` cross-cuts the training split and is optimistic. For the project-level
80
+ comparison against uniform replacement (prh) and a jōyō-kanji checker, see the
81
+ [benchmarks](https://github.com/NagaYu/scribe-koyobun/tree/main/benchmarks).
82
+
83
+ ## Intended use & limitations
84
+
85
+ - **Intended**: assisting drafters of Japanese official/public documents by flagging context-dependent kanji/kana
86
+ choices with a cited clause of the norm; output is phrased as "the norm would write it this way," never as a verdict
87
+ on the writer.
88
+ - **Proof-of-concept scale**: trained on a small synthetic corpus. It demonstrates that the distinction is learnable
89
+ from context, but is **not** production-grade on natural text. For real deployment, train on a licensed real corpus
90
+ (see `scribe/collect.py`) and expand the monitored vocabulary (`data/norms/lexicon.json`).
91
+ - **Coverage**: limited to the monitored formal nouns and auxiliary verbs; other orthographic issues (numbers,
92
+ punctuation, okurigana) are handled by Scribe's deterministic rule layer, not this model.
93
+ - The base model is multilingual; a dedicated Japanese encoder may improve quality if paired with a custom
94
+ offset aligner.
95
+
96
+ ## License
97
+
98
+ Apache-2.0. The normative documents' rights belong to their issuers; this model references clause numbers only.
config.json ADDED
@@ -0,0 +1,44 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "add_cross_attention": false,
3
+ "architectures": [
4
+ "BertForTokenClassification"
5
+ ],
6
+ "attention_probs_dropout_prob": 0.1,
7
+ "bos_token_id": null,
8
+ "classifier_dropout": null,
9
+ "directionality": "bidi",
10
+ "dtype": "float32",
11
+ "eos_token_id": null,
12
+ "hidden_act": "gelu",
13
+ "hidden_dropout_prob": 0.1,
14
+ "hidden_size": 768,
15
+ "id2label": {
16
+ "0": "O",
17
+ "1": "TARGET-KANA",
18
+ "2": "TARGET-KANJI"
19
+ },
20
+ "initializer_range": 0.02,
21
+ "intermediate_size": 3072,
22
+ "is_decoder": false,
23
+ "label2id": {
24
+ "O": 0,
25
+ "TARGET-KANA": 1,
26
+ "TARGET-KANJI": 2
27
+ },
28
+ "layer_norm_eps": 1e-12,
29
+ "max_position_embeddings": 512,
30
+ "model_type": "bert",
31
+ "num_attention_heads": 12,
32
+ "num_hidden_layers": 12,
33
+ "pad_token_id": 0,
34
+ "pooler_fc_size": 768,
35
+ "pooler_num_attention_heads": 12,
36
+ "pooler_num_fc_layers": 3,
37
+ "pooler_size_per_head": 128,
38
+ "pooler_type": "first_token_transform",
39
+ "tie_word_embeddings": true,
40
+ "transformers_version": "5.16.1",
41
+ "type_vocab_size": 2,
42
+ "use_cache": false,
43
+ "vocab_size": 119547
44
+ }
model.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:8003afd2916adbbb1118d76fc679be04f54490defba0e6f56238c14d02e32529
3
+ size 709083948
tokenizer.json ADDED
The diff for this file is too large to render. See raw diff
 
tokenizer_config.json ADDED
@@ -0,0 +1,15 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "backend": "tokenizers",
3
+ "cls_token": "[CLS]",
4
+ "do_lower_case": false,
5
+ "is_local": false,
6
+ "local_files_only": false,
7
+ "mask_token": "[MASK]",
8
+ "model_max_length": 512,
9
+ "pad_token": "[PAD]",
10
+ "sep_token": "[SEP]",
11
+ "strip_accents": null,
12
+ "tokenize_chinese_chars": true,
13
+ "tokenizer_class": "BertTokenizer",
14
+ "unk_token": "[UNK]"
15
+ }
training_args.bin ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:631c3506af9594859ac253f6468c1414de3871737ff8df957af648cdae2a2b64
3
+ size 5201