kestrel-edu-ner-22l v1.2.1 โ€” ํ•œ๊ตญ์–ด ๊ต์œก ๋„๋ฉ”์ธ 22์ธต ๊ต์‚ฌ PII ๊ฐœ์ฒด๋ช… ์ธ์‹ ๋ชจ๋ธ (edu-v1+fmt)

Korean | English

1. ๋ชจ๋ธ ์š”์•ฝ (v1.2.1, ๋ผ๋ฒจ ํ”„๋กœํŒŒ์ผ edu-v1+fmt ยท 29ํด๋ž˜์Šค BIO 22์ธต ๊ต์‚ฌ ๋ชจ๋ธ)

mulkzz/kestrel-edu-ner-22l(v1.2.1, ๋ผ๋ฒจ ํ”„๋กœํŒŒ์ผ edu-v1+fmt)๋Š” ๊ฒฝ๋Ÿ‰ ์„œ๋น™ ํ•™์ƒ ๋ชจ๋ธ(mulkzz/kestrel-edu-ner-6l)์˜ ๋‹ค์ค‘ ์‹ ํ˜ธ ์ง€์‹ ์ฆ๋ฅ˜(Stage 2)๋ฅผ ๊ฐ๋…ํ•˜๊ธฐ ์œ„ํ•ด ํ•™์Šต๋œ 22์ธต ์ƒ์œ„ ๊ต์‚ฌ(Teacher) ํ† ํฐ ๋ถ„๋ฅ˜ ๋ชจ๋ธ(149.34M ํŒŒ๋ผ๋ฏธํ„ฐ / ๋น„์ž„๋ฒ ๋”ฉ 110.94M)์ž…๋‹ˆ๋‹ค. skt/A.X-Encoder-base(9708f9c, Apache-2.0) ์œ„์— 4๋Œ€ ํ•ต์‹ฌ ๋น„์ •ํ˜• ๊ฐœ์ฒด(PERSON, SCHOOL, ORG, ADDRESS) + 10์ข… ์ •ํ˜• ๋ณด์กฐ ๊ฐœ์ฒด(PHONE, EMAIL, RRN, PASSPORT, DRIVER_LICENSE, ACCOUNT, CARD, BRN, URL, CODE) ์ด 29ํด๋ž˜์Šค BIO ํ—ค๋“œ๋ฅผ ์–น๊ณ , ์กฐ์‚ฌ ๋ถ„๋ฆฌ ์ •๋ ฌ(particle_aware)๊ณผ ๋™ํ˜•์ด์˜์–ด(amb ๋Œ€ amb_neg) + ๋™๋ช…์ด์ธ(D1=B ๊ต๊ณผ ๊ณต์ธ O ๋Œ€ ์‹ค์ œ ํ•™์ƒ PERSON) ๋™์ผ ๋ฏธ๋‹ˆ๋ฐฐ์น˜ ๋Œ€์กฐ ์ƒ˜ํ”Œ๋Ÿฌ๋กœ ๋ฏธ์„ธ์กฐ์ •ํ–ˆ์Šต๋‹ˆ๋‹ค.

  • ๋ธŒ๋žœ์น˜ ยท ๋ฆฌ๋น„์ „ ๋ฐ ์„œ๋น™ ์•ˆ๋‚ด:
    • ๊ฒฝ๋Ÿ‰ 6์ธต ์„œ๋น™ ํ•™์ƒ ๋ชจ๋ธ (์šด์˜ ๊ถŒ์žฅ): mulkzz/kestrel-edu-ner-6l (69.62 MB INT8 ONNX, CPU 2.17ร— ๊ฐ€์†, dorms-pii-ner-bench v1.1 post-v1.1 ์ ์ค‘๋ฅ  91.6% [589/643] / ๋ˆ„๋ฝ 8.4%).
    • main ยท v1.2.1 ยท v1.2: 29ํด๋ž˜์Šค edu-v1+fmt ๊ต์‚ฌ ๊ฐ€์ค‘์น˜(sha256: 9998e608...) ๋ฐ v1.2.1 ์ตœ์‹  ๋ฒค์น˜๋งˆํฌ ๋„ํ‘œ๋ฅผ ์ œ๊ณตํ•ฉ๋‹ˆ๋‹ค. ์ดˆ๊ธฐ 9ํด๋ž˜์Šค(edu-v1) ๋‹จ๋ฐœ ๊ธฐ์ค€์„ ์€ revision="v1.0" (133302653e95c30d0e8ea0729f0e6f9649d4a0eb)์— ์˜๊ตฌ ๋ณด์กด๋ฉ๋‹ˆ๋‹ค.
    • ์‹ค์‹œ๊ฐ„ CPU ์„œ๋น™ ์ฐธ๊ณ : 22์ธต ๊ต์‚ฌ ๋ชจ๋ธ์€ ECI C-4(4 vCPU) ๊ธฐ์ค€ 20,000์žร—๋™์‹œ 4์š”์ฒญ ํˆฌ์˜ ์ง€์—ฐ์ด ~54.5s(40s ์ƒํ•œ ์ดˆ๊ณผ)์ด๋ฏ€๋กœ ์‹ค์‹œ๊ฐ„ ๊ฒŒ์ดํŠธ์›จ์ดยท์˜จ๋””๋ฐ”์ด์Šค ๋ธŒ๋ฆฌ์ง€์—๋Š” 6์ธต ์ฆ๋ฅ˜ ํ•™์ƒ ๋ชจ๋ธ์„ ์‚ฌ์šฉํ•˜๋ฉฐ, ๋ณธ ์ €์žฅ์†Œ๋Š” ์ง€์‹ ์ฆ๋ฅ˜ ์žฌํ˜„ยท์˜คํ”„๋ผ์ธ ๊ณ ์ •๋ฐ€ ๊ธฐ์ค€์„ ยทํ›„์† ์—ฐ๊ตฌ์šฉ์œผ๋กœ ๊ณต๊ฐœํ•ฉ๋‹ˆ๋‹ค.
  • ํ•™์Šต ๋ฐ์ดํ„ฐ ์›์น™: ์‹ค์ œ ํ•™์ƒ ์ž๋ฃŒ 0๊ฑด, ์™ธ๋ถ€ ์ƒ์šฉ LLM ์ƒ์„ฑ ํ•™์Šต ๋ฐ์ดํ„ฐ 0๊ฑด(28,000๊ฑด ๊ฒฐ์ •๋ก ์  ํ•ฉ์„ฑ ๊ต์œก ๋ฌธ์„œ).

6์ธต ์ฆ๋ฅ˜ ํ•™์ƒ์˜ post-v1.1 ๋ฒค์น˜๋งˆํฌ ๋ฐ ๊ต์‚ฌ-ํ•™์ƒ ์šฉ๋Ÿ‰ ๋น„๊ต

Fig. 1 | v1.2.1 capacity, latency, quantization parity, and 4-Gate summary

Fig. 1 | 22์ธต ๊ต์‚ฌ(kestrel-edu-ner-22l) โ†’ 6์ธต ํ•™์ƒ(kestrel-edu-ner-6l) ์••์ถ• ์‹œ ์ง€์—ฐยท๋ฉ”๋ชจ๋ฆฌ ์ ˆ๊ฐ ๋ฐ v1.2.1 4์ค‘ ๊ฒŒ์ดํŠธ ์š”์•ฝ. ๋น„์ž„๋ฒ ๋”ฉ ์—ฐ์‚ฐ ํŒŒ๋ผ๋ฏธํ„ฐ๋ฅผ 110.94M โ†’ 30.70M(-72.3%)์œผ๋กœ ์ค„์—ฌ Local Bridge 20k์ž ์ง€์—ฐ์„ 6.49s โ†’ 1.82s, PII Worker 20kร—4 ์ง€์—ฐ์„ 17.03s โ†’ 4.40s๋กœ ๋‹จ์ถ•ํ•˜๋ฉด์„œ๋„ ๋ฏธ๋…ธ์ถœ ๋‹จ๋ฐœ itest_v1_2(3,000๊ฑด) ๋งคํฌ๋กœ ํ•ต์‹ฌ ์žฌํ˜„์œจ 97.84% โ†’ 97.44%(์ •ํ˜• 1,817 span 100.0% โ†’ 100.0%)๋ฅผ ์œ ์ง€ํ•จ. 6์ธต ํ•™์ƒ์˜ dorms-pii-ner-bench v1.1(post-v1.1, ๋ˆ„๋ฝ ๋ฌธํ•ญ 17.0% โ†’ 8.4%, $p < 0.0001$) ์ƒ์„ธ ์„ฑ์ ์€ mulkzz/kestrel-edu-ner-6l ์นด๋“œ ๋ฐ ./figures/ner_bench_v1_1_6l.svg ์ฐธ์กฐ.

Fig. 2 | Stage 1 22L teacher fine-tuning dynamics and itest_v1_2 evaluation

Fig. 2 | Stage 1 22์ธต ๊ต์‚ฌ ๋ฏธ์„ธ์กฐ์ • ์ˆ˜๋ ด ๋™์—ญํ•™ ๋ฐ ๋ถ„ํฌ ๋‚ด(dev_v1_2 / ๋‹จ๋ฐœ itest_v1_2 3,000๊ฑด) ์„ฑ์ . 28,000๊ฑด ํ•ฉ์„ฑ ๊ต์œก ๋ฌธ์„œ์—์„œ ์ฐจ๋“ฑ ํ•™์Šต๋ฅ (backbone 3e-5, classifier 1e-4, ์ฝ”์‚ฌ์ธ ์Šค์ผ€์ค„, ์‹คํšจ ๋ฐฐ์น˜ 32, bf16)๋กœ ํ•™์Šตํ•˜์—ฌ 1.5์—ํญ(step=1,783, 7.44M ํ† ํฐ)์—์„œ ์ตœ๊ณ  ๊ฒฐํ•ฉ ์ ์ˆ˜(selection_score=1.7843, ์ •ํ˜• ์žฌํ˜„์œจ 100.0%, ๊ณต์ธ ์˜คํƒ 2.31%)๋ฅผ ๋‹ฌ์„ฑํ•จ.


2. Stage 1 ๊ต์‚ฌ ํ•™์Šต ์„ค์ • ๋ฐ ์—ํญ๋ณ„ ์ˆ˜๋ ด ์ถ”์ด

  • ๋ฐ์ดํ„ฐ์…‹: ๊ต์‚ฌ ์—…๋ฌด 12์žฅ๋ฅด ๋™๊ฒฐ ํ‹€ ์€ํ–‰ ๊ธฐ๋ฐ˜ ๊ฒฐ์ •๋ก ์  ํ•ฉ์„ฑ ๋ฐ์ดํ„ฐ(train 28,000 / dev_v1_2 3,000 / ๋‹จ๋ฐœ itest_v1_2 3,000). ์‹ค์ œ ํ•™์ƒ ๋ฐ์ดํ„ฐ 0๊ฑด, ์™ธ๋ถ€ ์ƒ์šฉ LLM ์ƒ์„ฑ๋ฌธ 0๊ฑด.
  • ์ตœ์ ํ™”: AdamW (backbone_lr=3e-5, classifier_lr=1e-4, weight_decay=0.01, warmup_ratio=0.06, max_grad_norm=1.0, micro_batch=16 ร— grad_accum=2), ์Šฌ๋กฏ ๋™์  ์น˜ํ™˜ ์ฆ๊ฐ• 15%, ๋‹ค์ค‘ ์Šค์ผ€์ผ ์ฐฝ(W-400 30% ํ˜ผํ•ฉ).
ํ‰๊ฐ€ ์‹œ์  (epoch) ๊ธ€๋กœ๋ฒŒ ์Šคํ… ๋ˆ„์  ์œ ํšจ ํ† ํฐ dev_loss selection_score macro_core_recall formal_recall homonym_student public_person_fp ๋น„๊ณ 
0.5 593 2.55M 0.00408 1.7837 98.58% 100.00% 97.57% 0.33% ์ดˆ๊ธฐ ์ˆ˜๋ ด
1.0 1,187 4.95M 0.00446 1.7700 98.55% 100.00% 93.69% 0.33% no_imp=1
1.5 1,783 7.44M 0.00620 1.7843 98.60% 100.00% 97.09% 2.31% ์ตœ์ข… ์ฑ„ํƒ (*BEST*)
2.0 2,379 9.91M 0.00496 1.7584 98.62% 100.00% 96.60% 8.91% no_imp=1
2.5 2,976 12.40M 0.00861 1.7708 98.64% 100.00% 96.12% 8.25% no_imp=2
3.0 3,573 14.88M 0.01840 1.7785 98.66% 100.00% 98.06% 9.24% ์กฐ๊ธฐ ์ข…๋ฃŒ (patience=3)

๋ถ„ํฌ ๋‚ด dev_v1_2 (3,000๊ฑด) ๋ฐ ๋ฏธ๋…ธ์ถœ ๋‹จ๋ฐœ itest_v1_2 (3,000๊ฑด) ๋น„๊ต (W-400, pipeline ๋ทฐ)

๋ถ„ํ•  ๋ชจ๋ธ macro_core_recall overall_core_recall formal_recall PERSON SCHOOL ORG ADDRESS public_person_fp (NER)
dev_v1_2 (3,000) kestrel-edu-ner-22l (๊ต์‚ฌ) 98.71% 98.32% 100.00% 98.24% 100.00% 96.60% 100.00% 2.31%
dev_v1_2 (3,000) kestrel-edu-ner-6l (๋ณด์ • ํ•™์ƒ) 98.45% 97.84% 100.00% 97.69% 99.60% 96.60% 99.90% 5.28%
itest_v1_2 (3,000) kestrel-edu-ner-22l (๊ต์‚ฌ) 97.84% 96.79% 100.00% 96.60% 100.00% 94.90% 99.85% 0.00%
itest_v1_2 (3,000) kestrel-edu-ner-6l (๋ณด์ • ํ•™์ƒ) 97.44% 95.77% 100.00% 95.44% 99.56% 94.75% 100.00% 3.74%

3. ํ˜ธ์ถœ ์˜ˆ์ œ (transformers) ๋ฐ ๊ณ ์ง€

import torch
from transformers import AutoModelForTokenClassification, AutoTokenizer

REPO_ID = "mulkzz/kestrel-edu-ner-22l"
REV = "v1.2.1"  # ๋˜๋Š” ๊ธฐ๋ณธ ๋ธŒ๋žœ์น˜ "main"
tokenizer = AutoTokenizer.from_pretrained(REPO_ID, revision=REV)
model = AutoModelForTokenClassification.from_pretrained(REPO_ID, revision=REV).eval()

text = "๋ณ„๋น›๊ณ  ๋ฐ• ๋ฏธ๋ž‘ ํ•™์ƒ๊ณผ ์ดˆ๋กฑ์ด๊ฐ€ ํ•ด์˜ค๋ฆ„์ƒ๋‹ด์„ผํ„ฐ(๋ฌผ์•ˆ๊ฐœ๋กœ 58, 1์ธต)์— ๋ฐฉ๋ฌธํ–ˆ๋‹ค."
inputs = tokenizer(text, return_offsets_mapping=True, return_tensors="pt")
offsets = inputs.pop("offset_mapping")[0].tolist()

with torch.no_grad():
    pred = model(**inputs).logits[0].argmax(dim=-1).tolist()

for pid, (s, e) in zip(pred, offsets):
    if e > s and pid != 0:
        print(model.config.id2label[pid], text[s:e], (s, e))
  • ์ฃผ์˜ (pad_token_id = 49999): ์ˆ˜๋™ ๋ฐฐ์น˜ ํŒจ๋”ฉ ์‹œ ๋ฐ˜๋“œ์‹œ 49999๋ฅผ ์‚ฌ์šฉํ•˜์‹ญ์‹œ์˜ค(0์€ [CLS]).
  • ํ•œ๊ณ„ยท์™ธ๋ถ€ ๋ฐ์ดํ„ฐ ๊ท€์†ยท๋ผ์ด์„ ์Šค: ๋ณธ ๋ชจ๋ธ์€ ๊ฐ€๋ช…ํ™” ๋ณด์กฐ ์—ฐ๊ตฌยท์ฆ๋ฅ˜์šฉ ์ƒ์œ„ ๋ชจ๋ธ์ด๋ฉฐ ๋ชจ๋“  ๊ฐœ์ธ์ •๋ณด์˜ ์™„์ „ํ•œ ํƒ์ง€๋‚˜ ๋ฒ•์  ๋ฉด์ฑ…์„ ๋ณด์ฆํ•˜์ง€ ์•Š์Šต๋‹ˆ๋‹ค. ์™ธ๋ถ€ ํ‰๊ฐ€ ์ฝ”ํผ์Šค(townboy/korean-pii-dataset CC BY 4.0, KDPII Zenodo 10968609 CC BY 4.0, ์ฝ๊ธฐ ์ „์šฉ ์ง‘๊ณ„๋งŒ ์‚ฌ์šฉ) ์ƒ์„ธ ๊ฒฐ๊ณผ๋Š” mulkzz/kestrel-edu-ner-6l ๋ชจ๋ธ ์นด๋“œ๋ฅผ ์ฐธ์กฐํ•˜์‹ญ์‹œ์˜ค. ๋ผ์ด์„ ์Šค๋Š” Apache-2.0 (NOTICE)์ž…๋‹ˆ๋‹ค.
Downloads last month
64
Safetensors
Model size
0.1B params
Tensor type
F32
ยท
Inference Providers NEW
This model isn't deployed by any Inference Provider. ๐Ÿ™‹ Ask for provider support

Model tree for mulkzz/kestrel-edu-ner-22l

Finetuned
(17)
this model