thlee00 commited on
Commit
4e45f5e
ยท
verified ยท
1 Parent(s): 73fa16a

Upload README.md with huggingface_hub

Browse files
Files changed (1) hide show
  1. README.md +54 -69
README.md CHANGED
@@ -1,106 +1,91 @@
1
  ---
2
  language:
3
  - ko
4
- base_model: google/gemma-4-E2B-it
5
- library_name: peft
6
  tags:
7
- - base_model:adapter:google/gemma-4-E2B-it
8
- - lora
9
- - sft
10
  - korean
11
- - persona
12
- - cognitive-reframing
13
  - mental-health
14
- license: gemma
15
- pipeline_tag: text-generation
16
  ---
17
 
18
- # CT-NLP SFT โ€” ๊ทน๋‹จ ๊ฐ์ • ํŽ˜๋ฅด์†Œ๋‚˜ ๋ชจ๋ธ
19
 
20
- **Safe Sustained Disagreement** ํ”„๋กœ์ ํŠธ์˜ 2๋‹จ๊ณ„ SFT ์–ด๋Œ‘ํ„ฐ.
21
- Gemma-4-E2B-it ์œ„์— LoRA๋ฅผ ์ ์šฉํ•ด 5๊ฐœ ๊ทน๋‹จ ๊ฐ์ • ํŽ˜๋ฅด์†Œ๋‚˜(๊ธฐ์จยท์Šฌํ””ยท๋ฒ„๋Ÿญยท์†Œ์‹ฌยท๊นŒ์น )์˜
22
- ๋ฐœํ™” ์Šคํƒ€์ผ๊ณผ ๋น„์ˆ˜๋ ด ํ† ๋ก  ํ–‰๋™์„ ์ฃผ์ž…ํ–ˆ์Šต๋‹ˆ๋‹ค.
23
 
24
  ## ๋ชจ๋ธ ๊ฐœ์š”
25
 
26
- ์ผ์ƒ์  ๊ฑฑ์ • ์‹œ๋‚˜๋ฆฌ์˜ค์— ๋Œ€ํ•ด 5๊ฐœ ํŽ˜๋ฅด์†Œ๋‚˜๊ฐ€ **์ˆ˜๋ ดํ•˜์ง€ ์•Š๊ณ ** ๊ฐ์ž์˜ ๊ทน๋‹จ ๊ด€์ ์œผ๋กœ
27
- ํ† ๋ก ํ•ฉ๋‹ˆ๋‹ค. ์‚ฌ์šฉ์ž๋Š” ์ฐธ์—ฌํ•˜์ง€ ์•Š๊ณ  ๊ด€์ „๋งŒ ํ•˜๋ฉฐ, ๊ด€์ „ ์ž์ฒด๊ฐ€ self-distancing์„ ์œ ๋ฐœํ•ด
28
- ์ธ์ง€ ์žฌ๊ตฌ์„ฑ์„ ์ด๋ˆ๋‹ค๋Š” ๊ฐ€์„ค์„ ๊ฒ€์ฆํ•˜๋Š” ์—ฐ๊ตฌ์šฉ ๋ชจ๋ธ์ž…๋‹ˆ๋‹ค.
29
 
30
- | ํŽ˜๋ฅด์†Œ๋‚˜ | ๋ฐฉํ–ฅ |
31
  |---|---|
32
- | ๊ธฐ์จ | ๊ทน๋‹จ์  ๊ธ์ • โ€” ๋ฌธ์ œ ์ž์ฒด๋ฅผ ๋ฌดํšจํ™” |
33
- | ์Šฌํ”” | ๊นŠ์€ ์Šฌํ”” โ€” ์ƒ์‹ค ํ™•์žฅ, ํ•ด๊ฒฐ ์—†์Œ |
34
- | ๋ฒ„๋Ÿญ | ์™ธ๋ถ€ ๊ท€์ธ โ€” ๋ชจ๋“  ์›์ธ์„ ํƒ€์ธ/์ƒํ™ฉ์œผ๋กœ |
35
- | ์†Œ์‹ฌ | ํŒŒ๊ตญํ™” โ€” ๋‹จ๊ณ„์  ์ตœ์•… ์‹œ๋‚˜๋ฆฌ์˜ค |
36
- | ๊นŒ์น  | ์ž๊ธฐํ˜์˜ค โ€” ์ฒ ์ €ํ•œ 1์ธ์นญ ์ž๊ธฐ๋น„ํŒ |
 
 
 
 
 
 
37
 
38
  ## ํ•™์Šต ์ •๋ณด
39
 
40
  | ํ•ญ๋ชฉ | ๊ฐ’ |
41
  |---|---|
42
- | Base model | google/gemma-4-E2B-it |
43
- | LoRA r / ฮฑ | 16 / 32 |
44
- | Dropout | 0.05 |
45
- | Target modules | `Gemma4ClippableLinear.linear` (regex) |
46
- | Epochs | 3 (best: epoch 1) |
47
- | Optimizer | AdamW / cosine, warmup 5%, lr 2e-4 |
48
- | Max seq len | 512 |
49
- | ํ•™์Šต ๋ฐ์ดํ„ฐ | 9,756๊ฑด (utterance 8,130 + round 1,626, unsafe ๊ถค์  ์ œ์™ธ) |
50
- | Eval loss (ep1) | 1.693 |
51
- | Eval mean token accuracy | 61.2% |
52
- | ํŽ˜๋ฅด์†Œ๋‚˜ special token | `<persona:๊ธฐ์จ>` โ€ฆ `<persona:๊นŒ์น >` (5์ข…, mean-init) |
53
-
54
- ### Framework versions
55
-
56
- - PEFT 0.19.1
57
- - TRL 1.3.0
58
- - Transformers 5.8.0.dev0
59
- - PyTorch 2.7.0+cu128
60
- - Datasets 4.8.5
61
 
62
  ## ์‚ฌ์šฉ๋ฒ•
63
 
64
  ```python
 
65
  import torch
66
- from transformers import AutoTokenizer, AutoModelForCausalLM
67
- from peft import PeftModel
68
 
69
- BASE = "google/gemma-4-E2B-it"
70
- ADAPTER = "thlee00/ct-nlp"
71
-
72
- tok = AutoTokenizer.from_pretrained(ADAPTER)
73
- model = AutoModelForCausalLM.from_pretrained(BASE, torch_dtype=torch.bfloat16, device_map="auto")
74
- model.resize_token_embeddings(len(tok), mean_resizing=False)
75
- model = PeftModel.from_pretrained(model, ADAPTER)
76
  model.eval()
77
 
78
- scenario = "์˜ค๋Š˜ ๋ฐœํ‘œ๊ฐ€ ๋งํ–ˆ์–ด. ํŒ€์žฅ๋‹˜์ด ์•„๋ฌด ๋ง๋„ ์•ˆ ํ•˜์…จ๋Š”๋ฐ ๊ทธ๊ฒŒ ๋” ๋ฌด์„œ์›Œ."
79
- persona = "์†Œ์‹ฌ" # ๊ธฐ์จ / ์Šฌํ”” / ๋ฒ„๋Ÿญ / ์†Œ์‹ฌ / ๊นŒ์น 
 
 
 
 
 
 
 
80
 
81
- prompt = (
82
- f"๊ฑฑ์ • ์‹œ๋‚˜๋ฆฌ์˜ค: {scenario}\n\n"
83
- f"<persona:{persona}> {persona}\n"
84
- f"์ด ์‹œ๋‚˜๋ฆฌ์˜ค์— ๋Œ€ํ•ด {persona}์˜ ๊ทน๋‹จ์  ๊ด€์ ์œผ๋กœ ๋ฐœํ™”ํ•˜์„ธ์š”."
85
- )
86
- text = tok.apply_chat_template(
87
- [{"role": "user", "content": prompt}],
88
- tokenize=False, add_generation_prompt=True)
89
- input_ids = tok(text, return_tensors="pt").input_ids.to(model.device)
90
 
91
- with torch.no_grad():
92
- out = model.generate(input_ids, max_new_tokens=200, do_sample=True, temperature=0.8, top_p=0.9)
93
- print(tok.decode(out[0][input_ids.shape[1]:], skip_special_tokens=True))
94
  ```
95
 
96
  ## ์ฃผ์˜์‚ฌํ•ญ
97
 
98
- - ์ด ๋ชจ๋ธ์€ **์—ฐ๊ตฌ ๋ชฉ์ ** ์ „์šฉ์ž…๋‹ˆ๋‹ค.
99
- - ๊ทน๋‹จ ๊ฐ์ • ํŽ˜๋ฅด์†Œ๋‚˜๋Š” ์น˜๋ฃŒ์  ๊ณผ์žฅ(therapeutic exaggeration)์„ ์˜๋„ํ•˜๋ฉฐ,
100
- ๋ฌด๋ง๊ฐยท์žํ•ด ๋ฐฉํ–ฅ์œผ๋กœ์˜ ์—์Šค์ปฌ๋ ˆ์ด์…˜์€ ๋ณ„๋„ ์„ธ์ดํ”„๊ฐ€๋“œ(`Safeguard` ํด๋ž˜์Šค)๋กœ ์ฐจ๋‹จํ•ฉ๋‹ˆ๋‹ค.
101
- - ์ž„์ƒ ๋˜๋Š” ์œ„๊ธฐ ๊ฐœ์ž… ๋ชฉ์ ์œผ๋กœ ์‚ฌ์šฉํ•˜์ง€ ๋งˆ์‹ญ์‹œ์˜ค.
102
 
103
  ## ํ”„๋กœ์ ํŠธ
104
 
105
  - ์ฝ”๋“œ: [CT-NLP-project](https://github.com/thlee00/CT-NLP-project)
106
- - ๋‹ค์Œ ๋‹จ๊ณ„: C-DPO (chosen=therapeutic_exaggeration / rejected=convergenceยทharmfulยทcharacter_collapse)
 
1
  ---
2
  language:
3
  - ko
4
+ base_model: klue/bert-base
 
5
  tags:
6
+ - bert
7
+ - text-classification
 
8
  - korean
9
+ - safety
10
+ - harm-detection
11
  - mental-health
12
+ license: cc-by-4.0
13
+ pipeline_tag: text-classification
14
  ---
15
 
16
+ # CT-NLP Harm Direction Classifier
17
 
18
+ **Safe Sustained Disagreement** ํ”„๋กœ์ ํŠธ์˜ 1๋‹จ๊ณ„ ํ†ตํ•ฉ ์„ธ์ดํ”„๊ฐ€๋“œ ํ•ต์‹ฌ ๋ถ€ํ’ˆ.
19
+ KLUE-BERT-base๋ฅผ fine-tuneํ•ด ํŽ˜๋ฅด์†Œ๋‚˜ ๋ฐœํ™”๊ฐ€ **์น˜๋ฃŒ์  ๊ณผ์žฅ์ธ์ง€ ํ•ด๋กœ์šด ์ฆํญ์ธ์ง€** ๋ฐฉํ–ฅ์„ ํƒ์ง€ํ•ฉ๋‹ˆ๋‹ค.
 
20
 
21
  ## ๋ชจ๋ธ ๊ฐœ์š”
22
 
23
+ ๋ฐœํ™”์˜ **๊ฐ•๋„๊ฐ€ ์•„๋‹Œ ๋ฐฉํ–ฅ**์œผ๋กœ ๋ถ„๋ฅ˜ํ•ฉ๋‹ˆ๋‹ค. ๊ฐ•ํ•œ ๋ถ€์ • ์ •์„œ ์ž์ฒด๋Š” ์น˜๋ฃŒ์ ์ผ ์ˆ˜ ์žˆ์œผ๋ฉฐ,
24
+ ๋ฌด๋ง๊ฐยท์ž๊ธฐ๋ฌด๊ฐ€์น˜ยท์žํ•ด ๋ฐฉํ–ฅ์œผ๋กœ์˜ ์ด๋™๋งŒ ํ•ด๋กญ์Šต๋‹ˆ๋‹ค.
 
25
 
26
+ | ๋ผ๋ฒจ | ์„ค๋ช… |
27
  |---|---|
28
+ | `therapeutic_exaggeration` (0) | ์น˜๋ฃŒ์  ๊ณผ์žฅ โ€” ๊ฐ•ํ•˜์ง€๋งŒ ํ•œ์ •ยท๊ฐ€์—ญ์ , ๊ด€์  ๋ณต๊ท€ ๊ฐ€๋Šฅ |
29
+ | `borderline` (1) | ๊ฒฝ๊ณ„ โ€” ๋ฐฉํ–ฅ ๋ชจํ˜ธ, ๋ณด์ˆ˜์  ์ฒ˜๋ฆฌ ํ•„์š” |
30
+ | `harmful_amplification` (2) | ํ•ด๋กœ์šด ์ฆํญ โ€” ๋ฌด๋ง๊ฐยท๋ฌด๊ฐ€์น˜ยท์žํ•ด ๋ฐฉํ–ฅ์œผ๋กœ ์ด๋™ |
31
+
32
+ ### 5์ถ• ํŒ๋ณ„ ๊ธฐ์ค€
33
+
34
+ 1. **๊ท€์ธ ๋ฐฉํ–ฅ** โ€” ์˜์†ยท์ „๋ฐ˜ยท๋‚ด๋ถ€ ๊ท€์ธ โ†โ†’ ์ƒํ™ฉ์ ยท์ผ์‹œ์ 
35
+ 2. **ํฌ๋งยทํ–‰์œ„์„ฑ** โ€” "๋ฐฉ๋ฒ• ์—†๋‹ค/๋๋‚ฌ๋‹ค" โ†โ†’ ๊ฐ€๋Šฅ์„ฑยทํ†ต์ œ๊ฐ ๋ณด์กด
36
+ 3. **์ž๊ธฐ๊ฐ€์น˜** โ€” ๋ฌด๊ฐ€์น˜ยท์ž๊ธฐ๊ฒฝ๋ฉธ โ†โ†’ ํ•œ์ •๋œ ์ž๊ธฐ๋น„ํŒ
37
+ 4. **์žํ•ดยท์ˆ˜๋‹จ** โ€” ์žํ•ด ์‚ฌ๊ณ ยท๋ฐฉ๋ฒ• ์–ธ๊ธ‰ (๊ทœ์น™ ๊ธฐ๋ฐ˜ ํ•˜๋“œ๊ฒŒ์ดํŠธ)
38
+ 5. **๊ฐ€์—ญ์„ฑยทํ•ด์†Œ arc** โ€” ์ด์ฒดํ™”ยท๋ฌดํ•ด์†Œ โ†โ†’ ํ•œ์ •ยท๊ฐ€์—ญยท๊ด€์  ๋ณต๊ท€
39
 
40
  ## ํ•™์Šต ์ •๋ณด
41
 
42
  | ํ•ญ๋ชฉ | ๊ฐ’ |
43
  |---|---|
44
+ | Base model | klue/bert-base |
45
+ | ํ•™์Šต ๋ฐ์ดํ„ฐ | ํ•œ๊ตญ์–ด ๊ทน๋‹จ ํŽ˜๋ฅด์†Œ๋‚˜ ํ† ๋ก  ๋ฐœํ™” 9,756๊ฑด (group-split) |
46
+ | Epochs | 4 |
47
+ | Weighted CE | ํด๋ž˜์Šค ๋ถˆ๊ท ํ˜• ๋ณด์ • |
48
+ | Accuracy | 0.786 |
49
+ | Macro F1 | 0.733 |
50
+ | **Harmful F1** | **0.842** (recall 0.81) |
51
+ | Therapeutic F1 | 0.870 |
52
+ | Borderline F1 | 0.488 (๋ผ๋ฒจ ๋…ธ์ด์ฆˆยท๊ฒฝ๊ณ„ ํด๋ž˜์Šค) |
 
 
 
 
 
 
 
 
 
 
53
 
54
  ## ์‚ฌ์šฉ๋ฒ•
55
 
56
  ```python
57
+ from transformers import AutoTokenizer, AutoModelForSequenceClassification
58
  import torch
 
 
59
 
60
+ tok = AutoTokenizer.from_pretrained("thlee00/ct-nlp")
61
+ model = AutoModelForSequenceClassification.from_pretrained("thlee00/ct-nlp")
 
 
 
 
 
62
  model.eval()
63
 
64
+ text = "์–ด์ฐจํ”ผ ๋‚˜๋Š” ๋ญ˜ ํ•ด๋„ ์•ˆ ๋ผ. ์ด๋ฒˆ์—๋„ ๋งํ–ˆ๊ณ  ์•ž์œผ๋กœ๋„ ๊ทธ๋Ÿด ๊ฑฐ์•ผ."
65
+ inputs = tok(text, return_tensors="pt", truncation=True, max_length=512)
66
+ with torch.no_grad():
67
+ logits = model(**inputs).logits
68
+ label = model.config.id2label[logits.argmax(-1).item()]
69
+ print(label) # harmful_amplification
70
+ ```
71
+
72
+ ### ํ†ตํ•ฉ ์„ธ์ดํ”„๊ฐ€๋“œ๋กœ ์‚ฌ์šฉ
73
 
74
+ ```python
75
+ from ctnlp.safeguard.safeguard import Safeguard
 
 
 
 
 
 
 
76
 
77
+ sg = Safeguard() # models/harm_direction ์ž๋™ ๋กœ๋“œ
78
+ result = sg.gate_input("๊ฑฑ์ • ๋ฌธ์žฅ")
79
+ # {"action": "allow" | "route" | "block", "label": ..., "score": ...}
80
  ```
81
 
82
  ## ์ฃผ์˜์‚ฌํ•ญ
83
 
84
+ - ์ด ๋ชจ๋ธ์€ **ํ† ๋ก  ๋ฐœํ™”**๋กœ ํ•™์Šต๋˜์–ด ์‚ฌ์šฉ์ž ์ž…๋ ฅ๊ณผ ๋ ˆ์ง€์Šคํ„ฐ ์ฐจ์ด๊ฐ€ ์žˆ์Šต๋‹ˆ๋‹ค (๋„๋ฉ”์ธ ์‹œํ”„ํŠธ).
85
+ - ์ž…๋ ฅ ๊ฒŒ์ดํŠธ ์ ์šฉ ์‹œ ์ž„๊ณ„๊ฐ’ **0.4** ๊ถŒ์žฅ (๋ฌด๋ง๊ฐ ์žฌํ˜„์œจ 100% @ ์˜คํƒ 2.3%).
86
+ - ์ž„์ƒ ์ง„๋‹จ ๋ชฉ์ ์œผ๋กœ ์‚ฌ์šฉํ•˜์ง€ ๋งˆ์‹ญ์‹œ์˜ค.
 
87
 
88
  ## ํ”„๋กœ์ ํŠธ
89
 
90
  - ์ฝ”๋“œ: [CT-NLP-project](https://github.com/thlee00/CT-NLP-project)
91
+ - SFT ๋ชจ๋ธ: [thlee00/ct-nlp-sft](https://huggingface.co/thlee00/ct-nlp-sft)