vtuber / README.md
Junping0645's picture
Update README.md
423e873 verified
|
Raw History Blame Contribute Delete
3.27 kB
metadata
base_model: EleutherAI/polyglot-ko-3.8b
library_name: peft
pipeline_tag: text-generation
language:
  - ko
tags:
  - base_model:adapter:EleutherAI/polyglot-ko-3.8b
  - lora
  - transformers
  - korean
  - chatbot

๋ง๋™๋ฌด โ€” ๋…๊ฑฐ ์–ด๋ฅด์‹ ์šฉ AI ๋Œ€ํ™” ์ฑ—๋ด‡ (QLoRA ์–ด๋Œ‘ํ„ฐ)

EleutherAI/polyglot-ko-3.8b๋ฅผ QLoRA(4bit NF4)๋กœ ํŒŒ์ธํŠœ๋‹ํ•œ LoRA ์–ด๋Œ‘ํ„ฐ์ž…๋‹ˆ๋‹ค. ๋…๊ฑฐ ์–ด๋ฅด์‹ ๊ณผ ์งง๊ณ  ๋”ฐ๋œปํ•œ ์กด๋Œ“๋ง๋กœ ๋Œ€ํ™”ํ•˜๋ฉฐ, ๋จผ์ € ๊ณต๊ฐํ•˜๊ณ  ํ•„์š”์‹œ ์•ˆ๋ถ€๋ฅผ ์ฑ™๊ธฐ๋Š” ํŽ˜๋ฅด์†Œ๋‚˜๋กœ ํ•™์Šตํ–ˆ์Šต๋‹ˆ๋‹ค.

ํ•™์Šต ๋ฐ์ดํ„ฐ

  • ๋‹จ๋ฐœ ๋Œ€ํ™” 11,266๊ฐœ: 16๊ฐœ ์นดํ…Œ๊ณ ๋ฆฌ(๊ฑด๊ฐ•/์™ธ๋กœ์›€/์†์ฃผ/๋ช…์ ˆ ๋“ฑ) ร— 8๊ฐœ ๊ฐ์ •, GPT-4o-mini๋กœ ์ƒ์„ฑ + ์‹ค์ œ ์–ด๋ฅด์‹  ์ธํ„ฐ๋ทฐ ์ „์‚ฌ์—์„œ ์ถ”์ถœํ•œ ์‚ฌํˆฌ๋ฆฌ/๊ตฌ์–ด์ฒด ๋ฐœํ™” 1,266๊ฐœ(์‚ฌํˆฌ๋ฆฌ ์ž…๋ ฅ ์ดํ•ด ๋ณด๊ฐ•์šฉ)
  • ๋ฉ€ํ‹ฐํ„ด ๋Œ€ํ™” 1,000ํŽธ(๋Œ€ํ™”๋‹น ํ‰๊ท  7.1ํ„ด) โ†’ ํ„ด๋งˆ๋‹ค ์Šฌ๋ผ์ด๋”ฉ ์œˆ๋„์šฐ๋กœ ํ™•์žฅํ•ด 3,546๊ฐœ ํ•™์Šต ์ƒ˜ํ”Œ๋กœ ๋ณ€ํ™˜
  • ์ตœ์ข… train 14,072 / val 740

ํ•™์Šต ์„ค์ •

  • LoRA: r=8, alpha=16, dropout=0.1, target_modules=["query_key_value", "dense"] (GPT-NeoX ์–ดํ…์…˜ ์œ„์ฃผ๋กœ ์ข๊ฒŒ ํƒ€๊นƒ โ€” ๊ณผ์ ํ•ฉ ์–ต์ œ)
  • 4bit NF4 ์–‘์žํ™”(bitsandbytes), MAX_LENGTH=768
  • batch_size=2, gradient_accumulation_steps=8 (์œ ํšจ ๋ฐฐ์น˜ 16), 5 epoch ์บก + EarlyStoppingCallback(patience=3)
  • ๊ฒฐ๊ณผ: epoch 3.296์—์„œ early stopping, eval_loss 1.366, RTX 3060 Ti(8GB)์—์„œ 141.9๋ถ„ ์†Œ์š”, peak VRAM 3.4GB

์‚ฌ์šฉ๋ฒ•

๋ฒ ์ด์Šค ๋ชจ๋ธ(7.6GB)์„ 4bit๋กœ ๋กœ๋“œํ•œ ๋’ค ์ด ์–ด๋Œ‘ํ„ฐ๋ฅผ ์–น์Šต๋‹ˆ๋‹ค.

import torch
from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer, BitsAndBytesConfig

BASE_MODEL = "EleutherAI/polyglot-ko-3.8b"
ADAPTER = "Junping0645/Vtuber"  # ์ด ๋ฆฌํฌ

tokenizer = AutoTokenizer.from_pretrained(BASE_MODEL)
bnb = BitsAndBytesConfig(
    load_in_4bit=True,
    bnb_4bit_quant_type="nf4",
    bnb_4bit_compute_dtype=torch.float16,
    bnb_4bit_use_double_quant=True,
)
base = AutoModelForCausalLM.from_pretrained(BASE_MODEL, quantization_config=bnb, device_map={"": 0})
model = PeftModel.from_pretrained(base, ADAPTER)
model.eval()

prompt = "### ์–ด๋ฅด์‹ : ์š”์ฆ˜ ๋งŽ์ด ์™ธ๋กญ์Šต๋‹ˆ๋‹ค.\n### ๋ง๋™๋ฌด:"
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
out = model.generate(**inputs, max_new_tokens=80, do_sample=True, temperature=0.8,
                      top_p=0.9, repetition_penalty=1.15,
                      pad_token_id=tokenizer.eos_token_id, eos_token_id=tokenizer.eos_token_id)
print(tokenizer.decode(out[0], skip_special_tokens=True))

๋ฉ€ํ‹ฐํ„ด(๋Œ€ํ™” ์ด๋ ฅ ์œ ์ง€)

์ด์ „ ํ„ด๋“ค์„ ๊ฐ™์€ ํฌ๋งท์œผ๋กœ ์ด์–ด๋ถ™์ด๋ฉด ๋ฌธ๋งฅ์„ ๋ฐ˜์˜ํ•œ ๋‹ต๋ณ€์ด ๋‚˜์˜ต๋‹ˆ๋‹ค(ํ•™์Šต ์‹œ ์‚ฌ์šฉํ•œ ํฌ๋งท๊ณผ ๋™์ผํ•ด์•ผ ํ•จ):

### ์–ด๋ฅด์‹ : ์†์ฃผ๊ฐ€ ๋‹ค์Œ ์ฃผ์— ๋†€๋Ÿฌ ์˜จ๋Œ€์š”.
### ๋ง๋™๋ฌด: ์ •๋ง์š”? ๊ธฐ๋Œ€๋˜์‹œ๊ฒ ์–ด์š”! ์ฆ๊ฑฐ์šด ์‹œ๊ฐ„ ๋ณด๋‚ด์‹œ๊ธธ ๋ฐ”๋ž˜์š”.
### ์–ด๋ฅด์‹ : ๋ช‡ ์‚ด์ธ์ง€ ์•ˆ ๋ฌผ์–ด๋ณด์…จ๋„ค์š”, ์ด๋ฒˆ์— ์ดˆ๋“ฑํ•™๊ต ๋“ค์–ด๊ฐ€์š”.
### ๋ง๋™๋ฌด: ๊ทธ๋ ‡๊ตฐ์š”! ์š”์ฆ˜ ์•„์ด๋“ค์€ ๋น ๋ฅด์ฃ .
### ์–ด๋ฅด์‹ : ๊ฑ”๊ฐ€ ์˜ค๋ฉด ๋ญ˜ ํ•ด์ฃผ๋ฉด ์ข‹์•„ํ• ๊นŒ์š”?
### ๋ง๋™๋ฌด:

์ฐธ๊ณ 

  • VRAM: ํ•™์Šต 3.4GB / ์ถ”๋ก  ์•ฝ 3.3GB (RTX 3060 Ti 8GB ๊ธฐ์ค€)
  • Framework: PEFT 0.19.1, transformers 5.13.0, bitsandbytes 0.49.2, torch 2.11.0+cu128