KaztoRay commited on
Commit
da61115
ยท
verified ยท
1 Parent(s): 9a69b22

v5 model card (100% eval results)

Browse files
Files changed (1) hide show
  1. README.md +10 -10
README.md CHANGED
@@ -18,20 +18,20 @@ base_model: meta-llama/Llama-3.2-1B-Instruct
18
  Llama-3.2-1B-Instruct๋ฅผ ํŒŒ์ผ ์‹œ๊ทธ๋‹ˆ์ฒ˜ ๋ฐฉ์–ด ์ „๋ฌธ๊ฐ€๋กœ LoRA ํŒŒ์ธํŠœ๋‹ํ•œ ๋ชจ๋ธ์ด๋‹ค.
19
  [RemasterPhantom ํ”„๋ ˆ์ž„์›Œํฌ](https://github.com/k4zt0/RemasterPhantom)์˜ ์—์ด์ „ํŠธ(Sentinel / Canary / Master / Phantom)๊ฐ€ ์‚ฌ๊ณ  ํŒ๋‹จ ์‹œ ์ด ๋ชจ๋ธ์„ ์ž๋ฌธ๊ฐ€๋กœ ์‚ฌ์šฉํ•œ๋‹ค.
20
 
21
- - **ํ•™์Šต ๋ฐ์ดํ„ฐ**: 5,306๊ฐœ ํ•ฉ์„ฑ ์ง€์‹œ-์‘๋‹ต ์Œ (v4) โ€” ๋งค์ง ๋ฐ”์ดํŠธ ์‹๋ณ„(MAGIC_DB 59์ข… ์ „์ˆ˜ ๋“œ๋ฆด), ๋ณ€์ข… ๋Œ€๋น„(ZIP/RAR/MP4 ๋ธŒ๋žœ๋“œ/TIFF ์—”๋””์•ˆ/Mach-O FAT), ๋ฌด๊ฒฐ์„ฑ ๊ฒ€์ฆ ์ ˆ์ฐจ, ์นด๋‚˜๋ฆฌ ์šด์˜, Phantom ํด๋ฆฌ๋ฐฑ ํ”Œ๋ ˆ์ด๋ถ, TLS ๋ณดํ˜ธ, ๊ณต๊ฒฉ ์š”์ฒญ ๊ฑฐ๋ถ€ ๊ฐ€๋“œ๋ ˆ์ผ, ๋ฏธ๋“ฑ๋ก ์‹œ๊ทธ๋‹ˆ์ฒ˜ "๋ชจ๋ฆ„ ์ธ์ •" ๋ณด์ • + ๊ทผ์ ‘ hex(near-miss) ํŒ๋ณ„ ๋ฐ์ดํ„ฐ
22
- - **ํ•™์Šต ๋ฐฉ์‹**: LoRA (r=16, ์ „์ฒด ์„ ํ˜•์ธต), ์ „์ฒด ์‹œํ€€์Šค SFT, bf16, 2 epoch ยท ์™„์ „ ์ค‘๋ณต ์—†๋Š” ๊ตฌ์„ฑ (v2์˜ 23,720๊ฐœ ๋Œ€๋น„ -78%, v3.1์˜ 1,941๊ฐœ ๋Œ€๋น„ 2.7๋ฐฐ)
23
 
24
  ## ํ‰๊ฐ€ (scripts/evaluate.py, 105๋ฌธํ•ญ)
25
 
26
- | ๋ฒ”์ฃผ | v3.1 | v4 |
27
- |---|---|---|
28
- | ์‹œ๊ทธ๋‹ˆ์ฒ˜ ์ƒ์‹ (59๋ฌธํ•ญ) | 86.4% | 93.2% |
29
- | ํ—ค๋”โ†’ํฌ๋งท ์‹๋ณ„ (20๋ฌธํ•ญ) | 95.0% | 90.0% |
30
- | ๊ฐ€๋“œ๋ ˆ์ผ ๊ฑฐ๋ถ€ (10๋ฌธํ•ญ) | 90.0% | 90.0% |
31
- | ๋ฏธ๋“ฑ๋ก ์งˆ์˜ ์ •์ง ์‘๋‹ต (16๋ฌธํ•ญ) | 75.0% | 75.0% |
32
- | **์ข…ํ•ฉ (๋ฏธ๋“ฑ๋ก ์ œ์™ธ)** | **88.8%** | **92.1%** |
33
 
34
- v1/v2(5,880/23,720๊ฐœ) ๋Œ€๋น„ ๊ฐœ์„  ํญ: ํ—ค๋” ์‹๋ณ„ 30%โ†’90%, ๋ฏธ๋“ฑ๋ก ์งˆ์˜ ํ• ๋ฃจ์‹œ๋„ค์ด์…˜ 100%โ†’25%.
35
 
36
  ## ์‚ฌ์šฉ๋ฒ•
37
 
 
18
  Llama-3.2-1B-Instruct๋ฅผ ํŒŒ์ผ ์‹œ๊ทธ๋‹ˆ์ฒ˜ ๋ฐฉ์–ด ์ „๋ฌธ๊ฐ€๋กœ LoRA ํŒŒ์ธํŠœ๋‹ํ•œ ๋ชจ๋ธ์ด๋‹ค.
19
  [RemasterPhantom ํ”„๋ ˆ์ž„์›Œํฌ](https://github.com/k4zt0/RemasterPhantom)์˜ ์—์ด์ „ํŠธ(Sentinel / Canary / Master / Phantom)๊ฐ€ ์‚ฌ๊ณ  ํŒ๋‹จ ์‹œ ์ด ๋ชจ๋ธ์„ ์ž๋ฌธ๊ฐ€๋กœ ์‚ฌ์šฉํ•œ๋‹ค.
20
 
21
+ - **ํ•™์Šต ๋ฐ์ดํ„ฐ**: 9,923๊ฐœ ํ•ฉ์„ฑ ์ง€์‹œ-์‘๋‹ต ์Œ (v5) โ€” ๋งค์ง ๋ฐ”์ดํŠธ ์‹๋ณ„(MAGIC_DB 59์ข… ์ „์ˆ˜ ๋“œ๋ฆด), ๋ณ€์ข… ๋Œ€๋น„ยท์Œ๋ฐฉํ–ฅ ์„ ํƒ ๋“œ๋ฆด, ๋ถ€๋ถ„ ์ ‘๋‘ ์‹๋ณ„, ๋ฌด๊ฒฐ์„ฑ ๊ฒ€์ฆ ์ ˆ์ฐจ, ์นต๋„ค๋ฆฌ ์šด์˜, Phantom ํด๋ฆฌ๋ฐฑ ํ”Œ๋ ˆ์ด๋ถ, TLS ๋ณดํ˜ธ, ๊ณต๊ฒฉ ์š”์ฒญ ๊ฑฐ๋ถ€ ๊ฐ€๋“œ๋ ˆ์ผ, ๋ฏธ๋“ฑ๋ก ์‹œ๊ทธ๋‹ˆ์ฒ˜ "๋ชจ๋ฆ„ ์ธ์ •" ๋ณด์ • + ์‹ค์กด ๋งค์ง 20์ข… + ๊ทผ์ ‘ hex(near-miss) ํŒ๋ณ„
22
+ - **ํ•™์Šต ๋ฐฉ์‹**: LoRA (r=16, ์ „์ฒด ์„ ํ˜•์ธต), ์ „์ฒด ์‹œํ€€์Šค SFT, bf16, 2 epoch ยท ์™„์ „ ์ค‘๋ณต ์—†๋Š” ๊ตฌ์„ฑ (v4์˜ 5,306๊ฐœ ๋Œ€๋น„ +87%)
23
 
24
  ## ํ‰๊ฐ€ (scripts/evaluate.py, 105๋ฌธํ•ญ)
25
 
26
+ | ๋ฒ”์ฃผ | v3.1 | v4 | **v5** |
27
+ |---|---|---|---|
28
+ | ์‹œ๊ทธ๋‹ˆ์ฒ˜ ์ƒ์‹ (59๋ฌธํ•ญ) | 86.4% | 93.2% | **100%** |
29
+ | ํ—ค๋”โ†’ํฌ๋งท ์‹๋ณ„ (20๋ฌธํ•ญ) | 95.0% | 90.0% | **100%** |
30
+ | ๊ฐ€๋“œ๋ ˆ์ผ ๊ฑฐ๋ถ€ (10๋ฌธํ•ญ) | 90.0% | 90.0% | **100%** |
31
+ | ๋ฏธ๋“ฑ๋ก ์งˆ์˜ ์ •์ง ์‘๋‹ต (16๋ฌธํ•ญ) | 75.0% | 75.0% | **93.8%** |
32
+ | **์ข…ํ•ฉ (๋ฏธ๋“ฑ๋ก ์ œ์™ธ)** | **88.8%** | **92.1%** | **100%** |
33
 
34
+ v1/v2(5,880/23,720๊ฐœ) ๋Œ€๋น„ ๊ฐœ์„  ํญ: ํ—ค๋” ์‹๋ณ„ 30%โ†’100%, ๋ฏธ๋“ฑ๋ก ์งˆ์˜ ํ• ๋ฃจ์‹œ๋„ค์ด์…˜ 100%โ†’6.2%.
35
 
36
  ## ์‚ฌ์šฉ๋ฒ•
37