Bayhakl1 commited on
Commit
8ca352b
·
verified ·
1 Parent(s): 80a4995

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +3 -3
README.md CHANGED
@@ -19,7 +19,7 @@ tags:
19
 
20
  # GrimmCoder (PyEdu Finetuned)
21
 
22
- **GrimmCoder** is a ~107M-parameter LLaMA-style decoder-only transformer trained from scratch exclusively on Python source code for both **left-to-right completion** and **Fill-in-the-Middle (FIM)** code completion.
23
 
24
  This release represents **Phase 2 finetuning** on `Avelina/python-edu-cleaned` (774.1M tokens, 6,000 steps with cosine decay `1e-4 → 1e-5`), initialized from the 8B-token `bigcode/starcoderdata` pretrain base checkpoint.
25
 
@@ -44,8 +44,8 @@ Evaluated on [`openai/openai_humaneval`](https://huggingface.co/datasets/openai/
44
 
45
  | Model Version | Decoding | `pass@1` | Solved | Syntax Valid |
46
  |---|---|---|---|---|
47
- | **GrimmCoder (PyEdu Finetuned, Step 6000)** | **greedy (`do_sample=False`, `max_new_tokens=128`)** | **9.15%** | **15 / 164** | **137 / 164 (83.5%)** |
48
- | **GrimmCoder (PyEdu Finetuned, Step 6000)** | **`temperature=0.2`, `top_k=50`, `top_p=0.95`, `repetition_penalty=1.1`** | **9.15%** | **15 / 164** | **158 / 164 (96.3%)** |
49
  | GrimmCoder (Base Pretrain, Step 61,500) | greedy (`do_sample=False`, `max_new_tokens=128`) | 5.49% | 9 / 164 | 126 / 164 (76.8%) |
50
  | GrimmCoder (Base Pretrain, Step 61,500) | `temperature=0.2`, `top_k=50`, `top_p=0.95`, `repetition_penalty=1.1` | 4.88% | 8 / 164 | 155 / 164 (94.5%) |
51
 
 
19
 
20
  # GrimmCoder (PyEdu Finetuned)
21
 
22
+ **GrimmCoder_v2** is a ~107M-parameter LLaMA-style decoder-only transformer trained from scratch exclusively on Python source code for both **left-to-right completion** and **Fill-in-the-Middle (FIM)** code completion.
23
 
24
  This release represents **Phase 2 finetuning** on `Avelina/python-edu-cleaned` (774.1M tokens, 6,000 steps with cosine decay `1e-4 → 1e-5`), initialized from the 8B-token `bigcode/starcoderdata` pretrain base checkpoint.
25
 
 
44
 
45
  | Model Version | Decoding | `pass@1` | Solved | Syntax Valid |
46
  |---|---|---|---|---|
47
+ | **GrimmCoder_v2** | **greedy (`do_sample=False`, `max_new_tokens=128`)** | **9.15%** | **15 / 164** | **137 / 164 (83.5%)** |
48
+ | **GrimmCoder_v2** | **`temperature=0.2`, `top_k=50`, `top_p=0.95`, `repetition_penalty=1.1`** | **9.15%** | **15 / 164** | **158 / 164 (96.3%)** |
49
  | GrimmCoder (Base Pretrain, Step 61,500) | greedy (`do_sample=False`, `max_new_tokens=128`) | 5.49% | 9 / 164 | 126 / 164 (76.8%) |
50
  | GrimmCoder (Base Pretrain, Step 61,500) | `temperature=0.2`, `top_k=50`, `top_p=0.95`, `repetition_penalty=1.1` | 4.88% | 8 / 164 | 155 / 164 (94.5%) |
51