Monibee-Fudgekins commited on
Commit
18b04d8
·
verified ·
1 Parent(s): b3352e6

Unsloth Model Card

Browse files
Files changed (1) hide show
  1. README.md +14 -77
README.md CHANGED
@@ -1,84 +1,21 @@
1
  ---
2
- license: gemma
3
- base_model: google/gemma-4-26B-A4B-it
4
- datasets:
5
- - ise-uiuc/Magicoder-OSS-Instruct-75K
6
- library_name: transformers
7
- pipeline_tag: text-generation
8
  tags:
9
- - code
10
- - coding-assistant
11
- - qlora
12
- - unsloth
13
- model-index:
14
- - name: gemma-coder
15
- results:
16
- - task:
17
- type: text-generation
18
- name: Code generation
19
- dataset:
20
- type: code-eval
21
- name: remote-agent-dev-platform coding_eval
22
- metrics:
23
- - type: pass@1
24
- value: 1.0000
25
- name: pass@1 (Python/JS/React/Go/Java/Swift)
26
  ---
27
 
28
- # gemma-coder
29
 
30
- Coding-focused fine-tune of [`google/gemma-4-26B-A4B-it`](https://huggingface.co/google/gemma-4-26B-A4B-it)
31
- (**Gemma 4 26B A4B**, an MoE with ~4B active params), produced automatically by the
32
- weekly retrain pipeline in [remote-agent-dev-platform](https://github.com/Monibee-Fudgekins/remote-agent-dev-platform).
33
- **Last updated: 2026-07-22 16:45 UTC** · run mode: `full` · promoted: **True**.
34
 
35
- ## Model description
36
- QLoRA fine-tune of google/gemma-4-26B-A4B-it specialized for coding assistance. It is the
37
- default agent model for the remote-agent-dev-platform (served via vLLM on Modal).
38
 
39
- ## Intended uses & limitations
40
- - **Intended:** code generation and assistance in Python, JavaScript/React, Go, Java,
41
- and Swift, inside a sandboxed agent that runs/tests the output.
42
- - **Not intended:** safety-critical use, or running generated code unreviewed.
43
- - **Limitations:** a small, free-tier-trained model — it can produce incorrect or
44
- insecure code. Always review and test. Quality tracks the training data, which is
45
- still being built out.
46
-
47
- ## Training data
48
- - Dataset: [`ise-uiuc/Magicoder-OSS-Instruct-75K`](https://huggingface.co/datasets/ise-uiuc/Magicoder-OSS-Instruct-75K)
49
-
50
- ## Training procedure
51
- - Method: QLoRA (Unsloth), 4-bit base, LoRA r=8 / alpha=16 on
52
- attention only, lr 2e-05,
53
- max seq len 768, optimizer adamw_8bit.
54
- - Progress: **cycle 1 — 2047 / 4000 steps** (trained
55
- in weekly ~8h chunks on Kaggle's free 2×T4, resuming each week; training is
56
- continuous — a finished cycle rolls into the next).
57
-
58
- ## Evaluation
59
- Sandboxed multi-language **pass@1** harness (`finetune/evaluate.py`): the model
60
- completes functions that are then compiled/run against unit tests. Languages whose
61
- toolchain is unavailable are skipped.
62
-
63
- **Overall pass@1: 100.00%** over 11 executed problems
64
- (0 skipped). Promotion threshold: 80%.
65
-
66
- | language | passed / run | pass@1 |
67
- |---|---|---|
68
- | javascript | 3/3 | 100.00% |
69
- | python | 8/8 | 100.00% |
70
-
71
- ## How to use
72
- ```python
73
- from transformers import AutoModelForCausalLM, AutoTokenizer
74
- tok = AutoTokenizer.from_pretrained("Monibee-Fudgekins/gemma-coder")
75
- model = AutoModelForCausalLM.from_pretrained("Monibee-Fudgekins/gemma-coder", device_map="auto")
76
- msgs = [{"role": "user", "content": "Write a Python function that reverses a string."}]
77
- ids = tok.apply_chat_template(msgs, add_generation_prompt=True, return_tensors="pt").to(model.device)
78
- print(tok.decode(model.generate(ids, max_new_tokens=256)[0]))
79
- ```
80
-
81
-
82
- ## Provenance
83
- Generated by `finetune/kaggle/run.py` in [https://github.com/Monibee-Fudgekins/remote-agent-dev-platform](https://github.com/Monibee-Fudgekins/remote-agent-dev-platform); see that repo for
84
- the full training + eval pipeline.
 
1
  ---
2
+ base_model: Monibee-Fudgekins/gemma-coder
 
 
 
 
 
3
  tags:
4
+ - text-generation-inference
5
+ - transformers
6
+ - unsloth
7
+ - gemma4
8
+ license: apache-2.0
9
+ language:
10
+ - en
 
 
 
 
 
 
 
 
 
 
11
  ---
12
 
13
+ # Uploaded finetuned model
14
 
15
+ - **Developed by:** Monibee-Fudgekins
16
+ - **License:** apache-2.0
17
+ - **Finetuned from model :** Monibee-Fudgekins/gemma-coder
 
18
 
19
+ This gemma4 model was trained 2x faster with [Unsloth](https://github.com/unslothai/unsloth) and Huggingface's TRL library.
 
 
20
 
21
+ [<img src="https://raw.githubusercontent.com/unslothai/unsloth/main/images/unsloth%20made%20with%20love.png" width="200"/>](https://github.com/unslothai/unsloth)