CoBeDigger commited on
Commit
5aa8aa7
·
verified ·
1 Parent(s): 91c2b90

Add model card and Ollama files

Browse files
Files changed (4) hide show
  1. Modelfile +60 -0
  2. README.md +140 -0
  3. system +1 -0
  4. template +51 -0
Modelfile ADDED
@@ -0,0 +1,60 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # Krama v0.1 for Ollama. Put this file next to krama-coder-1.5b-q4_k_m.gguf, then run:
2
+ # ollama create krama -f Modelfile
3
+
4
+ FROM ./krama-coder-1.5b-q4_k_m.gguf
5
+
6
+ TEMPLATE """{{- if .Suffix }}<|fim_prefix|>{{ .Prompt }}<|fim_suffix|>{{ .Suffix }}<|fim_middle|>
7
+ {{- else if .Messages }}
8
+ {{- if or .System .Tools }}<|im_start|>system
9
+ {{- if .System }}
10
+ {{ .System }}
11
+ {{- end }}
12
+ {{- if .Tools }}
13
+
14
+ # Tools
15
+
16
+ You may call one or more functions to assist with the user query.
17
+
18
+ You are provided with function signatures within <tools></tools>:
19
+ <tools>
20
+ {{- range .Tools }}
21
+ {"type": "function", "function": {{ .Function }}}
22
+ {{- end }}
23
+ </tools>
24
+
25
+ For each function call, return a json object with function name and arguments within <tool_call></tool_call> with NO other text. Do not include any backticks or ```json.
26
+ <tool_call>
27
+ {"name": <function-name>, "arguments": <args-json-object>}
28
+ </tool_call>
29
+ {{- end }}<|im_end|>
30
+ {{ end }}
31
+ {{- range $i, $_ := .Messages }}
32
+ {{- $last := eq (len (slice $.Messages $i)) 1 -}}
33
+ {{- if eq .Role "user" }}<|im_start|>user
34
+ {{ .Content }}<|im_end|>
35
+ {{ else if eq .Role "assistant" }}<|im_start|>assistant
36
+ {{ if .Content }}{{ .Content }}
37
+ {{- else if .ToolCalls }}<tool_call>
38
+ {{ range .ToolCalls }}{"name": "{{ .Function.Name }}", "arguments": {{ .Function.Arguments }}}
39
+ {{ end }}</tool_call>
40
+ {{- end }}{{ if not $last }}<|im_end|>
41
+ {{ end }}
42
+ {{- else if eq .Role "tool" }}<|im_start|>user
43
+ <tool_response>
44
+ {{ .Content }}
45
+ </tool_response><|im_end|>
46
+ {{ end }}
47
+ {{- if and (ne .Role "assistant") $last }}<|im_start|>assistant
48
+ {{ end }}
49
+ {{- end }}
50
+ {{- else }}
51
+ {{- if .System }}<|im_start|>system
52
+ {{ .System }}<|im_end|>
53
+ {{ end }}{{ if .Prompt }}<|im_start|>user
54
+ {{ .Prompt }}<|im_end|>
55
+ {{ end }}<|im_start|>assistant
56
+ {{ end }}{{ .Response }}{{ if .Response }}<|im_end|>{{ end }}"""
57
+
58
+ SYSTEM You are Krama, a Python and FastAPI coding assistant for beginners, fine-tuned by CoBeDigger from Qwen2.5-Coder-1.5B-Instruct.
59
+
60
+ LICENSE Apache-2.0
README.md ADDED
@@ -0,0 +1,140 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: apache-2.0
3
+ base_model: Qwen/Qwen2.5-Coder-1.5B-Instruct
4
+ base_model_relation: finetune
5
+ language:
6
+ - en
7
+ pipeline_tag: text-generation
8
+ tags:
9
+ - code
10
+ - python
11
+ - fastapi
12
+ - education
13
+ - gguf
14
+ - ollama
15
+ - lora
16
+ ---
17
+
18
+ # Krama Coder 1.5B (v0.1 preview)
19
+
20
+ Krama is a small, offline coding tutor for **Python and FastAPI beginners**.
21
+ For a coding task it answers in three short parts: **Plan** (numbered steps), **Code**, and **Why** (plain-language reasons).
22
+ For a quick question it gives a short, direct answer.
23
+ It runs on an ordinary laptop CPU with no internet.
24
+
25
+ "Krama" is Sanskrit for order or sequence.
26
+
27
+ > **Preview.** This is an early release for trying out and giving feedback.
28
+ > It is not a general-purpose coding model and it is not production-ready.
29
+ > Always run the code it gives you.
30
+
31
+ Krama is a fine-tune of [Qwen2.5-Coder-1.5B-Instruct](https://huggingface.co/Qwen/Qwen2.5-Coder-1.5B-Instruct) by the Qwen team (Apache 2.0). All of its coding ability comes from that model. Krama only changes how it answers.
32
+
33
+ ## Run it
34
+
35
+ With [Ollama](https://ollama.com):
36
+
37
+ ```sh
38
+ ollama run hf.co/CoBeDigger/krama-coder-1.5b
39
+ ```
40
+
41
+ The file is 986 MB (4-bit Q4_K_M). On a laptop CPU (Intel Core Ultra 7 155H) the base model runs at about 33-36 tokens per second, and Krama is the same size.
42
+
43
+ You can also use it from code through Ollama's local API (`http://localhost:11434`), or in VS Code through an extension that supports Ollama, such as Continue.
44
+
45
+ To build it yourself from the downloaded file, use the `Modelfile` in this repo:
46
+
47
+ ```sh
48
+ ollama create krama -f Modelfile
49
+ ```
50
+
51
+ ## What an answer looks like
52
+
53
+ These are real answers from the release file, at temperature 0.
54
+
55
+ **"Write a Python function safe_divide(a, b) that returns a divided by b. If b is zero, return None instead of crashing."**
56
+
57
+ ````text
58
+ Plan:
59
+ 1. Define the function `safe_divide`.
60
+ 2. Check if `b` is zero.
61
+ 3. Return `None` if `b` is zero, otherwise return the division result.
62
+
63
+ Code:
64
+ ```python
65
+ def safe_divide(a, b):
66
+ if b == 0:
67
+ return None
68
+ return a / b
69
+ ```
70
+
71
+ Why:
72
+ - The function checks if `b` is zero before performing the division to avoid division by zero errors.
73
+ - If `b` is zero, the function returns `None`, which can be handled gracefully in the calling code.
74
+ ````
75
+
76
+ **"What is the difference between == and is in Python?"**
77
+
78
+ ````text
79
+ `==` checks if two values are equal, while `is` checks if both variables point to the same object. For example:
80
+ ```python
81
+ x = [1, 2]
82
+ y = [1, 2]
83
+ print(x == y) # True because they have the same elements
84
+ print(x is y) # False because they are two different lists
85
+ ```
86
+ ````
87
+
88
+ ## How it was tested
89
+
90
+ All four systems below are the same 986 MB Q4_K_M file size, run in Ollama at temperature 0.
91
+ "Base + instruction" is the base model given a written instruction to answer like Krama.
92
+
93
+ **Correctness.** 30 tasks of our own (12 Python, 12 FastAPI, 6 quick questions), where the code is run against tests. The 164-task [HumanEval](https://github.com/openai/human-eval) benchmark is the standard check.
94
+
95
+ | System | Python (12) | FastAPI (12) | Quick answers, average length | HumanEval (164) |
96
+ |---|---|---|---|---|
97
+ | Base model | 9 | 6 | 273 tokens | 120 (73.2%) |
98
+ | Base + instruction | 9 | 5 | 218 tokens | 98 (59.8%) |
99
+ | **Krama v0.1** | **10** | **6** | **121 tokens** | **112 (68.3%)** |
100
+
101
+ **Teaching quality.** Each answer was scored blind by a different model (Llama 3.1 8B), which never saw which system wrote it. Scoring rules were written before training. Averages are out of 2 per rule.
102
+
103
+ | System | Plan | Why | Clarity | Length | Coding total (of 8) | Quick answers (of 4) |
104
+ |---|---|---|---|---|---|---|
105
+ | Base model | 1.08 | 0.67 | 1.04 | 1.58 | 4.38 | 1.33 |
106
+ | Base + instruction | 1.83 | 1.17 | 1.17 | 1.79 | 5.96 | 1.33 |
107
+ | **Krama v0.1** | **1.92** | **1.17** | **1.21** | **1.96** | **6.25** | **2.00** |
108
+
109
+ What this means, honestly:
110
+
111
+ - Krama matches the base model's correctness on our tasks, and adds a plan and an explanation.
112
+ - On HumanEval it is **8 tasks below** the plain base model. Asking the base model to plan and explain costs it far more (22 tasks), so the fine-tune is the better way to get this answer style, but it is not free.
113
+ - Against the instructed base, Krama wins 7, ties 15 and loses 2 of the 24 coding tasks on teaching quality, and wins 3 and ties 3 of the 6 quick questions. Its short answers are about half as long.
114
+ - Our 30 tasks share topics with the training data. Exact copies were removed, but those scores may flatter Krama a little. HumanEval is the fairer measure.
115
+ - The blind grader is a small model and was lenient. It sometimes gave the base model credit for a plan it did not have. That error favours the base, not Krama.
116
+
117
+ ## Known weaknesses
118
+
119
+ - FastAPI: 6 of 12 tasks still fail. Krama sometimes adds a Pydantic model where none is needed, and once it read a JSON body as a plain string parameter.
120
+ - Explanations are short and sometimes generic, and they are not always fully accurate. Treat the "Why" as a starting point.
121
+ - English only. Python and FastAPI only.
122
+ - 1.5B parameters: it makes logic mistakes that larger models would not.
123
+
124
+ ## How it was made
125
+
126
+ - **Data:** 1,578 examples: 776 Python, 471 FastAPI, 316 short answers, and 15 about who made it. Most answers were written by [Qwen2.5-Coder-7B-Instruct](https://huggingface.co/Qwen/Qwen2.5-Coder-7B-Instruct) (Apache 2.0) and kept **only if their code passed their own tests**. 92 examples were written by hand. Some questions were inspired by short pieces of the author's own code. Any example whose code passed one of our test tasks, or which asked one of our test questions, was removed. The dataset is not released in this preview.
127
+ - **Training:** LoRA on all linear layers (rank 16, alpha 32, dropout 0.05), learning rate 2e-4 with cosine decay, 2 epochs, max length 1,024, effective batch 16, fp16, on one free Kaggle T4 GPU (11 minutes). Held-out loss went from 0.522 to 0.474. The adapter was merged and converted to Q4_K_M GGUF with llama.cpp.
128
+ - **Instruction it was trained with:** "You are Krama, a Python and FastAPI coding assistant for beginners, fine-tuned by CoBeDigger from Qwen2.5-Coder-1.5B-Instruct." The `system` file in this repo sets it for you.
129
+
130
+ ## What comes next
131
+
132
+ - A `krama` command-line tool that runs Krama's code against tests on your own machine before showing it.
133
+ - Version 0.2, trained on the mistakes real beginners find.
134
+ - Later, a larger version.
135
+
136
+ Feedback is very welcome in the Community tab, especially questions where Krama got it wrong.
137
+
138
+ ## License
139
+
140
+ Apache 2.0, the same as the base model. Built on Qwen2.5-Coder-1.5B-Instruct by the Qwen team.
system ADDED
@@ -0,0 +1 @@
 
 
1
+ You are Krama, a Python and FastAPI coding assistant for beginners, fine-tuned by CoBeDigger from Qwen2.5-Coder-1.5B-Instruct.
template ADDED
@@ -0,0 +1,51 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {{- if .Suffix }}<|fim_prefix|>{{ .Prompt }}<|fim_suffix|>{{ .Suffix }}<|fim_middle|>
2
+ {{- else if .Messages }}
3
+ {{- if or .System .Tools }}<|im_start|>system
4
+ {{- if .System }}
5
+ {{ .System }}
6
+ {{- end }}
7
+ {{- if .Tools }}
8
+
9
+ # Tools
10
+
11
+ You may call one or more functions to assist with the user query.
12
+
13
+ You are provided with function signatures within <tools></tools>:
14
+ <tools>
15
+ {{- range .Tools }}
16
+ {"type": "function", "function": {{ .Function }}}
17
+ {{- end }}
18
+ </tools>
19
+
20
+ For each function call, return a json object with function name and arguments within <tool_call></tool_call> with NO other text. Do not include any backticks or ```json.
21
+ <tool_call>
22
+ {"name": <function-name>, "arguments": <args-json-object>}
23
+ </tool_call>
24
+ {{- end }}<|im_end|>
25
+ {{ end }}
26
+ {{- range $i, $_ := .Messages }}
27
+ {{- $last := eq (len (slice $.Messages $i)) 1 -}}
28
+ {{- if eq .Role "user" }}<|im_start|>user
29
+ {{ .Content }}<|im_end|>
30
+ {{ else if eq .Role "assistant" }}<|im_start|>assistant
31
+ {{ if .Content }}{{ .Content }}
32
+ {{- else if .ToolCalls }}<tool_call>
33
+ {{ range .ToolCalls }}{"name": "{{ .Function.Name }}", "arguments": {{ .Function.Arguments }}}
34
+ {{ end }}</tool_call>
35
+ {{- end }}{{ if not $last }}<|im_end|>
36
+ {{ end }}
37
+ {{- else if eq .Role "tool" }}<|im_start|>user
38
+ <tool_response>
39
+ {{ .Content }}
40
+ </tool_response><|im_end|>
41
+ {{ end }}
42
+ {{- if and (ne .Role "assistant") $last }}<|im_start|>assistant
43
+ {{ end }}
44
+ {{- end }}
45
+ {{- else }}
46
+ {{- if .System }}<|im_start|>system
47
+ {{ .System }}<|im_end|>
48
+ {{ end }}{{ if .Prompt }}<|im_start|>user
49
+ {{ .Prompt }}<|im_end|>
50
+ {{ end }}<|im_start|>assistant
51
+ {{ end }}{{ .Response }}{{ if .Response }}<|im_end|>{{ end }}