louhless commited on
Commit
71c7ed3
·
verified ·
1 Parent(s): 540257a

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +31 -19
README.md CHANGED
@@ -14,7 +14,7 @@ tags:
14
  - bash
15
  - experimental
16
  license: apache-2.0
17
- base_model: distilgpt2
18
  pipeline_tag: text-generation
19
  library_name: transformers
20
  model_creator: louhless
@@ -22,8 +22,9 @@ model_creator: louhless
22
 
23
  # Ycoder-small
24
 
25
- `Ycoder-small` is a tiny experimental code-focused language model created by **louhless** and fine-tuned for short programming prompts and lightweight problem solving.
26
- **join Discord↠https://discord.gg/Dq4MWuJm**
 
27
 
28
  Its main focus is:
29
 
@@ -33,15 +34,17 @@ Its main focus is:
33
  - SQL
34
  - Bash
35
 
36
- It also retains limited support for simple math-style prompts.
37
 
38
  ## Model Details
39
 
40
  - **Model name:** `Ycoder-small`
41
  - **Creator:** `louhless`
42
- - **Base model:** `distilgpt2`
43
  - **Model type:** fine-tuned causal language model
44
- - **Language:** English
 
 
45
  - **Current export:** GGUF available
46
  - **Status:** experimental
47
 
@@ -49,11 +52,13 @@ It also retains limited support for simple math-style prompts.
49
 
50
  `Ycoder-small` is **not trained from scratch**.
51
 
52
- It is a **fine-tuned version of `distilgpt2`**, adapted on a very small custom instruction dataset focused mainly on short code tasks. It is intended as a compact coding experiment and local demo model, not as a production-grade coding assistant.
 
 
53
 
54
  ## Training Goal
55
 
56
- The goal of this project is to explore how small a fine-tuned model can be while still producing somewhat useful code outputs in a narrow setup.
57
 
58
  The project is optimized for:
59
 
@@ -61,16 +66,18 @@ The project is optimized for:
61
  - fast fine-tuning
62
  - tiny datasets
63
  - compact GGUF export workflows
 
 
64
 
65
  ## Training Data
66
 
67
- The model was fine-tuned on a small custom English instruction dataset with three categories:
68
 
69
  - `router_text`
70
  - `code`
71
  - `math`
72
 
73
- The `code` portion was emphasized and includes examples such as:
74
 
75
  - Python utility functions
76
  - Python syntax fixes
@@ -81,6 +88,13 @@ The `code` portion was emphasized and includes examples such as:
81
  - GLSL vertex shaders
82
  - short code explanations
83
 
 
 
 
 
 
 
 
84
  Because the dataset is very small, the model is strongly biased toward short prompt patterns and limited example styles.
85
 
86
  ## Training Setup
@@ -91,17 +105,15 @@ Training approach:
91
  - causal language modeling objective
92
  - small instruction-style Q&A dataset
93
  - local Mac training workflow
 
94
 
95
- Observed local training characteristics in this project:
96
 
97
- - **1 epoch:** about **4 minutes 14 seconds**
98
- - **2 epochs:** about **8 to 9 minutes**
99
- - **peak memory footprint:** about **3.6 GB**
100
 
101
- ## Example Prompts
102
 
103
- ### Python
104
 
105
- **Prompt**
106
- ```text
107
- write python code to reverse a string
 
14
  - bash
15
  - experimental
16
  license: apache-2.0
17
+ base_model: HuggingFaceTB/SmolLM2-135M-Instruct
18
  pipeline_tag: text-generation
19
  library_name: transformers
20
  model_creator: louhless
 
22
 
23
  # Ycoder-small
24
 
25
+ `Ycoder-small` is a tiny experimental code-focused language model created by **louhless** and fine-tuned for short programming prompts, lightweight problem solving, and simple chat behavior.
26
+
27
+ Join Discord: https://discord.gg/Dq4MWuJm
28
 
29
  Its main focus is:
30
 
 
34
  - SQL
35
  - Bash
36
 
37
+ It also has limited support for simple math prompts and short normal conversation.
38
 
39
  ## Model Details
40
 
41
  - **Model name:** `Ycoder-small`
42
  - **Creator:** `louhless`
43
+ - **Base model:** `HuggingFaceTB/SmolLM2-135M-Instruct`
44
  - **Model type:** fine-tuned causal language model
45
+ - **Architecture:** Llama-style instruct model
46
+ - **Language:** English, with small German greeting support
47
+ - **Context length:** 8192
48
  - **Current export:** GGUF available
49
  - **Status:** experimental
50
 
 
52
 
53
  `Ycoder-small` is **not trained from scratch**.
54
 
55
+ It is a fine-tuned version of `HuggingFaceTB/SmolLM2-135M-Instruct`, adapted on a small custom instruction dataset focused mainly on short code tasks, simple math, normal replies, and optional thinking-style output.
56
+
57
+ It is intended as a compact coding experiment and local demo model, not as a production-grade coding assistant.
58
 
59
  ## Training Goal
60
 
61
+ The goal of this project is to explore how small a fine-tuned model can be while still producing somewhat useful short outputs in a narrow setup.
62
 
63
  The project is optimized for:
64
 
 
66
  - fast fine-tuning
67
  - tiny datasets
68
  - compact GGUF export workflows
69
+ - short code/math/chat responses
70
+ - optional thinking-style output
71
 
72
  ## Training Data
73
 
74
+ The model was fine-tuned on a small custom instruction dataset with three categories:
75
 
76
  - `router_text`
77
  - `code`
78
  - `math`
79
 
80
+ The code portion was emphasized and includes examples such as:
81
 
82
  - Python utility functions
83
  - Python syntax fixes
 
88
  - GLSL vertex shaders
89
  - short code explanations
90
 
91
+ The text and math portions include examples such as:
92
+
93
+ - greetings like `hello`, `hi`, and `hallo`
94
+ - short normal assistant replies
95
+ - simple arithmetic
96
+ - simple equations such as `3x = 21`
97
+
98
  Because the dataset is very small, the model is strongly biased toward short prompt patterns and limited example styles.
99
 
100
  ## Training Setup
 
105
  - causal language modeling objective
106
  - small instruction-style Q&A dataset
107
  - local Mac training workflow
108
+ - continued fine-tuning from an instruct base model
109
 
110
+ Observed local training characteristics vary by run and base model. The current stronger version uses `SmolLM2-135M-Instruct`, so it is slower than the earlier `distilgpt2` experiment but has much better chat/context behavior.
111
 
112
+ ## Thinking Toggle
 
 
113
 
114
+ `Ycoder-small` supports a Qwen-style thinking toggle in the local chat script.
115
 
116
+ Thinking off:
117
 
118
+ ```bash
119
+ python3 scripts/chat_hf_model.py --prompt "write python code to reverse a string" --thinking off