Dikshan1234 commited on
Commit
7ab72b3
·
verified ·
1 Parent(s): 09376e6

Update root README — checkpoint-14769 logged

Browse files
Files changed (1) hide show
  1. README.md +14 -110
README.md CHANGED
@@ -1,28 +1,3 @@
1
- ---
2
- language:
3
- - en
4
- license: cc-by-nc-4.0
5
- tags:
6
- - code
7
- - python
8
- - django
9
- - flask
10
- - fastapi
11
- - react
12
- - nextjs
13
- - typescript
14
- - sql
15
- - docker
16
- - qlora
17
- - fine-tuned
18
- - coding-assistant
19
- - web-development
20
- base_model: Qwen/Qwen2.5-Coder-7B-Instruct
21
- pipeline_tag: text-generation
22
- model_type: qwen2
23
- library_name: transformers
24
- ---
25
-
26
  # ScriptSwiftAI v12 (Dikshan)
27
 
28
  > QLoRA fine-tuned full-stack code generation model by **Dikshan** (India).
@@ -47,101 +22,19 @@ https://creativecommons.org/licenses/by-nc/4.0/
47
  | **Method** | QLoRA (4-bit, r=64) |
48
  | **Hardware** | NVIDIA RTX 5070 Ti 16GB |
49
  | **Target Steps** | 40,000 |
50
- | **Dataset** | ~570k samples (17 datasets) |
51
  | **Max Length** | 1536 tokens |
52
  | **Developer** | Dikshan (India) |
53
  | **Training Started** | June 2026 |
54
 
55
  ---
56
 
57
- ## How to Use
58
-
59
- Load the adapter on top of the base model:
60
-
61
- ```python
62
- from transformers import AutoModelForCausalLM, AutoTokenizer, BitsAndBytesConfig
63
- from peft import PeftModel
64
- import torch
65
-
66
- bnb_config = BitsAndBytesConfig(
67
- load_in_4bit=True,
68
- bnb_4bit_use_double_quant=True,
69
- bnb_4bit_quant_type="nf4",
70
- bnb_4bit_compute_dtype=torch.bfloat16
71
- )
72
-
73
- base = AutoModelForCausalLM.from_pretrained(
74
- "Qwen/Qwen2.5-Coder-7B-Instruct",
75
- quantization_config=bnb_config,
76
- device_map="auto"
77
- )
78
- model = PeftModel.from_pretrained(base, "Dikshan1234/ScriptSwiftAIV2/checkpoint-14000")
79
- tokenizer = AutoTokenizer.from_pretrained("Qwen/Qwen2.5-Coder-7B-Instruct")
80
-
81
- prompt = """You are ScriptSwiftAI, a professional full-stack and Python web development assistant made by Dikshan.
82
-
83
- ### Instruction:
84
- Build a Django REST API with JWT authentication.
85
-
86
- ### Response:
87
- """
88
-
89
- inputs = tokenizer(prompt, return_tensors="pt").to("cuda")
90
- output = model.generate(**inputs, max_new_tokens=1536, temperature=0.3, top_p=0.85, top_k=40)
91
- print(tokenizer.decode(output[0], skip_special_tokens=True))
92
- ```
93
-
94
- > GGUF version coming soon for easy local use with Ollama.
95
-
96
- ---
97
-
98
- ## Dataset Sources (17 datasets, ~570k samples)
99
-
100
- | # | Dataset | Focus |
101
- |---|---------|-------|
102
- | 1 | sahil2801/CodeAlpaca-20k | General coding |
103
- | 2 | TokenBender/code_instructions_122k_alpaca_style | Python |
104
- | 3 | flytech/python-codes-25k | Python |
105
- | 4 | AhmedSSoliman/DJANGO | Django |
106
- | 5 | nickrosh/Evol-Instruct-Code-80k-v1 | Web dev |
107
- | 6 | iamtarun/python_code_instructions_18k_alpaca | Python |
108
- | 7 | ise-uiuc/Magicoder-OSS-Instruct-75K | General coding |
109
- | 8 | glaiveai/glaive-code-assistant-v3 | Web dev |
110
- | 9 | ajibawa-2023/Python-Code-23k-ShareGPT | Python |
111
- | 10 | m-a-p/CodeFeedback-Filtered-Instruction | Web dev |
112
- | 11 | cfahlgren1/react-code-instructions | React |
113
- | 12 | Tesslate/Next.js-Dataset | Next.js |
114
- | 13 | mhhmm/typescript-instruct-20k | TypeScript |
115
- | 14 | grenishrai/typescript-dataset | TypeScript |
116
- | 15 | NumbersStation/NSText2SQL | SQL |
117
- | 16 | nvidia/OpenCodeInstruct | General coding |
118
- | 17 | Identity QA | ScriptSwiftAI identity |
119
-
120
- ---
121
-
122
- ## Training Config
123
-
124
- | Setting | Value |
125
- |---|---|
126
- | **Base Model** | Qwen2.5-Coder-7B-Instruct |
127
- | **Method** | QLoRA (4-bit NF4) |
128
- | **LoRA Rank** | r=64, alpha=128 |
129
- | **Target Steps** | 40,000 |
130
- | **Batch Size** | 1 (effective 16 with grad accum) |
131
- | **Learning Rate** | 2e-4 cosine, 1000 warmup steps |
132
- | **Optimizer** | paged_adamw_8bit |
133
- | **Max Length** | 1536 tokens |
134
- | **Hardware** | RTX 5070 Ti 16GB |
135
-
136
- ---
137
-
138
- ---
139
-
140
  ## Changelog / Training Timeline
141
 
142
  *Auto-updated by train_v12.py on every checkpoint save.*
143
  *Append-only — entries never overwritten. HuggingFace commit history provides timestamps.*
144
 
 
145
  ---
146
 
147
  ### 🔵 Checkpoint — Step 13
@@ -459,4 +352,15 @@ print(tokenizer.decode(output[0], skip_special_tokens=True))
459
  **Loss:** 0.2634
460
  **Epoch:** 0.409
461
  **Zone:** 📈 Good
462
- **Status:** Training ongoing — auto-uploaded via train_v12.py
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
  # ScriptSwiftAI v12 (Dikshan)
2
 
3
  > QLoRA fine-tuned full-stack code generation model by **Dikshan** (India).
 
22
  | **Method** | QLoRA (4-bit, r=64) |
23
  | **Hardware** | NVIDIA RTX 5070 Ti 16GB |
24
  | **Target Steps** | 40,000 |
25
+ | **Dataset** | ~450-500k samples (17 datasets) |
26
  | **Max Length** | 1536 tokens |
27
  | **Developer** | Dikshan (India) |
28
  | **Training Started** | June 2026 |
29
 
30
  ---
31
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
32
  ## Changelog / Training Timeline
33
 
34
  *Auto-updated by train_v12.py on every checkpoint save.*
35
  *Append-only — entries never overwritten. HuggingFace commit history provides timestamps.*
36
 
37
+
38
  ---
39
 
40
  ### 🔵 Checkpoint — Step 13
 
352
  **Loss:** 0.2634
353
  **Epoch:** 0.409
354
  **Zone:** 📈 Good
355
+ **Status:** Training ongoing — auto-uploaded via train_v12.py
356
+
357
+ ---
358
+
359
+ ### 🔵 Checkpoint — Step 14,769
360
+ **Date:** June 27, 2026 at 10:48 IST
361
+ **Checkpoint:** checkpoint-14769
362
+ **Step:** 14,769 / 40,000 (36.9% complete)
363
+ **Loss:** 0.1741
364
+ **Epoch:** 0.431
365
+ **Zone:** 🎯 Sweet Spot
366
+ **Status:** Training ongoing — auto-uploaded via train_v12.py