Instructions to use ahpah/YokoCoder-4B with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- llama.cpp
How to use ahpah/YokoCoder-4B with llama.cpp:
Install (macOS, Linux)
curl -LsSf https://llama.app/install.sh | sh # Start a local OpenAI-compatible server with a web UI: llama serve -hf ahpah/YokoCoder-4B:Q4_K_M # Run inference directly in the terminal: llama cli -hf ahpah/YokoCoder-4B:Q4_K_M
Install from WinGet (Windows)
winget install llama.cpp # Start a local OpenAI-compatible server with a web UI: llama serve -hf ahpah/YokoCoder-4B:Q4_K_M # Run inference directly in the terminal: llama cli -hf ahpah/YokoCoder-4B:Q4_K_M
Build from source code
git clone https://github.com/ggml-org/llama.cpp.git cd llama.cpp cmake -B build cmake --build build -j --target llama-app # Start a local OpenAI-compatible server with a web UI: ./build/bin/llama serve -hf ahpah/YokoCoder-4B:Q4_K_M # Run inference directly in the terminal: ./build/bin/llama cli -hf ahpah/YokoCoder-4B:Q4_K_M
Use Docker
docker model run hf.co/ahpah/YokoCoder-4B:Q4_K_M
- Llama new
- LM Studio
- Jan
- vLLM
How to use ahpah/YokoCoder-4B with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "ahpah/YokoCoder-4B" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "ahpah/YokoCoder-4B", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker
docker model run hf.co/ahpah/YokoCoder-4B:Q4_K_M
- Ollama
How to use ahpah/YokoCoder-4B with Ollama:
ollama run hf.co/ahpah/YokoCoder-4B:Q4_K_M
- Unsloth Desktop
- Pi
How to use ahpah/YokoCoder-4B with Pi:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf ahpah/YokoCoder-4B:Q4_K_M
Configure the model in Pi
# Install Pi: npm install -g @earendil-works/pi-coding-agent # Add to ~/.pi/agent/models.json: { "providers": { "llama-cpp": { "baseUrl": "http://localhost:9931/v1", "api": "openai-completions", "apiKey": "none", "models": [ { "id": "ahpah/YokoCoder-4B:Q4_K_M" } ] } } }Run Pi
# Start Pi in your project directory: pi
- Docker Model Runner
How to use ahpah/YokoCoder-4B with Docker Model Runner:
docker model run hf.co/ahpah/YokoCoder-4B:Q4_K_M
- Lemonade
How to use ahpah/YokoCoder-4B with Lemonade:
Pull the model
# Download Lemonade from https://lemonade-server.ai/ lemonade pull ahpah/YokoCoder-4B:Q4_K_M
Run and chat with the model
lemonade run user.YokoCoder-4B-Q4_K_M
List all available models
lemonade list
- Hermes Agent
How to use ahpah/YokoCoder-4B with Hermes Agent:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf ahpah/YokoCoder-4B:Q4_K_M
Configure Hermes
# Install Hermes: curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash hermes setup # Point Hermes at the local server: hermes config set model.provider custom hermes config set model.base_url http://127.0.0.1:9931/v1 hermes config set model.default ahpah/YokoCoder-4B:Q4_K_M
Run Hermes
hermes
- Atomic Chat
- OpenClaw
How to use ahpah/YokoCoder-4B with OpenClaw:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf ahpah/YokoCoder-4B:Q4_K_M
Configure OpenClaw
# Install OpenClaw: npm install -g openclaw@latest # Register the local server and set it as the default model: openclaw onboard --non-interactive --mode local \ --auth-choice custom-api-key \ --custom-base-url http://127.0.0.1:9931/v1 \ --custom-model-id "ahpah/YokoCoder-4B:Q4_K_M" \ --custom-provider-id llama-cpp \ --custom-compatibility openai \ --custom-text-input \ --accept-risk \ --skip-health
Run OpenClaw
openclaw agent --local --agent main --message "Hello from Hugging Face"
Update Kaggle notebook with Unsloth 16-bit pipeline
Browse files
notebooks/yokocoder_4b_kaggle_finetune_gguf_hf.ipynb
CHANGED
|
@@ -4,22 +4,7 @@
|
|
| 4 |
"cell_type": "markdown",
|
| 5 |
"metadata": {},
|
| 6 |
"source": [
|
| 7 |
-
"# YokoCoder:4B (Qwen3.5-4B) \u2014
|
| 8 |
-
"\n",
|
| 9 |
-
"### Quick Setup on Kaggle:\n",
|
| 10 |
-
"1. **Accelerator**: Select **GPU T4 x2** or **GPU P100** in Kaggle Notebook Settings.\n",
|
| 11 |
-
"2. **No Manual Dataset Upload Needed!**: This notebook automatically downloads datasets/yokocoder_4b_reasoning_gamedev_bugfix.jsonl and your existing adapter directly from \u0007hpah/YokoCoder-4B on Hugging Face.\n",
|
| 12 |
-
"3. **Hugging Face Secret**: Ensure HF_TOKEN is enabled in **Add-ons -> Secrets** so the quantized YokoCoder-4B-5epochs-Q4_K_M.gguf pushes back to \u0007hpah/YokoCoder-4B automatically.\n"
|
| 13 |
-
]
|
| 14 |
-
},
|
| 15 |
-
{
|
| 16 |
-
"cell_type": "code",
|
| 17 |
-
"execution_count": null,
|
| 18 |
-
"metadata": {},
|
| 19 |
-
"outputs": [],
|
| 20 |
-
"source": [
|
| 21 |
-
"# Step 1: Install latest Transformers (with Qwen3.5 support), PEFT, BitsAndBytes, and HuggingFace Hub\n",
|
| 22 |
-
"!pip install -q -U transformers peft bitsandbytes accelerate datasets huggingface_hub gguf mistral_common protobuf sentencepiece\n"
|
| 23 |
]
|
| 24 |
},
|
| 25 |
{
|
|
@@ -30,75 +15,56 @@
|
|
| 30 |
"source": [
|
| 31 |
"#!/usr/bin/env python3\n",
|
| 32 |
"\"\"\"\n",
|
| 33 |
-
"YokoCoder:4B \u2014
|
| 34 |
-
"GGUF (Q4_K_M) Quantization, and Hugging Face
|
| 35 |
-
"\n",
|
| 36 |
-
"Features:\n",
|
| 37 |
-
"- Automatically pulls `datasets/yokocoder_4b_reasoning_gamedev_bugfix.jsonl` and\n",
|
| 38 |
-
" `adapters/dataset4-5epochs/final_adapter` directly from `ahpah/YokoCoder-4B`\n",
|
| 39 |
-
" on Hugging Face if not attached as a local Kaggle dataset!\n",
|
| 40 |
-
"- Trains with Assistant-Only Loss Masking + Sliding Windows so 100% of long\n",
|
| 41 |
-
" 16K HTML/Three.js game examples are learned without truncation.\n",
|
| 42 |
-
"- Merges into 16-bit, converts and quantizes to `Q4_K_M` via `llama.cpp`, and\n",
|
| 43 |
-
" pushes the updated GGUF, adapter, and Ollama `Modelfile` back to `ahpah/YokoCoder-4B`.\n",
|
| 44 |
"\"\"\"\n",
|
| 45 |
"\n",
|
| 46 |
-
"from __future__ import annotations\n",
|
| 47 |
-
"\n",
|
| 48 |
"import gc\n",
|
|
|
|
| 49 |
"import json\n",
|
|
|
|
| 50 |
"import os\n",
|
| 51 |
"import subprocess\n",
|
| 52 |
"import sys\n",
|
| 53 |
"import zipfile\n",
|
|
|
|
| 54 |
"from pathlib import Path\n",
|
| 55 |
"\n",
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 56 |
"import torch\n",
|
| 57 |
"from datasets import Dataset\n",
|
| 58 |
-
"from huggingface_hub import HfApi, hf_hub_download,
|
| 59 |
-
"from peft import
|
| 60 |
-
"from transformers import
|
| 61 |
-
" AutoModelForCausalLM,\n",
|
| 62 |
-
" AutoTokenizer,\n",
|
| 63 |
-
" BitsAndBytesConfig,\n",
|
| 64 |
-
" Trainer,\n",
|
| 65 |
-
" TrainingArguments,\n",
|
| 66 |
-
")\n",
|
| 67 |
-
"\n",
|
| 68 |
-
"# ============================================================================\n",
|
| 69 |
-
"# 1. CONFIGURATION\n",
|
| 70 |
-
"# ============================================================================\n",
|
| 71 |
-
"BASE_MODEL_ID = \"wangzhang/Qwen3.5-4B-abliterated\"\n",
|
| 72 |
-
"HF_TARGET_REPO = \"ahpah/YokoCoder-4B\"\n",
|
| 73 |
-
"QUANT_TYPE = \"Q4_K_M\"\n",
|
| 74 |
-
"OUTPUT_GGUF_NAME = \"YokoCoder-4B-5epochs-Q4_K_M.gguf\"\n",
|
| 75 |
"\n",
|
| 76 |
-
"
|
| 77 |
-
"RESUME_FROM_EXISTING_ADAPTER = True\n",
|
| 78 |
"\n",
|
| 79 |
-
"
|
| 80 |
-
"
|
| 81 |
-
"MAX_SEQ_LENGTH = 8192\n",
|
| 82 |
-
"WINDOW_STRIDE = 6144\n",
|
| 83 |
-
"LORA_R = 16\n",
|
| 84 |
-
"LORA_ALPHA = 32\n",
|
| 85 |
"\n",
|
| 86 |
-
"
|
| 87 |
-
"
|
| 88 |
-
"
|
| 89 |
-
"
|
| 90 |
-
"F16_GGUF_PATH = WORK_DIR / \"yokocoder-4b-f16.gguf\"\n",
|
| 91 |
-
"FINAL_GGUF_PATH = Path(\"/kaggle/working\") / OUTPUT_GGUF_NAME\n",
|
| 92 |
"\n",
|
|
|
|
|
|
|
|
|
|
| 93 |
"\n",
|
| 94 |
-
"
|
| 95 |
-
"
|
| 96 |
-
" if not root.exists():\n",
|
| 97 |
-
" continue\n",
|
| 98 |
-
" matches = list(root.rglob(filename_pattern))\n",
|
| 99 |
-
" if matches:\n",
|
| 100 |
-
" return matches[0]\n",
|
| 101 |
-
" return None\n",
|
| 102 |
"\n",
|
| 103 |
"\n",
|
| 104 |
"def get_hf_token() -> str | None:\n",
|
|
@@ -112,290 +78,325 @@
|
|
| 112 |
" return None\n",
|
| 113 |
"\n",
|
| 114 |
"\n",
|
| 115 |
-
"
|
| 116 |
-
"
|
| 117 |
-
"#
|
| 118 |
-
"
|
| 119 |
-
"
|
| 120 |
-
"
|
| 121 |
-
"
|
| 122 |
-
"
|
| 123 |
-
"
|
| 124 |
-
"\n",
|
| 125 |
-
"
|
| 126 |
-
"
|
| 127 |
-
"\n",
|
| 128 |
-
"
|
| 129 |
-
"
|
| 130 |
-
"
|
| 131 |
-
"
|
| 132 |
-
"\n",
|
| 133 |
-
"
|
| 134 |
-
"
|
| 135 |
-
"
|
| 136 |
-
"
|
| 137 |
-
"
|
| 138 |
-
"
|
| 139 |
-
"
|
| 140 |
-
"
|
| 141 |
-
"
|
| 142 |
-
"
|
| 143 |
-
"\n",
|
| 144 |
-
"
|
| 145 |
-
"\n",
|
| 146 |
-
"
|
| 147 |
-
"
|
| 148 |
-
"
|
| 149 |
-
"
|
| 150 |
-
"
|
| 151 |
-
"
|
| 152 |
-
"
|
| 153 |
-
"
|
| 154 |
-
"
|
| 155 |
-
"
|
| 156 |
-
"
|
| 157 |
-
"
|
| 158 |
-
"
|
| 159 |
-
"
|
| 160 |
-
"
|
| 161 |
-
"\n",
|
| 162 |
-
"
|
| 163 |
-
"
|
| 164 |
-
"\n",
|
| 165 |
-
"\n",
|
| 166 |
-
"
|
| 167 |
-
" def __init__(self, pad_token_id: int):\n",
|
| 168 |
-
" self.pad_token_id = pad_token_id\n",
|
| 169 |
-
"\n",
|
| 170 |
-
" def __call__(self, features: list[dict]) -> dict[str, torch.Tensor]:\n",
|
| 171 |
-
" max_len = max(len(f[\"input_ids\"]) for f in features)\n",
|
| 172 |
-
" input_ids, attention_mask, labels = [], [], []\n",
|
| 173 |
-
" for f in features:\n",
|
| 174 |
-
" ids = f[\"input_ids\"]\n",
|
| 175 |
-
" lbl = f[\"labels\"]\n",
|
| 176 |
-
" pad_len = max_len - len(ids)\n",
|
| 177 |
-
" input_ids.append(ids + [self.pad_token_id] * pad_len)\n",
|
| 178 |
-
" attention_mask.append([1] * len(ids) + [0] * pad_len)\n",
|
| 179 |
-
" labels.append(lbl + [-100] * pad_len)\n",
|
| 180 |
-
" return {\n",
|
| 181 |
-
" \"input_ids\": torch.tensor(input_ids, dtype=torch.long),\n",
|
| 182 |
-
" \"attention_mask\": torch.tensor(attention_mask, dtype=torch.long),\n",
|
| 183 |
-
" \"labels\": torch.tensor(labels, dtype=torch.long),\n",
|
| 184 |
-
" }\n",
|
| 185 |
-
"\n",
|
| 186 |
-
"\n",
|
| 187 |
-
"# ============================================================================\n",
|
| 188 |
-
"# 3. MAIN PIPELINE\n",
|
| 189 |
-
"# ============================================================================\n",
|
| 190 |
-
"def main():\n",
|
| 191 |
-
" WORK_DIR.mkdir(parents=True, exist_ok=True)\n",
|
| 192 |
-
" hf_token = get_hf_token()\n",
|
| 193 |
-
" if hf_token:\n",
|
| 194 |
-
" login(token=hf_token)\n",
|
| 195 |
-
"\n",
|
| 196 |
-
" # 1. Locate dataset locally or auto-download from `ahpah/YokoCoder-4B`\n",
|
| 197 |
-
" dataset_path = find_input_file(\"*yokocoder_4b_reasoning_gamedev_bugfix*.jsonl\")\n",
|
| 198 |
-
" if dataset_path is None:\n",
|
| 199 |
-
" print(f\"Downloading dataset from Hugging Face repo {HF_TARGET_REPO}...\")\n",
|
| 200 |
-
" downloaded = hf_hub_download(\n",
|
| 201 |
" repo_id=HF_TARGET_REPO,\n",
|
| 202 |
" filename=\"datasets/yokocoder_4b_reasoning_gamedev_bugfix.jsonl\",\n",
|
| 203 |
" repo_type=\"model\",\n",
|
| 204 |
-
"
|
|
|
|
| 205 |
" )\n",
|
| 206 |
-
"
|
| 207 |
-
" print(f\"Using dataset: {dataset_path}\")\n",
|
| 208 |
"\n",
|
| 209 |
-
"
|
| 210 |
-
"
|
| 211 |
-
"
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 212 |
"\n",
|
| 213 |
-
" train_dataset = build_sliding_window_dataset(dataset_path, tokenizer)\n",
|
| 214 |
"\n",
|
| 215 |
-
"
|
| 216 |
-
"
|
| 217 |
-
"
|
| 218 |
-
"
|
| 219 |
-
" bnb_4bit_use_double_quant=True,\n",
|
| 220 |
" )\n",
|
| 221 |
-
"\n",
|
| 222 |
-
"
|
| 223 |
-
" model = AutoModelForCausalLM.from_pretrained(\n",
|
| 224 |
-
" BASE_MODEL_ID,\n",
|
| 225 |
-
" quantization_config=bnb_config,\n",
|
| 226 |
-
" device_map=\"auto\",\n",
|
| 227 |
-
" trust_remote_code=True,\n",
|
| 228 |
" )\n",
|
| 229 |
-
"
|
| 230 |
-
"
|
| 231 |
-
"\n",
|
| 232 |
-
"
|
| 233 |
-
"
|
| 234 |
-
"
|
| 235 |
-
"
|
| 236 |
-
"
|
| 237 |
-
"
|
| 238 |
-
"
|
| 239 |
-
"
|
| 240 |
-
"
|
| 241 |
-
"
|
| 242 |
-
"
|
| 243 |
-
"
|
| 244 |
-
"
|
| 245 |
-
"
|
| 246 |
-
"
|
| 247 |
-
"
|
| 248 |
-
"
|
| 249 |
-
"\n",
|
| 250 |
-
"
|
| 251 |
-
"
|
| 252 |
-
"
|
| 253 |
-
"
|
| 254 |
-
"
|
| 255 |
-
"
|
| 256 |
-
"
|
| 257 |
-
"
|
| 258 |
-
"
|
| 259 |
-
"
|
| 260 |
-
"
|
| 261 |
-
"
|
| 262 |
-
" \"q_proj\", \"k_proj\", \"v_proj\", \"o_proj\",\n",
|
| 263 |
-
" \"gate_proj\", \"up_proj\", \"down_proj\",\n",
|
| 264 |
-
" \"in_proj_qkv\", \"in_proj_z\", \"in_proj_b\", \"in_proj_a\", \"out_proj\",\n",
|
| 265 |
-
" ],\n",
|
| 266 |
" )\n",
|
| 267 |
-
"
|
| 268 |
-
"\n",
|
| 269 |
-
"
|
| 270 |
-
"\n",
|
| 271 |
-
"
|
| 272 |
-
"
|
| 273 |
-
"
|
| 274 |
-
"
|
| 275 |
-
"
|
| 276 |
-
"
|
| 277 |
-
"
|
| 278 |
-
"
|
| 279 |
-
"
|
| 280 |
-
"
|
| 281 |
-
"
|
| 282 |
-
"
|
| 283 |
-
"
|
| 284 |
-
"
|
| 285 |
-
"
|
| 286 |
-
" report_to=\"none\",\n",
|
| 287 |
" )\n",
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 288 |
"\n",
|
| 289 |
-
" trainer = Trainer(\n",
|
| 290 |
-
" model=model,\n",
|
| 291 |
-
" args=training_args,\n",
|
| 292 |
-
" train_dataset=train_dataset,\n",
|
| 293 |
-
" data_collator=CausalLMDataCollator(tokenizer.pad_token_id),\n",
|
| 294 |
-
" )\n",
|
| 295 |
"\n",
|
| 296 |
-
"
|
| 297 |
-
"
|
| 298 |
-
"\n",
|
| 299 |
-
"
|
| 300 |
-
"
|
| 301 |
-
"
|
| 302 |
-
"
|
| 303 |
-
"\n",
|
| 304 |
-
"
|
| 305 |
-
"
|
| 306 |
-
"
|
| 307 |
-
"\n",
|
| 308 |
-
"
|
| 309 |
-
"
|
| 310 |
-
"
|
| 311 |
-
"
|
| 312 |
-
"
|
| 313 |
-
"
|
| 314 |
-
"
|
| 315 |
-
"
|
| 316 |
-
"
|
| 317 |
-
"
|
| 318 |
-
"
|
| 319 |
-
"
|
| 320 |
-
" merged_model = merged_model.merge_and_unload()\n",
|
| 321 |
-
"\n",
|
| 322 |
-
" MERGED_OUT_DIR.mkdir(parents=True, exist_ok=True)\n",
|
| 323 |
-
" merged_model.save_pretrained(str(MERGED_OUT_DIR), safe_serialization=True)\n",
|
| 324 |
-
" tokenizer.save_pretrained(str(MERGED_OUT_DIR))\n",
|
| 325 |
-
" del merged_model, base_fp16\n",
|
| 326 |
-
" gc.collect()\n",
|
| 327 |
-
" print(f\"Merged 16-bit model saved to {MERGED_OUT_DIR}\")\n",
|
| 328 |
-
"\n",
|
| 329 |
-
" # ========================================================================\n",
|
| 330 |
-
" # 5. CONVERT TO GGUF & QUANTIZE TO Q4_K_M VIA LLAMA.CPP\n",
|
| 331 |
-
" # ========================================================================\n",
|
| 332 |
-
" if not LLAMA_CPP_DIR.exists():\n",
|
| 333 |
-
" subprocess.run(\n",
|
| 334 |
-
" [\"git\", \"clone\", \"--depth\", \"1\", \"https://github.com/ggerganov/llama.cpp\", str(LLAMA_CPP_DIR)],\n",
|
| 335 |
-
" check=True,\n",
|
| 336 |
" )\n",
|
| 337 |
-
" subprocess.run(\n",
|
| 338 |
-
" [\"cmake\", \"-B\", str(LLAMA_CPP_DIR / \"build\"), \"-S\", str(LLAMA_CPP_DIR), \"-DGGML_CUDA=OFF\"],\n",
|
| 339 |
-
" check=True,\n",
|
| 340 |
-
" )\n",
|
| 341 |
-
" subprocess.run(\n",
|
| 342 |
-
" [\"cmake\", \"--build\", str(LLAMA_CPP_DIR / \"build\"), \"--target\", \"llama-quantize\", \"-j\", \"4\"],\n",
|
| 343 |
-
" check=True,\n",
|
| 344 |
-
" )\n",
|
| 345 |
"\n",
|
| 346 |
-
"
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 347 |
" [\n",
|
| 348 |
-
"
|
| 349 |
-
"
|
| 350 |
-
"
|
| 351 |
-
" \"--outfile\",\n",
|
| 352 |
-
" str(F16_GGUF_PATH),\n",
|
| 353 |
-
" \"--outtype\",\n",
|
| 354 |
-
" \"f16\",\n",
|
| 355 |
" ],\n",
|
| 356 |
-
"
|
| 357 |
" )\n",
|
| 358 |
-
"\n",
|
| 359 |
-
"
|
| 360 |
-
"
|
| 361 |
-
" [str(quantize_bin), str(F16_GGUF_PATH), str(FINAL_GGUF_PATH), QUANT_TYPE],\n",
|
| 362 |
-
" check=True,\n",
|
| 363 |
" )\n",
|
| 364 |
-
"
|
| 365 |
-
"
|
| 366 |
-
"\n",
|
| 367 |
-
"
|
| 368 |
-
"
|
| 369 |
-
"
|
| 370 |
-
"
|
| 371 |
-
"
|
| 372 |
-
"
|
| 373 |
-
"
|
| 374 |
-
"\n",
|
| 375 |
-
"
|
| 376 |
-
"
|
| 377 |
-
"
|
| 378 |
-
"
|
| 379 |
-
"
|
| 380 |
-
"
|
| 381 |
-
"
|
| 382 |
-
"
|
| 383 |
-
"
|
| 384 |
-
"
|
| 385 |
-
"
|
| 386 |
-
"
|
| 387 |
-
"
|
| 388 |
-
"
|
| 389 |
-
"
|
| 390 |
-
"
|
| 391 |
-
"
|
| 392 |
-
"
|
| 393 |
-
"
|
| 394 |
-
"
|
| 395 |
-
"\n",
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 396 |
"\n",
|
| 397 |
-
"
|
| 398 |
-
"
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 399 |
]
|
| 400 |
}
|
| 401 |
],
|
|
|
|
| 4 |
"cell_type": "markdown",
|
| 5 |
"metadata": {},
|
| 6 |
"source": [
|
| 7 |
+
"# YokoCoder:4B (Qwen3.5-4B) \u2014 Unsloth 16-Bit LoRA Fine-Tuning, Safe Merge & GGUF Export\n"
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 8 |
]
|
| 9 |
},
|
| 10 |
{
|
|
|
|
| 15 |
"source": [
|
| 16 |
"#!/usr/bin/env python3\n",
|
| 17 |
"\"\"\"\n",
|
| 18 |
+
"YokoCoder:4B \u2014 Battle-Tested Unsloth 16-Bit LoRA Fine-Tuning, Safe Merge,\n",
|
| 19 |
+
"GGUF (Q4_K_M) Quantization (`--no-mtp`), and Hugging Face Export (`ahpah/YokoCoder-4B`).\n",
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 20 |
"\"\"\"\n",
|
| 21 |
"\n",
|
|
|
|
|
|
|
| 22 |
"import gc\n",
|
| 23 |
+
"import hashlib\n",
|
| 24 |
"import json\n",
|
| 25 |
+
"import math\n",
|
| 26 |
"import os\n",
|
| 27 |
"import subprocess\n",
|
| 28 |
"import sys\n",
|
| 29 |
"import zipfile\n",
|
| 30 |
+
"from concurrent.futures import ThreadPoolExecutor\n",
|
| 31 |
"from pathlib import Path\n",
|
| 32 |
"\n",
|
| 33 |
+
"os.environ[\"CUDA_VISIBLE_DEVICES\"] = \"0\"\n",
|
| 34 |
+
"os.environ[\"TOKENIZERS_PARALLELISM\"] = \"false\"\n",
|
| 35 |
+
"os.environ[\"WANDB_DISABLED\"] = \"true\"\n",
|
| 36 |
+
"\n",
|
| 37 |
+
"# 1. Ensure exact battle-tested Unsloth + Transformers versions are installed\n",
|
| 38 |
+
"subprocess.check_call([\n",
|
| 39 |
+
" sys.executable, \"-m\", \"pip\", \"install\", \"--quiet\", \"--upgrade\",\n",
|
| 40 |
+
" \"unsloth==2026.10.10\", \"unsloth_zoo\", \"transformers==5.17.0\",\n",
|
| 41 |
+
" \"peft\", \"accelerate\", \"datasets\", \"bitsandbytes\",\n",
|
| 42 |
+
" \"huggingface_hub\", \"sentencepiece\", \"protobuf\", \"gguf\"\n",
|
| 43 |
+
"])\n",
|
| 44 |
+
"\n",
|
| 45 |
+
"from unsloth import FastLanguageModel\n",
|
| 46 |
"import torch\n",
|
| 47 |
"from datasets import Dataset\n",
|
| 48 |
+
"from huggingface_hub import HfApi, hf_hub_download, snapshot_download\n",
|
| 49 |
+
"from peft import PeftModel\n",
|
| 50 |
+
"from transformers import AutoTokenizer, Trainer, TrainerCallback, TrainingArguments, set_seed\n",
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 51 |
"\n",
|
| 52 |
+
"assert torch.cuda.is_available(), \"Enable a GPU accelerator in Kaggle Settings!\"\n",
|
|
|
|
| 53 |
"\n",
|
| 54 |
+
"SEED = 3407\n",
|
| 55 |
+
"set_seed(SEED)\n",
|
|
|
|
|
|
|
|
|
|
|
|
|
| 56 |
"\n",
|
| 57 |
+
"EPOCHS = 3\n",
|
| 58 |
+
"MAX_SEQ_LENGTH = 4096\n",
|
| 59 |
+
"OVERLAP = 512\n",
|
| 60 |
+
"GRADIENT_ACCUMULATION = 8\n",
|
|
|
|
|
|
|
| 61 |
"\n",
|
| 62 |
+
"HF_TARGET_REPO = \"ahpah/YokoCoder-4B\"\n",
|
| 63 |
+
"BASE_NAME = \"wangzhang/Qwen3.5-4B-abliterated\"\n",
|
| 64 |
+
"BASE_REVISION = \"bbf0cdd540907b95ca8430fd34e4b2971f73df57\"\n",
|
| 65 |
"\n",
|
| 66 |
+
"ROOT = Path(\"/kaggle/working/yokocoder_v2_run\")\n",
|
| 67 |
+
"ROOT.mkdir(parents=True, exist_ok=True)\n",
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 68 |
"\n",
|
| 69 |
"\n",
|
| 70 |
"def get_hf_token() -> str | None:\n",
|
|
|
|
| 78 |
" return None\n",
|
| 79 |
"\n",
|
| 80 |
"\n",
|
| 81 |
+
"HF_TOKEN = get_hf_token()\n",
|
| 82 |
+
"\n",
|
| 83 |
+
"# ---------------------------------------------------------------------------\n",
|
| 84 |
+
"# 2. LOCATE OR DOWNLOAD STARTING ADAPTER FROM `ahpah/YokoCoder-4B`\n",
|
| 85 |
+
"# ---------------------------------------------------------------------------\n",
|
| 86 |
+
"found = list(Path(\"/kaggle/input\").glob(\"**/final_adapter/adapter_config.json\"))\n",
|
| 87 |
+
"if found:\n",
|
| 88 |
+
" ckpt = found[0].parent\n",
|
| 89 |
+
"else:\n",
|
| 90 |
+
" print(f\"Downloading starting adapter from {HF_TARGET_REPO}...\", flush=True)\n",
|
| 91 |
+
" snap_dir = snapshot_download(\n",
|
| 92 |
+
" repo_id=HF_TARGET_REPO,\n",
|
| 93 |
+
" allow_patterns=[\"adapters/dataset4-5epochs/final_adapter/*\"],\n",
|
| 94 |
+
" token=HF_TOKEN,\n",
|
| 95 |
+
" )\n",
|
| 96 |
+
" ckpt = Path(snap_dir) / \"adapters\" / \"dataset4-5epochs\" / \"final_adapter\"\n",
|
| 97 |
+
"\n",
|
| 98 |
+
"adapter_sha = hashlib.sha256((ckpt / \"adapter_model.safetensors\").read_bytes()).hexdigest()\n",
|
| 99 |
+
"print(\"START_ADAPTER \" + json.dumps({\"path\": str(ckpt), \"sha256\": adapter_sha}), flush=True)\n",
|
| 100 |
+
"\n",
|
| 101 |
+
"tokenizer = AutoTokenizer.from_pretrained(ckpt)\n",
|
| 102 |
+
"if tokenizer.pad_token_id is None:\n",
|
| 103 |
+
" tokenizer.pad_token = tokenizer.eos_token\n",
|
| 104 |
+
"tokenizer.padding_side = \"right\"\n",
|
| 105 |
+
"\n",
|
| 106 |
+
"bf16 = torch.cuda.is_bf16_supported()\n",
|
| 107 |
+
"base, _ = FastLanguageModel.from_pretrained(\n",
|
| 108 |
+
" model_name=BASE_NAME,\n",
|
| 109 |
+
" revision=BASE_REVISION,\n",
|
| 110 |
+
" max_seq_length=MAX_SEQ_LENGTH,\n",
|
| 111 |
+
" dtype=torch.bfloat16 if bf16 else torch.float16,\n",
|
| 112 |
+
" load_in_4bit=False,\n",
|
| 113 |
+
" load_in_16bit=True,\n",
|
| 114 |
+
" full_finetuning=False,\n",
|
| 115 |
+
" fast_inference=False,\n",
|
| 116 |
+
")\n",
|
| 117 |
+
"model = PeftModel.from_pretrained(base, ckpt, is_trainable=True)\n",
|
| 118 |
+
"FastLanguageModel.for_training(model, use_gradient_checkpointing=\"unsloth\")\n",
|
| 119 |
+
"model.config.use_cache = False\n",
|
| 120 |
+
"model.print_trainable_parameters()\n",
|
| 121 |
+
"print(\"TRAINABLE_ADAPTER_LOADED\", flush=True)\n",
|
| 122 |
+
"\n",
|
| 123 |
+
"# ---------------------------------------------------------------------------\n",
|
| 124 |
+
"# 3. LOCATE OR DOWNLOAD DATASET (`yokocoder_4b_reasoning_gamedev_bugfix.jsonl`)\n",
|
| 125 |
+
"# ---------------------------------------------------------------------------\n",
|
| 126 |
+
"data_files = list(Path(\"/kaggle/input\").glob(\"**/yokocoder_4b_reasoning_gamedev_bugfix.jsonl\"))\n",
|
| 127 |
+
"if data_files:\n",
|
| 128 |
+
" dataset_path = data_files[0]\n",
|
| 129 |
+
"else:\n",
|
| 130 |
+
" print(f\"Downloading dataset from {HF_TARGET_REPO}...\", flush=True)\n",
|
| 131 |
+
" dataset_path = Path(\n",
|
| 132 |
+
" hf_hub_download(\n",
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 133 |
" repo_id=HF_TARGET_REPO,\n",
|
| 134 |
" filename=\"datasets/yokocoder_4b_reasoning_gamedev_bugfix.jsonl\",\n",
|
| 135 |
" repo_type=\"model\",\n",
|
| 136 |
+
" force_download=True,\n",
|
| 137 |
+
" token=HF_TOKEN,\n",
|
| 138 |
" )\n",
|
| 139 |
+
" )\n",
|
|
|
|
| 140 |
"\n",
|
| 141 |
+
"raw = dataset_path.read_bytes()\n",
|
| 142 |
+
"records = [json.loads(line) for line in raw.decode(\"utf-8-sig\").splitlines() if line.strip()]\n",
|
| 143 |
+
"for i, r in enumerate(records):\n",
|
| 144 |
+
" assert isinstance(r, dict) and isinstance(r.get(\"messages\"), list), f\"Bad record {i}\"\n",
|
| 145 |
+
" assert [m.get(\"role\") for m in r[\"messages\"]] == [\"system\", \"user\", \"assistant\"], f\"Role order error in {i}\"\n",
|
| 146 |
+
" assert all(isinstance(m.get(\"content\"), str) and m[\"content\"].strip() for m in r[\"messages\"]), f\"Empty msg in {i}\"\n",
|
| 147 |
+
"\n",
|
| 148 |
+
"DATA_SHA = hashlib.sha256(raw).hexdigest()\n",
|
| 149 |
+
"print(\n",
|
| 150 |
+
" \"DATA_VERIFIED \"\n",
|
| 151 |
+
" + json.dumps({\"path\": str(dataset_path), \"examples\": len(records), \"bytes\": len(raw), \"sha256\": DATA_SHA}),\n",
|
| 152 |
+
" flush=True,\n",
|
| 153 |
+
")\n",
|
| 154 |
"\n",
|
|
|
|
| 155 |
"\n",
|
| 156 |
+
"def encode_record(record):\n",
|
| 157 |
+
" messages = record[\"messages\"]\n",
|
| 158 |
+
" prefix = tokenizer.apply_chat_template(\n",
|
| 159 |
+
" messages[:-1], tokenize=False, add_generation_prompt=True, enable_thinking=True\n",
|
|
|
|
| 160 |
" )\n",
|
| 161 |
+
" full = tokenizer.apply_chat_template(\n",
|
| 162 |
+
" messages, tokenize=False, add_generation_prompt=False, enable_thinking=True\n",
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 163 |
" )\n",
|
| 164 |
+
" assert full.startswith(prefix), \"Chat template prefix mismatch\"\n",
|
| 165 |
+
" encoded = tokenizer(full, add_special_tokens=False, return_offsets_mapping=True)\n",
|
| 166 |
+
" ids = encoded[\"input_ids\"]\n",
|
| 167 |
+
" labels = [\n",
|
| 168 |
+
" tok if end > len(prefix) and start >= len(prefix) else -100\n",
|
| 169 |
+
" for tok, (start, end) in zip(ids, encoded[\"offset_mapping\"])\n",
|
| 170 |
+
" ]\n",
|
| 171 |
+
" return ids, labels\n",
|
| 172 |
+
"\n",
|
| 173 |
+
"\n",
|
| 174 |
+
"def make_windows(rows):\n",
|
| 175 |
+
" windows = []\n",
|
| 176 |
+
" stats = []\n",
|
| 177 |
+
" stride = MAX_SEQ_LENGTH - OVERLAP\n",
|
| 178 |
+
" for idx, record in enumerate(rows):\n",
|
| 179 |
+
" ids, labels = encode_record(record)\n",
|
| 180 |
+
" expected = sum(v != -100 for v in labels[1:])\n",
|
| 181 |
+
" actual = 0\n",
|
| 182 |
+
" for start in range(0, len(ids), stride):\n",
|
| 183 |
+
" end = min(start + MAX_SEQ_LENGTH, len(ids))\n",
|
| 184 |
+
" x = ids[start:end]\n",
|
| 185 |
+
" y = labels[start:end].copy()\n",
|
| 186 |
+
" for j in range(min(OVERLAP if start else 1, len(y))):\n",
|
| 187 |
+
" y[j] = -100\n",
|
| 188 |
+
" count = sum(v != -100 for v in y)\n",
|
| 189 |
+
" if count:\n",
|
| 190 |
+
" windows.append({\"input_ids\": x, \"attention_mask\": [1] * len(x), \"labels\": y})\n",
|
| 191 |
+
" actual += count\n",
|
| 192 |
+
" if end == len(ids):\n",
|
| 193 |
+
" break\n",
|
| 194 |
+
" assert actual == expected, (idx, actual, expected)\n",
|
| 195 |
+
" stats.append(\n",
|
| 196 |
+
" {\"example\": idx, \"tokens\": len(ids), \"assistant_targets\": expected, \"covered_targets\": actual}\n",
|
|
|
|
|
|
|
|
|
|
|
|
|
| 197 |
" )\n",
|
| 198 |
+
" assert windows\n",
|
| 199 |
+
" return windows, stats\n",
|
| 200 |
+
"\n",
|
| 201 |
+
"\n",
|
| 202 |
+
"train_windows, coverage = make_windows(records)\n",
|
| 203 |
+
"train_dataset = Dataset.from_list(train_windows)\n",
|
| 204 |
+
"(ROOT / \"run_config.json\").write_text(\n",
|
| 205 |
+
" json.dumps(\n",
|
| 206 |
+
" {\n",
|
| 207 |
+
" \"dataset_sha256\": DATA_SHA,\n",
|
| 208 |
+
" \"starting_adapter_sha256\": adapter_sha,\n",
|
| 209 |
+
" \"base_model\": BASE_NAME,\n",
|
| 210 |
+
" \"base_revision\": BASE_REVISION,\n",
|
| 211 |
+
" \"additional_epochs\": EPOCHS,\n",
|
| 212 |
+
" \"train_examples\": len(records),\n",
|
| 213 |
+
" \"train_windows\": len(train_windows),\n",
|
| 214 |
+
" \"coverage\": coverage,\n",
|
| 215 |
+
" },\n",
|
| 216 |
+
" indent=2,\n",
|
|
|
|
| 217 |
" )\n",
|
| 218 |
+
")\n",
|
| 219 |
+
"print(\n",
|
| 220 |
+
" \"TRAIN_DATA_READY \"\n",
|
| 221 |
+
" + json.dumps(\n",
|
| 222 |
+
" {\n",
|
| 223 |
+
" \"examples\": len(records),\n",
|
| 224 |
+
" \"windows\": len(train_windows),\n",
|
| 225 |
+
" \"targets\": sum(x[\"covered_targets\"] for x in coverage),\n",
|
| 226 |
+
" }\n",
|
| 227 |
+
" ),\n",
|
| 228 |
+
" flush=True,\n",
|
| 229 |
+
")\n",
|
| 230 |
"\n",
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 231 |
"\n",
|
| 232 |
+
"# ---------------------------------------------------------------------------\n",
|
| 233 |
+
"# 4. FINE-TUNE WITH UNSLOTH 16-BIT + ADAMW_8BIT\n",
|
| 234 |
+
"# ---------------------------------------------------------------------------\n",
|
| 235 |
+
"def collate(features):\n",
|
| 236 |
+
" size = math.ceil(max(len(f[\"input_ids\"]) for f in features) / 8) * 8\n",
|
| 237 |
+
" return {\n",
|
| 238 |
+
" \"input_ids\": torch.tensor(\n",
|
| 239 |
+
" [f[\"input_ids\"] + [tokenizer.pad_token_id] * (size - len(f[\"input_ids\"])) for f in features]\n",
|
| 240 |
+
" ),\n",
|
| 241 |
+
" \"attention_mask\": torch.tensor(\n",
|
| 242 |
+
" [f[\"attention_mask\"] + [0] * (size - len(f[\"input_ids\"])) for f in features]\n",
|
| 243 |
+
" ),\n",
|
| 244 |
+
" \"labels\": torch.tensor([f[\"labels\"] + [-100] * (size - len(f[\"labels\"])) for f in features]),\n",
|
| 245 |
+
" }\n",
|
| 246 |
+
"\n",
|
| 247 |
+
"\n",
|
| 248 |
+
"class Progress(TrainerCallback):\n",
|
| 249 |
+
" def on_log(self, args, state, control, logs=None, **kwargs):\n",
|
| 250 |
+
" if logs and \"loss\" in logs and not math.isfinite(logs[\"loss\"]):\n",
|
| 251 |
+
" raise FloatingPointError(\"non-finite loss\")\n",
|
| 252 |
+
" print(\n",
|
| 253 |
+
" \"TRAIN_PROGRESS \"\n",
|
| 254 |
+
" + json.dumps({\"epoch\": state.epoch, \"step\": state.global_step, \"metrics\": logs}),\n",
|
| 255 |
+
" flush=True,\n",
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 256 |
" )\n",
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 257 |
"\n",
|
| 258 |
+
"\n",
|
| 259 |
+
"args = TrainingArguments(\n",
|
| 260 |
+
" output_dir=str(ROOT / \"checkpoints\"),\n",
|
| 261 |
+
" num_train_epochs=EPOCHS,\n",
|
| 262 |
+
" per_device_train_batch_size=1,\n",
|
| 263 |
+
" gradient_accumulation_steps=GRADIENT_ACCUMULATION,\n",
|
| 264 |
+
" learning_rate=2e-5,\n",
|
| 265 |
+
" warmup_steps=0,\n",
|
| 266 |
+
" lr_scheduler_type=\"constant\",\n",
|
| 267 |
+
" weight_decay=0.01,\n",
|
| 268 |
+
" max_grad_norm=0.3,\n",
|
| 269 |
+
" fp16=False,\n",
|
| 270 |
+
" bf16=bf16,\n",
|
| 271 |
+
" optim=\"adamw_8bit\",\n",
|
| 272 |
+
" save_strategy=\"epoch\",\n",
|
| 273 |
+
" save_total_limit=2,\n",
|
| 274 |
+
" logging_steps=1,\n",
|
| 275 |
+
" logging_nan_inf_filter=False,\n",
|
| 276 |
+
" report_to=\"none\",\n",
|
| 277 |
+
" disable_tqdm=True,\n",
|
| 278 |
+
" seed=SEED,\n",
|
| 279 |
+
" data_seed=SEED,\n",
|
| 280 |
+
" dataloader_num_workers=0,\n",
|
| 281 |
+
" remove_unused_columns=False,\n",
|
| 282 |
+
")\n",
|
| 283 |
+
"\n",
|
| 284 |
+
"trainer = Trainer(\n",
|
| 285 |
+
" model=model,\n",
|
| 286 |
+
" args=args,\n",
|
| 287 |
+
" train_dataset=train_dataset,\n",
|
| 288 |
+
" data_collator=collate,\n",
|
| 289 |
+
" processing_class=tokenizer,\n",
|
| 290 |
+
" callbacks=[Progress()],\n",
|
| 291 |
+
")\n",
|
| 292 |
+
"result = trainer.train()\n",
|
| 293 |
+
"trainer.save_model(str(ROOT / \"final_adapter\"))\n",
|
| 294 |
+
"tokenizer.save_pretrained(ROOT / \"final_adapter\")\n",
|
| 295 |
+
"trainer.save_state()\n",
|
| 296 |
+
"trainer.save_metrics(\"train\", result.metrics)\n",
|
| 297 |
+
"\n",
|
| 298 |
+
"epoch = float(trainer.state.epoch or 0)\n",
|
| 299 |
+
"status = {\n",
|
| 300 |
+
" \"completed\": epoch >= EPOCHS - 1e-5,\n",
|
| 301 |
+
" \"requested_epochs\": EPOCHS,\n",
|
| 302 |
+
" \"completed_epochs\": epoch,\n",
|
| 303 |
+
" \"global_step\": trainer.state.global_step,\n",
|
| 304 |
+
" \"final_adapter\": str(ROOT / \"final_adapter\"),\n",
|
| 305 |
+
" \"metrics\": result.metrics,\n",
|
| 306 |
+
"}\n",
|
| 307 |
+
"(ROOT / \"training_status.json\").write_text(json.dumps(status, indent=2))\n",
|
| 308 |
+
"print(\"TRAINING_COMPLETE \" + json.dumps(status), flush=True)\n",
|
| 309 |
+
"assert status[\"completed\"], f\"Training stopped before {EPOCHS} epochs\"\n",
|
| 310 |
+
"\n",
|
| 311 |
+
"# ---------------------------------------------------------------------------\n",
|
| 312 |
+
"# 5. MERGE TO 16-BIT & QUANTIZE TO Q4_K_M GGUF (`--no-mtp`)\n",
|
| 313 |
+
"# ---------------------------------------------------------------------------\n",
|
| 314 |
+
"def build_tools():\n",
|
| 315 |
+
" repo = Path(\"/kaggle/temp/llama.cpp\")\n",
|
| 316 |
+
" if not repo.exists():\n",
|
| 317 |
+
" subprocess.check_call([\"git\", \"clone\", \"--depth\", \"1\", \"https://github.com/ggml-org/llama.cpp\", str(repo)])\n",
|
| 318 |
+
" subprocess.check_call(\n",
|
| 319 |
" [\n",
|
| 320 |
+
" \"cmake\", \"-S\", str(repo), \"-B\", str(repo / \"build\"),\n",
|
| 321 |
+
" \"-DGGML_CUDA=OFF\", \"-DLLAMA_CURL=OFF\", \"-DLLAMA_BUILD_TESTS=OFF\",\n",
|
| 322 |
+
" \"-DLLAMA_BUILD_EXAMPLES=OFF\", \"-DLLAMA_BUILD_TOOLS=ON\", \"-DBUILD_SHARED_LIBS=OFF\",\n",
|
|
|
|
|
|
|
|
|
|
|
|
|
| 323 |
" ],\n",
|
| 324 |
+
" stdout=subprocess.DEVNULL,\n",
|
| 325 |
" )\n",
|
| 326 |
+
" subprocess.check_call(\n",
|
| 327 |
+
" [\"cmake\", \"--build\", str(repo / \"build\"), \"--config\", \"Release\", \"--target\", \"llama-quantize\", \"-j\", \"2\"],\n",
|
| 328 |
+
" stdout=subprocess.DEVNULL,\n",
|
|
|
|
|
|
|
| 329 |
" )\n",
|
| 330 |
+
" return repo\n",
|
| 331 |
+
"\n",
|
| 332 |
+
"\n",
|
| 333 |
+
"pool = ThreadPoolExecutor(max_workers=1)\n",
|
| 334 |
+
"future = pool.submit(build_tools)\n",
|
| 335 |
+
"\n",
|
| 336 |
+
"print(\"MERGE_START\", flush=True)\n",
|
| 337 |
+
"merged = Path(\"/kaggle/temp/YokoCoder-4B-v2-merged\")\n",
|
| 338 |
+
"model = model.merge_and_unload(safe_merge=True)\n",
|
| 339 |
+
"model = model.to(dtype=torch.bfloat16)\n",
|
| 340 |
+
"model.config.eos_token_id = tokenizer.eos_token_id\n",
|
| 341 |
+
"model.generation_config.eos_token_id = tokenizer.eos_token_id\n",
|
| 342 |
+
"model.save_pretrained(merged, safe_serialization=True, max_shard_size=\"4GB\")\n",
|
| 343 |
+
"tokenizer.save_pretrained(merged)\n",
|
| 344 |
+
"print(\"MERGED_WEIGHTS_SAVED\", flush=True)\n",
|
| 345 |
+
"\n",
|
| 346 |
+
"del model, base, trainer\n",
|
| 347 |
+
"gc.collect()\n",
|
| 348 |
+
"torch.cuda.empty_cache()\n",
|
| 349 |
+
"\n",
|
| 350 |
+
"repo = future.result()\n",
|
| 351 |
+
"pool.shutdown()\n",
|
| 352 |
+
"\n",
|
| 353 |
+
"intermediate = Path(\"/kaggle/temp/YokoCoder-4B-v2-f16.gguf\")\n",
|
| 354 |
+
"subprocess.check_call(\n",
|
| 355 |
+
" [\n",
|
| 356 |
+
" sys.executable,\n",
|
| 357 |
+
" str(repo / \"convert_hf_to_gguf.py\"),\n",
|
| 358 |
+
" str(merged),\n",
|
| 359 |
+
" \"--no-mtp\",\n",
|
| 360 |
+
" \"--outtype\",\n",
|
| 361 |
+
" \"f16\",\n",
|
| 362 |
+
" \"--outfile\",\n",
|
| 363 |
+
" str(intermediate),\n",
|
| 364 |
+
" ],\n",
|
| 365 |
+
" stdout=subprocess.DEVNULL,\n",
|
| 366 |
+
")\n",
|
| 367 |
+
"print(\"GGUF_F16_READY\", flush=True)\n",
|
| 368 |
"\n",
|
| 369 |
+
"output = Path(\"/kaggle/working/YokoCoder-4B-5epochs-Q4_K_M.gguf\")\n",
|
| 370 |
+
"subprocess.check_call(\n",
|
| 371 |
+
" [str(repo / \"build/bin/llama-quantize\"), str(intermediate), str(output), \"Q4_K_M\", \"4\"],\n",
|
| 372 |
+
" stdout=subprocess.DEVNULL,\n",
|
| 373 |
+
")\n",
|
| 374 |
+
"intermediate.unlink(missing_ok=True)\n",
|
| 375 |
+
"print(f\"GGUF_Q4_K_M_READY: {output} ({output.stat().st_size:,} bytes)\", flush=True)\n",
|
| 376 |
+
"\n",
|
| 377 |
+
"# ---------------------------------------------------------------------------\n",
|
| 378 |
+
"# 6. PUSH UPDATED GGUF & ADAPTER TO HUGGING FACE (`ahpah/YokoCoder-4B`)\n",
|
| 379 |
+
"# ---------------------------------------------------------------------------\n",
|
| 380 |
+
"if HF_TOKEN:\n",
|
| 381 |
+
" print(f\"Pushing {output.name} and v2 adapter to Hugging Face ({HF_TARGET_REPO})...\", flush=True)\n",
|
| 382 |
+
" api = HfApi(token=HF_TOKEN)\n",
|
| 383 |
+
" api.upload_file(\n",
|
| 384 |
+
" path_or_fileobj=str(output),\n",
|
| 385 |
+
" path_in_repo=\"YokoCoder-4B-5epochs-Q4_K_M.gguf\",\n",
|
| 386 |
+
" repo_id=HF_TARGET_REPO,\n",
|
| 387 |
+
" repo_type=\"model\",\n",
|
| 388 |
+
" commit_message=\"Update YokoCoder-4B-5epochs-Q4_K_M.gguf with v2 Structured Reasoning, Bug-Fixing & 2D/3D GameDev fine-tune\",\n",
|
| 389 |
+
" )\n",
|
| 390 |
+
" api.upload_folder(\n",
|
| 391 |
+
" folder_path=str(ROOT / \"final_adapter\"),\n",
|
| 392 |
+
" path_in_repo=\"adapters/v2-reasoning-gamedev-bugfix/final_adapter\",\n",
|
| 393 |
+
" repo_id=HF_TARGET_REPO,\n",
|
| 394 |
+
" repo_type=\"model\",\n",
|
| 395 |
+
" commit_message=\"Upload YokoCoder:4B v2 final_adapter\",\n",
|
| 396 |
+
" )\n",
|
| 397 |
+
" print(f\"HF_UPLOAD_COMPLETE: https://huggingface.co/{HF_TARGET_REPO}\", flush=True)\n",
|
| 398 |
+
"else:\n",
|
| 399 |
+
" print(\"WARNING: HF_TOKEN not found; GGUF saved to /kaggle/working/YokoCoder-4B-5epochs-Q4_K_M.gguf\", flush=True)\n"
|
| 400 |
]
|
| 401 |
}
|
| 402 |
],
|