Instructions to use zerolimites/Qwen3-0.6B-Code-Surgery-v2 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Notebooks
Google Colab
Kaggle
Local Apps Settings

How to use zerolimites/Qwen3-0.6B-Code-Surgery-v2 with llama.cpp:

Install (macOS, Linux)

curl -LsSf https://llama.app/install.sh | sh
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf zerolimites/Qwen3-0.6B-Code-Surgery-v2:F16
# Run inference directly in the terminal:
llama cli -hf zerolimites/Qwen3-0.6B-Code-Surgery-v2:F16

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf zerolimites/Qwen3-0.6B-Code-Surgery-v2:F16
# Run inference directly in the terminal:
llama cli -hf zerolimites/Qwen3-0.6B-Code-Surgery-v2:F16

Build from source code

git clone https://github.com/ggml-org/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-app
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama serve -hf zerolimites/Qwen3-0.6B-Code-Surgery-v2:F16
# Run inference directly in the terminal:
./build/bin/llama cli -hf zerolimites/Qwen3-0.6B-Code-Surgery-v2:F16

Use Docker

docker model run hf.co/zerolimites/Qwen3-0.6B-Code-Surgery-v2:F16

Llama new
LM Studio
Jan

vLLM

How to use zerolimites/Qwen3-0.6B-Code-Surgery-v2 with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "zerolimites/Qwen3-0.6B-Code-Surgery-v2"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "zerolimites/Qwen3-0.6B-Code-Surgery-v2",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/zerolimites/Qwen3-0.6B-Code-Surgery-v2:F16

Ollama
How to use zerolimites/Qwen3-0.6B-Code-Surgery-v2 with Ollama:
```
ollama run hf.co/zerolimites/Qwen3-0.6B-Code-Surgery-v2:F16
```
Unsloth Desktop

How to use zerolimites/Qwen3-0.6B-Code-Surgery-v2 with Pi:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf zerolimites/Qwen3-0.6B-Code-Surgery-v2:F16

Configure the model in Pi

# Install Pi:
npm install -g @earendil-works/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "llama-cpp": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "zerolimites/Qwen3-0.6B-Code-Surgery-v2:F16"
        }
      ]
    }
  }
}

Run Pi

# Start Pi in your project directory:
pi

Docker Model Runner
How to use zerolimites/Qwen3-0.6B-Code-Surgery-v2 with Docker Model Runner:
```
docker model run hf.co/zerolimites/Qwen3-0.6B-Code-Surgery-v2:F16
```

Lemonade

How to use zerolimites/Qwen3-0.6B-Code-Surgery-v2 with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull zerolimites/Qwen3-0.6B-Code-Surgery-v2:F16

Run and chat with the model

lemonade run user.Qwen3-0.6B-Code-Surgery-v2-F16

List all available models

lemonade list

Hermes Agent

How to use zerolimites/Qwen3-0.6B-Code-Surgery-v2 with Hermes Agent:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf zerolimites/Qwen3-0.6B-Code-Surgery-v2:F16

Configure Hermes

# Install Hermes:
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
# Point Hermes at the local server:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default zerolimites/Qwen3-0.6B-Code-Surgery-v2:F16

Run Hermes

hermes

Atomic Chat

OpenClaw

How to use zerolimites/Qwen3-0.6B-Code-Surgery-v2 with OpenClaw:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf zerolimites/Qwen3-0.6B-Code-Surgery-v2:F16

Configure OpenClaw

# Install OpenClaw:
npm install -g openclaw@latest
# Register the local server and set it as the default model:
openclaw onboard --non-interactive --mode local \
  --auth-choice custom-api-key \
  --custom-base-url http://127.0.0.1:8080/v1 \
  --custom-model-id "zerolimites/Qwen3-0.6B-Code-Surgery-v2:F16" \
  --custom-provider-id llama-cpp \
  --custom-compatibility openai \
  --custom-text-input \
  --accept-risk \
  --skip-health

Run OpenClaw

openclaw agent --local --agent main --message "Hello from Hugging Face"

Qwen3-0.6B-Code-Surgery-v2 / code_eval_engine.py

zerolimites

Upload Qwen3-0.6B-Code-Surgery-v2 (96.43% Pass@1 - Safetensors + GGUF Q8_0 & F16 + JSONs)

f10f49d verified 2 days ago

Raw History Blame Contribute Delete

4.32 kB

import os
import gc
import time
import json
import subprocess
import sys
import torch
from safetensors.torch import load_file
from eval_engine import MODEL_DIR, Qwen3FastRunner, decode_in_subprocess
from code_benchmark_problems import CODE_BENCHMARK_PROBLEMS, verify_code_solution

torch.set_num_threads(2)
CODE_TOK_CACHE_PATH = "/home/user/.cache/code_benchmark_tokens.pt"


def ensure_code_tokenized_cache():
    if os.path.exists(CODE_TOK_CACHE_PATH):
        return
    cmd = [
        sys.executable,
        "-c",
        (
            "import torch; from transformers import AutoTokenizer; "
            "from code_benchmark_problems import CODE_BENCHMARK_PROBLEMS; "
            f"tok = AutoTokenizer.from_pretrained('{MODEL_DIR}'); "
            "pids = [tok.encode(tok.apply_chat_template([{'role':'user','content':p['prompt']}], tokenize=False, add_generation_prompt=True, enable_thinking=False), add_special_tokens=False) for p in CODE_BENCHMARK_PROBLEMS]; "
            f"torch.save({{'prompt_ids_list': pids, 'eos_id': tok.eos_token_id, 'pad_id': tok.pad_token_id}}, '{CODE_TOK_CACHE_PATH}')"
        ),
    ]
    subprocess.run(cmd, check=True)


def evaluate_code_state_dict(sd, problems=CODE_BENCHMARK_PROBLEMS, max_new_tokens=90, batch_size=7, verbose=True):
    ensure_code_tokenized_cache()
    tok_data = torch.load(CODE_TOK_CACHE_PATH, weights_only=True)
    prompt_ids_list = tok_data["prompt_ids_list"]
    eos_id = tok_data["eos_id"]
    pad_id = tok_data["pad_id"]

    order = sorted(range(len(problems)), key=lambda i: len(prompt_ids_list[i]))
    runner = Qwen3FastRunner(sd)

    t0 = time.time()
    ordered_gen_ids = []
    num_batches = (len(problems) + batch_size - 1) // batch_size
    for b_idx, start in enumerate(range(0, len(problems), batch_size)):
        end = min(start + batch_size, len(problems))
        batch_indices = order[start:end]
        batch_prompts = [prompt_ids_list[i] for i in batch_indices]
        tb0 = time.time()
        gen_ids = runner.generate_batch(batch_prompts, eos_id, pad_id, max_new_tokens=max_new_tokens)
        ordered_gen_ids.extend(gen_ids)
        if verbose:
            avg_t = sum(len(g) for g in gen_ids) / len(gen_ids)
            print(f"  Batch {b_idx+1}/{num_batches} ({end-start} tasks, avg {avg_t:.1f} toks) in {time.time()-tb0:.1f}s", flush=True)
    elapsed = time.time() - t0

    all_gen_ids = [None] * len(problems)
    for orig_idx, g_ids in zip(order, ordered_gen_ids):
        all_gen_ids[orig_idx] = g_ids

    del runner
    gc.collect()

    responses = decode_in_subprocess(all_gen_ids)
    results = []
    passed = 0
    for prob, resp, g_ids in zip(problems, responses, all_gen_ids):
        ok, detail = verify_code_solution(prob, resp)
        if ok:
            passed += 1
        results.append({
            "id": prob["id"],
            "category": prob["category"],
            "prompt": prob["prompt"],
            "passed": ok,
            "detail": detail,
            "num_tokens": len(g_ids),
            "response": resp,
        })

    summary = {
        "total": len(problems),
        "passed": passed,
        "accuracy": round(100.0 * passed / len(problems), 2),
        "avg_tokens": round(sum(r["num_tokens"] for r in results) / len(results), 1),
        "elapsed_sec": round(elapsed, 2),
        "results": results,
    }
    return summary


if __name__ == "__main__":
    print("Evaluating Baseline Qwen3-0.6B on 28-task Executable Code Benchmark...", flush=True)
    sd = load_file(os.path.join(MODEL_DIR, "model.safetensors"), device="cpu")
    summary = evaluate_code_state_dict(sd, CODE_BENCHMARK_PROBLEMS, max_new_tokens=90, batch_size=7, verbose=True)
    with open("/home/user/code_baseline_results.json", "w") as f:
        json.dump(summary, f, indent=2)
    print(f"\n==========================================", flush=True)
    print(f"CODE BASELINE QWEN3-0.6B: {summary['passed']}/{summary['total']} ({summary['accuracy']}%) | Avg Toks: {summary['avg_tokens']} | {summary['elapsed_sec']}s", flush=True)
    print(f"==========================================", flush=True)
    for r in summary["results"]:
        status = "PASS" if r["passed"] else "FAIL"
        print(f"[{status:4s}] {r['id']:8s} ({r['category']:16s}) | Toks: {r['num_tokens']:3d} | {r['detail']}", flush=True)

	import os
	import gc
	import time
	import json
	import subprocess
	import sys
	import torch
	from safetensors.torch import load_file
	from eval_engine import MODEL_DIR, Qwen3FastRunner, decode_in_subprocess
	from code_benchmark_problems import CODE_BENCHMARK_PROBLEMS, verify_code_solution

	torch.set_num_threads(2)
	CODE_TOK_CACHE_PATH = "/home/user/.cache/code_benchmark_tokens.pt"


	def ensure_code_tokenized_cache():
	if os.path.exists(CODE_TOK_CACHE_PATH):
	return
	cmd = [
	sys.executable,
	"-c",
	(
	"import torch; from transformers import AutoTokenizer; "
	"from code_benchmark_problems import CODE_BENCHMARK_PROBLEMS; "
	f"tok = AutoTokenizer.from_pretrained('{MODEL_DIR}'); "
	"pids = [tok.encode(tok.apply_chat_template([{'role':'user','content':p['prompt']}], tokenize=False, add_generation_prompt=True, enable_thinking=False), add_special_tokens=False) for p in CODE_BENCHMARK_PROBLEMS]; "
	f"torch.save({{'prompt_ids_list': pids, 'eos_id': tok.eos_token_id, 'pad_id': tok.pad_token_id}}, '{CODE_TOK_CACHE_PATH}')"
	),
	]
	subprocess.run(cmd, check=True)


	def evaluate_code_state_dict(sd, problems=CODE_BENCHMARK_PROBLEMS, max_new_tokens=90, batch_size=7, verbose=True):
	ensure_code_tokenized_cache()
	tok_data = torch.load(CODE_TOK_CACHE_PATH, weights_only=True)
	prompt_ids_list = tok_data["prompt_ids_list"]
	eos_id = tok_data["eos_id"]
	pad_id = tok_data["pad_id"]

	order = sorted(range(len(problems)), key=lambda i: len(prompt_ids_list[i]))
	runner = Qwen3FastRunner(sd)

	t0 = time.time()
	ordered_gen_ids = []
	num_batches = (len(problems) + batch_size - 1) // batch_size
	for b_idx, start in enumerate(range(0, len(problems), batch_size)):
	end = min(start + batch_size, len(problems))
	batch_indices = order[start:end]
	batch_prompts = [prompt_ids_list[i] for i in batch_indices]
	tb0 = time.time()
	gen_ids = runner.generate_batch(batch_prompts, eos_id, pad_id, max_new_tokens=max_new_tokens)
	ordered_gen_ids.extend(gen_ids)
	if verbose:
	avg_t = sum(len(g) for g in gen_ids) / len(gen_ids)
	print(f" Batch {b_idx+1}/{num_batches} ({end-start} tasks, avg {avg_t:.1f} toks) in {time.time()-tb0:.1f}s", flush=True)
	elapsed = time.time() - t0

	all_gen_ids = [None] * len(problems)
	for orig_idx, g_ids in zip(order, ordered_gen_ids):
	all_gen_ids[orig_idx] = g_ids

	del runner
	gc.collect()

	responses = decode_in_subprocess(all_gen_ids)
	results = []
	passed = 0
	for prob, resp, g_ids in zip(problems, responses, all_gen_ids):
	ok, detail = verify_code_solution(prob, resp)
	if ok:
	passed += 1
	results.append({
	"id": prob["id"],
	"category": prob["category"],
	"prompt": prob["prompt"],
	"passed": ok,
	"detail": detail,
	"num_tokens": len(g_ids),
	"response": resp,
	})

	summary = {
	"total": len(problems),
	"passed": passed,
	"accuracy": round(100.0 * passed / len(problems), 2),
	"avg_tokens": round(sum(r["num_tokens"] for r in results) / len(results), 1),
	"elapsed_sec": round(elapsed, 2),
	"results": results,
	}
	return summary


	if __name__ == "__main__":
	print("Evaluating Baseline Qwen3-0.6B on 28-task Executable Code Benchmark...", flush=True)
	sd = load_file(os.path.join(MODEL_DIR, "model.safetensors"), device="cpu")
	summary = evaluate_code_state_dict(sd, CODE_BENCHMARK_PROBLEMS, max_new_tokens=90, batch_size=7, verbose=True)
	with open("/home/user/code_baseline_results.json", "w") as f:
	json.dump(summary, f, indent=2)
	print(f"\n==========================================", flush=True)
	print(f"CODE BASELINE QWEN3-0.6B: {summary['passed']}/{summary['total']} ({summary['accuracy']}%) \| Avg Toks: {summary['avg_tokens']} \| {summary['elapsed_sec']}s", flush=True)
	print(f"==========================================", flush=True)
	for r in summary["results"]:
	status = "PASS" if r["passed"] else "FAIL"
	print(f"[{status:4s}] {r['id']:8s} ({r['category']:16s}) \| Toks: {r['num_tokens']:3d} \| {r['detail']}", flush=True)