Jay2003Bhatt/alpha-coder-14b

Qwen Coder Lab

Starter lab for building a specialized coding assistant on top of:

Qwen/Qwen3-Coder-30B-A3B-Instruct

This project is set up for three phases:

1. Run the base model locally or on a GPU machine.
2. Prepare high-quality coding instruction data.
3. Fine-tune with QLoRA, then quantize/deploy.

Hardware Target

For the 30B-A3B model, practical setups are:

• Inference: 24 GB+ VRAM with 4-bit quantization, or a larger unified-memory machine.
• QLoRA fine-tuning: 24-48 GB+ VRAM depending on sequence length, batch size, and optimizer.
• CPU-only: possible only with GGUF/llama.cpp style inference, but slow.

If your current machine cannot run the model, use the same project on a rented GPU instance.

MacBook M-Series Path

For a MacBook with 24 GB unified memory, 1 TB SSD, and a 10-core GPU:

• Run the 30B model through a GGUF runtime such as Ollama, llama.cpp, or LM Studio.
• Prefer Q3 or Q4 quantization for local inference. Higher precision will likely exceed comfortable memory limits.
• Keep context length modest at first, such as 4K-16K tokens. Long context can consume a lot of memory.
• Do not plan to QLoRA fine-tune the 30B model locally. Use a rented NVIDIA GPU for that phase.
• For local fine-tuning experiments, use a smaller model first, such as a 7B coder model.

Recommended local workflow:

1. Use the Mac to test prompts, collect examples, validate datasets, and run quantized inference.
2. Use a cloud GPU to fine-tune Qwen3-Coder-30B-A3B-Instruct with QLoRA.
3. Bring the resulting adapter or merged/quantized model back to the Mac for local inference.

Start here: docs/mac-local-run.md.

Training and usage roadmap: docs/train-and-use-roadmap.md.

Specialist dataset strategy: docs/dataset-strategy.md.

Baseline findings: docs/baseline-findings.md.

Light deployment strategy: docs/light-deployment-strategy.md.

Runtime system prompt: prompts/product_engineer_system.md.

Complex benchmark findings: docs/complex-benchmark-findings.md.

Current performance report: docs/current-performance-report.md.

Install

cd qwen-coder-lab
python3 -m venv .venv
source .venv/bin/activate
pip install -r requirements.txt

If macOS reports missing Command Line Tools when running python3, install them with xcode-select --install, or use a uv Python environment instead.

Run Chat Inference

python scripts/run_chat.py \
  --model Qwen/Qwen3-Coder-30B-A3B-Instruct \
  --load-in-4bit

Then type coding requests interactively.

Dataset Format

Training examples use JSONL:

{"instruction":"Fix this Python bug.","input":"def add(a,b): return a-b","output":"def add(a, b):\n    return a + b"}

Validate examples:

python scripts/prepare_dataset.py \
  --input data/examples.jsonl \
  --output data/prepared.jsonl

QLoRA Fine-Tuning

Start with the config in configs/qwen3_coder_30b_lora.yaml.

python scripts/train_lora.py \
  --config configs/qwen3_coder_30b_lora.yaml

The default config is intentionally conservative. Increase sequence length and batch size only after a small training run succeeds.

What To Fine-Tune On

Best data for a coding assistant:

• issue -> patch examples
• failing test -> fix examples
• compiler/type error -> corrected code
• code review comment -> improved diff
• refactoring requests
• multi-language explanations plus executable solutions
• repo-specific style examples

Avoid low-quality scraped code. For coding models, clean task/answer pairs beat giant noisy dumps.