---
base_model: unsloth/Llama-3.2-3B-Instruct-unsloth-bnb-4bit
library_name: unsloth
language:
  - en
license: llama3.2
tags:
  - llama
  - llama-3.2
  - gguf
  - unsloth
  - qlora
  - web-development
  - code
pipeline_tag: text-generation
---

# Flow Models — Web Dev (Llama 3.2 3B Instruct, Q4_K_M)

A QLoRA fine-tune of **Llama-3.2-3B-Instruct** on a web-development instruction dataset, exported to GGUF for use with `llama.cpp`, Ollama, LM Studio, or any GGUF-compatible runtime.

## Model details

| | |
|---|---|
| Base model | `unsloth/Llama-3.2-3B-Instruct-unsloth-bnb-4bit` |
| Fine-tuning | QLoRA (4-bit), Unsloth |
| Export | Merged and quantized to **Q4_K_M** (1.88 GB) |
| Context length | 2048 |
| Language | English |

## Training

| Hyperparameter | Value |
|---|---|
| LoRA rank / alpha | 16 / 16 |
| LoRA dropout | 0 |
| Target modules | `q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj` |
| Epochs | 3 |
| Batch size | 2 |
| Gradient accumulation | 4 |
| Learning rate | 2e-4 (linear schedule) |
| Warmup steps | 5 |
| Weight decay | 0.001 |
| Optimizer | `adamw_8bit` |
| Max sequence length | 2048 |
| Packing | enabled |
| Seed | 3407 |

Result: 87 steps, 1,376,556 tokens, training loss **1.20 → 0.39**.

## Dataset

`formatted_webdev_dataset.json` — 3,723 instruction/response examples covering HTML, CSS, JavaScript, Python, SQL, and general web development. Examples use the Alpaca prompt format and responses end with a `[COMPLETE]` marker.

Training text format:

```
Below is an instruction that describes a task, paired with an input that provides further context. Write a response that appropriately completes the request.

### Instruction:
<instruction>

### Input:
<input>

### Response:
<response>
[COMPLETE]
```

## Usage

### llama.cpp

```bash
llama-cli -m Llama-3.2-3B-Instruct.Q4_K_M.gguf \
  -p "### Instruction:\nWrite a JavaScript function to count the number of vowels in a string.\n\n### Input:\n\n\n### Response:\n" \
  -n 256 --temp 0.2 -st
```

### Ollama

```bash
ollama create flow-webdev -f Modelfile
ollama run flow-webdev
```

### Expected output style

```javascript
function countVowels(str) {
    let vowels = 'aeiou';
    let count = 0;
    for (let i = 0; i < str.length; i++) {
        if (vowels.includes(str[i].toLowerCase())) {
            count++;
        }
    }
    return count;
}
[COMPLETE]
```

## Limitations

- Trained on a relatively small, synthetic-style web-dev dataset; it can reproduce errors or outdated patterns from that data.
- Best results come from the Alpaca prompt format it was trained on, not free-form chat.
- The `[COMPLETE]` suffix is part of the learned response style.
- Not evaluated for correctness, safety, or production use. Review generated code before running it.

## License

Llama 3.2 Community License, inherited from the base model.