File size: 2,424 Bytes
7e6ab14 64a3d35 7e6ab14 64a3d35 7e6ab14 64a3d35 7e6ab14 64a3d35 7e6ab14 64a3d35 7e6ab14 64a3d35 7e6ab14 64a3d35 7e6ab14 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 | ---
license: apache-2.0
library_name: gguf
base_model: Qwen/Qwen3-14B
tags:
- code
- qwen3
- gguf
- fine-tuned
- conversational
model-index:
- name: Codeas Model
results: []
pipeline_tag: text-generation
language:
- en
---
# Codeas Model
A fine-tuned **Qwen3-14B** model optimized for code generation and reasoning tasks. Available in GGUF Q6_K format for efficient local inference.
## Model Details
| | |
|---|---|
| **Base Model** | Qwen3-14B |
| **Parameters** | ~15B |
| **Architecture** | Qwen3 (GQA, RoPE) |
| **Context Length** | 40,960 tokens |
| **Precision** | BF16 (original), Q6_K (GGUF) |
| **License** | Apache 2.0 |
## Architecture
- 40 transformer blocks
- 40 attention heads, 8 KV heads (Grouped Query Attention)
- 5,120 hidden size / 17,408 FFN size
- RoPE with 1M frequency base
- SiLU activation
- 151,936 vocab size (GPT-2 tokenizer, Qwen2 pre-tokenizer)
## Capabilities
- Chain-of-thought reasoning via `<think>` blocks
- Tool/function calling via `<tool_call>` format
- Thinking mode can be toggled on/off per request
## GGUF Quantizations
| File | Quant | Size | Quality |
|------|-------|------|---------|
| `codeas-model-Q6_K.gguf` | Q6_K | 12.1 GB | Near-lossless |
## Usage
### llama.cpp
```bash
./llama-cli -m codeas-model-Q6_K.gguf -p "Write a Python function to merge two sorted lists" -n 512
```
### Ollama
Create a `Modelfile` with the following content:
```dockerfile
FROM ./codeas-model-Q6_K.gguf
PARAMETER temperature 0.6
PARAMETER top_p 0.95
PARAMETER top_k 20
TEMPLATE """{{- if .System }}<|im_start|>system
{{ .System }}<|im_end|>
{{ end }}<|im_start|>user
{{ .Prompt }}<|im_end|>
<|im_start|>assistant
"""
SYSTEM "You are Codeas, a helpful coding assistant."
```
Then run:
```bash
ollama create codeas -f Modelfile
ollama run codeas
```
## Hardware Requirements
| Format | VRAM / RAM |
|--------|-----------|
| Q6_K GGUF | ~14 GB |
## Training
| | |
|---|---|
| **Method** | Full fine-tune (no LoRA) |
| **Framework** | Axolotl 0.13.0 + Transformers 4.55.4 |
| **Hardware** | 8x GPU (FSDP) |
| **Optimizer** | AdamW (fused) |
| **LR Schedule** | Cosine, 1e-5 peak |
| **Sequence Length** | 8,192 |
| **Batch Size** | 24 (3 per device) |
| **Epochs** | 3 |
| **Precision** | BF16 + TF32 |
| **Techniques** | Flash Attention, Sample Packing, Gradient Checkpointing, Activation Offloading |
## Sampling Defaults
```
temperature: 0.6
top_p: 0.95
top_k: 20
``` |