---
language:
- en
license: mit
library_name: transformers
tags:
- code
- coding
- code-generation
- distillation
- gpt2-xl
- causal-lm
- qwen
base_model: gpt2-xl
pipeline_tag: text-generation
---

# Aura-1-Coding (Standalone FP16)

**Aura-1-Coding** is a lightweight, distilled causal language model designed for Python code generation and algorithmic problem solving. It adapts **GPT-2 XL (1.5B parameters)** through live on-policy knowledge distillation from **Qwen2.5-3B-Instruct**, fusing modern reasoning and coding patterns into a compact, standalone footprint.

This repository hosts the **fully merged standalone FP16 weights** (`safetensors`). No PEFT or bitsandbytes runtime dependencies are required for deployment.

---

## Model Details

- **Architecture:** Causal Transformer (Decoder-Only)
- **Base Model:** `gpt2-xl` (1.5B parameters)
- **Teacher Model:** `Qwen/Qwen2.5-3B-Instruct`
- **Distillation Method:** Live On-Policy Sequence Distillation via QLoRA (NF4) merged into FP16
- **Precision:** Float16 (`torch.float16`)
- **Context Window:** Up to 1024 tokens (optimized for prompts/solutions $\le 512$ tokens)
- **Primary Domain:** Python algorithms, data structures, and utility functions

---

## Prompt Format

Aura-1-Coding is trained on structured prompt-solution pairs. For optimal performance, structure your input as follows:

```text
Problem: <Your description or programming question task>
Solution: