| --- |
| language: |
| - en |
| license: mit |
| library_name: transformers |
| tags: |
| - code |
| - coding |
| - code-generation |
| - distillation |
| - gpt2-xl |
| - causal-lm |
| - qwen |
| base_model: gpt2-xl |
| pipeline_tag: text-generation |
| --- |
| |
| # Aura-1-Coding (Standalone FP16) |
|
|
| **Aura-1-Coding** is a lightweight, distilled causal language model designed for Python code generation and algorithmic problem solving. It adapts **GPT-2 XL (1.5B parameters)** through live on-policy knowledge distillation from **Qwen2.5-3B-Instruct**, fusing modern reasoning and coding patterns into a compact, standalone footprint. |
|
|
| This repository hosts the **fully merged standalone FP16 weights** (`safetensors`). No PEFT or bitsandbytes runtime dependencies are required for deployment. |
|
|
| --- |
|
|
| ## Model Details |
|
|
| - **Architecture:** Causal Transformer (Decoder-Only) |
| - **Base Model:** `gpt2-xl` (1.5B parameters) |
| - **Teacher Model:** `Qwen/Qwen2.5-3B-Instruct` |
| - **Distillation Method:** Live On-Policy Sequence Distillation via QLoRA (NF4) merged into FP16 |
| - **Precision:** Float16 (`torch.float16`) |
| - **Context Window:** Up to 1024 tokens (optimized for prompts/solutions $\le 512$ tokens) |
| - **Primary Domain:** Python algorithms, data structures, and utility functions |
|
|
| --- |
|
|
| ## Prompt Format |
|
|
| Aura-1-Coding is trained on structured prompt-solution pairs. For optimal performance, structure your input as follows: |
|
|
| ```text |
| Problem: <Your description or programming question task> |
| Solution: |
