| --- |
| license: openmdw-1.1 |
| language: |
| - en |
| pipeline_tag: text-generation |
| tags: |
| - preview |
| - edge |
| - code |
| - luau |
| datasets: |
| - Pinkstack/LuauDev-instructions-SFT-full |
| --- |
| # ⚠️ THIS IS A PREVIEW VARIANT OF LUAUDEV! DO NOT USE THIS MODEL FOR PRODUCTION. FOR PRODUCTION WAIT FOR THE OFFICIAL RELEASE |
|
|
|  |
|
|
| When luaudev officially releases, its readme will be more full with information. this is after one stage of sft and pre-training. The base model will be released along side the official instruct model. |
|
|
| This is the preview instruct variant. |
|
|
| ## pre-training info: |
| - Luaudev was pre-traiend on 21 billion unique tokens, at 4096 sequence length with packing enabled, with a knowledge cutoff date of march 2026 for code and web and for math may 2025. |
| - the model was pre-trained with fp8 matmuls with bf16 weights using the adamwfp8 optimizer and an a effective batch size of 288. All training was done on a single Nvidia RTX 4080 (overclocked) |
|
|
| ## post-training info: |
| - this luaudev variant was post-trained with sft on about 1.5 billion tokens of mixed luau specific code, math, q&a and so on. This model supports 4 reasoning efforts: disabled(no reasoning), low, medium, high. Note the model may loop with high reasoning more than normal. Luaudev also supports tool calling and agentic output and inputs yet for this preview the performance is not ideal. |
| - the model was post trained with an effective batch size of 32, and with a custom home-made ademamix based 2bit fp8 optimizer for real memory savings. |
| - post training was done with a sequence length of 32k, and thus that is the supported max sequence length by the model. |
| ## PLB benchmark results: |
| text: |
| Model|Publisher|Elo |
| |:---|:---:|:---:| |
| GPT OSS 20B **|OpenAI|__**75.4**__ 🏆| |
| LuauDev 177M Preview *|**Pinkstack**|56.1| |
| Qwen3.5 2B|Qwen|17.3| |
| LFM2.5 350M|Liquid AI|11.0| |
| Gemma 3 1B|Google|9.5| |
| |
| <small>.* luaudev supports reasoning efforts — set to disabled (reasoning off) for a fair comparison.</small> |
| |
| <small>.** gpt-oss-20b supports reasoning efforts — lowest available is low, so it was set to low.</small> |
|
|
| ## GGUF |
| https://huggingface.co/Pinkstack/Luaudev-177M-Preview-inst-GGUF |
|
|
| ### License: |
| Luaudev is released under openmdw 1.1 |
