Vaingloryy/luau-agent-3b

luau-agent 3b

A 3B paramater coding model for Roblox and Luau, with light fine tuning from Nanbeige 4.2 3B. Quality can be inconsistent due to the fine tuning, plus small errors within datasets. (buffers are the most prevalent incorrect data currently) Highly suggested to use a more competent model as an orchestrator, for the reason above.

Training

• Base model: Nanbeige 4.2 3B
• Fine-tuned on around 100 million tokens of data due to budget constraints.
• Stages: continued pretraining on luau code, then ran sft with AdamW optimizer on both, RL not performed due to budget constraints, this is the SFT checkpoint.
• Method: BF16 LoRA (r 64, alpha 64) on attention output and MLP projections, merged back into the full weights
• Knowledge cutoff: October 11th 2026
• Time taken to train: ~20 hours
• Hardware: 1x RTX PRO 6000 Blackwell @ Google Colab

Files

| model-0000*-of-00002.safetensors | Full merged BF16 weights for Transformers |

| luau-agent-3b-Q8_0.gguf | Q8_0 quant for llama.cpp, LM Studio, etc. |

Example output from one turn:

type HitInfo = {
    hitPart: BasePart,
    hitModel: Model?,
    hitHumanoid: Humanoid?,
    hitPosition: Vector3?,
}

type HitboxConfig = {
    size: Vector3,
    position: Vector3,
    overlap: boolean,   -- true → detect parts inside the box, false → detect parts overlapping the box
    excludeList: { Instance }?,   -- parts to ignore (usually the hitbox itself)
}

type Hitbox = {
    config: HitboxConfig,
    overlapParams: OverlapParams,
}

local Hitbox = {}
Hitbox.__index = Hitbox

function Hitbox.new(config: HitboxConfig): Hitbox
    local self = setmetatable({}, Hitbox)

    self.config = config

    -- Default OverlapParams
    local overlapParams = OverlapParams.new()
    overlapParams.FilterType = Enum.RaycastFilterType.Exclude
    overlapParams.FilterDescendantsInstances = {}
    overlapParams.MaxParts = 64
    overlapParams.RespectCanCollide = true

    self.overlapParams = overlapParams

    return self
end

function Hitbox.setExclude(self: Hitbox, list: { Instance })
    self.overlapParams.FilterDescendantsInstances = list
end

function Hitbox.queryHitbox(self: Hitbox): {HitInfo}
    local size = self.config.size
    local pos = self.config.position

    local region = Region3.new(pos - size / 2, pos + size / 2)

    local parts = workspace:GetPartsInRegion3(region, self.overlapParams)

    local results: {HitInfo} = {}
    for _, part in parts do
        local model = part:FindFirstAncestorOfClass("Model")
        local humanoid = model and model:FindFirstChildOfClass("Humanoid")

        table.insert(results, {
            hitPart = part,
            hitModel = model,
            hitHumanoid = humanoid,
        })
    end
    return results
end

function Hitbox.queryOverlap(self: Hitbox): {HitInfo}
    local size = self.config.size
    local pos = self.config.position

    local parts = workspace:GetPartBoundsInBox(
        pos - size / 2,
        size,
        self.overlapParams
    )

    local results: {HitInfo} = {}
    for _, part in parts do
        local model = part:FindFirstAncestorOfClass("Model")
        local humanoid = model and model:FindFirstChildOfClass("Humanoid")

        table.insert(results, {
            hitPart = part,
            hitModel = model,
            hitHumanoid = humanoid,
        })
    end
    return results
end

function Hitbox.applyDamage(self: Hitbox, humanoid: Humanoid, amount: number, force: number)
    if humanoid then
        humanoid:TakeDamage(amount * force)
    end
end

return Hitbox

Setup

llama.cpp (recommended)

Needs a recent llama.cpp build with Nanbeige support.

# get the gguf
hf download Vaingloryy/luau-agent-3b luau-agent-3b-Q8_0.gguf --local-dir .

# run a server (OpenAI-compatible API on http://localhost:8080)
llama-server -m luau-agent-3b-Q8_0.gguf \
  -ngl 99 -c 16384 -np 1 -fa on --jinja \
  -ctk q8_0 -ctv q8_0 --spec-type ngram-mod \
  --temp 1.0 --top-p 0.95 --top-k 20

llama.cpp settings

Setting	Value	Why
-ngl	99	Put every layer on the GPU
-c	16384	Context it was trained on (long SFT rows go up to 16k)
-np	1	One slot, keeps all the context for one chat
-fa	on	Flash attention, less VRAM
--jinja	on	Uses the built-in chat template, needed for tool calls
-ctk / -ctv	q8_0	Quantized KV cache, roughly halves its memory
--spec-type	ngram-mod	Free speculative decoding, speeds up code edits that repeat text
--temp	1.0	Sampling settings from the base model
--top-p	0.95
--top-k	20

Q8_0 needs about 5 GB of VRAM plus the KV cache.

Transformers

The model uses custom code, so trust_remote_code=True is required.

from transformers import AutoModelForCausalLM, AutoTokenizer
import torch

repo = "Vaingloryy/luau-agent-3b"
tokenizer = AutoTokenizer.from_pretrained(repo, trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(repo, torch_dtype=torch.bfloat16, device_map="auto", trust_remote_code=True)

messages = [{"role": "user", "content": "Write a Luau module that tracks per-player cooldowns."}]
inputs = tokenizer.apply_chat_template(messages, add_generation_prompt=True, return_tensors="pt").to(model.device)
output = model.generate(inputs, max_new_tokens=1024, do_sample=True, temperature=1.0, top_p=0.95, top_k=20)
print(tokenizer.decode(output[0][inputs.shape[-1]:], skip_special_tokens=True))

Credits

Thanks to the people behind these datasets and Pinkstack especially for helping me start:

• BackpropBuff/Principia_SimpleMath_Integers_1m
• Pinkstack/RobloxAgent-v2
• khtsly/LuauCoder-1.0-Preview-SFT
• nvidia/Nemotron-SFT-Instruction-Following-Chat-v2
• nvidia/Nemotron-SFT-SWE-v3.5
• khtsly/roblox-luau-corpus
• TorpedoSoftware/the-luau-stack
• Pinkstack/LuauDev-instructions-SFT-full
• Pinkstack/2026-Roblox-Luaudev2-dataset-sft
• Pinkstack/oct2026-Roblox-Luaudev2-dataset-pretrain

And to the Nanbeige team for the base model.

License

Apache License 2.0, same as the base model. See LICENSE.

This is a modified version of Nanbeige 4.2 3B: the weights were changed by fine-tuning as described above. The original model's copyright and license notices are kept. Datasets keep their own licenses and terms.