LFM3-VL-2B-3D-Coder

Supervised specialisation of LiquidAI/LFM3-VL-2B for generating executable Three.js 3D scenes and GLB-exportable assets from natural-language instructions.

This is an independent, exploratory fine-tune — a quasi-reproduction and extension of ccore/Qwen3.5-4B-3D-Coder, targeting a model half the size (1.92B vs 4.6B parameters) but with higher reported benchmark performance.

What changed relative to ccore's work

	ccore (Qwen3.5-4B-3D-Coder)	This model
Base	Qwen3.5-4B	LFM3-VL-2B
Adaptation	Full fine-tune (+ short LoRA stage)	High-rank QLoRA from the base model (r=64, α=128)
Quantisation	NF4 only for the short LoRA stage	NF4 4-bit QLoRA throughout
Objective	cross-entropy SFT	DFT (Dynamic Fine-Tuning, ICLR 2026) — one-line loss reweighting
Framework	custom full-FT loop	Unsloth + TRL 1.17.1 SFTTrainer
RL	none	none
Data	3,933 scenes	all 3,933 scenes + 1,500 GLB-asset examples

DFT is implemented natively by TRL 1.17.1 via SFTConfig(loss_type="dft"), which computes per_token_loss * log_probs.exp().detach() using a fused/chunked LM head (no full logit materialisation). No RL stage of any kind was used.

Training configuration

• Base: LiquidAI/LFM3-VL-2B (ChatML, <think>…</think> reasoning, 128K context), transformers 5.22.1, trust_remote_code=True, attn_implementation="sdpa".
• QLoRA: bitsandbytes NF4 + double quant, bf16 compute. LoRA r=64, α=128, dropout 0, bias none. Target modules (language model only; vision encoder frozen): q_proj, k_proj, v_proj, o_proj, gate_proj, in_proj, out_proj, w1, w2, w3 — including Liquid AI's per-head attention gate_proj. 65,874,944 trainable params (3.32%).
• Loss: DFT, assistant-only (labels pre-masked; ChatML {% generation %} markers).
• Data: ccore/ThreeJS-3D-Scenes (default, all 3,933 train / 213 validation) plus 1,500 randomly sampled (seed=42) examples from ccore/ThreeJS-3D-GLB-Assets (conversations, 3,501 train / 417 validation). Real validation splits used for eval. max_length=None — no truncation; every example trained complete (max 66,831 tokens).
• Batch: per_device_train_batch_size=1, gradient_accumulation_steps=4 → effective batch size 4. Micro-batch 1 (instead of 2) was required to fit full-length sequences on a single L4; effective batch size is unchanged.
• Optimiser: AdamW fused, peak LR 1e-4, cosine decay, 59 warmup steps, weight decay 0.01, β=(0.9, 0.999), ε=1e-8, bf16, 2 epochs.
• Memory/compute: Unsloth "unsloth" gradient checkpointing on one NVIDIA L4 (24 GB) via Google Colab; ~15 GB peak VRAM at 67K tokens; ~1,300 tok/s.
• Monitoring: TensorBoard event files, validation loss every 200 steps, generated HTML samples every 400 steps, LoRA checkpoints pushed to this repo every 100 steps.

Repository layout

Path	Contents
merged/	Final LoRA merged into the bf16 base weights (Transformers, trust_remote_code=True)
final_adapter/	Final LoRA adapter
checkpoints/step-*/	Interim checkpoints every 100 steps
evals/baseline/	Samples from the original, untrained checkpoint
evals/step-*/	Generated Three.js HTML + results at eval steps
gguf/	4-bit Q4_K_M GGUF
training_config.json	Full machine-readable config
TECHNICAL_REPORT.md	Technical report

Usage

Use the same system prompt as the training data:

Use only the Three.js core ES module from
https://cdnjs.cloudflare.com/ajax/libs/three.js/0.160.0/three.module.min.js.
Do not import addons or legacy examples/js paths;
implement any camera controls directly inside the HTML.

For GLB assets, the dataset's second system prompt is used (Godot / Export GLB button).

Recommended sampling — the LFM3-VL-2B model-card defaults, used for all evaluations in this repo:

output_ids = model.generate(
    **inputs, do_sample=True,
    temperature=0.1, top_p=1.0, top_k=50, min_p=0.0,
    repetition_penalty=1.1, max_new_tokens=16384,
)

For comparability with ccore's study, a secondary regime (temperature=0.6, top_p=0.95, top_k=20, repetition_penalty=1.0) is also reported. Allow a large output budget — training examples reach 66K tokens, so in-training evals use max_new_tokens=24576; an 8K cap truncates reasoning before the HTML is emitted. Keep <think> reasoning enabled. During training, evals load the base in NF4 4-bit (GPU is shared); the final evaluation uses the unquantised bf16 merged model.

Limitations

• Exploratory study with synthetic data; evaluation is qualitative (rendered scenes), not a uniform first-attempt benchmark.
• Long generations can loop in <think> before emitting HTML; retries are sometimes needed.
• LFM3-VL-2B is not recommended by its creators for coding; this fine-tune specialises it for one narrow code-generation domain.
• The GGUF requires llama.cpp ≥ b11938 for Lfm3VlForConditionalGeneration support.

Attribution

Base model: Liquid AI, LFM Open License v1.0. Datasets: ccore (MIT / Apache-2.0). DFT: Wu et al., On the Generalization of SFT: A Reinforcement Learning Perspective with Reward Rectification, ICLR 2026 (arXiv:2508.05629). Framework: Unsloth, TRL, PEFT, bitsandbytes. Technical report: ccore's Qwen3.5-4B-3D-Coder study.