license: apache-2.0
base_model: Qwen/Qwen3.5-0.8B
library_name: mlx
tags:
- lora
- bend
- program-synthesis
- in-weights-execution
QwenBend: LoRA adapter
Qwen3.5-0.8B writes a Bend program. Then the same model type-checks it and runs it inside its own forward pass. Code, design notes and the 15-second demo: https://github.com/voidash/qwenbend
This repository holds the LoRA adapter that teaches Qwen3.5-0.8B to write BendLite, a subset of Bend2.
The engine half of QwenBend is not trained. Its weights are set by construction, from code in the GitHub repo:
- a 16-register CPU with 8,192 words of memory held in Gated DeltaNet heads;
- the CPU applies Bend2's static rules (types, affine use, termination) and executes the program.
python bendvm/qwenbend.py "your task" there builds the full 3.97B model in memory in about 15 s
from Qwen3.5-0.8B and this adapter:
- 0.753B nonzero weights are Qwen's;
- 52.9M are the engine's;
- the rest is zero padding from widening the hidden size to 4096.
The adapter
adapter/adapters.safetensors+adapter/adapter_config.json: mlx_lm LoRA format.- rank 8, scale 20, the last 16 layers;
- trained with lr 1e-4, batch 2;
- the 400-step checkpoint was best by validation loss (training to 1,200 steps overfit).
qwen_fused.pyin the GitHub repo fuses it into the HF weights in torch: W += 20·(A·B)ᵀ.- sha256 of
adapters.safetensors:4577f177fdaabd10ae0f920f71a93d8f89603236b05484822bf0393e4caf315c.
Measured
On 200 test items: greedy, each program checked and run with real Bend2.
| model | valid | correct | in-distribution correct | held-out families correct |
|---|---|---|---|---|
| Qwen3.5-0.8B (no adapter) | 0% | 0% | 0% | 0% |
| + this adapter | 58% | 42% | 80% | 4% |
End to end in the merged model (Apple M4 Pro, MPS, bf16): 5/5 programs correct, and every execution trace equals the reference interpreter's token for token. About 20 tokens/s.
Limits
The writer is a 0.8B model: it often writes the wrong program for unfamiliar tasks. The engine guarantees the answer is exactly what the program computes, not that the program fits the question.
License: Apache-2.0 (same as the base model). Built at Ampixa.