---
license: apache-2.0
base_model: Qwen/Qwen3.5-0.8B
library_name: mlx
tags:
  - lora
  - bend
  - program-synthesis
  - in-weights-execution
---

# QwenBend: LoRA adapter

**Qwen3.5-0.8B writes a Bend program. Then the same model type-checks it and runs it inside its own
forward pass.** Code, design notes and the 15-second demo: **https://github.com/voidash/qwenbend**

This repository holds the **LoRA adapter** that teaches Qwen3.5-0.8B to write BendLite, a subset
of [Bend2](https://github.com/HigherOrderCO/Bend).

The engine half of QwenBend is not trained. Its weights are set by construction, from code in the
GitHub repo:
- a 16-register CPU with 8,192 words of memory held in Gated DeltaNet heads;
- the CPU applies Bend2's static rules (types, affine use, termination) and executes the program.

`python bendvm/qwenbend.py "your task"` there builds the full 3.97B model in memory in about 15 s
from Qwen3.5-0.8B and this adapter:
- 0.753B nonzero weights are Qwen's;
- 52.9M are the engine's;
- the rest is zero padding from widening the hidden size to 4096.

## The adapter

- `adapter/adapters.safetensors` + `adapter/adapter_config.json`: **mlx_lm** LoRA format.
  - rank 8, scale 20, the last 16 layers;
  - trained with lr 1e-4, batch 2;
  - the 400-step checkpoint was best by validation loss (training to 1,200 steps overfit).
- `qwen_fused.py` in the GitHub repo fuses it into the HF weights in torch: W += 20·(A·B)ᵀ.
- sha256 of `adapters.safetensors`: `4577f177fdaabd10ae0f920f71a93d8f89603236b05484822bf0393e4caf315c`.

## Measured

On 200 test items: greedy, each program checked and run with real Bend2.

| model | valid | correct | in-distribution correct | held-out families correct |
|---|---:|---:|---:|---:|
| Qwen3.5-0.8B (no adapter) | 0% | 0% | 0% | 0% |
| **+ this adapter** | 58% | **42%** | **80%** | 4% |

End to end in the merged model (Apple M4 Pro, MPS, bf16): **5/5** programs correct, and every
execution trace equals the reference interpreter's token for token. About 20 tokens/s.

## Limits

The writer is a 0.8B model: it often writes the wrong program for unfamiliar tasks. The engine
guarantees the answer is exactly what the program computes, not that the program fits the question.

License: Apache-2.0 (same as the base model). Built at [Ampixa](https://ampixa.com).