Text Generation
PEFT
Safetensors
English
qlora
lora
structured-output

Qwen3-4B-Structured-Conversion-LoRA-v3

This repository provides a LoRA adapter fine-tuned from Qwen/Qwen3-4B-Instruct-2507 using QLoRA (4-bit, Unsloth).

This repository contains LoRA adapter weights only. The base model must be loaded separately.

Training Objective

This adapter is trained to improve structured output accuracy (JSON / YAML / XML / TOML / CSV).

Loss is applied only to the final assistant output. Chain-of-Thought reasoning is masked during training.

Training Configuration

  • Base model: Qwen/Qwen3-4B-Instruct-2507
  • Method: QLoRA (4-bit)
  • Max sequence length: 1024
  • Epochs: 2
  • Learning rate: 8e-06
  • LoRA: r=64, alpha=128

Training Data

This model was trained on a balanced mixture of:

Usage

from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel
import torch

base = "Qwen/Qwen3-4B-Instruct-2507"
adapter = "AshleyQu0311/Qwen3-4B-Structured-Conversion-LoRA-v3"

tokenizer = AutoTokenizer.from_pretrained(base)
model = AutoModelForCausalLM.from_pretrained(
    base,
    torch_dtype=torch.float16,
    device_map="auto",
)
model = PeftModel.from_pretrained(model, adapter)

License and Terms

Users must comply with:

  • The base model license

  • The license of each dataset used (MIT)

Dataset License: MIT License. This dataset is used and distributed under the terms of the MIT License. Compliance: Users must comply with the MIT license (including copyright notice) and the base model's original terms of use.

Downloads last month
10
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Model tree for AshleyQu0311/Qwen3-4B-Structured-Conversion-LoRA-v3

Adapter
(5,861)
this model

Datasets used to train AshleyQu0311/Qwen3-4B-Structured-Conversion-LoRA-v3