Image-Text-to-Text
PEFT
Safetensors
qlora
lora
unsloth
screenshot-to-code
html
tailwind

Screenshot to HTML: QLoRA adapter for Qwen3-VL-2B-Instruct (ECNG6102 CW4)

A LoRA adapter (rank 16) that turns a web page screenshot into one HTML file styled with Tailwind CSS 2.2.19. It was trained for coursework (ECNG6102, CW4). The repo also holds the data splits, saved predictions, scores and figures, so the coursework notebook can rerun every evaluation without retraining.

Results (100 pages per test set)

Model Synthetic composite Real composite Synthetic complete HTML
B0 blank page 0.275 0.262 n/a
B1 PCA + nearest neighbour 0.627 0.401 100%
B2 zero-shot Qwen3-VL-2B 0.345 0.373 4%
This adapter (step 664) 0.896 0.604 98%

The composite score is the mean of visible-text, HTML tag-structure and colour-histogram similarity between the rendered reference page and the rendered prediction. Every improvement over a baseline has a paired bootstrap 95% confidence interval that excludes zero. "Synthetic" means held-out WebSight pages from business types never seen in training; "real" means Design2Code pages.

Contents

Path What it is
adapter_best/ Selected adapter (checkpoint at step 664 of 1,000)
adapter_final/ Adapter after the full epoch
config.json All training and data settings (seed 42)
manifest.json, real_manifest.json Pages in each split
images.tar Screenshots for the splits, resized to 1,024 px wide
results/ Per-page predictions and scores, summaries, confidence intervals, training log, figures
requirements_frozen.txt Package versions used for training
train_info.json Hardware and duration of the training run

Training

  • Base model: unsloth/Qwen3-VL-2B-Instruct-unsloth-bnb-4bit (4-bit QLoRA)
  • LoRA r=16, alpha=16, dropout 0, on the language model's attention and MLP layers; vision layers frozen
  • 8,000 WebSight v0.2 pages, 1 epoch, effective batch 8, learning rate 2e-4 with a cosine schedule, 8-bit AdamW
  • Images at 1,024 px wide, sequences up to 2,048 tokens
  • One NVIDIA RTX 4090; training took 22 minutes

Usage

Load the base model with Unsloth, add LoRA adapters with the settings above, then load adapter_best/adapter_model.safetensors. The coursework notebook does this when TRAIN = False and ARTIFACT_REPO points to this repo. Prompt: "Generate one complete HTML file using Tailwind CSS that reproduces this web page screenshot. Output only the HTML." Use greedy decoding with up to 1,024 new tokens.

Intended use and limitations

  • Intended use: a first-draft HTML file for a developer to review and edit. It is not meant for copying another organisation's design, or for shipping code to production without review.
  • Length limit: it was trained on short synthetic pages. Real pages are about 50 times longer, so output is cut off at 1,024 tokens: only 45% of real-page outputs are complete.
  • Known failures: lower page sections (forms, footers) go missing; it invents text when a screenshot has little content; layouts are simplified to Tailwind defaults.
  • Safety: render outputs only in a sandbox with scripts disabled.
  • Privacy: the model copies text it reads in a screenshot, so do not upload screenshots that contain personal data.

Data and licences

  • WebSight v0.2, HuggingFaceM4, CC-BY-4.0. Laurençon, Tronchon and Sanh (2024), arXiv:2403.09029.
  • Design2Code, SALT-NLP, ODC-By. Si et al. (2025), arXiv:2403.03163.
  • Base model: Qwen3-VL-2B-Instruct, Apache-2.0.

The adapter weights and results are released under CC-BY-4.0. The included data keeps its original licence.

Downloads last month
-
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Model tree for sramsingh-env/cw4-screenshot2html-v0.2

Adapter
(130)
this model

Datasets used to train sramsingh-env/cw4-screenshot2html-v0.2

Papers for sramsingh-env/cw4-screenshot2html-v0.2