wesjos/Qwen3.8-27B-HTML

Qwen3.8-27B · HTML

A fine-tuned Qwen3.8-27B that turns a prompt into an .ai-view page instead of a wall of Markdown.

_{27.78B params  ·  BF16  ·  18 shards  ·  55.6 GB  ·  Apache-2.0}

---

📦 What this repo is

A fine-tune of Qwen3.8-27B, distributed as full BF16 weights so it loads with a plain from_pretrained — no adapter to mount. The same tune is also published as a 42 MB LoRA adapter if you prefer to attach it to the base.

Release	Size	Use it when
This repo — fine-tuned, full BF16	55.6 GB (18 shards)	You want from_pretrained with no adapter step
Adapter	42 MB	You have the base cached, or you want the bundled WebUI
GGUF adapter	21 MB	llama.cpp / llama-server, mounted at runtime

⚠️ The trigger prompt is not baked into this repo's chat template. This repo ships the generic Qwen3.8 template; the Mario-html system prompt exists only in the adapter repo's chat_template.jinja. Pass it explicitly, as shown below, or the model falls back to ordinary 27B behaviour (long Markdown answers).

---

🖼 Showcase

_{28 s screen recording · .ai-view pages streaming in — cards, callouts, a comparison table, inline SVG}

OpenWebUI测试

_{12 s screen recording · the same tune running inside OpenWebUI (model chip reads qwen-3.8-27b): the Mario-html persona picks up the English instruction, a run_command tool step renders the sandbox spec as an .ai-view card grid, and “is qwen4 coming?” comes back as a rendered page — short-answer callout, timeline cards, warning callout, bottom line.}

_{“what does sigmoid look like” → formula block, an SVG plot with asymptotes and the inflection point, three property cards, a reference table}

_{“can you show me what a sloth looks like.” → thinking panel, an SVG sloth hanging from a branch, two fact cards, a “Did you know?” callout}

---

✨ What it does

The trigger is one system prompt:

You are Mario-html. You are a helpful assistant who generates nice visual content.

Output convention (from the training data; the reference WebUI keys off these class names):

<div class="ai-view">
  <h2>Title</h2>
  <div class="summary"><p class="lead">…</p></div>
  <div class="card">…</div>
  <table class="modern-table">…</table>
</div>

The chain of thought is kept intact — it was masked out of the loss, so the tune changes how it writes, not how it thinks.

---

🚀 Quickstart

transformers

import torch
from transformers import AutoProcessor, AutoModelForImageTextToText

MODEL = "wesjos/Qwen3.8-27B-HTML"
SYSTEM = "You are Mario-html. You are a helpful assistant who generates nice visual content."

processor = AutoProcessor.from_pretrained(MODEL)
model = AutoModelForImageTextToText.from_pretrained(
    MODEL, dtype=torch.bfloat16, device_map="auto"
)

messages = [
    {"role": "system", "content": SYSTEM},   # required here: not baked into the chat template
    {"role": "user", "content": "Build a page introducing black holes"},
]
text = processor.apply_chat_template(
    messages, tokenize=False, add_generation_prompt=True,
    enable_thinking=True, reasoning_effort="medium",
)
inputs = processor(text=text, return_tensors="pt").to(model.device)
out = model.generate(
    **inputs, max_new_tokens=4096, do_sample=True,
    temperature=1.0, top_p=0.95, top_k=20, min_p=0.0,
)
print(processor.decode(out[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True))

4-bit (smaller cards)

from transformers import BitsAndBytesConfig

bnb = BitsAndBytesConfig(
    load_in_4bit=True, bnb_4bit_quant_type="nf4",
    bnb_4bit_compute_dtype=torch.bfloat16, bnb_4bit_use_double_quant=True,
)
model = AutoModelForImageTextToText.from_pretrained(
    MODEL, quantization_config=bnb, device_map="auto"
)

Download

hf download wesjos/Qwen3.8-27B-HTML --local-dir Qwen3.8-27B-HTML

WebUI

The FastAPI + single-page frontend that renders .ai-view live ships with the adapter repo. It speaks the OpenAI /v1/chat/completions protocol (backend_mode: "api"), so serve this fine-tune behind any OpenAI-compatible server that supports Qwen3.5 and set API_URL / MODEL_ALIAS to that endpoint.

llama.cpp

This repo ships safetensors only. For llama.cpp, mount the 21 MB adapter at runtime:

llama-server -m Qwen3.8-27B-UD-Q4_K_M.gguf \
  --lora mario-html-27b-lora-f16.gguf \
  --mmproj mmproj-F16.gguf \
  -c 131072 -ngl 99 -fa on --jinja --port 18080

---

🎛 Sampling

Situation	Setting
Normal page	temperature 1.0 / top_p 0.95 / top_k 20 / min_p 0
Thinking length	reasoning_effort="medium" — low is shorter; xhigh burns 20k+ tokens and truncates the page
HTML only, no reasoning	enable_thinking=False — under ~90 tokens of thinking, but the format occasionally drifts
Context	131,072 recommended (xhigh eats 8k–10k tokens per turn), max_new_tokens 16384

_{The template in this repo accepts xhigh / medium / low (high is mapped to xhigh) and defaults to xhigh. For this tune, override it.}

---

📊 Measured

Prompt (no format instructions)	Result
Explain how neural networks learn, with a visual summary and a comparison table.	.ai-view + 5 cards + 3 tables
Build a page introducing black holes	.ai-view + 13 cards + 2 tables + 7 h2

---

⚙️ Training setup

Item	Value
Base	Qwen/Qwen3.8-27B (QLoRA training in 4-bit nf4)
Method	QLoRA, r=16 / alpha=32 / dropout=0
Target modules	q_proj, k_proj, v_proj, o_proj of the 16 full-attention layers
Trainable params	≈ 10.5M (0.04% of 27.78B), 42 MB adapter
Release format	full BF16 weights, 18 shards, 55.6 GB (trained as a QLoRA adapter)

---

💡 Notes

• The system prompt is a hard trigger. Without it this repo behaves like the base model: long Markdown, no .ai-view blocks.
• The tune learned form, not facts — facts and arithmetic still come from the base model.
• Training ran against the 4-bit base; this release contains full-precision weights. If you need the exact adapter behaviour, mount the LoRA adapter on Qwen/Qwen3.8-27B instead.
• Image input still works — the vision tower is untouched — but this adapter was trained on text only; it does not make the model better at looking at screenshots.

⚠️ Limitations

• Complex nesting and interactive JavaScript are out of reach for a 10.5M-parameter attention-only tune.
• Keep ctx ≥ 32,768. xhigh alone can consume 8k–10k tokens, and a tight context slices the page mid-sentence (finish_reason=length).
• Training data is mostly English — Chinese prompts work, layout stability is slightly lower.
• Markdown is not suppressed at the token level; a page can still contain a stray Markdown block alongside the HTML.

---

📁 Files

File	What it is
model-00001-of-00018.safetensors … model-00018-of-00018.safetensors	BF16 weights, 55.6 GB total
model.safetensors.index.json	shard map (27,781,427,952 parameters)
config.json	Qwen3_5ForConditionalGeneration, 64 layers, 262,144 max positions
generation_config.json	generation defaults
chat_template.jinja	generic Qwen3.8 template — no Mario-html trigger
tokenizer.json · tokenizer_config.json	tokenizer, 248,320 vocab
processor_config.json	image / video preprocessing
demos/	screenshots, the 28 s recording and the 12 s OpenWebUI clip used above
LICENSE	Apache-2.0

---

📜 License & attribution

Apache-2.0, inherited from Qwen/Qwen3.8-27B. Qwen is a trademark of its respective owner; this is an unofficial community tune and is not affiliated with or endorsed by the Qwen team.