PaperQwen3.5-4B

A fine-tune of Qwen/Qwen3.5-4B that rewrites dense, hard-to-read research prose into clear writing of the kind expert researchers produce, while keeping every fact, number, citation and LaTeX command.

It works on LaTeX or plain-text paragraphs from any scientific field and also turns bullet notes into paper prose.

Usage

Non-thinking mode, one paragraph (or a few) per request.

vllm serve Thermostatic/PaperQwen3.5-4B --served-model-name paperqwen --max-model-len 8192

from openai import OpenAI
client = OpenAI(base_url="http://localhost:8000/v1", api_key="x")
draft = r"""LoRA on the speech encoder instead of the LLM decoder cuts WER on new sentences by 3.7 points ..."""
r = client.chat.completions.create(
    model="paperqwen",
    messages=[{"role": "user", "content": "Rewrite this passage from my paper so it is clear and easy to read. "
               "Keep every fact, number, citation and LaTeX command.\n\n" + draft}],
    temperature=0.3, top_p=0.9,
    extra_body={"chat_template_kwargs": {"enable_thinking": False}},
)
print(r.choices[0].message.content)

Other instructions it was trained on include "Turn these notes into a polished paragraph for the Methods section" and "Make this digestible for a reader outside my subfield, while keeping all the technical content and citations."

Always check the output against your draft. A simple safeguard is to compare the sets of numbers, \cite{} keys and \ref{} labels between input and output, and to keep your original paragraph when they differ.

How it was trained

1. Continued pre-training (LoRA r=256, all linear layers, ~29M tokens over 2 epochs) on the LaTeX source of 1,349 machine-learning, NLP and speech papers that passed two filters: a readability score (sentence length, parenthetical, acronym and number density, function-word ratio) and a grammar/clarity rating. The pool came from best-paper award winners and the most-cited papers per venue and year. Distill.pub and transformer-circuits.pub articles were included and upweighted.
2. Supervised fine-tuning (LoRA r=128, merged) on ~40k examples, in two self-training rounds (this is round 2):
   • Human-written targets (12k): clear, well-written paragraphs from ~5,000 papers across ML, NLP, speech, physics, biology, neuroscience, medicine, economics, psychology, earth science, computer vision and HCI (arXiv and Europe PMC open-access journals). A local model turned each into a bad draft (compressed, number-cluttered, awkward, notes, disorganized, or fluent but dense). It was told to keep every fact, and pairs whose numbers, citations or refs changed were discarded.
   • Real author revisions (1.1k): an author's draft paragraph paired with their own revised version, wording-only. Sources: ParaRev and arXiv v1 → final LaTeX revisions.
   • Self-training on real drafts (12.6k): genuinely unclear human paragraphs paired with a rewrite by the round-1 model, written from lossless notes. Each rewrite was kept only if numbers, citations, refs and headings were identical to the original, it was a substantial rewrite, and a judge model found the same content and rated it clearer. Without this step, the model learned to return real human paragraphs unchanged.

Evaluation

Test paragraphs come from papers held out from all training. The clarity score is the readability heuristic: higher is clearer, and well-regarded papers score around −1 to 0.

	Qwen3.5-4B	PaperQwen3.5-4B
200 real unclear paragraphs (all fields): clarity score (input −4.24)	−4.23	−2.97
… paragraphs with an invented number	3.0%	0%
… paragraphs with a dropped number	6.0%	2.0%
… returned nearly unchanged	5%	11%
Synthetic drafts: chrF against the expert's original text	73.7	79.9

Compared with the round-1 model, round 2 returns fewer real paragraphs unchanged (11% vs. 18%) at the same clarity and fact preservation.

Limitations

• About 1 in 10 real paragraphs is returned nearly unchanged, especially ones that are already reasonably clear.
• An LLM judge (Qwen3.5-35B-A3B) preferred the base model's rewrites in blind pairwise comparison. The same judge also preferred the base model over the original expert-written text, so we do not treat it as a measure of writing quality.
• Continued pre-training covered only ML/NLP/speech papers; other fields are covered by fine-tuning data only.
• Math-heavy passages, tables and long sections were not trained on. Rewrite paragraph by paragraph.
• Only general capabilities relevant to rewriting were checked. Coding, math and other skills of the base model may have degraded.

License

This model is released under CC BY-NC-SA 4.0 (non-commercial, share-alike), because part of the fine-tuning data (ParaRev) carries that license. The base model, Qwen3.5-4B, is licensed under Apache 2.0 by the Qwen team; its license is included as LICENSE.Qwen-Apache-2.0. Training text from arXiv and Europe PMC remains the copyright of its authors.