FinChart-VL-4B (FinChart-OCR-Qwen3VL-4B, LoRA)

A LoRA adapter for Qwen3-VL-4B-Instruct specialised for financial chart understanding: question answering over charts, and chart to markdown-table extraction. Called FinChart-VL-4B in the ICAIF '26 paper From Curriculum to Rehearsal Learning: Towards Small Financial Chart VLMs.

At 4B parameters it scores above every open-source model reported in the FinChart-Bench paper's Table 2, including models 6x its size (published numbers, different harness), and above every baseline we re-ran under the same harness, including three chart-specialised models.

Results

FinChart-Bench (7,019 questions, our released harness)

Scored with our released harness (the official FinChart-Bench repository leaves answer extraction to the user): answers are read from the Result = [[ label ]] bracket and graded by strict equality (quotes before an MC letter are skipped).

model	params	TF	MC	QA	AVG
Qwen3-VL-4B (base)	4B	.931	.784	.316	.677
this model	4B	.954	.894	.548	.798
(same recipe, mean of 3 seeds)	4B	.950	.894	.535	.793 +/- .005
re-run under the same harness:
Qwen2.5-VL-7B	7B	.951	.745	.370	.689
Qwen2-VL-7B	7B	.035	.166	.220	.140
DeepSeek-VL2-small	2.8B active	.770	.252	.240	.421
TinyChart-3B (chart specialist)	3B	.001	.000	.002	.001
ChartGemma (chart specialist)	3B	.004	.000	.001	.002
Chart-R1 (chart specialist)	7B	.058	.003	.172	.077
published, not re-run:
Mistral-3.1-24B *	24B	.930	.842	.449	.740
Qwen2.5-VL-7B *	7B	.951	.828	.373	.717
Claude Sonnet 4 *	-	.969	.917	.636	.841

AVG is the unweighted mean of TF, MC and QA in every row (the FinChart-Bench paper prints question-weighted averages, which differ by up to .005).

* published numbers from the FinChart-Bench paper, not re-run under this harness -- prompts and image budgets may differ. The rows for this model, its base and the re-run block were run under an identical harness, released in the eval/ folder of the dataset repository.

Most of the re-run baselines' losses are answer format, not chart reading: they answer with a bare True/False, the option's text instead of its letter, program code (TinyChart) or their own <answer> tags (Chart-R1), which the protocol scores wrong. Read leniently as diagnostics, Qwen2.5-VL-7B averages .720, Qwen2-VL .625, DeepSeek-VL2 .600, TinyChart .318, ChartGemma .244 and Chart-R1 .782, against .789 for this recipe at the same seed, whose answers are unaffected.

The released checkpoint is the best of three training seeds (.798). The honest summary statistic for the recipe is the mean, .793 +/- .005; individual seeds scored .789 / .798 / .791.

Chart to markdown extraction (50 held-out real financial charts)

model	value F1	cell F1	header F1
Qwen3-VL-4B (base)	.745	.401	.671
this model	.785	.483	.683
(same recipe, mean of 3 seeds)	.771	.459	.671
Chart-R1 (harness prompt)	.746	.413	.593
ChartGemma (harness prompt)	.649	.268	.672
TinyChart-3B (harness prompt)	.508	.153	.347
DePlot (its own instruction)	.481	.073	.369

A response without a parseable markdown table scores as empty. Counting numbers anywhere in the response instead, this recipe averages .773 (the base model also .773).

Intended use

• Question answering over financial charts (values, comparisons, trends, simple arithmetic over chart data)
• Transcribing financial charts into markdown tables
• A drafting aid for chart digitisation with a human in the loop

Out of scope / limitations

Read this before deploying.

• Not an unattended table extractor. Cell-level F1 is about .48, i.e. roughly half of cells land wrong or under a mismatched header. Suitable for drafting, not for autonomous ingestion.

• Degraded on general financial document OCR. Chart specialisation costs general document fidelity. On FinCriticalED (848 pages, markdown-OCR prompt), measured on the seed-4242 sibling checkpoint:

  model	fact recall	numeric precision
  Qwen3-VL-4B (base)	.966	.949
  this recipe	.916	.720
  Qwen3-VL-8B	.953	.992

  Numeric precision of .720 means roughly 28 percent of the numbers it emits on dense document pages do not appear on the page. For document (non-chart) OCR, use the base model or a larger one -- Qwen3-VL-8B is strictly better at that task.

• Trained on English-language charts.

• No safety tuning beyond the base model's.

Training

base	Qwen3-VL-4B-Instruct
method	LoRA r=32, alpha=64, dropout .05, on q/k/v/o/gate/up/down_proj
data	19,039 samples: 5,359 TF / 2,212 MC / 8,967 QA / 2,501 chart-to-table (the question generator is seeded, so counts vary slightly by seed; the paper reports the seed-4242 mixture, 19,024)
schedule	1 epoch (1,190 steps), bf16, lr 1e-4 cosine, warmup 15
batch	4 x 4 grad accum, gradient checkpointing
seed	1337
hardware	A100-80GB, about 139 min
versions	transformers 5.14.1, peft 0.20.0, torch 2.11.0+cu128

Training data derives from varadsrivastava/financial-chart-ocr: real financial charts whose markdown tables were drafted by Claude Opus 4.6 and verified against the image by human annotators, plus Matplotlib-rendered synthetic charts whose tables are the generating data. QA/TF/MC supervision is generated programmatically from the gold tables, so every derived answer is exact with respect to its table.

Contamination control. The corpus was screened against FinChart-Bench by document-and-crop identifier before training, and the 50-chart held-out extraction benchmark (the benchmark split of the dataset) was excluded from training, enforced by an assertion inside the training loop.

What the extraction data actually does

An ablation removing only the 2,501 chart-to-table samples, holding QA/TF/MC identical, over 3 seeds:

• chart reasoning is unaffected: +.003 +/- .004 on FinChart-Bench (n.s.)
• chart extraction collapses without it: -.182 +/- .056 value F1 (p about .03), falling below the untrained base model

So the extraction objective acts as rehearsal against catastrophic forgetting rather than as a source of new capability. The reasoning gains come from the derived computational supervision, not from extraction training.

Usage

from transformers import AutoModelForImageTextToText, AutoProcessor
from peft import PeftModel

BASE = "Qwen/Qwen3-VL-4B-Instruct"
model = AutoModelForImageTextToText.from_pretrained(
    BASE, dtype="bfloat16", device_map="cuda")
model = PeftModel.from_pretrained(model, "varadsrivastava/FinChart-VL-4B")
processor = AutoProcessor.from_pretrained("varadsrivastava/FinChart-VL-4B")

messages = [{"role": "user", "content": [
    {"type": "image", "image": "chart.png"},
    {"type": "text", "text": "Convert this financial chart into a markdown "
                             "table. Include the chart title and units."}]}]
text = processor.apply_chat_template(messages, tokenize=False,
                                     add_generation_prompt=True)

For benchmark-style QA, append the official instruction so the answer is parseable: Output correct answer only in this format: 'Result = [[ answer ]]'

Reproducibility note

Set the vision budget when loading the processor: AutoProcessor.from_pretrained(..., min_pixels=256*28*28, max_pixels=1280*28*28) (the paper's setting, ~0.2–1.0 MP). Assigning processor.image_processor.max_pixels after loading does not change the resize the processor applies.

Citation

@inproceedings{srivastava2026finchartvl,
  title     = {From Curriculum to Rehearsal Learning: Towards Small Financial Chart {VLMs}},
  author    = {Srivastava, Varad and Shukla, Abhay and Casas Hernandez, Victor Daniel},
  booktitle = {7th ACM International Conference on AI in Finance (ICAIF '26)},
  year      = {2026},
  address   = {Milan, Italy},
  publisher = {ACM},
  doi       = {10.1145/3858063.3858868}
}

Disclaimer

This work was carried out at AI Lab, Barclays. The views expressed here are those of the authors and do not represent the views of Barclays.