TinyDoc-VLM-256M

⚠️ RETIRED RESEARCH CHECKPOINT β€” no performance claims

Measured 0.0% on OCRBench (n=1,000, all 10 categories) in 2026-09. The model degenerates under every prompt tried, including the exact training format ("Extract document information: <image>"), with or without repetition controls; the LoRA adapter is worse (literal degeneration). The checkpoint is kept for reproducibility and research only. Inference is disabled on this card so the Hub widget does not serve a retired model.

Earlier marketing numbers for this model (DocVQA ~65%, OCRBench ~60%) were never measured and have been withdrawn by the project. Evidence & artifacts: evaluation/phase0/results/ocrbench_256m_full.* and docs/BENCHMARKS.md in the GitHub repo.

What actually ships

The working product is the local grounded-extraction SDK (tinydoc 0.2.0), running a perception-strong Qwen base through Ollama with schema validation, OCR-span evidence, and confidence. No API key.

System SROIE clean-100 field F1
Qwen2.5-VL-3B β€” quality default 0.9475
Qwen3-VL-2B-Instruct β€” fast tier 0.9200
First-party hybrid v2 0.603
PP-OCR + heuristics 0.376

⚠️ clean-100 is a duplicate-filtered split of the public SROIE train pool, not an external test; Qwen may have seen it in pretraining. The original 100-doc set is 60% leaked. These are same-document engine comparisons.

All numbers recompute from committed artifacts (python3 evaluation/phase0/recompute_scores.py). Full evidence and the v3 plan: docs/v3_plan.md.

Install (0.2.0 is a source install until the PyPI upload lands):

pip install "tinydoc @ git+https://github.com/eulogik/TinyDoc-VLM.git#subdirectory=sdk"
ollama pull qwen2.5vl:3b
from tinydoc import ReceiptPipeline

pipe = ReceiptPipeline("ollama:qwen2.5vl:3b")
doc = pipe.extract("receipt.jpg")
print(doc.fields, doc.schema_valid, doc.confidence)
for fr in doc.field_results:
    print(fr.name, fr.value, fr.evidence["quote"], fr.evidence["bbox"])

About this checkpoint (kept for research)

Experimental 256M-parameter document VLM (2026-06 run), legacy 384px pipeline:

Image (384Γ—384) β†’ SigLIP-B/16 (93M) β†’ PixelShuffle compressor Γ—3 (β†’64 tokens)
β†’ SmolLM2-135M decoder (30 layers, GQA, 8192 ctx) β†’ multi-task heads

Verified facts (not claims):

  • Vision tower is alive (features stdβ‰ˆ1, cross-image cosine βˆ’0.57) β€” the dead-tower defect was specific to the separate 768px run.
  • Processor image_token_id mismatch was fixed in evaluation code; it does not explain the failures.
  • A 768px retrain (TinyDoc-VLM-768-checkpoints) was abandoned: vision tower dead at init.

Loading (research use only)

from tinydoc_vlm import TinyDocVLMForConditionalGeneration, TinyDocVLMProcessor
model = TinyDocVLMForConditionalGeneration.from_pretrained("eulogik/TinyDoc-VLM-256M")
processor = TinyDocVLMProcessor()

Links

Resource URL
GitHub (eval artifacts, SDK, decision record) github.com/eulogik/TinyDoc-VLM
v3 plan + evidence docs/v3_plan.md
PyPI SDK pypi.org/project/tinydoc
LoRA adapter (also retired) eulogik/TinyDoc-VLM-LoRA
Benchmark write-up (measured only) docs/BENCHMARKS.md

License

Apache 2.0. Free for commercial use.


Built by eulogik

Downloads last month
1,110
Safetensors
Model size
0.3B params
Tensor type
F32
Β·
Inference Providers NEW
This model isn't deployed by any Inference Provider. πŸ™‹ Ask for provider support

Model tree for eulogik/TinyDoc-VLM-256M

Adapters
1 model

Space using eulogik/TinyDoc-VLM-256M 1

Collections including eulogik/TinyDoc-VLM-256M