Instructions to use eulogik/TinyDoc-VLM-256M with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use eulogik/TinyDoc-VLM-256M with Transformers:
# Use a pipeline as a high-level helper # Warning: Pipeline type "visual-question-answering" is no longer supported in transformers v5. # You must load the model directly (see below) or downgrade to v4.x with: # pip install "transformers<5.0.0" from transformers import pipeline pipe = pipeline("visual-question-answering", model="eulogik/TinyDoc-VLM-256M")# pip install -U transformers accelerate # Load model directly from transformers import AutoModelForSeq2SeqLM model = AutoModelForSeq2SeqLM.from_pretrained("eulogik/TinyDoc-VLM-256M", device_map="auto") - PEFT
How to use eulogik/TinyDoc-VLM-256M with PEFT:
Task type is invalid.
- Notebooks
- Google Colab
- Kaggle
TinyDoc-VLM-256M
β οΈ RETIRED RESEARCH CHECKPOINT β no performance claims
Measured 0.0% on OCRBench (n=1,000, all 10 categories) in 2026-09. The model degenerates under every prompt tried, including the exact training format (
"Extract document information: <image>"), with or without repetition controls; the LoRA adapter is worse (literal degeneration). The checkpoint is kept for reproducibility and research only. Inference is disabled on this card so the Hub widget does not serve a retired model.Earlier marketing numbers for this model (DocVQA ~65%, OCRBench ~60%) were never measured and have been withdrawn by the project. Evidence & artifacts:
evaluation/phase0/results/ocrbench_256m_full.*anddocs/BENCHMARKS.mdin the GitHub repo.
What actually ships
The working product is the local grounded-extraction SDK (tinydoc 0.2.0),
running a perception-strong Qwen base through Ollama with schema validation,
OCR-span evidence, and confidence. No API key.
| System | SROIE clean-100 field F1 |
|---|---|
| Qwen2.5-VL-3B β quality default | 0.9475 |
| Qwen3-VL-2B-Instruct β fast tier | 0.9200 |
| First-party hybrid v2 | 0.603 |
| PP-OCR + heuristics | 0.376 |
β οΈ clean-100 is a duplicate-filtered split of the public SROIE train pool, not an external test; Qwen may have seen it in pretraining. The original 100-doc set is 60% leaked. These are same-document engine comparisons.
All numbers recompute from committed artifacts
(python3 evaluation/phase0/recompute_scores.py). Full evidence and the v3 plan:
docs/v3_plan.md.
Install (0.2.0 is a source install until the PyPI upload lands):
pip install "tinydoc @ git+https://github.com/eulogik/TinyDoc-VLM.git#subdirectory=sdk"
ollama pull qwen2.5vl:3b
from tinydoc import ReceiptPipeline
pipe = ReceiptPipeline("ollama:qwen2.5vl:3b")
doc = pipe.extract("receipt.jpg")
print(doc.fields, doc.schema_valid, doc.confidence)
for fr in doc.field_results:
print(fr.name, fr.value, fr.evidence["quote"], fr.evidence["bbox"])
About this checkpoint (kept for research)
Experimental 256M-parameter document VLM (2026-06 run), legacy 384px pipeline:
Image (384Γ384) β SigLIP-B/16 (93M) β PixelShuffle compressor Γ3 (β64 tokens)
β SmolLM2-135M decoder (30 layers, GQA, 8192 ctx) β multi-task heads
Verified facts (not claims):
- Vision tower is alive (features stdβ1, cross-image cosine β0.57) β the dead-tower defect was specific to the separate 768px run.
- Processor
image_token_idmismatch was fixed in evaluation code; it does not explain the failures. - A 768px retrain (
TinyDoc-VLM-768-checkpoints) was abandoned: vision tower dead at init.
Loading (research use only)
from tinydoc_vlm import TinyDocVLMForConditionalGeneration, TinyDocVLMProcessor
model = TinyDocVLMForConditionalGeneration.from_pretrained("eulogik/TinyDoc-VLM-256M")
processor = TinyDocVLMProcessor()
Links
| Resource | URL |
|---|---|
| GitHub (eval artifacts, SDK, decision record) | github.com/eulogik/TinyDoc-VLM |
| v3 plan + evidence | docs/v3_plan.md |
| PyPI SDK | pypi.org/project/tinydoc |
| LoRA adapter (also retired) | eulogik/TinyDoc-VLM-LoRA |
| Benchmark write-up (measured only) | docs/BENCHMARKS.md |
License
Apache 2.0. Free for commercial use.
Built by eulogik
- Downloads last month
- 1,110