Document Question Answering
Transformers
Safetensors
English
layoutlmv3
ocr
document-understanding
paddleocr
multilingual
layout-aware
lakshya-singh
# pip install -U transformers accelerate
# Load model directly
from transformers import AutoProcessor, AutoModelForDocumentQuestionAnswering
processor = AutoProcessor.from_pretrained("lakshya-rawat/document-qa-model")
model = AutoModelForDocumentQuestionAnswering.from_pretrained("lakshya-rawat/document-qa-model", device_map="auto")Quick Links
Document QA Model
This is a fine-tuned document question-answering model based on layoutlmv3-base. It is trained to understand documents using OCR data (via PaddleOCR) and accurately answer questions related to structured information in the document layout.
Model Details
Model Description
- Model Name:
document-qa-model - Base Model:
microsoft/layoutlmv3-base - Fine-tuned by: Lakshya Singh (solo contributor)
- Languages: English, Spanish, French, German, Italian
- License: Apache-2.0 (inherited from base model)
- Intended Use: Extract answers to structured queries from scanned documents
- Not funded — this project was completed independently.
Model Sources
- Repository:
Github Link - Trained on: Adapted version of
nielsr/docvqa_1200_examples - Model metrics: See
Uses
Direct Use
This model can be used for:
- Question Answering on document images (PDFs, invoices, utility bills)
- Information extraction tasks using OCR and layout-aware understanding
Out-of-Scope Use
- Not suitable for conversational QA
- Not suitable for images with no OCR-processed text
Training Details
Dataset
The dataset consisted of:
- Images of utility bills and documents
- OCR data with bounding boxes (from PaddleOCR)
- Queries in English, Spanish, and Chinese
- Answer spans with match scores and positions
Training Procedure
- Preprocessing: PaddleOCR was used to extract tokens, positions, and structure
- Model: LayoutLMv3-base
- Epochs: 4
- Learning rate schedule: Shown in image below
Training Metrics
- F1 Score (validation):
- Loss & Learning Rate Chart:
Evaluation
Metrics Used
- F1 score
- Match score of predicted spans
- Token overlap vs ground truth
Summary
The model performs well on document-style QA tasks, especially with:
- Clearly structured OCR results
- Document types similar to utility bills, invoices, and forms
How to Use
- Available on my
Github
- Downloads last month
- 14
Model tree for lakshya-rawat/document-qa-model
Base model
microsoft/layoutlmv3-base
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("document-question-answering", model="lakshya-rawat/document-qa-model")