Japanese-Receipt-VL-3B-JSON

Model Description

Japanese-Receipt-VL-3B-JSON is a fine-tuned vision-language model based on Qwen2.5-VL-3B, specifically optimized for Japanese receipt OCR and structured data extraction. The model processes mobile phone-captured receipt images and outputs structured JSON containing store information, itemized purchases, tax calculations, and payment details.

Dataset: Trained on the Japanese-Mobile-Receipt-OCR-1K dataset.

Model Details

  • Model Name: Japanese-Receipt-VL-3B-JSON
  • Dataset Name: Japanese-Mobile-Receipt-OCR-1K
  • Base Model: Qwen/Qwen2.5-VL-3B-Instruct
  • Model Type: Vision-Language Model (Multimodal)
  • Language: Japanese (preserves original text exactly as printed)
  • License: Same as base model
  • Fine-tuning Method: LoRA (Low-Rank Adaptation)
  • Training Images: 1,000 real-world Japanese receipt images
  • Total Dataset: 1,147 collected images
  • Output Format: Structured JSON with Japanese keys
  • Extraction Approach: Comprehensive, exact preservation without translation or interpretation

Intended Use

Primary Use Cases

  • Japanese retail receipt digitization
  • Expense tracking and management systems
  • Business accounting automation
  • Mobile receipt scanning applications
  • E-commerce and point-of-sale integrations

Input

  • Mobile phone-captured images of Japanese receipts
  • Supported formats: JPG, PNG
  • Optimal resolution: 640-896px (portrait) or 896-640px (landscape)

Output

Structured JSON containing:

{
    "ๅบ—่ˆ—ๅ": "ใ‚ปใƒ–ใƒณใ‚คใƒฌใƒ–ใƒณๆธ‹่ฐทๅบ—",
    "ๆ—ฅไป˜": "2024ๅนด01ๆœˆ15ๆ—ฅ",
    "ๆ™‚ๅˆป": "14:30",
    "ใƒฌใ‚ทใƒผใƒˆNo": "0001234",
    "ๅ•†ๅ“ใƒชใ‚นใƒˆ": [
        {
            "ๅ•†ๅ“ๅ": "ใŠใซใŽใ‚Š้ฎญ",
            "ๆ•ฐ้‡": 1,
            "ๅ˜ไพก": 128,
            "้‡‘้ก": 128
        }
    ],
    "ๅฐ่จˆ": 840,
    "ๆถˆ่ฒป็จŽ": 84,
    "ๅˆ่จˆ": 924,
    "ๆ”ฏๆ‰•ๆ–นๆณ•": "็พ้‡‘",
    "ใŠ้ ใ‚Š": 1000,
    "ใŠ้‡ฃใ‚Š": 76
}

Prompting Guidelines

Optimized Instruction Prompt

This model was trained with a specific instruction prompt that ensures comprehensive and accurate extraction. For best results, use the exact prompt shown in the usage example above.

Key Prompt Features

  • Comprehensive extraction: Captures all visible information including store details, timestamps, itemized products, tax calculations, and payment information
  • Exact preservation: Maintains original Japanese text, formatting, and symbols without translation or modification
  • JSON structure: Uses Japanese field names as keys for cultural and linguistic consistency
  • No inference: Only extracts explicitly visible information, avoiding assumptions or corrections
  • Complete coverage: Includes often-missed elements like receipt numbers, cashier names, footer messages, and promotional text

Sample Results

Example Input

Sample Receipt

Example Output

{
  "้ ˜ๅŽๆ›ธ": {
    "ๅบ—่ˆ—ๆƒ…ๅ ฑ": [
      {
        "ๅบ—ๅ": "ๆฅญๅ‹™ใ‚นใƒผใƒ‘ใƒผๆฒณๅ†…ๅฑ‹",
        "ๅ ดๆ‰€": "้’่‘‰ๅฐๅบ—",
        "้›ป่ฉฑ": "045-985-9603"
      }
    ]
  },
  "็™ป้Œฒ็•ชๅท": "T7011701002269",
  "ใƒกใƒƒใ‚ปใƒผใ‚ธ": {
    "ๅ†…ๅฎน": "ใƒ—ใƒญใฎๅ“่ณชใจใƒ—ใƒญใฎไพกๆ ผ ๆฅญๅ‹™ใ‚นใƒผใƒ‘ใƒผใงใฏๆฏŽๆ—ฅใฎใŠ่ฒทใ„ๅพ—ใจใชใฃใฆใ„ใพใ™ใ€‚"
  },
  "็™บ่กŒๆ—ฅ": {
    "ๆœˆๆ—ฅๅนดๅˆ†้กž": "72025ๅนด 7ๆœˆ12ๆ—ฅ",
    "ๆ‹…ๅฝ“่€…": "ๅœŸ็”ฐ"
  },
  "ๅ•†ๅ“ๆƒ…ๅ ฑ": [
    {
      "ๅๅ‰": "้ถๅต่ตค็މMSP",
      "ๅ€‹ๆ•ฐ": "10ๅ€‹ๅ…ฅใ‚Š",
      "ไพกๆ ผ": "๏ฟฅ237"
    },
    {
      "ๅๅ‰": "ใƒžใ‚ซใƒญใƒ‹(ใ‚ปใƒ€ใƒผใƒ‹ใƒผใƒ‹ใƒผ)500Gใ€‚",
      "ๅ€‹ๆ•ฐ": "๏ฟฅ138",
      "ๅ•†ๅ“่ฉณ็ดฐ": "021 4901995400119"
    },
    {
      "ๅๅ‰": "JUCO VIA(ๆฅญ)ใƒใƒฅใƒผใ‚ฟใƒผใ‚นใƒฉใ‚คใ‚ตใ‚คใ‚บ",
      "ไพกๆ ผ": "๏ฟฅ2418"
    },
    {
      "ๅๅ‰": "ๅ”ๅŒ็‰›ไนณ ้…ช่พฒ็‰›ไนณ 1L",
      "ไพกๆ ผ": "๏ฟฅ199",
      "ๅ•†ๅ“่ฉณ็ดฐ": "021 496168110063"
    },
    {
      "ๅๅ‰": "ใŠใ‹ใ‚‚็ด่ฑ†ๆฅตๅฐ็ฒ’ใƒŸใƒ‹",
      "ไพกๆ ผ": "๏ฟฅ76"
    },
    {
      "ๅๅ‰": "โ˜…ๅ”ๅŒ็‰›ไนณ",
      "ๅ€‹ๆ•ฐ": "2418",
      "ๅ•†ๅ“่ฉณ็ดฐ": "021 4901160010143"
    }
  ],
  "็จŽๆƒ…ๅ ฑ": {
    "่ฝปๆธ›็จŽ็އๅฏพ่ฑก": {
      "้‡‘้ก": "1,068",
      "ๅ‰ฒๅˆ": "๏ผ…",
      "ๅค–็จŽ่ฒทไธŠ้ก": "๏ฟฅ1,068",
      "ๅค–็จŽ้ก": "๏ฟฅ85",
      "็จŽ้กๅˆ่จˆ": "๏ฟฅ85"
    }
  },
  "ๅˆ่จˆๆƒ…ๅ ฑ": {
    "้‡‘้ก": {
      "ๅˆ่จˆ": "ยฅ1,153",
      "ๅค–็จŽๅˆ่จˆ": {
        "้‡‘้ก": "1,068",
        "ๅ‰ฒๅˆ": "๏ผ…",
        "ๅค–็จŽๅˆ่จˆ": "ยฅ1,068",
        "ๅค–็จŽๅˆ่จˆๅ‰ฒๅˆ": "๏ผ…",
        "็จŽ้กๅˆ่จˆ": "๏ฟฅ85"
      }
    },
    "ใŠ้ ใ‚Š": "ยฅ5,000",
    "ใŠ้‡ฃใ‚Š": "๏ฟฅ3,847"
  },
  "ๅ‚™่€ƒ": {
    "ๅ†…ๅฎน": "ๆœฌๅ•ไฟ็ฎกไธŠใฎใŠ้ก˜ใ„่ฒกๅธƒใƒปๆ‰‹ๅธณ็ญ‰ใซใฏใ•ใ‚“ใงไฟ็ฎก้ ‚ใๅ ดๅˆใฏใ€ๅฐๅˆท้ขใ‚’ๅ†…ๅดใซๆŠ˜ใ‚Šไฟ็ฎกใ‚’ใŠ้ก˜ใ„ใ—ใพใ™"
  },
  "ใƒใƒผใ‚ณใƒผใƒ‰": {
    "6829": "6821",
    "ๆ—ฅไป˜": "11:20"
  }
}

This example demonstrates the model's ability to:

  • Extract complete store information including registration numbers
  • Capture all product details with prices and barcodes
  • Parse complex tax calculations and reduced tax rates
  • Preserve promotional messages and handling instructions
  • Maintain exact Japanese formatting and terminology

Training Details

Dataset

  • Source: Custom collected real-world Japanese receipts
  • Collection Method: Mobile phone photography in various lighting conditions
  • Training Split: 1,000 images
  • Validation Split: 147 images
  • Receipt Types: Convenience stores, restaurants, supermarkets, department stores
  • Image Preprocessing: Balanced resize (640x896 target), aspect ratio preservation

Training Configuration

  • Sequence Length: 1,536 tokens (optimized for Japanese text)
  • Batch Size: 1 (with gradient accumulation)
  • Learning Rate: 5e-6 (conservative for stability)
  • LoRA Configuration:
    • Rank (r): 8
    • Alpha: 16
    • Dropout: 0.1
    • Target modules: Attention layers (q_proj, k_proj, v_proj, o_proj)
  • Training Hardware: Tesla T4 GPU
  • Training Framework: Unsloth + HuggingFace Transformers

Training Optimizations

  • Custom image preprocessing for mobile receipt photos
  • Balanced aspect ratio handling (portrait/landscape)
  • Even dimension sizing to prevent tokenization issues
  • Conservative training parameters to avoid NaN issues
  • Japanese text tokenization optimizations

Performance

Receipt Types Supported

  • โœ… Convenience store receipts (ใ‚ปใƒ–ใƒณใ‚คใƒฌใƒ–ใƒณใ€ใƒญใƒผใ‚ฝใƒณใ€ใƒ•ใ‚กใƒŸใƒž)
  • โœ… Restaurant bills and cafรฉ receipts
  • โœ… Supermarket and grocery receipts
  • โœ… Department store purchases
  • โœ… Pharmacy and drugstore receipts
  • โœ… Mixed Japanese/English text receipts

Text Recognition Capabilities

  • Japanese Scripts: Kanji, Hiragana, Katakana
  • Numerical Data: Prices, quantities, tax calculations
  • Dates & Times: Various Japanese date formats
  • Special Characters: Currency symbols (๏ฟฅ), percentages (%)

Usage

Basic Usage

from transformers import Qwen2_5_VLForConditionalGeneration, AutoProcessor
from PIL import Image

# Load model and processor
model = Qwen2_5_VLForConditionalGeneration.from_pretrained(
    "your-username/Japanese-Receipt-VL-3B-JSON"
)
processor = AutoProcessor.from_pretrained("your-username/Japanese-Receipt-VL-3B-JSON")

# Load receipt image
image = Image.open("japanese_receipt.jpg")

# Optimized instruction prompt for Japanese receipt extraction
instruct_prompt = """You are an intelligent document parser. Read the following Japanese receipt and extract every piece of information exactly as it appears, and present it in a well-structured JSON format using Japanese keys and values. Please strictly follow these rules: Only extract information that is actually present on the receipt. Do not include any missing, blank, or inferred fields. Do not summarize, omit, translate, or modify any part of the receipt. Every character, number, symbol, and line must be retained exactly as printed. Extract all available content including but not limited to: store details, receipt number, date, time, cashier name, product list, prices, tax breakdowns, payment details, receipt bags, barcodes, notices, and any footer messages. Preserve original formatting such as line breaks, symbols, and full-width characters (hiragana, katakana, kanji, numbers, etc.). Do not perform any translation, correction, interpretation, or reformatting of content. Use only what is present. Output the result in JSON format, using Japanese field names as keys."""

# Prepare input
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "image": image},
            {"type": "text", "text": instruct_prompt}
        ]
    }
]

# Process
text = processor.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = processor(text=[text], images=[image], return_tensors="pt")
outputs = model.generate(**inputs, max_new_tokens=512)
result = processor.batch_decode(outputs, skip_special_tokens=True)[0]
print(result)

Recommended Preprocessing

from your_preprocessing import process_receipt_image

# Preprocess image for optimal results
processed_image = process_receipt_image(
    "receipt.jpg",
    target_width=640,
    target_height=896
)

Limitations

Current Limitations

  • Language: Specifically trained for Japanese receipts; maintains original Japanese text in output
  • Image Quality: Best results with clear, well-lit mobile photos
  • Receipt Layout: Optimized for standard Japanese retail receipt formats
  • Handwritten Text: Limited support for handwritten items or annotations
  • Damaged Receipts: May struggle with torn, folded, or severely faded receipts
  • Text Preservation: Designed to preserve exact text as printed; does not perform translation or correction

Technical Limitations

  • Input Resolution: Optimized for mobile-captured images (not high-DPI scans)
  • Sequence Length: 1,536 token limit may truncate very long receipts
  • JSON Structure: Fixed schema; doesn't adapt to unusual receipt formats

Ethical Considerations

Privacy & Data Protection

  • Personal Information: May extract personal data from receipts
  • Data Handling: Users responsible for complying with privacy regulations
  • Business Data: Consider data sensitivity in commercial applications

Bias & Fairness

  • Regional Bias: Trained primarily on specific Japanese retail formats
  • Store Type Bias: Performance may vary across different business types
  • Demographic Bias: Dataset reflects specific geographic collection area

Citation

@model{japanese-receipt-vl-3b-json,
  title={Japanese-Receipt-VL-3B-JSON: Fine-tuned Vision-Language Model for Japanese Receipt OCR},
  author={[Your Name]},
  year={2024},
  publisher={Hugging Face},
  journal={Hugging Face Model Hub},
  howpublished={\url{https://huggingface.co/your-username/Japanese-Receipt-VL-3B-JSON}},
  note={Trained on Japanese-Mobile-Receipt-OCR-1K dataset}
}

Acknowledgments

  • Base Model: Qwen2.5-VL-3B by Alibaba Cloud
  • Training Framework: Unsloth for efficient fine-tuning
  • Dataset: Japanese-Mobile-Receipt-OCR-1K - Custom collected Japanese receipt images (1,147 samples)
  • Community: Hugging Face transformers library

Model Card Authors

Sabari Nathan / Couger Inc,Japan

Model Card Contact

sabarinathantce@gmail.com


Tags: vision-language, japanese, ocr, receipt-processing, json-extraction, qwen2.5-vl, multimodal, fine-tuned

Downloads last month
129
Safetensors
Model size
4B params
Tensor type
BF16
ยท
F16
ยท
Inference Providers NEW
This model isn't deployed by any Inference Provider. ๐Ÿ™‹ Ask for provider support

Model tree for sabaridsnfuji/Japanese-Receipt-VL-3B-JSON

Finetuned
(933)
this model
Quantizations
1 model

Space using sabaridsnfuji/Japanese-Receipt-VL-3B-JSON 1