Image-to-Text
PEFT
Safetensors
Portuguese
English
vision-language
table-extraction
scientific-figures
markdown-table
qwen2.5-vl
lora
icdar-metric-loss
Instructions to use lucasoc/sci-image-models with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- PEFT
How to use lucasoc/sci-image-models with PEFT:
from peft import PeftModel from transformers import AutoModelForCausalLM base_model = AutoModelForCausalLM.from_pretrained("Qwen/Qwen2.5-VL-3B-Instruct") model = PeftModel.from_pretrained(base_model, "lucasoc/sci-image-models") - Notebooks
- Google Colab
- Kaggle
File size: 1,061 Bytes
be90b31 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 | """
Data preprocessing and conversation template building.
"""
from typing import Any, Dict, List, Optional
from PIL import Image
def format_qwen_vl_conversation(
image: Image.Image,
target_table: Optional[str] = None,
system_prompt: str = "You are an expert scientific figure analyzer. Extract the plotted quantitative data into a clean Markdown table."
) -> List[Dict[str, Any]]:
"""Formats an image and optional target into Qwen2.5-VL chat template format."""
messages = [
{
"role": "system",
"content": system_prompt
},
{
"role": "user",
"content": [
{"type": "image", "image": image},
{"type": "text", "text": "Extract all numerical data points from this scientific figure panel into a Markdown table with clear column headers."}
]
}
]
if target_table is not None:
messages.append({
"role": "assistant",
"content": target_table.strip()
})
return messages
|