Jingcz commited on
Commit
7d7ccef
·
verified ·
1 Parent(s): e3fe7f5

Upload README.md with huggingface_hub

Browse files
Files changed (1) hide show
  1. README.md +53 -0
README.md ADDED
@@ -0,0 +1,53 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: other
3
+ library_name: transformers
4
+ tags:
5
+ - chemistry
6
+ - molecular-structure
7
+ - reaction-diagram
8
+ - robustrdp
9
+ base_model: Qwen/Qwen2.5-VL-3B-Instruct
10
+ pipeline_tag: image-text-to-text
11
+ ---
12
+
13
+ # RobustRDP
14
+
15
+ Fine-tuned model for the paper: *RobustRDP: Advancing Reaction Diagram Parsing via Synthetic-to-Real Data Scaling and Robustness-Oriented Training*.
16
+
17
+ ## Description
18
+
19
+ This model is a fine-tuned checkpoint based on **Qwen2.5-VL-3B-Instruct**, trained with a three-stage pipeline:
20
+
21
+ 1. **Pretrain stage**: Synthetic data pretraining on large-scale synthetic reaction diagrams.
22
+ 2. **SFT stage**: Supervised fine-tuning with three specialized tasks:
23
+ - **Vanilla Reaction Parsing (VRP)**: Standard reaction diagram parsing
24
+ - **Region-Guided Reaction Parsing (RGRP)**: Region-aware parsing with spatial guidance
25
+ - **Prefix-Perturbed Reaction Parsing (PPRP)**: Robustness-oriented parsing with prefix perturbations
26
+ 3. **DPO stage**: Direct Preference Optimization to further align model outputs with ground-truth annotations.
27
+
28
+ ## Training Details
29
+
30
+ | Config | Pretrain | SFT | DPO |
31
+ |--------|----------|-----|-----|
32
+ | Base Model | Qwen2.5-VL-3B-Instruct | Qwen2.5-VL-3B-Instruct | Qwen2.5-VL-3B-Instruct |
33
+ | Learning Rate | 1.0×10⁻⁶ | 1.0×10⁻⁵ | 3.0×10⁻⁷ |
34
+ | Batch Size | 16 | 4 | 64 |
35
+ | Epochs | 1 | 1 | 1 |
36
+ | Scheduler | Cosine (warmup 0.03) | Cosine (warmup 0.03) | Cosine (warmup 0.03) |
37
+ | Optimizer | AdamW | AdamW | AdamW |
38
+ | Trainable Params | LLM only | Full (vision + LLM) | LLM only |
39
+
40
+ ## Usage
41
+
42
+ ```python
43
+ from transformers import AutoModelForCausalLM, AutoTokenizer
44
+
45
+ model_name = "Jingcz/RobustRDP"
46
+ tokenizer = AutoTokenizer.from_pretrained(model_name)
47
+ model = AutoModelForCausalLM.from_pretrained(model_name)
48
+ ```
49
+
50
+ ## Related Resources
51
+
52
+ - **Dataset**: [RxnLabelData](https://huggingface.co/datasets/Jingcz/RxnLabelData)
53
+ - **Annotation Platform**: [RxnLabel](https://github.com/jaydetang/RxnLabel)