Upload MolParser-Mobile-V2 (private)

README.md

@@ -0,0 +1,153 @@
+---
+library_name: transformers
+pipeline_tag: image-to-text
+tags:
+- chemistry
+- image-to-text
+- ocsr
+- markush
+- e-smiles2.0
+datasets:
+- UniParser/MolParser-7M
+- UniParser/MolGallery
+license: cc-by-nc-sa-4.0
+---
+
+# MolParser Mobile V2
+
+<p align="center">
+💻 <a href="https://github.com/dptech-corp/MolParser">GitHub</a> |
+📘 <a href="https://github.com/dptech-corp/MolParser/blob/main/skills/molparser-extended-smiles/extended-smiles-spec.md">E-SMILES 2.0 Spec</a> |
+📄 <a href="https://arxiv.org/abs/2609.05807">Report</a> |
+🚀 <a href="https://ocsr.dp.tech/">Demo</a>
+</p>
+
+**MolParser-Mobile-V2** is a lightweight Optical Chemical Structure Recognition (OCSR) model that converts molecular structure images directly into **E-SMILES 2.0**. It upgrades MolParser-Mobile for broader recognition of structures found in chemical literature, especially complex Markush structures, while retaining a compact 10M parameter architecture.
+
+
+## 🚀 What's New
+
+* **E-SMILES 2.0 output** with substantially broader coverage of literature molecules and Markush structures.
+* **Richer Markush type coverage** for literature molecules, including atom- and ring-indexed substituents, explicit dummy attachments, nested substructures, structural repeating units and polymers, virtual arcs, colored endpoint balls, and axial-chirality annotations.
+* **384 × 384 input resolution**, increased from 224 × 224 in MolParser-Mobile.
+* **384-token maximum output length**, increased from 256 tokens.
+* **Improved recognition accuracy**, particularly for complex and stereochemical structures.
+
+For notation details, examples, validation, normalization, substitution, and rendering utilities, see the [MolParser Repo](https://github.com/dptech-corp/MolParser) and the [E-SMILES specification](https://github.com/dptech-corp/MolParser/blob/main/skills/molparser-extended-smiles/extended-smiles-spec.md).
+
+
+## 📊 Performance
+
+Accuracy for MolParser-Mobile-V2 was measured with FP16 inference, greedy decoding, and batch size 512. Deltas are relative to MolParser-Mobile.
+
+| Model                    | Parameters | Throughput (RTX 4090D) |     Uni-Parser Bench |             BioVista |          WildMol-10k |               USPTO |
+| ------------------------ | ---------: | ---------------------: | -------------------: | -------------------: | -------------------: | ------------------: |
+| MolParser-Mobile         |      9.98M |           1,520 Mol/s  |                0.823 |                0.801 |                0.734 |               0.836 |
+| **MolParser-Mobile-V2**  |     10.00M |           1,296 Mol/s  | **0.850** (+0.027)   | **0.820** (+0.019)   | **0.762** (+0.028)   | **0.909** (+0.073)  |
+
+
+## ⚡ Usage
+
+### Option 1. MolParser Library (Recommended)
+
+The [MolParser library](https://github.com/dptech-corp/MolParser) provides a convenient interface for molecule detection, recognition, E-SMILES 2.0 post-processing, and rendering.
+
+Clone the repository and install the package:
+
+```bash
+git clone https://github.com/dptech-corp/MolParser.git
+cd MolParser
+pip install -e .
+```
+
+Then run:
+
+```python
+from molparser import MolParser
+
+parser = MolParser() 
+# default setting (updated to latest main): molparser_hf_repo="UniParser/MolParser-Mobile-V2", max_length=384,
+
+result = parser.parse("mol.png", rec_only=True)
+```
+
+To render the predicted E-SMILES as SVG or PNG, see [Render E-SMILES](https://github.com/dptech-corp/MolParser/tree/main#render-e-smiles):
+
+```python
+from pathlib import Path
+from molparser import utils as mutils
+
+raw = "*C(O)c1cc(C(=O)N(*)*)cc(-c2*ccc*2)c1<sep><a>0:CF3</a><a>9:R[3]</a><a>10:R[2]</a><a>14:X</a><a>18:Y</a><r>1:R[1]?1-3</r>"
+svg_text = mutils.draw(raw, output_format="svg")
+Path("molecule.svg").write_text(svg_text, encoding="utf-8")
+
+png_bytes = mutils.draw(raw, output_format="png")
+Path("molecule.png").write_bytes(png_bytes)
+```
+
+### Option 2. 🤗 Transformers
+
+Load MolParser-Mobile-V2 directly with the Hugging Face transformers library.
+
+```python
+import torch
+from PIL import Image
+from transformers import AutoModelForImageTextToText, AutoProcessor
+
+repo_id = "UniParser/MolParser-Mobile-V2"
+device = "cuda" if torch.cuda.is_available() else "cpu"
+dtype = torch.float16 if device == "cuda" else torch.float32
+
+processor = AutoProcessor.from_pretrained(repo_id, trust_remote_code=True)
+model = AutoModelForImageTextToText.from_pretrained(
+    repo_id,
+    dtype=dtype,
+    trust_remote_code=True,
+).to(device).eval()
+
+image = Image.open("mol.png").convert("RGB")
+inputs = processor(images=image, return_tensors="pt")
+inputs = {k: v.to(device, dtype=dtype) for k, v in inputs.items()}
+
+output_ids = model.generate(**inputs, max_length=384, num_beams=1, do_sample=False)
+caption = processor.batch_decode(output_ids, skip_special_tokens=True)[0]
+print(caption)
+```
+
+
+## 📜 License
+
+### MolParser-Mobile-V2 Weight
+
+The **MolParser-Mobile-V2 model weights** are provided for **non-commercial use only** under CC BY-NC-SA 4.0.
+
+For commercial licensing, please contact **fangxi@dp.tech** or open a discussion on Hugging Face.
+
+### MolParser Github Repo
+
+The **MolParser** library (including E-SMILES post-processing and rendering) is available at https://github.com/dptech-corp/MolParser and is licensed under the **Apache License 2.0**, which permits commercial use, modification, and distribution, provided that the license and copyright notices are retained.
+
+**Note:** Model weights, datasets, and third-party dependencies are subject to their respective licenses.
+
+## 📖 Citation
+
+If you use this model, please cite:
+
+```
+@article{fang2026molparserm,
+  title={MolParser-Mobile: Ultrafast OCSR System for Large-Scale Chemical Literature Mining},
+  author={Fang, Xi and Lu, Haocheng and Lyu, Han and Luo, Chengxiang and Zhang, Linfeng and Ke, Guolin},
+  journal={arXiv preprint arXiv:2609.05807},
+  year={2026}
+}
+```
+
+```
+@inproceedings{fang2025molparser,
+  title={Molparser: End-to-end visual recognition of molecule structures in the wild},
+  author={Fang, Xi and Wang, Jiankun and Cai, Xiaochen and Chen, Shangqian and Yang, Shuwen and Tao, Haoyi and Wang, Nan and Yao, Lin and Zhang, Linfeng and Ke, Guolin},
+  booktitle={Proceedings of the IEEE/CVF International Conference on Computer Vision},
+  pages={24528--24538},
+  year={2025}
+}
+```

config.json

@@ -0,0 +1,101 @@
+{
+  "architectures": [
+    "MolParserVisionEncoderDecoderModel"
+  ],
+  "auto_map": {
+    "AutoConfig": "modeling_molparser_mobile.MolParserVisionEncoderDecoderConfig",
+    "AutoModelForImageTextToText": "modeling_molparser_mobile.MolParserVisionEncoderDecoderModel"
+  },
+  "decoder": {
+    "_name_or_path": "",
+    "activation_dropout": 0.0,
+    "activation_function": "gelu",
+    "add_cross_attention": true,
+    "architectures": null,
+    "attention_dropout": 0.0,
+    "bos_token_id": 0,
+    "chunk_size_feed_forward": 0,
+    "classifier_dropout": 0.0,
+    "d_model": 192,
+    "decoder_attention_heads": 4,
+    "decoder_ffn_dim": 768,
+    "decoder_layerdrop": 0.0,
+    "decoder_layers": 6,
+    "decoder_start_token_id": 2,
+    "dropout": 0.1,
+    "dtype": "float16",
+    "encoder_attention_heads": 4,
+    "encoder_ffn_dim": 768,
+    "encoder_hidden_size": 192,
+    "encoder_layerdrop": 0.0,
+    "encoder_layers": 0,
+    "eos_token_id": 2,
+    "forced_eos_token_id": 2,
+    "id2label": {
+      "0": "LABEL_0",
+      "1": "LABEL_1",
+      "2": "LABEL_2"
+    },
+    "init_std": 0.02,
+    "is_decoder": true,
+    "is_encoder_decoder": false,
+    "label2id": {
+      "LABEL_0": 0,
+      "LABEL_1": 1,
+      "LABEL_2": 2
+    },
+    "max_position_embeddings": 1024,
+    "model_type": "bart",
+    "output_attentions": false,
+    "output_hidden_states": false,
+    "pad_token_id": 1,
+    "problem_type": null,
+    "return_dict": true,
+    "scale_embedding": false,
+    "tie_word_embeddings": true,
+    "use_cache": true,
+    "vocab_size": 414
+  },
+  "decoder_start_token_id": 0,
+  "dtype": "float16",
+  "encoder": {
+    "_name_or_path": "",
+    "architectures": null,
+    "auto_map": {
+      "AutoConfig": "modeling_molparser_mobile.CustomEncoderConfig",
+      "AutoModel": "modeling_molparser_mobile.CustomTimmEncoder"
+    },
+    "chunk_size_feed_forward": 0,
+    "dtype": "float16",
+    "hidden_size": 192,
+    "id2label": {
+      "0": "LABEL_0",
+      "1": "LABEL_1"
+    },
+    "initializer_range": 0.02,
+    "is_encoder_decoder": false,
+    "label2id": {
+      "LABEL_0": 0,
+      "LABEL_1": 1
+    },
+    "model_input_size": 384,
+    "model_type": "custom_timm_encoder",
+    "output_attentions": false,
+    "output_hidden_states": false,
+    "pixel_unshuffle": 1,
+    "problem_type": null,
+    "return_dict": true,
+    "timm_model_name": "vit_tiny_r_s16_p8_384.augreg_in21k_ft_in1k",
+    "timm_output_dim": 192,
+    "timm_pretrained": false
+  },
+  "is_encoder_decoder": true,
+  "model_type": "molparser_vision_encoder_decoder",
+  "pad_token_id": 1,
+  "tie_word_embeddings": false,
+  "transformers_version": "5.4.0",
+  "use_cache": false,
+  "vocab_size": 414,
+  "torch_dtype": "float16",
+  "model_name": "MolParser Mobile V2"
+}

generation_config.json

@@ -0,0 +1,39 @@
+{
+  "_from_model_config": false,
+  "assistant_confidence_threshold": 0.4,
+  "assistant_lookbehind": 10,
+  "bos_token_id": 0,
+  "decoder_start_token_id": 0,
+  "diversity_penalty": 0.0,
+  "do_sample": false,
+  "early_stopping": false,
+  "encoder_no_repeat_ngram_size": 0,
+  "encoder_repetition_penalty": 1.0,
+  "eos_token_id": 2,
+  "epsilon_cutoff": 0.0,
+  "eta_cutoff": 0.0,
+  "forced_eos_token_id": 2,
+  "length_penalty": 1.0,
+  "max_length": 384,
+  "min_length": 0,
+  "no_repeat_ngram_size": 0,
+  "num_assistant_tokens": 20,
+  "num_assistant_tokens_schedule": "constant",
+  "num_beam_groups": 1,
+  "num_beams": 1,
+  "num_return_sequences": 1,
+  "output_attentions": false,
+  "output_hidden_states": false,
+  "output_scores": false,
+  "pad_token_id": 1,
+  "remove_invalid_values": false,
+  "repetition_penalty": 1.0,
+  "return_dict_in_generate": false,
+  "target_lookbehind": 10,
+  "temperature": 1.0,
+  "top_k": 50,
+  "top_p": 1.0,
+  "transformers_version": "5.4.0",
+  "typical_p": 1.0,
+  "use_cache": false
+}

image_processing_molparser_mobile.py

@@ -0,0 +1,95 @@
+"""Image processor for MolParser Mobile."""
+
+from __future__ import annotations
+
+from typing import List, Sequence, Union
+
+import cv2
+import numpy as np
+import torch
+from PIL import Image
+from transformers import BaseImageProcessor
+from transformers.feature_extraction_utils import BatchFeature
+
+
+ImageInput = Union[str, Image.Image, np.ndarray, torch.Tensor]
+
+
+class MolParserImageProcessor(BaseImageProcessor):
+    model_input_names = ["pixel_values"]
+
+    def __init__(
+        self,
+        image_size: int = 384,
+        do_resize: bool = True,
+        do_normalize: bool = True,
+        image_mean: Sequence[float] = (0.485, 0.456, 0.406),
+        image_std: Sequence[float] = (0.229, 0.224, 0.225),
+        **kwargs,
+    ):
+        super().__init__(**kwargs)
+        self.image_size = int(image_size)
+        self.do_resize = bool(do_resize)
+        self.do_normalize = bool(do_normalize)
+        self.image_mean = list(image_mean)
+        self.image_std = list(image_std)
+
+    @property
+    def size(self):
+        return {"height": self.image_size, "width": self.image_size}
+
+    def _to_pil(self, image: ImageInput) -> Image.Image:
+        if isinstance(image, Image.Image):
+            return image.convert("RGB")
+        if isinstance(image, str):
+            return Image.open(image).convert("RGB")
+        if isinstance(image, torch.Tensor):
+            tensor = image.detach().cpu()
+            if tensor.ndim == 3 and tensor.shape[0] in {1, 3, 4}:
+                tensor = tensor.permute(1, 2, 0)
+            array = tensor.numpy()
+        else:
+            array = np.asarray(image)
+        if array.dtype != np.uint8:
+            if array.max() <= 1.0:
+                array = array * 255.0
+            array = np.clip(np.rint(array), 0, 255).astype(np.uint8)
+        if array.ndim == 2:
+            return Image.fromarray(array, mode="L").convert("RGB")
+        if array.shape[-1] == 4:
+            return Image.fromarray(array, mode="RGBA").convert("RGB")
+        return Image.fromarray(array).convert("RGB")
+
+    def _preprocess_one(self, image: ImageInput) -> np.ndarray:
+        pil_image = self._to_pil(image)
+        array = np.asarray(pil_image).astype(np.uint8)
+        if self.do_resize:
+            # Match deploy/transform.py: albumentations.Resize defaults to OpenCV INTER_LINEAR.
+            array = cv2.resize(array, (self.image_size, self.image_size), interpolation=cv2.INTER_LINEAR)
+        array = array.astype(np.float32) / 255.0
+        if self.do_normalize:
+            mean = np.asarray(self.image_mean, dtype=np.float32).reshape(1, 1, 3)
+            std = np.asarray(self.image_std, dtype=np.float32).reshape(1, 1, 3)
+            array = (array - mean) / std
+        return np.transpose(array, (2, 0, 1))
+
+    def preprocess(
+        self,
+        images: Union[ImageInput, Sequence[ImageInput]],
+        return_tensors: str | None = None,
+        **kwargs,
+    ) -> BatchFeature:
+        if not isinstance(images, (list, tuple)):
+            images = [images]
+        pixel_values: List[np.ndarray] = [self._preprocess_one(image) for image in images]
+        data = {"pixel_values": np.stack(pixel_values, axis=0)}
+        encoded = BatchFeature(data=data)
+        if return_tensors is not None:
+            encoded = encoded.convert_to_tensors(return_tensors)
+        return encoded
+
+    def __call__(self, images: Union[ImageInput, Sequence[ImageInput]], return_tensors: str | None = None, **kwargs):
+        return self.preprocess(images=images, return_tensors=return_tensors, **kwargs)
+
+
+__all__ = ["MolParserImageProcessor"]

model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:2204a871f7975780ead6eee8ac354de11f859c4eccc8863c3462e6f9eb110f02
+size 20037304

modeling_molparser_mobile.py

@@ -0,0 +1,204 @@
+"""MolParser Mobile model code for Hugging Face Hub remote loading."""
+
+from __future__ import annotations
+
+import torch
+import torch.nn as nn
+import timm
+from transformers import (
+    AutoConfig,
+    AutoModel,
+    PretrainedConfig,
+    PreTrainedModel,
+    VisionEncoderDecoderConfig,
+    VisionEncoderDecoderModel,
+)
+from transformers.modeling_outputs import BaseModelOutput
+
+
+MOBILE_TIMM_MODEL_NAME = "vit_tiny_r_s16_p8_384.augreg_in21k_ft_in1k"
+MOBILE_IMAGE_SIZE = 384
+MOBILE_HIDDEN_SIZE = 192
+MOBILE_PIXEL_UNSHUFFLE = 1
+
+
+class CustomEncoderConfig(PretrainedConfig):
+    model_type = "custom_timm_encoder"
+
+    def __init__(
+        self,
+        timm_model_name: str = MOBILE_TIMM_MODEL_NAME,
+        timm_pretrained: bool = False,
+        timm_output_dim: int = MOBILE_HIDDEN_SIZE,
+        pixel_unshuffle: int = MOBILE_PIXEL_UNSHUFFLE,
+        hidden_size: int = MOBILE_HIDDEN_SIZE,
+        initializer_range: float = 0.02,
+        model_input_size: int = MOBILE_IMAGE_SIZE,
+        **kwargs,
+    ):
+        self.timm_model_name = timm_model_name
+        self.timm_pretrained = timm_pretrained
+        self.timm_output_dim = timm_output_dim
+        self.pixel_unshuffle = pixel_unshuffle
+        self.hidden_size = hidden_size
+        self.initializer_range = initializer_range
+        self.model_input_size = model_input_size
+        super().__init__(**kwargs)
+        if getattr(self, "auto_map", None) is None:
+            self.auto_map = {
+                "AutoConfig": "modeling_molparser_mobile.CustomEncoderConfig",
+                "AutoModel": "modeling_molparser_mobile.CustomTimmEncoder",
+            }
+
+
+class CustomTimmEncoder(PreTrainedModel):
+    config_class = CustomEncoderConfig
+    main_input_name = "pixel_values"
+
+    def __init__(self, config: CustomEncoderConfig):
+        super().__init__(config)
+        self.config = config
+        self.pixel_unshuffle_factor = int(config.pixel_unshuffle)
+        self.unshuffle = (
+            nn.PixelUnshuffle(self.pixel_unshuffle_factor)
+            if self.pixel_unshuffle_factor > 1
+            else nn.Identity()
+        )
+
+        timm_kwargs = {
+            "pretrained": bool(config.timm_pretrained),
+            "features_only": True,
+            "num_classes": 0,
+            "global_pool": "",
+        }
+        if getattr(config, "model_input_size", None):
+            timm_kwargs["img_size"] = int(config.model_input_size)
+        self.timm_model = timm.create_model(config.timm_model_name, **timm_kwargs)
+
+        in_channels = int(config.timm_output_dim) * self.pixel_unshuffle_factor**2
+        self.use_projection = not (
+            self.pixel_unshuffle_factor == 1 and in_channels == int(config.hidden_size)
+        )
+        if self.use_projection:
+            self.projection = nn.Sequential(
+                nn.Conv2d(in_channels=in_channels, out_channels=config.hidden_size, kernel_size=1),
+                nn.GELU(),
+            )
+        else:
+            self.projection = nn.Identity()
+
+    def forward(self, pixel_values: torch.Tensor, **kwargs):
+        encoder_features = self.timm_model(pixel_values)[-1]
+        if encoder_features.ndim != 4:
+            raise ValueError(f"Expected 4D feature map, got shape={tuple(encoder_features.shape)}")
+        if encoder_features.shape[1] == self.config.timm_output_dim:
+            pass
+        elif encoder_features.shape[-1] == self.config.timm_output_dim:
+            encoder_features = encoder_features.permute(0, 3, 1, 2).contiguous()
+        else:
+            raise ValueError(
+                "Unexpected timm feature shape "
+                f"{tuple(encoder_features.shape)} for timm_output_dim={self.config.timm_output_dim}"
+            )
+
+        encoder_features = self.unshuffle(encoder_features)
+        encoder_features = self.projection(encoder_features)
+        _, channels, _, _ = encoder_features.shape
+        if channels != self.config.hidden_size:
+            raise ValueError(
+                f"Unexpected encoder channels={channels}, expected hidden_size={self.config.hidden_size}."
+            )
+        encoder_hidden_states = encoder_features.flatten(2).transpose(1, 2)
+        return BaseModelOutput(last_hidden_state=encoder_hidden_states)
+
+
+try:
+    AutoConfig.register("custom_timm_encoder", CustomEncoderConfig)
+except ValueError:
+    pass
+try:
+    AutoModel.register(CustomEncoderConfig, CustomTimmEncoder)
+except ValueError:
+    pass
+
+CustomEncoderConfig.register_for_auto_class()
+CustomTimmEncoder.register_for_auto_class("AutoModel")
+
+
+class MolParserVisionEncoderDecoderConfig(VisionEncoderDecoderConfig):
+    model_type = "molparser_vision_encoder_decoder"
+
+
+class MolParserVisionEncoderDecoderModel(VisionEncoderDecoderModel):
+    config_class = MolParserVisionEncoderDecoderConfig
+
+    def __init__(self, config=None, encoder=None, decoder=None):
+        if config is not None and not isinstance(config, self.config_class):
+            config = self.config_class.from_dict(config.to_dict())
+        super().__init__(config=config, encoder=encoder, decoder=decoder)
+
+    @classmethod
+    def get_init_context(cls, dtype, is_quantized, _is_ds_init_called, allow_all_kernels):
+        contexts = super().get_init_context(dtype, is_quantized, _is_ds_init_called, allow_all_kernels)
+        if is_quantized:
+            return contexts
+        # timm ViT initialization calls tensor.item(), which cannot run on meta tensors.
+        return [ctx for ctx in contexts if not (isinstance(ctx, torch.device) and ctx.type == "meta")]
+
+
+try:
+    AutoConfig.register("molparser_vision_encoder_decoder", MolParserVisionEncoderDecoderConfig)
+except ValueError:
+    pass
+
+MolParserVisionEncoderDecoderConfig.register_for_auto_class()
+MolParserVisionEncoderDecoderModel.register_for_auto_class("AutoModelForImageTextToText")
+
+
+def assert_mobile_config(config: MolParserVisionEncoderDecoderConfig) -> None:
+    encoder = getattr(config, "encoder", None)
+    decoder = getattr(config, "decoder", None)
+    if encoder is None or decoder is None:
+        raise ValueError("MolParser Mobile config must contain encoder and decoder sub-configs.")
+
+    checks = {
+        "encoder.timm_model_name": getattr(encoder, "timm_model_name", None) == MOBILE_TIMM_MODEL_NAME,
+        "encoder.model_input_size": int(getattr(encoder, "model_input_size", 0) or 0) == MOBILE_IMAGE_SIZE,
+        "encoder.hidden_size": int(getattr(encoder, "hidden_size", 0) or 0) == MOBILE_HIDDEN_SIZE,
+        "encoder.pixel_unshuffle": int(getattr(encoder, "pixel_unshuffle", 0) or 0) == MOBILE_PIXEL_UNSHUFFLE,
+        "decoder.d_model": int(getattr(decoder, "d_model", 0) or 0) == MOBILE_HIDDEN_SIZE,
+        "decoder.decoder_layers": int(getattr(decoder, "decoder_layers", 0) or 0) == 6,
+        "decoder.decoder_attention_heads": int(getattr(decoder, "decoder_attention_heads", 0) or 0) == 4,
+    }
+    failed = [name for name, ok in checks.items() if not ok]
+    if failed:
+        raise ValueError(
+            "This Hub package is mobile-only; non-mobile config values found: "
+            + ", ".join(failed)
+        )
+
+
+def load_molparser_mobile_model(checkpoint_path: str) -> MolParserVisionEncoderDecoderModel:
+    config = MolParserVisionEncoderDecoderConfig.from_pretrained(
+        checkpoint_path,
+        trust_remote_code=True,
+    )
+    assert_mobile_config(config)
+    if getattr(config, "encoder", None) is not None:
+        config.encoder.timm_pretrained = False
+    model = MolParserVisionEncoderDecoderModel.from_pretrained(
+        checkpoint_path,
+        config=config,
+        trust_remote_code=True,
+    )
+    return model
+
+
+__all__ = [
+    "CustomEncoderConfig",
+    "CustomTimmEncoder",
+    "MolParserVisionEncoderDecoderConfig",
+    "MolParserVisionEncoderDecoderModel",
+    "assert_mobile_config",
+    "load_molparser_mobile_model",
+]

preprocessor_config.json

@@ -0,0 +1,20 @@
+{
+  "image_processor_type": "MolParserImageProcessor",
+  "model_name": "MolParser Mobile V2",
+  "auto_map": {
+    "AutoImageProcessor": "image_processing_molparser_mobile.MolParserImageProcessor"
+  },
+  "image_size": 384,
+  "do_resize": true,
+  "do_normalize": true,
+  "image_mean": [
+    0.485,
+    0.456,
+    0.406
+  ],
+  "image_std": [
+    0.229,
+    0.224,
+    0.225
+  ]
+}

processing_molparser_mobile.py

@@ -0,0 +1,71 @@
+"""Processor that combines MolParser Mobile image preprocessing and tokenizer."""
+
+from __future__ import annotations
+
+from pathlib import Path
+from typing import Sequence
+
+from .image_processing_molparser_mobile import MolParserImageProcessor
+from .tokenization_molparser_mobile import MolParserTokenizer
+
+
+class MolParserProcessor:
+    attributes = ["image_processor", "tokenizer"]
+    image_processor_class = "MolParserImageProcessor"
+    tokenizer_class = "MolParserTokenizer"
+
+    def __init__(
+        self,
+        image_processor: MolParserImageProcessor | None = None,
+        tokenizer: MolParserTokenizer | None = None,
+    ):
+        self.image_processor = image_processor or MolParserImageProcessor()
+        self.tokenizer = tokenizer
+
+    @classmethod
+    def register_for_auto_class(cls, auto_class: str = "AutoProcessor"):
+        cls._auto_class = auto_class
+
+    @classmethod
+    def from_pretrained(cls, pretrained_model_name_or_path: str, **kwargs) -> "MolParserProcessor":
+        path = str(pretrained_model_name_or_path)
+        image_processor = MolParserImageProcessor.from_pretrained(path, **kwargs)
+        tokenizer = MolParserTokenizer.from_pretrained(path, **kwargs)
+        return cls(image_processor=image_processor, tokenizer=tokenizer)
+
+    def save_pretrained(self, save_directory: str, **kwargs):
+        Path(save_directory).mkdir(parents=True, exist_ok=True)
+        image_files = self.image_processor.save_pretrained(save_directory, **kwargs)
+        tokenizer_files = ()
+        if self.tokenizer is not None:
+            tokenizer_files = self.tokenizer.save_pretrained(save_directory, **kwargs)
+        return tuple(image_files) + tuple(tokenizer_files)
+
+    def __call__(
+        self,
+        images=None,
+        text: str | Sequence[str] | None = None,
+        return_tensors: str | None = None,
+        **kwargs,
+    ):
+        encoded = {}
+        if images is not None:
+            encoded.update(self.image_processor(images=images, return_tensors=return_tensors, **kwargs))
+        if text is not None:
+            if self.tokenizer is None:
+                raise ValueError("MolParserProcessor was created without a tokenizer.")
+            encoded.update(self.tokenizer(text, return_tensors=return_tensors, **kwargs))
+        return encoded
+
+    def decode(self, *args, **kwargs):
+        if self.tokenizer is None:
+            raise ValueError("MolParserProcessor was created without a tokenizer.")
+        return self.tokenizer.decode(*args, **kwargs)
+
+    def batch_decode(self, *args, **kwargs):
+        if self.tokenizer is None:
+            raise ValueError("MolParserProcessor was created without a tokenizer.")
+        return self.tokenizer.batch_decode(*args, **kwargs)
+
+
+__all__ = ["MolParserProcessor"]

processor_config.json

@@ -0,0 +1,7 @@
+{
+  "processor_class": "MolParserProcessor",
+  "model_name": "MolParser Mobile V2",
+  "auto_map": {
+    "AutoProcessor": "processing_molparser_mobile.MolParserProcessor"
+  }
+}

tokenization_molparser_mobile.py

@@ -0,0 +1,191 @@
+"""MolParser Mobile tokenizer for Hugging Face Hub remote loading."""
+
+from __future__ import annotations
+
+import json
+import os
+import re
+from pathlib import Path
+from typing import Dict, Iterable, List, Optional, Sequence, Union
+
+from huggingface_hub import hf_hub_download
+from transformers import PreTrainedTokenizer
+
+
+TOKENIZER_CONFIG_NAME = "tokenizer_config.json"
+VOCAB_NAME = "vocab.txt"
+
+
+class MolParserTokenizer(PreTrainedTokenizer):
+    model_input_names = ["input_ids", "attention_mask"]
+    padding_side = "right"
+
+    def __init__(
+        self,
+        vocab_list: Optional[List[str]] = None,
+        special_tokens: Optional[Dict[str, str]] = None,
+        additional_special_tokens: Optional[Sequence[str]] = None,
+        **kwargs,
+    ):
+        if vocab_list is None:
+            vocab_list = []
+        if special_tokens is None:
+            special_tokens = {}
+        if additional_special_tokens is None:
+            additional_special_tokens = []
+
+        self.special_tokens = {
+            "cls_token": special_tokens.get("cls_token", "[CLS]"),
+            "pad_token": special_tokens.get("pad_token", "[PAD]"),
+            "sep_token": special_tokens.get("sep_token", "[SEP]"),
+            "unk_token": special_tokens.get("unk_token", "[UNK]"),
+        }
+        self.additional_special_tokens = list(dict.fromkeys(additional_special_tokens))
+        self.vocab_list = list(vocab_list)
+        all_tokens = self._build_full_vocab(self.vocab_list)
+        self.vocab = {token: idx for idx, token in enumerate(all_tokens)}
+        self.ids_to_tokens = {idx: token for token, idx in self.vocab.items()}
+        self._decode_skip_tokens = set(self.special_tokens.values())
+        self._compile_pattern()
+
+        super().__init__(
+            cls_token=self.special_tokens["cls_token"],
+            pad_token=self.special_tokens["pad_token"],
+            sep_token=self.special_tokens["sep_token"],
+            unk_token=self.special_tokens["unk_token"],
+            bos_token=self.special_tokens["cls_token"],
+            eos_token=self.special_tokens["sep_token"],
+            additional_special_tokens=self.additional_special_tokens,
+            **kwargs,
+        )
+
+    def _build_full_vocab(self, vocab_list: Sequence[str]) -> List[str]:
+        ordered_tokens: List[str] = []
+        for token in list(self.special_tokens.values()) + list(vocab_list) + list(self.additional_special_tokens):
+            if token not in ordered_tokens:
+                ordered_tokens.append(token)
+        return ordered_tokens
+
+    def _compile_pattern(self) -> None:
+        multi_char_tokens = sorted(self.vocab.keys(), key=len, reverse=True)
+        pattern = "(" + "|".join(re.escape(token) for token in multi_char_tokens) + "|.)"
+        self.pattern = re.compile(pattern)
+
+    @property
+    def vocab_size(self) -> int:
+        return len(self.vocab)
+
+    def __len__(self) -> int:
+        return len(self.vocab)
+
+    def get_vocab(self) -> Dict[str, int]:
+        return dict(self.vocab)
+
+    def _tokenize(self, text: str) -> List[str]:
+        return [token for token in self.pattern.findall(str(text)) if token]
+
+    def tokenize(self, text: str, **kwargs) -> List[str]:
+        return self._tokenize(text)
+
+    def _convert_token_to_id(self, token: str) -> int:
+        return self.vocab.get(token, self.unk_token_id)
+
+    def _convert_id_to_token(self, index: int) -> str:
+        return self.ids_to_tokens.get(int(index), self.unk_token)
+
+    def convert_tokens_to_string(self, tokens: Sequence[str]) -> str:
+        return "".join(tokens)
+
+    def build_inputs_with_special_tokens(self, token_ids_0, token_ids_1=None):
+        if token_ids_1 is None:
+            return list(token_ids_0)
+        return list(token_ids_0) + list(token_ids_1)
+
+    def encode(self, text: str, add_special_tokens: bool = False, **kwargs) -> List[int]:
+        token_ids = [self._convert_token_to_id(token) for token in self._tokenize(text)]
+        if add_special_tokens:
+            return [self.bos_token_id] + token_ids + [self.eos_token_id]
+        return token_ids
+
+    def decode(self, token_ids: Iterable[int], skip_special_tokens: bool = False, **kwargs) -> str:
+        tokens = [self._convert_id_to_token(idx) for idx in token_ids]
+        if skip_special_tokens:
+            # Match deploy/tokenizer.py: keep MolParser business tokens such as
+            # <sep>, <a>, </a>, <r>, </r>, <c>, </c>, and |Sg:n|.
+            tokens = [token for token in tokens if token not in self._decode_skip_tokens]
+        return "".join(tokens)
+
+    def batch_encode(self, texts: Sequence[str], add_special_tokens: bool = False) -> List[List[int]]:
+        return [self.encode(text, add_special_tokens=add_special_tokens) for text in texts]
+
+    def batch_decode(
+        self,
+        sequences: Sequence[Sequence[int]],
+        skip_special_tokens: bool = False,
+        **kwargs,
+    ) -> List[str]:
+        return [self.decode(ids, skip_special_tokens=skip_special_tokens, **kwargs) for ids in sequences]
+
+    def to_dict(self) -> Dict[str, object]:
+        return {
+            "vocab_list": self.vocab_list,
+            "special_tokens": self.special_tokens,
+            "additional_special_tokens": self.additional_special_tokens,
+            "tokenizer_class": self.__class__.__name__,
+            "auto_map": {
+                "AutoTokenizer": [
+                    "tokenization_molparser_mobile.MolParserTokenizer",
+                    None,
+                ]
+            },
+        }
+
+    @classmethod
+    def from_dict(cls, config: Dict[str, object]) -> "MolParserTokenizer":
+        return cls(
+            vocab_list=list(config["vocab_list"]),
+            special_tokens=dict(config["special_tokens"]),
+            additional_special_tokens=list(config.get("additional_special_tokens", [])),
+        )
+
+    def save_vocabulary(self, save_directory: str, filename_prefix: Optional[str] = None):
+        path = Path(save_directory)
+        path.mkdir(parents=True, exist_ok=True)
+        name = f"{filename_prefix}-{VOCAB_NAME}" if filename_prefix else VOCAB_NAME
+        vocab_path = path / name
+        vocab_path.write_text("\n".join(self.vocab_list) + "\n", encoding="utf-8")
+        return (str(vocab_path),)
+
+    def save_pretrained(self, save_directory: str, **kwargs):
+        os.makedirs(save_directory, exist_ok=True)
+        config_path = os.path.join(save_directory, TOKENIZER_CONFIG_NAME)
+        with open(config_path, "w", encoding="utf-8") as f:
+            json.dump(self.to_dict(), f, ensure_ascii=False, indent=2)
+        vocab_files = self.save_vocabulary(save_directory)
+        return (config_path, *vocab_files)
+
+    @classmethod
+    def from_pretrained(cls, pretrained_model_name_or_path: str, *args, **kwargs) -> "MolParserTokenizer":
+        config_path = Path(pretrained_model_name_or_path)
+        if config_path.is_dir():
+            config_path = config_path / TOKENIZER_CONFIG_NAME
+        elif config_path.is_file():
+            pass
+        else:
+            config_path = Path(
+                hf_hub_download(
+                    repo_id=str(pretrained_model_name_or_path),
+                    filename=TOKENIZER_CONFIG_NAME,
+                    repo_type=kwargs.get("repo_type"),
+                    revision=kwargs.get("revision"),
+                    cache_dir=kwargs.get("cache_dir"),
+                    token=kwargs.get("token"),
+                    local_files_only=kwargs.get("local_files_only", False),
+                )
+            )
+        with open(config_path, "r", encoding="utf-8") as f:
+            config = json.load(f)
+        return cls.from_dict(config)
+
+
+__all__ = ["MolParserTokenizer"]

tokenizer_config.json

@@ -0,0 +1,429 @@
+{
+  "vocab_list": [
+    "!",
+    "\"",
+    "#",
+    "$",
+    "%",
+    "&",
+    "'",
+    "(",
+    ")",
+    "*",
+    "+",
+    ",",
+    "-",
+    ".",
+    "/",
+    "0",
+    "1",
+    "2",
+    "3",
+    "4",
+    "5",
+    "6",
+    "7",
+    "8",
+    "9",
+    "10",
+    "11",
+    "12",
+    "13",
+    "14",
+    "15",
+    "16",
+    "17",
+    "18",
+    "19",
+    "20",
+    "21",
+    "22",
+    "23",
+    "24",
+    "25",
+    "26",
+    "27",
+    "28",
+    "29",
+    "30",
+    "31",
+    "32",
+    "33",
+    "34",
+    "35",
+    "36",
+    "37",
+    "38",
+    "39",
+    "40",
+    "41",
+    "42",
+    "43",
+    "44",
+    "45",
+    "46",
+    "47",
+    "48",
+    "49",
+    "50",
+    "51",
+    "52",
+    "53",
+    "54",
+    "55",
+    "56",
+    "57",
+    "58",
+    "59",
+    "60",
+    "61",
+    "62",
+    "63",
+    "64",
+    "65",
+    "66",
+    "67",
+    "68",
+    "69",
+    "70",
+    "71",
+    "72",
+    "73",
+    "74",
+    "75",
+    "76",
+    "77",
+    "78",
+    "79",
+    "80",
+    "81",
+    "82",
+    "83",
+    "84",
+    "85",
+    "86",
+    "87",
+    "88",
+    "89",
+    "90",
+    "91",
+    "92",
+    "93",
+    "94",
+    "95",
+    "96",
+    "97",
+    "98",
+    "99",
+    "100",
+    "101",
+    "102",
+    "103",
+    "104",
+    "105",
+    "106",
+    "107",
+    "108",
+    "109",
+    "110",
+    "111",
+    "112",
+    "113",
+    "114",
+    "115",
+    "116",
+    "117",
+    "118",
+    "119",
+    "120",
+    "121",
+    "122",
+    "123",
+    "124",
+    "125",
+    "126",
+    "127",
+    "128",
+    "129",
+    "130",
+    "131",
+    "132",
+    "133",
+    "134",
+    "135",
+    "136",
+    "137",
+    "138",
+    "139",
+    "140",
+    "141",
+    "142",
+    "143",
+    "144",
+    "145",
+    "146",
+    "147",
+    "148",
+    "149",
+    "150",
+    "151",
+    "152",
+    "153",
+    "154",
+    "155",
+    "156",
+    "157",
+    "158",
+    "159",
+    "160",
+    "161",
+    "162",
+    "163",
+    "164",
+    "165",
+    "166",
+    "167",
+    "168",
+    "169",
+    "170",
+    "171",
+    "172",
+    "173",
+    "174",
+    "175",
+    "176",
+    "177",
+    "178",
+    "179",
+    "180",
+    "181",
+    "182",
+    "183",
+    "184",
+    "185",
+    "186",
+    "187",
+    "188",
+    "189",
+    "190",
+    "191",
+    "192",
+    "193",
+    "194",
+    "195",
+    "196",
+    "197",
+    "198",
+    "199",
+    "200",
+    "201",
+    "202",
+    "203",
+    "204",
+    "205",
+    "206",
+    "207",
+    "208",
+    "209",
+    "210",
+    "211",
+    "212",
+    "213",
+    "214",
+    "215",
+    "216",
+    "217",
+    "218",
+    "219",
+    "220",
+    "221",
+    "222",
+    "223",
+    "224",
+    "225",
+    "226",
+    "227",
+    "228",
+    "229",
+    "230",
+    "231",
+    "232",
+    "233",
+    "234",
+    "235",
+    "236",
+    "237",
+    "238",
+    "239",
+    "240",
+    "241",
+    "242",
+    "243",
+    "244",
+    "245",
+    "246",
+    "247",
+    "248",
+    "249",
+    "250",
+    "251",
+    "252",
+    "253",
+    "254",
+    "255",
+    ":",
+    ";",
+    "<",
+    "=",
+    ">",
+    "?",
+    "@@",
+    "@",
+    "A",
+    "B",
+    "C",
+    "D",
+    "E",
+    "F",
+    "G",
+    "H",
+    "I",
+    "J",
+    "K",
+    "L",
+    "M",
+    "N",
+    "O",
+    "P",
+    "Q",
+    "R",
+    "S",
+    "T",
+    "U",
+    "V",
+    "W",
+    "X",
+    "Y",
+    "Z",
+    "[",
+    "\\",
+    "]",
+    "^",
+    "_",
+    "`",
+    "a",
+    "b",
+    "c",
+    "d",
+    "e",
+    "f",
+    "g",
+    "h",
+    "i",
+    "j",
+    "k",
+    "l",
+    "m",
+    "n",
+    "o",
+    "p",
+    "q",
+    "r",
+    "s",
+    "t",
+    "u",
+    "v",
+    "w",
+    "x",
+    "y",
+    "z",
+    "{",
+    "|",
+    "}",
+    "~",
+    "Ag",
+    "Al",
+    "As",
+    "Au",
+    "Br",
+    "Ca",
+    "Cl",
+    "Cr",
+    "Cu",
+    "Fe",
+    "Gd",
+    "Hg",
+    "Li",
+    "Mg",
+    "Mn",
+    "Na",
+    "Ni",
+    "Pb",
+    "Pt",
+    "Sb",
+    "Se",
+    "Si",
+    "Sn",
+    "Ti",
+    "Zn",
+    "Zr",
+    "2H",
+    "3H",
+    "->",
+    "<-",
+    "ball",
+    "grey",
+    "black",
+    "red",
+    "green",
+    "blue",
+    "yellow",
+    "purple",
+    "orange",
+    "pink",
+    "brown",
+    "DNA",
+    "RNA",
+    "star",
+    "capsule",
+    "other",
+    "radioactive",
+    "Sg:",
+    "Ra",
+    "Sa"
+  ],
+  "special_tokens": {
+    "cls_token": "[CLS]",
+    "pad_token": "[PAD]",
+    "sep_token": "[SEP]",
+    "unk_token": "[UNK]"
+  },
+  "additional_special_tokens": [
+    "<sep>",
+    "<dum>",
+    "<id>",
+    "<a>",
+    "</a>",
+    "<r>",
+    "</r>",
+    "<c>",
+    "</c>",
+    "<d>",
+    "</d>",
+    "<s>",
+    "</s>",
+    "<g>",
+    "</g>",
+    "<v>",
+    "</v>",
+    "<x>",
+    "</x>"
+  ],
+  "tokenizer_class": "MolParserTokenizer",
+  "auto_map": {
+    "AutoTokenizer": [
+      "tokenization_molparser_mobile.MolParserTokenizer",
+      null
+    ]
+  }
+}

vocab.txt

@@ -0,0 +1,391 @@
+!
+"
+#
+$
+%
+&
+'
+(
+)
+*
++
+,
+-
+.
+/
+0
+1
+2
+3
+4
+5
+6
+7
+8
+9
+10
+11
+12
+13
+14
+15
+16
+17
+18
+19
+20
+21
+22
+23
+24
+25
+26
+27
+28
+29
+30
+31
+32
+33
+34
+35
+36
+37
+38
+39
+40
+41
+42
+43
+44
+45
+46
+47
+48
+49
+50
+51
+52
+53
+54
+55
+56
+57
+58
+59
+60
+61
+62
+63
+64
+65
+66
+67
+68
+69
+70
+71
+72
+73
+74
+75
+76
+77
+78
+79
+80
+81
+82
+83
+84
+85
+86
+87
+88
+89
+90
+91
+92
+93
+94
+95
+96
+97
+98
+99
+100
+101
+102
+103
+104
+105
+106
+107
+108
+109
+110
+111
+112
+113
+114
+115
+116
+117
+118
+119
+120
+121
+122
+123
+124
+125
+126
+127
+128
+129
+130
+131
+132
+133
+134
+135
+136
+137
+138
+139
+140
+141
+142
+143
+144
+145
+146
+147
+148
+149
+150
+151
+152
+153
+154
+155
+156
+157
+158
+159
+160
+161
+162
+163
+164
+165
+166
+167
+168
+169
+170
+171
+172
+173
+174
+175
+176
+177
+178
+179
+180
+181
+182
+183
+184
+185
+186
+187
+188
+189
+190
+191
+192
+193
+194
+195
+196
+197
+198
+199
+200
+201
+202
+203
+204
+205
+206
+207
+208
+209
+210
+211
+212
+213
+214
+215
+216
+217
+218
+219
+220
+221
+222
+223
+224
+225
+226
+227
+228
+229
+230
+231
+232
+233
+234
+235
+236
+237
+238
+239
+240
+241
+242
+243
+244
+245
+246
+247
+248
+249
+250
+251
+252
+253
+254
+255
+:
+;
+<
+=
+>
+?
+@@
+@
+A
+B
+C
+D
+E
+F
+G
+H
+I
+J
+K
+L
+M
+N
+O
+P
+Q
+R
+S
+T
+U
+V
+W
+X
+Y
+Z
+[
+\
+]
+^
+_
+`
+a
+b
+c
+d
+e
+f
+g
+h
+i
+j
+k
+l
+m
+n
+o
+p
+q
+r
+s
+t
+u
+v
+w
+x
+y
+z
+{
+|
+}
+~
+Ag
+Al
+As
+Au
+Br
+Ca
+Cl
+Cr
+Cu
+Fe
+Gd
+Hg
+Li
+Mg
+Mn
+Na
+Ni
+Pb
+Pt
+Sb
+Se
+Si
+Sn
+Ti
+Zn
+Zr
+2H
+3H
+->
+<-
+ball
+grey
+black
+red
+green
+blue
+yellow
+purple
+orange
+pink
+brown
+DNA
+RNA
+star
+capsule
+other
+radioactive
+Sg:
+Ra
+Sa