Delete files model_files/README.md model_files/config.json model_files/generation_config.json model_files/image_processing_molparser_mobile.py model_files/modeling_molparser_mobile.py model_files/preprocessor_config.json model_files/processing_molparser_mobile.py model_files/processor_config.json model_files/tokenization_molparser_mobile.py model_files/tokenizer_config.json model_files/vocab.txt with huggingface_hub

model_files/README.md

@@ -1,127 +0,0 @@
----
-library_name: transformers
-pipeline_tag: image-to-text
-tags:
-- chemistry
-- image-to-text
-datasets:
-- UniParser/MolParser-7M
-- UniParser/MolGallery
-license: cc-by-nc-sa-4.0
----
-
-# MolParser Mobile
-
-<p align="center">
-💻 <a href="https://github.com/dptech-corp/MolParser">Github</a> |
-📄 <a>Report (Coming soon...)</a> |
-🚀 <a href="https://ocsr.dp.tech/">Demo</a>
-</p>
-
-**MolParser-Mobile** is the next-generation lightweight version of **MolParser**, designed for **ultrafast Optical Chemical Structure Recognition (OCSR)**. It converts molecular structure images into **E-SMILES** representations with significantly higher throughput while maintaining state-of-the-art recognition accuracy.
-
-Compared with MolParser 1.0, MolParser-Mobile reduces the model size by **95.4%** (216M → **9.98M** parameters) and increases inference throughput by **38×**, reaching **1,520 molecules/second** on a single NVIDIA RTX 4090D GPU. Despite its compact architecture, it achieves better accuracy on multiple real-world OCSR benchmarks, making it suitable for large-scale chemical literature mining and industrial deployment.
-
-
-## 🚀 Highlights
-
-* **38× faster** than MolParser 1.0 (**1,520 Mol/s** on RTX 4090D)
-* **9.98M parameters** (**95.4% smaller** than MolParser 1.0)
-* **State-of-the-art** accuracy on challenging real-world OCSR benchmarks (BioVista)
-* Designed for **web-scale** chemical literature mining
-* Converts molecular images directly into **E-SMILES** strings
-
-
-## 📊 Performance
-
-| Model                |         Parameters |     Throughput (RTX 4090D) |   Uni-Parser Bench |           BioVista |    WildMol-10k |
-| -------------------- | -----------------: | -------------------------: | -----------------: | -----------------: | -------------: |
-| Gemma4-31B           |                31B |                        OOM |              0.073 |              0.118 |          0.232 |
-| Qwen3.5-397B-A17B    |               397B |                        OOM |              0.272 |              0.347 |          0.303 |
-| MolParser 1.0        |               216M |                 39.8 Mol/s |              0.800 |              0.703 |      **0.769** |
-| **MolParser-Mobile** | **9.98M (-95.4%)** | **1,520 Mol/s (+3819.1%)** | **0.823 (+0.023)** | **0.801 (+0.098)** | 0.734 (-0.035) |
-
-
-## ⚡ Usage
-
-### Option 1. MolParser Library (Recommended)
-
-The [MolParser library](https://github.com/dptech-corp/MolParser) provides a convenient interface for molecule detection and recognition.
-
-Clone the repository and install the package:
-
-```bash
-git clone https://github.com/dptech-corp/MolParser.git
-cd MolParser
-pip install -e .
-```
-
-Then run:
-
-```python
-from molparser import MolParser
-
-parser = MolParser()
-
-result = parser.parse(
-    "mol.png",
-    rec_only=True,   # Recognition only
-)
-```
-
-### Option 2. 🤗 Transformers
-Load MolParser-Mobile directly with the Hugging Face transformers library.
-
-```python
-import torch
-from PIL import Image
-from transformers import AutoModelForImageTextToText, AutoProcessor
-
-repo_id = "UniParser/MolParser-Mobile"
-device = "cuda" if torch.cuda.is_available() else "cpu"
-dtype = torch.float16 if device == "cuda" else torch.float32
-
-processor = AutoProcessor.from_pretrained(repo_id, trust_remote_code=True)
-model = AutoModelForImageTextToText.from_pretrained(
-    repo_id,
-    dtype=dtype,
-    trust_remote_code=True,
-).to(device).eval()
-
-image = Image.open("mol.png").convert("RGB")
-inputs = processor(images=image, return_tensors="pt")
-inputs = {k: v.to(device, dtype=dtype) for k, v in inputs.items()}
-
-output_ids = model.generate(**inputs, max_length=256, num_beams=1, do_sample=False)
-caption = processor.batch_decode(output_ids, skip_special_tokens=True)[0]
-print(caption)
-```
-
-
-## 📜 License
-
-### MolParser-Mobile Weight
-
-The **MolParser-Mobile model weights** are provided for **non-commercial use only**.
-
-For commercial licensing, please contact **fangxi@dp.tech** or open a discussion on Hugging Face.
-
-### MolParser Github Repo
-
-The **MolParser** library (including E-SMILES post-processing and rendering) is available at https://github.com/dptech-corp/MolParser and is licensed under the **Apache License 2.0**, which permits commercial use, modification, and distribution, provided that the license and copyright notices are retained.
-
-**Note:** Model weights, datasets, and third-party dependencies are subject to their respective licenses.
-
-## 📖 Citation
-
-If you use this datasets in your work, please cite:
-
-```
-@inproceedings{fang2025molparser,
-  title={Molparser: End-to-end visual recognition of molecule structures in the wild},
-  author={Fang, Xi and Wang, Jiankun and Cai, Xiaochen and Chen, Shangqian and Yang, Shuwen and Tao, Haoyi and Wang, Nan and Yao, Lin and Zhang, Linfeng and Ke, Guolin},
-  booktitle={Proceedings of the IEEE/CVF International Conference on Computer Vision},
-  pages={24528--24538},
-  year={2025}
-}
-```

model_files/config.json

@@ -1,101 +0,0 @@
-{
-  "architectures": [
-    "MolParserVisionEncoderDecoderModel"
-  ],
-  "auto_map": {
-    "AutoConfig": "modeling_molparser_mobile.MolParserVisionEncoderDecoderConfig",
-    "AutoModelForImageTextToText": "modeling_molparser_mobile.MolParserVisionEncoderDecoderModel"
-  },
-  "decoder": {
-    "_name_or_path": "",
-    "activation_dropout": 0.0,
-    "activation_function": "gelu",
-    "add_cross_attention": true,
-    "architectures": null,
-    "attention_dropout": 0.0,
-    "bos_token_id": 0,
-    "chunk_size_feed_forward": 0,
-    "classifier_dropout": 0.0,
-    "d_model": 192,
-    "decoder_attention_heads": 4,
-    "decoder_ffn_dim": 768,
-    "decoder_layerdrop": 0.0,
-    "decoder_layers": 6,
-    "decoder_start_token_id": 2,
-    "dropout": 0.1,
-    "dtype": "float16",
-    "encoder_attention_heads": 4,
-    "encoder_ffn_dim": 768,
-    "encoder_hidden_size": 192,
-    "encoder_layerdrop": 0.0,
-    "encoder_layers": 0,
-    "eos_token_id": 2,
-    "forced_eos_token_id": 2,
-    "id2label": {
-      "0": "LABEL_0",
-      "1": "LABEL_1",
-      "2": "LABEL_2"
-    },
-    "init_std": 0.02,
-    "is_decoder": true,
-    "is_encoder_decoder": false,
-    "label2id": {
-      "LABEL_0": 0,
-      "LABEL_1": 1,
-      "LABEL_2": 2
-    },
-    "max_position_embeddings": 1024,
-    "model_type": "bart",
-    "output_attentions": false,
-    "output_hidden_states": false,
-    "pad_token_id": 1,
-    "problem_type": null,
-    "return_dict": true,
-    "scale_embedding": false,
-    "tie_word_embeddings": true,
-    "use_cache": true,
-    "vocab_size": 385
-  },
-  "decoder_start_token_id": 0,
-  "dtype": "float16",
-  "encoder": {
-    "_name_or_path": "",
-    "architectures": null,
-    "auto_map": {
-      "AutoConfig": "modeling_molparser_mobile.CustomEncoderConfig",
-      "AutoModel": "modeling_molparser_mobile.CustomTimmEncoder"
-    },
-    "chunk_size_feed_forward": 0,
-    "dtype": "float16",
-    "hidden_size": 192,
-    "id2label": {
-      "0": "LABEL_0",
-      "1": "LABEL_1"
-    },
-    "initializer_range": 0.02,
-    "is_encoder_decoder": false,
-    "label2id": {
-      "LABEL_0": 0,
-      "LABEL_1": 1
-    },
-    "model_input_size": 224,
-    "model_type": "custom_timm_encoder",
-    "output_attentions": false,
-    "output_hidden_states": false,
-    "pixel_unshuffle": 1,
-    "problem_type": null,
-    "return_dict": true,
-    "timm_model_name": "vit_tiny_r_s16_p8_224.augreg_in21k_ft_in1k",
-    "timm_output_dim": 192,
-    "timm_pretrained": false
-  },
-  "is_encoder_decoder": true,
-  "model_type": "molparser_vision_encoder_decoder",
-  "pad_token_id": 1,
-  "tie_word_embeddings": false,
-  "transformers_version": "5.4.0",
-  "use_cache": true,
-  "vocab_size": 385,
-  "model_name": "MolParser Mobile",
-  "torch_dtype": "float16"
-}

model_files/generation_config.json

@@ -1,39 +0,0 @@
-{
-  "_from_model_config": false,
-  "assistant_confidence_threshold": 0.4,
-  "assistant_lookbehind": 10,
-  "bos_token_id": 0,
-  "decoder_start_token_id": 0,
-  "diversity_penalty": 0.0,
-  "do_sample": false,
-  "early_stopping": false,
-  "encoder_no_repeat_ngram_size": 0,
-  "encoder_repetition_penalty": 1.0,
-  "eos_token_id": 2,
-  "epsilon_cutoff": 0.0,
-  "eta_cutoff": 0.0,
-  "forced_eos_token_id": 2,
-  "length_penalty": 1.0,
-  "max_length": 256,
-  "min_length": 0,
-  "no_repeat_ngram_size": 0,
-  "num_assistant_tokens": 20,
-  "num_assistant_tokens_schedule": "constant",
-  "num_beam_groups": 1,
-  "num_beams": 1,
-  "num_return_sequences": 1,
-  "output_attentions": false,
-  "output_hidden_states": false,
-  "output_scores": false,
-  "pad_token_id": 1,
-  "remove_invalid_values": false,
-  "repetition_penalty": 1.0,
-  "return_dict_in_generate": false,
-  "target_lookbehind": 10,
-  "temperature": 1.0,
-  "top_k": 50,
-  "top_p": 1.0,
-  "transformers_version": "5.4.0",
-  "typical_p": 1.0,
-  "use_cache": true
-}

model_files/image_processing_molparser_mobile.py

@@ -1,95 +0,0 @@
-"""Image processor for MolParser Mobile."""
-
-from __future__ import annotations
-
-from typing import List, Sequence, Union
-
-import cv2
-import numpy as np
-import torch
-from PIL import Image
-from transformers import BaseImageProcessor
-from transformers.feature_extraction_utils import BatchFeature
-
-
-ImageInput = Union[str, Image.Image, np.ndarray, torch.Tensor]
-
-
-class MolParserImageProcessor(BaseImageProcessor):
-    model_input_names = ["pixel_values"]
-
-    def __init__(
-        self,
-        image_size: int = 224,
-        do_resize: bool = True,
-        do_normalize: bool = True,
-        image_mean: Sequence[float] = (0.485, 0.456, 0.406),
-        image_std: Sequence[float] = (0.229, 0.224, 0.225),
-        **kwargs,
-    ):
-        super().__init__(**kwargs)
-        self.image_size = int(image_size)
-        self.do_resize = bool(do_resize)
-        self.do_normalize = bool(do_normalize)
-        self.image_mean = list(image_mean)
-        self.image_std = list(image_std)
-
-    @property
-    def size(self):
-        return {"height": self.image_size, "width": self.image_size}
-
-    def _to_pil(self, image: ImageInput) -> Image.Image:
-        if isinstance(image, Image.Image):
-            return image.convert("RGB")
-        if isinstance(image, str):
-            return Image.open(image).convert("RGB")
-        if isinstance(image, torch.Tensor):
-            tensor = image.detach().cpu()
-            if tensor.ndim == 3 and tensor.shape[0] in {1, 3, 4}:
-                tensor = tensor.permute(1, 2, 0)
-            array = tensor.numpy()
-        else:
-            array = np.asarray(image)
-        if array.dtype != np.uint8:
-            if array.max() <= 1.0:
-                array = array * 255.0
-            array = np.clip(np.rint(array), 0, 255).astype(np.uint8)
-        if array.ndim == 2:
-            return Image.fromarray(array, mode="L").convert("RGB")
-        if array.shape[-1] == 4:
-            return Image.fromarray(array, mode="RGBA").convert("RGB")
-        return Image.fromarray(array).convert("RGB")
-
-    def _preprocess_one(self, image: ImageInput) -> np.ndarray:
-        pil_image = self._to_pil(image)
-        array = np.asarray(pil_image).astype(np.uint8)
-        if self.do_resize:
-            # Match deploy/transform.py: albumentations.Resize defaults to OpenCV INTER_LINEAR.
-            array = cv2.resize(array, (self.image_size, self.image_size), interpolation=cv2.INTER_LINEAR)
-        array = array.astype(np.float32) / 255.0
-        if self.do_normalize:
-            mean = np.asarray(self.image_mean, dtype=np.float32).reshape(1, 1, 3)
-            std = np.asarray(self.image_std, dtype=np.float32).reshape(1, 1, 3)
-            array = (array - mean) / std
-        return np.transpose(array, (2, 0, 1))
-
-    def preprocess(
-        self,
-        images: Union[ImageInput, Sequence[ImageInput]],
-        return_tensors: str | None = None,
-        **kwargs,
-    ) -> BatchFeature:
-        if not isinstance(images, (list, tuple)):
-            images = [images]
-        pixel_values: List[np.ndarray] = [self._preprocess_one(image) for image in images]
-        data = {"pixel_values": np.stack(pixel_values, axis=0)}
-        encoded = BatchFeature(data=data)
-        if return_tensors is not None:
-            encoded = encoded.convert_to_tensors(return_tensors)
-        return encoded
-
-    def __call__(self, images: Union[ImageInput, Sequence[ImageInput]], return_tensors: str | None = None, **kwargs):
-        return self.preprocess(images=images, return_tensors=return_tensors, **kwargs)
-
-
-__all__ = ["MolParserImageProcessor"]

model_files/modeling_molparser_mobile.py

@@ -1,204 +0,0 @@
-"""MolParser Mobile model code for Hugging Face Hub remote loading."""
-
-from __future__ import annotations
-
-import torch
-import torch.nn as nn
-import timm
-from transformers import (
-    AutoConfig,
-    AutoModel,
-    PretrainedConfig,
-    PreTrainedModel,
-    VisionEncoderDecoderConfig,
-    VisionEncoderDecoderModel,
-)
-from transformers.modeling_outputs import BaseModelOutput
-
-
-MOBILE_TIMM_MODEL_NAME = "vit_tiny_r_s16_p8_224.augreg_in21k_ft_in1k"
-MOBILE_IMAGE_SIZE = 224
-MOBILE_HIDDEN_SIZE = 192
-MOBILE_PIXEL_UNSHUFFLE = 1
-
-
-class CustomEncoderConfig(PretrainedConfig):
-    model_type = "custom_timm_encoder"
-
-    def __init__(
-        self,
-        timm_model_name: str = MOBILE_TIMM_MODEL_NAME,
-        timm_pretrained: bool = False,
-        timm_output_dim: int = MOBILE_HIDDEN_SIZE,
-        pixel_unshuffle: int = MOBILE_PIXEL_UNSHUFFLE,
-        hidden_size: int = MOBILE_HIDDEN_SIZE,
-        initializer_range: float = 0.02,
-        model_input_size: int = MOBILE_IMAGE_SIZE,
-        **kwargs,
-    ):
-        self.timm_model_name = timm_model_name
-        self.timm_pretrained = timm_pretrained
-        self.timm_output_dim = timm_output_dim
-        self.pixel_unshuffle = pixel_unshuffle
-        self.hidden_size = hidden_size
-        self.initializer_range = initializer_range
-        self.model_input_size = model_input_size
-        super().__init__(**kwargs)
-        if getattr(self, "auto_map", None) is None:
-            self.auto_map = {
-                "AutoConfig": "modeling_molparser_mobile.CustomEncoderConfig",
-                "AutoModel": "modeling_molparser_mobile.CustomTimmEncoder",
-            }
-
-
-class CustomTimmEncoder(PreTrainedModel):
-    config_class = CustomEncoderConfig
-    main_input_name = "pixel_values"
-
-    def __init__(self, config: CustomEncoderConfig):
-        super().__init__(config)
-        self.config = config
-        self.pixel_unshuffle_factor = int(config.pixel_unshuffle)
-        self.unshuffle = (
-            nn.PixelUnshuffle(self.pixel_unshuffle_factor)
-            if self.pixel_unshuffle_factor > 1
-            else nn.Identity()
-        )
-
-        timm_kwargs = {
-            "pretrained": bool(config.timm_pretrained),
-            "features_only": True,
-            "num_classes": 0,
-            "global_pool": "",
-        }
-        if getattr(config, "model_input_size", None):
-            timm_kwargs["img_size"] = int(config.model_input_size)
-        self.timm_model = timm.create_model(config.timm_model_name, **timm_kwargs)
-
-        in_channels = int(config.timm_output_dim) * self.pixel_unshuffle_factor**2
-        self.use_projection = not (
-            self.pixel_unshuffle_factor == 1 and in_channels == int(config.hidden_size)
-        )
-        if self.use_projection:
-            self.projection = nn.Sequential(
-                nn.Conv2d(in_channels=in_channels, out_channels=config.hidden_size, kernel_size=1),
-                nn.GELU(),
-            )
-        else:
-            self.projection = nn.Identity()
-
-    def forward(self, pixel_values: torch.Tensor, **kwargs):
-        encoder_features = self.timm_model(pixel_values)[-1]
-        if encoder_features.ndim != 4:
-            raise ValueError(f"Expected 4D feature map, got shape={tuple(encoder_features.shape)}")
-        if encoder_features.shape[1] == self.config.timm_output_dim:
-            pass
-        elif encoder_features.shape[-1] == self.config.timm_output_dim:
-            encoder_features = encoder_features.permute(0, 3, 1, 2).contiguous()
-        else:
-            raise ValueError(
-                "Unexpected timm feature shape "
-                f"{tuple(encoder_features.shape)} for timm_output_dim={self.config.timm_output_dim}"
-            )
-
-        encoder_features = self.unshuffle(encoder_features)
-        encoder_features = self.projection(encoder_features)
-        _, channels, _, _ = encoder_features.shape
-        if channels != self.config.hidden_size:
-            raise ValueError(
-                f"Unexpected encoder channels={channels}, expected hidden_size={self.config.hidden_size}."
-            )
-        encoder_hidden_states = encoder_features.flatten(2).transpose(1, 2)
-        return BaseModelOutput(last_hidden_state=encoder_hidden_states)
-
-
-try:
-    AutoConfig.register("custom_timm_encoder", CustomEncoderConfig)
-except ValueError:
-    pass
-try:
-    AutoModel.register(CustomEncoderConfig, CustomTimmEncoder)
-except ValueError:
-    pass
-
-CustomEncoderConfig.register_for_auto_class()
-CustomTimmEncoder.register_for_auto_class("AutoModel")
-
-
-class MolParserVisionEncoderDecoderConfig(VisionEncoderDecoderConfig):
-    model_type = "molparser_vision_encoder_decoder"
-
-
-class MolParserVisionEncoderDecoderModel(VisionEncoderDecoderModel):
-    config_class = MolParserVisionEncoderDecoderConfig
-
-    def __init__(self, config=None, encoder=None, decoder=None):
-        if config is not None and not isinstance(config, self.config_class):
-            config = self.config_class.from_dict(config.to_dict())
-        super().__init__(config=config, encoder=encoder, decoder=decoder)
-
-    @classmethod
-    def get_init_context(cls, dtype, is_quantized, _is_ds_init_called, allow_all_kernels):
-        contexts = super().get_init_context(dtype, is_quantized, _is_ds_init_called, allow_all_kernels)
-        if is_quantized:
-            return contexts
-        # timm ViT initialization calls tensor.item(), which cannot run on meta tensors.
-        return [ctx for ctx in contexts if not (isinstance(ctx, torch.device) and ctx.type == "meta")]
-
-
-try:
-    AutoConfig.register("molparser_vision_encoder_decoder", MolParserVisionEncoderDecoderConfig)
-except ValueError:
-    pass
-
-MolParserVisionEncoderDecoderConfig.register_for_auto_class()
-MolParserVisionEncoderDecoderModel.register_for_auto_class("AutoModelForImageTextToText")
-
-
-def assert_mobile_config(config: MolParserVisionEncoderDecoderConfig) -> None:
-    encoder = getattr(config, "encoder", None)
-    decoder = getattr(config, "decoder", None)
-    if encoder is None or decoder is None:
-        raise ValueError("MolParser Mobile config must contain encoder and decoder sub-configs.")
-
-    checks = {
-        "encoder.timm_model_name": getattr(encoder, "timm_model_name", None) == MOBILE_TIMM_MODEL_NAME,
-        "encoder.model_input_size": int(getattr(encoder, "model_input_size", 0) or 0) == MOBILE_IMAGE_SIZE,
-        "encoder.hidden_size": int(getattr(encoder, "hidden_size", 0) or 0) == MOBILE_HIDDEN_SIZE,
-        "encoder.pixel_unshuffle": int(getattr(encoder, "pixel_unshuffle", 0) or 0) == MOBILE_PIXEL_UNSHUFFLE,
-        "decoder.d_model": int(getattr(decoder, "d_model", 0) or 0) == MOBILE_HIDDEN_SIZE,
-        "decoder.decoder_layers": int(getattr(decoder, "decoder_layers", 0) or 0) == 6,
-        "decoder.decoder_attention_heads": int(getattr(decoder, "decoder_attention_heads", 0) or 0) == 4,
-    }
-    failed = [name for name, ok in checks.items() if not ok]
-    if failed:
-        raise ValueError(
-            "This Hub package is mobile-only; non-mobile config values found: "
-            + ", ".join(failed)
-        )
-
-
-def load_molparser_mobile_model(checkpoint_path: str) -> MolParserVisionEncoderDecoderModel:
-    config = MolParserVisionEncoderDecoderConfig.from_pretrained(
-        checkpoint_path,
-        trust_remote_code=True,
-    )
-    assert_mobile_config(config)
-    if getattr(config, "encoder", None) is not None:
-        config.encoder.timm_pretrained = False
-    model = MolParserVisionEncoderDecoderModel.from_pretrained(
-        checkpoint_path,
-        config=config,
-        trust_remote_code=True,
-    )
-    return model
-
-
-__all__ = [
-    "CustomEncoderConfig",
-    "CustomTimmEncoder",
-    "MolParserVisionEncoderDecoderConfig",
-    "MolParserVisionEncoderDecoderModel",
-    "assert_mobile_config",
-    "load_molparser_mobile_model",
-]

model_files/preprocessor_config.json

@@ -1,20 +0,0 @@
-{
-  "image_processor_type": "MolParserImageProcessor",
-  "model_name": "MolParser Mobile",
-  "auto_map": {
-    "AutoImageProcessor": "image_processing_molparser_mobile.MolParserImageProcessor"
-  },
-  "image_size": 224,
-  "do_resize": true,
-  "do_normalize": true,
-  "image_mean": [
-    0.485,
-    0.456,
-    0.406
-  ],
-  "image_std": [
-    0.229,
-    0.224,
-    0.225
-  ]
-}

model_files/processing_molparser_mobile.py

@@ -1,71 +0,0 @@
-"""Processor that combines MolParser Mobile image preprocessing and tokenizer."""
-
-from __future__ import annotations
-
-from pathlib import Path
-from typing import Sequence
-
-from .image_processing_molparser_mobile import MolParserImageProcessor
-from .tokenization_molparser_mobile import MolParserTokenizer
-
-
-class MolParserProcessor:
-    attributes = ["image_processor", "tokenizer"]
-    image_processor_class = "MolParserImageProcessor"
-    tokenizer_class = "MolParserTokenizer"
-
-    def __init__(
-        self,
-        image_processor: MolParserImageProcessor | None = None,
-        tokenizer: MolParserTokenizer | None = None,
-    ):
-        self.image_processor = image_processor or MolParserImageProcessor()
-        self.tokenizer = tokenizer
-
-    @classmethod
-    def register_for_auto_class(cls, auto_class: str = "AutoProcessor"):
-        cls._auto_class = auto_class
-
-    @classmethod
-    def from_pretrained(cls, pretrained_model_name_or_path: str, **kwargs) -> "MolParserProcessor":
-        path = str(pretrained_model_name_or_path)
-        image_processor = MolParserImageProcessor.from_pretrained(path, **kwargs)
-        tokenizer = MolParserTokenizer.from_pretrained(path, **kwargs)
-        return cls(image_processor=image_processor, tokenizer=tokenizer)
-
-    def save_pretrained(self, save_directory: str, **kwargs):
-        Path(save_directory).mkdir(parents=True, exist_ok=True)
-        image_files = self.image_processor.save_pretrained(save_directory, **kwargs)
-        tokenizer_files = ()
-        if self.tokenizer is not None:
-            tokenizer_files = self.tokenizer.save_pretrained(save_directory, **kwargs)
-        return tuple(image_files) + tuple(tokenizer_files)
-
-    def __call__(
-        self,
-        images=None,
-        text: str | Sequence[str] | None = None,
-        return_tensors: str | None = None,
-        **kwargs,
-    ):
-        encoded = {}
-        if images is not None:
-            encoded.update(self.image_processor(images=images, return_tensors=return_tensors, **kwargs))
-        if text is not None:
-            if self.tokenizer is None:
-                raise ValueError("MolParserProcessor was created without a tokenizer.")
-            encoded.update(self.tokenizer(text, return_tensors=return_tensors, **kwargs))
-        return encoded
-
-    def decode(self, *args, **kwargs):
-        if self.tokenizer is None:
-            raise ValueError("MolParserProcessor was created without a tokenizer.")
-        return self.tokenizer.decode(*args, **kwargs)
-
-    def batch_decode(self, *args, **kwargs):
-        if self.tokenizer is None:
-            raise ValueError("MolParserProcessor was created without a tokenizer.")
-        return self.tokenizer.batch_decode(*args, **kwargs)
-
-
-__all__ = ["MolParserProcessor"]

model_files/processor_config.json

@@ -1,7 +0,0 @@
-{
-  "processor_class": "MolParserProcessor",
-  "model_name": "MolParser Mobile",
-  "auto_map": {
-    "AutoProcessor": "processing_molparser_mobile.MolParserProcessor"
-  }
-}

model_files/tokenization_molparser_mobile.py

@@ -1,191 +0,0 @@
-"""MolParser Mobile tokenizer for Hugging Face Hub remote loading."""
-
-from __future__ import annotations
-
-import json
-import os
-import re
-from pathlib import Path
-from typing import Dict, Iterable, List, Optional, Sequence, Union
-
-from huggingface_hub import hf_hub_download
-from transformers import PreTrainedTokenizer
-
-
-TOKENIZER_CONFIG_NAME = "tokenizer_config.json"
-VOCAB_NAME = "vocab.txt"
-
-
-class MolParserTokenizer(PreTrainedTokenizer):
-    model_input_names = ["input_ids", "attention_mask"]
-    padding_side = "right"
-
-    def __init__(
-        self,
-        vocab_list: Optional[List[str]] = None,
-        special_tokens: Optional[Dict[str, str]] = None,
-        additional_special_tokens: Optional[Sequence[str]] = None,
-        **kwargs,
-    ):
-        if vocab_list is None:
-            vocab_list = []
-        if special_tokens is None:
-            special_tokens = {}
-        if additional_special_tokens is None:
-            additional_special_tokens = []
-
-        self.special_tokens = {
-            "cls_token": special_tokens.get("cls_token", "[CLS]"),
-            "pad_token": special_tokens.get("pad_token", "[PAD]"),
-            "sep_token": special_tokens.get("sep_token", "[SEP]"),
-            "unk_token": special_tokens.get("unk_token", "[UNK]"),
-        }
-        self.additional_special_tokens = list(dict.fromkeys(additional_special_tokens))
-        self.vocab_list = list(vocab_list)
-        all_tokens = self._build_full_vocab(self.vocab_list)
-        self.vocab = {token: idx for idx, token in enumerate(all_tokens)}
-        self.ids_to_tokens = {idx: token for token, idx in self.vocab.items()}
-        self._decode_skip_tokens = set(self.special_tokens.values())
-        self._compile_pattern()
-
-        super().__init__(
-            cls_token=self.special_tokens["cls_token"],
-            pad_token=self.special_tokens["pad_token"],
-            sep_token=self.special_tokens["sep_token"],
-            unk_token=self.special_tokens["unk_token"],
-            bos_token=self.special_tokens["cls_token"],
-            eos_token=self.special_tokens["sep_token"],
-            additional_special_tokens=self.additional_special_tokens,
-            **kwargs,
-        )
-
-    def _build_full_vocab(self, vocab_list: Sequence[str]) -> List[str]:
-        ordered_tokens: List[str] = []
-        for token in list(self.special_tokens.values()) + list(vocab_list) + list(self.additional_special_tokens):
-            if token not in ordered_tokens:
-                ordered_tokens.append(token)
-        return ordered_tokens
-
-    def _compile_pattern(self) -> None:
-        multi_char_tokens = sorted(self.vocab.keys(), key=len, reverse=True)
-        pattern = "(" + "|".join(re.escape(token) for token in multi_char_tokens) + "|.)"
-        self.pattern = re.compile(pattern)
-
-    @property
-    def vocab_size(self) -> int:
-        return len(self.vocab)
-
-    def __len__(self) -> int:
-        return len(self.vocab)
-
-    def get_vocab(self) -> Dict[str, int]:
-        return dict(self.vocab)
-
-    def _tokenize(self, text: str) -> List[str]:
-        return [token for token in self.pattern.findall(str(text)) if token]
-
-    def tokenize(self, text: str, **kwargs) -> List[str]:
-        return self._tokenize(text)
-
-    def _convert_token_to_id(self, token: str) -> int:
-        return self.vocab.get(token, self.unk_token_id)
-
-    def _convert_id_to_token(self, index: int) -> str:
-        return self.ids_to_tokens.get(int(index), self.unk_token)
-
-    def convert_tokens_to_string(self, tokens: Sequence[str]) -> str:
-        return "".join(tokens)
-
-    def build_inputs_with_special_tokens(self, token_ids_0, token_ids_1=None):
-        if token_ids_1 is None:
-            return list(token_ids_0)
-        return list(token_ids_0) + list(token_ids_1)
-
-    def encode(self, text: str, add_special_tokens: bool = False, **kwargs) -> List[int]:
-        token_ids = [self._convert_token_to_id(token) for token in self._tokenize(text)]
-        if add_special_tokens:
-            return [self.bos_token_id] + token_ids + [self.eos_token_id]
-        return token_ids
-
-    def decode(self, token_ids: Iterable[int], skip_special_tokens: bool = False, **kwargs) -> str:
-        tokens = [self._convert_id_to_token(idx) for idx in token_ids]
-        if skip_special_tokens:
-            # Match deploy/tokenizer.py: keep MolParser business tokens such as
-            # <sep>, <a>, </a>, <r>, </r>, <c>, </c>, and |Sg:n|.
-            tokens = [token for token in tokens if token not in self._decode_skip_tokens]
-        return "".join(tokens)
-
-    def batch_encode(self, texts: Sequence[str], add_special_tokens: bool = False) -> List[List[int]]:
-        return [self.encode(text, add_special_tokens=add_special_tokens) for text in texts]
-
-    def batch_decode(
-        self,
-        sequences: Sequence[Sequence[int]],
-        skip_special_tokens: bool = False,
-        **kwargs,
-    ) -> List[str]:
-        return [self.decode(ids, skip_special_tokens=skip_special_tokens, **kwargs) for ids in sequences]
-
-    def to_dict(self) -> Dict[str, object]:
-        return {
-            "vocab_list": self.vocab_list,
-            "special_tokens": self.special_tokens,
-            "additional_special_tokens": self.additional_special_tokens,
-            "tokenizer_class": self.__class__.__name__,
-            "auto_map": {
-                "AutoTokenizer": [
-                    "tokenization_molparser_mobile.MolParserTokenizer",
-                    None,
-                ]
-            },
-        }
-
-    @classmethod
-    def from_dict(cls, config: Dict[str, object]) -> "MolParserTokenizer":
-        return cls(
-            vocab_list=list(config["vocab_list"]),
-            special_tokens=dict(config["special_tokens"]),
-            additional_special_tokens=list(config.get("additional_special_tokens", [])),
-        )
-
-    def save_vocabulary(self, save_directory: str, filename_prefix: Optional[str] = None):
-        path = Path(save_directory)
-        path.mkdir(parents=True, exist_ok=True)
-        name = f"{filename_prefix}-{VOCAB_NAME}" if filename_prefix else VOCAB_NAME
-        vocab_path = path / name
-        vocab_path.write_text("\n".join(self.vocab_list) + "\n", encoding="utf-8")
-        return (str(vocab_path),)
-
-    def save_pretrained(self, save_directory: str, **kwargs):
-        os.makedirs(save_directory, exist_ok=True)
-        config_path = os.path.join(save_directory, TOKENIZER_CONFIG_NAME)
-        with open(config_path, "w", encoding="utf-8") as f:
-            json.dump(self.to_dict(), f, ensure_ascii=False, indent=2)
-        vocab_files = self.save_vocabulary(save_directory)
-        return (config_path, *vocab_files)
-
-    @classmethod
-    def from_pretrained(cls, pretrained_model_name_or_path: str, *args, **kwargs) -> "MolParserTokenizer":
-        config_path = Path(pretrained_model_name_or_path)
-        if config_path.is_dir():
-            config_path = config_path / TOKENIZER_CONFIG_NAME
-        elif config_path.is_file():
-            pass
-        else:
-            config_path = Path(
-                hf_hub_download(
-                    repo_id=str(pretrained_model_name_or_path),
-                    filename=TOKENIZER_CONFIG_NAME,
-                    repo_type=kwargs.get("repo_type"),
-                    revision=kwargs.get("revision"),
-                    cache_dir=kwargs.get("cache_dir"),
-                    token=kwargs.get("token"),
-                    local_files_only=kwargs.get("local_files_only", False),
-                )
-            )
-        with open(config_path, "r", encoding="utf-8") as f:
-            config = json.load(f)
-        return cls.from_dict(config)
-
-
-__all__ = ["MolParserTokenizer"]

model_files/tokenizer_config.json

@@ -1,400 +0,0 @@
-{
-  "vocab_list": [
-    "!",
-    "\"",
-    "#",
-    "$",
-    "%",
-    "&",
-    "'",
-    "(",
-    ")",
-    "*",
-    "+",
-    ",",
-    "-",
-    ".",
-    "/",
-    "0",
-    "1",
-    "2",
-    "3",
-    "4",
-    "5",
-    "6",
-    "7",
-    "8",
-    "9",
-    "10",
-    "11",
-    "12",
-    "13",
-    "14",
-    "15",
-    "16",
-    "17",
-    "18",
-    "19",
-    "20",
-    "21",
-    "22",
-    "23",
-    "24",
-    "25",
-    "26",
-    "27",
-    "28",
-    "29",
-    "30",
-    "31",
-    "32",
-    "33",
-    "34",
-    "35",
-    "36",
-    "37",
-    "38",
-    "39",
-    "40",
-    "41",
-    "42",
-    "43",
-    "44",
-    "45",
-    "46",
-    "47",
-    "48",
-    "49",
-    "50",
-    "51",
-    "52",
-    "53",
-    "54",
-    "55",
-    "56",
-    "57",
-    "58",
-    "59",
-    "60",
-    "61",
-    "62",
-    "63",
-    "64",
-    "65",
-    "66",
-    "67",
-    "68",
-    "69",
-    "70",
-    "71",
-    "72",
-    "73",
-    "74",
-    "75",
-    "76",
-    "77",
-    "78",
-    "79",
-    "80",
-    "81",
-    "82",
-    "83",
-    "84",
-    "85",
-    "86",
-    "87",
-    "88",
-    "89",
-    "90",
-    "91",
-    "92",
-    "93",
-    "94",
-    "95",
-    "96",
-    "97",
-    "98",
-    "99",
-    "100",
-    "101",
-    "102",
-    "103",
-    "104",
-    "105",
-    "106",
-    "107",
-    "108",
-    "109",
-    "110",
-    "111",
-    "112",
-    "113",
-    "114",
-    "115",
-    "116",
-    "117",
-    "118",
-    "119",
-    "120",
-    "121",
-    "122",
-    "123",
-    "124",
-    "125",
-    "126",
-    "127",
-    "128",
-    "129",
-    "130",
-    "131",
-    "132",
-    "133",
-    "134",
-    "135",
-    "136",
-    "137",
-    "138",
-    "139",
-    "140",
-    "141",
-    "142",
-    "143",
-    "144",
-    "145",
-    "146",
-    "147",
-    "148",
-    "149",
-    "150",
-    "151",
-    "152",
-    "153",
-    "154",
-    "155",
-    "156",
-    "157",
-    "158",
-    "159",
-    "160",
-    "161",
-    "162",
-    "163",
-    "164",
-    "165",
-    "166",
-    "167",
-    "168",
-    "169",
-    "170",
-    "171",
-    "172",
-    "173",
-    "174",
-    "175",
-    "176",
-    "177",
-    "178",
-    "179",
-    "180",
-    "181",
-    "182",
-    "183",
-    "184",
-    "185",
-    "186",
-    "187",
-    "188",
-    "189",
-    "190",
-    "191",
-    "192",
-    "193",
-    "194",
-    "195",
-    "196",
-    "197",
-    "198",
-    "199",
-    "200",
-    "201",
-    "202",
-    "203",
-    "204",
-    "205",
-    "206",
-    "207",
-    "208",
-    "209",
-    "210",
-    "211",
-    "212",
-    "213",
-    "214",
-    "215",
-    "216",
-    "217",
-    "218",
-    "219",
-    "220",
-    "221",
-    "222",
-    "223",
-    "224",
-    "225",
-    "226",
-    "227",
-    "228",
-    "229",
-    "230",
-    "231",
-    "232",
-    "233",
-    "234",
-    "235",
-    "236",
-    "237",
-    "238",
-    "239",
-    "240",
-    "241",
-    "242",
-    "243",
-    "244",
-    "245",
-    "246",
-    "247",
-    "248",
-    "249",
-    "250",
-    "251",
-    "252",
-    "253",
-    "254",
-    "255",
-    ":",
-    ";",
-    "<",
-    "=",
-    ">",
-    "?",
-    "@@",
-    "@",
-    "A",
-    "B",
-    "C",
-    "D",
-    "E",
-    "F",
-    "G",
-    "H",
-    "I",
-    "J",
-    "K",
-    "L",
-    "M",
-    "N",
-    "O",
-    "P",
-    "Q",
-    "R",
-    "S",
-    "T",
-    "U",
-    "V",
-    "W",
-    "X",
-    "Y",
-    "Z",
-    "[",
-    "\\",
-    "]",
-    "^",
-    "_",
-    "`",
-    "a",
-    "b",
-    "c",
-    "d",
-    "e",
-    "f",
-    "g",
-    "h",
-    "i",
-    "j",
-    "k",
-    "l",
-    "m",
-    "n",
-    "o",
-    "p",
-    "q",
-    "r",
-    "s",
-    "t",
-    "u",
-    "v",
-    "w",
-    "x",
-    "y",
-    "z",
-    "{",
-    "|",
-    "}",
-    "~",
-    "Ag",
-    "Al",
-    "As",
-    "Au",
-    "Br",
-    "Ca",
-    "Cl",
-    "Cr",
-    "Cu",
-    "Fe",
-    "Gd",
-    "Hg",
-    "Li",
-    "Mg",
-    "Mn",
-    "Na",
-    "Ni",
-    "Pb",
-    "Pt",
-    "Sb",
-    "Se",
-    "Si",
-    "Sn",
-    "Ti",
-    "Zn",
-    "Zr",
-    "2H",
-    "3H",
-    "->",
-    "<-"
-  ],
-  "special_tokens": {
-    "cls_token": "[CLS]",
-    "pad_token": "[PAD]",
-    "sep_token": "[SEP]",
-    "unk_token": "[UNK]"
-  },
-  "additional_special_tokens": [
-    "<sep>",
-    "<dum>",
-    "<id>",
-    "<a>",
-    "</a>",
-    "<r>",
-    "</r>",
-    "<c>",
-    "</c>",
-    "|Sg:n|"
-  ],
-  "tokenizer_class": "MolParserTokenizer",
-  "auto_map": {
-    "AutoTokenizer": [
-      "tokenization_molparser_mobile.MolParserTokenizer",
-      null
-    ]
-  }
-}

model_files/vocab.txt

@@ -1,371 +0,0 @@
-!
-"
-#
-$
-%
-&
-'
-(
-)
-*
-+
-,
--
-.
-/
-0
-1
-2
-3
-4
-5
-6
-7
-8
-9
-10
-11
-12
-13
-14
-15
-16
-17
-18
-19
-20
-21
-22
-23
-24
-25
-26
-27
-28
-29
-30
-31
-32
-33
-34
-35
-36
-37
-38
-39
-40
-41
-42
-43
-44
-45
-46
-47
-48
-49
-50
-51
-52
-53
-54
-55
-56
-57
-58
-59
-60
-61
-62
-63
-64
-65
-66
-67
-68
-69
-70
-71
-72
-73
-74
-75
-76
-77
-78
-79
-80
-81
-82
-83
-84
-85
-86
-87
-88
-89
-90
-91
-92
-93
-94
-95
-96
-97
-98
-99
-100
-101
-102
-103
-104
-105
-106
-107
-108
-109
-110
-111
-112
-113
-114
-115
-116
-117
-118
-119
-120
-121
-122
-123
-124
-125
-126
-127
-128
-129
-130
-131
-132
-133
-134
-135
-136
-137
-138
-139
-140
-141
-142
-143
-144
-145
-146
-147
-148
-149
-150
-151
-152
-153
-154
-155
-156
-157
-158
-159
-160
-161
-162
-163
-164
-165
-166
-167
-168
-169
-170
-171
-172
-173
-174
-175
-176
-177
-178
-179
-180
-181
-182
-183
-184
-185
-186
-187
-188
-189
-190
-191
-192
-193
-194
-195
-196
-197
-198
-199
-200
-201
-202
-203
-204
-205
-206
-207
-208
-209
-210
-211
-212
-213
-214
-215
-216
-217
-218
-219
-220
-221
-222
-223
-224
-225
-226
-227
-228
-229
-230
-231
-232
-233
-234
-235
-236
-237
-238
-239
-240
-241
-242
-243
-244
-245
-246
-247
-248
-249
-250
-251
-252
-253
-254
-255
-:
-;
-<
-=
->
-?
-@@
-@
-A
-B
-C
-D
-E
-F
-G
-H
-I
-J
-K
-L
-M
-N
-O
-P
-Q
-R
-S
-T
-U
-V
-W
-X
-Y
-Z
-[
-\
-]
-^
-_
-`
-a
-b
-c
-d
-e
-f
-g
-h
-i
-j
-k
-l
-m
-n
-o
-p
-q
-r
-s
-t
-u
-v
-w
-x
-y
-z
-{
-|
-}
-~
-Ag
-Al
-As
-Au
-Br
-Ca
-Cl
-Cr
-Cu
-Fe
-Gd
-Hg
-Li
-Mg
-Mn
-Na
-Ni
-Pb
-Pt
-Sb
-Se
-Si
-Sn
-Ti
-Zn
-Zr
-2H
-3H
-->
-<-