DoL-1.2-Base / Tokenizer /__init__.py
GABIRI's picture
Add DoL 1.2 model card and inference runtime
431ffa3 verified
Raw History Blame Contribute Delete
103 Bytes
# -*- coding: utf-8 -*-
"""Tokenizer package for text, Mongolian morphology, and multimodal inputs."""