Instructions to use zai-org/GLM-OCR with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use zai-org/GLM-OCR with Transformers:
# Use a pipeline as a high-level helper # Warning: Pipeline type "image-to-text" is no longer supported in transformers v5. # You must load the model directly (see below) or downgrade to v4.x with: # 'pip install "transformers<5.0.0' from transformers import pipeline pipe = pipeline("image-to-text", model="zai-org/GLM-OCR") messages = [ { "role": "user", "content": [ {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"}, {"type": "text", "text": "What animal is on the candy?"} ] }, ] pipe(text=messages)# Load model directly from transformers import AutoTokenizer, AutoModelForMultimodalLM tokenizer = AutoTokenizer.from_pretrained("zai-org/GLM-OCR") model = AutoModelForMultimodalLM.from_pretrained("zai-org/GLM-OCR", device_map="auto") messages = [ { "role": "user", "content": [ {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"}, {"type": "text", "text": "What animal is on the candy?"} ] }, ] inputs = tokenizer.apply_chat_template( messages, add_generation_prompt=True, tokenize=True, return_dict=True, return_tensors="pt", ).to(model.device) outputs = model.generate(**inputs, max_new_tokens=40) print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:])) - Notebooks
- Google Colab
- Kaggle
- AMD Developer Cloud
Poor Russian language generation.
I am trying to use GLM-OCR to recognize Russian accounting and financial documents.
I am satisfied with table recognition, but the quality of the generated Russian text is quite poor, despite the good quality of the input documents.
What is the recommended way to fine-tune the model for language and domain adaptation?
@borisbek Hi, thank you for your interest in GLM-OCR.
The current version still has limitations in multilingual scenarios, the recognition quality for non-Chinese/English text is not yet fully optimized. Enhancing multilingual text (including Russian) performance is one of our key priorities in the next release.
For language and domain adaptation, we recommend fine-tuning the model with your own Russian financial document dataset. You can find the step-by-step fine-tuning guide here:
GLM-OCR Fine-tuning Guide
If possible, please share some bad cases with us so that we can use them for evaluation.