Instructions to use zai-org/GLM-OCR with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use zai-org/GLM-OCR with Transformers:
# Use a pipeline as a high-level helper # Warning: Pipeline type "image-to-text" is no longer supported in transformers v5. # You must load the model directly (see below) or downgrade to v4.x with: # pip install "transformers<5.0.0" from transformers import pipeline pipe = pipeline("image-to-text", model="zai-org/GLM-OCR") messages = [ { "role": "user", "content": [ {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"}, {"type": "text", "text": "What animal is on the candy?"} ] }, ] pipe(text=messages)# pip install -U transformers accelerate # Load model directly from transformers import AutoTokenizer, AutoModelForMultimodalLM tokenizer = AutoTokenizer.from_pretrained("zai-org/GLM-OCR") model = AutoModelForMultimodalLM.from_pretrained("zai-org/GLM-OCR", device_map="auto") messages = [ { "role": "user", "content": [ {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"}, {"type": "text", "text": "What animal is on the candy?"} ] }, ] inputs = tokenizer.apply_chat_template( messages, add_generation_prompt=True, tokenize=True, return_dict=True, return_tensors="pt", ).to(model.device) outputs = model.generate(**inputs, max_new_tokens=256) print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:])) - Notebooks
- Google Colab
- Kaggle
- AMD Developer Cloud
Commit History
-
Remove hardcode pipeline_tag (must be image-text-to-text)
Verified signatureI believe the pipeline_tag must be `image-text-to-text` which is the underlying task of the model in transformers/vllm/etc. The Image-to-Text label itself it not wrong but takes precedence as "main pipeline tag" and therefore breaks a few integrations with the Hub (no "conversational" tag, broken inference widget, no chat template parser, etc.) This PR should solve this by letting the Hub auto detect the correct pipeline tag. This should also solve this issue reported here: https://github.com/huggingface/hub-docs/issues/24816b7a858
-
Add OlmOCRBench evaluation results (#43)
- Add OlmOCRBench evaluation results (24d1a23a936160395eddb503904507a648f7b19e) - Update .eval_results/olmocrbench.yaml (0365ee8603499a3f64d3fb1e68f75f98d90dcd49) - Update .eval_results/olmocrbench.yaml (0b03da6b10eaa2c1751159a7312bb85e0b0cddc4) Co-authored-by: Said Taghadouini <staghado@users.noreply.huggingface.co>cb34f33 -
Add MDPBench evaluation results (#51)
- Add MDPBench evaluation results (ceb7f67cdcca12957b1fbdb764338a4b72e51094) Co-authored-by: Delores Lin <Delores-Lin@users.noreply.huggingface.co>b04a4f9
-
add technical report (#44)
- add technical report (1d070ac87e860e594c102a3f67a7492f9708a79f) Co-authored-by: Jared Wen <Jared4Real@users.noreply.huggingface.co>1c97a90
-
Update README.md (#25)
- Update README.md (a8a31f9345ad2c774da831ad19b592de49920d90) Co-authored-by: Jared Wen <Jared4Real@users.noreply.huggingface.co>677c6ba
-
Update README.md (#3)
- Update README.md (9759f40871fe8ec6dec430dfa2502bea8ca051c2) Co-authored-by: Jared Wen <Jared4Real@users.noreply.huggingface.co>47b6bf4
-
update
zRzRzRzRzRzRzR1525284 -
Update Acknowledgement
zRzRzRzRzRzRzRef113d3 -
update
zRzRzRzRzRzRzRe9134f4 -
Add files using upload-large-folder tool
Verified signaturec676960
-
initial commit
Verified signatured4f5b7f