Basalt v0.11.0 โ Inference, Speech & Quantization Stack
Collection
42+ verified MLX/ANE models (QTIP, PV-Tuning, GSQ-RCO, TTS, OCR) with 1-click launch in Basalt v0.11.0 (Swift 6.4, 84 Compose profiles). โข 43 items โข Updated
How to use goodmai/GLM-OCR-8bit with Transformers:
# Use a pipeline as a high-level helper
# Warning: Pipeline type "image-to-text" is no longer supported in transformers v5.
# You must load the model directly (see below) or downgrade to v4.x with:
# pip install "transformers<5.0.0"
from transformers import pipeline
pipe = pipeline("image-to-text", model="goodmai/GLM-OCR-8bit") # pip install -U transformers accelerate
# Load model directly
from transformers import AutoTokenizer, AutoModelForMultimodalLM
tokenizer = AutoTokenizer.from_pretrained("goodmai/GLM-OCR-8bit")
model = AutoModelForMultimodalLM.from_pretrained("goodmai/GLM-OCR-8bit", device_map="auto")How to use goodmai/GLM-OCR-8bit with MLX:
# Download the model from the Hub pip install huggingface_hub[hf_xet] hf download goodmai/GLM-OCR-8bit --local-dir GLM-OCR-8bit
โก ะะฐะฟัััะธัั ะฒ Basalt v0.11.0 ยท bas-alt.vercel.app (macOS 27+ ยท Swift 6.4 ยท Apple Silicon, ะฑะตะท Python) ยท Upstream source:
mlx-community/GLM-OCR-8bit:basalt serve --model goodmai/gemma-4-12B-it-qat-4bit --pipeline-ocr-model goodmai/GLM-OCR-8bit --rest
This model was converted to MLX format from zai-org/GLM-OCR using mlx-vlm version 0.3.11.
Refer to the original model card for more details on the model.
pip install -U mlx-vlm
python -m mlx_vlm.generate --model mlx-community/GLM-OCR-8bit --max-tokens 100 --temperature 0.0 --prompt "Describe this image." --image <path_to_image>
8-bit