Image-Text-to-Text
MLX
Safetensors
prism_hadamard_qwen35
zen6
hanzo
zenlm
apple-silicon
ternary
2-bit
mcp
multimodal
conversational
Instructions to use hanzoai/zen6-mlx with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- MLX
How to use hanzoai/zen6-mlx with MLX:
# Make sure mlx-vlm is installed # pip install --upgrade mlx-vlm from mlx_vlm import load, generate from mlx_vlm.prompt_utils import apply_chat_template from mlx_vlm.utils import load_config # Load the model model, processor = load("hanzoai/zen6-mlx") config = load_config("hanzoai/zen6-mlx") # Prepare input image = ["http://images.cocodataset.org/val2017/000000039769.jpg"] prompt = "Describe this image." # Apply chat template formatted_prompt = apply_chat_template( processor, config, prompt, num_images=1 ) # Generate output output = generate(model, processor, formatted_prompt, image) print(output) - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- LM Studio
- Pi
How to use hanzoai/zen6-mlx with Pi:
Start the MLX server
# Install MLX LM: uv tool install mlx-lm # Start a local OpenAI-compatible server: mlx_lm.server --model "hanzoai/zen6-mlx"
Configure the model in Pi
# Install Pi: npm install -g @earendil-works/pi-coding-agent # Add to ~/.pi/agent/models.json: { "providers": { "mlx-lm": { "baseUrl": "http://localhost:8080/v1", "api": "openai-completions", "apiKey": "none", "models": [ { "id": "hanzoai/zen6-mlx" } ] } } }Run Pi
# Start Pi in your project directory: pi
- Hermes Agent
How to use hanzoai/zen6-mlx with Hermes Agent:
Start the MLX server
# Install MLX LM: uv tool install mlx-lm # Start a local OpenAI-compatible server: mlx_lm.server --model "hanzoai/zen6-mlx"
Configure Hermes
# Install Hermes: curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash hermes setup # Point Hermes at the local server: hermes config set model.provider custom hermes config set model.base_url http://127.0.0.1:8080/v1 hermes config set model.default hanzoai/zen6-mlx
Run Hermes
hermes
- Atomic Chat
- OpenClaw
How to use hanzoai/zen6-mlx with OpenClaw:
Start the MLX server
# Install MLX LM: uv tool install mlx-lm # Start a local OpenAI-compatible server: mlx_lm.server --model "hanzoai/zen6-mlx"
Configure OpenClaw
# Install OpenClaw: npm install -g openclaw@latest # Register the local server and set it as the default model: openclaw onboard --non-interactive --mode local \ --auth-choice custom-api-key \ --custom-base-url http://127.0.0.1:8080/v1 \ --custom-model-id "hanzoai/zen6-mlx" \ --custom-provider-id mlx-lm \ --custom-compatibility openai \ --custom-text-input \ --accept-risk \ --skip-health
Run OpenClaw
openclaw agent --local --agent main --message "Hello from Hugging Face"
Download runtime/codec.py from hanzoai/zen6-mlx: direct link, hf CLI and curl.
- Browser
- Download file 2.57 kB
-
https://huggingface.co/hanzoai/zen6-mlx/resolve/main/runtime/codec.py
- Command line
-
hf download hf://hanzoai/zen6-mlx/runtime/codec.py
-
curl -L -o codec.py https://huggingface.co/hanzoai/zen6-mlx/resolve/main/runtime/codec.py
2.57 kB
| """Lossless PQ2_0/PTQ1_0 to MLX affine 2-bit block transcoding. | |
| Preserves folded weights. Runtime Hadamard metadata remains mandatory; this | |
| module alone does not produce a loadable or validated model. | |
| """ | |
| import numpy as np | |
| def transcode(raw: bytes, shape: tuple[int, int], source: str): | |
| """Return packed uint32 weights and FP16 affine scales/biases, group size 128.""" | |
| rows, width = shape | |
| if rows <= 0 or width <= 0 or width % 128: | |
| raise ValueError( | |
| "Expected positive [output, input] shape with input divisible by 128" | |
| ) | |
| sizes = {"PQ2_0": 34, "PTQ1_0": 28} | |
| if source not in sizes: | |
| raise ValueError(f"Unsupported format: {source}") | |
| blocks = rows * width // 128 | |
| if len(raw) != blocks * sizes[source]: | |
| raise ValueError("Raw byte length does not match shape and format") | |
| data = np.frombuffer(raw, dtype=np.uint8).reshape(blocks, sizes[source]) | |
| scale_bytes = data[:, :2] if source == "PQ2_0" else data[:, 26:28] | |
| scales = scale_bytes.copy().view("<f2").reshape(rows, width // 128) | |
| if not np.isfinite(scales).all(): | |
| raise ValueError("Non-finite quantization scale") | |
| if source == "PQ2_0": | |
| words = data[:, 2:].copy().view("<u4").reshape(rows, width // 16) | |
| return words, np.ascontiguousarray(scales), np.ascontiguousarray(-scales) | |
| else: | |
| pieces = [] | |
| # GGML PTQ stages consume 16 bytes, then 8, then the two-byte tail. | |
| for lo, hi, count in [(0, 16, 5), (16, 24, 5), (24, 26, 4)]: | |
| packed = data[:, lo:hi].astype(np.uint16) | |
| for trit in range(count): | |
| remainder = (packed * (3**trit)) & 255 | |
| pieces.append(((remainder * 3) >> 8).astype(np.uint8)) | |
| codes = np.concatenate(pieces, axis=1) | |
| words = np.bitwise_or.reduce( | |
| codes.astype(np.uint32).reshape(rows, width // 16, 16) | |
| << (2 * np.arange(16, dtype=np.uint32)), | |
| axis=-1, | |
| ).astype("<u4") | |
| return ( | |
| np.ascontiguousarray(words), | |
| np.ascontiguousarray(scales), | |
| np.ascontiguousarray(-scales), | |
| ) | |
| def unpack(weight, scales, biases): | |
| """Independent affine reconstruction in float32 for packing validation.""" | |
| rows, words = weight.shape | |
| values = np.empty((rows, words * 16), dtype=np.float32) | |
| for lane in range(16): | |
| values[:, lane::16] = (weight >> (2 * lane)) & 3 | |
| groups = values.reshape(rows, -1, 128) | |
| return ( | |
| groups * scales.astype(np.float32)[..., None] | |
| + biases.astype(np.float32)[..., None] | |
| ).reshape(rows, -1) | |