Image-Text-to-Text
MLX
Safetensors
prism_hadamard_qwen35
zen6
hanzo
zenlm
apple-silicon
ternary
2-bit
mcp
multimodal
conversational
Instructions to use zenlm/zen6-mlx with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- MLX
How to use zenlm/zen6-mlx with MLX:
# Make sure mlx-vlm is installed # pip install --upgrade mlx-vlm from mlx_vlm import load, generate from mlx_vlm.prompt_utils import apply_chat_template from mlx_vlm.utils import load_config # Load the model model, processor = load("zenlm/zen6-mlx") config = load_config("zenlm/zen6-mlx") # Prepare input image = ["http://images.cocodataset.org/val2017/000000039769.jpg"] prompt = "Describe this image." # Apply chat template formatted_prompt = apply_chat_template( processor, config, prompt, num_images=1 ) # Generate output output = generate(model, processor, formatted_prompt, image) print(output) - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- LM Studio
- Pi
How to use zenlm/zen6-mlx with Pi:
Start the MLX server
# Install MLX LM: uv tool install mlx-lm # Start a local OpenAI-compatible server: mlx_lm.server --model "zenlm/zen6-mlx"
Configure the model in Pi
# Install Pi: npm install -g @earendil-works/pi-coding-agent # Add to ~/.pi/agent/models.json: { "providers": { "mlx-lm": { "baseUrl": "http://localhost:8080/v1", "api": "openai-completions", "apiKey": "none", "models": [ { "id": "zenlm/zen6-mlx" } ] } } }Run Pi
# Start Pi in your project directory: pi
- Hermes Agent
How to use zenlm/zen6-mlx with Hermes Agent:
Start the MLX server
# Install MLX LM: uv tool install mlx-lm # Start a local OpenAI-compatible server: mlx_lm.server --model "zenlm/zen6-mlx"
Configure Hermes
# Install Hermes: curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash hermes setup # Point Hermes at the local server: hermes config set model.provider custom hermes config set model.base_url http://127.0.0.1:8080/v1 hermes config set model.default zenlm/zen6-mlx
Run Hermes
hermes
- Atomic Chat
- OpenClaw
How to use zenlm/zen6-mlx with OpenClaw:
Start the MLX server
# Install MLX LM: uv tool install mlx-lm # Start a local OpenAI-compatible server: mlx_lm.server --model "zenlm/zen6-mlx"
Configure OpenClaw
# Install OpenClaw: npm install -g openclaw@latest # Register the local server and set it as the default model: openclaw onboard --non-interactive --mode local \ --auth-choice custom-api-key \ --custom-base-url http://127.0.0.1:8080/v1 \ --custom-model-id "zenlm/zen6-mlx" \ --custom-provider-id mlx-lm \ --custom-compatibility openai \ --custom-text-input \ --accept-risk \ --skip-health
Run OpenClaw
openclaw agent --local --agent main --message "Hello from Hugging Face"
File size: 2,572 Bytes
38bdb31 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 | """Lossless PQ2_0/PTQ1_0 to MLX affine 2-bit block transcoding.
Preserves folded weights. Runtime Hadamard metadata remains mandatory; this
module alone does not produce a loadable or validated model.
"""
import numpy as np
def transcode(raw: bytes, shape: tuple[int, int], source: str):
"""Return packed uint32 weights and FP16 affine scales/biases, group size 128."""
rows, width = shape
if rows <= 0 or width <= 0 or width % 128:
raise ValueError(
"Expected positive [output, input] shape with input divisible by 128"
)
sizes = {"PQ2_0": 34, "PTQ1_0": 28}
if source not in sizes:
raise ValueError(f"Unsupported format: {source}")
blocks = rows * width // 128
if len(raw) != blocks * sizes[source]:
raise ValueError("Raw byte length does not match shape and format")
data = np.frombuffer(raw, dtype=np.uint8).reshape(blocks, sizes[source])
scale_bytes = data[:, :2] if source == "PQ2_0" else data[:, 26:28]
scales = scale_bytes.copy().view("<f2").reshape(rows, width // 128)
if not np.isfinite(scales).all():
raise ValueError("Non-finite quantization scale")
if source == "PQ2_0":
words = data[:, 2:].copy().view("<u4").reshape(rows, width // 16)
return words, np.ascontiguousarray(scales), np.ascontiguousarray(-scales)
else:
pieces = []
# GGML PTQ stages consume 16 bytes, then 8, then the two-byte tail.
for lo, hi, count in [(0, 16, 5), (16, 24, 5), (24, 26, 4)]:
packed = data[:, lo:hi].astype(np.uint16)
for trit in range(count):
remainder = (packed * (3**trit)) & 255
pieces.append(((remainder * 3) >> 8).astype(np.uint8))
codes = np.concatenate(pieces, axis=1)
words = np.bitwise_or.reduce(
codes.astype(np.uint32).reshape(rows, width // 16, 16)
<< (2 * np.arange(16, dtype=np.uint32)),
axis=-1,
).astype("<u4")
return (
np.ascontiguousarray(words),
np.ascontiguousarray(scales),
np.ascontiguousarray(-scales),
)
def unpack(weight, scales, biases):
"""Independent affine reconstruction in float32 for packing validation."""
rows, words = weight.shape
values = np.empty((rows, words * 16), dtype=np.float32)
for lane in range(16):
values[:, lane::16] = (weight >> (2 * lane)) & 3
groups = values.reshape(rows, -1, 128)
return (
groups * scales.astype(np.float32)[..., None]
+ biases.astype(np.float32)[..., None]
).reshape(rows, -1)
|