Card: zero-shot-classification tag, RAM guide for all variants, limitations, pinned mlx-vlm; clef_mlx: warn on untested mlx-vlm/mlx-lm versions

README.md

@@ -3,7 +3,7 @@ license: apache-2.0
 library_name: mlx
 base_model: Cloudflare/clef
 base_model_relation: quantized
-pipeline_tag: image-text-to-text
+pipeline_tag: zero-shot-classification
 tags:
 - mlx
 - clef
@@ -25,12 +25,26 @@ probability for every allowed option, in a single forward pass. **It is not a ch
 meaningless text. Use the bundled `clef_mlx.py` loader, which runs the backbone and the
 joint schema head.
 
+## Which variant fits your Mac?
+
+| Variant | Base | Download | Peak memory (1k / 4k / 16k tokens) | Latency (1k / 16k tokens) | Minimum Mac RAM |
+|---|---|---|---|---|---|
+| [clef-flash-4bit](https://huggingface.co/mlx-community/clef-flash-4bit) | 9B | 6.2 GB | 7.0 / 7.2 / 8.6 GB | 0.31 s / 7.0 s | 16 GB |
+| [clef-flash-8bit](https://huggingface.co/mlx-community/clef-flash-8bit) | 9B | 10.7 GB | 11.4 / 11.6 / 13.0 GB | 0.34 s / 7.7 s | 24 GB (16 GB for short prompts) |
+| **clef-4bit (this repo)** | 27B | 16.3 GB | 17.1 / 17.5 / 19.6 GB | 1.4 s / 26.0 s | 32 GB |
+| [clef-8bit](https://huggingface.co/mlx-community/clef-8bit) | 27B | 29.8 GB | 30.5 / 30.8 / 33.0 GB | 1.5 s / 32.0 s | 48 GB |
+
+Measured on an M5 Max with text input. macOS lets the GPU use only about 70–75% of RAM by default, so the minimum RAM is higher than the peak. The 9B variants are much faster; the 27B variants score higher (see the quality check below).
+
 ## Usage
 
 ```bash
-pip install mlx-vlm huggingface_hub   # no torch needed
+pip install "mlx-vlm>=0.7.4,<0.8" huggingface_hub   # no torch needed
 ```
 
+Tested with mlx 0.32.3, mlx-lm 0.32.0 and mlx-vlm 0.7.4. `clef_mlx.py` uses mlx-vlm internals, so it
+warns if you load it with an untested mlx-vlm minor version.
+
 ```python
 import sys
 from huggingface_hub import snapshot_download
@@ -69,6 +83,20 @@ model.predict({
 
 See the [original model card](https://huggingface.co/Cloudflare/clef) for the input format, question types, and benchmarks.
 
+## Limitations
+
+- **Not a chat model.** Text generation tools (`mlx_lm.generate`, `mlx_vlm.generate`, LM Studio, Ollama) load
+  the backbone but give meaningless output. Only `clef_mlx.py` runs the decision head.
+- **Long inputs are truncated by default**, exactly like the reference implementation: the state is cut so
+  the whole prompt fits in 16,384 tokens. Pass `truncate=False` to `predict()`/`systemone()` to get a
+  `clef_mlx.ContextTooLong` error instead, or raise `max_length` (the head was trained at 16k).
+- **One record per call.** There is no batching; requests run one after another.
+- **Images or videos, not both in one record.** Several images, or several videos, are fine.
+- **Speed depends heavily on the chip.** Latencies above are from an M5 Max; long prompts on base
+  M-series chips will be several times slower.
+- **Custom code.** Like the original release, this repo ships Python (`clef_mlx.py`) that you import and run.
+  Read it before use if that matters in your environment.
+
 ## Conversion
 
 - Backbone: `mlx_vlm.convert -q --q-bits 4 --q-group-size 64` (vision tower kept in bf16).

clef_mlx.py

@@ -27,11 +27,34 @@ SYSTEM_PROMPT = (
 )
 QUESTION_TYPES = {"noul": 0, "choice": 1, "score": 2}
 
+# This loader uses mlx-vlm internals (get_input_embeddings, inputs_embeds/position_ids on the
+# text model), which can change between minor releases. Tested: mlx 0.32.3, mlx-lm 0.32.0, mlx-vlm 0.7.4.
+TESTED_VERSIONS = {"mlx-vlm": (0, 7), "mlx-lm": (0, 32)}
+
 
 class ContextTooLong(ValueError):
     pass
 
 
+def _check_versions(packages: tuple[str, ...]) -> None:
+    import importlib.metadata
+    import warnings
+
+    for name in packages:
+        try:
+            version = importlib.metadata.version(name)
+        except importlib.metadata.PackageNotFoundError:
+            continue
+        major_minor = tuple(int(p) for p in version.split(".")[:2] if p.isdigit())
+        major, minor = TESTED_VERSIONS[name]
+        if major_minor != (major, minor):
+            warnings.warn(
+                f"clef_mlx was tested with {name} {major}.{minor}.x but {version} is installed; "
+                f'if loading fails or results look wrong, run: pip install "{name}>={major}.{minor},<{major}.{minor + 1}"',
+                stacklevel=3,
+            )
+
+
 # --------------------------------------------------------------------------- encoding
 # Mirrors joint_schema_model.encode_record (text-only path) exactly.
 
@@ -533,6 +556,7 @@ def load(path: str | Path, head_dtype=mx.bfloat16, backend: str = "auto") -> Cle
     if backend == "auto":
         backend = "vlm" if _has_vision_weights(path) else "lm"
     processor = None
+    _check_versions(("mlx-vlm",) if backend == "vlm" else ("mlx-lm",))
     if backend == "vlm":
         from mlx_vlm import load as vlm_load