Image-Text-to-Text
MLX
Safetensors
English
inkling2_loop
mlx-vlm
inkling2
Mixture of Experts
quantized
gptq
apple-silicon
tool-calling
agents
conversational
8-bit precision
Instructions to use VisualInference/Inkling-2-Mini-MLX-2bit with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- MLX
How to use VisualInference/Inkling-2-Mini-MLX-2bit with MLX:
# Make sure mlx-vlm is installed # pip install --upgrade mlx-vlm from mlx_vlm import load, generate from mlx_vlm.prompt_utils import apply_chat_template from mlx_vlm.utils import load_config # Load the model model, processor = load("VisualInference/Inkling-2-Mini-MLX-2bit") config = load_config("VisualInference/Inkling-2-Mini-MLX-2bit") # Prepare input image = ["http://images.cocodataset.org/val2017/000000039769.jpg"] prompt = "Describe this image." # Apply chat template formatted_prompt = apply_chat_template( processor, config, prompt, num_images=1 ) # Generate output output = generate(model, processor, formatted_prompt, image) print(output) - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- LM Studio
- Pi
How to use VisualInference/Inkling-2-Mini-MLX-2bit with Pi:
Start the MLX server
# Install MLX LM: uv tool install mlx-lm # Start a local OpenAI-compatible server: mlx_lm.server --model "VisualInference/Inkling-2-Mini-MLX-2bit"
Configure the model in Pi
# Install Pi: npm install -g @earendil-works/pi-coding-agent # Add to ~/.pi/agent/models.json: { "providers": { "mlx-lm": { "baseUrl": "http://localhost:8080/v1", "api": "openai-completions", "apiKey": "none", "models": [ { "id": "VisualInference/Inkling-2-Mini-MLX-2bit" } ] } } }Run Pi
# Start Pi in your project directory: pi
- Hermes Agent
How to use VisualInference/Inkling-2-Mini-MLX-2bit with Hermes Agent:
Start the MLX server
# Install MLX LM: uv tool install mlx-lm # Start a local OpenAI-compatible server: mlx_lm.server --model "VisualInference/Inkling-2-Mini-MLX-2bit"
Configure Hermes
# Install Hermes: curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash hermes setup # Point Hermes at the local server: hermes config set model.provider custom hermes config set model.base_url http://127.0.0.1:8080/v1 hermes config set model.default VisualInference/Inkling-2-Mini-MLX-2bit
Run Hermes
hermes
- Atomic Chat
- OpenClaw
How to use VisualInference/Inkling-2-Mini-MLX-2bit with OpenClaw:
Start the MLX server
# Install MLX LM: uv tool install mlx-lm # Start a local OpenAI-compatible server: mlx_lm.server --model "VisualInference/Inkling-2-Mini-MLX-2bit"
Configure OpenClaw
# Install OpenClaw: npm install -g openclaw@latest # Register the local server and set it as the default model: openclaw onboard --non-interactive --mode local \ --auth-choice custom-api-key \ --custom-base-url http://127.0.0.1:8080/v1 \ --custom-model-id "VisualInference/Inkling-2-Mini-MLX-2bit" \ --custom-provider-id mlx-lm \ --custom-compatibility openai \ --custom-text-input \ --accept-risk \ --skip-health
Run OpenClaw
openclaw agent --local --agent main --message "Hello from Hugging Face"
Download extras/scripts/build_sequences.py from VisualInference/Inkling-2-Mini-MLX-2bit: direct link, hf CLI and curl.
- Browser
- Download file 3.43 kB
-
https://huggingface.co/VisualInference/Inkling-2-Mini-MLX-2bit/resolve/main/extras/scripts/build_sequences.py
- Command line
-
hf download hf://VisualInference/Inkling-2-Mini-MLX-2bit/extras/scripts/build_sequences.py
-
curl -L -o build_sequences.py https://huggingface.co/VisualInference/Inkling-2-Mini-MLX-2bit/resolve/main/extras/scripts/build_sequences.py
3.43 kB
| """Build calibration / evaluation token sequences. Usage: build_sequences.py <chats.jsonl> <out.json>""" | |
| import glob, json, os, sys, sysconfig | |
| import pyarrow.parquet as pq | |
| from transformers import AutoTokenizer | |
| sys.path.insert(0, os.path.dirname(os.path.abspath(__file__))) | |
| from scenarios import TOOLS # noqa: E402 | |
| ROOT = os.path.dirname(os.path.dirname(os.path.abspath(__file__))) | |
| tok = AutoTokenizer.from_pretrained(os.path.join(ROOT, "upstream", "Inkling-2-Mini")) | |
| MAX = 900 | |
| out = [] | |
| def render(messages, tools=None, effort="none"): | |
| r = tok.apply_chat_template(messages, tools=tools, reasoning_effort=effort, tokenize=True) | |
| return list(r["input_ids"]) if hasattr(r, "keys") else list(r) | |
| def clip_text(text, budget): | |
| ids = tok.encode(text, add_special_tokens=False)[:budget] | |
| return tok.decode(ids) | |
| # 1. model-generated chats | |
| for line in open(sys.argv[1]): | |
| rec = json.loads(line) | |
| tools = [TOOLS[t] for t in rec.get("tools", [])] or None | |
| ids = render(rec["messages"], tools, rec.get("effort")) | |
| out.append(dict(name="chat:" + rec["id"], split=rec["split"], kind="chat:" + rec["category"], ids=ids)) | |
| # 2. WikiText-2 articles framed as an assistant answer | |
| def articles(path): | |
| lines = pq.read_table(path).column("text").to_pylist() | |
| cur_title, cur = None, [] | |
| for ln in lines: | |
| s = ln.strip() | |
| if s.startswith("= ") and s.endswith(" =") and not s.startswith("= ="): | |
| if cur_title and cur: | |
| yield cur_title, "".join(cur) | |
| cur_title, cur = s.strip("= ").strip(), [] | |
| elif cur_title is not None: | |
| cur.append(ln) | |
| if cur_title and cur: | |
| yield cur_title, "".join(cur) | |
| for split, fname, count in (("eval", "test", 30), ("calib", "train", 30)): | |
| for i, (title, body) in enumerate(articles(os.path.join(ROOT, "data/wikitext", fname + ".parquet"))): | |
| if i >= count: | |
| break | |
| msgs = [{"role": "user", "content": f"Write the encyclopedia article titled “{title}”."}, | |
| {"role": "assistant", "content": clip_text(body.strip(), MAX)}] | |
| out.append(dict(name=f"wikitext:{fname}:{i}", split=split, kind="wikitext", ids=render(msgs))) | |
| # 3. Python standard-library source framed as an assistant answer | |
| stdlib = sysconfig.get_paths()["stdlib"] | |
| for split, names in (("calib", ["heapq", "bisect", "textwrap", "colorsys", "fnmatch", "glob", "sched", "queue"]), | |
| ("eval", ["shlex", "statistics", "difflib", "copy", "keyword", "reprlib"])): | |
| for name in names: | |
| src = open(os.path.join(stdlib, name + ".py"), encoding="utf-8").read() | |
| for part, offset in enumerate((0, 6000)): | |
| piece = src[offset:offset + 12000] | |
| if len(piece) < 800: | |
| continue | |
| msgs = [{"role": "user", "content": f"Show part {part + 1} of the source of Python's `{name}` module."}, | |
| {"role": "assistant", "content": "```python\n" + clip_text(piece, MAX) + "\n```"}] | |
| out.append(dict(name=f"code:{name}:{part}", split=split, kind="code", ids=render(msgs))) | |
| json.dump(out, open(sys.argv[2], "w")) | |
| for split in ("calib", "eval"): | |
| sel = [s for s in out if s["split"] == split] | |
| kinds = {} | |
| for s in sel: | |
| k = s["kind"].split(":")[0] | |
| kinds[k] = kinds.get(k, 0) + len(s["ids"]) | |
| print(split, "sequences", len(sel), "tokens", sum(len(s["ids"]) for s in sel), kinds) | |