Image-Text-to-Text
MLX
Safetensors
English
inkling2_loop
mlx-vlm
inkling2
Mixture of Experts
quantized
gptq
apple-silicon
tool-calling
agents
conversational
8-bit precision
Instructions to use VisualInference/Inkling-2-Mini-MLX-4bit with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- MLX
How to use VisualInference/Inkling-2-Mini-MLX-4bit with MLX:
# Make sure mlx-vlm is installed # pip install --upgrade mlx-vlm from mlx_vlm import load, generate from mlx_vlm.prompt_utils import apply_chat_template from mlx_vlm.utils import load_config # Load the model model, processor = load("VisualInference/Inkling-2-Mini-MLX-4bit") config = load_config("VisualInference/Inkling-2-Mini-MLX-4bit") # Prepare input image = ["http://images.cocodataset.org/val2017/000000039769.jpg"] prompt = "Describe this image." # Apply chat template formatted_prompt = apply_chat_template( processor, config, prompt, num_images=1 ) # Generate output output = generate(model, processor, formatted_prompt, image) print(output) - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- LM Studio
- Pi
How to use VisualInference/Inkling-2-Mini-MLX-4bit with Pi:
Start the MLX server
# Install MLX LM: uv tool install mlx-lm # Start a local OpenAI-compatible server: mlx_lm.server --model "VisualInference/Inkling-2-Mini-MLX-4bit"
Configure the model in Pi
# Install Pi: npm install -g @earendil-works/pi-coding-agent # Add to ~/.pi/agent/models.json: { "providers": { "mlx-lm": { "baseUrl": "http://localhost:8080/v1", "api": "openai-completions", "apiKey": "none", "models": [ { "id": "VisualInference/Inkling-2-Mini-MLX-4bit" } ] } } }Run Pi
# Start Pi in your project directory: pi
- Hermes Agent
How to use VisualInference/Inkling-2-Mini-MLX-4bit with Hermes Agent:
Start the MLX server
# Install MLX LM: uv tool install mlx-lm # Start a local OpenAI-compatible server: mlx_lm.server --model "VisualInference/Inkling-2-Mini-MLX-4bit"
Configure Hermes
# Install Hermes: curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash hermes setup # Point Hermes at the local server: hermes config set model.provider custom hermes config set model.base_url http://127.0.0.1:8080/v1 hermes config set model.default VisualInference/Inkling-2-Mini-MLX-4bit
Run Hermes
hermes
- Atomic Chat
- OpenClaw
How to use VisualInference/Inkling-2-Mini-MLX-4bit with OpenClaw:
Start the MLX server
# Install MLX LM: uv tool install mlx-lm # Start a local OpenAI-compatible server: mlx_lm.server --model "VisualInference/Inkling-2-Mini-MLX-4bit"
Configure OpenClaw
# Install OpenClaw: npm install -g openclaw@latest # Register the local server and set it as the default model: openclaw onboard --non-interactive --mode local \ --auth-choice custom-api-key \ --custom-base-url http://127.0.0.1:8080/v1 \ --custom-model-id "VisualInference/Inkling-2-Mini-MLX-4bit" \ --custom-provider-id mlx-lm \ --custom-compatibility openai \ --custom-text-input \ --accept-risk \ --skip-health
Run OpenClaw
openclaw agent --local --agent main --message "Hello from Hugging Face"
File size: 6,868 Bytes
d19188c | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 | {%- set effort_map = {"none": 0.0, "minimal": 0.1, "low": 0.2, "medium": 0.7, "high": 0.9, "xhigh": 0.99, "max": 0.99} -%}
{%- set role_token = {"user": "<|message_user|>", "assistant": "<|message_model|>", "system": "<|message_system|>", "tool": "<|message_tool|>"} -%}
{%- macro emit_thinking_effort() -%}
{%- set eff = reasoning_effort if reasoning_effort is defined and reasoning_effort is not none else 0.9 -%}
{%- if eff is string -%}
{%- set key = eff | trim -%}
{%- if key not in effort_map -%}
{{- raise_exception("Unknown reasoning_effort: " ~ eff) -}}
{%- endif -%}
{%- set num = effort_map[key] -%}
{%- else -%}
{%- set num = eff | float -%}
{%- endif -%}
{%- if num < 0.0 or num > 0.99 -%}
{{- raise_exception("reasoning_effort must be in [0.0, 0.99]") -}}
{%- endif -%}
{{- "<|message_system|><|content_text|>Thinking effort level: " -}}
{%- if num == 0.0 -%}0{%- else -%}{{ num }}{%- endif -%}
{{- "<|end_message|>" -}}
{%- endmacro -%}
{%- if tools -%}
{%- set tool_state = namespace(specs=[]) -%}
{%- for tool in tools -%}
{%- set fn = tool.function if tool.function is defined else tool -%}
{%- set spec = {
"description": (fn.description if fn.description is defined and fn.description else ""),
"name": fn.name,
"parameters": (fn.parameters if fn.parameters is defined and fn.parameters else {}),
"type": (tool.type if tool.type is defined and tool.type else "function"),
} -%}
{%- set tool_state.specs = tool_state.specs + [spec] -%}
{%- endfor -%}
{{- "<|message_system|>tool_declare<|content_xml|>" -}}
{{- tool_state.specs | tojson(sort_keys=true, separators=(",", ":")) -}}
{{- "<|end_message|>" -}}
{%- endif -%}
{%- set state = namespace(effort_emitted=false) -%}
{%- for message in messages -%}
{%- if message.role not in role_token -%}
{{- raise_exception("Unknown message role: " ~ message.role) -}}
{%- endif -%}
{%- if not state.effort_emitted and message.role != "system" -%}
{{- emit_thinking_effort() -}}
{%- set state.effort_emitted = true -%}
{%- endif -%}
{%- set header = namespace(text=role_token[message.role]) -%}
{%- if message.role == "tool" -%}
{%- if message.name is defined and message.name -%}
{%- set header.text = header.text ~ message.name -%}
{%- elif message.tool_call_id is defined and message.tool_call_id -%}
{%- for prev in messages -%}
{%- if prev.role == "assistant" and prev.tool_calls -%}
{%- for tc in prev.tool_calls -%}
{%- if tc.id is defined and tc.id == message.tool_call_id and tc.function.name is defined -%}
{%- set header.text = header.text ~ tc.function.name -%}
{%- endif -%}
{%- endfor -%}
{%- endif -%}
{%- endfor -%}
{%- endif -%}
{%- elif message.role == "assistant" and message.name is defined and message.name -%}
{%- set header.text = header.text ~ message.name -%}
{%- endif -%}
{%- set rtok = header.text -%}
{%- set rendered -%}
{%- if message.role == "assistant" and message.reasoning_content is defined and message.reasoning_content -%}
{{- rtok ~ "<|content_thinking|>" ~ message.reasoning_content ~ "<|end_message|>" -}}
{%- endif -%}
{%- if message.content is string and (message.content or not (message.role == "assistant" and (message.tool_calls or message.reasoning_content))) -%}
{{- rtok ~ "<|content_text|>" ~ message.content ~ "<|end_message|>" -}}
{%- elif message.content -%}
{%- for part in message.content -%}
{%- if part is string -%}
{{- rtok ~ "<|content_text|>" ~ part ~ "<|end_message|>" -}}
{%- elif part.type is not defined or part.type in ("text", "input_text") -%}
{%- set text_part = (part.text if part.text is defined and part.text is string else "") -%}
{{- rtok ~ "<|content_text|>" ~ text_part ~ "<|end_message|>" -}}
{%- elif part.type in ("image", "input_image", "image_url") -%}
{%- if message.role not in ("user", "tool") -%}
{{- raise_exception("Image content is only supported for user and tool messages") -}}
{%- endif -%}
{{- rtok ~ "<|content_image|><|unused_200054|><|end_message|>" -}}
{%- elif part.type in ("audio", "input_audio", "audio_url") -%}
{{- raise_exception("Inkling 2 does not accept audio input") -}}
{%- else -%}
{{- raise_exception("Unsupported content part type: " ~ part.type) -}}
{%- endif -%}
{%- endfor -%}
{%- endif -%}
{%- if message.role == "assistant" and message.tool_calls -%}
{%- for tc in message.tool_calls -%}
{%- set fn = tc.function -%}
{%- if fn.name is not defined or fn.name is not string -%}
{{- raise_exception("tool call function name must be a string") -}}
{%- endif -%}
{%- if message.name is defined and message.name and message.name != fn.name -%}
{{- raise_exception("assistant name " ~ message.name ~ " does not match its tool call " ~ fn.name) -}}
{%- endif -%}
{%- set args = fn.arguments if fn.arguments is defined and fn.arguments else {} -%}
{%- if args is string -%}
{{- raise_exception("tool call arguments must be a parsed object, not a JSON string; canonicalize upstream") -}}
{%- endif -%}
{%- if args is not mapping -%}
{{- raise_exception("tool call arguments must be an object") -}}
{%- endif -%}
{{- "<|message_model|>" ~ fn.name ~ "<|content_invoke_tool_json|>" -}}
{{- '{"name":' ~ (fn.name | tojson(sort_keys=true, separators=(",", ":"))) ~ ',"args":' -}}
{{- "{" -}}
{%- for key, value in args | dictsort(case_sensitive=true) -%}
{{- (key | tojson) ~ ":" ~ (value | tojson(separators=(",", ":"))) -}}
{%- if not loop.last -%},{%- endif -%}
{%- endfor -%}
{{- "}}<|end_message|>" -}}
{%- endfor -%}
{%- endif -%}
{%- if message.role == "assistant" -%}
{{- "<|content_model_end_sampling|>" -}}
{%- endif -%}
{%- endset -%}
{%- if message.role == "assistant" -%}
{%- generation -%}{{- rendered -}}{%- endgeneration -%}
{%- else -%}
{{- rendered -}}
{%- endif -%}
{%- endfor -%}
{%- if not state.effort_emitted -%}
{{- emit_thinking_effort() -}}
{%- endif -%}
|