Instructions to use IsValorum/Inkling-2-Mini-APEX-I-NanoPlus-V2.1-GGUF with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- llama.cpp
How to use IsValorum/Inkling-2-Mini-APEX-I-NanoPlus-V2.1-GGUF with llama.cpp:
Install (macOS, Linux)
curl -LsSf https://llama.app/install.sh | sh # Start a local OpenAI-compatible server with a web UI: llama serve -hf IsValorum/Inkling-2-Mini-APEX-I-NanoPlus-V2.1-GGUF:Q8_0 # Run inference directly in the terminal: llama cli -hf IsValorum/Inkling-2-Mini-APEX-I-NanoPlus-V2.1-GGUF:Q8_0
Install from WinGet (Windows)
winget install llama.cpp # Start a local OpenAI-compatible server with a web UI: llama serve -hf IsValorum/Inkling-2-Mini-APEX-I-NanoPlus-V2.1-GGUF:Q8_0 # Run inference directly in the terminal: llama cli -hf IsValorum/Inkling-2-Mini-APEX-I-NanoPlus-V2.1-GGUF:Q8_0
Build from source code
git clone https://github.com/ggml-org/llama.cpp.git cd llama.cpp cmake -B build cmake --build build -j --target llama-app # Start a local OpenAI-compatible server with a web UI: ./build/bin/llama serve -hf IsValorum/Inkling-2-Mini-APEX-I-NanoPlus-V2.1-GGUF:Q8_0 # Run inference directly in the terminal: ./build/bin/llama cli -hf IsValorum/Inkling-2-Mini-APEX-I-NanoPlus-V2.1-GGUF:Q8_0
Use Docker
docker model run hf.co/IsValorum/Inkling-2-Mini-APEX-I-NanoPlus-V2.1-GGUF:Q8_0
- Llama new
- LM Studio
- Jan
- vLLM
How to use IsValorum/Inkling-2-Mini-APEX-I-NanoPlus-V2.1-GGUF with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "IsValorum/Inkling-2-Mini-APEX-I-NanoPlus-V2.1-GGUF" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "IsValorum/Inkling-2-Mini-APEX-I-NanoPlus-V2.1-GGUF", "messages": [ { "role": "user", "content": [ { "type": "text", "text": "Describe this image in one sentence." }, { "type": "image_url", "image_url": { "url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg" } } ] } ] }'Use Docker
docker model run hf.co/IsValorum/Inkling-2-Mini-APEX-I-NanoPlus-V2.1-GGUF:Q8_0
- Ollama
How to use IsValorum/Inkling-2-Mini-APEX-I-NanoPlus-V2.1-GGUF with Ollama:
ollama run hf.co/IsValorum/Inkling-2-Mini-APEX-I-NanoPlus-V2.1-GGUF:Q8_0
- Unsloth Desktop
- Pi
How to use IsValorum/Inkling-2-Mini-APEX-I-NanoPlus-V2.1-GGUF with Pi:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf IsValorum/Inkling-2-Mini-APEX-I-NanoPlus-V2.1-GGUF:Q8_0
Configure the model in Pi
# Install Pi: npm install -g @earendil-works/pi-coding-agent # Add to ~/.pi/agent/models.json: { "providers": { "llama-cpp": { "baseUrl": "http://localhost:9931/v1", "api": "openai-completions", "apiKey": "none", "models": [ { "id": "IsValorum/Inkling-2-Mini-APEX-I-NanoPlus-V2.1-GGUF:Q8_0" } ] } } }Run Pi
# Start Pi in your project directory: pi
- Docker Model Runner
How to use IsValorum/Inkling-2-Mini-APEX-I-NanoPlus-V2.1-GGUF with Docker Model Runner:
docker model run hf.co/IsValorum/Inkling-2-Mini-APEX-I-NanoPlus-V2.1-GGUF:Q8_0
- Lemonade
How to use IsValorum/Inkling-2-Mini-APEX-I-NanoPlus-V2.1-GGUF with Lemonade:
Pull the model
# Download Lemonade from https://lemonade-server.ai/ lemonade pull IsValorum/Inkling-2-Mini-APEX-I-NanoPlus-V2.1-GGUF:Q8_0
Run and chat with the model
lemonade run user.Inkling-2-Mini-APEX-I-NanoPlus-V2.1-GGUF-Q8_0
List all available models
lemonade list
- Hermes Agent
How to use IsValorum/Inkling-2-Mini-APEX-I-NanoPlus-V2.1-GGUF with Hermes Agent:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf IsValorum/Inkling-2-Mini-APEX-I-NanoPlus-V2.1-GGUF:Q8_0
Configure Hermes
# Install Hermes: curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash hermes setup # Point Hermes at the local server: hermes config set model.provider custom hermes config set model.base_url http://127.0.0.1:9931/v1 hermes config set model.default IsValorum/Inkling-2-Mini-APEX-I-NanoPlus-V2.1-GGUF:Q8_0
Run Hermes
hermes
- Atomic Chat
- OpenClaw
How to use IsValorum/Inkling-2-Mini-APEX-I-NanoPlus-V2.1-GGUF with OpenClaw:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf IsValorum/Inkling-2-Mini-APEX-I-NanoPlus-V2.1-GGUF:Q8_0
Configure OpenClaw
# Install OpenClaw: npm install -g openclaw@latest # Register the local server and set it as the default model: openclaw onboard --non-interactive --mode local \ --auth-choice custom-api-key \ --custom-base-url http://127.0.0.1:9931/v1 \ --custom-model-id "IsValorum/Inkling-2-Mini-APEX-I-NanoPlus-V2.1-GGUF:Q8_0" \ --custom-provider-id llama-cpp \ --custom-compatibility openai \ --custom-text-input \ --accept-risk \ --skip-health
Run OpenClaw
openclaw agent --local --agent main --message "Hello from Hugging Face"
Inkling 2 Mini β NanoPlus
APEX Quantization Series by IsValorum
Independently computed using dedicated cloud compute clusters, funded 100% out-of-pocket.
If you consider these releases to be of high quality and they help you in your day-to-day workflow, consider supporting on Ko-fi. Every contribution directly sustains future cloud compute hours and helps support my university studies.
Run it in V-Agent
Get the latest Windows installer Β· Release notes
Your model. Your PC. Real tools. V-Agent is a free, open-source Windows desktop agent developed by the author of these APEX quantizations. Run GGUF locally, or connect a supported remote API.
- Four native local engines: llama.cpp (CUDA, Vulkan, CPU), Strata / PolyStrata, Strata Rust / RustingStrata, and optional Project Maya. Use official builds where available or select compatible custom builds; Project Maya has its own prerequisites and setup.
- Real agent workflows: seven integrated MCP servers for files and documents, commands, DuckDuckGo web search and page reading, autonomous browser, desktop tools, HTTP downloads and Hugging Face. Access modes, per-action approvals and isolated workspaces are user-controlled.
- Find and run GGUF models: browse Hugging Face with hardware-fit estimates, import from LM Studio or Unsloth Studio, and download model files with SHA-256 verification. Local engine options adapt to the executable's reported capabilities.
- Code checks and privacy: syntax and HTML runtime checks, context management and recovery from some transient failures. Local inference requires no V-Agent account or telemetry; optional web tools and remote APIs access their respective networks.
Open Models, search Inkling-2-Mini-APEX-I-NanoPlus-V2.1-GGUF, choose a compatible GGUF and confirm the download. V-Agent guides engine and model loading where supported.
Family cross-links
| Tier | Footprint | Base | Abliterated |
|---|---|---|---|
| MiniPlus | 12.09 GiB | Base | Abliterated |
| NanoPlus | 10.23 GiB | Current | Abliterated |
Architecture and precision
- Base model:
thinkingmachines/Inkling-2-Mini - Architecture:
inkling2_loop - Quantization source precision:
BF16 - iMatrix:
ggml-org/Inkling-2-Mini-GGUF/imatrix.gguf(used file; external provenance not inferred) - Recurrent protections (if present): F32 (58 tensors observed)
Surgical Tensor Quantization Map (Audited from GGUF)
The table below is generated from the audited physical GGUF.
| Layer Group | Sub-Component / Tensor | Qty | Precision | Engineering Rationale |
|---|---|---|---|---|
| Global tensors | output.weight |
1 | Q6_K |
Audited global tensor |
| Global tensors | output_res_query.weight |
1 | F32 |
Audited global tensor |
| Global tensors | token_embd.weight |
1 | Q4_K |
Audited global tensor |
| Attention and gates | blk.{0..23}.attn_gate.weight |
24 | Q8_0 |
Audited attention projection |
| Attention and gates | blk.{0..23}.attn_k.weight |
24 | Q4_K |
Audited attention projection |
| Attention and gates | blk.{0..23}.attn_k_conv.weight |
24 | F32 |
Audited attention projection |
| Attention and gates | blk.{5..23:6}.attn_k_conv_proj.weight |
4 | Q8_0 |
Audited attention projection |
| Attention and gates | blk.{0..4,6..10,12..16,18..22}.attn_output.weight |
20 | Q4_K |
Audited attention projection |
| Attention and gates | blk.{5..23:6}.attn_output.weight |
4 | Q6_K |
Audited attention projection |
| Attention and gates | blk.{0..23}.attn_post_conv.weight |
24 | F32 |
Audited attention projection |
| Attention and gates | blk.{0..23}.attn_q.weight |
24 | Q4_K |
Audited attention projection |
| Attention and gates | blk.{0..23}.attn_q_clip.weight |
24 | F32 |
Audited attention projection |
| Attention and gates | blk.{0..23}.attn_rel.weight |
24 | Q4_K |
Audited attention projection |
| Attention and gates | blk.{0..23}.attn_rel_bias.weight |
24 | F32 |
Audited attention projection |
| Attention and gates | blk.{0..23}.attn_rel_clip.weight |
24 | F32 |
Audited attention projection |
| Attention and gates | blk.{1..23}.attn_res_query.weight |
23 | F32 |
Audited attention projection |
| Attention and gates | blk.{0..23}.attn_sink_proj.weight |
24 | Q8_0 |
Audited attention projection |
| Attention and gates | blk.{0..23}.attn_v.weight |
24 | Q4_K |
Audited attention projection |
| Attention and gates | blk.{0..23}.attn_v_conv.weight |
24 | F32 |
Audited attention projection |
| Attention and gates | blk.{5..23:6}.attn_v_conv_proj.weight |
4 | Q8_0 |
Audited attention projection |
| Dense FFN / MLP | blk.{0..23}.ffn_post_conv.weight |
24 | F32 |
Audited dense MLP projection |
| Dense FFN / MLP | blk.{0..23}.ffn_res_query.weight |
24 | F32 |
Audited dense MLP projection |
| Routed experts | blk.{0..1,22..23}.ffn_down_exps.weight |
4 | Q5_0 |
Audited routed-expert projection |
| Routed experts | blk.{2..21}.ffn_down_exps.weight |
20 | IQ4_NL |
Audited routed-expert projection |
| Routed experts | blk.{0..1,22..23}.ffn_gate_up_exps.weight |
4 | IQ3_XXS |
Audited routed-expert projection |
| Routed experts | blk.{2..11}.ffn_gate_up_exps.weight |
10 | IQ2_S |
Audited routed-expert projection |
| Routed experts | blk.{12..21}.ffn_gate_up_exps.weight |
10 | IQ2_XXS |
Audited routed-expert projection |
| Routing and indexer | blk.{0..23}.ffn_gate_inp.weight |
24 | F32 |
Audited routing/indexer parameter |
| Normalizations | blk.{5..23:6}.attn_k_inp_norm.weight |
4 | F32 |
Verified physical normalization type |
| Normalizations | blk.{0..4,6..10,12..16,18..22}.attn_norm.weight |
20 | F32 |
Verified physical normalization type |
| Normalizations | blk.{5..23:6}.attn_q_inp_norm.weight |
4 | F32 |
Verified physical normalization type |
| Normalizations | blk.{5..23:6}.attn_v_inp_norm.weight |
4 | F32 |
Verified physical normalization type |
| Normalizations | blk.{0..23}.ffn_norm.weight |
24 | F32 |
Verified physical normalization type |
| Normalizations | output_norm.weight |
1 | F32 |
Verified physical normalization type |
| Normalizations | token_embd_norm.weight |
1 | F32 |
Verified physical normalization type |
| Other block components | blk.{0..23}.exp_probs_b.bias |
24 | F32 |
Audited structural parameter |
| Other block components | blk.{6..17}.loop_ffn_ctrl_down.weight |
12 | F32 |
Audited structural parameter |
| Other block components | blk.{6..17}.loop_ffn_ctrl_forget.weight |
12 | F32 |
Audited structural parameter |
| Other block components | blk.{6..17}.loop_ffn_ctrl_gain.weight |
12 | F32 |
Audited structural parameter |
| Other block components | blk.{6..17}.loop_k_ctrl_down.weight |
12 | F32 |
Audited structural parameter |
| Other block components | blk.{6..17}.loop_k_ctrl_forget.weight |
12 | F32 |
Audited structural parameter |
| Other block components | blk.{6..17}.loop_k_ctrl_gain.weight |
12 | F32 |
Audited structural parameter |
| Other block components | blk.{6..17}.loop_msg_query.weight |
12 | F32 |
Audited structural parameter |
| Other block components | blk.{6..17}.loop_q_ctrl_down.weight |
12 | F32 |
Audited structural parameter |
| Other block components | blk.{6..17}.loop_q_ctrl_forget.weight |
12 | F32 |
Audited structural parameter |
| Other block components | blk.{6..17}.loop_q_ctrl_gain.weight |
12 | F32 |
Audited structural parameter |
| Other block components | blk.{6..17}.loop_v_ctrl_down.weight |
12 | F32 |
Audited structural parameter |
| Other block components | blk.{6..17}.loop_v_ctrl_forget.weight |
12 | F32 |
Audited structural parameter |
| Other block components | blk.{6..17}.loop_v_ctrl_gain.weight |
12 | F32 |
Audited structural parameter |
- Downloads last month
- 454
We're not able to determine the quantization variants.
Model tree for IsValorum/Inkling-2-Mini-APEX-I-NanoPlus-V2.1-GGUF
Base model
thinkingmachines/Inkling-2-Mini