How to use from
llama.cpp
Install (macOS, Linux)
curl -LsSf https://llama.app/install.sh | sh
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf IsValorum/Inkling-2-Mini-APEX-I-MiniPlus-V2.1-GGUF:Q8_0
# Run inference directly in the terminal:
llama cli -hf IsValorum/Inkling-2-Mini-APEX-I-MiniPlus-V2.1-GGUF:Q8_0
Install from WinGet (Windows)
winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf IsValorum/Inkling-2-Mini-APEX-I-MiniPlus-V2.1-GGUF:Q8_0
# Run inference directly in the terminal:
llama cli -hf IsValorum/Inkling-2-Mini-APEX-I-MiniPlus-V2.1-GGUF:Q8_0
Build from source code
git clone https://github.com/ggml-org/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-app
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama serve -hf IsValorum/Inkling-2-Mini-APEX-I-MiniPlus-V2.1-GGUF:Q8_0
# Run inference directly in the terminal:
./build/bin/llama cli -hf IsValorum/Inkling-2-Mini-APEX-I-MiniPlus-V2.1-GGUF:Q8_0
Use Docker
docker model run hf.co/IsValorum/Inkling-2-Mini-APEX-I-MiniPlus-V2.1-GGUF:Q8_0
Quick Links

Inkling 2 Mini β€” MiniPlus

APEX Quantization Series by IsValorum

Support on Ko-fi

Independently computed using dedicated cloud compute clusters, funded 100% out-of-pocket.

If you consider these releases to be of high quality and they help you in your day-to-day workflow, consider supporting on Ko-fi. Every contribution directly sustains future cloud compute hours and helps support my university studies.

Run it in V-Agent

Download V-Agent for Windows 10/11

Get the latest Windows installer Β· Release notes

Windows 10/11 Four local engines Seven MCP servers Apache 2.0 license

Your model. Your PC. Real tools. V-Agent is a free, open-source Windows desktop agent developed by the author of these APEX quantizations. Run GGUF locally, or connect a supported remote API.

  • Four native local engines: llama.cpp (CUDA, Vulkan, CPU), Strata / PolyStrata, Strata Rust / RustingStrata, and optional Project Maya. Use official builds where available or select compatible custom builds; Project Maya has its own prerequisites and setup.
  • Real agent workflows: seven integrated MCP servers for files and documents, commands, DuckDuckGo web search and page reading, autonomous browser, desktop tools, HTTP downloads and Hugging Face. Access modes, per-action approvals and isolated workspaces are user-controlled.
  • Find and run GGUF models: browse Hugging Face with hardware-fit estimates, import from LM Studio or Unsloth Studio, and download model files with SHA-256 verification. Local engine options adapt to the executable's reported capabilities.
  • Code checks and privacy: syntax and HTML runtime checks, context management and recovery from some transient failures. Local inference requires no V-Agent account or telemetry; optional web tools and remote APIs access their respective networks.

Open Models, search Inkling-2-Mini-APEX-I-MiniPlus-V2.1-GGUF, choose a compatible GGUF and confirm the download. V-Agent guides engine and model loading where supported.

Family cross-links

Tier Footprint Base Abliterated
MiniPlus 12.09 GiB Current Abliterated
NanoPlus 10.23 GiB Base Abliterated

Architecture and precision

  • Base model: thinkingmachines/Inkling-2-Mini
  • Architecture: inkling2_loop
  • Quantization source precision: BF16
  • iMatrix: ggml-org/Inkling-2-Mini-GGUF/imatrix.gguf (used file; external provenance not inferred)
  • Recurrent protections (if present): F32 (58 tensors observed)

Surgical Tensor Quantization Map (Audited from GGUF)

The table below is generated from the audited physical GGUF.

Complete per-tensor map.

Layer Group Sub-Component / Tensor Qty Precision Engineering Rationale
Global tensors output.weight 1 Q6_K Audited global tensor
Global tensors output_res_query.weight 1 F32 Audited global tensor
Global tensors token_embd.weight 1 Q4_K Audited global tensor
Attention and gates blk.{0..23}.attn_gate.weight 24 Q8_0 Audited attention projection
Attention and gates blk.{0..23}.attn_k.weight 24 Q4_K Audited attention projection
Attention and gates blk.{0..23}.attn_k_conv.weight 24 F32 Audited attention projection
Attention and gates blk.{5..23:6}.attn_k_conv_proj.weight 4 Q8_0 Audited attention projection
Attention and gates blk.{0..4,6..10,12..16,18..22}.attn_output.weight 20 Q4_K Audited attention projection
Attention and gates blk.{5..23:6}.attn_output.weight 4 Q6_K Audited attention projection
Attention and gates blk.{0..23}.attn_post_conv.weight 24 F32 Audited attention projection
Attention and gates blk.{0..23}.attn_q.weight 24 Q4_K Audited attention projection
Attention and gates blk.{0..23}.attn_q_clip.weight 24 F32 Audited attention projection
Attention and gates blk.{0..23}.attn_rel.weight 24 Q4_K Audited attention projection
Attention and gates blk.{0..23}.attn_rel_bias.weight 24 F32 Audited attention projection
Attention and gates blk.{0..23}.attn_rel_clip.weight 24 F32 Audited attention projection
Attention and gates blk.{1..23}.attn_res_query.weight 23 F32 Audited attention projection
Attention and gates blk.{0..23}.attn_sink_proj.weight 24 Q8_0 Audited attention projection
Attention and gates blk.{0..23}.attn_v.weight 24 Q4_K Audited attention projection
Attention and gates blk.{0..23}.attn_v_conv.weight 24 F32 Audited attention projection
Attention and gates blk.{5..23:6}.attn_v_conv_proj.weight 4 Q8_0 Audited attention projection
Dense FFN / MLP blk.{0..23}.ffn_post_conv.weight 24 F32 Audited dense MLP projection
Dense FFN / MLP blk.{0..23}.ffn_res_query.weight 24 F32 Audited dense MLP projection
Routed experts blk.{0..5,18..23}.ffn_down_exps.weight 12 Q5_0 Audited routed-expert projection
Routed experts blk.{6..17}.ffn_down_exps.weight 12 IQ4_NL Audited routed-expert projection
Routed experts blk.{0..5,18..23}.ffn_gate_up_exps.weight 12 IQ3_S Audited routed-expert projection
Routed experts blk.{6..17}.ffn_gate_up_exps.weight 12 IQ3_XXS Audited routed-expert projection
Routing and indexer blk.{0..23}.ffn_gate_inp.weight 24 F32 Audited routing/indexer parameter
Normalizations blk.{5..23:6}.attn_k_inp_norm.weight 4 F32 Verified physical normalization type
Normalizations blk.{0..4,6..10,12..16,18..22}.attn_norm.weight 20 F32 Verified physical normalization type
Normalizations blk.{5..23:6}.attn_q_inp_norm.weight 4 F32 Verified physical normalization type
Normalizations blk.{5..23:6}.attn_v_inp_norm.weight 4 F32 Verified physical normalization type
Normalizations blk.{0..23}.ffn_norm.weight 24 F32 Verified physical normalization type
Normalizations output_norm.weight 1 F32 Verified physical normalization type
Normalizations token_embd_norm.weight 1 F32 Verified physical normalization type
Other block components blk.{0..23}.exp_probs_b.bias 24 F32 Audited structural parameter
Other block components blk.{6..17}.loop_ffn_ctrl_down.weight 12 F32 Audited structural parameter
Other block components blk.{6..17}.loop_ffn_ctrl_forget.weight 12 F32 Audited structural parameter
Other block components blk.{6..17}.loop_ffn_ctrl_gain.weight 12 F32 Audited structural parameter
Other block components blk.{6..17}.loop_k_ctrl_down.weight 12 F32 Audited structural parameter
Other block components blk.{6..17}.loop_k_ctrl_forget.weight 12 F32 Audited structural parameter
Other block components blk.{6..17}.loop_k_ctrl_gain.weight 12 F32 Audited structural parameter
Other block components blk.{6..17}.loop_msg_query.weight 12 F32 Audited structural parameter
Other block components blk.{6..17}.loop_q_ctrl_down.weight 12 F32 Audited structural parameter
Other block components blk.{6..17}.loop_q_ctrl_forget.weight 12 F32 Audited structural parameter
Other block components blk.{6..17}.loop_q_ctrl_gain.weight 12 F32 Audited structural parameter
Other block components blk.{6..17}.loop_v_ctrl_down.weight 12 F32 Audited structural parameter
Other block components blk.{6..17}.loop_v_ctrl_forget.weight 12 F32 Audited structural parameter
Other block components blk.{6..17}.loop_v_ctrl_gain.weight 12 F32 Audited structural parameter
Downloads last month
-
GGUF
Model size
26B params
Architecture
inkling2_loop
Hardware compatibility
Log In to add your hardware

We're not able to determine the quantization variants.

Inference Providers NEW
This model isn't deployed by any Inference Provider. πŸ™‹ Ask for provider support

Model tree for IsValorum/Inkling-2-Mini-APEX-I-MiniPlus-V2.1-GGUF

Quantized
(41)
this model

Collection including IsValorum/Inkling-2-Mini-APEX-I-MiniPlus-V2.1-GGUF