How to use from
llama.cpp
Install (macOS, Linux)
curl -LsSf https://llama.app/install.sh | sh
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf ITLL/IBM-Grok4-UltraFast-Coder-1B-GGUF:
# Run inference directly in the terminal:
llama cli -hf ITLL/IBM-Grok4-UltraFast-Coder-1B-GGUF:
Install from WinGet (Windows)
winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf ITLL/IBM-Grok4-UltraFast-Coder-1B-GGUF:
# Run inference directly in the terminal:
llama cli -hf ITLL/IBM-Grok4-UltraFast-Coder-1B-GGUF:
Build from source code
git clone https://github.com/ggml-org/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-app
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama serve -hf ITLL/IBM-Grok4-UltraFast-Coder-1B-GGUF:
# Run inference directly in the terminal:
./build/bin/llama cli -hf ITLL/IBM-Grok4-UltraFast-Coder-1B-GGUF:
Use Docker
docker model run hf.co/ITLL/IBM-Grok4-UltraFast-Coder-1B-GGUF:
Quick Links

IBM-Grok4-UltraFast-Coder-1B-GGUF

1. Model overview

The name decodes as follows: IBM stands for the Granite 4.0 base model, Grok4 for the xAI Grok distillation lineage, UltraFast for the Grok-4.1-Fast and Grok Code Fast that generated the training data, Coder because coding and technical reasoning dominate the training mix, and 1B for the parameter class. The result is a IBM model with Distillation of a Grok model whose behavior was shaped by Grok-generated training data.

2. Base model: granite-4.0-h-1b

The base model was built by the Granite Team at IBM and released under Apache 2.0 in October 2025 as part of the Granite 4.0 Nano family. It is a decoder-only hybrid: 4 attention layers and 36 Mamba2 layers, with a 128K-token context window. IBM positions the 1B class for on-device deployment, research, and fine-tuning on specialized domains without massive compute resources — exactly the use case this repo targets. The base model's published capability profile covers summarization, text classification and extraction, question answering, retrieval-augmented generation (RAG), code-related tasks, function calling, multilingual dialog, and fill-in-the-middle (FIM) code completion.

For reference, the benchmark scores IBM published for granite-4.0-h-1b (the base model, not this fine-tune):

Benchmark Base model score
MMLU (5-shot) 59.74
GSM8K (8-shot) 69.83
HumanEval (pass@1) 73.0
BFCL v3 (tool calling) 50.21
SALAD-Bench (safety) 96.4

These numbers describe the checkpoint before fine-tuning. Fine-tuning redistributes capability rather than simply adding to it, so treat the table as the starting point, not a measurement of this model. No formal re-benchmark of the fine-tuned checkpoint has been published yet; see Section 8 (Limitations).

3. Fine-tuning methodology

ITLL performed a full fine-tune: every weight of the base network was updated on the nine-dataset mixture described below. No LoRA, no adapters, no frozen layers. The trained checkpoint was then converted to GGUF and quantized into the twelve files available in this repository (Section 6). The GGUF retains the base granite architecture metadata, so runtimes treat it like any other Granite 4.0 hybrid model — including the chat template embedded in the file.

4. Training data

The fine-tune drew on nine publicly available Hugging Face datasets, all built around outputs from the xAI Grok family — Grok-4, Grok-4.1-Fast, and Grok Code Fast 1. Together they contribute roughly 15.2K rows (counts below are as published on each dataset card; two entries are estimates and marked as such).

# Dataset Domain Format Size Teacher
1 Antired/tradehax-xai-grok-trading-visual-prompts Trading-scene visual prompts JSON 10 rows Grok-style prompt curation
2 lvogel123/arc-agi-1-grok-4 Abstract reasoning (ARC-AGI-1) Parquet 402 rows (400 samples) xAI Grok-4
3 Crownelius/Hyper-Creative-Grok-V1 Creative writing, stylistic mirroring ShareGPT JSONL ≈3,400 samples · 8.08M tokens x-ai/grok-4.1-fast
4 Crownelius/Hyper-Logic-Grok-V1 Logic, reasoning, chain-of-thought ShareGPT JSONL ≈3,402 rows (est.) · 10.65M tokens x-ai/grok-4.1-fast
5 Crownelius/Hyper-UltraData-Grok-V1 Doctoral-level knowledge synthesis ShareGPT JSONL 3,873 rows x-ai/grok-4.1-fast
6 TeichAI/brainstorm-v3.1-grok-4-fast-200x Brainstorming dialogues JSON 192 rows Grok 4 Fast
7 TeichAI/grok-code-fast-1-1000x Coding & reasoning Q&A JSON 1,017 rows Grok Code Fast 1
8 nmayorga7/math-grok-4 Competition-math evaluation traces Parquet 2,628 rows xAI Grok-4
9 Liontix/grok-code-fast-1-200x Coding & reasoning Q&A JSON 247 rows Grok Code Fast 1

5. Expected capabilities

  • Code generation and repair. Python scripting, web development, and embedded-systems tasks formed the core of the Grok Code Fast 1 sets (1,264 Q&A rows), so quick code writing, debugging, and explanation are the model's headline skill — matching the "Coder" in its name.
  • Step-by-step reasoning and logic. The Hyper-Logic set (≈10.65M tokens) and the ARC-AGI traces push the model toward explicit, structured chain-of-thought on puzzles, deductions, and multi-step technical problems.
  • Mathematical problem solving. Competition-math traces (AIME/HMMT/SMT-class tasks) plus math questions in the coding sets give the model a working grasp of algebra, combinatorics, and proof-style exposition at the 1B scale.
  • Long-form creative writing. The Hyper-Creative set contributes author-style mirroring and narrative state tracking, so fiction and stylized prose stay coherent over long generations.
  • Divergent brainstorming. The brainstorm set trains multi-option, exploratory answers — useful as a sparring partner for ideas rather than a single-answer oracle.
  • Expert-level Q&A. The Hyper-UltraData set adds graduate-level exposition across physics, CS, operations research, and mathematics.

6. GGUF quantizations

Twelve quantizations are available in this repository. Sizes are as published on the repo file listing:

Quantization File size Suggested use
F16 3.27 GB Reference quality; archival or CPU-only with plenty of RAM
Q8_0 1.74 GB Near-lossless; best quality-per-byte for serious use
Q6_K 1.34 GB Very high quality with a smaller footprint
Q5_K_M 1.18 GB High quality for daily driving
Q5_K_S 1.15 GB Slightly tighter variant of Q5_K_M
Q4_K_M 1.02 GB Recommended default — best balance of quality and size
Q4_K_S 981 MB Slightly faster/smaller than Q4_K_M
IQ4_XS 944 MB Smallest 4-bit option
Q3_K_L 926 MB Tight-RAM setups
Q3_K_M 860 MB Tight-RAM setups
Q3_K_S 786 MB Tight-RAM setups; noticeable quality loss
Q2_K 702 MB Extreme constraints only; expect degraded reasoning

For a 1B-class model, Q4_K_M keeps nearly all of the fine-tune's behavior in about a gigabyte; Q8_0 is worth the extra 700 MB if the model is used for code, where dropped precision shows up as syntax slips. Because the architecture is a Mamba-2/Transformer hybrid, use a current llama.cpp build — older releases predate full support for Granite 4.0 hybrid models.

7. Quickstart

llama.cpp (CLI):

llama-cli -hf ITLL/IBM-Grok4-UltraFast-Coder-1B-GGUF:Q4_K_M

llama.cpp (OpenAI-compatible local server):

llama-server -hf ITLL/IBM-Grok4-UltraFast-Coder-1B-GGUF:Q4_K_M --port 8080

Ollama:

ollama run hf.co/ITLL/IBM-Grok4-UltraFast-Coder-1B-GGUF

LM Studio, Jan, GPT4All, or any llama.cpp-based app: search for the repo ID ITLL/IBM-Grok4-UltraFast-Coder-1B-GGUF and pick a quantization — Q4_K_M is a sensible first choice.

Python (llama-cpp-python):

from llama_cpp import Llama

llm = Llama.from_pretrained(
    repo_id="ITLL/IBM-Grok4-UltraFast-Coder-1B-GGUF",
    filename="*Q4_K_M*",
    n_ctx=8192,
)
print(llm.create_chat_completion(messages=[
    {"role": "user", "content": "Write a Python script that watches a folder and renames screenshots by date."}
]))

The GGUF carries the Granite chat template in its metadata, so llama.cpp, Ollama, and most front-ends apply it automatically; pass --jinja to llama.cpp if you want to force the embedded Jinja template (recommended when using tools). The 128K context window is inherited from the base model, but on a 1B model quality degrades gracefully rather than perfectly at very long contexts — keep prompts focused.

10. Acknowledgements

Thanks to the IBM Granite Team for the open Apache 2.0 base model; to the dataset authors — Antired, lvogel123, Crownelius (three sets), TeichAI (two sets), nmayorga7, and Liontix — for publishing the Grok distillation data; to and to the DevQuasar and reedmayhew sources that several datasets build upon.

11. Citation

If you use this model, please cite it along with the base model:

@misc{itll2025ibmgrok4,
  author       = {{In The Loop Labs}},
  title        = {IBM-Grok4-UltraFast-Coder-1B-GGUF},
  year         = {2025},
  howpublished = {\url{https://huggingface.co/ITLL/IBM-Grok4-UltraFast-Coder-1B-GGUF}},
  note         = {Full fine-tune of ibm-granite/granite-4.0-h-1b on 9 Grok-distilled datasets; GGUF quantization}
}

Base model citation, as provided by IBM:

@misc{granite2025,
  author       = {{IBM Research}},
  title        = {Granite 4.0 Language Models},
  year         = {2025},
  howpublished = {\url{https://github.com/ibm-granite/granite-4.0-language-models}},
  note         = {Accessed: 2025-10-01}
}

12. Links

Downloads last month
363
GGUF
Model size
2B params
Architecture
granite
Hardware compatibility
Log In to add your hardware

2-bit

3-bit

4-bit

5-bit

6-bit

8-bit

16-bit

Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Model tree for ITLL/IBM-Grok4-UltraFast-Coder-1B-GGUF

Quantized
(48)
this model

Datasets used to train ITLL/IBM-Grok4-UltraFast-Coder-1B-GGUF

Collection including ITLL/IBM-Grok4-UltraFast-Coder-1B-GGUF