11-47 commited on
Commit
ee3eeed
·
verified ·
1 Parent(s): f908a51

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +55 -112
README.md CHANGED
@@ -1,6 +1,5 @@
1
- base\_model:
2
-
3
- - ibm-granite/granite-4.0-h-1b
4
  datasets:
5
  - Antired/tradehax-xai-grok-trading-visual-prompts
6
  - lvogel123/arc-agi-1-grok-4
@@ -11,137 +10,81 @@ datasets:
11
  - TeichAI/grok-code-fast-1-1000x
12
  - nmayorga7/math-grok-4
13
  - Liontix/grok-code-fast-1-200x
14
- - Antired/tradehax-xai-grok-image-capabilities
15
  language:
16
- - en
17
- library\_name: transformers
 
 
 
18
  tags:
19
  - granite
20
  - ibm
21
- - granite-4.0
22
  - full-finetune
23
- - grok-distill
 
24
  - code
25
  - text-generation
26
- - gguf
27
- license: apache-2.0
28
-
29
  ---
 
30
 
31
- # IBM-Grok4-UltraFast-Coder-1B (GGUF)
32
-
33
- GGUF quantizations of [WithinUsAI/IBM-Grok4-UltraFast-Coder-1B](https://huggingface.co/WithinUsAI/IBM-Grok4-UltraFast-Coder-1B), a **full fine-tune** of IBM's [ibm-granite/granite-4.0-h-1b](https://huggingface.co/ibm-granite/granite-4.0-h-1b) on 10 Grok-4 / Grok Code Fast-derived datasets focused on coding, math, reasoning, and creative generation.
34
-
35
- ## Model Summary
36
-
37
-
38
- | | |
39
- | ---------------------- | -------------------------------------------------------------------------------------------------------------------- |
40
- | **Fine-tuned from** | [ibm-granite/granite-4.0-h-1b](https://huggingface.co/ibm-granite/granite-4.0-h-1b) (IBM Granite 4.0 Nano, instruct) |
41
- | **Fine-tuning method** | Full fine-tune (all parameters updated) |
42
- | **Training data** | 10 open Grok-4 / Grok Code Fast–derived datasets (see below) |
43
- | **Parameters** | \~1.46B (1,461,538,368) |
44
- | **Context length** | 128K tokens (131,072) |
45
- | **Languages** | English |
46
- | **License** | Apache 2.0 |
47
-
48
-
49
- ## Base Model: Granite-4.0-H-1B
50
-
51
- [Granite-4.0-H-1B](https://huggingface.co/ibm-granite/granite-4.0-h-1b) is IBM's lightweight instruct model from the Granite 4.0 Nano family, fine-tuned by IBM from Granite-4.0-H-1B-Base using supervised fine-tuning, reinforcement learning, and model merging. It inherits a **hybrid Mamba-2 / transformer ("granitemoehybrid") decoder architecture** combining:
52
-
53
- - 4 grouped-query-attention (GQA) transformer layers + 36 Mamba-2 layers
54
- - MLP with SwiGLU activation, RMSNorm, and shared input/output embeddings
55
- - NoPE position encoding (no RoPE on the hybrid variant)
56
-
57
- Key specs of the base model: embedding size 1536, attention head size 128, 12 attention heads / 4 KV heads, Mamba-2 state size 128, MLP hidden size 4096, validated 128K context. The hybrid Mamba-2 design gives the model substantially lower memory use and faster inference than a comparable dense transformer — which carries over to this fine-tune, making it well suited for edge, on-device, and latency-sensitive coding workloads.
58
-
59
- ## Fine-Tuning Datasets
60
-
61
- The full fine-tune used the following 10 datasets, largely consisting of Grok-4 and Grok Code Fast outputs covering code generation, math, ARC-style reasoning, brainstorming, trading/visual prompting, and general instruction data:
62
 
 
63
 
64
- | # | Dataset | Focus |
65
- | --- | ------------------------------------------------------------------------------------------------------------------------------------ | ---------------------------------- |
66
- | 1 | [TeichAI/grok-code-fast-1-1000x](https://huggingface.co/datasets/TeichAI/grok-code-fast-1-1000x) | Coding (Grok Code Fast) |
67
- | 2 | [Liontix/grok-code-fast-1-200x](https://huggingface.co/datasets/Liontix/grok-code-fast-1-200x) | Coding (Grok Code Fast) |
68
- | 3 | [nmayorga7/math-grok-4](https://huggingface.co/datasets/nmayorga7/math-grok-4) | Math |
69
- | 4 | [lvogel123/arc-agi-1-grok-4](https://huggingface.co/datasets/lvogel123/arc-agi-1-grok-4) | Abstract reasoning (ARC-AGI-1) |
70
- | 5 | [Crownelius/Hyper-Logic-Grok-V1](https://huggingface.co/datasets/Crownelius/Hyper-Logic-Grok-V1) | Logical reasoning |
71
- | 6 | [Crownelius/Hyper-Creative-Grok-V1](https://huggingface.co/datasets/Crownelius/Hyper-Creative-Grok-V1) | Creative writing |
72
- | 7 | [Crownelius/Hyper-UltraData-Grok-V1](https://huggingface.co/datasets/Crownelius/Hyper-UltraData-Grok-V1) | General knowledge / data |
73
- | 8 | [TeichAI/brainstorm-v3.1-grok-4-fast-200x](https://huggingface.co/datasets/TeichAI/brainstorm-v3.1-grok-4-fast-200x) | Brainstorming / ideation |
74
- | 9 | [Antired/tradehax-xai-grok-trading-visual-prompts](https://huggingface.co/datasets/Antired/tradehax-xai-grok-trading-visual-prompts) | Trading prompts / visual prompting |
75
- | 10 | [Antired/tradehax-xai-grok-image-capabilities](https://huggingface.co/datasets/Antired/tradehax-xai-grok-image-capabilities) | Image-capability prompting |
76
 
 
 
77
 
78
- > Note: the base Granite-4.0-H-1B is a **text-only** model, so the image- and visual-related datasets contribute text-only prompting behavior, not vision inputs.
79
-
80
- ## Training Details
81
-
82
-
83
-
84
- - **Method:** Full fine-tune (all weights updated, no LoRA/adapters)
85
- - **Hardware:**
86
- - **Hyperparameters:**
87
- - **Framework:**
88
- - **Chat template:** inherited from Granite-4.0-H-1B
89
-
90
- ## Intended Use
91
-
92
- - Local code generation and completion with low latency and a small footprint
93
- - Math, logic, and reasoning tasks on consumer hardware
94
- - Edge / on-device deployment via llama.cpp, LM Studio, Ollama, or any GGUF-compatible runtime
95
 
96
  ## Provided Quants
97
 
 
98
 
99
- | Type | Size (GB) | Notes |
100
- | :-------- | ---------: | :------------------------ |
101
- | Q2\_K | 0.8 | smallest, lowest quality |
102
- | Q3\_K\_S | 0.9 | |
103
- | Q3\_K\_M | 1.0 | lower quality |
104
- | Q3\_K\_L | 1.0 | |
105
- | IQ4\_XS | 1.0 | |
106
- | Q4\_K\_S | 1.1 | fast, recommended |
107
- | Q4\_K\_M | 1.1 | fast, recommended |
108
- | Q5\_K\_S | 1.3 | |
109
- | Q5\_K\_M | 1.3 | |
110
- | Q6\_K | 1.4 | very good quality |
111
- | Q8\_0 | 1.8 | fast, best quality |
112
- | f16 | 3.4 | 16 bpw, overkill |
113
-
114
-
115
- ## Usage
116
-
117
- Example with llama.cpp:
118
-
119
- ```bash
120
- ./llama-cli -m IBM-Grok4-UltraFast-Coder-1B.Q4_K_M.gguf -c 8192 -p "Write a Python function that merges two sorted lists."
121
- ```
122
 
123
- Or in Python with `llama-cpp-python`:
 
124
 
125
- ```python
126
- from llama_cpp import Llama
127
 
128
- llm = Llama(model_path="IBM-Grok4-UltraFast-Coder-1B.Q4_K_M.gguf", n_ctx=8192)
129
- print(llm.create_chat_completion(messages=[
130
- {"role": "user", "content": "Write a Python function that merges two sorted lists."}
131
- ]))
132
- ```
133
 
134
- If you are unsure how to use GGUF files, refer to one of [TheBloke's READMEs](https://huggingface.co/TheBloke/KafkaLM-70B-German-V0.1-GGUF) for more details, including how to concatenate multi-part files.
135
 
136
- ## Limitations & Safety
 
137
 
138
- - The model may produce inaccurate, biased, or unsafe outputs; it has not undergone additional safety alignment beyond the base Granite-4.0-H-1B.
139
- - Distilled from Grok-4 / Grok Code Fast outputs: behavior, style, and refusal patterns reflect those datasets, not IBM's Granite alignment.
140
- - Text-only: it cannot accept image inputs.
141
- - Not intended for high-stakes trading or financial decision-making, despite trading-related training data.
142
 
143
- ## Acknowledgements
 
 
144
 
145
- - [IBM Granite team](https://huggingface.co/ibm-granite) for the Granite 4.0 base models (Apache 2.0)
146
- - The creators of the 10 open datasets listed above
147
- - [mradermacher](https://huggingface.co/mradermacher) for GGUF quantization tooling
 
1
+ --
2
+ base_model: WithinUsAI/IBM-Grok4-Ultra.Fast.Coder-1B
 
3
  datasets:
4
  - Antired/tradehax-xai-grok-trading-visual-prompts
5
  - lvogel123/arc-agi-1-grok-4
 
10
  - TeichAI/grok-code-fast-1-1000x
11
  - nmayorga7/math-grok-4
12
  - Liontix/grok-code-fast-1-200x
13
+ - Antired/tradehax-xai-grok-image-capabilities
14
  language:
15
+ - en
16
+ library_name: transformers
17
+ mradermacher:
18
+ readme_rev: 1
19
+ quantized_by: mradermacher
20
  tags:
21
  - granite
22
  - ibm
 
23
  - full-finetune
24
+ - multi-gpu
25
+ - pytorch
26
  - code
27
  - text-generation
 
 
 
28
  ---
29
+ ## About
30
 
31
+ <!-- ### quantize_version: 2 -->
32
+ <!-- ### output_tensor_quantised: 1 -->
33
+ <!-- ### convert_type: hf -->
34
+ <!-- ### vocab_type: -->
35
+ <!-- ### tags: -->
36
+ <!-- ### quants: x-f16 Q4_K_S Q2_K Q8_0 Q6_K Q3_K_M Q3_K_S Q3_K_L Q4_K_M Q5_K_S Q5_K_M IQ4_XS -->
37
+ <!-- ### quants_skip: -->
38
+ <!-- ### skip_mmproj: -->
39
+ static quants of https://huggingface.co/WithinUsAI/IBM-Grok4-Ultra.Fast.Coder-1B
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
40
 
41
+ <!-- provided-files -->
42
 
43
+ ***For a convenient overview and download list, visit our [model page for this model](https://hf.tst.eu/model#IBM-Grok4-UltraFast-Coder-1B-GGUF).***
 
 
 
 
 
 
 
 
 
 
 
44
 
45
+ weighted/imatrix quants seem not to be available (by me) at this time. If they do not show up a week or so after the static ones, I have probably not planned for them. Feel free to request them by opening a Community Discussion.
46
+ ## Usage
47
 
48
+ If you are unsure how to use GGUF files, refer to one of [TheBloke's
49
+ READMEs](https://huggingface.co/TheBloke/KafkaLM-70B-German-V0.1-GGUF) for
50
+ more details, including on how to concatenate multi-part files.
 
 
 
 
 
 
 
 
 
 
 
 
 
 
51
 
52
  ## Provided Quants
53
 
54
+ (sorted by size, not necessarily quality. IQ-quants are often preferable over similar sized non-IQ quants)
55
 
56
+ | Link | Type | Size/GB | Notes |
57
+ |:-----|:-----|--------:|:------|
58
+ | [GGUF](https://huggingface.co/mradermacher/IBM-Grok4-UltraFast-Coder-1B-GGUF/resolve/main/IBM-Grok4-UltraFast-Coder-1B.Q2_K.gguf) | Q2_K | 0.8 | |
59
+ | [GGUF](https://huggingface.co/mradermacher/IBM-Grok4-UltraFast-Coder-1B-GGUF/resolve/main/IBM-Grok4-UltraFast-Coder-1B.Q3_K_S.gguf) | Q3_K_S | 0.9 | |
60
+ | [GGUF](https://huggingface.co/mradermacher/IBM-Grok4-UltraFast-Coder-1B-GGUF/resolve/main/IBM-Grok4-UltraFast-Coder-1B.Q3_K_M.gguf) | Q3_K_M | 1.0 | lower quality |
61
+ | [GGUF](https://huggingface.co/mradermacher/IBM-Grok4-UltraFast-Coder-1B-GGUF/resolve/main/IBM-Grok4-UltraFast-Coder-1B.Q3_K_L.gguf) | Q3_K_L | 1.0 | |
62
+ | [GGUF](https://huggingface.co/mradermacher/IBM-Grok4-UltraFast-Coder-1B-GGUF/resolve/main/IBM-Grok4-UltraFast-Coder-1B.IQ4_XS.gguf) | IQ4_XS | 1.0 | |
63
+ | [GGUF](https://huggingface.co/mradermacher/IBM-Grok4-UltraFast-Coder-1B-GGUF/resolve/main/IBM-Grok4-UltraFast-Coder-1B.Q4_K_S.gguf) | Q4_K_S | 1.1 | fast, recommended |
64
+ | [GGUF](https://huggingface.co/mradermacher/IBM-Grok4-UltraFast-Coder-1B-GGUF/resolve/main/IBM-Grok4-UltraFast-Coder-1B.Q4_K_M.gguf) | Q4_K_M | 1.1 | fast, recommended |
65
+ | [GGUF](https://huggingface.co/mradermacher/IBM-Grok4-UltraFast-Coder-1B-GGUF/resolve/main/IBM-Grok4-UltraFast-Coder-1B.Q5_K_S.gguf) | Q5_K_S | 1.3 | |
66
+ | [GGUF](https://huggingface.co/mradermacher/IBM-Grok4-UltraFast-Coder-1B-GGUF/resolve/main/IBM-Grok4-UltraFast-Coder-1B.Q5_K_M.gguf) | Q5_K_M | 1.3 | |
67
+ | [GGUF](https://huggingface.co/mradermacher/IBM-Grok4-UltraFast-Coder-1B-GGUF/resolve/main/IBM-Grok4-UltraFast-Coder-1B.Q6_K.gguf) | Q6_K | 1.4 | very good quality |
68
+ | [GGUF](https://huggingface.co/mradermacher/IBM-Grok4-UltraFast-Coder-1B-GGUF/resolve/main/IBM-Grok4-UltraFast-Coder-1B.Q8_0.gguf) | Q8_0 | 1.8 | fast, best quality |
69
+ | [GGUF](https://huggingface.co/mradermacher/IBM-Grok4-UltraFast-Coder-1B-GGUF/resolve/main/IBM-Grok4-UltraFast-Coder-1B.f16.gguf) | f16 | 3.4 | 16 bpw, overkill |
 
 
 
 
 
 
 
 
 
70
 
71
+ Here is a handy graph by ikawrakow comparing some lower-quality quant
72
+ types (lower is better):
73
 
74
+ ![image.png](https://www.nethype.de/huggingface_embed/quantpplgraph.png)
 
75
 
76
+ And here are Artefact2's thoughts on the matter:
77
+ https://gist.github.com/Artefact2/b5f810600771265fc1e39442288e8ec9
 
 
 
78
 
79
+ ## FAQ / Model Request
80
 
81
+ See https://huggingface.co/mradermacher/model_requests for some answers to
82
+ questions you might have and/or if you want some other model quantized.
83
 
84
+ ## Thanks
 
 
 
85
 
86
+ I thank my company, [nethype GmbH](https://www.nethype.de/), for letting
87
+ me use its servers and providing upgrades to my workstation to enable
88
+ this work in my free time.
89
 
90
+ <!-- end -->