Instructions to use Bayhakl1/MiniGrimm with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use Bayhakl1/MiniGrimm with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="Bayhakl1/MiniGrimm")# pip install -U transformers accelerate # Load model directly from transformers import AutoTokenizer, AutoModelForCausalLM tokenizer = AutoTokenizer.from_pretrained("Bayhakl1/MiniGrimm") model = AutoModelForCausalLM.from_pretrained("Bayhakl1/MiniGrimm", device_map="auto") - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use Bayhakl1/MiniGrimm with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "Bayhakl1/MiniGrimm" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "Bayhakl1/MiniGrimm", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker
docker model run hf.co/Bayhakl1/MiniGrimm
- SGLang
How to use Bayhakl1/MiniGrimm with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "Bayhakl1/MiniGrimm" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "Bayhakl1/MiniGrimm", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "Bayhakl1/MiniGrimm" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "Bayhakl1/MiniGrimm", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }' - Docker Model Runner
How to use Bayhakl1/MiniGrimm with Docker Model Runner:
docker model run hf.co/Bayhakl1/MiniGrimm
|
Download README.md from Bayhakl1/MiniGrimm: direct link, hf CLI and curl.
- Browser
- Download file 5.95 kB
-
https://huggingface.co/Bayhakl1/MiniGrimm/resolve/main/README.md
- Command line
-
hf download hf://Bayhakl1/MiniGrimm/README.md
-
curl -L -o README.md https://huggingface.co/Bayhakl1/MiniGrimm/resolve/main/README.md
5.95 kB
| license: apache-2.0 | |
| language: | |
| - en | |
| library_name: transformers | |
| pipeline_tag: text-generation | |
| tags: | |
| - reasoning | |
| - muon | |
| datasets: | |
| - openai/gsm8k | |
| - HuggingFaceTB/smoltalk | |
| - HuggingFaceH4/ultrachat_200k | |
| - allenai/tulu-3-sft-personas-instruction-following | |
| - HuggingFaceH4/no_robots | |
| - HuggingFaceFW/fineweb-edu | |
| - HuggingFaceTB/smollm-corpus | |
| - wikimedia/wikipedia | |
| - Open-Orca/OpenOrca | |
| - TIGER-Lab/WebInstructSub | |
| - nvidia/OpenMathInstruct-2 | |
| - kaist-ai/CoT-Collection | |
| - microsoft/orca-math-word-problems-200k | |
| - allenai/WildChat-1M | |
| - Magpie-Align/Llama-3-Magpie-Pro-1M-v0.1 | |
| - openbmb/UltraInteract_sft | |
| - WizardLMTeam/WizardLM_evol_instruct_V2_196k | |
| # MiniGrimm (48M SFT) | |
| **MiniGrimm** is a tiny **48.29M-parameter** (42.00M non-embedding) decoder-only Transformer trained from scratch and fine-tuned with Supervised Fine-Tuning (SFT). | |
| This is a small-scale experimental model built to test a deep-and-thin Llama-style architecture trained with the **Muon** optimizer. Because it is under 50M parameters and has a 512-token context window, **keep your expectations realistic**—it is not a general-purpose assistant and will hallucinate facts or stumble on complex logic. That said, it can hold short casual chats and do basic step-by-step arithmetic surprisingly well for its size. | |
| --- | |
| ## Benchmarks | |
| Evaluated after SFT on **ArithMark 3.0** (arithmetic reasoning): | |
| | Benchmark | Metric | Score | | |
| | :--- | :--- | :---: | | |
| | **ArithMark 3.0** | Raw Accuracy | **38.1%** | | |
| | **ArithMark 3.0** | Normalized Accuracy | **38.2%** | | |
| *(Not bad for a 42M non-embedding model, though it will still make plenty of mistakes on multi-step word problems or larger numbers.)* | |
| --- | |
| Other Benchmarks: | |
| | Benchmark | Accuracy (Raw) | Accuracy (Norm) | Notes | | |
| | :--- | :---: | :---: | :--- | | |
| | **SciQ** | 70.7% (±1.44) | 63.0% (±1.53) | Science QA | | |
| | **PIQA** | 61.7% (±1.13) | 62.0% (±1.13) | Physical commonsense | | |
| | **Winogrande** | 52.4% (±1.40) | — | Pronoun resolution | | |
| | **BoolQ** | 48.6% (±0.87) | — | Yes/No reading comprehension | | |
| | **ARC-Easy** | 48.6% (±1.03) | 43.8% (±1.02) | Grade-school science | | |
| | **HellaSwag** | 29.1% (±0.45) | 31.7% (±0.46) | Sentence completion | | |
| | **OpenBookQA** | 19.2% (±1.76) | 30.0% (±2.05) | Elementary science facts | | |
| | **ARC-Challenge** | 22.3% (±1.22) | 27.2% (±1.30) | Harder science reasoning | | |
| | **LAMBADA (OpenAI)** | 26.9% (±0.62) | — | Perplexity: 83.15 (±3.71) | | |
| --- | |
| ## Architecture & Training Summary | |
| Instead of a wide, shallow network, MiniGrimm uses a tall, narrow layout (28 layers at `d_model=384`) paired with modern stability tweaks: | |
| | Spec | Detail | | |
| | :--- | :--- | | |
| | **Parameters** | 48.29M total (~42.00M non-embedding, tied embeddings) | | |
| | **Layers / Hidden Dim** | 28 layers, `d_model = 384`, `ffn_hidden = 960` (SwiGLU) | | |
| | **Attention** | Grouped-Query Attention (6 query heads, 2 KV heads) + QK-Norm | | |
| | **Positional Encoding** | RoPE (`theta = 10000.0`) | | |
| | **Logit Soft-Capping** | `30.0` | | |
| | **Context Window** | `512` tokens | | |
| | **Tokenizer** | Custom 16,384-vocab Byte-Level BPE (with single-digit number splitting) | | |
| | **Pretraining** | ~12.55B tokens (FineWeb-Edu, Cosmopedia-v2, Wikipedia, CoT/math, chat) using Muon + AdamW | | |
| | **SFT** | Assistant-only loss masking across GSM8K, SmolTalk, UltraChat-200k, Tülu-3, and No Robots | | |
| --- | |
| ## Usage | |
| Because MiniGrimm uses custom modeling code (for QK-Norm and logit soft-capping), you need to pass `trust_remote_code=True` when loading the model. | |
| ### Quickstart with `transformers` | |
| ```python | |
| import torch | |
| from transformers import AutoModelForCausalLM, AutoTokenizer | |
| repo_id = "Bayhakl1/MiniGrimm" | |
| device = "cuda" if torch.cuda.is_available() else "cpu" | |
| tokenizer = AutoTokenizer.from_pretrained(repo_id) | |
| model = AutoModelForCausalLM.from_pretrained( | |
| repo_id, | |
| trust_remote_code=True, | |
| ).to(device) | |
| model.eval() | |
| # Format the prompt using MiniGrimm's special chat tokens | |
| prompt = ( | |
| "<|system|>You are a helpful assistant.<|end|>" | |
| "<|user|>If I have 3 apples and buy 5 more, then eat 2, how many apples do I have?<|end|>" | |
| "<|assistant|>" | |
| ) | |
| inputs = tokenizer(prompt, return_tensors="pt").to(device) | |
| # Stop when the model emits either <|end|> or <|endoftext|> | |
| stop_token_ids = [ | |
| tokenizer.convert_tokens_to_ids("<|end|>"), | |
| tokenizer.eos_token_id, | |
| ] | |
| with torch.no_grad(): | |
| output_ids = model.generate( | |
| **inputs, | |
| max_new_tokens=150, | |
| do_sample=True, | |
| temperature=0.6, | |
| top_p=0.9, | |
| eos_token_id=stop_token_ids, | |
| pad_token_id=tokenizer.pad_token_id, | |
| ) | |
| # Slice off the prompt tokens so only the assistant's reply is printed | |
| reply_ids = output_ids[0][inputs["input_ids"].shape[1]:] | |
| print(tokenizer.decode(reply_ids, skip_special_tokens=True)) | |
| ``` | |
| ### Important Usage Notes | |
| 1. **Chat Template Format:** Always wrap turns using the exact single-token delimiters `<|system|>`, `<|user|>`, `<|assistant|>`, and `<|end|>`: | |
| ```text | |
| <|system|>Optional system prompt.<|end|><|user|>First message<|end|><|assistant|>First reply<|end|><|user|>Follow-up<|end|><|assistant|> | |
| ``` | |
| *(The `<|system|>...<|end|>` block is optional—you can start directly with `<|user|>`.)* | |
| 2. **512-Token Context Limit:** The model was trained with a maximum sequence length of **512 tokens** (prompt + generated tokens combined). Keep prompts short and `max_new_tokens` around `100–250` so the total length stays within 512 tokens. | |
| 3. **Greedy vs. Sampling:** For math questions, `do_sample=False` (greedy decoding) usually gives cleaner step-by-step arithmetic. For casual chat, `do_sample=True` with `temperature=0.6`–`0.7` helps avoid repetitive loops. | |
| --- | |
| ## Limitations | |
| * **Small Model Scale (~48M):** Limited factual recall—it will confidently make up dates, names, or trivia. | |
| * **Short Memory:** Anything past 512 tokens is outside its training window. | |
| * **English Only:** Trained almost exclusively on English text and basic math. |