Instructions to use JallyAI/Nomi-2 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use JallyAI/Nomi-2 with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("image-text-to-text", model="JallyAI/Nomi-2") messages = [ { "role": "user", "content": [ {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"}, {"type": "text", "text": "What animal is on the candy?"} ] }, ] pipe(text=messages)# Load model directly from transformers import AutoProcessor, AutoModelForMultimodalLM processor = AutoProcessor.from_pretrained("JallyAI/Nomi-2") model = AutoModelForMultimodalLM.from_pretrained("JallyAI/Nomi-2", device_map="auto") messages = [ { "role": "user", "content": [ {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"}, {"type": "text", "text": "What animal is on the candy?"} ] }, ] inputs = processor.apply_chat_template( messages, add_generation_prompt=True, tokenize=True, return_dict=True, return_tensors="pt", ).to(model.device) outputs = model.generate(**inputs, max_new_tokens=40) print(processor.decode(outputs[0][inputs["input_ids"].shape[-1]:])) - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use JallyAI/Nomi-2 with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "JallyAI/Nomi-2" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "JallyAI/Nomi-2", "messages": [ { "role": "user", "content": [ { "type": "text", "text": "Describe this image in one sentence." }, { "type": "image_url", "image_url": { "url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg" } } ] } ] }'Use Docker
docker model run hf.co/JallyAI/Nomi-2
- SGLang
How to use JallyAI/Nomi-2 with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "JallyAI/Nomi-2" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "JallyAI/Nomi-2", "messages": [ { "role": "user", "content": [ { "type": "text", "text": "Describe this image in one sentence." }, { "type": "image_url", "image_url": { "url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg" } } ] } ] }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "JallyAI/Nomi-2" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "JallyAI/Nomi-2", "messages": [ { "role": "user", "content": [ { "type": "text", "text": "Describe this image in one sentence." }, { "type": "image_url", "image_url": { "url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg" } } ] } ] }' - Unsloth Studio
How to use JallyAI/Nomi-2 with Unsloth Studio:
Install Unsloth Studio (macOS, Linux, WSL)
curl -fsSL https://unsloth.ai/install.sh | sh # Run unsloth studio unsloth studio -H 0.0.0.0 -p 8888 # Then open http://localhost:8888 in your browser # Search for JallyAI/Nomi-2 to start chatting
Install Unsloth Studio (Windows)
irm https://unsloth.ai/install.ps1 | iex # Run unsloth studio unsloth studio -H 0.0.0.0 -p 8888 # Then open http://localhost:8888 in your browser # Search for JallyAI/Nomi-2 to start chatting
Using HuggingFace Spaces for Unsloth
# No setup required # Open https://huggingface.co/spaces/unsloth/studio in your browser # Search for JallyAI/Nomi-2 to start chatting
Load model with FastModel
pip install unsloth from unsloth import FastModel model, tokenizer = FastModel.from_pretrained( model_name="JallyAI/Nomi-2", max_seq_length=2048, ) - Docker Model Runner
How to use JallyAI/Nomi-2 with Docker Model Runner:
docker model run hf.co/JallyAI/Nomi-2
Nomi 2.0
Introduction
Introducing Nomi 2, an 4B LLM based on Qwen 3.5 4B, that thinks more efficient while keeping great performance.
In this training, we used another base and improved it's reasoning. It's base, Qwen 3.5 4B was great, but overthought most of the requests und looped. Nomi 2 fixes that. Even at more complicated prompts it only thinks for about 10 sec on consumer hardware (~500 Reasoning tokens).
Nomi 2 works by using one specific reasoning style: Instead of human like CoT, Nomi uses what I like to call RASV. In it's reasoning, it got 4 main parts:
- Restatement
- Approach
- Step-by-step derivation
- Verification
These steps help the LLM to keep the reasoning short, while keeping it effective, and avoiding loops.
🌟 Key Features & Improvements
- Architecture: Qwen-3.5-4B (runs on 8 GB VRAM GPUs like the RTX 4060).
- Multilingual Support: Can understand and generate text in German and English and many other languages.
- Efficiency: Runs with 60+ Tokens/s at Q4 with an Context Window of almost 100k tokens on consumer hardware.
📃 Evals and Showcase
Evals
| Model Name | GSM8K* | GPQA Diamond (0-Shot)* |
|---|---|---|
| Nomi 2.0 | 41/100 | 32/50 |
*Benchmarks were done without reasoning.
Reasoning Style Showcase
Q: What is 8x8x8+43+(432+432x3)-39?
Nomi
Restatement
We need to evaluate the arithmetic expression 8 × 8 × 8 + 43 + (432 + 432 × 3) − 39. The operations include multiplication, addition, and subtraction, and we must apply the standard order of operations (PEMDAS/BODMAS).
Approach
Multiplication has higher precedence than addition and subtraction, so we compute all products first, then handle the parentheses grouping, then add/subtract from left to right. Specifically:
- Inside the parentheses: 432 × 3 first, then add 432
- Outside: 8 × 8 × 8, then combine everything
Step-by-step derivation
Step 1 — Inside parentheses: 432 × 3 = 1296. Then 432 + 1296 = 1728. So (432 + 432 × 3) = 1728.
Step 2 — Outside: 8 × 8 × 8 = 64 × 8 = 512.
Step 3 — Combine: 512 + 43 + 1728 − 39.
- 512 + 43 = 555
- 555 + 1728 = 2283
- 2283 − 39 = 2244
Answer: 2244.
Verification
Check by recombining: 512 + 43 + 1728 = 2283, and 2283 − 39 = 2244. Alternatively, note 512 + 1728 = 2240, then 2240 + 43 − 39 = 2240 + 4 = 2244. Both methods agree.
2244
To evaluate 8 × 8 × 8 + 43 + (432 + 432 × 3) − 39, we follow the order of operations: multiplication first, then parentheses, then addition and subtraction from left to right.
Inside the parentheses, we compute 432 × 3 = 1296, then 432 + 1296 = 1728. Outside, we compute 8 × 8 × 8 = 64 × 8 = 512. The expression now reads 512 + 43 + 1728 − 39.
Adding from left to right: 512 + 43 = 555, then 555 + 1728 = 2283, and finally 2283 − 39 = 2244.
As a sanity check, I can reorder the additions: 512 + 1728 = 2240, and 2240 + 43 − 39 = 2240 + 4 = 2244. The two independent paths give the same result, confirming the answer.
Qwen
Here's my thought process for solving this math problem:
Analyze the Request: The user wants to solve a specific arithmetic expression: $8 \times 8 \times 8 + 43 + (432 + 432 \times 3) - 39$.
Identify the Operations: I need to perform multiplication, addition, and subtraction.
Apply Order of Operations (PEMDAS/BODMAS):
- P/B: Parentheses/Brackets. I need to solve the expression inside the parentheses first.
- E/O: Exponents/Orders. There are none here (unless $8 \times 8 \times 8$ is interpreted as a power, but standard notation implies multiplication).
- MD/DM: Multiplication and Division (left to right).
- AS: Addition and Subtraction (left to right).
Step-by-Step Breakdown:
Expression: $8 \times 8 \times 8 + 43 + (432 + 432 \times 3) - 39$
Step 1: Parentheses
- Inside the parentheses: $(432 + 432 \times 3)$
- Within the parentheses, multiplication comes before addition.
- Calculate $432 \times 3$.
- $400 \times 3 = 1200$
- $30 \times 3 = 90$
- $2 \times 3 = 6$
- $1200 + 90 + 6 = 1296$
- Now add 432 to the result.
- $432 + 1296$
- $2 + 6 = 8$
- $3 + 9 = 12$ (carry 1)
- $4 + 2 + 1 = 7$
- $1$
- Result: $1728$
- So, the expression becomes: $8 \times 8 \times 8 + 43 + 1728 - 39$
Step 2: Multiplication (outside the parentheses)
- Calculate $8 \times 8 \times 8$.
- $8 \times 8 = 64$
- $64 \times 8 = 512$
- (Note: $8^3 = 512$)
- The expression becomes: $512 + 43 + 1728 - 39$
Step 3: Addition and Subtraction (Left to Right)
- Current expression: $512 + 43 + 1728 - 39$
- First addition: $512 + 43$
- $512 + 40 = 552$
- $552 + 3 = 555$
- Current expression: $555 + 1728 - 39$
- Second addition: $555 + 1728$
- $500 + 1700 = 2200$
- $55 + 28 = 83$
- $2200 + 83 = 2283$
- Current expression: $2283 - 39$
- Final subtraction: $2283 - 39$
- $2283 - 30 = 2253$
- $2253 - 9 = 2244$
- Alternatively: $2283 - 40 + 1 = 2243 + 1 = 2244$
Double Check:
Original: $8 \times 8 \times 8 + 43 + (432 + 432 \times 3) - 39$
$8^3 = 512$
$432 \times 3 = 1296$
$432 + 1296 = 1728$
Sum so far: $512 + 43 + 1728 - 39$
$512 + 43 = 555$
$555 + 1728 = 2283$
$2283 - 39 = 2244$
Wait, let me re-read the expression carefully.
$8 \times 8 \times 8 + 43 + (432 + 432 \times 3) - 39$
Is it possible the user meant $432 + (432 \times 3)$? Yes, that's what I did.
Is it possible there is a typo in my reading? "8x8x8+43+(432+432x3)-39"
Let's check the numbers again.
$512 + 43 = 555$
$432 \times 3 = 1296$
$432 + 1296 = 1728$
$555 + 1728 = 2283$
$2283 - 39 = 2244$
Let's check if there's any ambiguity. "432+432x3" is inside parentheses.
Multiplication takes precedence inside the parentheses.
$432 + 1296 = 1728$. Correct.
$8^3 = 512$. Correct.
$512 + 43 + 1728 - 39$.
$512 + 43 = 555$.
$555 + 1728 = 2283$.
$2283 - 39 = 2244$.
Wait, is there a trick?
$432 + 432 \times 3$...
$432(1+3) = 432 \times 4 = 1728$.
Ah, that's a nice shortcut. $432 \times 4$.
$432 \times 2 = 864$.
$864 \times 2 = 1728$.
Yes, calculation holds up.
$8 \times 8 \times 8 = 512$.
$512 + 43 + 1728 - 39$.
$512 + 1728 = 2240$.
$2240 + 43 = 2283$.
$2283 - 39 = 2244$.
Wait, $2240 + 43 - 39$.
$43 - 39 = 4$.
$2240 + 4 = 2244$.
This confirms the previous calculation ($2283 - 39 = 2244$) but is slightly faster to verify.
Construct Final Output:
- State the expression.
- Show the order of operations steps clearly.
- Calculate the power ($8^3$).
- Calculate the multiplication inside parentheses.
- Add inside parentheses.
- Perform the final additions and subtractions.
- State the final answer clearly.
Final Polish:
- Expression: $8 \times 8 \times 8 + 43 + (432 + 432 \times 3) - 39$
- Step 1: Parentheses. $432 \times 3 = 1296$. Then $432 + 1296 = 1728$.
- Step 2: Multiplication. $8 \times 8 \times 8 = 512$.
- Step 3: Remaining additions/subtractions. $512 + 43 + 1728 - 39$.
- $555 + 1728 - 39$.
- $2283 - 39 = 2244$.
Looks solid. Ready to write.
Wait, just a quick sanity check on the numbers. $432 + 432 \times 3$. $432 \times (1+3) = 432 \times 4$. $400 \times 4 = 1600$. $32 \times 4 = 128$. $1600 + 128 = 1728$. Correct.
$8^3 = 512$. Correct.
$512 + 43 = 555$. Correct.
$555 + 1728 = 2283$. Correct.
$2283 - 39 = 2244$. Correct.
All checks pass.
Wait, could the input be interpreted differently? "8x8x8+43+(432+432x3)-39" Is it possible "432" is a variable? No, context implies arithmetic. Is it possible "8x8x8" is $8 \times 8 \times 8$? Yes. Is it possible the user meant a specific math puzzle with a trick? Usually, these are just straight arithmetic. The number $432$ is specific. $432 \times 3 = 1296$. $432 + 1296 = 1728$. $8^3 = 512$. $512 + 43 + 1728 - 39$. Maybe there is a pattern? $1728 = 12^3$. $512 = 8^3$. $43 = 43$. $-39$. So $8^3 + 43 + 12^3 - 39$. $8^3 + 12^3 + (43 - 39)$. $512 + 1728 + 4$. $2240 + 4 = 2244$. Interesting, but not a "trick", just observation. The calculation stands.
Okay, proceed with generating the response. Structure:
- State order of operations (PEMDAS).
- Solve parentheses.
- Solve multiplication.
- Solve addition/subtraction.
- Final result. Keep it clear and concise.
Wait, re-read the input string one more time to ensure no typos in my transcription. "8x8x8+43+(432+432x3)-39" My transcription: $8 \times 8 \times 8 + 43 + (432 + 432 \times 3) - 39$ Matches perfectly.
Ready.
To solve the expression 8x8x8+43+(432+432x3)-39, we follow the order of operations (PEMDAS/BODMAS): Parentheses, Exponents/Multiplication, then Addition and Subtraction.
Step 1: Simplify the parentheses
Inside the parentheses, we have: (432 + 432x3)
Multiplication comes before addition.
- $432 \times 3 = 1296$
- $432 + 1296 = 1728$
Now the expression looks like this: $8 \times 8 \times 8 + 43 + 1728 - 39$
Step 2: Perform the multiplication
- $8 \times 8 \times 8 = 512$
Now the expression looks like this: $512 + 43 + 1728 - 39$
Step 3: Perform addition and subtraction (left to right)
- $512 + 43 = 555$
- $555 + 1728 = 2283$
- $2283 - 39 = 2244$
Answer: 2244
🧠 Training Details
- Base Model:
Qwen/Qwen3.5-4B - Fine-tuning: SFT (Supervised Fine-Tuning).
- Training Tool: Unsloth (for 4-bit optimized training).
😎 Cool License
Feel free to use or improve Nomi! Benchmark results are always welcome.
- Downloads last month
- 7