Instructions to use CMSManhattan/JiRackDeltaNet_70b with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Notebooks
Google Colab
Kaggle
Local Apps Settings

How to use CMSManhattan/JiRackDeltaNet_70b with llama.cpp:

Install (macOS, Linux)

curl -LsSf https://llama.app/install.sh | sh
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf CMSManhattan/JiRackDeltaNet_70b:Q4_K_M
# Run inference directly in the terminal:
llama cli -hf CMSManhattan/JiRackDeltaNet_70b:Q4_K_M

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf CMSManhattan/JiRackDeltaNet_70b:Q4_K_M
# Run inference directly in the terminal:
llama cli -hf CMSManhattan/JiRackDeltaNet_70b:Q4_K_M

Build from source code

git clone https://github.com/ggml-org/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-app
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama serve -hf CMSManhattan/JiRackDeltaNet_70b:Q4_K_M
# Run inference directly in the terminal:
./build/bin/llama cli -hf CMSManhattan/JiRackDeltaNet_70b:Q4_K_M

Use Docker

docker model run hf.co/CMSManhattan/JiRackDeltaNet_70b:Q4_K_M

Llama new
LM Studio
Jan

vLLM

How to use CMSManhattan/JiRackDeltaNet_70b with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "CMSManhattan/JiRackDeltaNet_70b"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "CMSManhattan/JiRackDeltaNet_70b",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/CMSManhattan/JiRackDeltaNet_70b:Q4_K_M

Ollama
How to use CMSManhattan/JiRackDeltaNet_70b with Ollama:
```
ollama run hf.co/CMSManhattan/JiRackDeltaNet_70b:Q4_K_M
```
Unsloth Desktop

How to use CMSManhattan/JiRackDeltaNet_70b with Pi:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf CMSManhattan/JiRackDeltaNet_70b:Q4_K_M

Configure the model in Pi

# Install Pi:
npm install -g @earendil-works/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "llama-cpp": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "CMSManhattan/JiRackDeltaNet_70b:Q4_K_M"
        }
      ]
    }
  }
}

Run Pi

# Start Pi in your project directory:
pi

Docker Model Runner
How to use CMSManhattan/JiRackDeltaNet_70b with Docker Model Runner:
```
docker model run hf.co/CMSManhattan/JiRackDeltaNet_70b:Q4_K_M
```

Lemonade

How to use CMSManhattan/JiRackDeltaNet_70b with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull CMSManhattan/JiRackDeltaNet_70b:Q4_K_M

Run and chat with the model

lemonade run user.JiRackDeltaNet_70b-Q4_K_M

List all available models

lemonade list

Hermes Agent

How to use CMSManhattan/JiRackDeltaNet_70b with Hermes Agent:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf CMSManhattan/JiRackDeltaNet_70b:Q4_K_M

Configure Hermes

# Install Hermes:
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
# Point Hermes at the local server:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default CMSManhattan/JiRackDeltaNet_70b:Q4_K_M

Run Hermes

hermes

Atomic Chat

OpenClaw

How to use CMSManhattan/JiRackDeltaNet_70b with OpenClaw:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf CMSManhattan/JiRackDeltaNet_70b:Q4_K_M

Configure OpenClaw

# Install OpenClaw:
npm install -g openclaw@latest
# Register the local server and set it as the default model:
openclaw onboard --non-interactive --mode local \
  --auth-choice custom-api-key \
  --custom-base-url http://127.0.0.1:8080/v1 \
  --custom-model-id "CMSManhattan/JiRackDeltaNet_70b:Q4_K_M" \
  --custom-provider-id llama-cpp \
  --custom-compatibility openai \
  --custom-text-input \
  --accept-risk \
  --skip-health

Run OpenClaw

openclaw agent --local --agent main --message "Hello from Hugging Face"

kgrabko commited on 14 days ago

Commit

b9683ee

verified ·

1 Parent(s): 6099837

NOTICE.md: base model, changes, training data and software licenses

Browse files

Files changed (2) hide show

NOTICE.md +68 -0
README.md +1 -1

NOTICE.md ADDED Viewed

	@@ -0,0 +1,68 @@

+# Notices
+**JiRackDeltaNet 70B**
+Copyright 2026 Konstantin Vladimirovich Grabko (CMSManhattan)
+This model is a derivative work of Qwen3.5-27B and is distributed under the Apache License, Version 2.0 (see
+`LICENSE`). Keep this file and the license text when you redistribute the model or a derivative of it.
+## Base model
+| Work | Source | License |
+|---|---|---|
+| Qwen3.5-27B, by the Qwen team, Alibaba Cloud | https://huggingface.co/Qwen/Qwen3.5-27B | Apache License 2.0 |
+The weights in this repository are derived from the text-model weights of Qwen3.5-27B. The tokenizer and the chat
+template (`chat_template.jinja`, also embedded in the GGUF files) come from Qwen3.5-27B.
+## Changes made by CMSManhattan
+As required by section 4(b) of the Apache License 2.0, these are the changes made to the original work:
+1. **FFN widening**: the feed-forward layers were widened from 17,408 to 26,624, with the new units initialized so that
+   the model's output is unchanged before training (intermediate model: JiRackDeltaNet 36B).
+2. **Depth expansion**: 64 to 128 layers. Every group of 4 layers is followed by a copy of itself whose output
+   projections are zero, so each new block starts as an exact identity.
+3. **Supervised fine-tuning** of all parameters except the multi-token-prediction (MTP) head, which is unchanged
+   (374 optimizer steps, about 45 M tokens).
+4. **Model code** written by CMSManhattan (`JiRackDeltaNet_70b.py`, `chat_jirack.py`, `chat_fp8.py`); parameter names
+   follow the Hugging Face `Qwen3_5ForCausalLM` layout.
+5. **Conversions**: an FP8 block-quantized copy (`sft_final_fp8/`) and GGUF quantizations (`gguf/`); the chat template
+   in the GGUF files adds a built-in system prompt.
+## Training data
+The fine-tuning data was drawn from the following datasets. Their licenses apply to the datasets themselves.
+| Dataset | License | Used for |
+|---|---|---|
+| [nvidia/OpenCodeReasoning](https://huggingface.co/datasets/nvidia/OpenCodeReasoning) | CC BY 4.0 | code |
+| [nvidia/OpenCodeReasoning-2](https://huggingface.co/datasets/nvidia/OpenCodeReasoning-2) | CC BY 4.0 | code |
+| [Magpie-Align/Magpie-Qwen2.5-Coder-Pro-300K-v0.1](https://huggingface.co/datasets/Magpie-Align/Magpie-Qwen2.5-Coder-Pro-300K-v0.1) | generated with Qwen2.5-Coder-32B-Instruct (Apache 2.0) | code |
+| [nvidia/OpenMathReasoning](https://huggingface.co/datasets/nvidia/OpenMathReasoning) | CC BY 4.0 | math |
+| [open-r1/OpenR1-Math-220k](https://huggingface.co/datasets/open-r1/OpenR1-Math-220k) | Apache 2.0 | math |
+| [open-thoughts/OpenThoughts3-1.2M](https://huggingface.co/datasets/open-thoughts/OpenThoughts3-1.2M) (science part) | Apache 2.0 | science |
+| [HuggingFaceTB/smoltalk](https://huggingface.co/datasets/HuggingFaceTB/smoltalk) | Apache 2.0 | general chat |
+| [allenai/tulu-3-sft-mixture](https://huggingface.co/datasets/allenai/tulu-3-sft-mixture) (subsets under non-commercial licenses removed) | ODC-BY 1.0 | general chat |
+| [Team-ACE/ToolACE](https://huggingface.co/datasets/Team-ACE/ToolACE) | Apache 2.0 | tool calling |
+| [glaiveai/glaive-function-calling-v2](https://huggingface.co/datasets/glaiveai/glaive-function-calling-v2) | Apache 2.0 | tool calling |
+| [Salesforce/xlam-function-calling-60k](https://huggingface.co/datasets/Salesforce/xlam-function-calling-60k) | CC BY 4.0 | tool calling |
+| [rubenroy/GammaCorpus-Fact-QA-450k](https://huggingface.co/datasets/rubenroy/GammaCorpus-Fact-QA-450k) (questions only; answers regenerated, see below) | Apache 2.0 | fact QA |
+| [allenai/c4](https://huggingface.co/datasets/allenai/c4) | ODC-BY 1.0 | replay |
+| [wikimedia/wikipedia](https://huggingface.co/datasets/wikimedia/wikipedia) (13 languages) | CC BY-SA 4.0 / GFDL | replay |
+Attribution for CC BY 4.0 data: contains data from NVIDIA (OpenCodeReasoning, OpenCodeReasoning-2, OpenMathReasoning)
+and Salesforce (xLAM function-calling 60k), licensed under CC BY 4.0
+(https://creativecommons.org/licenses/by/4.0/).
+**AI-generated data.** Parts of the data were generated by AI models: the datasets above were largely generated with
+the models named on their cards, and CMSManhattan generated two parts with the DeepSeek API, whose terms of use allow
+outputs to be used for training: answers to the GammaCorpus fact questions, and situational dialogues written from
+summaries of TV and film scenes (no script text is included).
+## Software
+| Software | License | Use |
+|---|---|---|
+| [llama.cpp](https://github.com/ggml-org/llama.cpp) | MIT | GGUF conversion and quantization |
+| [flash-linear-attention](https://github.com/fla-org/flash-linear-attention) | MIT | Gated DeltaNet kernels during training (not included in this repository) |

README.md CHANGED Viewed

@@ -151,7 +151,7 @@ The reference chat scripts recompute the whole sequence for every new token (no
 ## License
-Apache License 2.0. The weights are derived from a Qwen model released under Apache 2.0. Changes by CMSManhattan (Konstantin Grabko): JiRack model code, FFN widening, depth expansion to 128 layers, SFT on the JiRack SFT mix, the FP8 and GGUF conversions. Keep this notice and the license text when redistributing.
 ## Contact

 ## License
+Apache License 2.0. The weights are derived from a Qwen model released under Apache 2.0. Changes by CMSManhattan (Konstantin Grabko): JiRack model code, FFN widening, depth expansion to 128 layers, SFT on the JiRack SFT mix, the FP8 and GGUF conversions. Keep this notice, `NOTICE.md` and the license text when redistributing.
 ## Contact