Entrit/llama.cpp-d2nl

Run Entrit d2 models with llama.cpp

This installs llama.cpp, the free program that runs AI models on your own computer, with Entrit's d2 kernel added. Our d2 models store each weight as one of 9 levels, and stock llama.cpp can't read that format. The install script downloads llama.cpp, adds our kernel, and builds it for your graphics card. With no graphics card it still builds and runs, but on the processor alone a 27B model takes minutes per word, so plan on a graphics card for these models.

You'll type a few commands into a terminal. You don't need to know how to program.

Which models it runs

Model	File to download	Runs here?
Qwen3.8-27B d2	Qwen3.8-27B-d2-D2NL.gguf (15.3 GB)	yes
Muse-Glimmer-30B d2	—	no: llama.cpp doesn't support this architecture yet. Use transformers.
DeepSeek-V4-Flash d2	—	no: needs DeepSeek's own inference code.

What you need

• A Mac with Apple Silicon (M1 or newer), or a Linux PC. Windows works through WSL (see the end of this page).
• About 5 GB of free disk space for the program, plus room for the model.
• Enough memory for the model. Qwen3.8-27B needs a graphics card with 24 GB (RTX 3090, 4090, or similar), or a Mac with at least 32 GB.
• Internet, for the first install.

Step 1: Open a terminal

• Mac: press Cmd-Space, type Terminal, press Enter.
• Linux: press Ctrl-Alt-T.

Every grey box below is a command. Copy it, paste it into the terminal, and press Enter.

Step 2: Install the build tools (once)

Mac:

xcode-select --install

A window pops up. Click Install and wait for it to finish. Then install Homebrew (a tool installer) if you don't have it, from https://brew.sh, and run:

brew install cmake

Linux (Ubuntu or Debian):

sudo apt update && sudo apt install -y build-essential cmake curl patch

Then add the part for your graphics card:

Your graphics card	Extra command
NVIDIA	Install the CUDA Toolkit from https://developer.nvidia.com/cuda-downloads
AMD or Intel	sudo apt install -y libvulkan-dev glslc
None	nothing; the processor version works but is far too slow for a 27B model

Step 3: Download and install

mkdir ~/entrit-llama && cd ~/entrit-llama
curl -fsSLO https://huggingface.co/Entrit/llama.cpp-d2nl/resolve/main/install.sh
bash install.sh

The first two lines make a folder and download the installer into it. The installer then downloads the rest of our files and llama.cpp, finds your graphics card, and builds for it. The first build takes 5 to 20 minutes. At the end you should see:

==> Self-test: Entrit kernel on ... vs the CPU
PASS
==> Done

PASS means our kernel gives the same answers on your graphics card as on the processor.

Step 4: Download a model

./get-model.sh Entrit/Qwen3.8-27B-d2-GGUF Qwen3.8-27B-d2-D2NL.gguf

It's 15 GB, so give it a while. If the download stops, run the same command again and it picks up where it left off.

Step 5: Chat with it

./run.sh models/Qwen3.8-27B-d2-D2NL.gguf

Wait until the terminal says server is listening. Then open http://localhost:8080 in your web browser and type a message. To stop, go back to the terminal and press Ctrl-C.

Other programs can talk to it too. It answers the OpenAI chat API at http://localhost:8080/v1.

Updating

When we release a new version of the kernel:

cd ~/entrit-llama
./install.sh --update

It downloads the newest files and rebuilds. Your downloaded models stay where they are.

If something goes wrong

What you see	What to do
'cmake' is missing (or c++, curl, patch)	Do Step 2 again.
could not download ...	Check your internet connection and try again.
Vulkan needs glslc	Run sudo apt install -y libvulkan-dev glslc, or install the Vulkan SDK from https://vulkan.lunarg.com
configure failed or build failed	Open build-config.log or build.log. The last 30 lines usually say what is missing.
the kernel patch did not apply	Run ./install.sh --update to get matching files.
The self-test doesn't say PASS	Open an issue on this page's Community tab with build.log and the name of your graphics card.
out of memory, or the model loads very slowly	The model is too big for your card. Add -ngl 30 to put only part of it on the graphics card: ./run.sh models/<file>.gguf -ngl 30
It picked the wrong backend	Choose one yourself: ./install.sh --backend vulkan (or cuda, metal, hip, cpu)

Windows

Install WSL, which gives you Linux inside Windows. Open PowerShell as administrator and run wsl --install, restart, then open "Ubuntu" from the Start menu. From there, follow the Linux steps. NVIDIA cards work inside WSL with NVIDIA's normal Windows driver.

How it works

d2 stores each weight as one of 9 levels, {0, ±1, ±2.83, ±5.20, ±8}, times one scale per group of 64 weights. The kernel adds a GGUF type, D2NL, that holds the level index in the same 4-bit block llama.cpp uses for its IQ4_NL type, and expands it through a 16-entry int8 table:

level   0    ±1    ±2.828   ±5.196   ±8
int8    0   ±11    ±31      ±57      ±88      (10.981 × level, within 0.19% of each level)

So every IQ4_NL code path serves D2NL with the table swapped in: dp4a and int8 tensor cores on NVIDIA, Vulkan shaders on AMD and Intel, Metal on Apple, and plain C on the processor. The AMD ROCm route (--backend hip) builds from the CUDA code but hasn't been tested; use --backend vulkan if it gives you trouble.

File	What it does
install.sh	downloads llama.cpp, adds our kernel, builds, runs the self-test
get-model.sh	downloads a model from Hugging Face into models/
run.sh	starts the chat server
patches/entrit-d2nl.diff	our changes to llama.cpp
LLAMA_CPP_VERSION	the exact llama.cpp version our changes apply to (from PrismML-Eng/llama.cpp)

After installing, the folder also holds llama.cpp/ (the source), build/ (the program) and models/. You can delete llama.cpp/ and build/ at any time; ./install.sh makes them again.

License and notice

The patch and scripts are released under the MIT License, the same as llama.cpp. The d2 codec is the subject of pending patent applications by Entrit Systems. Contact: eric@entrit.io