Say plainly that CPU-only is too slow for 27B models
Browse files
README.md
CHANGED
|
@@ -13,8 +13,8 @@ tags:
|
|
| 13 |
This installs **llama.cpp**, the free program that runs AI models on your own computer, with
|
| 14 |
Entrit's **d2 kernel** added. Our d2 models store each weight as one of 9 levels, and stock
|
| 15 |
llama.cpp can't read that format. The install script downloads llama.cpp, adds our kernel, and
|
| 16 |
-
builds it for your graphics card. With no graphics card
|
| 17 |
-
|
| 18 |
|
| 19 |
You'll type a few commands into a terminal. You don't need to know how to program.
|
| 20 |
|
|
@@ -64,7 +64,7 @@ Then add the part for your graphics card:
|
|
| 64 |
|---|---|
|
| 65 |
| NVIDIA | Install the CUDA Toolkit from https://developer.nvidia.com/cuda-downloads |
|
| 66 |
| AMD or Intel | `sudo apt install -y libvulkan-dev glslc` |
|
| 67 |
-
| None | nothing;
|
| 68 |
|
| 69 |
## Step 3: Download and install
|
| 70 |
|
|
|
|
| 13 |
This installs **llama.cpp**, the free program that runs AI models on your own computer, with
|
| 14 |
Entrit's **d2 kernel** added. Our d2 models store each weight as one of 9 levels, and stock
|
| 15 |
llama.cpp can't read that format. The install script downloads llama.cpp, adds our kernel, and
|
| 16 |
+
builds it for your graphics card. With no graphics card it still builds and runs, but on the
|
| 17 |
+
processor alone a 27B model takes minutes per word, so plan on a graphics card for these models.
|
| 18 |
|
| 19 |
You'll type a few commands into a terminal. You don't need to know how to program.
|
| 20 |
|
|
|
|
| 64 |
|---|---|
|
| 65 |
| NVIDIA | Install the CUDA Toolkit from https://developer.nvidia.com/cuda-downloads |
|
| 66 |
| AMD or Intel | `sudo apt install -y libvulkan-dev glslc` |
|
| 67 |
+
| None | nothing; the processor version works but is far too slow for a 27B model |
|
| 68 |
|
| 69 |
## Step 3: Download and install
|
| 70 |
|