Entrit commited on
Commit
1006bd3
·
verified ·
1 Parent(s): 7acb327

Say plainly that CPU-only is too slow for 27B models

Browse files
Files changed (1) hide show
  1. README.md +3 -3
README.md CHANGED
@@ -13,8 +13,8 @@ tags:
13
  This installs **llama.cpp**, the free program that runs AI models on your own computer, with
14
  Entrit's **d2 kernel** added. Our d2 models store each weight as one of 9 levels, and stock
15
  llama.cpp can't read that format. The install script downloads llama.cpp, adds our kernel, and
16
- builds it for your graphics card. With no graphics card, it builds a slower version that runs on
17
- the processor.
18
 
19
  You'll type a few commands into a terminal. You don't need to know how to program.
20
 
@@ -64,7 +64,7 @@ Then add the part for your graphics card:
64
  |---|---|
65
  | NVIDIA | Install the CUDA Toolkit from https://developer.nvidia.com/cuda-downloads |
66
  | AMD or Intel | `sudo apt install -y libvulkan-dev glslc` |
67
- | None | nothing; you'll get the processor version |
68
 
69
  ## Step 3: Download and install
70
 
 
13
  This installs **llama.cpp**, the free program that runs AI models on your own computer, with
14
  Entrit's **d2 kernel** added. Our d2 models store each weight as one of 9 levels, and stock
15
  llama.cpp can't read that format. The install script downloads llama.cpp, adds our kernel, and
16
+ builds it for your graphics card. With no graphics card it still builds and runs, but on the
17
+ processor alone a 27B model takes minutes per word, so plan on a graphics card for these models.
18
 
19
  You'll type a few commands into a terminal. You don't need to know how to program.
20
 
 
64
  |---|---|
65
  | NVIDIA | Install the CUDA Toolkit from https://developer.nvidia.com/cuda-downloads |
66
  | AMD or Intel | `sudo apt install -y libvulkan-dev glslc` |
67
+ | None | nothing; the processor version works but is far too slow for a 27B model |
68
 
69
  ## Step 3: Download and install
70