Instructions to use TheBloke/MPT-7B-Instruct-GGML with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use TheBloke/MPT-7B-Instruct-GGML with Transformers:
# pip install -U transformers accelerate # Load model directly from transformers import AutoModel model = AutoModel.from_pretrained("TheBloke/MPT-7B-Instruct-GGML", device_map="auto") - Notebooks
- Google Colab
- Kaggle
Update README.md
Browse files
README.md
CHANGED
|
@@ -15,7 +15,7 @@ This is GGML format quantised 4-bit, 5-bit and 8-bit GGML models of [MosaicML's
|
|
| 15 |
|
| 16 |
This repo is the result of converting to GGML and quantising.
|
| 17 |
|
| 18 |
-
Please note that these MPT GGMLs are **not compatbile with llama.cpp**.
|
| 19 |
|
| 20 |
## Repositories available
|
| 21 |
|
|
|
|
| 15 |
|
| 16 |
This repo is the result of converting to GGML and quantising.
|
| 17 |
|
| 18 |
+
Please note that these MPT GGMLs are **not compatbile with llama.cpp**. Please see below for a list of tools known to work with these model files.
|
| 19 |
|
| 20 |
## Repositories available
|
| 21 |
|