Instructions to use ibm-ai-platform/codellama-13b-accelerator with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use ibm-ai-platform/codellama-13b-accelerator with Transformers:
# pip install -U transformers accelerate # Load model directly from transformers import AutoModel model = AutoModel.from_pretrained("ibm-ai-platform/codellama-13b-accelerator", device_map="auto") - Notebooks
- Google Colab
- Kaggle
Update README.md
Browse files
README.md
CHANGED
|
@@ -184,6 +184,4 @@ python fms-extras/scripts/paged_speculative_inference.py \
|
|
| 184 |
--top_k_tokens_per_head=4,3,2,2,2,2,2 \
|
| 185 |
--prompt_type=code \
|
| 186 |
--compile \
|
| 187 |
-
```
|
| 188 |
-
|
| 189 |
-
Sample code can be found [here](https://github.com/foundation-model-stack/fms-extras)
|
|
|
|
| 184 |
--top_k_tokens_per_head=4,3,2,2,2,2,2 \
|
| 185 |
--prompt_type=code \
|
| 186 |
--compile \
|
| 187 |
+
```
|
|
|
|
|
|