llama-cpp-python
llama-cpp
wheel
windows
cuda-12
blackwell
sm_100
sm_90
sm_89
sm_86
sm_80
sm_75
sm_72
sm_70
sm_62
sm_61
cp312
Instructions to use trajis-tech/llama-cpp-python-trajis-tech-nonavx512-cuda with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- llama-cpp-python
How to use trajis-tech/llama-cpp-python-trajis-tech-nonavx512-cuda with llama-cpp-python:
# !pip install llama-cpp-python from llama_cpp import Llama llm = Llama.from_pretrained( repo_id="trajis-tech/llama-cpp-python-trajis-tech-nonavx512-cuda", filename="{{GGUF_FILE}}", )output = llm( "Once upon a time,", max_tokens=512, echo=True ) print(output)
- Notebooks
- Google Colab
- Kaggle
Update README.md
Browse files
README.md
CHANGED
|
@@ -2,7 +2,7 @@
|
|
| 2 |
license: mit
|
| 3 |
tags:
|
| 4 |
- llama-cpp
|
| 5 |
-
- llama-cpp-python
|
| 6 |
- windows
|
| 7 |
- cuda
|
| 8 |
- wheel
|
|
@@ -15,11 +15,13 @@ library_name: llama-cpp-python
|
|
| 15 |
|
| 16 |
Prebuilt wheel for:
|
| 17 |
|
|
|
|
| 18 |
- Windows x64
|
| 19 |
- Python 3.12 (cp312)
|
| 20 |
- CUDA enabled
|
| 21 |
- AVX512 disabled
|
| 22 |
- Supports NVIDIA 10 / 20 / 30 / 40 / 50 series GPUs
|
|
|
|
| 23 |
|
| 24 |
---
|
| 25 |
|
|
|
|
| 2 |
license: mit
|
| 3 |
tags:
|
| 4 |
- llama-cpp
|
| 5 |
+
- llama-cpp-python
|
| 6 |
- windows
|
| 7 |
- cuda
|
| 8 |
- wheel
|
|
|
|
| 15 |
|
| 16 |
Prebuilt wheel for:
|
| 17 |
|
| 18 |
+
- llama_cpp_python 0.3.16
|
| 19 |
- Windows x64
|
| 20 |
- Python 3.12 (cp312)
|
| 21 |
- CUDA enabled
|
| 22 |
- AVX512 disabled
|
| 23 |
- Supports NVIDIA 10 / 20 / 30 / 40 / 50 series GPUs
|
| 24 |
+
- Trajis SmartSRT 1.0.0
|
| 25 |
|
| 26 |
---
|
| 27 |
|