The public image ghcr.io/spinfinc/spinf-serve
Browse files
README.md
CHANGED
|
@@ -27,6 +27,7 @@ gives the same score on your GPU and on the hosted API at [spinf.com](https://sp
|
|
| 27 |
```sh
|
| 28 |
pip install "spinf-serve[server]"
|
| 29 |
spinf-serve serve spinf/gemma-4-12b-fp8 # downloads the checkpoint on first start; an OpenAI-compatible server on port 8000
|
|
|
|
| 30 |
```
|
| 31 |
|
| 32 |
- **[spinf-serve](https://github.com/spinfinc/spinf-serve)**: the engine and server (Apache 2.0), the quantization
|
|
|
|
| 27 |
```sh
|
| 28 |
pip install "spinf-serve[server]"
|
| 29 |
spinf-serve serve spinf/gemma-4-12b-fp8 # downloads the checkpoint on first start; an OpenAI-compatible server on port 8000
|
| 30 |
+
# or the image: docker run --gpus all -p 8000:8000 -v spinf-cache:/root/.cache ghcr.io/spinfinc/spinf-serve
|
| 31 |
```
|
| 32 |
|
| 33 |
- **[spinf-serve](https://github.com/spinfinc/spinf-serve)**: the engine and server (Apache 2.0), the quantization
|