Spaces:
Running
Running
| # Minimal patch notes | |
| If the existing `Leon4gr45/llama-swap` Space already starts a recent | |
| `llama-swap` + `llama-server` successfully, the smallest change is: | |
| 1. Replace its current `config.yaml` with the supplied `config.yaml`. | |
| 2. Add the `HF_TOKEN` Space secret after accepting the Gemma license. | |
| 3. Rebuild/restart. | |
| 4. Test `GET /v1/models`, then `POST /v1/chat/completions`. | |
| You do **not** need to replace the existing Dockerfile merely to change | |
| Bonsai/MiniCPM -> Gemma if its current llama.cpp build is recent enough. | |
| Use the supplied `Dockerfile` only if you want the Space reduced to a clean, | |
| generic llama-swap CPU deployment based on the current upstream image. | |
| ## Resource-first defaults | |
| - model: `google/gemma-3-4b-it-qat-q4_0-gguf:Q4_0` | |
| - exposed model id: `gemma-3-4b` | |
| - context allocation: 8192 | |
| - parallel slots: 1 | |
| - CPU threads: 2 | |
| - KV cache: q8_0 | |
| - idle TTL: 300 s | |
| - text-only by default (`--no-mmproj`) | |
| ## Multimodal | |
| Copy `config.multimodal.yaml` over `config.yaml` to allow llama.cpp to load | |
| Gemma's multimodal projector. This costs additional download time and memory, | |
| so it is intentionally not the default. | |