| <!-- SPDX-License-Identifier: Apache-2.0 --> |
| # oblivion_vllm |
| |
| vLLM plugin that registers the `OblivionForCausalLM` architecture so vLLM can |
| **serve** the [Oblivion](https://huggingface.co/IcyApril/Oblivion) checkpoint. |
| This is runtime code only — install it and serve. |
| |
| ```bash |
| pip install . |
| |
| VLLM_PLUGINS=oblivion_register vllm serve <oblivion-checkpoint> \ |
| --mamba-cache-mode=align --trust-remote-code |
| # add --quantization fp8 to serve in FP8 (dynamic, fits one GPU) |
| ``` |
| |
| The model is hybrid (its Gated-DeltaNet donor is a linear-attention layer), so |
| `--mamba-cache-mode=align` is required. Needs a vLLM build with MuseGlimmer |
| support; the plugin declares no dependencies of its own so it won't shadow it. |
| |
| Apache-2.0. |
| |
