plugin/README.md

oblivion_vllm

vLLM plugin that registers the OblivionForCausalLM architecture so vLLM can serve the Oblivion checkpoint. This is runtime code only — install it and serve.

pip install .

VLLM_PLUGINS=oblivion_register vllm serve <oblivion-checkpoint> \
  --mamba-cache-mode=align --trust-remote-code
# add --quantization fp8 to serve in FP8 (dynamic, fits one GPU)

The model is hybrid (its Gated-DeltaNet donor is a linear-attention layer), so --mamba-cache-mode=align is required. Needs a vLLM build with MuseGlimmer support; the plugin declares no dependencies of its own so it won't shadow it.

Apache-2.0.