Instructions to use LightshadeML/Roleshade-230M with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use LightshadeML/Roleshade-230M with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="LightshadeML/Roleshade-230M")# Load model directly from transformers import AutoModel model = AutoModel.from_pretrained("LightshadeML/Roleshade-230M", device_map="auto") - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use LightshadeML/Roleshade-230M with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "LightshadeML/Roleshade-230M" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "LightshadeML/Roleshade-230M", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker
docker model run hf.co/LightshadeML/Roleshade-230M
- SGLang
How to use LightshadeML/Roleshade-230M with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "LightshadeML/Roleshade-230M" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "LightshadeML/Roleshade-230M", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "LightshadeML/Roleshade-230M" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "LightshadeML/Roleshade-230M", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }' - Docker Model Runner
How to use LightshadeML/Roleshade-230M with Docker Model Runner:
docker model run hf.co/LightshadeML/Roleshade-230M
Roleshade 230M
Status: π§ Weights pending release β expected in 2β3 weeks.
Roleshade 230M is a 230-million-parameter model built by Lightshade for roleplay and creative writing. It is designed to run well on hardware people actually own β phones, laptops, and CPU-only machines β without giving up long-context conversations.
This repository is a placeholder. Model files, quantizations, and usage examples will be published here on release.
Highlights
- Purpose-built for character work. Trained for persona consistency, scene memory, and prose that resists the repetition collapse common at this parameter count.
- Fast prefill on CPUs and phones. Roleshade 230M uses a custom architecture that addresses the prefill bottleneck that makes long-context local models painful on non-GPU hardware. Loading a long chat history doesn't mean waiting on a spinner.
- Long context, anywhere. Deep, persistent roleplay sessions on a laptop, desktop, or phone.
- Extremely lightweight. At 230M parameters, this is Lightshade's smallest Roleshade variant β built for devices where even 1B-class models are too heavy.
- Local by default. Runs entirely on your own machine. Nothing leaves the device.
Intended Use
Primary: Interactive roleplay, character simulation, collaborative fiction, and creative writing assistance on constrained hardware.
Out of scope: Factual question answering, coding assistance, math, retrieval-grounded work, safety-critical applications, or any use where the model's output is treated as authoritative. Roleshade 230M is a creative model and will confidently produce fiction.
Usage
Code samples will be added when weights are published.
# Available on release
from transformers import AutoModelForCausalLM, AutoTokenizer
model_id = "lightshade/roleshade-230m"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(model_id)
Roleshade 230M will also be available directly on LSAI at launch β one click, no setup required.
Recommended sampler settings and prompt/character-card formatting will be documented here at release.
Limitations
- At 230M parameters, world knowledge is very limited. Expect confident hallucination on factual topics, more so than larger Roleshade variants.
- Optimized for creative output; general instruction-following outside of roleplay contexts should not be expected to be strong.
- Long-session coherence is solid for the size class but not unlimited, and simpler than larger models on complex, multi-thread scenes.
- English-first. Other languages are untested.
- Best suited for lightweight, casual roleplay rather than nuanced or highly demanding creative writing tasks β for those, consider a larger Roleshade variant.
Responsible Use
This model is intended for adult creative fiction and roleplay. Users are responsible for how they deploy it and for complying with applicable law and the terms of any platform they run it on. Outputs are fiction and should not be relied on as advice of any kind.
License
Apache 2.0 (subject to confirmation at release).
Citation
@misc{roleshade230m,
title = {Roleshade 230M},
author = {Lightshade},
year = {2026},
url = {https://huggingface.co/LightshadeML/Roleshade-230M}
}