Instructions to use Vortex5/Starry-Shadow-12B with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use Vortex5/Starry-Shadow-12B with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="Vortex5/Starry-Shadow-12B")# Load model directly from transformers import AutoTokenizer, AutoModelForCausalLM tokenizer = AutoTokenizer.from_pretrained("Vortex5/Starry-Shadow-12B") model = AutoModelForCausalLM.from_pretrained("Vortex5/Starry-Shadow-12B", device_map="auto") - Inference
- Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use Vortex5/Starry-Shadow-12B with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "Vortex5/Starry-Shadow-12B" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "Vortex5/Starry-Shadow-12B", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker
docker model run hf.co/Vortex5/Starry-Shadow-12B
- SGLang
How to use Vortex5/Starry-Shadow-12B with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "Vortex5/Starry-Shadow-12B" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "Vortex5/Starry-Shadow-12B", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "Vortex5/Starry-Shadow-12B" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "Vortex5/Starry-Shadow-12B", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }' - Docker Model Runner
How to use Vortex5/Starry-Shadow-12B with Docker Model Runner:
docker model run hf.co/Vortex5/Starry-Shadow-12B
| base_model: | |
| - Vortex5/NoctyxCosma-12B | |
| - Vortex5/Sunlit-Shadow-12B | |
| - Vortex5/Red-Synthesis-12B | |
| library_name: transformers | |
| tags: | |
| - mergekit | |
| - merge | |
| - roleplay | |
| <link rel="preconnect" href="https://fonts.googleapis.com"> | |
| <link rel="preconnect" href="https://fonts.gstatic.com" crossorigin> | |
| <link href="https://fonts.googleapis.com/css2?family=Inter:wght@400;600;700;800;900&family=Space+Grotesk:wght@400;600;700&family=JetBrains+Mono:wght@400;500&display=swap" rel="stylesheet"> | |
| <style> | |
| :root{ | |
| --night-0:#040814; | |
| --night-1:#070c1e; | |
| --nebula-blue:#2a6bff; | |
| --ice-blue:#8fb7ff; | |
| --aurora-green:#34f5c6; | |
| --text:#eaf0ff; | |
| --muted:rgba(234,240,255,.72); | |
| --line:rgba(143,183,255,.22); | |
| --font-title:"Space Grotesk",Inter,sans-serif; | |
| --font-body:Inter,sans-serif; | |
| --font-mono:"JetBrains Mono",monospace; | |
| --radius-xl:22px; | |
| --radius-lg:18px; | |
| } | |
| body{ | |
| margin:0; | |
| background: | |
| radial-gradient(1200px 700px at 15% 25%, rgba(42,107,255,.10), transparent 55%), | |
| linear-gradient(180deg,var(--night-0),var(--night-1)); | |
| color:var(--text); | |
| font-family:var(--font-body); | |
| } | |
| .ss12b{ | |
| max-width:1100px; | |
| margin:18px auto; | |
| padding:42px; | |
| border-radius:var(--radius-xl); | |
| background:linear-gradient(145deg,rgba(10,16,35,.92),rgba(7,12,30,.92)); | |
| border:1px solid var(--line); | |
| box-shadow:0 20px 60px rgba(0,0,0,.65); | |
| } | |
| .ss12b-hero{ | |
| padding:60px 36px 50px; | |
| border-radius:26px; | |
| background: | |
| radial-gradient(1000px 260px at 50% -20%, rgba(42,107,255,.35), transparent 60%), | |
| linear-gradient(145deg,rgba(12,20,46,.85),rgba(10,16,35,.85)); | |
| border:1px solid var(--line); | |
| text-align:center; | |
| } | |
| .ss12b-title{ | |
| margin:12px 0 10px; | |
| font-family:var(--font-title); | |
| font-weight:900; | |
| letter-spacing:4px; | |
| text-transform:uppercase; | |
| font-size: clamp(2.1rem, 3.5vw, 3.0rem); | |
| line-height:1.05; | |
| color:#eaf0ff; | |
| background:linear-gradient( | |
| 90deg, | |
| #b9d0ff 0%, | |
| #ffffff 25%, | |
| #6ea2ff 50%, | |
| #34f5c6 75%, | |
| #b9d0ff 100% | |
| ); | |
| -webkit-background-clip:text; | |
| background-clip:text; | |
| color:transparent; | |
| text-shadow: | |
| 0 0 18px rgba(42,107,255,.45), | |
| 0 0 36px rgba(143,183,255,.25), | |
| 0 0 64px rgba(52,245,198,.18); | |
| } | |
| .ss12b-subtitle{ | |
| margin-top:16px; | |
| max-width:860px; | |
| margin-left:auto; | |
| margin-right:auto; | |
| color:var(--muted); | |
| font-size:1.05rem; | |
| } | |
| .ss12b-image{ | |
| width: 100%; | |
| text-align: center; | |
| margin: 22px auto 0 auto; | |
| } | |
| .ss12b-image img{ | |
| display: inline-block; | |
| width: auto; | |
| max-width: 94%; | |
| height: auto; | |
| border-radius: 12px; | |
| border: 1.5px solid rgba(143,183,255,.35); | |
| box-shadow: | |
| 0 8px 24px rgba(0,0,0,.45), | |
| 0 0 18px rgba(42,107,255,.20); | |
| } | |
| .caption{ | |
| margin-top:10px; | |
| text-align:center; | |
| font-family:var(--font-mono); | |
| font-size:.85rem; | |
| color:rgba(234,240,255,.55); | |
| } | |
| .sections{ | |
| display:grid; | |
| grid-template-columns:1fr; | |
| gap:32px; | |
| margin-top:22px; | |
| } | |
| .card{ | |
| border-radius:var(--radius-lg); | |
| background: | |
| radial-gradient(600px 260px at 20% 10%, rgba(42,107,255,.14), transparent 60%), | |
| linear-gradient(145deg,rgba(10,16,35,.85),rgba(12,20,46,.60)); | |
| border:1px solid var(--line); | |
| padding:40px; | |
| min-height:420px; | |
| box-shadow:0 12px 36px rgba(0,0,0,.55); | |
| } | |
| .card h2{ | |
| margin:0 0 18px; | |
| font-family:var(--font-title); | |
| font-weight:800; | |
| letter-spacing:2.5px; | |
| text-transform:uppercase; | |
| display:flex; | |
| align-items:center; | |
| gap:10px; | |
| font-size: 1.7rem; | |
| } | |
| .dot{ | |
| width:10px;height:10px;border-radius:50%; | |
| background:var(--ice-blue); | |
| box-shadow:0 0 18px rgba(42,107,255,.4); | |
| } | |
| .dot.aurora{ | |
| background:var(--aurora-green); | |
| box-shadow:0 0 18px rgba(52,245,198,.35); | |
| } | |
| .card p,.card li{ | |
| color:var(--muted); | |
| line-height:1.7; | |
| } | |
| .card ul{padding-left:18px} | |
| .badge-row{ | |
| display:flex; | |
| gap:10px; | |
| flex-wrap:wrap; | |
| margin-top:22px; | |
| } | |
| .badge{ | |
| font-family:var(--font-mono); | |
| font-size:.82rem; | |
| padding:8px 10px; | |
| border-radius:12px; | |
| border:1px solid var(--line); | |
| background:rgba(4,8,20,.45); | |
| } | |
| .card{ | |
| font-size: 1.3rem; | |
| } | |
| @media(max-width:900px){ | |
| .ss12b{padding:22px} | |
| .card{padding:28px;min-height:unset} | |
| } | |
| </style> | |
| <div class="ss12b"> | |
| <div class="ss12b-hero"> | |
| <h1 class="ss12b-title">Starry-Shadow-12B</h1> | |
| </div> | |
| <div class="ss12b-image"> | |
| <img src="https://cdn-uploads.huggingface.co/production/uploads/6669a3a617b838fda45637b8/5ix2OtG3mDva9CTQD9a3Z.png"> | |
| </div> | |
| <div class="sections"> | |
| <section class="card"> | |
| <h2><span class="dot"></span>Model Overview</h2> | |
| <p><strong style="color:#fff">Starry-Shadow-12B</strong> was created by merging | |
| <a href="https://huggingface.co/Vortex5/Red-Synthesis-12B">Red-Synthesis-12B</a>, | |
| <a href="https://huggingface.co/Vortex5/NoctyxCosma-12B">NoctyxCosma-12B</a>, | |
| and | |
| <a href="https://huggingface.co/Vortex5/Sunlit-Shadow-12B">Sunlit-Shadow-12B</a> | |
| using a custom method. | |
| </p> | |
| <details> | |
| <summary>Merge configuration</summary> | |
| <pre><code>merge_method: acl | |
| base_model: Vortex5/Red-Synthesis-12B | |
| models: | |
| - model: Vortex5/NoctyxCosma-12B | |
| - model: Vortex5/Sunlit-Shadow-12B | |
| parameters: | |
| strength: 0.75 | |
| selectivity: 0.95 | |
| dtype: bfloat16 | |
| tokenizer: | |
| source: Vortex5/Red-Synthesis-12B</code></pre> | |
| </details> | |
| </section> | |
| <section class="card"> | |
| <h2><span class="dot aurora"></span>Intended Use</h2> | |
| <ul> | |
| <li>Creative writing</li> | |
| <li>Roleplay and character-driven dialogue</li> | |
| <li>Storytelling</li> | |
| </ul> | |
| </section> | |
| </div> |