Instructions to use DoppelReflEx/MiniusLight-24B with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use DoppelReflEx/MiniusLight-24B with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="DoppelReflEx/MiniusLight-24B")# Load model directly from transformers import AutoTokenizer, AutoModelForCausalLM tokenizer = AutoTokenizer.from_pretrained("DoppelReflEx/MiniusLight-24B") model = AutoModelForCausalLM.from_pretrained("DoppelReflEx/MiniusLight-24B", device_map="auto") - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use DoppelReflEx/MiniusLight-24B with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "DoppelReflEx/MiniusLight-24B" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "DoppelReflEx/MiniusLight-24B", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker
docker model run hf.co/DoppelReflEx/MiniusLight-24B
- SGLang
How to use DoppelReflEx/MiniusLight-24B with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "DoppelReflEx/MiniusLight-24B" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "DoppelReflEx/MiniusLight-24B", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "DoppelReflEx/MiniusLight-24B" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "DoppelReflEx/MiniusLight-24B", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }' - Docker Model Runner
How to use DoppelReflEx/MiniusLight-24B with Docker Model Runner:
docker model run hf.co/DoppelReflEx/MiniusLight-24B
| base_model: | |
| - TheDrummer/Cydonia-24B-v2 | |
| - PocketDoc/Dans-PersonalityEngine-V1.2.0-24b | |
| library_name: transformers | |
| tags: | |
| - mergekit | |
| - merge | |
| license: cc-by-nc-4.0 | |
| <style> | |
| @import url('https://fonts.googleapis.com/css2?family=Playwrite+CA+Guides&display=swap'); | |
| .playwrite-ca-guides-regular { | |
| font-family: "Playwrite CA Guides", cursive !important; | |
| font-weight: 400; | |
| font-style: normal; | |
| } | |
| body { | |
| margin:0; | |
| padding:0; | |
| font-size: 16px; | |
| } | |
| .main-container { | |
| background-color: #ebf3ff; | |
| border: 1px solid #466db9; | |
| border-radius: 8px; | |
| color: #050315; | |
| margin:16px; | |
| padding:16px; | |
| font-size: 16px; | |
| width: 95%; | |
| } | |
| h1, h2, h3 { | |
| color: #050315; | |
| margin-top: 16px; | |
| } | |
| .soft-blue-custom { | |
| color: #466db9 !important; | |
| } | |
| .alink { | |
| font-weight:400; | |
| text-decoration:none; | |
| } | |
| .main-banner-image { | |
| max-width:100%; | |
| border-radius:8px; | |
| align-self:center; | |
| justify-self: center; | |
| border: 1px solid #466db9; | |
| margin: 8px 16px | |
| } | |
| pre.code-block, pre { | |
| font-size: clamp(10px, 1.3vw, 14px); | |
| white-space: pre; | |
| margin: 1em 0; | |
| background-color: #1a1a1a; | |
| padding: 1em; | |
| border-radius: 4px; | |
| color: #a9a6de; | |
| overflow-x: auto; | |
| } | |
| p { | |
| font-weight:500; | |
| } | |
| .pb { | |
| padding-bottom: 8px; | |
| } | |
| .mb { | |
| margin-bottom: 8px; | |
| } | |
| .bold { | |
| font-weight: 600; | |
| } | |
| .secondary { | |
| color: #a9a6de; | |
| } | |
| .accent { | |
| color: #403bb7; | |
| } | |
| .tac { | |
| text-align:center; | |
| } | |
| .border-custom-dot { | |
| border: 1px dashed #466db9; | |
| border-radius: 16px; | |
| padding:0 8px; | |
| } | |
| .border-custom { | |
| border: 1px solid #466db9; | |
| border-radius: 8px; | |
| padding:0 8px; | |
| } | |
| .as { | |
| padding-left: 16px; | |
| } | |
| .as2 { | |
| padding-left: 24px; | |
| } | |
| </style> | |
| <html lang="en"> | |
| <head> | |
| <meta charset="UTF-8"> | |
| <meta name="viewport" content="width=device-width, initial-scale=1.0"> | |
| <link href="https://fonts.googleapis.com/css2?family=Playwrite+CA+Guides&display=swap" rel="stylesheet"> | |
| </head> | |
| <body> | |
| <div class="main-container"> | |
| <div class="playwrite-ca-guides-regular pb tac"> | |
| <h1 class="soft-blue-custom">MiniusLight-24B</h1> | |
| <h2 class="soft-blue-custom"><a href="https://huggingface.co/DoppelReflEx/MN-12B-Mimicore-Nocturne" class="accent bold">12B</a> - 24B-v1 - <a href="https://huggingface.co/DoppelReflEx/MiniusLight-24B-v1.01" class="accent bold">24B-v1.01</a></h2> | |
| <img src="https://cdn.donmai.us/original/8c/1f/__shorekeeper_wuthering_waves_drawn_by_void_0__8c1f85311759aea5d15ebe12bcfcab32.jpg" alt="cover image" class="main-banner-image"/> | |
| <a href="https://twitter.com/void_ling/status/1855164935141200230" class="alink soft-blue-custom">Origin Content (Click Here)</a> | |
| </div> | |
| <div class="info"> | |
| <div class="border-custom-dot mb"> | |
| <h2 class="soft-blue-custom">What is this?</h2> | |
| <div class="as"> | |
| <p> | |
| A nice, simple Slerp merge of 2 Mistral "Small" model and well-known HuggingFace users, <a class="alink soft-blue-custom bold" href="https://huggingface.co/TheDrummer/Cydonia-24B-v2">TheDrummer/Cydonia-24B-v2</a> | |
| <span class="secondary">&</span> | |
| <a class="alink soft-blue-custom bold" href="https://huggingface.co/PocketDoc/Dans-PersonalityEngine-V1.2.0-24b">PocketDoc/Dans-PersonalityEngine-V1.2.0-24b</a>. | |
| </p> | |
| <p>This version is the best merge version and recipe I have tried with a good eval scores. Strong in ERP, RP, Story Writing and orther purpose.</p> | |
| <p>Overall, nice to try model, if you want to try. :)</p> | |
| </div> | |
| </div> | |
| <div class="border-custom-dot mb"> | |
| <h2 class="soft-blue-custom">GGUF</h2> | |
| <h3 class="accent"><a href="https://huggingface.co/mradermacher/MiniusLight-24B-GGUF" class="accent bold">Static</a> - <a class="accent bold" href="https://huggingface.co/mradermacher/MiniusLight-24B-i1-GGUF">iMatrix</a></h3> | |
| </div> | |
| <div class="border-custom-dot"> | |
| <h2 class="soft-blue-custom">Other information</h2> | |
| <div class="as"> | |
| <h3><span class="soft-blue-custom">Chat Template? </span>ChatML<span class="soft-blue-custom">, of course!</span></h3> | |
| <h3 class="soft-blue-custom">Merge Method<h3/> | |
| <details class="border-custom"> | |
| <summary class="soft-blue-custom">Detail YAML Config</summary> | |
| <pre> | |
| { | |
| models: | |
| - model: TheDrummer/Cydonia-24B-v2 | |
| - model: PocketDoc/Dans-PersonalityEngine-V1.2.0-24b | |
| merge_method: slerp | |
| base_model: TheDrummer/Cydonia-24B-v2 | |
| parameters: | |
| t: [0.1, 0.3, 0.6, 0.3, 0.1] | |
| dtype: bfloat16 | |
| } | |
| </pre> | |
| </detail> | |
| </div> | |
| </div> | |
| </div> | |
| </div> | |
| </body> | |