Instructions to use avlp12/Solar-Open2-250B-Alis-MLX-Dynamic with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- MLX
How to use avlp12/Solar-Open2-250B-Alis-MLX-Dynamic with MLX:
# Make sure mlx-lm is installed # pip install --upgrade mlx-lm # Generate text with mlx-lm from mlx_lm import load, generate model, tokenizer = load("avlp12/Solar-Open2-250B-Alis-MLX-Dynamic") prompt = "Write a story about Einstein" messages = [{"role": "user", "content": prompt}] prompt = tokenizer.apply_chat_template( messages, add_generation_prompt=True ) text = generate(model, tokenizer, prompt=prompt, verbose=True) - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- LM Studio
- Pi
How to use avlp12/Solar-Open2-250B-Alis-MLX-Dynamic with Pi:
Start the MLX server
# Install MLX LM: uv tool install mlx-lm # Start a local OpenAI-compatible server: mlx_lm.server --model "avlp12/Solar-Open2-250B-Alis-MLX-Dynamic"
Configure the model in Pi
# Install Pi: npm install -g @earendil-works/pi-coding-agent # Add to ~/.pi/agent/models.json: { "providers": { "mlx-lm": { "baseUrl": "http://localhost:8080/v1", "api": "openai-completions", "apiKey": "none", "models": [ { "id": "avlp12/Solar-Open2-250B-Alis-MLX-Dynamic" } ] } } }Run Pi
# Start Pi in your project directory: pi
- MLX LM
How to use avlp12/Solar-Open2-250B-Alis-MLX-Dynamic with MLX LM:
Generate or start a chat session
# Install MLX LM uv tool install mlx-lm # Interactive chat REPL mlx_lm.chat --model "avlp12/Solar-Open2-250B-Alis-MLX-Dynamic"
Run an OpenAI-compatible server
# Install MLX LM uv tool install mlx-lm # Start the server mlx_lm.server --model "avlp12/Solar-Open2-250B-Alis-MLX-Dynamic" # Calling the OpenAI-compatible server with curl curl -X POST "http://localhost:8080/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "avlp12/Solar-Open2-250B-Alis-MLX-Dynamic", "messages": [ {"role": "user", "content": "Hello"} ] }' - Hermes Agent
How to use avlp12/Solar-Open2-250B-Alis-MLX-Dynamic with Hermes Agent:
Start the MLX server
# Install MLX LM: uv tool install mlx-lm # Start a local OpenAI-compatible server: mlx_lm.server --model "avlp12/Solar-Open2-250B-Alis-MLX-Dynamic"
Configure Hermes
# Install Hermes: curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash hermes setup # Point Hermes at the local server: hermes config set model.provider custom hermes config set model.base_url http://127.0.0.1:8080/v1 hermes config set model.default avlp12/Solar-Open2-250B-Alis-MLX-Dynamic
Run Hermes
hermes
- Atomic Chat
- OpenClaw
How to use avlp12/Solar-Open2-250B-Alis-MLX-Dynamic with OpenClaw:
Start the MLX server
# Install MLX LM: uv tool install mlx-lm # Start a local OpenAI-compatible server: mlx_lm.server --model "avlp12/Solar-Open2-250B-Alis-MLX-Dynamic"
Configure OpenClaw
# Install OpenClaw: npm install -g openclaw@latest # Register the local server and set it as the default model: openclaw onboard --non-interactive --mode local \ --auth-choice custom-api-key \ --custom-base-url http://127.0.0.1:8080/v1 \ --custom-model-id "avlp12/Solar-Open2-250B-Alis-MLX-Dynamic" \ --custom-provider-id mlx-lm \ --custom-compatibility openai \ --custom-text-input \ --accept-risk \ --skip-health
Run OpenClaw
openclaw agent --local --agent main --message "Hello from Hugging Face"
Solar-Open2-250B โ Alis MLX Dynamic (DWQ) Builds
Built with Solar โ this is a derivative of upstage/Solar-Open2-250B, distributed under the Upstage Solar License. A copy of the license ships in every branch.
MLX (Apple Silicon) port of Solar Open 2 (250B-A15B Hybrid-Attention MoE) in three quantization tiers, produced with alis-dwq โ data-aware quantization with teacher-guided distillation (DWQ). Both distilled builds are fine-tuned against the Q8 reference under a per-slice acceptance gate, with an EN sentinel gate as the final safety net.
โ Status: all three tiers final. Q v3 (4-bit/g32 + DWQ, lr 5e-7 ร 3 rounds): EN loop cleared with teacher-level diversity (distinct4 0.399 vs teacher 0.411), code KLD better than its own base. F v2 (2.34 bpw, 73.3 GB): KLD 0.42 (โ76% vs the legacy 2-bit path), all loops clean, EN sentinel ACCEPT. A fourth tier (M, 4-bit/g64) was trained twice (Q-teacher and Q8-teacher) but is not shipped โ it lost every metric to Q v3 at the same memory tier.
Builds (branches)
One repo, three weights โ pick a branch by machine memory:
| branch | build | recipe | measured bpw | size | fits |
|---|---|---|---|---|---|
main |
Q v3 โ daily driver, best quality | 4-bit / g32 + DWQ (Q8 teacher, per-slice gate, lr 5e-7 ร 3 rounds) | 5.001 | 156.5 GB (145.7 GiB, 31 shards) | 192 GiB stock (tight) / 256 GiB comfortable |
t96 |
F v2 โ smallest usable | 2-bit / g128 experts only + clip + DWQ rank 8 (Q8 teacher); chokepoints 8b, embed+head 6b, attn/shared 4b via quant_predicate_builder |
2.344 | 73.3 GB (68.3 GiB, 15 shards) + 4.0 GB LoRA adapters | 96 GiB stock (tight) / 128 GiB comfortable |
t384 |
T โ Q8 reference / teacher | 8-bit / g64 | 8.500 | 265.9 GB (247.7 GiB, 62 shards) | 384 GiB stock / 512 GiB for full 1M ctx |
Sizes are decimal GB (10โน bytes, as shown on the HF page) with binary GiB (2ยณโฐ bytes) in parentheses. Measured bpw is computed on the weight shards over 250.29B params; the t96 adapters (4.0 GB) ship separately and are not fused into the shards.
Every build keeps all MoE router gates at 8-bit / g64 (48/48 layers) โ router quantization was found to be the dominant quality risk, so only expert/attention weights are quantized per the tier recipe. Why no mid tier? An M build (4-bit/g64, 131โ141 GB) was distilled twice (Q-teacher and Q8-teacher arms). The Q8 arm fixed its EN regression but still lost every metric to Q v3 at the same memory tier (โฅ160 GB machines), so it is retained locally as a control, not shipped.
Install & run
โ ๏ธ Requires the
solar_open2architecture, which is not yet in PyPI mlx-lm (0.31.3). Stockpip install mlx-lmwill fail withValueError: Model type solar_open2 not supported. Two options:# option A โ the fork that adds solar_open2 (recommended; the fork's default branch is `solar-open2`) pip install "git+https://github.com/avlp12/mlx-lm.git"Option B โ keep stock mlx-lm and drop the bundled
solar_open2.py(shipped in every branch) into.../site-packages/mlx_lm/models/. Same file, same result.
# main = Q v3 (recommended)
mlx_lm.generate --model avlp12/Solar-Open2-250B-Alis-MLX-Dynamic \
--prompt "์๋
ํ์ธ์, ๊ฐ๋จํ ์๊ธฐ์๊ฐ๋ฅผ ํด์ฃผ์ธ์." --max-tokens 512
Branch selection (t96 = F v2, t384 = T Q8) uses the Python API โ the CLI has no --revision flag:
from mlx_lm import load, generate
model, tok = load("avlp12/Solar-Open2-250B-Alis-MLX-Dynamic", revision="t96") # F v2 (77 GB)
# model, tok = load("avlp12/Solar-Open2-250B-Alis-MLX-Dynamic", revision="t384") # T Q8 (266 GB)
print(generate(model, tok, prompt="Explain MoE routing in three sentences.", max_tokens=512, verbose=False))
CLI-only alternative: hf download avlp12/Solar-Open2-250B-Alis-MLX-Dynamic --revision t96, then point --model at the downloaded snapshot directory.
F v2 LoRA adapters (t96 branch only): the DWQ compensator adapters ship un-fused in alis_adapters/ (rank 8). Download that folder and pass --adapter-path <local alis_adapters dir> to use them. All numbers in this card are measured without the adapters (same convention as the control arm), so a fused deployment may do better.
Chat, reasoning & sampling. The chat template (chat_template.jinja) ships in every branch; mlx_lm applies it automatically in chat/server mode (raw mlx_lm.generate with a plain string prompt does not โ apply the template yourself). Client-side settings follow the base model card: temperature 1.0 / top_p 1.0 for both reasoning modes โ avoid ~0.7-class temperatures, which measurably raise loop/runaway-thinking risk on quantized builds. reasoning_effort defaults to "high" (thinking ON); use "none" for a direct answer (--chat-template-args '{"reasoning_effort":"none"}' on mlx_lm.server, or the apply_chat_template kwarg). Reasoning markers are dedicated tokens <|think:start|> / <|think:end|> (ids 130/131) โ there is no </think> or "Final Answer" convention. Split reasoning from the final answer on <|think:end|>, and do not add it as a stop token: the answer comes after it, and termination is <|im:end|> (id 129, already in generation_config.json). Budget max_tokens for thinking + answer together โ thinking typically runs 0.5โ2K tokens on chat prompts, so โฅ4K total is recommended; an empty answer or a reply cut off mid-thought means the budget ran out inside the reasoning block. Server + LoRA adapters (F v2): fork revisions โค 9d67a0c silently dropped --adapter-path in mlx_lm.server โ the adapter lookup ran after the model-name remap, always missed, and the server then served the bare 2-bit build. Fixed in 7937fda (branch solar-open2); update the fork, or pass "adapters": "<local alis_adapters dir>" in the request body as a workaround.
Hardware compatibility (memory planning)
Weights are loaded fully into GPU-wired unified memory; context costs extra on top. KV cache โ 48 KiB/token fp16 โ only 12 of 48 layers (the softmax GQA layers, every 4th) hold a KV cache; the 36 KDA linear-attention layers carry a fixed-size recurrent state (~150 MiB total) that does not grow with context. โ ~4.8 GiB per 100K tokens ยท 256K โ 12 GiB ยท 1M โ 48 GiB.
Default GPU wired limit on macOS โ 80% of RAM; raise it with sudo sysctl iogpu.wired_limit_mb=<MB> (revert by lowering again; needs admin). Build footprints below use exact GiB: F v2 68.3 GiB (+3.9 GiB adapters) ยท Q v3 145.7 GiB ยท T 247.7 GiB.
| machine RAM | default GPU limit (~80%) | what fits stock | with wired-limit bump |
|---|---|---|---|
| 96 GiB | ~77 GiB | F v2 โ ํ๊ณ ์ง์ (์ฌ์ ~5 GiB โ 100K tok ctx), bump ๊ถ์ฅ | |
| 128 GiB | ~103 GiB | F v2 + |
โ |
| 192 GiB | ~154 GiB | Q v3 โ ํ๊ณ ์ง์ (์ฌ์ ~8 GiB โ 170K tok ctx), bump ๊ถ์ฅ | |
| 256 GiB | ~206 GiB | Q v3 + ~60 GiB ctx (full 1M tok) | โ |
| 384 GiB | ~309 GiB | T (Q8) + ~61 GiB ctx (full 1M tok) | โ |
| 512 GiB | ~412 GiB | T (Q8) + full 1M ctx | โ |
Rule of thumb: pick the largest build whose weights fit your machine with the context you need โ quality scales with bits (see the measured table below). DWQ training peak observed: 275 GB wired+compressed during Q v3 distillation (training-only; inference needs are far lower).
Note on the HF sidebar panel: the native Hardware compatibility widget indexes only the repo's root model (
main= Q v3, shown as "4-bit ยท MLX ยท 156 GB") โ unlike GGUF repos, where every quant file gets its own row, an MLX repo gets a single row and branch builds (t96,t384) are not indexed. Use the table above for F/T hardware sizing.
Measured quality (vs Q8 reference, ยง2-10 eval, ref = T)
| Metric | T (ref) | Q v3 (main) |
F v2 (t96) |
|---|---|---|---|
| KLD โ overall | 0 (ref) | 0.06369 (flip 0.0928) | 0.41976 (flip 0.2122) |
| KLD โ English | 0 (ref) | 0.04858 (flip 0.0713, clean) | 0.30345 (flip 0.1514, clean) |
| KLD โ code | 0 (ref) | 0.02811 (flip 0.0439) | 0.20571 (flip 0.1426, clean) |
| KLD โ Korean | 0 (ref) | 0.11438 (flip 0.1631) | 0.75013 (flip 0.3535, clean) |
| loop probe (256) | โ | clean (all slices, distinct4 EN 0.399) โ | clean (all slices, 0.356) โ |
| kv probe (4) | โ | selfKL=0.02305, flip=0.0557 | selfKL=0.02195, flip=0.0495 |
| DWQ valid loss (init โ final) | โ | 0.01597 โ 0.01381 (โ13.5%, 3/3 accepted) | 0.12102 โ 0.07242 (โ40.2%) |
| EN sentinel gate (eps=0, cap 1%) | โ | mean โ8.8% (paired 3/16) | ACCEPT (โ15.9%, no breach) โ |
Q tier journey (disclosed): the 4-bit/g32 base shipped a greedy EN loop (len23). Attempt 1 (Q-dwq, legacy scalar gate) made it worse (len27) โ the scalar gate's blind spot that motivated the per-slice gate. Attempt 2 (Q v2, gate + Q8 teacher, lr 1e-6 ร 6 rounds) cleared the loop but collapsed EN diversity (distinct4 0.221) and regressed KLD +6.7% โ over-distillation. Attempt 3 (Q v3, same gate/teacher but lr 5e-7 ร 3 rounds) keeps the loop fix, restores EN diversity to 0.399 (97% of the Q8 teacher's 0.411), brings KLD EN back to base level (+0.7%), and improves code KLD โ5.1% below its own base. The residual trade-off is Korean KLD +8.4% vs base (0.10553 โ 0.11438) โ see Known Limitations.
F tier: the legacy path (uniform 2-bit/g128 base + clip + DWQ) failed the usability gate (KLD 1.23, all slices LOOPED) โ a uniform 2-bit predicate destroys the KDA chokepoints and embed/head beyond DWQ recovery. F v2 re-quantizes with
quant_predicate_builder(chokepoint 8b / embed+head 6b / attn+shared 4b / experts 2b, 578/578 per-layer entries verified) and distills from the Q8 teacher under a per-slice acceptance gate with rank-8 LoRA compensators. Result: KLD 1.73 (legacy base) โ 1.23 (legacy DWQ) โ 0.42 (F v2 + DWQ), all loop probes clean, EN sentinel ACCEPT. The LoRA adapters ship separately (alis_adapters/); the numbers above are measured without them (same convention as the legacy arm), so a fused deployment may do better.
DWQ training detail: 200 calibration samples (180 train / 20 valid), expert-routing coverage 99.8%; teacher = Q8 reference for all shipped builds; deepest-first layer order, 3 rounds (Q v3) / 6 rounds (F v2); per-slice acceptance gate (EN ceiling ฮต=1%) + EN sentinel gate (paired, cap 1%).
Quantization quality (pre-distillation measurements)
Effective bit utilization (code-entropy audit, ยง2-2):
| Metric | M (4-bit/g64) | F (2-bit/g128 clipped) |
|---|---|---|
| Effective bits/weight | 3.72 (93% of nominal) | 1.65 (82% of nominal) |
| Low-entropy groups | 0.0% | 0.5% (concentrated in L0, worst 31.7%) |
| Weak (<50% util) experts | 0 / 144 stacks | 0 / 144 stacks |
F clip requantization gate (T-dequant clipping before 2-bit requant):
| Held-out PPL | Before clip | After clip | ฮ |
|---|---|---|---|
| wikitext | 110.56 | 49.51 | โ55.2% |
| code | 21.81 | 11.63 | โ46.7% |
| Korean | 23.94 | 18.93 | โ20.9% |
Base model (bf16, official card numbers)
Solar Open 2 is Upstage's 250B-A15B open-weight model: Hybrid-Attention MoE โ [Softmax, Linearร3] ร 12 (only 12 of 48 layers hold a KV cache), NoPE on linear-attention layers, 321 experts (320 routed top-8 + 1 shared), vocab 196,608, 1M-token context, ~12T pre-training tokens, English/Korean/Japanese.
| Benchmark | Solar Open 2 (bf16) |
|---|---|
| MMLU-Pro | 86.2 |
| GPQA-Diamond | 86.3 |
| SWE-Bench Verified | 70.4 |
| AIME 2026 | 95.7 |
| KMMLU-Pro | 78.4 |
(Official numbers from the base model card; quantized-build deltas are tracked in the KLD table above.)
Known Limitations
- Q v3 carries a disclosed Korean KLD regression (+8.4% vs its 4-bit/g32 base: 0.10553 โ 0.11438) โ the price of clearing the base's EN greedy loop while restoring EN diversity. English and code KLD are at or below base level; if your workload is Korean-critical and you can tolerate a known EN loop, prefer the T (Q8) tier or the base model.
- F v2 is a 2.34 bpw build: Korean KLD (0.75) is materially higher than the Q tier โ expected at this bit budget; prefer Q v3 for Korean-heavy workloads. The DWQ LoRA adapters ship un-fused (
alis_adapters/); all reported numbers are without them. - Reasoning-block runaway (root-caused, disclosed): an external integrator reported F v2 thinking that never closed (22.6K chars, no
<|think:end|>). This was traced to a serving bug, not the weights: forkmlx_lm.serverโค9d67a0csilently dropped--adapter-path, so that serve ran the bare 2-bit build without its DWQ LoRA compensators (fixed in7937fda, see Install & run). With adapters actually applied, thinking terminated in 5/5 direct-path runs across both teams โ at the official temp 1.0 / top_p 1.0 and at 0.7/0.95. Guidance stands regardless: official sampling, generousmax_tokens,reasoning_effort="none"for short direct answers โ and treat the adapters as part of the deployed F v2 configuration (a bare serve can ramble). - Architecture support:
solar_open2is not yet in PyPI mlx-lm (0.31.3) โ use the fork (default branchsolar-open2; the adapter-drop fix is7937fda) or the bundledsolar_open2.py. - Calibration coverage saturates at 200 samples (99.8% expert routing), but median tokens/expert (4,314) is ~19% of the 753B reference run; very rare experts may be under-distilled.
- Quantized builds inherit the base model's knowledge cutoff (2026-02) and language focus (EN/KO/JA).
License
Distributed under the Upstage Solar License (same as the base model). Per the license's derivative-model terms:
- Model name carries the "Solar" prefix โ
- "Built with Solar" attribution displayed above โ
- A copy of the Upstage Solar License is included in this repo โ
Credits
- Base model: Upstage โ Solar-Open2-250B (Upstage Solar License).
- MLX & mlx-lm: Apple
ml-exploreโsolar_open2architecture port (KDA hybrid-attention MoE, NoPE, sigmoid-gated GQA): Alis (avlp12) (fork). - Mixed-precision DWQ quantization, per-slice acceptance gate + EN sentinel gate, F-tier
quant_predicate_builderrecipe (chokepoint 8b / embed+head 6b / attn+shared 4b / experts 2b): Alis (avlp12) (alis-dwq).
Citation
@misc{solar-open-2-2026,
title={Solar Open 2 Technical Report},
author={Upstage AI},
year={2026},
url={https://huggingface.co/upstage/Solar-Open2-250B}
}
- Downloads last month
- 49
4-bit
Model tree for avlp12/Solar-Open2-250B-Alis-MLX-Dynamic
Base model
upstage/Solar-Open2-250B![Solar Open 2 architecture & quantization map โ hybrid attention stack [Softmax, Linearร3]ร12, MoE 320 routed + 1 shared experts, routers pinned to 8-bit](/avlp12/Solar-Open2-250B-Alis-MLX-Dynamic/resolve/main/images/architecture_map.png)



