Text Generation
MLX
Safetensors
English
Chinese
qwen3_5
mixed-precision
coding
tool-use
experimental
conversational
4-bit precision
Instructions to use Shiftedx/Qwopus3.8-27B-Coder-Flash-MLX-Balanced with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- MLX
How to use Shiftedx/Qwopus3.8-27B-Coder-Flash-MLX-Balanced with MLX:
# Make sure mlx-lm is installed # pip install --upgrade mlx-lm # Generate text with mlx-lm from mlx_lm import load, generate model, tokenizer = load("Shiftedx/Qwopus3.8-27B-Coder-Flash-MLX-Balanced") prompt = "Write a story about Einstein" messages = [{"role": "user", "content": prompt}] prompt = tokenizer.apply_chat_template( messages, add_generation_prompt=True ) text = generate(model, tokenizer, prompt=prompt, verbose=True) - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- LM Studio
- Pi
How to use Shiftedx/Qwopus3.8-27B-Coder-Flash-MLX-Balanced with Pi:
Start the MLX server
# Install MLX LM: uv tool install mlx-lm # Start a local OpenAI-compatible server: mlx_lm.server --model "Shiftedx/Qwopus3.8-27B-Coder-Flash-MLX-Balanced"
Configure the model in Pi
# Install Pi: npm install -g @earendil-works/pi-coding-agent # Add to ~/.pi/agent/models.json: { "providers": { "mlx-lm": { "baseUrl": "http://localhost:8080/v1", "api": "openai-completions", "apiKey": "none", "models": [ { "id": "Shiftedx/Qwopus3.8-27B-Coder-Flash-MLX-Balanced" } ] } } }Run Pi
# Start Pi in your project directory: pi
- MLX LM
How to use Shiftedx/Qwopus3.8-27B-Coder-Flash-MLX-Balanced with MLX LM:
Generate or start a chat session
# Install MLX LM uv tool install mlx-lm # Interactive chat REPL mlx_lm.chat --model "Shiftedx/Qwopus3.8-27B-Coder-Flash-MLX-Balanced"
Run an OpenAI-compatible server
# Install MLX LM uv tool install mlx-lm # Start the server mlx_lm.server --model "Shiftedx/Qwopus3.8-27B-Coder-Flash-MLX-Balanced" # Calling the OpenAI-compatible server with curl curl -X POST "http://localhost:8080/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "Shiftedx/Qwopus3.8-27B-Coder-Flash-MLX-Balanced", "messages": [ {"role": "user", "content": "Hello"} ] }' - Hermes Agent
How to use Shiftedx/Qwopus3.8-27B-Coder-Flash-MLX-Balanced with Hermes Agent:
Start the MLX server
# Install MLX LM: uv tool install mlx-lm # Start a local OpenAI-compatible server: mlx_lm.server --model "Shiftedx/Qwopus3.8-27B-Coder-Flash-MLX-Balanced"
Configure Hermes
# Install Hermes: curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash hermes setup # Point Hermes at the local server: hermes config set model.provider custom hermes config set model.base_url http://127.0.0.1:8080/v1 hermes config set model.default Shiftedx/Qwopus3.8-27B-Coder-Flash-MLX-Balanced
Run Hermes
hermes
- Atomic Chat
- OpenClaw
How to use Shiftedx/Qwopus3.8-27B-Coder-Flash-MLX-Balanced with OpenClaw:
Start the MLX server
# Install MLX LM: uv tool install mlx-lm # Start a local OpenAI-compatible server: mlx_lm.server --model "Shiftedx/Qwopus3.8-27B-Coder-Flash-MLX-Balanced"
Configure OpenClaw
# Install OpenClaw: npm install -g openclaw@latest # Register the local server and set it as the default model: openclaw onboard --non-interactive --mode local \ --auth-choice custom-api-key \ --custom-base-url http://127.0.0.1:8080/v1 \ --custom-model-id "Shiftedx/Qwopus3.8-27B-Coder-Flash-MLX-Balanced" \ --custom-provider-id mlx-lm \ --custom-compatibility openai \ --custom-text-input \ --accept-risk \ --skip-health
Run OpenClaw
openclaw agent --local --agent main --message "Hello from Hugging Face"
Document tested strict JSON profiles, vision results and repeated speed comparison
Browse files- README.md +7 -2
- structured_output_profiles.json +108 -0
- structured_output_validation.json +63 -0
README.md
CHANGED
|
@@ -40,9 +40,14 @@ kapitsa serve --private --local --model ./qwopus-balanced \
|
|
| 40 |
|
| 41 |
Use --context 65536 for 64K. Send temperature=0, top_p=1, top_k=0 and chat_template_kwargs={"enable_thinking":false} to /v1/chat/completions. Prompt and output share the limit. DFlash2 is separate.
|
| 42 |
|
| 43 |
-
|
| 44 |
|
| 45 |
-
Kapitsa 0.2.31
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 46 |
|
| 47 |
Experimental refusal-direction edit, strength 0.5 on attention outputs in layers 10–17. Refusal reduction and BF16 fidelity unverified. Text, still-image and short MTP checks only; long-context MTP untested.
|
| 48 |
|
|
|
|
| 40 |
|
| 41 |
Use --context 65536 for 64K. Send temperature=0, top_p=1, top_k=0 and chat_template_kwargs={"enable_thinking":false} to /v1/chat/completions. Prompt and output share the limit. DFlash2 is separate.
|
| 42 |
|
| 43 |
+
**Structured JSON / vision.** With the request schemas, both models passed **8/8 original cases twice, 12/12 held-out cases and 8/8 with native MTP3**, graded as exact bare JSON. Apply the matching response_format from [structured_output_profiles.json](structured_output_profiles.json) for OCR, chart, spatial-array or two-image replies. The schemas constrain structure without supplying answers. This request profile fixes the previously observed fences and nested arrays. [Validation](structured_output_validation.json).
|
| 44 |
|
| 45 |
+
| Kapitsa 0.2.31 + DFlash2 | Balanced | Fast |
|
| 46 |
+
|---|---:|---:|
|
| 47 |
+
| Structured JSON decode | 128.1 tok/s | 165.6 tok/s |
|
| 48 |
+
| Code/prose decode | 90.2 tok/s | 108.0 tok/s |
|
| 49 |
+
|
| 50 |
+
Median of three repetitions on M4 Max / 64 GiB. Native MTP depths 1–3 and DFlash2 each matched plain greedy text on 4/4 short probes. Use --dflash none --spec 3 for native MTP or --spec 0 for plain decoding. [Earlier capability checks](capability_validation.json).
|
| 51 |
|
| 52 |
Experimental refusal-direction edit, strength 0.5 on attention outputs in layers 10–17. Refusal reduction and BF16 fidelity unverified. Text, still-image and short MTP checks only; long-context MTP untested.
|
| 53 |
|
structured_output_profiles.json
ADDED
|
@@ -0,0 +1,108 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"ocr": {
|
| 3 |
+
"type": "json_schema",
|
| 4 |
+
"json_schema": {
|
| 5 |
+
"name": "ocr",
|
| 6 |
+
"schema": {
|
| 7 |
+
"type": "object",
|
| 8 |
+
"properties": {
|
| 9 |
+
"identifier": {
|
| 10 |
+
"type": "string"
|
| 11 |
+
}
|
| 12 |
+
},
|
| 13 |
+
"required": [
|
| 14 |
+
"identifier"
|
| 15 |
+
],
|
| 16 |
+
"additionalProperties": false
|
| 17 |
+
},
|
| 18 |
+
"strict": true
|
| 19 |
+
}
|
| 20 |
+
},
|
| 21 |
+
"chart": {
|
| 22 |
+
"type": "json_schema",
|
| 23 |
+
"json_schema": {
|
| 24 |
+
"name": "chart",
|
| 25 |
+
"schema": {
|
| 26 |
+
"type": "object",
|
| 27 |
+
"properties": {
|
| 28 |
+
"tallest": {
|
| 29 |
+
"type": "string"
|
| 30 |
+
},
|
| 31 |
+
"sum": {
|
| 32 |
+
"type": "integer"
|
| 33 |
+
}
|
| 34 |
+
},
|
| 35 |
+
"required": [
|
| 36 |
+
"tallest",
|
| 37 |
+
"sum"
|
| 38 |
+
],
|
| 39 |
+
"additionalProperties": false
|
| 40 |
+
},
|
| 41 |
+
"strict": true
|
| 42 |
+
}
|
| 43 |
+
},
|
| 44 |
+
"spatial": {
|
| 45 |
+
"type": "json_schema",
|
| 46 |
+
"json_schema": {
|
| 47 |
+
"name": "spatial",
|
| 48 |
+
"schema": {
|
| 49 |
+
"type": "array",
|
| 50 |
+
"items": {
|
| 51 |
+
"type": "string"
|
| 52 |
+
},
|
| 53 |
+
"minItems": 3,
|
| 54 |
+
"maxItems": 3
|
| 55 |
+
},
|
| 56 |
+
"strict": true
|
| 57 |
+
}
|
| 58 |
+
},
|
| 59 |
+
"multi_image": {
|
| 60 |
+
"type": "json_schema",
|
| 61 |
+
"json_schema": {
|
| 62 |
+
"name": "multi_image",
|
| 63 |
+
"schema": {
|
| 64 |
+
"type": "object",
|
| 65 |
+
"properties": {
|
| 66 |
+
"job": {
|
| 67 |
+
"type": "integer"
|
| 68 |
+
},
|
| 69 |
+
"from": {
|
| 70 |
+
"type": "string"
|
| 71 |
+
},
|
| 72 |
+
"to": {
|
| 73 |
+
"type": "string"
|
| 74 |
+
}
|
| 75 |
+
},
|
| 76 |
+
"required": [
|
| 77 |
+
"job",
|
| 78 |
+
"from",
|
| 79 |
+
"to"
|
| 80 |
+
],
|
| 81 |
+
"additionalProperties": false
|
| 82 |
+
},
|
| 83 |
+
"strict": true
|
| 84 |
+
}
|
| 85 |
+
},
|
| 86 |
+
"sequence": {
|
| 87 |
+
"type": "json_schema",
|
| 88 |
+
"json_schema": {
|
| 89 |
+
"name": "sequence",
|
| 90 |
+
"schema": {
|
| 91 |
+
"type": "object",
|
| 92 |
+
"properties": {
|
| 93 |
+
"items": {
|
| 94 |
+
"type": "array",
|
| 95 |
+
"items": {
|
| 96 |
+
"type": "integer"
|
| 97 |
+
}
|
| 98 |
+
}
|
| 99 |
+
},
|
| 100 |
+
"required": [
|
| 101 |
+
"items"
|
| 102 |
+
],
|
| 103 |
+
"additionalProperties": false
|
| 104 |
+
},
|
| 105 |
+
"strict": true
|
| 106 |
+
}
|
| 107 |
+
}
|
| 108 |
+
}
|
structured_output_validation.json
ADDED
|
@@ -0,0 +1,63 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"runtime": "Kapitsa 0.2.31 (84249c0)",
|
| 3 |
+
"host": "Apple M4 Max, 40-core GPU, 64 GiB",
|
| 4 |
+
"context_limit": 131072,
|
| 5 |
+
"sampler": "greedy; thinking disabled",
|
| 6 |
+
"implementation": "request-level strict JSON Schema via response_format; no weight or tokenizer changes; no response postprocessing",
|
| 7 |
+
"schema_inputs": "Field types and structural constraints only; expected answers are excluded from requests.",
|
| 8 |
+
"test_source_sha256": "b0e5013aecfe29e2b966afed82120926b700ffd8860a8129ee08e5cdecb62088",
|
| 9 |
+
"schema_file": "structured_output_profiles.json",
|
| 10 |
+
"fixture_seeds": {
|
| 11 |
+
"original": [
|
| 12 |
+
7319,
|
| 13 |
+
173
|
| 14 |
+
],
|
| 15 |
+
"held_out": [
|
| 16 |
+
27183,
|
| 17 |
+
65537,
|
| 18 |
+
42017
|
| 19 |
+
]
|
| 20 |
+
},
|
| 21 |
+
"vision": [
|
| 22 |
+
{
|
| 23 |
+
"mode": "dflash2",
|
| 24 |
+
"lane": "baseline",
|
| 25 |
+
"passed": 4,
|
| 26 |
+
"total": 8
|
| 27 |
+
},
|
| 28 |
+
{
|
| 29 |
+
"mode": "dflash2",
|
| 30 |
+
"lane": "schema-r1",
|
| 31 |
+
"passed": 8,
|
| 32 |
+
"total": 8
|
| 33 |
+
},
|
| 34 |
+
{
|
| 35 |
+
"mode": "dflash2",
|
| 36 |
+
"lane": "schema-r2",
|
| 37 |
+
"passed": 8,
|
| 38 |
+
"total": 8
|
| 39 |
+
},
|
| 40 |
+
{
|
| 41 |
+
"mode": "dflash2",
|
| 42 |
+
"lane": "holdout",
|
| 43 |
+
"passed": 12,
|
| 44 |
+
"total": 12
|
| 45 |
+
},
|
| 46 |
+
{
|
| 47 |
+
"mode": "mtp3",
|
| 48 |
+
"lane": "schema-mtp",
|
| 49 |
+
"passed": 8,
|
| 50 |
+
"total": 8
|
| 51 |
+
}
|
| 52 |
+
],
|
| 53 |
+
"coding_passed": 6,
|
| 54 |
+
"coding_total": 8,
|
| 55 |
+
"coding_grader_note": "The same standard-library typing allowance is applied to both models.",
|
| 56 |
+
"speed": {
|
| 57 |
+
"text_median_decode_tps": 90.2,
|
| 58 |
+
"structured_json_median_decode_tps": 128.1,
|
| 59 |
+
"text_probes": 2,
|
| 60 |
+
"repeats": 3,
|
| 61 |
+
"measurement": "Median server-reported decode; code/prose capped at 256 tokens; JSON sequence independently checked for exact correctness. Six text and three JSON measurements per model."
|
| 62 |
+
}
|
| 63 |
+
}
|