Instructions to use autotrust/hy4-flash-concise with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use autotrust/hy4-flash-concise with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="autotrust/hy4-flash-concise") messages = [ {"role": "user", "content": "Who are you?"}, ] pipe(messages)# pip install -U transformers accelerate # Load model directly from transformers import AutoModelForSeq2SeqLM model = AutoModelForSeq2SeqLM.from_pretrained("autotrust/hy4-flash-concise", device_map="auto") - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use autotrust/hy4-flash-concise with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "autotrust/hy4-flash-concise" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "autotrust/hy4-flash-concise", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker
docker model run hf.co/autotrust/hy4-flash-concise
- SGLang
How to use autotrust/hy4-flash-concise with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "autotrust/hy4-flash-concise" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "autotrust/hy4-flash-concise", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "autotrust/hy4-flash-concise" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "autotrust/hy4-flash-concise", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }' - Docker Model Runner
How to use autotrust/hy4-flash-concise with Docker Model Runner:
docker model run hf.co/autotrust/hy4-flash-concise
Download README.md from autotrust/hy4-flash-concise: direct link, hf CLI and curl.
- Browser
- Download file 7.04 kB
-
https://huggingface.co/autotrust/hy4-flash-concise/resolve/main/README.md
- Command line
-
hf download hf://autotrust/hy4-flash-concise/README.md
-
curl -L -o README.md https://huggingface.co/autotrust/hy4-flash-concise/resolve/main/README.md
license: other
license_name: tencent-hy-license
license_link: https://huggingface.co/tencent/Hy4-flash/blob/main/LICENSE
base_model:
- tencent/Hy4-flash
- atakans/Hy4-flash-NVFP4
tags:
- hy4-flash
- weight-editing
- weight-space-programming
- reasoning
- nvfp4
- moe
library_name: transformers
pipeline_tag: text-generation
hy4-flash-concise
Hy4-flash that thinks less. Nine shared-expert down-projections of tencent/Hy4-flash are rewritten with a rank-one, gradient-free edit (weight-space programming). There are no gradient steps and no fine-tuning data. The edit comes from the model's own activations and is compiled into nine weight matrices.
Thinking tokens drop by 37–55% on every benchmark we ran. Accuracy also drops, by a little on GPQA and by a meaningful amount on code and instruction following (see below). Use this model when shorter, cheaper reasoning matters more than the last few points of accuracy.
Status: experimental. The root of this repo is the aggressive edit (α = 0.1). A milder edit (α = 0.05) that keeps reasoning and code accuracy within noise is in
alpha-0.05/.
What was changed
| Base weights | atakans/Hy4-flash-NVFP4 (NVFP4 MLPs, FP8 Engram tables, BF16 elsewhere), derived from tencent/Hy4-flash |
| Edited tensors | model.layers.{22,24,…,38}.mlp.shared_experts.down_proj.weight (9 matrices) |
| Edit | W ← W_bf16_original + α · n · d″ uᵀ / ‖u‖², α = 0.1, stored in BF16 (excluded from NVFP4) |
| Relative size | ‖ΔW‖/‖W‖ = 0.56–1.09% per layer |
| Everything else | byte-identical to atakans/Hy4-flash-NVFP4 |
dis the residual-stream direction that separates the last 48 thinking tokens before</think>from the rest of the thinking, measured on 288 MATH-train (level 5) traces.d″correctsdfor the FFN post-norm (combine_output_norm), so after that RMSNorm the written vector points alongd.uis the mean input to the shared-expert down-projection on thinking tokens, andnis the mean norm of the FFN output before the post-norm.- The shared expert runs on every token, so a rank-one term in its down-projection acts like a constant steering vector, compiled into the weights.
- Directions, layers and α were chosen on MATH train only. No evaluation benchmark was used for selection.
The direction tensors and scripts are in edit/.
Results
Setup: vLLM, this checkpoint against the matched base (the same nine matrices in BF16, unedited), thinking on
(reasoning_effort: high), temperature 0.9, max 32k tokens, one H200. Δ is paired by problem, with a 95%
bootstrap CI.
| Benchmark | Base | concise (α=0.1) | Δ accuracy [95% CI] | Δ thinking tokens [95% CI] |
|---|---|---|---|---|
| GPQA Diamond (198 × 2) | 91.2 | 88.6 | −2.5 [−5.3, +0.3] | −39.6% [−43.3, −35.6] |
| LiveCodeBench v6 (175, 2025-01…04) | 84.6 | 78.3 | −6.3 [−12.0, −0.6] | −36.8% [−43.3, −30.3] |
| IFBench (300, loose) | 76.7 | 66.0 | −10.7 [−16.0, −5.3] | −54.8% [−58.3, −51.0] |
| 13 coding/math/science tasks × 3 (held-out) | 39/39 | 38/39 | −2.6 | −54.2% |
| MATH train screen (64 × 2) | 92.2 | 89.8 | −2.3 [−6.2, +0.8] | −40.2% [−47.8, −29.9] |
Mean thinking tokens, base → concise: GPQA 5448 → 3291, LCB 8564 → 5409, IFBench 5275 → 2382, held-out tasks 5415 → 2482.
Milder version: alpha-0.05/
| Benchmark | Base | α=0.05 | Δ accuracy [95% CI] | Δ thinking tokens [95% CI] |
|---|---|---|---|---|
| GPQA Diamond (198 × 2) | 91.2 | 89.6 | −1.5 [−3.8, +0.5] | −20.6% [−25.6, −15.7] |
| LiveCodeBench v6 (175) | 84.6 | 82.3 | −2.3 [−8.0, +3.4] | −25.0% [−31.2, −18.8] |
| IFBench (300, loose) | 76.7 | 70.3 | −6.3 [−11.0, −2.0] | −36.6% [−40.8, −31.7] |
| 13 held-out tasks × 3 | 39/39 | 37/39 | −5.1 | −29.1% |
Which one to pick: choose alpha-0.05 if accuracy on reasoning and code must hold. Choose the root (α = 0.1) for
the largest saving. Both lose points on strict instruction following (IFBench).
Control. A norm-matched random edit (same layers, same ‖ΔW‖, random direction orthogonal to d) on the
MATH screen gives +0.0 points of accuracy and +8% tokens. The token saving comes from the direction, not from the
perturbation.
Dose response (MATH screen): α = 0.05 → −23% tokens; α = 0.075 → −32%; α = 0.1 → −40%; α = 0.3 → answers
become wrong; α = 1.0 → the model loops on </think>. Do not raise α.
Usage
vLLM (Hopper / Blackwell)
Hy4-flash needs the vllm/vllm-openai:hy4-flash image. NVFP4 expert loading needs the
vllm/models/hy_v4_flash/nvidia/model.py from
atakann/dgx-spark-recipes (copy included as
edit/vllm_hy_v4_flash_model.py). On DGX Spark (SM120), use that recipe's full image.
vllm serve autotrust/hy4-flash-concise --served-model-name hy4-flash \
--dtype bfloat16 --max-model-len 40960 --max-num-seqs 64 \
--reasoning-parser hy_v4 --limit-mm-per-prompt '{"image":0,"video":0}'
# milder version (local path):
hf download autotrust/hy4-flash-concise --include "alpha-0.05/*" --local-dir hy4-flash-concise
vllm serve hy4-flash-concise/alpha-0.05 --served-model-name hy4-flash --reasoning-parser hy_v4 ...
On one H200: 97 GiB of weights on the GPU, Engram tables in host RAM, about 1.5k tok/s aggregate decode at 100 concurrent requests.
llama.cpp (GGUF, e.g. DGX Spark)
gguf-lora/hy4-flash-concise-lora.gguf (167 KB) applies the same rank-one edit to the stock
AngelSlim/Hy4-flash-GGUF files as a LoRA adapter on
blk.{22..38 even}.ffn_down_shexp. The scale is α (0.1 = root version, 0.05 = milder version):
llama-server -m Hy4-flash-SPARK-4Bit-Fast.gguf ... --lora-scaled hy4-flash-concise-lora.gguf:0.1
The adapter was validated on Q4_K_M (MATH screen: −38% tokens, −2.3 points). It is not separately validated on the SPARK-4Bit builds. Note that llama-server will not batch requests whose per-request LoRA settings differ, so set the scale globally.
Limitations
- Accuracy loss on code (−6 points) and instruction following (−11 points) at α = 0.1 is statistically significant.
- The direction was read from a Q4_K_M GGUF of the same model. Q8_0 shared experts differ from BF16 by 0.6%.
- The edit was tested only with thinking on. With
no_thinkit is untested. - One edit form, one layer set, single runs per benchmark.
License and attribution
Tencent Hy License 1.0 (see LICENSE), inherited from tencent/Hy4-flash. Modified from
atakans/Hy4-flash-NVFP4: nine shared_experts.down_proj tensors replaced as described above, and the
hf_quant_config.json exclude list extended to cover them.
Method after weight-space programming (Takehi et al., 2026) and the norm-matched replication by Concept Country (2026).