Instructions to use tiny-random/raon-1-mini with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use tiny-random/raon-1-mini with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="tiny-random/raon-1-mini", trust_remote_code=True)
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoModelForCausalLM
model = AutoModelForCausalLM.from_pretrained("tiny-random/raon-1-mini", trust_remote_code=True, device_map="auto")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use tiny-random/raon-1-mini with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "tiny-random/raon-1-mini"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "tiny-random/raon-1-mini",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/tiny-random/raon-1-mini

SGLang

How to use tiny-random/raon-1-mini with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "tiny-random/raon-1-mini" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "tiny-random/raon-1-mini",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "tiny-random/raon-1-mini" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "tiny-random/raon-1-mini",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use tiny-random/raon-1-mini with Docker Model Runner:
```
docker model run hf.co/tiny-random/raon-1-mini
```

raon-1-mini / README.md

yujiepan

Upload folder using huggingface_hub

6466a4d verified 3 days ago

preview code

Raw History Blame Contribute Delete

15.4 kB

---
library_name: transformers
base_model:
- KRAFTON/Raon-1-mini-preview
---
This tiny model is intended for debugging. It is randomly initialized using the configuration adapted from [KRAFTON/Raon-1-mini-preview](https://huggingface.co/KRAFTON/Raon-1-mini-preview).

Structural comparison with `KRAFTON/Raon-1-mini-preview`:

| Feature | Source | Tiny |
|---|---:|---:|
| Vocabulary / tied embeddings | 259,840 / untied | same |
| Layers | 25 | 8; two 3-RDA + 1-MLA cycles |
| RDA / NoPE MLA layer cadence | 18 / 7 | 6 / 2 |
| Raon mHC streams | 4 | same |
| Dense prefix / MoE layers | layer 0 / 24 layers | layer 0 / 7 layers |
| Routed experts / active experts | 128 / 6 | 64 / 6 |
| Hidden width / attention head dimension | 2,560 / 128 | 32 / 32 |
| RDA, MLA and LoRA dimensions | source widths | 32-wide compatible projections |
| Routed-expert storage | MXFP8 E4M3 + E8M0 1x32 scales | same |
| Router / shared expert / output gate | sigmoid top-k / present / present | same |

Transformers constructs Raon's architecture, then the definition rewrites only
routed-expert tensors to the source MXFP8 storage representation.  This is
necessary because random CPU initialization of `float8_e4m3fn` parameters is
not implemented by PyTorch; it does not change the serialized precision.

| File path | Size |
|------|------|
| model.safetensors | 35.3MB |


### Example usage:

- Transformers (✅ verified)

```python
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer
model_id = "tiny-random/raon-1-mini"
device = "cuda" if torch.cuda.is_available() else "cpu"
model = AutoModelForCausalLM.from_pretrained(model_id, trust_remote_code=True).to(device).eval()
tokenizer = AutoTokenizer.from_pretrained(model_id, trust_remote_code=True)
inputs = tokenizer("Hello", return_tensors="pt").to(device)
output = model.generate(**inputs, do_sample=False, min_new_tokens=10, max_new_tokens=10)
print(tokenizer.decode(output[0, inputs["input_ids"].shape[1]:], skip_special_tokens=True))
```

### Codes to create this repo (✅ verified):

<details><summary>Click to expand</summary>

```python
import json
from pathlib import Path
from shutil import copyfile
import torch
from huggingface_hub import hf_hub_download, list_repo_files
from safetensors.torch import save_file
from transformers import AutoConfig, AutoModelForCausalLM, set_seed
source_model_id = "KRAFTON/Raon-1-mini-preview"
save_folder = "/tmp/tiny-random/raon-1-mini"
source_revision = "7bf87c45fc7b658d3c9532305f3aaed106bbac4f"
set_seed(42)
folder = Path(save_folder)
folder.mkdir(parents=True, exist_ok=True)
raw = json.loads(Path(hf_hub_download(source_model_id, "config.json", revision=source_revision)).read_text())
raw.update(
    hidden_size=32, head_dim=32, num_attention_heads=1, num_hidden_layers=8, intermediate_size=128,
    num_experts=64, num_experts_per_token=6, moe_intermediate_size=32,
    q_lora_rank=32, kv_lora_rank=32, qk_nope_head_dim=32,
    qk_rope_head_dim=32, v_head_dim=32, rda_decay_rank=32,
    max_position_embeddings=4096,
)
raw["rda_config"].update(
    num_heads=1, head_dim=32,
    rda_layers=[1, 2, 3, 5, 6, 7], mla_layers=[4, 8],
)
# Build the public remote-code graph in BF16 first.  The source's MXFP8
# modules cannot initialize random float8 parameters directly, then the
# routed experts are converted back to the source's E4M3 + E8M0 format.
construction_raw = json.loads(json.dumps(raw))
construction_raw["quantization_config"]["dequantize"] = True
(folder / "config.json").write_text(json.dumps(construction_raw, indent=2) + "\n", encoding="utf-8")
for filename in ("configuration_raon1_mini.py", "modeling_raon1_mini.py"):
    copyfile(hf_hub_download(source_model_id, filename, revision=source_revision), folder / filename)
config = AutoConfig.from_pretrained(folder, trust_remote_code=True)
model = AutoModelForCausalLM.from_config(config, trust_remote_code=True).eval()
with torch.no_grad():
    for name, parameter in sorted(model.named_parameters()):
        if name.endswith(("weight", "phi_pre", "phi_post", "phi_z")) and not name.endswith("norm.weight"):
            torch.nn.init.normal_(parameter, mean=0, std=0.02)
        elif name.endswith("norm.weight") or name.endswith("layernorm.weight"):
            parameter.fill_(1)
def quantize_mxfp8(weight: torch.Tensor) -> tuple[torch.Tensor, torch.Tensor]:
    rows, columns = weight.shape
    assert columns % 32 == 0, f"MXFP8 columns must divide into 32-value blocks, got {columns}"
    blocks = weight.float().reshape(rows, columns // 32, 32)
    bits = (blocks.abs().amax(-1) / 448.0).view(torch.int32)
    exponent, mantissa = (bits >> 23) & 0xFF, bits & 0x7FFFFF
    round_up = (mantissa > 0) & (exponent != 0xFE) & ~((exponent == 0) & (mantissa <= 0x400000))
    scale = (exponent + round_up.int()).clamp(max=254)
    reciprocal = ((254 - scale) << 23).view(torch.float32)
    reciprocal = torch.where(scale == 254, torch.full_like(reciprocal, 2.0**-127), reciprocal)
    values = (blocks * reciprocal.unsqueeze(-1)).clamp(-448.0, 448.0)
    return values.to(torch.float8_e4m3fn).reshape(rows, columns), scale.to(torch.uint8)
state = model.state_dict()
for name in sorted(tuple(state)):
    if ".moe.experts." in name and name.endswith(".weight"):
        values, scales = quantize_mxfp8(state[name])
        state[name] = values
        state[name.removesuffix(".weight") + ".weight_scale_inv"] = scales
raw["quantization_config"]["dequantize"] = False
(folder / "config.json").write_text(json.dumps(raw, indent=2) + "\n", encoding="utf-8")
save_file(state, folder / "model.safetensors", metadata={"format": "pt"})
assets = {"LICENSE", "chat_template.jinja", "generation_config.json", "tokenizer.json", "tokenizer_config.json"}
for filename in sorted(set(list_repo_files(source_model_id, revision=source_revision)) & assets):
    copyfile(hf_hub_download(source_model_id, filename, revision=source_revision), folder / filename)
```

</details>

### Printing the model:

<details><summary>Click to expand</summary>

```text
Raon1MiniForCausalLM(
  (model): Raon1MiniModel(
    (embed_tokens): Embedding(259840, 32, padding_idx=259200)
    (layers): ModuleList(
      (0): Raon1MiniDecoderLayer(
        (input_layernorm): Raon1MiniRMSNorm((32,), eps=1e-05)
        (post_attention_layernorm): Raon1MiniRMSNorm((32,), eps=1e-05)
        (self_attn): Raon1MiniRDA(
          (q_proj): Linear(in_features=32, out_features=32, bias=False)
          (k_proj): Linear(in_features=32, out_features=32, bias=False)
          (v_proj): Linear(in_features=32, out_features=32, bias=False)
          (q_conv1d): Raon1MiniShortConv()
          (k_conv1d): Raon1MiniShortConv()
          (v_conv1d): Raon1MiniShortConv()
          (f_a_proj): Linear(in_features=32, out_features=32, bias=False)
          (f_b_proj): Linear(in_features=32, out_features=32, bias=False)
          (b_proj): Linear(in_features=32, out_features=1, bias=False)
          (rda_w_proj): Linear(in_features=32, out_features=1, bias=False)
          (g_proj): Linear(in_features=32, out_features=32, bias=False)
          (o_norm): Raon1MiniRMSNorm((32,), eps=1e-05)
          (o_proj): Linear(in_features=32, out_features=32, bias=False)
        )
        (self_attn_raon_mhc): RaonMHC(
          (res): RaonMHCStreamMix()
        )
        (mlp_raon_mhc): RaonMHC(
          (res): RaonMHCStreamMix()
        )
        (mlp): Raon1MiniMLP(
          (gate_proj): Linear(in_features=32, out_features=128, bias=False)
          (up_proj): Linear(in_features=32, out_features=128, bias=False)
          (down_proj): Linear(in_features=128, out_features=32, bias=False)
        )
      )
      (1-2): 2 x Raon1MiniDecoderLayer(
        (input_layernorm): Raon1MiniRMSNorm((32,), eps=1e-05)
        (post_attention_layernorm): Raon1MiniRMSNorm((32,), eps=1e-05)
        (self_attn): Raon1MiniRDA(
          (q_proj): Linear(in_features=32, out_features=32, bias=False)
          (k_proj): Linear(in_features=32, out_features=32, bias=False)
          (v_proj): Linear(in_features=32, out_features=32, bias=False)
          (q_conv1d): Raon1MiniShortConv()
          (k_conv1d): Raon1MiniShortConv()
          (v_conv1d): Raon1MiniShortConv()
          (f_a_proj): Linear(in_features=32, out_features=32, bias=False)
          (f_b_proj): Linear(in_features=32, out_features=32, bias=False)
          (b_proj): Linear(in_features=32, out_features=1, bias=False)
          (rda_w_proj): Linear(in_features=32, out_features=1, bias=False)
          (g_proj): Linear(in_features=32, out_features=32, bias=False)
          (o_norm): Raon1MiniRMSNorm((32,), eps=1e-05)
          (o_proj): Linear(in_features=32, out_features=32, bias=False)
        )
        (self_attn_raon_mhc): RaonMHC(
          (res): RaonMHCStreamMix()
        )
        (mlp_raon_mhc): RaonMHC(
          (res): RaonMHCStreamMix()
        )
        (moe): Raon1MiniMoE(
          (gate): Raon1MiniRouter()
          (experts): ModuleList(
            (0-63): 64 x Raon1MiniExpert(
              (gate_proj): Raon1MiniRoutedLinear(in_features=32, out_features=32, bias=False)
              (up_proj): Raon1MiniRoutedLinear(in_features=32, out_features=32, bias=False)
              (down_proj): Raon1MiniRoutedLinear(in_features=32, out_features=32, bias=False)
            )
          )
          (shared_experts): Raon1MiniMLP(
            (gate_proj): Linear(in_features=32, out_features=32, bias=False)
            (up_proj): Linear(in_features=32, out_features=32, bias=False)
            (down_proj): Linear(in_features=32, out_features=32, bias=False)
          )
        )
      )
      (3): Raon1MiniDecoderLayer(
        (input_layernorm): Raon1MiniRMSNorm((32,), eps=1e-05)
        (post_attention_layernorm): Raon1MiniRMSNorm((32,), eps=1e-05)
        (self_attn): Raon1MiniMLA(
          (q_a_proj): Linear(in_features=32, out_features=32, bias=False)
          (q_a_layernorm): Raon1MiniRMSNorm((32,), eps=1e-06)
          (q_b_proj): Linear(in_features=32, out_features=64, bias=False)
          (kv_a_proj_with_mqa): Linear(in_features=32, out_features=64, bias=False)
          (kv_a_layernorm): Raon1MiniRMSNorm((32,), eps=1e-06)
          (kv_b_proj): Linear(in_features=32, out_features=64, bias=False)
          (g_proj): Linear(in_features=32, out_features=32, bias=False)
          (o_proj): Linear(in_features=32, out_features=32, bias=False)
        )
        (self_attn_raon_mhc): RaonMHC(
          (res): RaonMHCStreamMix()
        )
        (mlp_raon_mhc): RaonMHC(
          (res): RaonMHCStreamMix()
        )
        (moe): Raon1MiniMoE(
          (gate): Raon1MiniRouter()
          (experts): ModuleList(
            (0-63): 64 x Raon1MiniExpert(
              (gate_proj): Raon1MiniRoutedLinear(in_features=32, out_features=32, bias=False)
              (up_proj): Raon1MiniRoutedLinear(in_features=32, out_features=32, bias=False)
              (down_proj): Raon1MiniRoutedLinear(in_features=32, out_features=32, bias=False)
            )
          )
          (shared_experts): Raon1MiniMLP(
            (gate_proj): Linear(in_features=32, out_features=32, bias=False)
            (up_proj): Linear(in_features=32, out_features=32, bias=False)
            (down_proj): Linear(in_features=32, out_features=32, bias=False)
          )
        )
      )
      (4-6): 3 x Raon1MiniDecoderLayer(
        (input_layernorm): Raon1MiniRMSNorm((32,), eps=1e-05)
        (post_attention_layernorm): Raon1MiniRMSNorm((32,), eps=1e-05)
        (self_attn): Raon1MiniRDA(
          (q_proj): Linear(in_features=32, out_features=32, bias=False)
          (k_proj): Linear(in_features=32, out_features=32, bias=False)
          (v_proj): Linear(in_features=32, out_features=32, bias=False)
          (q_conv1d): Raon1MiniShortConv()
          (k_conv1d): Raon1MiniShortConv()
          (v_conv1d): Raon1MiniShortConv()
          (f_a_proj): Linear(in_features=32, out_features=32, bias=False)
          (f_b_proj): Linear(in_features=32, out_features=32, bias=False)
          (b_proj): Linear(in_features=32, out_features=1, bias=False)
          (rda_w_proj): Linear(in_features=32, out_features=1, bias=False)
          (g_proj): Linear(in_features=32, out_features=32, bias=False)
          (o_norm): Raon1MiniRMSNorm((32,), eps=1e-05)
          (o_proj): Linear(in_features=32, out_features=32, bias=False)
        )
        (self_attn_raon_mhc): RaonMHC(
          (res): RaonMHCStreamMix()
        )
        (mlp_raon_mhc): RaonMHC(
          (res): RaonMHCStreamMix()
        )
        (moe): Raon1MiniMoE(
          (gate): Raon1MiniRouter()
          (experts): ModuleList(
            (0-63): 64 x Raon1MiniExpert(
              (gate_proj): Raon1MiniRoutedLinear(in_features=32, out_features=32, bias=False)
              (up_proj): Raon1MiniRoutedLinear(in_features=32, out_features=32, bias=False)
              (down_proj): Raon1MiniRoutedLinear(in_features=32, out_features=32, bias=False)
            )
          )
          (shared_experts): Raon1MiniMLP(
            (gate_proj): Linear(in_features=32, out_features=32, bias=False)
            (up_proj): Linear(in_features=32, out_features=32, bias=False)
            (down_proj): Linear(in_features=32, out_features=32, bias=False)
          )
        )
      )
      (7): Raon1MiniDecoderLayer(
        (input_layernorm): Raon1MiniRMSNorm((32,), eps=1e-05)
        (post_attention_layernorm): Raon1MiniRMSNorm((32,), eps=1e-05)
        (self_attn): Raon1MiniMLA(
          (q_a_proj): Linear(in_features=32, out_features=32, bias=False)
          (q_a_layernorm): Raon1MiniRMSNorm((32,), eps=1e-06)
          (q_b_proj): Linear(in_features=32, out_features=64, bias=False)
          (kv_a_proj_with_mqa): Linear(in_features=32, out_features=64, bias=False)
          (kv_a_layernorm): Raon1MiniRMSNorm((32,), eps=1e-06)
          (kv_b_proj): Linear(in_features=32, out_features=64, bias=False)
          (g_proj): Linear(in_features=32, out_features=32, bias=False)
          (o_proj): Linear(in_features=32, out_features=32, bias=False)
        )
        (self_attn_raon_mhc): RaonMHC(
          (res): RaonMHCStreamMix()
        )
        (mlp_raon_mhc): RaonMHC(
          (res): RaonMHCStreamMix()
        )
        (moe): Raon1MiniMoE(
          (gate): Raon1MiniRouter()
          (experts): ModuleList(
            (0-63): 64 x Raon1MiniExpert(
              (gate_proj): Raon1MiniRoutedLinear(in_features=32, out_features=32, bias=False)
              (up_proj): Raon1MiniRoutedLinear(in_features=32, out_features=32, bias=False)
              (down_proj): Raon1MiniRoutedLinear(in_features=32, out_features=32, bias=False)
            )
          )
          (shared_experts): Raon1MiniMLP(
            (gate_proj): Linear(in_features=32, out_features=32, bias=False)
            (up_proj): Linear(in_features=32, out_features=32, bias=False)
            (down_proj): Linear(in_features=32, out_features=32, bias=False)
          )
        )
      )
    )
    (norm): Raon1MiniRMSNorm((32,), eps=1e-05)
  )
  (lm_head): Linear(in_features=32, out_features=259840, bias=False)
)
```

</details>

### Test environment:

- huggingface_hub: 1.27.0
- safetensors: 0.8.0
- tokenizers: 0.23.0rc0
- torch: 2.13.0+cu126
- transformers: 5.15.0.dev0
### Change log:
- 2026-10-23: Initial version.
This repo is generated by GPT-6.1-Sol

	---
	library_name: transformers
	base_model:
	- KRAFTON/Raon-1-mini-preview
	---

	This tiny model is intended for debugging. It is randomly initialized using the configuration adapted from [KRAFTON/Raon-1-mini-preview](https://huggingface.co/KRAFTON/Raon-1-mini-preview).

	Structural comparison with `KRAFTON/Raon-1-mini-preview`:

	\| Feature \| Source \| Tiny \|
	\|---\|---:\|---:\|
	\| Vocabulary / tied embeddings \| 259,840 / untied \| same \|
	\| Layers \| 25 \| 8; two 3-RDA + 1-MLA cycles \|
	\| RDA / NoPE MLA layer cadence \| 18 / 7 \| 6 / 2 \|
	\| Raon mHC streams \| 4 \| same \|
	\| Dense prefix / MoE layers \| layer 0 / 24 layers \| layer 0 / 7 layers \|
	\| Routed experts / active experts \| 128 / 6 \| 64 / 6 \|
	\| Hidden width / attention head dimension \| 2,560 / 128 \| 32 / 32 \|
	\| RDA, MLA and LoRA dimensions \| source widths \| 32-wide compatible projections \|
	\| Routed-expert storage \| MXFP8 E4M3 + E8M0 1x32 scales \| same \|
	\| Router / shared expert / output gate \| sigmoid top-k / present / present \| same \|

	Transformers constructs Raon's architecture, then the definition rewrites only
	routed-expert tensors to the source MXFP8 storage representation. This is
	necessary because random CPU initialization of `float8_e4m3fn` parameters is
	not implemented by PyTorch; it does not change the serialized precision.

	\| File path \| Size \|
	\|------\|------\|
	\| model.safetensors \| 35.3MB \|


	### Example usage:

	- Transformers (✅ verified)

	```python
	import torch
	from transformers import AutoModelForCausalLM, AutoTokenizer

	model_id = "tiny-random/raon-1-mini"
	device = "cuda" if torch.cuda.is_available() else "cpu"
	model = AutoModelForCausalLM.from_pretrained(model_id, trust_remote_code=True).to(device).eval()
	tokenizer = AutoTokenizer.from_pretrained(model_id, trust_remote_code=True)
	inputs = tokenizer("Hello", return_tensors="pt").to(device)
	output = model.generate(**inputs, do_sample=False, min_new_tokens=10, max_new_tokens=10)
	print(tokenizer.decode(output[0, inputs["input_ids"].shape[1]:], skip_special_tokens=True))
	```

	### Codes to create this repo (✅ verified):

	<details><summary>Click to expand</summary>

	```python
	import json
	from pathlib import Path
	from shutil import copyfile

	import torch
	from huggingface_hub import hf_hub_download, list_repo_files
	from safetensors.torch import save_file
	from transformers import AutoConfig, AutoModelForCausalLM, set_seed

	source_model_id = "KRAFTON/Raon-1-mini-preview"
	save_folder = "/tmp/tiny-random/raon-1-mini"
	source_revision = "7bf87c45fc7b658d3c9532305f3aaed106bbac4f"
	set_seed(42)
	folder = Path(save_folder)
	folder.mkdir(parents=True, exist_ok=True)
	raw = json.loads(Path(hf_hub_download(source_model_id, "config.json", revision=source_revision)).read_text())
	raw.update(
	hidden_size=32, head_dim=32, num_attention_heads=1, num_hidden_layers=8, intermediate_size=128,
	num_experts=64, num_experts_per_token=6, moe_intermediate_size=32,
	q_lora_rank=32, kv_lora_rank=32, qk_nope_head_dim=32,
	qk_rope_head_dim=32, v_head_dim=32, rda_decay_rank=32,
	max_position_embeddings=4096,
	)
	raw["rda_config"].update(
	num_heads=1, head_dim=32,
	rda_layers=[1, 2, 3, 5, 6, 7], mla_layers=[4, 8],
	)
	# Build the public remote-code graph in BF16 first. The source's MXFP8
	# modules cannot initialize random float8 parameters directly, then the
	# routed experts are converted back to the source's E4M3 + E8M0 format.
	construction_raw = json.loads(json.dumps(raw))
	construction_raw["quantization_config"]["dequantize"] = True
	(folder / "config.json").write_text(json.dumps(construction_raw, indent=2) + "\n", encoding="utf-8")
	for filename in ("configuration_raon1_mini.py", "modeling_raon1_mini.py"):
	copyfile(hf_hub_download(source_model_id, filename, revision=source_revision), folder / filename)
	config = AutoConfig.from_pretrained(folder, trust_remote_code=True)
	model = AutoModelForCausalLM.from_config(config, trust_remote_code=True).eval()
	with torch.no_grad():
	for name, parameter in sorted(model.named_parameters()):
	if name.endswith(("weight", "phi_pre", "phi_post", "phi_z")) and not name.endswith("norm.weight"):
	torch.nn.init.normal_(parameter, mean=0, std=0.02)
	elif name.endswith("norm.weight") or name.endswith("layernorm.weight"):
	parameter.fill_(1)

	def quantize_mxfp8(weight: torch.Tensor) -> tuple[torch.Tensor, torch.Tensor]:
	rows, columns = weight.shape
	assert columns % 32 == 0, f"MXFP8 columns must divide into 32-value blocks, got {columns}"
	blocks = weight.float().reshape(rows, columns // 32, 32)
	bits = (blocks.abs().amax(-1) / 448.0).view(torch.int32)
	exponent, mantissa = (bits >> 23) & 0xFF, bits & 0x7FFFFF
	round_up = (mantissa > 0) & (exponent != 0xFE) & ~((exponent == 0) & (mantissa <= 0x400000))
	scale = (exponent + round_up.int()).clamp(max=254)
	reciprocal = ((254 - scale) << 23).view(torch.float32)
	reciprocal = torch.where(scale == 254, torch.full_like(reciprocal, 2.0**-127), reciprocal)
	values = (blocks * reciprocal.unsqueeze(-1)).clamp(-448.0, 448.0)
	return values.to(torch.float8_e4m3fn).reshape(rows, columns), scale.to(torch.uint8)

	state = model.state_dict()
	for name in sorted(tuple(state)):
	if ".moe.experts." in name and name.endswith(".weight"):
	values, scales = quantize_mxfp8(state[name])
	state[name] = values
	state[name.removesuffix(".weight") + ".weight_scale_inv"] = scales
	raw["quantization_config"]["dequantize"] = False
	(folder / "config.json").write_text(json.dumps(raw, indent=2) + "\n", encoding="utf-8")
	save_file(state, folder / "model.safetensors", metadata={"format": "pt"})
	assets = {"LICENSE", "chat_template.jinja", "generation_config.json", "tokenizer.json", "tokenizer_config.json"}
	for filename in sorted(set(list_repo_files(source_model_id, revision=source_revision)) & assets):
	copyfile(hf_hub_download(source_model_id, filename, revision=source_revision), folder / filename)
	```

	</details>

	### Printing the model:

	<details><summary>Click to expand</summary>

	```text
	Raon1MiniForCausalLM(
	(model): Raon1MiniModel(
	(embed_tokens): Embedding(259840, 32, padding_idx=259200)
	(layers): ModuleList(
	(0): Raon1MiniDecoderLayer(
	(input_layernorm): Raon1MiniRMSNorm((32,), eps=1e-05)
	(post_attention_layernorm): Raon1MiniRMSNorm((32,), eps=1e-05)
	(self_attn): Raon1MiniRDA(
	(q_proj): Linear(in_features=32, out_features=32, bias=False)
	(k_proj): Linear(in_features=32, out_features=32, bias=False)
	(v_proj): Linear(in_features=32, out_features=32, bias=False)
	(q_conv1d): Raon1MiniShortConv()
	(k_conv1d): Raon1MiniShortConv()
	(v_conv1d): Raon1MiniShortConv()
	(f_a_proj): Linear(in_features=32, out_features=32, bias=False)
	(f_b_proj): Linear(in_features=32, out_features=32, bias=False)
	(b_proj): Linear(in_features=32, out_features=1, bias=False)
	(rda_w_proj): Linear(in_features=32, out_features=1, bias=False)
	(g_proj): Linear(in_features=32, out_features=32, bias=False)
	(o_norm): Raon1MiniRMSNorm((32,), eps=1e-05)
	(o_proj): Linear(in_features=32, out_features=32, bias=False)
	)
	(self_attn_raon_mhc): RaonMHC(
	(res): RaonMHCStreamMix()
	)
	(mlp_raon_mhc): RaonMHC(
	(res): RaonMHCStreamMix()
	)
	(mlp): Raon1MiniMLP(
	(gate_proj): Linear(in_features=32, out_features=128, bias=False)
	(up_proj): Linear(in_features=32, out_features=128, bias=False)
	(down_proj): Linear(in_features=128, out_features=32, bias=False)
	)
	)
	(1-2): 2 x Raon1MiniDecoderLayer(
	(input_layernorm): Raon1MiniRMSNorm((32,), eps=1e-05)
	(post_attention_layernorm): Raon1MiniRMSNorm((32,), eps=1e-05)
	(self_attn): Raon1MiniRDA(
	(q_proj): Linear(in_features=32, out_features=32, bias=False)
	(k_proj): Linear(in_features=32, out_features=32, bias=False)
	(v_proj): Linear(in_features=32, out_features=32, bias=False)
	(q_conv1d): Raon1MiniShortConv()
	(k_conv1d): Raon1MiniShortConv()
	(v_conv1d): Raon1MiniShortConv()
	(f_a_proj): Linear(in_features=32, out_features=32, bias=False)
	(f_b_proj): Linear(in_features=32, out_features=32, bias=False)
	(b_proj): Linear(in_features=32, out_features=1, bias=False)
	(rda_w_proj): Linear(in_features=32, out_features=1, bias=False)
	(g_proj): Linear(in_features=32, out_features=32, bias=False)
	(o_norm): Raon1MiniRMSNorm((32,), eps=1e-05)
	(o_proj): Linear(in_features=32, out_features=32, bias=False)
	)
	(self_attn_raon_mhc): RaonMHC(
	(res): RaonMHCStreamMix()
	)
	(mlp_raon_mhc): RaonMHC(
	(res): RaonMHCStreamMix()
	)
	(moe): Raon1MiniMoE(
	(gate): Raon1MiniRouter()
	(experts): ModuleList(
	(0-63): 64 x Raon1MiniExpert(
	(gate_proj): Raon1MiniRoutedLinear(in_features=32, out_features=32, bias=False)
	(up_proj): Raon1MiniRoutedLinear(in_features=32, out_features=32, bias=False)
	(down_proj): Raon1MiniRoutedLinear(in_features=32, out_features=32, bias=False)
	)
	)
	(shared_experts): Raon1MiniMLP(
	(gate_proj): Linear(in_features=32, out_features=32, bias=False)
	(up_proj): Linear(in_features=32, out_features=32, bias=False)
	(down_proj): Linear(in_features=32, out_features=32, bias=False)
	)
	)
	)
	(3): Raon1MiniDecoderLayer(
	(input_layernorm): Raon1MiniRMSNorm((32,), eps=1e-05)
	(post_attention_layernorm): Raon1MiniRMSNorm((32,), eps=1e-05)
	(self_attn): Raon1MiniMLA(
	(q_a_proj): Linear(in_features=32, out_features=32, bias=False)
	(q_a_layernorm): Raon1MiniRMSNorm((32,), eps=1e-06)
	(q_b_proj): Linear(in_features=32, out_features=64, bias=False)
	(kv_a_proj_with_mqa): Linear(in_features=32, out_features=64, bias=False)
	(kv_a_layernorm): Raon1MiniRMSNorm((32,), eps=1e-06)
	(kv_b_proj): Linear(in_features=32, out_features=64, bias=False)
	(g_proj): Linear(in_features=32, out_features=32, bias=False)
	(o_proj): Linear(in_features=32, out_features=32, bias=False)
	)
	(self_attn_raon_mhc): RaonMHC(
	(res): RaonMHCStreamMix()
	)
	(mlp_raon_mhc): RaonMHC(
	(res): RaonMHCStreamMix()
	)
	(moe): Raon1MiniMoE(
	(gate): Raon1MiniRouter()
	(experts): ModuleList(
	(0-63): 64 x Raon1MiniExpert(
	(gate_proj): Raon1MiniRoutedLinear(in_features=32, out_features=32, bias=False)
	(up_proj): Raon1MiniRoutedLinear(in_features=32, out_features=32, bias=False)
	(down_proj): Raon1MiniRoutedLinear(in_features=32, out_features=32, bias=False)
	)
	)
	(shared_experts): Raon1MiniMLP(
	(gate_proj): Linear(in_features=32, out_features=32, bias=False)
	(up_proj): Linear(in_features=32, out_features=32, bias=False)
	(down_proj): Linear(in_features=32, out_features=32, bias=False)
	)
	)
	)
	(4-6): 3 x Raon1MiniDecoderLayer(
	(input_layernorm): Raon1MiniRMSNorm((32,), eps=1e-05)
	(post_attention_layernorm): Raon1MiniRMSNorm((32,), eps=1e-05)
	(self_attn): Raon1MiniRDA(
	(q_proj): Linear(in_features=32, out_features=32, bias=False)
	(k_proj): Linear(in_features=32, out_features=32, bias=False)
	(v_proj): Linear(in_features=32, out_features=32, bias=False)
	(q_conv1d): Raon1MiniShortConv()
	(k_conv1d): Raon1MiniShortConv()
	(v_conv1d): Raon1MiniShortConv()
	(f_a_proj): Linear(in_features=32, out_features=32, bias=False)
	(f_b_proj): Linear(in_features=32, out_features=32, bias=False)
	(b_proj): Linear(in_features=32, out_features=1, bias=False)
	(rda_w_proj): Linear(in_features=32, out_features=1, bias=False)
	(g_proj): Linear(in_features=32, out_features=32, bias=False)
	(o_norm): Raon1MiniRMSNorm((32,), eps=1e-05)
	(o_proj): Linear(in_features=32, out_features=32, bias=False)
	)
	(self_attn_raon_mhc): RaonMHC(
	(res): RaonMHCStreamMix()
	)
	(mlp_raon_mhc): RaonMHC(
	(res): RaonMHCStreamMix()
	)
	(moe): Raon1MiniMoE(
	(gate): Raon1MiniRouter()
	(experts): ModuleList(
	(0-63): 64 x Raon1MiniExpert(
	(gate_proj): Raon1MiniRoutedLinear(in_features=32, out_features=32, bias=False)
	(up_proj): Raon1MiniRoutedLinear(in_features=32, out_features=32, bias=False)
	(down_proj): Raon1MiniRoutedLinear(in_features=32, out_features=32, bias=False)
	)
	)
	(shared_experts): Raon1MiniMLP(
	(gate_proj): Linear(in_features=32, out_features=32, bias=False)
	(up_proj): Linear(in_features=32, out_features=32, bias=False)
	(down_proj): Linear(in_features=32, out_features=32, bias=False)
	)
	)
	)
	(7): Raon1MiniDecoderLayer(
	(input_layernorm): Raon1MiniRMSNorm((32,), eps=1e-05)
	(post_attention_layernorm): Raon1MiniRMSNorm((32,), eps=1e-05)
	(self_attn): Raon1MiniMLA(
	(q_a_proj): Linear(in_features=32, out_features=32, bias=False)
	(q_a_layernorm): Raon1MiniRMSNorm((32,), eps=1e-06)
	(q_b_proj): Linear(in_features=32, out_features=64, bias=False)
	(kv_a_proj_with_mqa): Linear(in_features=32, out_features=64, bias=False)
	(kv_a_layernorm): Raon1MiniRMSNorm((32,), eps=1e-06)
	(kv_b_proj): Linear(in_features=32, out_features=64, bias=False)
	(g_proj): Linear(in_features=32, out_features=32, bias=False)
	(o_proj): Linear(in_features=32, out_features=32, bias=False)
	)
	(self_attn_raon_mhc): RaonMHC(
	(res): RaonMHCStreamMix()
	)
	(mlp_raon_mhc): RaonMHC(
	(res): RaonMHCStreamMix()
	)
	(moe): Raon1MiniMoE(
	(gate): Raon1MiniRouter()
	(experts): ModuleList(
	(0-63): 64 x Raon1MiniExpert(
	(gate_proj): Raon1MiniRoutedLinear(in_features=32, out_features=32, bias=False)
	(up_proj): Raon1MiniRoutedLinear(in_features=32, out_features=32, bias=False)
	(down_proj): Raon1MiniRoutedLinear(in_features=32, out_features=32, bias=False)
	)
	)
	(shared_experts): Raon1MiniMLP(
	(gate_proj): Linear(in_features=32, out_features=32, bias=False)
	(up_proj): Linear(in_features=32, out_features=32, bias=False)
	(down_proj): Linear(in_features=32, out_features=32, bias=False)
	)
	)
	)
	)
	(norm): Raon1MiniRMSNorm((32,), eps=1e-05)
	)
	(lm_head): Linear(in_features=32, out_features=259840, bias=False)
	)
	```

	</details>

	### Test environment:

	- huggingface_hub: 1.27.0
	- safetensors: 0.8.0
	- tokenizers: 0.23.0rc0
	- torch: 2.13.0+cu126
	- transformers: 5.15.0.dev0

	### Change log:

	- 2026-10-23: Initial version.

	This repo is generated by GPT-6.1-Sol