README.md

---
library_name: sglang
base_model:
- RecursiveSI/flint
---

This tiny model is intended for debugging. It is randomly initialized using the configuration adapted from [RecursiveSI/flint](https://huggingface.co/RecursiveSI/flint).

Structural comparison with `RecursiveSI/flint`:

| Feature | Source | Tiny |
|---|---:|---:|
| Loading implementation | source SGLang plugin | same source plugin wheel |
| Stored precision | BF16 | BF16 |
| Vocabulary / embedding and LM-head ties | 150,144 / untied | same |
| Layers / dense prefix | 49 / layers 0-1 | 8 / layers 0-1 |
| Attention cadence / RoPE-free layers | 49-layer source schedule | first 8 source layers |
| Hidden size / Q and KV heads / head dimension | 2,048 / 32 and 4 / 128 | 32 / 1 and 1 / 32 |
| Routed experts / active experts | 256 / 6 | 64 / 6 |
| MoE / shared-expert widths | 512 / 1,024 | 32 / 64 |
| Sigmoid router, top-k renormalization, learned sinks, QK norms | present | same checkpoint schema |

Direct checkpoint generation is required because Transformers does not implement
Flint.  Tensor names and shapes come from the pinned source safetensors headers
and the source-provided `flint_sglang` loader wheel.  The source plugin explicitly
validates TP=1 only, so this model carries no TP=4 claim.

| File path | Size |
|------|------|
| model.safetensors | 21.9MB |


### Example usage:

- SGLang (unverified)

- The source Flint plugin validates one GPU per replica only; it explicitly marks tensor/expert/pipeline parallelism as unvalidated.  The tiny checkpoint therefore does not claim TP=4 support.

- Install the source-provided plugin wheel before this command.  Its learned attention sinks, sigmoid-router top-k, shared expert, tool grammar and reasoning parser are not implemented by stock SGLang.

```bash
pip install ./sglang/flint_sglang-0.1.4-py3-none-any.whl
SGLANG_EXTERNAL_MODEL_PACKAGE=flint_sglang FLASHINFER_USE_CUDA_NORM=1 \
python -m sglang.launch_server \
  --model-path tiny-random/flint --served-model-name flint --tp-size 1 \
  --attention-backend fa3 --moe-runner-backend triton --enable-fp32-lm-head \
  --tool-call-parser flint --reasoning-parser flint --grammar-backend flint
```

### Codes to create this repo:

<details><summary>Click to expand</summary>

```python
import json
from pathlib import Path
from shutil import copyfile

import torch
from huggingface_hub import hf_hub_download
from safetensors.torch import save_file

source_model_id = "RecursiveSI/flint"
save_folder = "/tmp/tiny-random/flint"
source_revision = "2a247be0c0aec895ccb0ac7ca92229ec6df37288"
folder = Path(save_folder)
folder.mkdir(parents=True, exist_ok=True)
raw = json.loads(Path(hf_hub_download(source_model_id, "config.json", revision=source_revision)).read_text())
# Transformers has no Flint model class.  The source SGLang plugin is the
# authoritative executable schema, so create its public checkpoint tensors directly.
# The fixed 150,144-row untied embedding and LM head make hidden_size=32 the
# smallest practical BF16 geometry; one 32-wide attention head retains the
# required LLM head-dimension floor.
layers = 8
raw.update(
    hidden_size=32, head_dim=32, num_attention_heads=1, num_key_value_heads=1,
    intermediate_size=128, moe_intermediate_size=32, shared_expert_intermediate_size=64,
    num_experts=64, num_experts_per_tok=6, max_position_embeddings=4096,
    num_hidden_layers=layers,
    layer_types=raw["layer_types"][:layers],
    no_rope_layer_indices=[index for index in raw["no_rope_layer_indices"] if index < layers],
)
(folder / "config.json").write_text(json.dumps(raw, indent=2) + "\n", encoding="utf-8")
generator = torch.Generator(device="cpu").manual_seed(42)

def random_tensor(*shape: int) -> torch.Tensor:
    return torch.randn(shape, generator=generator, dtype=torch.bfloat16).mul_(0.02)

hidden = raw["hidden_size"]
head_dim = raw["head_dim"]
experts = raw["num_experts"]
state: dict[str, torch.Tensor] = {
    "model.embed_tokens.weight": random_tensor(raw["vocab_size"], hidden),
    "lm_head.weight": random_tensor(raw["vocab_size"], hidden),
    "model.norm.weight": torch.ones(hidden, dtype=torch.bfloat16),
}
dense_layers = set(raw["mlp_only_layers"])
for layer in range(raw["num_hidden_layers"]):
    prefix = f"model.layers.{layer}"
    state[f"{prefix}.input_layernorm.weight"] = torch.ones(hidden, dtype=torch.bfloat16)
    state[f"{prefix}.post_attention_layernorm.weight"] = torch.ones(hidden, dtype=torch.bfloat16)
    state[f"{prefix}.self_attn.q_proj.weight"] = random_tensor(hidden, hidden)
    state[f"{prefix}.self_attn.k_proj.weight"] = random_tensor(head_dim, hidden)
    state[f"{prefix}.self_attn.v_proj.weight"] = random_tensor(head_dim, hidden)
    state[f"{prefix}.self_attn.o_proj.weight"] = random_tensor(hidden, hidden)
    state[f"{prefix}.self_attn.q_norm.weight"] = torch.ones(head_dim, dtype=torch.bfloat16)
    state[f"{prefix}.self_attn.k_norm.weight"] = torch.ones(head_dim, dtype=torch.bfloat16)
    state[f"{prefix}.self_attn.softmax_offset"] = random_tensor(raw["num_attention_heads"])
    if layer in dense_layers:
        for projection in ("gate_proj", "up_proj"):
            state[f"{prefix}.mlp.{projection}.weight"] = random_tensor(raw["intermediate_size"], hidden)
        state[f"{prefix}.mlp.down_proj.weight"] = random_tensor(hidden, raw["intermediate_size"])
        continue
    state[f"{prefix}.mlp.gate.weight"] = random_tensor(experts, hidden)
    for projection in ("gate_proj", "up_proj"):
        state[f"{prefix}.mlp.shared_expert.{projection}.weight"] = random_tensor(
            raw["shared_expert_intermediate_size"], hidden
        )
    state[f"{prefix}.mlp.shared_expert.down_proj.weight"] = random_tensor(
        hidden, raw["shared_expert_intermediate_size"]
    )
    for expert in range(experts):
        expert_prefix = f"{prefix}.mlp.experts.{expert}"
        for projection in ("gate_proj", "up_proj"):
            state[f"{expert_prefix}.{projection}.weight"] = random_tensor(raw["moe_intermediate_size"], hidden)
        state[f"{expert_prefix}.down_proj.weight"] = random_tensor(hidden, raw["moe_intermediate_size"])
save_file(state, folder / "model.safetensors", metadata={"format": "pt"})
for filename in (
    "LICENSE", "chat_template.jinja", "generation_config.json", "tokenizer.json", "tokenizer_config.json",
    "sglang/flint_sglang-0.1.4-py3-none-any.whl", "sglang/serve_h200.sh", "sglang/README.md",
):
    destination = folder / filename
    destination.parent.mkdir(parents=True, exist_ok=True)
    copyfile(hf_hub_download(source_model_id, filename, revision=source_revision), destination)
```

</details>

### Test environment:

- huggingface_hub: 1.27.0
- safetensors: 0.8.0
- tokenizers: 0.23.0rc0
- torch: 2.13.0+cu126
- transformers: 5.15.0.dev0

### Change log:

- 2026-10-23: Initial version.

This repo is generated by GPT-6.1-Sol