README.md

---
license: mit
base_model:
  - deepseek-ai/DeepSeek-V4-Flash-Vision-Exp
tags:
  - deepseek_v4
  - fixture
  - testing
  - moe
  - mla
pipeline_tag: text-generation
---

# DeepSeek-V4-Flash-Vision-Exp tiny architecture fixture (deepseek_v4)

A ~0.29M-parameter random-init TEXT checkpoint plus a reduced config that keeps the
real **DeepSeek-V4-Flash-Vision-Exp** `deepseek_v4` field names, so loaders, quant
planners, and CI jobs can exercise the new architecture's config parsing, expert-table
sizing, and safetensors load path without the real weights.

## What this is
- Base model: `deepseek-ai/DeepSeek-V4-Flash-Vision-Exp` (created 2026-08-31),
  `model_type: deepseek_v4`, `architectures: ["DeepseekV4ForCausalLM"]`, MIT license.
  The base is a large multimodal MoE (image-text-to-text) with MLA attention, routed +
  shared experts, a DSA index, hash layers, an MTP head, hyper-connections (`hc_*`),
  a `dspark_*` block, and a full vision encoder, so it cannot be instantiated in a unit
  test, in CI, or on a laptop.
- What this fixture is: a byte-reproducible **random-init** text checkpoint plus a
  reduced config that preserves the real `deepseek_v4` field names — MLA
  (`q_lora_rank`, `o_lora_rank`, `o_groups`, `qk_rope_head_dim`), MoE
  (`n_routed_experts`, `n_shared_experts`, `num_experts_per_tok`, `topk_method`,
  `scoring_func`, `routed_scaling_factor`, `expert_dtype`), the DSA/hash knobs
  (`num_hash_layers`), the MTP knob (`num_nextn_predict_layers`), `swiglu_limit`,
  `sliding_window`, and the `yarn` `rope_scaling` block.
- How it is changed from the base: same schema, tiny geometry, float32, and a
  **reduced standard-attention + MoE tensor set**. It is a schema/load fixture, not a
  quantization or a distilled model.
- What it is **not**: not trained, not distilled, not a quality or benchmark claim.
- Why it is useful: it lets you test config parsing, weight-name mapping,
  expert-table sizing, router/top-k bookkeeping, and safetensors load paths in
  milliseconds, and it exercises the `deepseek_v4` text-config shape that a plain
  Llama-style fixture does not.

## Fixture geometry (what this checkpoint actually contains)
| Field | Value |
|---|---|
| num_hidden_layers | 4 |
| hidden_size | 64 |
| num_attention_heads / num_key_value_heads | 4 / 1 |
| head_dim | 16 |
| q_lora_rank / o_lora_rank / o_groups | 32 / 32 / 4 |
| n_routed_experts / n_shared_experts | 8 / 1 |
| num_experts_per_tok | 2 |
| moe_intermediate_size | 32 |
| num_nextn_predict_layers (MTP) | 0 (omitted) |
| num_hash_layers | 0 (omitted) |
| dtype | float32 |
| vocab_size | 256 |

Total: 150 tensors, 1,162,496 data bytes = 290,624 float32 parameters.

## Intentional omissions (documented, not silent)
- **Text-only.** No vision encoder or projector tensors. A full multimodal loader must
  supply vision/projector tensors.
- The tensor set is a **reduced standard-attention + MoE convention**. The real model's
  MLA compressed tensors, DSA index tensors, hash-layer tensors, MTP head, and
  hyper-connection / `dspark_*` tensors are NOT included.
- No `lm_head` tensor; a loader must tie to `model.embed_tokens.weight` or supply its
  own head.
- Tokenizer metadata files are placeholders (no vocab file); use your own tokenizer.

## Verification actually performed (stdlib only, no torch in this environment)
- safetensors header parses: 150 tensors, contiguous `data_offsets`, header padded to
  8-byte alignment; `__metadata__` records the generator string.
- `config.json` parses; `model_type` is `deepseek_v4` with the real field names listed
  above.
- `checksums.txt` records the SHA-256 of every tensor blob.
- Deterministic regeneration: SplitMix64 seed 20260903, Box-Muller normals, scale 0.02,
  float32 row-major, consumed in sorted-name order.
- Generator script is included as `build_fixture.py` so the folder can be rebuilt and
  diffed.

**Not yet verified:** loading under a specific `transformers` version (no
torch/transformers in the build environment), and whether `DeepseekV4ForCausalLM`
accepts this reduced text-only geometry without the MLA / DSA / hash / MTP / vision
tensors. Treat those as open until run against a real install.

## How to run it
Read the tensors with the standard library (no torch needed, matching how this was
built):

```python
import json, struct
with open("model.safetensors", "rb") as f:
    n = struct.unpack("<Q", f.read(8))[0]
    header = json.loads(f.read(n))
    # header[name] = {"dtype", "shape", "data_offsets"}; data starts at byte 8+n
```

Or with the `safetensors` package:

```python
from safetensors.torch import load_file
tensors = load_file("model.safetensors")   # {name: tensor}
```

To exercise a real loader, build a config from `config.json` (the `deepseek_v4`
model type; use `AutoConfig.from_pretrained(..., trust_remote_code=True)` where
needed) and feed these weights in. There is no `lm_head` tensor, no vision stack, and
the tokenizer files are placeholders, so supply your own head/tokenizer/vision.

## License
The generated fixture content (random weights, config, scripts) is released under MIT
(see `LICENSE`). The `deepseek_v4` architecture and config schema belong to the base
model `deepseek-ai/DeepSeek-V4-Flash-Vision-Exp` under its own terms, which were
**not** independently re-verified this cycle — check the base repository before
redistribution.

## Citation
DeepSeek, DeepSeek-V4-Flash-Vision-Exp, 2026.

---

Buy me a beer:
**BTC** `bc1q5ayht3fxhj0v95fk0z8l2f6900g3awdsw5842p`