SOVEREIGN — sovereign AI model card (derived from DeepSeek-V4-Flash-0731): specs, config, tokenizer, generation config

README.md

@@ -0,0 +1,56 @@
+# 🌌 deepseek-v4-sovereign — the new sovereign AI model
+
+**SOVEREIGN's flagship model.** A sovereign-tuned derivative of
+[`deepseek-ai/DeepSeek-V4-Flash-0731`](https://huggingface.co/deepseek-ai/DeepSeek-V4-Flash-0731)
+(304B MoE, MIT license) with an extended reasoning budget, 1M-token context,
+and DSpark speculative decoding — tuned on private, user-owned corpora only.
+
+## Model card
+
+| Field | Value |
+|---|---|
+| Base | `deepseek-ai/DeepSeek-V4-Flash-0731` |
+| Architecture | Mixture-of-Experts (MoE) + speculative decoding (DSpark) |
+| Parameters | 304B total, fraction activated per token |
+| Context window | 1,048,576 tokens (1M) |
+| Max output | 384K tokens (high/max reasoning) |
+| Reasoning effort | `low` / `high` / `max` |
+| Precision | BF16 / FP16 / FP32 / FP8 (E4M3, E2M1) / INT8 |
+| Quantizations | 90+ model tree (GGUF/safetensors) |
+| License | MIT (derived) |
+| Paper | arXiv:2606.19348 |
+| Sovereign property | weights + fine-tunes stored locally; zero mandatory telemetry |
+
+## Why "sovereign"
+
+- Runs fully locally (vLLM / SGLang / transformers) or via your own VPC.
+- No mandatory external API calls; HF router (`router.huggingface.co/v1`)
+  is an *option*, not a dependency.
+- Fine-tuning data, adapters, and inference logs stay under your control.
+
+## Deployment
+
+```bash
+# vLLM (4×GB300 node) — DSpark speculative decoding enabled with one flag
+vllm serve deepseek-ai/DeepSeek-V4-Flash-0731 \
+  --trust-remote-code --kv-cache-dtype fp8 --block-size 256 \
+  --data-parallel-size 4 --enable-expert-parallel \
+  --moe-backend deep_gemm_mega_moe \
+  --speculative-config '{"method":"dspark","num_speculative_tokens":7,"draft_sample_method":"greedy"}'
+
+# SGLang
+sglang serve --trust-remote-code \
+  --model-path deepseek-ai/DeepSeek-V4-Flash-0731 \
+  --tp 4 --moe-runner-backend flashinfer_mxfp4 \
+  --speculative-algorithm DSPARK --chunked-prefill-size 4096
+```
+
+Recommended sampling for agentic scenarios: `temperature=1.0, top_p=0.95`;
+otherwise `top_p=1.0`.
+
+## Chat template
+
+No Jinja template ships with this release — encode via the OpenAI-compatible
+contract: `encoding_dsv4.encode_messages(messages, thinking_mode="thinking",
+reasoning_effort="max")` (see `models/deepseek-v4-flash-0731/` docs and
+`api_examples/`).

config.json

@@ -0,0 +1,38 @@
+{
+  "architectures": ["DeepseekV4ForCausalLM"],
+  "model_type": "deepseek_v4",
+  "model_id": "sovereign/deepseek-v4-sovereign",
+  "base_model": "deepseek-ai/DeepSeek-V4-Flash-0731",
+  "torch_dtype": "bfloat16",
+  "vocab_size": 129280,
+  "hidden_size": 7168,
+  "num_hidden_layers": 61,
+  "num_attention_heads": 64,
+  "num_key_value_heads": 8,
+  "head_dim": 128,
+  "num_experts": 256,
+  "num_experts_per_tok": 8,
+  "intermediate_size": 18432,
+  "max_position_embeddings": 1048576,
+  "rope_theta": 1000000.0,
+  "speculative_decoding": {
+    "method": "dspark",
+    "num_speculative_tokens": 7,
+    "draft_sample_method": "greedy"
+  },
+  "moe": {
+    "backend": "deep_gemm_mega_moe",
+    "expert_parallel": true
+  },
+  "attention": {
+    "use_fp4_indexer_cache": true,
+    "swa_full_tokens_ratio": 0.1
+  },
+  "quantization_config": {
+    "quant_method": "gptq",
+    "bits": 4,
+    "group_size": 128,
+    "damp_percent": 0.01
+  },
+  "transformers_version": "4.46.0"
+}

generation_config.json

@@ -0,0 +1,14 @@
+{
+  "temperature": 1.0,
+  "top_p": 0.95,
+  "top_k": 40,
+  "max_new_tokens": 384000,
+  "repetition_penalty": 1.0,
+  "do_sample": true,
+  "reasoning_effort": "max",
+  "thinking_mode": "thinking",
+  "use_cache": true,
+  "pad_token_id": 129277,
+  "bos_token_id": 129278,
+  "eos_token_id": [129279, 129280]
+}

model_specs.json

@@ -0,0 +1,37 @@
+{
+  "model_id": "sovereign/deepseek-v4-sovereign",
+  "display_name": "DeepSeek-V4-Sovereign",
+  "base_model": "deepseek-ai/DeepSeek-V4-Flash-0731",
+  "license": "MIT",
+  "model_size_params": "304B",
+  "architecture": "Mixture-of-Experts (MoE) with DSpark speculative decoding",
+  "precision_supported": ["BF16", "F16", "F32", "F8", "E4M3", "E2M1", "INT8"],
+  "context_window": 1048576,
+  "max_output_tokens": 384000,
+  "reasoning_effort_levels": ["low", "high", "max"],
+  "recommended_sampling": {
+    "agentic": {"temperature": 1.0, "top_p": 0.95},
+    "default": {"temperature": 1.0, "top_p": 1.0}
+  },
+  "speculative_decoding": {
+    "method": "dspark",
+    "num_speculative_tokens": 7,
+    "draft_sample_method": "greedy"
+  },
+  "serving": {
+    "vllm": "vllm serve deepseek-ai/DeepSeek-V4-Flash-0731 --trust-remote-code --kv-cache-dtype fp8 --block-size 256 --data-parallel-size 4 --enable-expert-parallel --moe-backend deep_gemm_mega_moe --speculative-config '{\"method\":\"dspark\",\"num_speculative_tokens\":7,\"draft_sample_method\":\"greedy\"}'",
+    "sglang": "sglang serve --trust-remote-code --model-path deepseek-ai/DeepSeek-V4-Flash-0731 --tp 4 --moe-runner-backend flashinfer_mxfp4 --speculative-algorithm DSPARK --mem-fraction-static 0.90 --chunked-prefill-size 4096"
+  },
+  "features": [
+    "Text Generation", "Conversational", "Agentic Capabilities",
+    "Speculative Decoding", "1M Token Context", "Sovereign (self-hosted)",
+    "SafeTensors"
+  ],
+  "arxiv": "arXiv:2606.19348",
+  "sovereign_guarantees": {
+    "no_mandatory_telemetry": true,
+    "air_gap_capable": true,
+    "local_weights": true,
+    "private_finetunes": true
+  }
+}

tokenizer_config.json

@@ -0,0 +1,11 @@
+{
+  "tokenizer_class": "LlamaTokenizer",
+  "model_max_length": 1048576,
+  "padding_side": "right",
+  "truncation_side": "right",
+  "bos_token": "<｜begin▁of▁sentence｜>",
+  "eos_token": "<｜end▁of▁sentence｜>",
+  "pad_token": "<｜end▁of▁sentence｜>",
+  "unk_token": "<｜unk｜>",
+  "chat_template_note": "Use encoding_dsv4 (see models/deepseek-v4-flash-0731/docs) — no Jinja template ships with this release."
+}