DFM-Mimir-v1.8

This release contains the XL EMA checkpoint at step 3,800,000, with 1,786,775,040 total parameters, exported as BF16 inference weights. This is the XL continuation checkpoint, not the smaller B architecture.

Architecture and training

HRM-Text uses hierarchical recurrent computation and a PrefixLM objective. The configuration records the exact architecture: H=2, L=3, a 262,144-token vocabulary and a 4,096-token default context. The model uses the full Gemma tokenizer with native chat rendering and thinking disabled. Mini does not use the separate 65,536-token experimental Mini tokenizer.

The final training mixture is DFM15 (MTC v1.8). Training combines public sources with agreement-backed DBC/Lex.dk material; public downloads alone are not sufficient to reconstruct that material. Architecture and training sources are documented in HRM-Text and Mimir. The weight license is Apache-2.0; tokenizer origins and applicable upstream terms remain relevant independently of the weight license.

Loading and serving

These weights require an HRM-aware runtime with PrefixLM support. The evaluated environment used Transformers 5.17.0 and vLLM 0.31.0 with HRM-specific support. Version numbers alone do not establish equivalence: do not assume an unmodified installation supports this architecture. See the Mimir repository for the HRM runtime implementation and installation. This repository does not provide an automatic remote-code loader.

After installing a compatible HRM runtime, download this repository to ./DFM-Mimir-v1.8 and run the following evaluation-style server command:

python -m vllm.entrypoints.openai.api_server \
  --model ./DFM-Mimir-v1.8 \
  --tokenizer ./DFM-Mimir-v1.8 \
  --served-model-name danish-foundation-models/DFM-Mimir-v1.8 \
  --chat-template ./DFM-Mimir-v1.8/chat_template.jinja \
  --host 127.0.0.1 --port 8200 \
  --tensor-parallel-size 1 --gpu-memory-utilization 0.95 \
  --attention-backend FLASH_ATTN \
  --max-model-len 8192 --max-num-seqs 64 --max-num-batched-tokens 8192 \
  --no-enable-prefix-caching --no-enable-chunked-prefill --enforce-eager \
  --default-chat-template-kwargs '{"enable_thinking":false}' \
  --hf-overrides '{"max_position_embeddings":8192}'

The explicit override reproduces the 8K evaluation setting; it does not change the exported 4K default. GPU memory requirements depend on context and concurrency. Preserve fix_mistral_regex=false: enabling regex repair changes tokenization relative to training and the reported evaluations.

Evaluation evidence and limitations

evaluations/ contains the standalone EuroEval zero-shot and few-shot submission results. Keep these protocols separate from training-integrated aggregates. model-verification.json and server-smoke.json preserve the local evaluation checks; they are historical checks, not a claim of a new Hub-based GPU test. Local paths in provenance identify the original run and are not dependencies for loading these weights.

The model may produce incorrect or unsupported answers. Benchmark coverage is not a guarantee of general capability across all tasks or languages.

Integrity

checksums.json records SHA-256 digests of release files. Weight bytes are unchanged from the evaluated export. Mini's tokenizer packaging uses the validated native tokenizer and chat template rather than the raw export's regex-repair setting. Optimizer state is not included.

Downloads last month
92
Safetensors
Model size
2B params
Tensor type
BF16
ยท
Inference Providers NEW
This model isn't deployed by any Inference Provider. ๐Ÿ™‹ Ask for provider support