GDN-1.3B-100B

Final native GDN/HLA repository checkpoint, trained from scratch on approximately 100B SlimPajama tokens (100,000,595,968 tokens), with 4,096-token sequences and global batch size 512. Completed optimizer step: 47,684.

Variant: GatedDeltaNet_Release_1.3B. Standard Gated DeltaNet baseline.

Files

  • final-model-ckpt.pth: original FP32 model checkpoint, unchanged; contains model weights, training hyperparameters and counters. Optimizer is None.
  • tokenizer.model: SentencePiece tokenizer used for training/evaluation.
  • model_config.json: configuration for this repository's lit_gpt.Config.
  • training_metadata.json: provenance, training metadata and SHA-256 checksum.
  • load_checkpoint.py: native model loading helper.

Loading

Use GDN/HLA source at revision 2c4d6d7d458b7d3a21041bbbb6954e204a43b9f8 and its training dependencies. This is a custom native Lit-GPT checkpoint, not a Transformers AutoModel export. CUDA/Triton dependencies are required for model execution. Download this repository with huggingface_hub.snapshot_download, then call load_checkpoint.load_model(downloaded_directory, device="cuda") from the GDN/HLA environment. The loader strictly validates all weight keys. MHLA cached autoregressive evaluation uses scripts/mhla_affine_eval.py; see scripts/eval_longbench_v2_local.py for full evaluation setup. No optimizer state is supplied for exact training resumption.

Checkpoint SHA-256: 89005ab816851bb1ebfa9ec6b83446ecee999eaa85a6fdbd49cf392553d321d1.

Downloads last month

-

Downloads are not tracked for this model. How to track
Inference Providers NEW
This model isn't deployed by any Inference Provider. ๐Ÿ™‹ Ask for provider support