LastKV-LLM-8B-Base

Final model checkpoints for LastKV: Last Layer KV Cache is All You Need.

Training stage	Final step	Configured context	Revision
Continued pretraining, including the learning-rate cooldown	59,904	8,192 tokens	main, also pinned as 8k-step-059904
Subsequent long-context training	1,209	262,144 tokens	256k-step-001209

Each checkpoint contains the raw model weights consolidated from its distributed training checkpoint: 8,030,269,952 parameters in 419 tensors, stored as four BF16 safetensors files (about 16 GB total), with the model configuration and tokenizer. The index combines the four files into one model checkpoint. Optimizer and dataloader states are excluded.

Download

from huggingface_hub import snapshot_download

model_path = snapshot_download(
    "ZefanCai/LastKV-LLM-8B-Base",
    revision="256k-step-001209",  # Use "main" for the 8K pretraining checkpoint.
)

Inference requires the matching CustomRecurrentMarinAKV254ReuseKVForCausalLM implementation, which is not bundled with this weights release. The tokenizer metadata is preserved from the training export; its model_max_length can differ from the model configuration's context length.

License: MIT.