Hugging Face's logo

nabin2004
/
nebium

PyTorch
GGUF
chess
causal-lm
nebium
Model card Files
xet
 Community

Nebium

Causal Transformer for self-supervised next-move prediction on chess games.

Architecture

  • d_model: 512
  • n_heads: 8
  • n_layers: 6
  • vocab_size: 2018
  • max_seq_len: 512
  • positional_encoding: rope
  • activation: swiglu
  • norm: rmsnorm

Artifacts & Formats

  • model.pt: Raw PyTorch state_dict
  • model_config.json: Architecture configuration
  • tokenizer.json: BPE Chess tokenizer
  • nebium.gguf: GGUF format model for local inference and quantization

Last validation metrics

  • val/loss: 2.376061725547775
  • val/accuracy: 0.36386593746665474
  • val/top5_accuracy: 0.705297920239525
  • val/perplexity: 10.762433873185277
  • val/legal_move_rate: 0.9833333333333333
Downloads last month
9
GGUF
Model size
21M params
Architecture
nebium
Hardware compatibility
Log In to add your hardware

We're not able to determine the quantization variants.

View all variants
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Collection including nabin2004/nebium