MonumentalSystems/rotor-25m-annealed

RotorNative 25M — annealed mixed-domain checkpoint

An experimental byte-level language-model checkpoint using Clifford rotor dynamics on the unit sphere. This is a research artifact from the Harmonic GPT program, not a supported assistant or a general-purpose production model.

Architecture

• Wire encoding: raw bytes mapped through QPSK + Hamming structure into seven oscillator groups.
• RotorAttention: bivector coupling, Cayley rotors, and sphere-valued dynamics.
• DenseLoheSync: signed mean-field coupling between oscillator groups.
• Readout: oscillator state and cumulative mean-field features mapped to 256 raw byte logits.

Property	Value
Model family	RotorNative25M
Harmonics	40
Layers	20
Hidden width	560
Oscillator groups	7
Byte vocabulary	256
Training sequence length	256 bytes
Saved step	64,545
Saved BPB	0.6137
Optimizer	RotationalAdamW

Sequence mechanism

This model is attention-free: it uses Clifford rotor and Lohe synchronization dynamics rather than softmax attention or an attention KV cache. The checkpoint is an inference-only Safetensors export, but this repository does not currently bundle a standalone transformers implementation. Reproduction requires the matching Harmonic GPT/HarmonicRust RotorNative architecture.

Loading the tensors

import json

from huggingface_hub import hf_hub_download
from safetensors.torch import load_file

weights = load_file(
    hf_hub_download("MonumentalSystems/rotor-25m-annealed", "model.safetensors")
)
with open(
    hf_hub_download("MonumentalSystems/rotor-25m-annealed", "config.json"),
    encoding="utf-8",
) as handle:
    config = json.load(handle)

Limitations

The saved BPB is a within-run selection metric, not a standardized benchmark. The training mixture is described only as mixed-domain in the surviving export metadata; do not infer broader dataset provenance or model capability from that metric. Outputs may be malformed, repetitive, incorrect, or unsafe. Do not use this checkpoint for production or consequential decisions.