grt-30m-lora / PROVENANCE.md
davidhouseholter's picture
Release MatterWaveAI/grt-30m-lora with GGUF & SHA256SUMS (T-018 / T-019)
aae99bf verified
|
Raw History Blame Contribute Delete
547 Bytes
# Provenance: `MatterWaveAI/grt-30m-lora`
- **Base Model**: `MatterWaveAI/grt-30m` (`release/30m/model.safetensors`)
- **Adapter Class**: `GRTRotationalLoRA` (`r=8`, `lora_alpha=16.0`, `tune_scales=True`)
- **Optimizer**: `HessianPairAdam` (`Sophia-Torus`, `lr=1.5e-2`, `80` steps)
- **Build Script**: `release/build_30m_lora.py`
- **Before Adaptation**: loss `6.172`, perplexity `479.1388`
- **After Adaptation**: loss `4.7464`, perplexity `115.1645`
- **Merge Equivalence**: `max_abs_diff = 0.0e+00` (`0` `nn.Linear`, `0` `nn.Conv1d`)