grt-30m-lora / PROVENANCE.md
davidhouseholter's picture
Release MatterWaveAI/grt-30m-lora with GGUF & SHA256SUMS (T-018 / T-019)
aae99bf verified
|
Raw History Blame Contribute Delete
547 Bytes

Provenance: MatterWaveAI/grt-30m-lora

  • Base Model: MatterWaveAI/grt-30m (release/30m/model.safetensors)
  • Adapter Class: GRTRotationalLoRA (r=8, lora_alpha=16.0, tune_scales=True)
  • Optimizer: HessianPairAdam (Sophia-Torus, lr=1.5e-2, 80 steps)
  • Build Script: release/build_30m_lora.py
  • Before Adaptation: loss 6.172, perplexity 479.1388
  • After Adaptation: loss 4.7464, perplexity 115.1645
  • Merge Equivalence: max_abs_diff = 0.0e+00 (0 nn.Linear, 0 nn.Conv1d)