ReFORM C10 Model
ReFORM is a reccurent transformer architecture based on RoPE Transformer that uses Huginn-like tricks mixed with RWKV and other stuff: token shifting, latent thinking vector, vector-mixing, etc.
So, it's ~10M model trained ~10 minutes on RTX 5070 TI.