Text Generation
English
ReFORM-C10 / README.md
Kotodobr's picture
Update README.md
325866a verified
|
Raw History Blame Contribute Delete
377 Bytes
metadata
license: apache-2.0
datasets:
  - starhopp3r/TinyChat
language:
  - en
pipeline_tag: text-generation

ReFORM C10 Model

ReFORM is a reccurent transformer architecture based on RoPE Transformer that uses Huginn-like tricks mixed with RWKV and other stuff: token shifting, latent thinking vector, vector-mixing, etc.

So, it's ~10M model trained ~10 minutes on RTX 5070 TI.