README.md

metadata

title: Tiny Transformers
emoji: 🧠
colorFrom: blue
colorTo: purple
sdk: gradio
app_file: app.py
pinned: false
short_description: A set of tiny Transformers, from scratch

Tiny Text Transformers

---

title: Tiny Transformers emoji: 🧠 colorFrom: blue colorTo: purple sdk: gradio app_file: app.py pinned: false short_description: A set of tiny Transformers, from scratch

Tiny Text Transformers

A set of small transformer models, less than 1M params for multiple NLP tasks built on the same Transformer core.

Goal: Light, highly specialized and useful in a specific domain

Tasks

Text Generation

Character-level autoregressive language model. Shakespear corpus used for proof of concept.

Text Classification

News / text classifier. AG News dataset used for proof of concept.

Structured Output

Generate structured output for a limited instruction-set problem. For instance a cli command.

Translation

Encoder Decoder block for experimental educational purposes.

Hugging Faces

https://huggingface.co/spaces/zeguru/tiny-transformers