---
title: Tiny Transformers
emoji: 🧠
colorFrom: blue
colorTo: purple
sdk: gradio
app_file: app.py
pinned: false
short_description: A set of tiny Transformers, from scratch
---

# Tiny Text Transformers

---
title: Tiny Transformers
emoji: 🧠
colorFrom: blue
colorTo: purple
sdk: gradio
app_file: app.py
pinned: false
short_description: A set of tiny Transformers, from scratch
---

# Tiny Text Transformers

A set of small transformer models, less than 1M params for multiple NLP tasks built on the same Transformer core.

Goal: Light, highly specialized and useful in a specific domain

## Tasks

### Text Generation
Character-level autoregressive language model. Shakespear corpus used for proof of concept.

### Text Classification
News / text classifier. AG News dataset used for proof of concept.

### Structured Output
Generate structured output for a limited instruction-set problem. For instance a cli command.

### Translation
Encoder Decoder block for experimental educational purposes.

### Hugging Faces
https://huggingface.co/spaces/zeguru/tiny-transformers