bhaav-e / README.md

Acknowledge Udgam Labs

b740f6a verified about 23 hours ago

7.09 kB

---
license: mit
language:
  - en
library_name: pytorch
pipeline_tag: text-to-video
tags:
  - text-to-video
  - video-generation
  - diffusion-transformer
  - rectified-flow
  - stickers
  - animation
---
<div align="center">

# Bhaav-E

**A compact text-to-video model for animated stickers**

[![Project Page](https://img.shields.io/badge/Project-Page-e8590c)](https://huggingface.co/spaces/Mohit5899/bhaav-e)
[![GitHub](https://img.shields.io/badge/GitHub-Code-181717?logo=github)](https://github.com/Mohit-5899/bhaav-e)
[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/Mohit-5899/bhaav-e/blob/main/bhaav_e_demo.ipynb)
[![License: MIT](https://img.shields.io/badge/License-MIT-blue.svg)](https://github.com/Mohit-5899/bhaav-e/blob/main/LICENSE)

</div>

| | | | |
|:---:|:---:|:---:|:---:|
| ![](examples/00_seed0.gif) | ![](examples/01_seed0.gif) | ![](examples/02_seed0.gif) | ![](examples/03_seed0.gif) |
| *a crying emoji bouncing, tears falling, light blue background* | *a red furious emoji pulsing, steam puffing from its head, dark navy background* | *a blue success checkmark drawing itself inside a circle, confetti bursting out, white background* | *a party popper emoji pulsing, sparkles twinkling around it, light grey background* |
| ![](examples/04_seed0.gif) | ![](examples/05_seed0.gif) | ![](examples/06_seed0.gif) | ![](examples/07_seed0.gif) |
| *a cool emoji with sunglasses floating gently, sparkles twinkling around it, peach background* | *a glowing star emoji wobbling side to side, sparkles twinkling around it, dark navy background* | *a purple loading spinner spinning round and round, pulsing, black background* | *a screaming in shock emoji wobbling side to side, peach background* |

<sub>Uncurated samples (seed 0, 50 steps, guidance 3.0): the first and only sample generated for each prompt.</sub>

## Model Details

Bhaav-E generates 2-second looping stickers from a text prompt. It is a 211.92M-parameter video diffusion
transformer trained from scratch with rectified flow, operating in the latent space of the frozen Wan2.2 video VAE and
conditioned on a frozen Flan-T5-Base text encoder.

- **Developed by:** Mohit
- **Model type:** latent video diffusion transformer (rectified flow)
- **Output:** 17 frames, 256 × 256, 8 fps, looping
- **Language:** English prompts
- **License:** MIT (frozen components: Apache-2.0)
- **Code:** [github.com/Mohit-5899/bhaav-e](https://github.com/Mohit-5899/bhaav-e)

| Component | Specification |
|---|---|
| Denoiser (trained) | 16 transformer blocks, width 768, 12 heads, 2 × 2 patches (320 tokens) |
| Text encoder (frozen) | [google/flan-t5-base](https://huggingface.co/google/flan-t5-base), max 35 tokens |
| Autoencoder (frozen) | [Wan2.2 VAE](https://huggingface.co/Wan-AI/Wan2.2-TI2V-5B-Diffusers), 48 latent channels |
| Sampler | 50 Euler steps, classifier-free guidance 3.0, time shift 5.0 |

## Files

| File | Description |
|---|---|
| `bhaav_e.safetensors` | Bhaav-E (recommended): EMA weights, step 40,000, fp32 |
| `config.json` | model size and sampler defaults |
| `text/info.json`, `latents/vae_config.json` | text-encoder settings and VAE latent statistics used at inference |
| `r1/` | baseline checkpoint trained on the natural attribute distribution, same format |
| `code/` | model definition (`videodit.py`) and sampler (`sample.py`) |

## Uses

**Intended use:** generating short animated stickers, reaction emoji and UI micro-animations, and research on small
video diffusion models, compositional generalization and evaluation with procedurally generated data.

**Out of scope:** subjects outside the training vocabulary, photorealistic video, high resolution or long clips.

## How to Use

**No installation:** open the [Colab notebook](https://colab.research.google.com/github/Mohit-5899/bhaav-e/blob/main/bhaav_e_demo.ipynb),
select **Runtime → Run all**, and open the `gradio.live` link it prints. This runs the full web demo on a free GPU.

**Python** (CPU or GPU):

```bash
git clone https://github.com/Mohit-5899/bhaav-e && cd bhaav-e
pip install torch diffusers transformers sentencepiece safetensors pillow numpy huggingface_hub
```

```python
import sys; sys.path.insert(0, "space")
from bhaav import Bhaav, save_gif
pipe = Bhaav()                       # downloads this checkpoint, the VAE and the text encoder
frames = pipe.generate("a crying emoji bouncing, tears falling, light blue background",
                       steps=50, cfg=3.0, seed=0)
save_gif(frames, "sticker.gif")
```

**Prompt format:** *subject, motion, effect, background*. The model supports 30 subjects (10 emotion faces, 10
animated emoji, 10 UI animations), 9 motions, 12 effects and 12 background colours; the full list is in the
[GitHub README](https://github.com/Mohit-5899/bhaav-e#prompt-format).

## Training

| | |
|---|---|
| Data | 85,816 procedurally rendered clips; 61,571 used for training, with effects balanced per subject |
| Objective | rectified flow, MSE on predicted velocity |
| Optimizer | Muon (lr 0.02) for transformer weight matrices, AdamW (lr 2e-4) for other parameters |
| Batch size × steps | 64 × 40,000 |
| Precision | bf16 autocast |
| Compute | 1 × NVIDIA RTX PRO 4500 (32 GB), ≈ 3.6 hours |

## Evaluation

256 prompts with attribute combinations seen in training and 256 with combinations held out from training. Automatic
checkers read subject, motion, effect, background and accent from each generated clip; errors are counted only where
the checker reads the corresponding real clip correctly.

| Model | Seen combinations | Unseen combinations |
|---|---|---|
| **Bhaav-E** | **2.0%** | **3.3%** |
| Bhaav-E r1 (baseline) | 2.0% | 5.3% |

Error rate on any attribute, guidance 3.0. The dominant failure mode is effect leakage: adding the effect most
associated with a subject when none is requested. The unseen difference between the models is not statistically
significant (exact sign test, p = 0.23; one training seed each). Full report:
[runs/r2/RESULTS.md](https://github.com/Mohit-5899/bhaav-e/blob/main/runs/r2/RESULTS.md).

## Limitations

- Closed vocabulary: only the 30 trained subjects, 12 effects and 12 backgrounds.
- 256 × 256 resolution and 2-second duration.
- Unrequested effects are occasionally added.
- Evaluated with one seed per model and about 250 prompts per split.

## Citation

```bibtex
@misc{bhaave2026,
  title        = {Bhaav-E: A Compact Text-to-Video Model for Animated Stickers},
  author       = {Mohit},
  year         = {2026},
  howpublished = {\url{https://github.com/Mohit-5899/bhaav-e}}
}
```

## Acknowledgements

Built on [Wan2.2](https://github.com/Wan-Video/Wan2.2), [Flan-T5](https://huggingface.co/google/flan-t5-base) and
[Microsoft Fluent Emoji](https://github.com/microsoft/fluentui-emoji) (MIT, training artwork).

Special thanks to [Udgam Labs](https://udgamlabs.com) for openly publishing the code, training logs and cost reports of their small models; their published recipe and model size served as the reference for this work.

	---
	license: mit
	language:
	- en
	library_name: pytorch
	pipeline_tag: text-to-video
	tags:
	- text-to-video
	- video-generation
	- diffusion-transformer
	- rectified-flow
	- stickers
	- animation
	---

	<div align="center">

	# Bhaav-E

	A compact text-to-video model for animated stickers

	[![Project Page](https://img.shields.io/badge/Project-Page-e8590c)](https://huggingface.co/spaces/Mohit5899/bhaav-e)
	[![GitHub](https://img.shields.io/badge/GitHub-Code-181717?logo=github)](https://github.com/Mohit-5899/bhaav-e)
	[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/Mohit-5899/bhaav-e/blob/main/bhaav_e_demo.ipynb)
	[![License: MIT](https://img.shields.io/badge/License-MIT-blue.svg)](https://github.com/Mohit-5899/bhaav-e/blob/main/LICENSE)

	</div>

	\| \| \| \| \|
	\|:---:\|:---:\|:---:\|:---:\|
	\| ![](examples/00_seed0.gif) \| ![](examples/01_seed0.gif) \| ![](examples/02_seed0.gif) \| ![](examples/03_seed0.gif) \|
	\| a crying emoji bouncing, tears falling, light blue background \| a red furious emoji pulsing, steam puffing from its head, dark navy background \| a blue success checkmark drawing itself inside a circle, confetti bursting out, white background \| a party popper emoji pulsing, sparkles twinkling around it, light grey background \|
	\| ![](examples/04_seed0.gif) \| ![](examples/05_seed0.gif) \| ![](examples/06_seed0.gif) \| ![](examples/07_seed0.gif) \|
	\| a cool emoji with sunglasses floating gently, sparkles twinkling around it, peach background \| a glowing star emoji wobbling side to side, sparkles twinkling around it, dark navy background \| a purple loading spinner spinning round and round, pulsing, black background \| a screaming in shock emoji wobbling side to side, peach background \|

	<sub>Uncurated samples (seed 0, 50 steps, guidance 3.0): the first and only sample generated for each prompt.</sub>

	## Model Details

	Bhaav-E generates 2-second looping stickers from a text prompt. It is a 211.92M-parameter video diffusion
	transformer trained from scratch with rectified flow, operating in the latent space of the frozen Wan2.2 video VAE and
	conditioned on a frozen Flan-T5-Base text encoder.

	- Developed by: Mohit
	- Model type: latent video diffusion transformer (rectified flow)
	- Output: 17 frames, 256 × 256, 8 fps, looping
	- Language: English prompts
	- License: MIT (frozen components: Apache-2.0)
	- Code: [github.com/Mohit-5899/bhaav-e](https://github.com/Mohit-5899/bhaav-e)

	\| Component \| Specification \|
	\|---\|---\|
	\| Denoiser (trained) \| 16 transformer blocks, width 768, 12 heads, 2 × 2 patches (320 tokens) \|
	\| Text encoder (frozen) \| [google/flan-t5-base](https://huggingface.co/google/flan-t5-base), max 35 tokens \|
	\| Autoencoder (frozen) \| [Wan2.2 VAE](https://huggingface.co/Wan-AI/Wan2.2-TI2V-5B-Diffusers), 48 latent channels \|
	\| Sampler \| 50 Euler steps, classifier-free guidance 3.0, time shift 5.0 \|

	## Files

	\| File \| Description \|
	\|---\|---\|
	\| `bhaav_e.safetensors` \| Bhaav-E (recommended): EMA weights, step 40,000, fp32 \|
	\| `config.json` \| model size and sampler defaults \|
	\| `text/info.json`, `latents/vae_config.json` \| text-encoder settings and VAE latent statistics used at inference \|
	\| `r1/` \| baseline checkpoint trained on the natural attribute distribution, same format \|
	\| `code/` \| model definition (`videodit.py`) and sampler (`sample.py`) \|

	## Uses

	Intended use: generating short animated stickers, reaction emoji and UI micro-animations, and research on small
	video diffusion models, compositional generalization and evaluation with procedurally generated data.

	Out of scope: subjects outside the training vocabulary, photorealistic video, high resolution or long clips.

	## How to Use

	No installation: open the [Colab notebook](https://colab.research.google.com/github/Mohit-5899/bhaav-e/blob/main/bhaav_e_demo.ipynb),
	select Runtime → Run all, and open the `gradio.live` link it prints. This runs the full web demo on a free GPU.

	Python (CPU or GPU):

	```bash
	git clone https://github.com/Mohit-5899/bhaav-e && cd bhaav-e
	pip install torch diffusers transformers sentencepiece safetensors pillow numpy huggingface_hub
	```

	```python
	import sys; sys.path.insert(0, "space")
	from bhaav import Bhaav, save_gif

	pipe = Bhaav() # downloads this checkpoint, the VAE and the text encoder
	frames = pipe.generate("a crying emoji bouncing, tears falling, light blue background",
	steps=50, cfg=3.0, seed=0)
	save_gif(frames, "sticker.gif")
	```

	Prompt format: subject, motion, effect, background. The model supports 30 subjects (10 emotion faces, 10
	animated emoji, 10 UI animations), 9 motions, 12 effects and 12 background colours; the full list is in the
	[GitHub README](https://github.com/Mohit-5899/bhaav-e#prompt-format).

	## Training

	\| \| \|
	\|---\|---\|
	\| Data \| 85,816 procedurally rendered clips; 61,571 used for training, with effects balanced per subject \|
	\| Objective \| rectified flow, MSE on predicted velocity \|
	\| Optimizer \| Muon (lr 0.02) for transformer weight matrices, AdamW (lr 2e-4) for other parameters \|
	\| Batch size × steps \| 64 × 40,000 \|
	\| Precision \| bf16 autocast \|
	\| Compute \| 1 × NVIDIA RTX PRO 4500 (32 GB), ≈ 3.6 hours \|

	## Evaluation

	256 prompts with attribute combinations seen in training and 256 with combinations held out from training. Automatic
	checkers read subject, motion, effect, background and accent from each generated clip; errors are counted only where
	the checker reads the corresponding real clip correctly.

	\| Model \| Seen combinations \| Unseen combinations \|
	\|---\|---\|---\|
	\| Bhaav-E \| 2.0% \| 3.3% \|
	\| Bhaav-E r1 (baseline) \| 2.0% \| 5.3% \|

	Error rate on any attribute, guidance 3.0. The dominant failure mode is effect leakage: adding the effect most
	associated with a subject when none is requested. The unseen difference between the models is not statistically
	significant (exact sign test, p = 0.23; one training seed each). Full report:
	[runs/r2/RESULTS.md](https://github.com/Mohit-5899/bhaav-e/blob/main/runs/r2/RESULTS.md).

	## Limitations

	- Closed vocabulary: only the 30 trained subjects, 12 effects and 12 backgrounds.
	- 256 × 256 resolution and 2-second duration.
	- Unrequested effects are occasionally added.
	- Evaluated with one seed per model and about 250 prompts per split.

	## Citation

	```bibtex
	@misc{bhaave2026,
	title = {Bhaav-E: A Compact Text-to-Video Model for Animated Stickers},
	author = {Mohit},
	year = {2026},
	howpublished = {\url{https://github.com/Mohit-5899/bhaav-e}}
	}
	```

	## Acknowledgements

	Built on [Wan2.2](https://github.com/Wan-Video/Wan2.2), [Flan-T5](https://huggingface.co/google/flan-t5-base) and
	[Microsoft Fluent Emoji](https://github.com/microsoft/fluentui-emoji) (MIT, training artwork).

	Special thanks to [Udgam Labs](https://udgamlabs.com) for openly publishing the code, training logs and cost reports of their small models; their published recipe and model size served as the reference for this work.