---
title: tokenizers v1 split
emoji: ✂️
colorFrom: gray
colorTo: gray
sdk: static
pinned: false
license: apache-2.0
---

A figure from [tokenizers v1](https://huggingface.co/blog/tokenizers-v1), split
out so the blog post can embed it. Generated by `make_embeds.py`; edit the post
source, not this Space.