---
language:
  - it
library_name: transformers
tags:
  - tokenizer
  - italian
  - bpe
  - bert
  - alberturin
---

# gettone

**gettone** is a Byte-Pair Encoding (BPE) tokenizer developed specifically
for Italian and used across the **AlBERTurin** family of encoder models.

It has a vocabulary size of **32,768 tokens** and was designed to provide
compact tokenization of Italian text while maintaining a moderate vocabulary
size.

## Usage

```python
from transformers import AutoTokenizer

tokenizer = AutoTokenizer.from_pretrained("AlBERTurin/gettone")

text = "Questa è una frase di esempio in italiano."

tokens = tokenizer.tokenize(text)
encoded = tokenizer(text)

print(tokens)
print(encoded)
```

## AlBERTurin Model Family

gettone is shared by all models in the AlBERTurin family:

| Model | Parameters | Training Tokens |
| --- | ---: | ---: |
| [AlBERTmini](https://huggingface.co/AlBERTurin/AlBERTmini) | 95M | 7B |
| [AlBERTina](https://huggingface.co/AlBERTurin/AlBERTina) | 140M | 14B |
| [AlBERTone101](https://huggingface.co/AlBERTurin/AlBERTone101) | 450M | ~101B |

## Tokenizer Evaluation

The tokenizer was evaluated using **fertility**, defined as the average
number of subword tokens produced per word. Lower values indicate more
compact tokenization.

On the evaluation corpus used in the AlBERTurin paper, gettone achieved
a fertility score of **1.429**, the lowest among the evaluated Italian and
multilingual tokenizers.

| Tokenizer | Type | Vocabulary Size | Fertility ↓ |
| --- | --- | ---: | ---: |
| **gettone** | BPE | 32,768 | **1.429** |
| Velvet-2B | BPE | 126,976 | 1.471 |
| UmBERTo | BPE | 32,005 | 1.493 |
| dbmdz | WordPiece | 31,102 | 1.497 |
| Minerva-7B | BPE | 51,200 | 1.560 |
| Minerva-350M | BPE | 32,768 | 1.644 |
| XLM-R | Unigram | 250,002 | 1.663 |
| mBERT | WordPiece | 119,547 | 1.719 |
| EuroBERT | BPE | 128,000 | 1.934 |


## Citation

If you use gettone in your research, please cite:

> Matteo Rinaldi, Marco Madeddu, Calogero Jerik Scozzaro,
> Matteo Delsanto, Daniele Paolo Radicioni, and Viviana Patti.  
> **AlBERTurin: A Fully Open Family of Italian Encoder Models with
> Modern Architectures.**  
> CLiC-it 2026.