Instructions to use EvoLenTokenizer/base-200k with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use EvoLenTokenizer/base-200k with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("feature-extraction", model="EvoLenTokenizer/base-200k")# pip install -U transformers accelerate # Load model directly from transformers import AutoTokenizer, AutoModelForMaskedLM tokenizer = AutoTokenizer.from_pretrained("EvoLenTokenizer/base-200k") model = AutoModelForMaskedLM.from_pretrained("EvoLenTokenizer/base-200k", device_map="auto") - Notebooks
- Google Colab
- Kaggle
Improve model card with metadata and links
#1
by nielsr HF Staff - opened
README.md
CHANGED
|
@@ -1,34 +1,34 @@
|
|
| 1 |
---
|
| 2 |
-
tags:
|
| 3 |
-
- generated_from_trainer
|
| 4 |
metrics:
|
| 5 |
- accuracy
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 6 |
model-index:
|
| 7 |
- name: base_5120
|
| 8 |
results: []
|
| 9 |
---
|
| 10 |
|
| 11 |
-
<!-- This model card has been generated automatically according to the information the Trainer had access to. You
|
| 12 |
-
should probably proofread and complete it, then remove this comment. -->
|
| 13 |
-
|
| 14 |
# base_5120
|
| 15 |
|
| 16 |
-
This model is a
|
| 17 |
-
|
| 18 |
-
-
|
| 19 |
-
-
|
| 20 |
|
| 21 |
## Model description
|
| 22 |
|
| 23 |
-
|
| 24 |
|
| 25 |
## Intended uses & limitations
|
| 26 |
|
| 27 |
-
|
| 28 |
|
| 29 |
## Training and evaluation data
|
| 30 |
|
| 31 |
-
|
| 32 |
|
| 33 |
## Training procedure
|
| 34 |
|
|
@@ -46,11 +46,13 @@ The following hyperparameters were used during training:
|
|
| 46 |
|
| 47 |
### Training results
|
| 48 |
|
|
|
|
|
|
|
|
|
|
| 49 |
|
| 50 |
-
|
| 51 |
-
### Framework versions
|
| 52 |
|
| 53 |
- Transformers 4.35.2
|
| 54 |
- Pytorch 2.8.0+cu128
|
| 55 |
- Datasets 4.4.1
|
| 56 |
-
- Tokenizers 0.15.2
|
|
|
|
| 1 |
---
|
|
|
|
|
|
|
| 2 |
metrics:
|
| 3 |
- accuracy
|
| 4 |
+
tags:
|
| 5 |
+
- generated_from_trainer
|
| 6 |
+
license: mit
|
| 7 |
+
library_name: transformers
|
| 8 |
+
pipeline_tag: feature-extraction
|
| 9 |
model-index:
|
| 10 |
- name: base_5120
|
| 11 |
results: []
|
| 12 |
---
|
| 13 |
|
|
|
|
|
|
|
|
|
|
| 14 |
# base_5120
|
| 15 |
|
| 16 |
+
This model is a BERT-based DNA language model trained with the standard byte-pair encoding (BPE) tokenizer with vocabulary size 5,120. It is the baseline control model from the paper [EvoLen: Evolution-Guided Tokenization for DNA Language Model](https://huggingface.co/papers/2604.08698) (COLM 2026).
|
| 17 |
+
|
| 18 |
+
- **Paper**: [EvoLen: Evolution-Guided Tokenization for DNA Language Model](https://huggingface.co/papers/2604.08698)
|
| 19 |
+
- **Code**: [https://github.com/HN020719/EvoLen](https://github.com/HN020719/EvoLen)
|
| 20 |
|
| 21 |
## Model description
|
| 22 |
|
| 23 |
+
This model uses a BERT-base architecture (12 layers, hidden size 768) with a vocabulary of 5,120 tokens. It was pretrained on human genome sequences using masked language modeling. The tokenizer is the standard BPE baseline described in the EvoLen repository.
|
| 24 |
|
| 25 |
## Intended uses & limitations
|
| 26 |
|
| 27 |
+
This model is primarily intended as a baseline for evaluating the EvoLen tokenizer in DNA language model benchmarks. It can be fine-tuned for downstream genomic tasks, but it is not specifically designed for production use.
|
| 28 |
|
| 29 |
## Training and evaluation data
|
| 30 |
|
| 31 |
+
The model was pretrained on the human genome (hg38). Training sequences were tokenized into 512-token windows and a held-out evaluation set was used to report the following results.
|
| 32 |
|
| 33 |
## Training procedure
|
| 34 |
|
|
|
|
| 46 |
|
| 47 |
### Training results
|
| 48 |
|
| 49 |
+
- Loss: 5.0263
|
| 50 |
+
- Accuracy: 0.2365
|
| 51 |
+
- Perplexity: 152.36
|
| 52 |
|
| 53 |
+
## Framework versions
|
|
|
|
| 54 |
|
| 55 |
- Transformers 4.35.2
|
| 56 |
- Pytorch 2.8.0+cu128
|
| 57 |
- Datasets 4.4.1
|
| 58 |
+
- Tokenizers 0.15.2
|