Instructions to use metaXu264/Generator_new_tokenizer with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use metaXu264/Generator_new_tokenizer with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="metaXu264/Generator_new_tokenizer")

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("metaXu264/Generator_new_tokenizer")
model = AutoModelForCausalLM.from_pretrained("metaXu264/Generator_new_tokenizer", device_map="auto")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use metaXu264/Generator_new_tokenizer with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "metaXu264/Generator_new_tokenizer"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "metaXu264/Generator_new_tokenizer",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/metaXu264/Generator_new_tokenizer

SGLang

How to use metaXu264/Generator_new_tokenizer with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "metaXu264/Generator_new_tokenizer" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "metaXu264/Generator_new_tokenizer",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "metaXu264/Generator_new_tokenizer" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "metaXu264/Generator_new_tokenizer",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Docker Model Runner
How to use metaXu264/Generator_new_tokenizer with Docker Model Runner:
```
docker model run hf.co/metaXu264/Generator_new_tokenizer
```

Generator_new_tokenizer / README.md

XuJP264

Add updated vocab with CRISPR/Cas special tokens

4357886 9 months ago

preview code

Raw History Blame Contribute Delete

6.44 kB

---
license: mit
pipeline_tag: text-generation
tags:
- biology
- genomics
- long-context
library_name: transformers
---
# GENERator-v2-prokaryote-3b-base model

## **Important Notice**
If you are using **GENERator** for sequence generation, please ensure that the length of each input sequence is a multiple of **6**. This can be achieved by either:  
1. Padding the sequence on the left with `'A'` (**left padding**);  
2. Truncating the sequence from the left (**left truncation**).  

This requirement arises because **GENERator** employs a 6-mer tokenizer. If the input sequence length is not a multiple of **6**, the tokenizer will append an `'<oov>'` (out-of-vocabulary) token to the end of the token sequence. This can result in uninformative subsequent generations, such as repeated `'AAAAAA'`.  

We apologize for any inconvenience this may cause and recommend adhering to the above guidelines to ensure accurate and meaningful generation results.

## Abouts
In this repository, we present GENERator-v2, a generative genomic foundation with enhanced performance in prokaryotic domain. More technical details are provided in the GENERator-v2 [technical report](https://www.biorxiv.org/content/10.64898/2026.01.27.702015v1).

Python scripts for downstream analysis are available on Github: [https://github.com/GenerTeam/GENERator](https://github.com/GenerTeam/GENERator).

## How to use

### Simple example1:  generation

```python
import torch
from transformers import AutoTokenizer, AutoModelForCausalLM
# Load the tokenizer and model.
tokenizer = AutoTokenizer.from_pretrained("GenerTeam/GENERator-v2-prokaryote-3b-base", trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained("GenerTeam/GENERator-v2-prokaryote-3b-base")
config = model.config
max_length = config.max_position_embeddings
# Define input sequences.
sequences = [
    "ATGAGGTGGCAAGAAATGGGCTAC",
    "GAATTCCATGAGGCTATAGAATAATCTAAGAGAAAT"
]
def left_padding(sequence, padding_char='A', multiple=6):
    remainder = len(sequence) % multiple
    if remainder != 0:
        padding_length = multiple - remainder
        return padding_char * padding_length + sequence
    return sequence
def left_truncation(sequence, multiple=6):
    remainder = len(sequence) % multiple
    if remainder != 0:
        return sequence[remainder:]
    return sequence
# Apply left_padding to all sequences
# padded_sequences = [left_padding(seq) for seq in sequences]
# Apply left_truncation to all sequences
truncated_sequences = [left_truncation(seq) for seq in sequences]
# Process the sequences
sequences = [tokenizer.bos_token + sequence for sequence in truncated_sequences]
# Tokenize the sequences
tokenizer.padding_side = "left"
inputs = tokenizer(
    sequences,
    add_special_tokens=False,
    return_tensors="pt",
    padding=True,
    truncation=True,
    max_length=max_length
)
# Generate the sequences
with torch.inference_mode():
    outputs = model.generate(**inputs, max_new_tokens=32, temperature=0.00001, top_k=1)
# Decode the generated sequences
decoded_sequences = tokenizer.batch_decode(outputs, skip_special_tokens=True)
# Print the decoded sequences
print(decoded_sequences)
# It is expected to observe non-sense decoded sequences (e.g., 'AAAAAA')
# The input sequences are too short to provide sufficient context.
```

### Simple example2: embedding

```python
import torch
from transformers import AutoTokenizer, AutoModelForCausalLM
# Load the tokenizer and model
tokenizer = AutoTokenizer.from_pretrained("GENERator-v2-prokaryote-3b-base", trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained("GENERator-v2-prokaryote-3b-base")
# Get model configuration
config = model.config
max_length = config.max_position_embeddings
# Define input sequences
sequences = [
    "ATGAGGTGGCAAGAAATGGGCTAC",
    "GAATTCCATGAGGCTATAGAATAATCTAAGAGAAAT"
]
# Truncate each sequence to the nearest multiple of 6
processed_sequences = [tokenizer.bos_token + seq[:len(seq)//6*6] for seq in sequences]
# Tokenization
tokenizer.padding_side = "right"
inputs = tokenizer(
    processed_sequences,
    add_special_tokens=True,
    return_tensors="pt",
    padding=True,
    truncation=True,
    max_length=max_length
)
# Model Inference
with torch.inference_mode():
    outputs = model(**inputs, output_hidden_states=True)
hidden_states = outputs.hidden_states[-1]
attention_mask = inputs["attention_mask"]
# Option 1: Last token (EOS) embedding
last_token_indices = attention_mask.sum(dim=1) - 1
eos_embeddings = hidden_states[torch.arange(hidden_states.size(0)), last_token_indices, :]
# Option 2: Mean pooling over all tokens
expanded_mask = attention_mask.unsqueeze(-1).expand(hidden_states.size()).to(torch.float32)
sum_embeddings = torch.sum(hidden_states * expanded_mask, dim=1)
mean_embeddings = sum_embeddings / expanded_mask.sum(dim=1)
# Output
print("EOS (Last Token) Embeddings:", eos_embeddings)
print("Mean Pooling Embeddings:", mean_embeddings)
# ============================================================================
# Additional notes:
# - The preprocessing step ensures sequences are multiples of 6 for 6-mer tokenizer
# - For causal LM, the last token embedding (EOS) is commonly used
# - Mean pooling considers all tokens including BOS and content tokens
# - The choice depends on your downstream task requirements
# - Both methods handle variable sequence lengths via attention mask
# ============================================================================
```

## Citation
```
@article {li2026generator2,
	author = {Li, Qiuyi and Zhan, Zhihao and Feng, Shikun and Zhu, Yiheng and He, Yuan and Wu, Wei and Shi, Zhenghang and Wang, Shengjie and Hu, Zongyong and Yang, Zhao and Li, Jiaoyang and Tang, Jian and Liu, Haiguang and Qin, Tao},
	title = {Functional In-Context Learning in Genomic Language Models with Nucleotide-Level Supervision and Genome Compression},
	elocation-id = {2026.01.27.702015},
	year = {2026},
	doi = {10.64898/2026.01.27.702015},
	publisher = {Cold Spring Harbor Laboratory},
	URL = {https://www.biorxiv.org/content/early/2026/01/29/2026.01.27.702015},
	journal = {bioRxiv}
}
@article{wu2025generator,
  title={GENERator: a long-context generative genomic foundation model},
  author={Wu, Wei and Li, Qiuyi and Li, Mingyang and Fu, Kun and Feng, Fuli and Ye, Jieping and Xiong, Hui and Wang, Zheng},
  journal={arXiv preprint arXiv:2502.07272},
  year={2025}
}
```

	---
	license: mit
	pipeline_tag: text-generation
	tags:
	- biology
	- genomics
	- long-context
	library_name: transformers
	---
	# GENERator-v2-prokaryote-3b-base model

	## Important Notice
	If you are using GENERator for sequence generation, please ensure that the length of each input sequence is a multiple of 6. This can be achieved by either:
	1. Padding the sequence on the left with `'A'` (left padding);
	2. Truncating the sequence from the left (left truncation).

	This requirement arises because GENERator employs a 6-mer tokenizer. If the input sequence length is not a multiple of 6, the tokenizer will append an `'<oov>'` (out-of-vocabulary) token to the end of the token sequence. This can result in uninformative subsequent generations, such as repeated `'AAAAAA'`.

	We apologize for any inconvenience this may cause and recommend adhering to the above guidelines to ensure accurate and meaningful generation results.

	## Abouts
	In this repository, we present GENERator-v2, a generative genomic foundation with enhanced performance in prokaryotic domain. More technical details are provided in the GENERator-v2 [technical report](https://www.biorxiv.org/content/10.64898/2026.01.27.702015v1).

	Python scripts for downstream analysis are available on Github: [https://github.com/GenerTeam/GENERator](https://github.com/GenerTeam/GENERator).

	## How to use

	### Simple example1: generation

	```python

	import torch
	from transformers import AutoTokenizer, AutoModelForCausalLM

	# Load the tokenizer and model.
	tokenizer = AutoTokenizer.from_pretrained("GenerTeam/GENERator-v2-prokaryote-3b-base", trust_remote_code=True)
	model = AutoModelForCausalLM.from_pretrained("GenerTeam/GENERator-v2-prokaryote-3b-base")
	config = model.config

	max_length = config.max_position_embeddings

	# Define input sequences.
	sequences = [
	"ATGAGGTGGCAAGAAATGGGCTAC",
	"GAATTCCATGAGGCTATAGAATAATCTAAGAGAAAT"
	]

	def left_padding(sequence, padding_char='A', multiple=6):
	remainder = len(sequence) % multiple
	if remainder != 0:
	padding_length = multiple - remainder
	return padding_char * padding_length + sequence
	return sequence

	def left_truncation(sequence, multiple=6):
	remainder = len(sequence) % multiple
	if remainder != 0:
	return sequence[remainder:]
	return sequence

	# Apply left_padding to all sequences
	# padded_sequences = [left_padding(seq) for seq in sequences]

	# Apply left_truncation to all sequences
	truncated_sequences = [left_truncation(seq) for seq in sequences]

	# Process the sequences
	sequences = [tokenizer.bos_token + sequence for sequence in truncated_sequences]

	# Tokenize the sequences
	tokenizer.padding_side = "left"
	inputs = tokenizer(
	sequences,
	add_special_tokens=False,
	return_tensors="pt",
	padding=True,
	truncation=True,
	max_length=max_length
	)

	# Generate the sequences
	with torch.inference_mode():
	outputs = model.generate(**inputs, max_new_tokens=32, temperature=0.00001, top_k=1)

	# Decode the generated sequences
	decoded_sequences = tokenizer.batch_decode(outputs, skip_special_tokens=True)

	# Print the decoded sequences
	print(decoded_sequences)

	# It is expected to observe non-sense decoded sequences (e.g., 'AAAAAA')
	# The input sequences are too short to provide sufficient context.
	```

	### Simple example2: embedding

	```python

	import torch
	from transformers import AutoTokenizer, AutoModelForCausalLM

	# Load the tokenizer and model
	tokenizer = AutoTokenizer.from_pretrained("GENERator-v2-prokaryote-3b-base", trust_remote_code=True)
	model = AutoModelForCausalLM.from_pretrained("GENERator-v2-prokaryote-3b-base")

	# Get model configuration
	config = model.config
	max_length = config.max_position_embeddings

	# Define input sequences
	sequences = [
	"ATGAGGTGGCAAGAAATGGGCTAC",
	"GAATTCCATGAGGCTATAGAATAATCTAAGAGAAAT"
	]

	# Truncate each sequence to the nearest multiple of 6
	processed_sequences = [tokenizer.bos_token + seq[:len(seq)//6*6] for seq in sequences]

	# Tokenization
	tokenizer.padding_side = "right"
	inputs = tokenizer(
	processed_sequences,
	add_special_tokens=True,
	return_tensors="pt",
	padding=True,
	truncation=True,
	max_length=max_length
	)

	# Model Inference
	with torch.inference_mode():
	outputs = model(**inputs, output_hidden_states=True)

	hidden_states = outputs.hidden_states[-1]
	attention_mask = inputs["attention_mask"]

	# Option 1: Last token (EOS) embedding
	last_token_indices = attention_mask.sum(dim=1) - 1
	eos_embeddings = hidden_states[torch.arange(hidden_states.size(0)), last_token_indices, :]

	# Option 2: Mean pooling over all tokens
	expanded_mask = attention_mask.unsqueeze(-1).expand(hidden_states.size()).to(torch.float32)
	sum_embeddings = torch.sum(hidden_states * expanded_mask, dim=1)
	mean_embeddings = sum_embeddings / expanded_mask.sum(dim=1)

	# Output
	print("EOS (Last Token) Embeddings:", eos_embeddings)
	print("Mean Pooling Embeddings:", mean_embeddings)

	# ============================================================================
	# Additional notes:
	# - The preprocessing step ensures sequences are multiples of 6 for 6-mer tokenizer
	# - For causal LM, the last token embedding (EOS) is commonly used
	# - Mean pooling considers all tokens including BOS and content tokens
	# - The choice depends on your downstream task requirements
	# - Both methods handle variable sequence lengths via attention mask
	# ============================================================================

	```

	## Citation
	```
	@article {li2026generator2,
	author = {Li, Qiuyi and Zhan, Zhihao and Feng, Shikun and Zhu, Yiheng and He, Yuan and Wu, Wei and Shi, Zhenghang and Wang, Shengjie and Hu, Zongyong and Yang, Zhao and Li, Jiaoyang and Tang, Jian and Liu, Haiguang and Qin, Tao},
	title = {Functional In-Context Learning in Genomic Language Models with Nucleotide-Level Supervision and Genome Compression},
	elocation-id = {2026.01.27.702015},
	year = {2026},
	doi = {10.64898/2026.01.27.702015},
	publisher = {Cold Spring Harbor Laboratory},
	URL = {https://www.biorxiv.org/content/early/2026/01/29/2026.01.27.702015},
	journal = {bioRxiv}
	}

	@article{wu2025generator,
	title={GENERator: a long-context generative genomic foundation model},
	author={Wu, Wei and Li, Qiuyi and Li, Mingyang and Fu, Kun and Feng, Fuli and Ye, Jieping and Xiong, Hui and Wang, Zheng},
	journal={arXiv preprint arXiv:2502.07272},
	year={2025}
	}

	```