metadata
library_name: transformers
pipeline_tag: fill-mask
tags:
- genomics
- dna
- masked-lm
- ntv3
- long-range
- base-model
license: other
language:
- code
model_parameter_count: 7692475
This repo holds the code for the initial NTv3 release (2025). The 2025 checkpoints load their modelling code from here. The revised NTv3 models ship their own code in each repo.
InstaDeepAI/ntv3_base_model
Unified base model repository for NTv3 models.
This repository contains shared modeling code used by both:
- Pre-trained models (masked language models)
- Post-trained models (conditioned multi-species models with functional genomics heads)
Note: This repo should not be used standalone. It provides modeling code that is referenced by individual model checkpoints via trust_remote_code=True.
Contents
| File | Purpose |
|---|---|
configuration_ntv3_pretrained.py |
Config class: Ntv3PreTrainedConfig |
configuration_ntv3_posttrained.py |
Config classes: DiscreteConditionedNTv3Config, NTv3PostTrainedConfig |
modeling_ntv3_pretrained.py |
Pre-trained model: NTv3PreTrained |
modeling_ntv3_posttrained.py |
Post-trained model: NTv3PostTrained with conditioned towers and heads |
tokenization_ntv3.py |
Tokenizer: NTv3Tokenizer (DNA) |
Architecture
- U-Net style conv tower → Transformer stack → deconv tower → LM head
- Post-trained models add adaptive layer norms and multi-species prediction heads
- Tokenizer: character-level over A T C G N + specials (
<unk><pad><mask><cls><eos><bos>)