Feature Extraction
Transformers
PyTorch
English
fill-mask
genomics
virology
dnabert
foundation-model
hvilm
viral-genomics
pathogenicity
transmissibility
host-tropism
hvue
custom_code
Instructions to use duttaprat/HViLM-base with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use duttaprat/HViLM-base with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("feature-extraction", model="duttaprat/HViLM-base", trust_remote_code=True)# pip install -U transformers accelerate # Load model directly from transformers import AutoModelForMaskedLM model = AutoModelForMaskedLM.from_pretrained("duttaprat/HViLM-base", trust_remote_code=True, device_map="auto") - Notebooks
- Google Colab
- Kaggle
Update README.md
Browse files
README.md
CHANGED
|
@@ -14,9 +14,9 @@ tags:
|
|
| 14 |
- pathogenicity
|
| 15 |
- transmissibility
|
| 16 |
- host-tropism
|
| 17 |
-
- hvue
|
| 18 |
datasets:
|
| 19 |
-
- duttaprat/HVUE
|
| 20 |
pipeline_tag: feature-extraction
|
| 21 |
widget:
|
| 22 |
- text: "ATGCGTACGTTAGCCGATCG"
|
|
@@ -29,15 +29,12 @@ widget:
|
|
| 29 |
|
| 30 |
[](https://www.biorxiv.org/content/10.64898/2026.03.18.712700v1)
|
| 31 |
[](https://github.com/duttaprat/HViLM)
|
| 32 |
-
[](LICENSE)
|
| 34 |
|
| 35 |
</div>
|
| 36 |
|
| 37 |
-
|
| 38 |
-
> **HVUE v2 supersedes the original HVUE benchmark.**
|
| 39 |
-
> The original HVUE v1 benchmark contained substantial cross-split sequence similarity that could inflate held-out performance estimates. HVUE v2 was rebuilt using source-sequence clustering **before** train/validation/test assignment and chunking, followed by exact- and near-match leakage auditing.
|
| 40 |
-
> **Use [duttaprat/HVUE-v2](https://huggingface.co/datasets/duttaprat/HVUE-v2) for current HViLM evaluation.**
|
| 41 |
|
| 42 |
## Model Description
|
| 43 |
|
|
@@ -45,7 +42,7 @@ widget:
|
|
| 45 |
|
| 46 |
The architecture and tokenizer remain those of DNABERT-2; continued pre-training updates the model weights using a masked-language-modeling objective on the virus-focused corpus.
|
| 47 |
|
| 48 |
-
**Preprint:** *HViLM: A Foundation Model for Viral Genomics Enables
|
| 49 |
[bioRxiv 2026.03.18.712700](https://www.biorxiv.org/content/10.64898/2026.03.18.712700v1)
|
| 50 |
|
| 51 |
**Authors:** Pratik Dutta, Jack Vaska, Pallavi Surana, Rekha Sathian, Max Chao, Zhihan Zhou, Han Liu, and Ramana V. Davuluri
|
|
@@ -56,7 +53,7 @@ The architecture and tokenizer remain those of DNABERT-2; continued pre-training
|
|
| 56 |
|
| 57 |
## HViLM Model Family
|
| 58 |
|
| 59 |
-
HViLM-base is the continued-pretrained foundation model. Official task-specific models fine-tuned on HVUE
|
| 60 |
|
| 61 |
| Resource | Purpose |
|
| 62 |
|---|---|
|
|
@@ -64,7 +61,7 @@ HViLM-base is the continued-pretrained foundation model. Official task-specific
|
|
| 64 |
| [HViLM-Patho](https://huggingface.co/duttaprat/HViLM-Patho) | Pathogenicity classification |
|
| 65 |
| [HViLM-R0](https://huggingface.co/duttaprat/HViLM-R0) | Transmissibility classification |
|
| 66 |
| [HViLM-Tropism](https://huggingface.co/duttaprat/HViLM-Tropism) | Human host-tropism classification |
|
| 67 |
-
| [HVUE
|
| 68 |
|
| 69 |
The complete project is also grouped in the **HViLM: Human Virome Language Model** collection on the [duttaprat Collections page](https://huggingface.co/duttaprat/collections).
|
| 70 |
|
|
@@ -75,9 +72,9 @@ The complete project is also grouped in the **HViLM: Human Virome Language Model
|
|
| 75 |
- **Virus-focused continued pre-training:** approximately 5M non-redundant fragments derived from VIRION-linked virus sequences.
|
| 76 |
- **DNABERT-2 initialization:** preserves the DNABERT-2 architecture and BPE tokenizer while adapting model weights to virus sequence data.
|
| 77 |
- **Three official downstream models:** pathogenicity, transmissibility, and host tropism.
|
| 78 |
-
- **HVUE
|
| 79 |
- **Parameter-efficient downstream adaptation:** official task models were trained with LoRA.
|
| 80 |
-
- **Public reproducibility resources:** base model, three task-specific checkpoints, HVUE
|
| 81 |
|
| 82 |
---
|
| 83 |
|
|
@@ -105,7 +102,7 @@ HViLM-base is derived from **DNABERT-2 (117M parameters)**.
|
|
| 105 |
| Approximate training time | 72 hours |
|
| 106 |
| Held-out MLM accuracy | 94.2% |
|
| 107 |
|
| 108 |
-
**Sequence-length note:** HViLM uses BPE tokenization, so nucleotide length and model-token length are not equivalent. The continued-pretraining corpus used 1000-nt sequence fragments; downstream configurations are described by nucleotide length in HVUE
|
| 109 |
|
| 110 |
---
|
| 111 |
|
|
@@ -151,7 +148,7 @@ For sequence-level representations, pooling strategy should be chosen according
|
|
| 151 |
|
| 152 |
## Use the Official Fine-tuned Models
|
| 153 |
|
| 154 |
-
If the goal is one of the three HVUE
|
| 155 |
|
| 156 |
### Pathogenicity
|
| 157 |
|
|
@@ -185,6 +182,7 @@ Labels:
|
|
| 185 |
- `0` → `R0_LT_1`
|
| 186 |
- `1` → `R0_GE_1`
|
| 187 |
|
|
|
|
| 188 |
### Host Tropism
|
| 189 |
|
| 190 |
Use:
|
|
@@ -202,9 +200,9 @@ See the individual model cards for full usage examples and task-specific limitat
|
|
| 202 |
|
| 203 |
---
|
| 204 |
|
| 205 |
-
## HVUE
|
| 206 |
|
| 207 |
-
[HVUE
|
| 208 |
|
| 209 |
The benchmark was reconstructed to reduce supervised train-test leakage using the following ordering:
|
| 210 |
|
|
@@ -213,9 +211,9 @@ The benchmark was reconstructed to reduce supervised train-test leakage using th
|
|
| 213 |
3. assign complete clusters to train/validation/test splits;
|
| 214 |
4. chunk sequences only **after** split assignment;
|
| 215 |
5. remove exact duplicate chunks;
|
| 216 |
-
6. audit cross-split exact
|
| 217 |
|
| 218 |
-
HVUE
|
| 219 |
|
| 220 |
- **Pathogenicity**
|
| 221 |
- **Transmissibility**
|
|
@@ -227,7 +225,7 @@ Across the benchmark, configurations evaluate different sequence lengths (500, 1
|
|
| 227 |
|
| 228 |
The primary results below use the **standard 1000-nt configuration** for each task.
|
| 229 |
|
| 230 |
-
| Task | HVUE
|
| 231 |
|---|---|---:|---:|---:|---|
|
| 232 |
| Pathogenicity | `standard_capped_1000bp` | **92.39** | **91.32** | **83.10** | [HViLM-Patho](https://huggingface.co/duttaprat/HViLM-Patho) |
|
| 233 |
| Transmissibility | `standard_capped_1000bp` | **87.50** | **86.16** | **72.66** | [HViLM-R0](https://huggingface.co/duttaprat/HViLM-R0) |
|
|
@@ -235,7 +233,7 @@ The primary results below use the **standard 1000-nt configuration** for each ta
|
|
| 235 |
|
| 236 |
The directory/configuration identifiers retain `bp` for release stability; manuscript and descriptive text use **nt** for nucleotide sequence length.
|
| 237 |
|
| 238 |
-
### Interpretation of the HVUE
|
| 239 |
|
| 240 |
Under leakage-controlled evaluation, the effect of virus-focused continued pre-training is **task dependent**:
|
| 241 |
|
|
@@ -243,9 +241,9 @@ Under leakage-controlled evaluation, the effect of virus-focused continued pre-t
|
|
| 243 |
- **Transmissibility:** HViLM and DNABERT-2 are close (86.16 vs. 85.81 F1), and HViLM is essentially tied with DNABERT-MB (86.16 vs. 86.15 F1).
|
| 244 |
- **Host Tropism:** HViLM shows the largest F1 improvement, reaching 74.49 compared with 64.82 for class-balanced DNABERT-2.
|
| 245 |
|
| 246 |
-
These results support a
|
| 247 |
|
| 248 |
-
For complete baseline comparisons, hard-split evaluations, temporal evaluations, and sequence-length analyses, see the [HViLM GitHub repository](https://github.com/duttaprat/HViLM) and [HVUE
|
| 249 |
|
| 250 |
---
|
| 251 |
|
|
@@ -277,11 +275,10 @@ These observations are **hypothesis-generating**. Sequence similarity between vi
|
|
| 277 |
|
| 278 |
## Limitations
|
| 279 |
|
| 280 |
-
- HVUE
|
| 281 |
- The complete historical training exposure of the original DNABERT-2 model cannot be reconstructed; therefore, absence of all possible ancestral pretraining exposure to benchmark-related sequences cannot be guaranteed.
|
| 282 |
- Host association is biologically context-dependent and may include multi-host, zoonotic, and reverse-zoonotic relationships; the benchmark uses a simplified binary formulation.
|
| 283 |
-
-
|
| 284 |
-
- Performance differences between closely matched models should not be interpreted as statistically meaningful without uncertainty estimates or repeated evaluations.
|
| 285 |
- Attention-based motif analyses should be considered exploratory rather than direct evidence of mechanism.
|
| 286 |
- HViLM predictions are research outputs and are not intended to replace experimental, clinical, epidemiological, or public-health assessment.
|
| 287 |
|
|
@@ -293,7 +290,7 @@ These observations are **hypothesis-generating**. Sequence similarity between vi
|
|
| 293 |
- **Pathogenicity model:** [duttaprat/HViLM-Patho](https://huggingface.co/duttaprat/HViLM-Patho)
|
| 294 |
- **Transmissibility model:** [duttaprat/HViLM-R0](https://huggingface.co/duttaprat/HViLM-R0)
|
| 295 |
- **Host Tropism model:** [duttaprat/HViLM-Tropism](https://huggingface.co/duttaprat/HViLM-Tropism)
|
| 296 |
-
- **Benchmark:** [duttaprat/HVUE
|
| 297 |
- **Code:** [github.com/duttaprat/HViLM](https://github.com/duttaprat/HViLM)
|
| 298 |
- **Collections:** [duttaprat's Hugging Face Collections](https://huggingface.co/duttaprat/collections)
|
| 299 |
|
|
|
|
| 14 |
- pathogenicity
|
| 15 |
- transmissibility
|
| 16 |
- host-tropism
|
| 17 |
+
- hvue
|
| 18 |
datasets:
|
| 19 |
+
- duttaprat/HVUE
|
| 20 |
pipeline_tag: feature-extraction
|
| 21 |
widget:
|
| 22 |
- text: "ATGCGTACGTTAGCCGATCG"
|
|
|
|
| 29 |
|
| 30 |
[](https://www.biorxiv.org/content/10.64898/2026.03.18.712700v1)
|
| 31 |
[](https://github.com/duttaprat/HViLM)
|
| 32 |
+
[](https://huggingface.co/datasets/duttaprat/HVUE)
|
| 33 |
[](LICENSE)
|
| 34 |
|
| 35 |
</div>
|
| 36 |
|
| 37 |
+
|
|
|
|
|
|
|
|
|
|
| 38 |
|
| 39 |
## Model Description
|
| 40 |
|
|
|
|
| 42 |
|
| 43 |
The architecture and tokenizer remain those of DNABERT-2; continued pre-training updates the model weights using a masked-language-modeling objective on the virus-focused corpus.
|
| 44 |
|
| 45 |
+
**Preprint:** *HViLM: A Foundation Model for Viral Genomics Enables Prediction of Pathogenicity, Transmissibility, and Host Tropism*
|
| 46 |
[bioRxiv 2026.03.18.712700](https://www.biorxiv.org/content/10.64898/2026.03.18.712700v1)
|
| 47 |
|
| 48 |
**Authors:** Pratik Dutta, Jack Vaska, Pallavi Surana, Rekha Sathian, Max Chao, Zhihan Zhou, Han Liu, and Ramana V. Davuluri
|
|
|
|
| 53 |
|
| 54 |
## HViLM Model Family
|
| 55 |
|
| 56 |
+
HViLM-base is the continued-pretrained foundation model. Official task-specific models fine-tuned on HVUE are released as standalone checkpoints:
|
| 57 |
|
| 58 |
| Resource | Purpose |
|
| 59 |
|---|---|
|
|
|
|
| 61 |
| [HViLM-Patho](https://huggingface.co/duttaprat/HViLM-Patho) | Pathogenicity classification |
|
| 62 |
| [HViLM-R0](https://huggingface.co/duttaprat/HViLM-R0) | Transmissibility classification |
|
| 63 |
| [HViLM-Tropism](https://huggingface.co/duttaprat/HViLM-Tropism) | Human host-tropism classification |
|
| 64 |
+
| [HVUE](https://huggingface.co/datasets/duttaprat/HVUE) | Leakage-controlled benchmark |
|
| 65 |
|
| 66 |
The complete project is also grouped in the **HViLM: Human Virome Language Model** collection on the [duttaprat Collections page](https://huggingface.co/duttaprat/collections).
|
| 67 |
|
|
|
|
| 72 |
- **Virus-focused continued pre-training:** approximately 5M non-redundant fragments derived from VIRION-linked virus sequences.
|
| 73 |
- **DNABERT-2 initialization:** preserves the DNABERT-2 architecture and BPE tokenizer while adapting model weights to virus sequence data.
|
| 74 |
- **Three official downstream models:** pathogenicity, transmissibility, and host tropism.
|
| 75 |
+
- **HVUE evaluation:** cluster-aware splitting before chunking, with multiple similarity stringencies and sequence lengths.
|
| 76 |
- **Parameter-efficient downstream adaptation:** official task models were trained with LoRA.
|
| 77 |
+
- **Public reproducibility resources:** base model, three task-specific checkpoints, HVUE benchmark, and project code are released publicly.
|
| 78 |
|
| 79 |
---
|
| 80 |
|
|
|
|
| 102 |
| Approximate training time | 72 hours |
|
| 103 |
| Held-out MLM accuracy | 94.2% |
|
| 104 |
|
| 105 |
+
**Sequence-length note:** HViLM uses BPE tokenization, so nucleotide length and model-token length are not equivalent. The continued-pretraining corpus used 1000-nt sequence fragments; downstream configurations are described by nucleotide length in HVUE.
|
| 106 |
|
| 107 |
---
|
| 108 |
|
|
|
|
| 148 |
|
| 149 |
## Use the Official Fine-tuned Models
|
| 150 |
|
| 151 |
+
If the goal is one of the three HVUE tasks, users can load the corresponding task model directly; `HViLM-base` does not need to be loaded separately.
|
| 152 |
|
| 153 |
### Pathogenicity
|
| 154 |
|
|
|
|
| 182 |
- `0` → `R0_LT_1`
|
| 183 |
- `1` → `R0_GE_1`
|
| 184 |
|
| 185 |
+
These label names are retained for checkpoint compatibility. The task is a binary classification of documented sustained human transmission capacity, informed by reported R₀ values rather than predicting R₀ itself.
|
| 186 |
### Host Tropism
|
| 187 |
|
| 188 |
Use:
|
|
|
|
| 200 |
|
| 201 |
---
|
| 202 |
|
| 203 |
+
## HVUE Benchmark
|
| 204 |
|
| 205 |
+
[HVUE](https://huggingface.co/datasets/duttaprat/HVUE) is the current evaluation benchmark for HViLM.
|
| 206 |
|
| 207 |
The benchmark was reconstructed to reduce supervised train-test leakage using the following ordering:
|
| 208 |
|
|
|
|
| 211 |
3. assign complete clusters to train/validation/test splits;
|
| 212 |
4. chunk sequences only **after** split assignment;
|
| 213 |
5. remove exact duplicate chunks;
|
| 214 |
+
6. audit cross-split exact matches.
|
| 215 |
|
| 216 |
+
HVUE includes:
|
| 217 |
|
| 218 |
- **Pathogenicity**
|
| 219 |
- **Transmissibility**
|
|
|
|
| 225 |
|
| 226 |
The primary results below use the **standard 1000-nt configuration** for each task.
|
| 227 |
|
| 228 |
+
| Task | HVUE configuration | Accuracy | F1 | MCC | Official model |
|
| 229 |
|---|---|---:|---:|---:|---|
|
| 230 |
| Pathogenicity | `standard_capped_1000bp` | **92.39** | **91.32** | **83.10** | [HViLM-Patho](https://huggingface.co/duttaprat/HViLM-Patho) |
|
| 231 |
| Transmissibility | `standard_capped_1000bp` | **87.50** | **86.16** | **72.66** | [HViLM-R0](https://huggingface.co/duttaprat/HViLM-R0) |
|
|
|
|
| 233 |
|
| 234 |
The directory/configuration identifiers retain `bp` for release stability; manuscript and descriptive text use **nt** for nucleotide sequence length.
|
| 235 |
|
| 236 |
+
### Interpretation of the HVUE Results
|
| 237 |
|
| 238 |
Under leakage-controlled evaluation, the effect of virus-focused continued pre-training is **task dependent**:
|
| 239 |
|
|
|
|
| 241 |
- **Transmissibility:** HViLM and DNABERT-2 are close (86.16 vs. 85.81 F1), and HViLM is essentially tied with DNABERT-MB (86.16 vs. 86.15 F1).
|
| 242 |
- **Host Tropism:** HViLM shows the largest F1 improvement, reaching 74.49 compared with 64.82 for class-balanced DNABERT-2.
|
| 243 |
|
| 244 |
+
These results support a specific conclusion: virus-focused continued pre-training provides its clearest benefit on the more challenging Host Tropism task, while gains on Pathogenicity and Transmissibility are smaller.
|
| 245 |
|
| 246 |
+
For complete baseline comparisons, hard-split evaluations, temporal evaluations, and sequence-length analyses, see the [HViLM GitHub repository](https://github.com/duttaprat/HViLM) and [HVUE](https://huggingface.co/datasets/duttaprat/HVUE).
|
| 247 |
|
| 248 |
---
|
| 249 |
|
|
|
|
| 275 |
|
| 276 |
## Limitations
|
| 277 |
|
| 278 |
+
- HVUE controls supervised split leakage through source-level clustering and exact-match auditing, but sequence-similarity thresholds cannot eliminate every form of biological relatedness.
|
| 279 |
- The complete historical training exposure of the original DNABERT-2 model cannot be reconstructed; therefore, absence of all possible ancestral pretraining exposure to benchmark-related sequences cannot be guaranteed.
|
| 280 |
- Host association is biologically context-dependent and may include multi-host, zoonotic, and reverse-zoonotic relationships; the benchmark uses a simplified binary formulation.
|
| 281 |
+
- Transmissibility labels reduce a continuous, context-dependent epidemiological quantity to a binary benchmark task.- Performance differences between closely matched models should not be interpreted as statistically meaningful without uncertainty estimates or repeated evaluations.
|
|
|
|
| 282 |
- Attention-based motif analyses should be considered exploratory rather than direct evidence of mechanism.
|
| 283 |
- HViLM predictions are research outputs and are not intended to replace experimental, clinical, epidemiological, or public-health assessment.
|
| 284 |
|
|
|
|
| 290 |
- **Pathogenicity model:** [duttaprat/HViLM-Patho](https://huggingface.co/duttaprat/HViLM-Patho)
|
| 291 |
- **Transmissibility model:** [duttaprat/HViLM-R0](https://huggingface.co/duttaprat/HViLM-R0)
|
| 292 |
- **Host Tropism model:** [duttaprat/HViLM-Tropism](https://huggingface.co/duttaprat/HViLM-Tropism)
|
| 293 |
+
- **Benchmark:** [duttaprat/HVUE](https://huggingface.co/datasets/duttaprat/HVUE)
|
| 294 |
- **Code:** [github.com/duttaprat/HViLM](https://github.com/duttaprat/HViLM)
|
| 295 |
- **Collections:** [duttaprat's Hugging Face Collections](https://huggingface.co/duttaprat/collections)
|
| 296 |
|