duttaprat commited on
Commit
3f125df
·
verified ·
1 Parent(s): 7cd4d3a

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +23 -26
README.md CHANGED
@@ -14,9 +14,9 @@ tags:
14
  - pathogenicity
15
  - transmissibility
16
  - host-tropism
17
- - hvue-v2
18
  datasets:
19
- - duttaprat/HVUE-v2
20
  pipeline_tag: feature-extraction
21
  widget:
22
  - text: "ATGCGTACGTTAGCCGATCG"
@@ -29,15 +29,12 @@ widget:
29
 
30
  [![Preprint](https://img.shields.io/badge/bioRxiv-2026-B31B1B)](https://www.biorxiv.org/content/10.64898/2026.03.18.712700v1)
31
  [![Code](https://img.shields.io/badge/Code-GitHub-black)](https://github.com/duttaprat/HViLM)
32
- [![Dataset](https://img.shields.io/badge/Dataset-HVUE--v2-yellow)](https://huggingface.co/datasets/duttaprat/HVUE-v2)
33
  [![License](https://img.shields.io/badge/License-Apache%202.0-green.svg)](LICENSE)
34
 
35
  </div>
36
 
37
- > [!IMPORTANT]
38
- > **HVUE v2 supersedes the original HVUE benchmark.**
39
- > The original HVUE v1 benchmark contained substantial cross-split sequence similarity that could inflate held-out performance estimates. HVUE v2 was rebuilt using source-sequence clustering **before** train/validation/test assignment and chunking, followed by exact- and near-match leakage auditing.
40
- > **Use [duttaprat/HVUE-v2](https://huggingface.co/datasets/duttaprat/HVUE-v2) for current HViLM evaluation.**
41
 
42
  ## Model Description
43
 
@@ -45,7 +42,7 @@ widget:
45
 
46
  The architecture and tokenizer remain those of DNABERT-2; continued pre-training updates the model weights using a masked-language-modeling objective on the virus-focused corpus.
47
 
48
- **Preprint:** *HViLM: A Foundation Model for Viral Genomics Enables Multi-Task Prediction of Pathogenicity, Transmissibility, and Host Tropism*
49
  [bioRxiv 2026.03.18.712700](https://www.biorxiv.org/content/10.64898/2026.03.18.712700v1)
50
 
51
  **Authors:** Pratik Dutta, Jack Vaska, Pallavi Surana, Rekha Sathian, Max Chao, Zhihan Zhou, Han Liu, and Ramana V. Davuluri
@@ -56,7 +53,7 @@ The architecture and tokenizer remain those of DNABERT-2; continued pre-training
56
 
57
  ## HViLM Model Family
58
 
59
- HViLM-base is the continued-pretrained foundation model. Official task-specific models fine-tuned on HVUE v2 are released as standalone checkpoints:
60
 
61
  | Resource | Purpose |
62
  |---|---|
@@ -64,7 +61,7 @@ HViLM-base is the continued-pretrained foundation model. Official task-specific
64
  | [HViLM-Patho](https://huggingface.co/duttaprat/HViLM-Patho) | Pathogenicity classification |
65
  | [HViLM-R0](https://huggingface.co/duttaprat/HViLM-R0) | Transmissibility classification |
66
  | [HViLM-Tropism](https://huggingface.co/duttaprat/HViLM-Tropism) | Human host-tropism classification |
67
- | [HVUE-v2](https://huggingface.co/datasets/duttaprat/HVUE-v2) | Leakage-controlled benchmark |
68
 
69
  The complete project is also grouped in the **HViLM: Human Virome Language Model** collection on the [duttaprat Collections page](https://huggingface.co/duttaprat/collections).
70
 
@@ -75,9 +72,9 @@ The complete project is also grouped in the **HViLM: Human Virome Language Model
75
  - **Virus-focused continued pre-training:** approximately 5M non-redundant fragments derived from VIRION-linked virus sequences.
76
  - **DNABERT-2 initialization:** preserves the DNABERT-2 architecture and BPE tokenizer while adapting model weights to virus sequence data.
77
  - **Three official downstream models:** pathogenicity, transmissibility, and host tropism.
78
- - **HVUE v2 evaluation:** cluster-aware splitting before chunking, with multiple similarity stringencies and sequence lengths.
79
  - **Parameter-efficient downstream adaptation:** official task models were trained with LoRA.
80
- - **Public reproducibility resources:** base model, three task-specific checkpoints, HVUE v2 benchmark, and project code are released publicly.
81
 
82
  ---
83
 
@@ -105,7 +102,7 @@ HViLM-base is derived from **DNABERT-2 (117M parameters)**.
105
  | Approximate training time | 72 hours |
106
  | Held-out MLM accuracy | 94.2% |
107
 
108
- **Sequence-length note:** HViLM uses BPE tokenization, so nucleotide length and model-token length are not equivalent. The continued-pretraining corpus used 1000-nt sequence fragments; downstream configurations are described by nucleotide length in HVUE v2.
109
 
110
  ---
111
 
@@ -151,7 +148,7 @@ For sequence-level representations, pooling strategy should be chosen according
151
 
152
  ## Use the Official Fine-tuned Models
153
 
154
- If the goal is one of the three HVUE v2 tasks, users can load the corresponding task model directly; `HViLM-base` does not need to be loaded separately.
155
 
156
  ### Pathogenicity
157
 
@@ -185,6 +182,7 @@ Labels:
185
  - `0` → `R0_LT_1`
186
  - `1` → `R0_GE_1`
187
 
 
188
  ### Host Tropism
189
 
190
  Use:
@@ -202,9 +200,9 @@ See the individual model cards for full usage examples and task-specific limitat
202
 
203
  ---
204
 
205
- ## HVUE v2 Benchmark
206
 
207
- [HVUE v2](https://huggingface.co/datasets/duttaprat/HVUE-v2) is the current evaluation benchmark for HViLM. It replaces HVUE v1.
208
 
209
  The benchmark was reconstructed to reduce supervised train-test leakage using the following ordering:
210
 
@@ -213,9 +211,9 @@ The benchmark was reconstructed to reduce supervised train-test leakage using th
213
  3. assign complete clusters to train/validation/test splits;
214
  4. chunk sequences only **after** split assignment;
215
  5. remove exact duplicate chunks;
216
- 6. audit cross-split exact and near matches.
217
 
218
- HVUE v2 includes:
219
 
220
  - **Pathogenicity**
221
  - **Transmissibility**
@@ -227,7 +225,7 @@ Across the benchmark, configurations evaluate different sequence lengths (500, 1
227
 
228
  The primary results below use the **standard 1000-nt configuration** for each task.
229
 
230
- | Task | HVUE v2 configuration | Accuracy | F1 | MCC | Official model |
231
  |---|---|---:|---:|---:|---|
232
  | Pathogenicity | `standard_capped_1000bp` | **92.39** | **91.32** | **83.10** | [HViLM-Patho](https://huggingface.co/duttaprat/HViLM-Patho) |
233
  | Transmissibility | `standard_capped_1000bp` | **87.50** | **86.16** | **72.66** | [HViLM-R0](https://huggingface.co/duttaprat/HViLM-R0) |
@@ -235,7 +233,7 @@ The primary results below use the **standard 1000-nt configuration** for each ta
235
 
236
  The directory/configuration identifiers retain `bp` for release stability; manuscript and descriptive text use **nt** for nucleotide sequence length.
237
 
238
- ### Interpretation of the HVUE v2 Results
239
 
240
  Under leakage-controlled evaluation, the effect of virus-focused continued pre-training is **task dependent**:
241
 
@@ -243,9 +241,9 @@ Under leakage-controlled evaluation, the effect of virus-focused continued pre-t
243
  - **Transmissibility:** HViLM and DNABERT-2 are close (86.16 vs. 85.81 F1), and HViLM is essentially tied with DNABERT-MB (86.16 vs. 86.15 F1).
244
  - **Host Tropism:** HViLM shows the largest F1 improvement, reaching 74.49 compared with 64.82 for class-balanced DNABERT-2.
245
 
246
- These results support a more specific conclusion than the original HVUE v1 evaluation: virus-focused continued pre-training provides its clearest benefit on the more challenging Host Tropism task, while gains on Pathogenicity and Transmissibility are smaller.
247
 
248
- For complete baseline comparisons, hard-split evaluations, temporal evaluations, and sequence-length analyses, see the [HViLM GitHub repository](https://github.com/duttaprat/HViLM) and [HVUE-v2](https://huggingface.co/datasets/duttaprat/HVUE-v2).
249
 
250
  ---
251
 
@@ -277,11 +275,10 @@ These observations are **hypothesis-generating**. Sequence similarity between vi
277
 
278
  ## Limitations
279
 
280
- - HVUE v2 controls supervised split leakage through source-level clustering and auditing, but sequence-similarity thresholds cannot eliminate every form of biological relatedness.
281
  - The complete historical training exposure of the original DNABERT-2 model cannot be reconstructed; therefore, absence of all possible ancestral pretraining exposure to benchmark-related sequences cannot be guaranteed.
282
  - Host association is biologically context-dependent and may include multi-host, zoonotic, and reverse-zoonotic relationships; the benchmark uses a simplified binary formulation.
283
- - R₀-based transmissibility labels simplify a continuous, context-dependent epidemiological quantity into a binary benchmark task.
284
- - Performance differences between closely matched models should not be interpreted as statistically meaningful without uncertainty estimates or repeated evaluations.
285
  - Attention-based motif analyses should be considered exploratory rather than direct evidence of mechanism.
286
  - HViLM predictions are research outputs and are not intended to replace experimental, clinical, epidemiological, or public-health assessment.
287
 
@@ -293,7 +290,7 @@ These observations are **hypothesis-generating**. Sequence similarity between vi
293
  - **Pathogenicity model:** [duttaprat/HViLM-Patho](https://huggingface.co/duttaprat/HViLM-Patho)
294
  - **Transmissibility model:** [duttaprat/HViLM-R0](https://huggingface.co/duttaprat/HViLM-R0)
295
  - **Host Tropism model:** [duttaprat/HViLM-Tropism](https://huggingface.co/duttaprat/HViLM-Tropism)
296
- - **Benchmark:** [duttaprat/HVUE-v2](https://huggingface.co/datasets/duttaprat/HVUE-v2)
297
  - **Code:** [github.com/duttaprat/HViLM](https://github.com/duttaprat/HViLM)
298
  - **Collections:** [duttaprat's Hugging Face Collections](https://huggingface.co/duttaprat/collections)
299
 
 
14
  - pathogenicity
15
  - transmissibility
16
  - host-tropism
17
+ - hvue
18
  datasets:
19
+ - duttaprat/HVUE
20
  pipeline_tag: feature-extraction
21
  widget:
22
  - text: "ATGCGTACGTTAGCCGATCG"
 
29
 
30
  [![Preprint](https://img.shields.io/badge/bioRxiv-2026-B31B1B)](https://www.biorxiv.org/content/10.64898/2026.03.18.712700v1)
31
  [![Code](https://img.shields.io/badge/Code-GitHub-black)](https://github.com/duttaprat/HViLM)
32
+ [![Dataset](https://img.shields.io/badge/Dataset-HVUE-yellow)](https://huggingface.co/datasets/duttaprat/HVUE)
33
  [![License](https://img.shields.io/badge/License-Apache%202.0-green.svg)](LICENSE)
34
 
35
  </div>
36
 
37
+
 
 
 
38
 
39
  ## Model Description
40
 
 
42
 
43
  The architecture and tokenizer remain those of DNABERT-2; continued pre-training updates the model weights using a masked-language-modeling objective on the virus-focused corpus.
44
 
45
+ **Preprint:** *HViLM: A Foundation Model for Viral Genomics Enables Prediction of Pathogenicity, Transmissibility, and Host Tropism*
46
  [bioRxiv 2026.03.18.712700](https://www.biorxiv.org/content/10.64898/2026.03.18.712700v1)
47
 
48
  **Authors:** Pratik Dutta, Jack Vaska, Pallavi Surana, Rekha Sathian, Max Chao, Zhihan Zhou, Han Liu, and Ramana V. Davuluri
 
53
 
54
  ## HViLM Model Family
55
 
56
+ HViLM-base is the continued-pretrained foundation model. Official task-specific models fine-tuned on HVUE are released as standalone checkpoints:
57
 
58
  | Resource | Purpose |
59
  |---|---|
 
61
  | [HViLM-Patho](https://huggingface.co/duttaprat/HViLM-Patho) | Pathogenicity classification |
62
  | [HViLM-R0](https://huggingface.co/duttaprat/HViLM-R0) | Transmissibility classification |
63
  | [HViLM-Tropism](https://huggingface.co/duttaprat/HViLM-Tropism) | Human host-tropism classification |
64
+ | [HVUE](https://huggingface.co/datasets/duttaprat/HVUE) | Leakage-controlled benchmark |
65
 
66
  The complete project is also grouped in the **HViLM: Human Virome Language Model** collection on the [duttaprat Collections page](https://huggingface.co/duttaprat/collections).
67
 
 
72
  - **Virus-focused continued pre-training:** approximately 5M non-redundant fragments derived from VIRION-linked virus sequences.
73
  - **DNABERT-2 initialization:** preserves the DNABERT-2 architecture and BPE tokenizer while adapting model weights to virus sequence data.
74
  - **Three official downstream models:** pathogenicity, transmissibility, and host tropism.
75
+ - **HVUE evaluation:** cluster-aware splitting before chunking, with multiple similarity stringencies and sequence lengths.
76
  - **Parameter-efficient downstream adaptation:** official task models were trained with LoRA.
77
+ - **Public reproducibility resources:** base model, three task-specific checkpoints, HVUE benchmark, and project code are released publicly.
78
 
79
  ---
80
 
 
102
  | Approximate training time | 72 hours |
103
  | Held-out MLM accuracy | 94.2% |
104
 
105
+ **Sequence-length note:** HViLM uses BPE tokenization, so nucleotide length and model-token length are not equivalent. The continued-pretraining corpus used 1000-nt sequence fragments; downstream configurations are described by nucleotide length in HVUE.
106
 
107
  ---
108
 
 
148
 
149
  ## Use the Official Fine-tuned Models
150
 
151
+ If the goal is one of the three HVUE tasks, users can load the corresponding task model directly; `HViLM-base` does not need to be loaded separately.
152
 
153
  ### Pathogenicity
154
 
 
182
  - `0` → `R0_LT_1`
183
  - `1` → `R0_GE_1`
184
 
185
+ These label names are retained for checkpoint compatibility. The task is a binary classification of documented sustained human transmission capacity, informed by reported R₀ values rather than predicting R₀ itself.
186
  ### Host Tropism
187
 
188
  Use:
 
200
 
201
  ---
202
 
203
+ ## HVUE Benchmark
204
 
205
+ [HVUE](https://huggingface.co/datasets/duttaprat/HVUE) is the current evaluation benchmark for HViLM.
206
 
207
  The benchmark was reconstructed to reduce supervised train-test leakage using the following ordering:
208
 
 
211
  3. assign complete clusters to train/validation/test splits;
212
  4. chunk sequences only **after** split assignment;
213
  5. remove exact duplicate chunks;
214
+ 6. audit cross-split exact matches.
215
 
216
+ HVUE includes:
217
 
218
  - **Pathogenicity**
219
  - **Transmissibility**
 
225
 
226
  The primary results below use the **standard 1000-nt configuration** for each task.
227
 
228
+ | Task | HVUE configuration | Accuracy | F1 | MCC | Official model |
229
  |---|---|---:|---:|---:|---|
230
  | Pathogenicity | `standard_capped_1000bp` | **92.39** | **91.32** | **83.10** | [HViLM-Patho](https://huggingface.co/duttaprat/HViLM-Patho) |
231
  | Transmissibility | `standard_capped_1000bp` | **87.50** | **86.16** | **72.66** | [HViLM-R0](https://huggingface.co/duttaprat/HViLM-R0) |
 
233
 
234
  The directory/configuration identifiers retain `bp` for release stability; manuscript and descriptive text use **nt** for nucleotide sequence length.
235
 
236
+ ### Interpretation of the HVUE Results
237
 
238
  Under leakage-controlled evaluation, the effect of virus-focused continued pre-training is **task dependent**:
239
 
 
241
  - **Transmissibility:** HViLM and DNABERT-2 are close (86.16 vs. 85.81 F1), and HViLM is essentially tied with DNABERT-MB (86.16 vs. 86.15 F1).
242
  - **Host Tropism:** HViLM shows the largest F1 improvement, reaching 74.49 compared with 64.82 for class-balanced DNABERT-2.
243
 
244
+ These results support a specific conclusion: virus-focused continued pre-training provides its clearest benefit on the more challenging Host Tropism task, while gains on Pathogenicity and Transmissibility are smaller.
245
 
246
+ For complete baseline comparisons, hard-split evaluations, temporal evaluations, and sequence-length analyses, see the [HViLM GitHub repository](https://github.com/duttaprat/HViLM) and [HVUE](https://huggingface.co/datasets/duttaprat/HVUE).
247
 
248
  ---
249
 
 
275
 
276
  ## Limitations
277
 
278
+ - HVUE controls supervised split leakage through source-level clustering and exact-match auditing, but sequence-similarity thresholds cannot eliminate every form of biological relatedness.
279
  - The complete historical training exposure of the original DNABERT-2 model cannot be reconstructed; therefore, absence of all possible ancestral pretraining exposure to benchmark-related sequences cannot be guaranteed.
280
  - Host association is biologically context-dependent and may include multi-host, zoonotic, and reverse-zoonotic relationships; the benchmark uses a simplified binary formulation.
281
+ - Transmissibility labels reduce a continuous, context-dependent epidemiological quantity to a binary benchmark task.- Performance differences between closely matched models should not be interpreted as statistically meaningful without uncertainty estimates or repeated evaluations.
 
282
  - Attention-based motif analyses should be considered exploratory rather than direct evidence of mechanism.
283
  - HViLM predictions are research outputs and are not intended to replace experimental, clinical, epidemiological, or public-health assessment.
284
 
 
290
  - **Pathogenicity model:** [duttaprat/HViLM-Patho](https://huggingface.co/duttaprat/HViLM-Patho)
291
  - **Transmissibility model:** [duttaprat/HViLM-R0](https://huggingface.co/duttaprat/HViLM-R0)
292
  - **Host Tropism model:** [duttaprat/HViLM-Tropism](https://huggingface.co/duttaprat/HViLM-Tropism)
293
+ - **Benchmark:** [duttaprat/HVUE](https://huggingface.co/datasets/duttaprat/HVUE)
294
  - **Code:** [github.com/duttaprat/HViLM](https://github.com/duttaprat/HViLM)
295
  - **Collections:** [duttaprat's Hugging Face Collections](https://huggingface.co/duttaprat/collections)
296