File size: 2,743 Bytes
30272b1 0f5ae29 30272b1 0f5ae29 30272b1 0f5ae29 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 | ---
language:
- en
license: eupl-1.2
tags:
- genomics
- dna
- model-inversion
- privacy
- pytorch
---
# DNA Embedding Inversion
Three final LCQD decoders from [How Private Are DNA Embeddings?](https://arxiv.org/abs/2603.06950), reconstructing DNA from mean-pooled foundation-model embeddings.
**[Code, embedding extraction and results](https://github.com/not-a-feature/DNA-Embedding-Inversion)**
| Model | Input dimensions | Extraction layer |
| --- | ---: | --- |
| `dnabert2` | 768 | Final hidden layer (12) |
| `evo2` | 4096 | `blocks.26.mlp.l3` (zero-based) |
| `ntv2` | 1024 | Final hidden layer (29) |
Each decoder covers 14 lengths from 10 to 100 nucleotides. These are the main-study checkpoints, trained on coordinate- and exact-sequence-disjoint hg38 data for 20 epochs, retaining the best validation checkpoint.
## Quick start
Python 3.12; CPU or CUDA. Download this repository with Git LFS:
```bash
git lfs install
git clone https://huggingface.co/not-a-feature/DNA_Embedding_Invesion
cd DNA_Embedding_Invesion
pip install -r requirements.txt
python infer.py --model ntv2 --embeddings embeddings.npy --output reconstructions.json
```
`embeddings.npy` must contain a finite float array shaped `[N, input_dimensions]` of **raw, unnormalized mean-pooled embeddings**, extracted using the matching model and recipe linked above. The script applies the bundled exact training normalization and returns a JSON list of DNA strings. Add `--device cuda` for GPU inference.
Python API:
```python
import numpy as np
from infer import Decoder
sequences = Decoder("ntv2").predict(np.load("embeddings.npy", allow_pickle=False))
```
No original sequence length is required. Decoding stops at the first predicted EOS; predictions are not cropped to a requested length. Tokenizers are bundled, so inference works offline after installation and download. Foundation-model weights and embedding extraction are separate.
## Validation and scope
CPU predictions matched the original evaluation code on 252 embeddings: three per length, per model, for hg38 and 1000 Genomes. CLI execution, EOS handling and invalid-input checks also passed. This verifies inference parity, not a new accuracy estimate; see `validation.json`.
Performance depends on sequence length and embedding source. Evaluation assumes reference orientation. These checkpoints are intended for research on genomic embedding privacy.
Weights and decoder code: **EUPL-1.2**. Bundled tokenizers originate from [DNABERT-2](https://huggingface.co/zhihan1996/DNABERT-2-117M) and [NTv2](https://huggingface.co/InstaDeepAI/nucleotide-transformer-v2-500m-multi-species); their upstream terms apply. Configurations, normalization provenance and SHA-256 checksums are included.
|