| --- |
| license: mit |
| library_name: pytorch |
| pipeline_tag: feature-extraction |
| tags: |
| - protein |
| - structural-biology |
| - representation-learning |
| - 3d-cnn |
| - foldvision |
| --- |
| |
| # FoldVision Encoder |
|
|
| ## Model Summary |
|
|
| FoldVision is a protein 3D-CNN encoder that maps a voxelized protein structure to a fixed-size embedding (`1024` dimensions). |
|
|
| Primary task: |
| - **Protein feature extraction** from 3D structure. |
|
|
| Typical downstream tasks (with finetuning heads): |
| - Protein-only regression/classification. |
| - PSI (**protein-small molecule interactions**) prediction when combined with a SMILES encoder. |
|
|
| GitHub code: [foldvision_github](https://github.com/AlexanderKroll/foldvision) |
|
|
| ## Model Details |
|
|
| - Model name: `AlexanderKroll/foldvision-encoder` |
| - Architecture: 3D CNN encoder with GroupNorm blocks and global pooling. |
| - Framework: PyTorch |
| - Input channels: 5 atom-type channels (`C`, `N`, `S`, `O`, `P`) |
| - Output: `(B, 1024)` embedding |
|
|
|
|
| ## Input and Preprocessing |
|
|
| This model expects FoldVision voxel tensors generated from PDB structures. |
|
|
| Recommended preprocessing pipeline: |
| 1. Convert `.pdb` files to sparse point lists (`numpy_3D_point_lists/*.npz`). |
| 2. Use `bounding_boxes.npy` + dataloader to construct dense tensors at runtime. |
|
|
| Repository scripts: |
| - `scripts/preprocess_pdb_dir.py` |
| - `scripts/embed_proteins.py` |
| - `scripts/train.py` |
| - `scripts/train_PSI.py` |
| - `scripts/evaluate.py` |
| - `scripts/evaluate_PSI.py` |
|
|
| ## Usage |
|
|
| ```python |
| from foldvision import FoldVisionEncoder |
| |
| model = FoldVisionEncoder.from_pretrained("AlexanderKroll/foldvision-encoder") |
| model.eval() |
| # x: (B, 5, Z, Y, X) |
| # z = model(x) # (B, 1024) |
| ``` |
|
|
| ## Multi-Run Embeddings and Predictions |
|
|
| FoldVision pipelines support repeated runs with random 3D rotations (test-time augmentation). |
|
|
| - Embeddings: |
| - per-run: keep each run-specific embedding, |
| - aggregated: use mean embedding for a stable representation. |
| - Predictions: |
| - per-run predictions can be used to inspect spread/uncertainty, |
| - averaged predictions are recommended for reporting. |
|
|
|
|
| ## Citation |
|
|
| If you use this model, cite: |
|
|
| 1. **FoldVision bioRxiv manuscript**: |
|
|
| ```bibtex |
| @article{foldvision_biorxiv, |
| title = {FoldVision: A compute-efficient atom-level 3D protein encoder}, |
| author = {Kroll, Alexander and Yadav, Shantanu and Lercher, Martin J.}, |
| journal = {bioRxiv}, |
| year = {2026}, |
| doi = {10.64898/2026.01.23.701326}, |
| url = {https://doi.org/10.64898/2026.01.23.701326} |
| } |
| ``` |
|
|
| 2. The GitHub repository: |
|
|
| ```bibtex |
| @misc{foldvision_github, |
| title = {FoldVision code repository}, |
| author = {Kroll, Alexander}, |
| year = {2026}, |
| howpublished = {\url{https://github.com/AlexanderKroll/foldvision}} |
| } |
| ``` |
|
|
| ## Model Card Contact |
|
|
| For issues or questions, use the GitHub issue tracker in the FoldVision repository. |
|
|
