dotan1111
/

BetaReconstruct_Configuration1_internal_nodes

@@ -1,199 +1,46 @@
 ---
-library_name: transformers
-tags: []
 ---
-# Model Card for Model ID
-<!-- Provide a quick summary of what the model is/does. -->
-## Model Details
-### Model Description
-<!-- Provide a longer summary of what this model is. -->
-This is the model card of a 🤗 transformers model that has been pushed on the Hub. This model card has been automatically generated.
-- **Developed by:** [More Information Needed]
-- **Funded by [optional]:** [More Information Needed]
-- **Shared by [optional]:** [More Information Needed]
-- **Model type:** [More Information Needed]
-- **Language(s) (NLP):** [More Information Needed]
-- **License:** [More Information Needed]
-- **Finetuned from model [optional]:** [More Information Needed]
 ### Model Sources [optional]
 <!-- Provide the basic links for the model. -->
-- **Repository:** [More Information Needed]
-- **Paper [optional]:** [More Information Needed]
-- **Demo [optional]:** [More Information Needed]
 ## Uses
-<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
-### Direct Use
-<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
-[More Information Needed]
-### Downstream Use [optional]
-<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
-[More Information Needed]
-### Out-of-Scope Use
-<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
-[More Information Needed]
-## Bias, Risks, and Limitations
-<!-- This section is meant to convey both technical and sociotechnical limitations. -->
-[More Information Needed]
-### Recommendations
-<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
-Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
-## How to Get Started with the Model
-Use the code below to get started with the model.
-[More Information Needed]
-## Training Details
-### Training Data
-<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
-[More Information Needed]
-### Training Procedure
-<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
-#### Preprocessing [optional]
-[More Information Needed]
-#### Training Hyperparameters
-- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
-#### Speeds, Sizes, Times [optional]
-<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
-[More Information Needed]
-## Evaluation
-<!-- This section describes the evaluation protocols and provides the results. -->
-### Testing Data, Factors & Metrics
-#### Testing Data
-<!-- This should link to a Dataset Card if possible. -->
-[More Information Needed]
-#### Factors
-<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
-[More Information Needed]
-#### Metrics
-<!-- These are the evaluation metrics being used, ideally with a description of why. -->
-[More Information Needed]
-### Results
-[More Information Needed]
-#### Summary
-## Model Examination [optional]
-<!-- Relevant interpretability work for the model goes here -->
-[More Information Needed]
-## Environmental Impact
-<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
-Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
-- **Hardware Type:** [More Information Needed]
-- **Hours used:** [More Information Needed]
-- **Cloud Provider:** [More Information Needed]
-- **Compute Region:** [More Information Needed]
-- **Carbon Emitted:** [More Information Needed]
-## Technical Specifications [optional]
-### Model Architecture and Objective
-[More Information Needed]
-### Compute Infrastructure
-[More Information Needed]
-#### Hardware
-[More Information Needed]
-#### Software
-[More Information Needed]
-## Citation [optional]
-<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
-**BibTeX:**
-[More Information Needed]
-**APA:**
-[More Information Needed]
-## Glossary [optional]
-<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
-[More Information Needed]
-## More Information [optional]
-[More Information Needed]
-## Model Card Authors [optional]
-[More Information Needed]
-## Model Card Contact
-[More Information Needed]

 ---
+tags:
+- Protein-Language-Models
+- PLM
+- Ancestral-Sequence-Reconstruction
+- ASR
+- Natural-Language-Processing
+- NLP
+- Geneartive-AI
+- GenAI
+- Biology
+- Bioinformatics
 ---
+# Ancestral sequence reconstruction using generative models
+## Abstract:
+Ancestral sequence reconstruction (ASR) is a foundational task in evolutionary biology, providing insights into the molecular past and guiding studies of protein function and adaptation. Conventional ASR methods rely on a multiple sequence alignment (MSA), a phylogenetic tree, and an evolutionary model. However, alignments and trees are often uncertain, and the evolutionary models typically assume a single, simplified process of substitutions and insertion–deletion (indel) events for all sites. Here, we introduce BetaReconstruct, a generative approach to ASR that builds on recent advances in natural language processing (NLP) and uses a hybrid transformer–state-space architecture. The model was first trained on large-scale simulated datasets with known ancestral sequences and then fine-tuned on empirical protein sequences. Reconstruction requires neither an MSA nor a phylogenetic tree; to reconstruct an internal node, the user only specifies which sequences descend from it (ingroup) and which do not (outgroup). On simulated data, BetaReconstruct achieved accuracy comparable to that of maximum-likelihood and Bayesian methods and inferred fewer false ancestral insertions. Its accuracy remained similar under evolutionary models not seen during training. When different regions of the sequence evolved under different processes, BetaReconstruct was more accurate than maximum-likelihood methods and close to the Bayesian method. In a structural analysis of mammalian proteins, the ancestors reconstructed by BetaReconstruct were, on average, more similar to their descendants than to the outgroup, whereas maximum-likelihood reconstructions were pulled toward the outgroup. This work provides a scalable, alignment-free strategy for ASR that can be adapted to new evolutionary processes by changing the simulations used for training.
+![alt text](https://github.com/technion-cs-nlp/BetaReconstruct/blob/main/outline_image.png)
+Illustration of ASR using BetaReconstruct. (a) The ‘true’ evolutionary dynamics, in which the ancestral sequence “AAMM” evolved along a phylogenetic tree. The leaf sequences are “AAL”, “AYM”, and “ATMMM”. The target internal node, “AAM”, is the ancestor of the ingroup sequences “AAL” and “AYM”, and the sequence "ATMMM" serves as the outgroup. (b) The BetaReconstruct pipeline: (Ⅰ) the unaligned protein sequences are provided as input; (Ⅱ) the sequences are concatenated into a single string, in which the outgroup sequence is followed by a “#” character, ingroup sequences are separated by a “|” character, and a special “<RECONSTRUCT>” token marks the start of the prediction; (Ⅲ) the model processes the input; (Ⅳ) the model generates the ancestral sequence. (c) A typical ASR pipeline, which runs multiple independent tools: (Ⅰ) the unaligned protein sequences are provided as input; (Ⅱ) an alignment program infers an MSA; (Ⅲ) the resulting MSA, including newly introduced gap characters; (Ⅳ) a tree inference program uses the MSA to infer the phylogeny; (Ⅴ) the inferred tree, shown in a Newick format; (Ⅵ) specialized ASR software uses both the MSA and the tree as inputs to perform the reconstruction; (Ⅶ) the final output is the reconstructed ancestral sequence. Joint methods such as BAli-Phy combine steps (Ⅱ)–(Ⅵ).
 ### Model Sources [optional]
 <!-- Provide the basic links for the model. -->
+- **Repository:** [[Github]](https://github.com/technion-cs-nlp/BetaReconstruct)
+- **Paper [optional]:** https://www.biorxiv.org/content/10.64898/2026.01.18.700141v1
 ## Uses
+See Github repository: https://github.com/technion-cs-nlp/BetaReconstruct
+**[IMPORTANT] Note on Generalization**: While we have verified the models' ability to generalize to out-of-distribution data (as detailed in the main text), performance is highest when inference is performed on data distributions similar to those used during training.
+### Tokenization
+The models utilize a data-driven tokenizer specifically optimized for evolutionary related protein sequences. Instead of relying on standard single-amino-acid characters, this approach allows the model to learn and represent frequently occurring sequence patterns as distinct tokens. For a comprehensive technical breakdown of the tokenizer architecture and its training process, please refer to the main text of our publication.
+Related Resources
+ - [BiologicalTokenizers repository](https://github.com/technion-cs-nlp/BiologicalTokenizers): You can find the source code and training scripts for biological tokenizers.
+ - [Research paper](https://academic.oup.com/bioinformatics/article/40/4/btae196/7645044): For detailed methodology and performance benchmarks, please see our paper: *"Effect of tokenization on transformers for biological sequences"* (2024; Bioinformatics).