Update README.md

README.md

@@ -18,13 +18,17 @@ tags:
 
 **PolyTAO Student** is a property-conditioned Transformer model developed as part of my M.Sc. thesis, **“Knowledge-Distilled Transformers for Property-Conditioned Polymer Generation.”**
 
-The project investigates whether smaller Transformer models can retain useful generative behavior after knowledge distillation from a larger property-conditioned teacher model.
+The project investigates whether a compressed Transformer student can retain useful generative behavior through knowledge distillation from a larger property-conditioned teacher model.
 
-The teacher was initialized from the publicly available **`hkqiu/PolyTAO-BigSMILES_Version`** checkpoint and subsequently trained with molecular-property conditioning. The trained teacher was then frozen and used for feature-level knowledge distillation into smaller student architectures.
+The teacher was initialized from the publicly available **`hkqiu/PolyTAO-BigSMILES_Version`** checkpoint and subsequently trained with molecular-property conditioning. The trained teacher was then frozen and used for feature-level knowledge distillation.
 
-Multiple student capacities were investigated by reducing the number of encoder and decoder layers while keeping the main Transformer configuration consistent.
+Multiple student configurations were investigated during the research. The checkpoint published in this repository corresponds to the **Student-20 configuration** used in the experiments.
 
-The checkpoint published in this repository corresponds to the **20% student capacity configuration**.
+The released checkpoint contains:
+
+- **2 encoder layers**
+- **12 decoder layers**
+- **hidden dimension (`d_model`) = 768**
 
 > **Research model:** This checkpoint is provided for research and reproducibility purposes. It is not intended for production use or for making chemical or experimental decisions.
 
@@ -34,37 +38,33 @@ The checkpoint published in this repository corresponds to the **20% student cap
 
 The central research question behind PolyTAO is:
 
-> **How does reducing Transformer model capacity affect the quality, chemical validity, and property fidelity of generated polymers?**
+> **How does Transformer compression affect the chemical validity, property fidelity, and generative behavior of property-conditioned polymer generation models?**
 
-Rather than training only one compressed model, the research evaluates multiple student capacities to investigate the trade-off between model compression and generative behavior.
+Rather than evaluating only one compressed model, the research investigated multiple student configurations to study the relationship between model capacity and generative performance.
 
-The work focuses particularly on whether knowledge distilled from a larger Transformer can preserve useful molecular-property information when the depth of the student architecture is substantially reduced.
+The work focuses particularly on whether internal representations learned by a larger property-conditioned teacher can be transferred to a smaller student through feature-level knowledge distillation.
 
 ---
 
 ## Model Lineage
 
-The model development pipeline can be summarized as:
+The overall model-development pipeline is:
 
 ```text
 hkqiu/PolyTAO-BigSMILES_Version
               │
               ▼
    Property-Conditioned Teacher
+              │
               │
               │ Feature-Level
               │ Knowledge Distillation
               ▼
        PolyTAO Students
               │
-      ┌───────┼────────┐
-      ▼       ▼        ▼
-   larger   medium    smaller
-  students  students  students
-              │
               ▼
-    Capacity–Performance
-         Evaluation
+   Capacity–Performance
+        Evaluation
 ```
 
 The publicly available `hkqiu/PolyTAO-BigSMILES_Version` checkpoint therefore serves as the starting point for the teacher model rather than being the final student model itself.
@@ -75,13 +75,13 @@ The publicly available `hkqiu/PolyTAO-BigSMILES_Version` checkpoint therefore se
 
 The PolyTAO teacher is based on a **T5-style encoder-decoder Transformer architecture**.
 
-The teacher was initialized using:
+The teacher was initialized from:
 
 ```text
 hkqiu/PolyTAO-BigSMILES_Version
 ```
 
-and loaded using Hugging Face Transformers.
+using Hugging Face Transformers.
 
 The teacher architecture contains:
 
@@ -91,14 +91,16 @@ Decoder layers: 12
 d_model:        768
 ```
 
-A custom property-conditioning mechanism was added to the model.
+A custom property-conditioning mechanism was added to the Transformer.
 
-The teacher receives both:
+The teacher receives:
 
 1. a tokenized polymer representation, and
 2. a 15-dimensional molecular-property vector.
 
-The property vector is mapped into the Transformer's hidden space using a learned linear projection:
+The property vector is projected into the Transformer's hidden space using a learned linear projection.
+
+Conceptually:
 
 ```text
 15 Molecular Properties
@@ -115,56 +117,52 @@ Normalized Property Embedding
           ▼
 Property-Conditioned
 Encoder Representation
+          │
+          ▼
+        Decoder
 ```
 
-The property embedding is added to the encoder hidden states before decoding.
+The property embedding is added to the encoder hidden representation before decoding.
 
-After training, the Transformer checkpoint, tokenizer, and learned property projection are saved separately.
+After training, the Transformer checkpoint, tokenizer, and learned property projection are stored separately.
 
 ---
 
-## Student Architecture
+## Published Student Architecture
 
-Student models are constructed from the teacher configuration while reducing the number of encoder and decoder layers.
+The checkpoint released in this repository corresponds to the **Student-20 experimental configuration**.
 
-Student depth is calculated as:
+The architecture stored in the released checkpoint is:
 
-```text
-student_layers = max(
-    1,
-    round(teacher_layers × capacity / 100)
-)
-```
-
-The main Transformer configuration is otherwise kept consistent so that the effect of reducing model depth can be investigated more directly.
-
-The student Transformer is initialized independently rather than directly copying the trained teacher weights.
+| Component | Teacher | Published Student-20 |
+|---|---:|---:|
+| Encoder layers | 12 | **2** |
+| Decoder layers | 12 | **12** |
+| Hidden dimension | 768 | **768** |
+| Attention heads | 12 | **12** |
 
-Knowledge is subsequently transferred from the frozen teacher during student training.
+Therefore, the released checkpoint applies substantial compression to the **encoder depth**, reducing it from 12 layers to 2 layers, while retaining the 12-layer decoder configuration.
 
-### Published Checkpoint
-
-The checkpoint published in this repository corresponds to:
+The architecture reported here is based on the actual configuration of the released checkpoint:
 
 ```text
-Student capacity: 20%
-
-Teacher encoder layers: 12
-Student encoder layers: 2
-
-Teacher decoder layers: 12
-Student decoder layers: 2
+num_layers = 2
+num_decoder_layers = 12
+d_model = 768
+num_heads = 12
 ```
 
-This represents an aggressive reduction in Transformer depth and is used to study how much useful generative behavior can be retained through knowledge distillation.
+The student Transformer is initialized independently rather than directly copying the trained teacher weights.
+
+Knowledge is transferred from the frozen teacher during training through feature-level knowledge distillation.
 
 ---
 
 ## Property Conditioning
 
-Both teacher and student models use the same property-conditioning mechanism.
+Both teacher and student use a molecular-property conditioning mechanism.
 
-Generation is conditioned on **15 molecular properties**:
+Generation is conditioned on the following **15 molecular properties**:
 
 1. `MolWt`
 2. `HeavyAtomCount`
@@ -184,106 +182,110 @@ Generation is conditioned on **15 molecular properties**:
 
 During preprocessing, normalized versions of these properties are used when available.
 
-The property vector is projected into the Transformer hidden space:
+The conditioning pipeline is:
 
 ```text
-Property Vector (15 dimensions)
-              │
-              ▼
-        Linear Projection
-              │
-              ▼
-        L2 Normalization
-              │
-              ▼
-        Property Scaling
-              │
-              ▼
-      Property Embedding
-              │
-              ▼
-     Encoder Hidden States
-              │
-              ▼
-  Conditioned Representation
+Property Vector
+(15 dimensions)
+       │
+       ▼
+Linear Projection
+       │
+       ▼
+L2 Normalization
+       │
+       ▼
+Property Scaling
+       │
+       ▼
+Property Embedding
+       │
+       ▼
+Encoder Hidden States
+       │
+       ▼
+Conditioned Encoder
+Representation
 ```
 
 The property embedding is scaled before being added to the encoder representation.
 
-The training configuration uses:
+The training implementation uses:
 
 ```text
 Property scale = 0.05
 ```
 
+The learned student property projection is distributed separately as:
+
+```text
+property_proj_student.pt
+```
+
 ---
 
 ## Knowledge Distillation
 
 The student is trained using a frozen property-conditioned teacher.
 
-For each training sample, both models receive the same tokenized polymer representation and molecular-property vector.
+For each training sample, the teacher and student receive the same tokenized polymer representation and molecular-property vector.
 
-The teacher generates conditioned encoder hidden representations.
+The teacher produces a conditioned encoder representation.
 
-The student simultaneously learns:
+The student learns both:
 
 - the sequence-generation task, and
 - the internal representation produced by the teacher.
 
 ### Sequence Generation Loss
 
-The student is trained using the standard sequence-to-sequence cross-entropy loss.
-
-### Feature-Level Knowledge Distillation
+The student is trained using the standard sequence-to-sequence cross-entropy objective.
 
-The conditioned encoder representation produced by the student is compared with the corresponding teacher representation.
+### Feature-Level Distillation Loss
 
-Mean Squared Error (MSE) is used for the feature-level distillation objective.
+The student encoder representation is compared with the corresponding teacher encoder representation using Mean Squared Error (MSE).
 
 Conceptually:
 
 ```text
-Teacher
-  │
-  ▼
-Conditioned Teacher Hidden States
-                 │
-                 │
-                 │ MSE
-                 │
-                 ▼
-Conditioned Student Hidden States
-  ▲
-  │
-Student
+Property-Conditioned Teacher
+             │
+             ▼
+   Teacher Hidden States
+             │
+             │
+             │ MSE Distillation
+             │
+             ▼
+   Student Hidden States
+             ▲
+             │
+Property-Conditioned Student
 ```
 
-The training objective combines the sequence-generation and knowledge-distillation objectives.
-
-Conceptually:
+The implemented student training objective is:
 
 ```text
 Total Loss
     =
-Sequence Generation Loss
+Cross-Entropy Loss
     +
-α × Knowledge Distillation Loss
+α × Knowledge-Distillation Loss
 ```
 
-with:
+where:
 
 ```text
 α = 0.5
 ```
 
-During student training, the teacher remains frozen.
+The teacher remains frozen during student training.
 
 ---
 
 ## Training Configuration
 
-### Teacher
+### Teacher Training
 
 The teacher training configuration includes:
 
@@ -298,13 +300,13 @@ The teacher training configuration includes:
 | Validation ratio | `0.1` |
 | Precision | `32-bit` |
 
-The teacher is trained using PyTorch Lightning.
+The teacher was trained using PyTorch Lightning.
 
-Training and validation losses are logged during training.
+The teacher model and tokenizer were saved using the Hugging Face `save_pretrained()` format, while the learned property projection was stored separately.
 
 ---
 
-### Student
+### Student Training
 
 The student training configuration includes:
 
@@ -318,7 +320,6 @@ The student training configuration includes:
 | Property scale | `0.05` |
 | Validation ratio | `0.1` |
 | Random seed | `42` |
-| Precision | `32-bit` |
 
 The implementation uses:
 
@@ -326,12 +327,7 @@ The implementation uses:
 - PyTorch Lightning
 - Hugging Face Transformers
 
-Training logs include:
-
-- total training loss
-- cross-entropy loss
-- knowledge-distillation loss
-- validation loss
+The training process monitors the sequence-generation and knowledge-distillation objectives together with validation loss.
 
 ---
 
@@ -339,9 +335,9 @@ Training logs include:
 
 The research dataset was derived from the **Open Macromolecular Genome (OMG)** polymer dataset.
 
-The experimental dataset contains approximately **99,000 polymer samples**, where each sample contains a polymer representation together with molecular-property information.
+The experimental dataset contains approximately **99,000 polymer samples**, with each sample containing a polymer representation together with molecular-property information.
 
-The preprocessing pipeline includes:
+The preprocessing workflow includes:
 
 ```text
 Raw Polymer Data
@@ -362,30 +358,30 @@ Polymer Tokenization
 Model Input
 ```
 
-The molecular descriptors used for conditioning and evaluation are calculated using **RDKit**.
+Molecular descriptors used for conditioning and evaluation were calculated using **RDKit**.
 
-The dataset itself is **not distributed in this model repository**.
+The research dataset itself is **not distributed in this model repository**.
 
 ---
 
 ## Evaluation
 
-The generated polymer representations are evaluated using several complementary metrics.
+Generated polymer representations are evaluated using several complementary metrics.
 
-Evaluation considers both:
+The evaluation considers:
 
 1. whether generated structures are chemically valid according to RDKit, and
-2. whether the molecular properties of valid generated structures agree with the requested property conditions.
+2. how closely the properties of valid generated structures correspond to the conditioning targets and reference distributions.
 
 ---
 
 ## Chemical Validity
 
-Generated sequences are cleaned and subsequently parsed and sanitized using RDKit.
+Generated sequences are parsed and sanitized using RDKit.
 
-A generated structure is counted as valid only when RDKit can successfully construct and sanitize the molecule.
+A generated structure is counted as valid only when RDKit can successfully construct and sanitize the corresponding molecule.
 
-For the final evaluation of the **20% student configuration** published here:
+For the final evaluation associated with the published **Student-20 checkpoint**:
 
 | Metric | Result |
 |---|---:|
@@ -393,7 +389,7 @@ For the final evaluation of the **20% student configuration** published here:
 | Valid samples | 250 |
 | **Chemical validity** | **50.0%** |
 
-Only valid generated structures are included in downstream molecular-property evaluation.
+Only chemically valid generated structures are included in downstream molecular-property evaluation.
 
 ---
 
@@ -401,37 +397,35 @@ Only valid generated structures are included in downstream molecular-property ev
 
 For valid generated structures, the 15 molecular properties are recalculated using RDKit.
 
-Generated properties are then compared with the corresponding conditioning targets.
-
-The evaluation uses four complementary metrics.
+Generated properties are then compared with the target/reference properties using several complementary metrics.
 
 ### Mean Absolute Error — MAE
 
-Measures the average absolute difference between generated and target property values.
+MAE measures the average absolute difference between generated and target property values.
 
 Lower values indicate smaller average errors.
 
 ### Root Mean Squared Error — RMSE
 
-Measures prediction error while giving greater weight to larger deviations.
+RMSE measures property error while giving greater weight to larger deviations.
 
 Lower values indicate smaller errors.
 
 ### Pearson Correlation
 
-Measures the linear relationship between generated and target property values.
+Pearson correlation measures the linear relationship between generated and target property values.
 
 Higher positive correlation indicates stronger agreement between changes in the target property and changes in the generated property.
 
 ### Maximum Mean Discrepancy — MMD
 
-MMD compares the distributions of generated and target property values using an RBF kernel.
+MMD is used to compare generated and reference property distributions.
 
-Lower MMD indicates closer distributional agreement.
+Lower values indicate closer distributional agreement under the selected kernel.
 
-However, MMD should **not be interpreted in isolation**.
+However, **MMD should not be interpreted in isolation**.
 
-A low MMD value does not necessarily imply superior generative performance. A compressed model may, for example, generate less diverse or structurally simpler outputs whose aggregate property distributions happen to resemble the target distribution.
+A low MMD value does not necessarily imply superior generative performance. A compressed model may produce simpler or less diverse structures whose aggregate property distributions nevertheless resemble the reference distribution.
 
 For this reason, MMD is interpreted jointly with:
 
@@ -445,7 +439,7 @@ For this reason, MMD is interpreted jointly with:
 
 ## Distribution Analysis
 
-Selected molecular-property distributions can be visualized by comparing valid generated samples with the corresponding reference distribution.
+Selected molecular-property distributions from valid generated molecules are compared with the corresponding test distributions.
 
 ### Molecular Weight
 
@@ -459,11 +453,13 @@ Selected molecular-property distributions can be visualized by comparing valid g
 
 ![Generated vs Test — Ring Count](RingCount_hist.png)
 
-These plots complement the numerical metrics and help illustrate whether the generated structures reproduce important aspects of the target property distributions.
+The generated distributions overlap substantially with the central regions of the reference distributions for these selected properties.
+
+At the same time, the generated distributions can be narrower and may underrepresent portions of the reference tails. These plots should therefore be interpreted as a qualitative complement to the numerical metrics rather than as evidence of complete distributional equivalence.
 
 ---
 
-## Complete Experimental Pipeline
+## Experimental Pipeline
 
 ```text
 Open Macromolecular Genome
@@ -480,7 +476,6 @@ Open Macromolecular Genome
  └────────────┬────────────┘
               │
               ▼
-   Base Transformer Model
 hkqiu/PolyTAO-BigSMILES_Version
               │
               ▼
@@ -490,6 +485,7 @@ hkqiu/PolyTAO-BigSMILES_Version
      Teacher Hidden States
               │
               │
+              │ Feature-Level
               │ Knowledge Distillation
               ▼
  Property-Conditioned Student
@@ -504,15 +500,15 @@ hkqiu/PolyTAO-BigSMILES_Version
  Valid Generated Structures
               │
               ▼
- RDKit Property Recalculation
+ RDKit Property Calculation
               │
               ▼
  ┌───────────────────────────┐
+ │ Chemical Validity         │
  │ MAE                       │
  │ RMSE                      │
  │ MMD                       │
  │ Pearson Correlation       │
- │ Chemical Validity         │
  └─────────────┬─────────────┘
                │
                ▼
@@ -522,19 +518,29 @@ hkqiu/PolyTAO-BigSMILES_Version
 
 ---
 
-## Model Files
+## Repository Files
 
-The complete property-conditioned student model requires more than the standard Transformer checkpoint.
-
-The model repository may contain files such as:
+The repository contains the Transformer checkpoint together with the files required for tokenization and property conditioning.
 
 ```text
-config.json
-generation_config.json
-model.safetensors
-tokenizer_config.json
-tokenizer files
-property_proj_student.pt
+PolyTAO-Student/
+│
+├── README.md
+│
+├── config.json
+├── generation_config.json
+├── model.safetensors
+│
+├── property_proj_student.pt
+│
+├── tokenizer.json
+├── tokenizer_config.json
+├── special_tokens_map.json
+├── added_tokens.json
+│
+├── MolWt_hist.png
+├── HeavyAtomCount_hist.png
+└── RingCount_hist.png
 ```
 
 The standard Transformer weights are stored in:
@@ -543,35 +549,64 @@ The standard Transformer weights are stored in:
 model.safetensors
 ```
 
-The learned property-conditioning projection is stored separately in:
+The learned molecular-property projection is stored separately in:
 
 ```text
 property_proj_student.pt
 ```
 
-This projection maps the 15-dimensional molecular-property vector into the Transformer hidden space.
+This projection maps the 15-dimensional molecular-property vector into the Transformer's hidden representation.
 
-Because the property-conditioning mechanism is implemented outside the standard T5 forward interface, loading only the Transformer checkpoint is **not sufficient to reproduce property-conditioned generation**.
+Because property conditioning is implemented outside the standard T5 forward interface, loading only `model.safetensors` is **not sufficient to reproduce property-conditioned generation**.
 
 ---
 
-## Usage
+## Loading the Model
+
+The Transformer checkpoint and tokenizer can be loaded directly from Hugging Face:
+
+```python
+from transformers import AutoTokenizer, AutoModelForSeq2SeqLM
+
+MODEL_ID = "FahimehBahman/PolyTAO-Student"
+
+tokenizer = AutoTokenizer.from_pretrained(MODEL_ID)
+
+model = AutoModelForSeq2SeqLM.from_pretrained(MODEL_ID)
 
-This model requires a custom property-conditioning step.
+print("Encoder layers:", len(model.encoder.block))
+print("Decoder layers:", len(model.decoder.block))
+print("Hidden dimension:", model.config.d_model)
+```
 
-It should therefore not be treated as a standard text-to-text T5 model that can reproduce the research pipeline using only a generic Hugging Face `pipeline()` call.
+For the published checkpoint, the expected architecture is:
 
-The basic loading procedure is:
+```text
+Encoder layers: 2
+Decoder layers: 12
+Hidden dimension: 768
+```
+
+---
+
+## Property-Conditioned Usage
+
+Property-conditioned inference additionally requires the learned property projection.
+
+A simplified loading example is:
 
 ```python
 import torch
 import torch.nn as nn
 
+from huggingface_hub import hf_hub_download
 from transformers import AutoTokenizer, AutoModelForSeq2SeqLM
 
 MODEL_ID = "FahimehBahman/PolyTAO-Student"
 
-device = "cuda" if torch.cuda.is_available() else "cpu"
+device = torch.device(
+    "cuda" if torch.cuda.is_available() else "cpu"
+)
 
 tokenizer = AutoTokenizer.from_pretrained(MODEL_ID)
 
@@ -585,50 +620,54 @@ property_proj = nn.Linear(
     15,
     model.config.d_model
 ).to(device)
-```
 
-The learned weights from:
+projection_path = hf_hub_download(
+    repo_id=MODEL_ID,
+    filename="property_proj_student.pt"
+)
+
+state_dict = torch.load(
+    projection_path,
+    map_location=device
+)
 
-```text
-property_proj_student.pt
+property_proj.load_state_dict(state_dict)
+property_proj.eval()
 ```
 
-must then be loaded into `property_proj`.
-
-Property-conditioned generation requires the following steps:
+The complete conditioning procedure is:
 
 ```text
-Polymer Input
-      │
-      ▼
- Tokenization
-      │
-      ▼
-Transformer Encoder
-      │
-      ▼
-Encoder Hidden States
-      ▲
-      │
-15 Molecular Properties
-      │
-      ▼
-Property Projection
-      │
-      ▼
-Normalized Property Embedding
-      │
-      ▼
+15-Dimensional Property Vector
+              │
+              ▼
+      Learned Projection
+              │
+              ▼
+        L2 Normalization
+              │
+              ▼
+       Scale by 0.05
+              │
+              ▼
+     Property Embedding
+              │
+              ▼
+      Encoder Hidden States
+              │
+              ▼
 Conditioned Encoder Representation
-      │
-      ▼
-Autoregressive Generation
-      │
-      ▼
-Generated Polymer
+              │
+              ▼
+     Autoregressive Decoder
+              │
+              ▼
+       Generated Polymer
 ```
 
-The complete training and evaluation implementation is provided in the accompanying PolyTAO source-code repository.
+The same property normalization used during training must be applied to conditioning values before generation.
+
+Therefore, arbitrary raw molecular-property values should **not** be passed directly to the model without reproducing the training preprocessing and normalization procedure.
 
 ---
 
@@ -638,13 +677,13 @@ PolyTAO Student is intended primarily for research involving:
 
 - knowledge distillation
 - Transformer model compression
-- property-conditioned generation
+- property-conditioned molecular generation
 - polymer generative modeling
-- capacity–performance trade-offs
 - efficient generative Transformer architectures
+- capacity–performance analysis
 - evaluation of compressed generative models
 
-The checkpoint can also serve as a reproducible artifact for studying aggressive Transformer depth reduction under feature-level knowledge distillation.
+The checkpoint can also serve as a research artifact for studying feature-level knowledge transfer between Transformer models.
 
 ---
 
@@ -658,10 +697,13 @@ This model has several important limitations.
 - Generated structures have not been experimentally validated.
 - RDKit-based validation is a computational validity check rather than experimental verification.
 - Property conditioning is limited to the molecular descriptors used during training.
+- Correct property-conditioned inference requires reproducing the property normalization used during training.
 - Performance depends on preprocessing and decoding configuration.
+- Property metrics are calculated only for generated structures that pass the validity checks.
 - Distributional similarity does not guarantee structural diversity.
-- MMD should not be interpreted independently of validity and other evaluation metrics.
-- Results from this model should not be used directly for safety-critical, medical, industrial, or experimental chemical decisions without independent expert validation.
+- MMD should not be interpreted independently of chemical validity and other evaluation metrics.
+- The released Student-20 checkpoint contains a 2-layer encoder and a 12-layer decoder; the architecture should therefore be interpreted according to the released checkpoint rather than as a uniform 20% reduction of every Transformer component.
+- Results should not be used directly for safety-critical, medical, industrial, or experimental chemical decisions without independent expert validation.
 
 ---
 
@@ -671,7 +713,7 @@ PolyTAO was developed as part of the M.Sc. thesis:
 
 **Knowledge-Distilled Transformers for Property-Conditioned Polymer Generation**
 
-The research investigates the effect of Transformer capacity reduction on property-conditioned polymer generation.
+The research investigates how Transformer compression and knowledge distillation affect property-conditioned polymer generation.
 
 The work combines:
 
@@ -679,12 +721,12 @@ The work combines:
 - molecular-property conditioning
 - feature-level knowledge distillation
 - model compression
-- computational chemistry descriptors
+- computational molecular descriptors
 - systematic quantitative evaluation
 
-Multiple student configurations were trained and evaluated to investigate how model depth affects generative behavior and property fidelity.
+Multiple student configurations were investigated as part of the broader research.
 
-The published checkpoint represents the **20% capacity student configuration**.
+The checkpoint released in this repository corresponds to the **Student-20 experimental configuration**, whose saved architecture contains **2 encoder layers and 12 decoder layers**.
 
 ---