Update README.md

README.md

@@ -8,12 +8,12 @@ base_model: BioMistral/BioMistral-7B
 </p>
 
 # BioChat Model
-Source Paper: [BioMistral: A Collection of Open-Source Pretrained Large Language Models for Medical Domains](https://arxiv.org/abs/2402.10373)
+- **Source Paper:** [BioMistral: A Collection of Open-Source Pretrained Large Language Models for Medical Domains](https://arxiv.org/abs/2402.10373)
 
-BioChat is a language model fine-tuned using the ChatDoctor dataset from [ChatDoctor-5k](https://huggingface.co/datasets/LinhDuong/chatdoctor-5k). Specifically designed for medical conversations, BioChat enables users to engage in interactive discussions with a virtual doctor. Whether you are seeking advice about symptoms you are experiencing, exploring possible health conditions, or looking for general medical insights, BioChat is built to assist in a reliable and informative manner.
+- **BioChat** is a language model fine-tuned using the ChatDoctor dataset from [ChatDoctor-5k](https://huggingface.co/datasets/LinhDuong/chatdoctor-5k). Specifically designed for medical conversations, BioChat enables users to engage in interactive discussions with a virtual doctor. Whether you are seeking advice about symptoms you are experiencing, exploring possible health conditions, or looking for general medical insights, BioChat is built to assist in a reliable and informative manner.
+- **NOTE**: We are still in the early stages of exploring the generation capabilities and limitations of this model. It is important to emphasize that its text generation features are intended solely for research purposes and are not yet suitable for production use.
 
-In addition to its ability to act as a virtual medical consultant, we are still in the early stages of exploring the generation capabilities and limitations of this model. It is important to emphasize that its text generation features are intended solely for research purposes and are not yet suitable for production use. By releasing this model, we aim to drive advancements in biomedical NLP applications and promote best practices for the responsible training and deployment of domain-specific language models. Ensuring reliability, accuracy, and explainability remains a top priority for us.
-- **Finetuned from model [optional]:** [BioMistral-7B](https://huggingface.co/BioMistral/BioMistral-7B).
+- **Finetuned from model:** [BioMistral-7B](https://huggingface.co/BioMistral/BioMistral-7B).
 
 # Using BioChat
 
@@ -36,40 +36,29 @@ model = PeftModel.from_pretrained(model, "Indah1/BioChat10")
 
 # Fine-Tuning Data
 
-
-[More Information Needed]
+The fine-tuning data used for BioChat is derived from the [ChatDoctor-5k](https://huggingface.co/datasets/LinhDuong/chatdoctor-5k) dataset. This dataset contains a collection of medical conversations tailored to simulate doctor-patient interactions, making it an ideal source for training a medical conversational model. The dataset was carefully curated to ensure relevance and diversity in medical topics.
 
 ### Training Procedure
 
 <!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
 
-#### Preprocessing [optional]
-
-[More Information Needed]
-
-
 #### Training Hyperparameters
 
 |     Hyperparameter    |             Value            |
 |:-------------------:|:----------------------------------:|
-|    Rank    | 16 | 
+|    Weigh Decay   | 0.01 | 
 |  Learning Rate | 2e-05 |
 |  Training Batch Size | 8  |
 | Batch Size | 8 |
 | Number of GPU | 1 |
 | Optimizer | AdamW_8Bit |
+| Warm Up Ratio | 0.03 |
 | Scheduler | Cosine |
 | Number of Epoch | 10 |
 
-#### Speeds, Sizes, Times [optional]
-
-<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
-
-[More Information Needed]
-
 ## Evaluation
 
-<!-- This section describes the evaluation protocols and provides the results. -->
+To determine the best model for fine-tuning, I used *perplexity* as a metric to evaluate performance and select the most optimal version. By leveraging the model's capabilities, I aim to evaluate its behavior and responses using tools like the *Word Embedding Association Test (WEAT)*. It is important to emphasize that its text generation features are intended solely for research purposes and are not yet suitable for production use. By releasing this model, we aim to drive advancements in biomedical NLP applications and contribute to best practices for the responsible development of domain-specific language models. Ensuring reliability, fairness, accuracy, and explainability remains a top priority for us.
 
 ### Testing Data, Factors & Metrics