elucidator8918/clinical-ehr-prototype-0.1

Overview

This model, elucidator8918/clinical-ehr-prototype-0.1, is tailored for clinical documentation, based on the Mistral-7B-Instruct-v0.1-sharded architecture fine-tuned on the Asclepius-Synthetic-Clinical-Notes dataset.

Key Information

• Model Name: Mistral-7B-Instruct-v0.1-sharded
• Fine-tuned Model Name: elucidator8918/apigen-prototype-0.1
• Dataset: starmpcc/Asclepius-Synthetic-Clinical-Notes
• Language: English (en)

Model Details

• LoRA Parameters (QLoRA):

  • LoRA attention dimension: 64
  • Alpha parameter for LoRA scaling: 16
  • Dropout probability for LoRA layers: 0.1

• bitsandbytes Parameters:

  • Activate 4-bit precision base model loading
  • Compute dtype for 4-bit base models: float16
  • Quantization type: nf4
  • Activate nested quantization for 4-bit base models: No

• TrainingArguments Parameters:

  • Number of training epochs: 1
  • Batch size per GPU for training: 4
  • Batch size per GPU for evaluation: 4
  • Gradient accumulation steps: 1
  • Enable gradient checkpointing: Yes
  • Maximum gradient norm: 0.3
  • Initial learning rate: 2e-4
  • Weight decay: 0.001
  • Optimizer: paged_adamw_32bit
  • Learning rate scheduler type: cosine
  • Warm-up ratio: 0.03
  • Group sequences into batches with the same length: Yes

Usage

• Example Code (API Generation):

from transformers import pipeline

pipe = pipeline(task="text-generation", model=model, tokenizer=tokenizer)

# Run text generation pipeline with our next model
prompt = """
You are an intelligent clinical language model.
Below is a snippet of patient's electronic health record note and a following instruction with question from healthcare professional.
Write a response that appropriately completes the instruction.
The response should provide the accurate answer to the instruction, while being concise.

### Instruction:
Abbreviation Expansion

### Patient's Electronic Health Record Note:
Hospital Course: 

This 66-year-old male patient was admitted due to an ischemic left-hemispheric stroke in addition to a dry cough and fever. The patient tested positive for SARS-CoV-2 and experienced severe ARDS, resulting in intubation and ICU admission. The patient underwent veno-venous extracorporeal membrane oxygenation and physical therapy was initiated to focus on perception training, movement exercises, airway-clearing techniques, dysphagia therapy, and mobilization. Despite a trial of sedation cessation, the patient remained somnolent and unable to communicate or follow commands. A side-edge positioning was initiated in combination with intensive exercise training including trunk and head control. Muscle tone and strength remained severely reduced, particularly on his hemiplegic side, and a second SOEB trial failed. Occupational therapy was involved to support functional initiation of upper limb movements and to integrate perception-training into activities of daily living. Currently, the patient remains functionally dependent, tolerates spontaneous breathing trials, and is alert during therapy, although he cannot communicate. He is considered stable and functionally dependent (CPAx 6/50).

### Question:
What are the abbreviated terms in the given discharge summary that require expansion?
"""
result = pipe(f"[INST] {prompt} [/INST]",max_length=584)[0]['generated_text']
start_index = result.find("[/INST]") + len("[/INST]")
end_index = result.find("'", start_index)
response = result[start_index:end_index]
print(response)

• Output Generation:

The abbreviated terms in the given discharge summary that requires expansion are SARS-CoV-2, ARDS, ICU, SOEB, CPAx, and CPAx 6/50.

### Response:
The abbreviated terms in the given discharge summary that requires expansion are SARS-CoV-2, ARDS, ICU, SOEB, CPAx, and CPAx 6/50. SARS-CoV-2 stands for severe acute respiratory syndrome coronavirus 2, ARDS stands for acute respiratory distress syndrome, ICU stands for intensive care unit, SOEB stands for spontaneous breathing exercise, CPAx stands for Canadian Physical Activity Assessment, and CPAx 6/50 stands for a score of 6 out of 50 on the Canadian Physical Activity Assessment

License

This model is released under the MIT License.