Update README.md

README.md

@@ -1,3 +1,70 @@
 ---
 license: agpl-3.0
+language:
+- en
+base_model:
+- cambridgeltl/SapBERT-from-PubMedBERT-fulltext
+pipeline_tag: feature-extraction
+datasets:
+- FiscaAI/icd10cm-multilabel-prompt
+metrics:
+- name: f1 macro
+  type: f1
+  value: 0.65
+- name: recall micro
+  type: recall
+  value: 0.76
+- name: precision micro
+  type: precision
+  value: 0.65
+- name: subset accuracy
+  type: set accuracy
+  value: 0.26
+library_name: transformers
+tags:
+- icd10
+- healthcare
+- medical
 ---
+
+We finetuned the SapBERT encoder model on the synthetic ICD10 set by [FiscaAI](https://huggingface.co/FiscaAI).
+
+We trimmed the ICD10 codes to ```X##```, i.e. we trimmed the dot, and then kept only the ICD10 codes that had more than 1000 occurrences, after which we kept the texts that had at least 
+one of the remaining ICD10 codes, this resulted in close to 950 ICD10 codes uses as labels for the finetuning with about 700.000 samples.
+
+We use multilabel finetuning with a head of 3 dense layers, with 10% dropout and GELU activation.
+
+We used class weighting with log smoothing.
+
+For the 
+
+TLDR;
+* ICD10 trimmed after dot, A10.42 -> A10
+* minimum_token_count=16
+* minimum_label_count=1000
+* sequence_length=256
+* log-smoothed class weights
+* 3 dense layers, 10% dropout, GELU
+* 5 epochs
+* batch size 16
+* max learning rate 2e-5
+* repeated linear warmup (10.000 steps) + cosine decay (50.000 steps) 
+
+When you load this model for feature extraction or classification use 
+
+```python
+from transformers import AutoModelForSequenceClassification
+
+model = AutoModelForSequenceClassification.from_pretrained(
+    "UMCU/ICD10_classifier_base_English",
+    trust_remote_code=True,
+)
+```
+
+or 
+
+```python
+from transformers import pipeline
+
+le_pipe = pipeline('text-classification',trust_remote_code=True)
+```