ICD2BERT / README.md
JonasHri's picture
added gated repo prompt and fields
b83d318 verified
|
Raw History Blame Contribute Delete
1.75 kB
metadata
license: gpl-3.0
pipeline_tag: feature-extraction
base_model:
  - google-bert/bert-base-uncased
extra_gated_prompt: >-
  Access requires a verified CITI training certificate using the same process
  outlined by PhysioNet (see https://physionet.org/about/citi-course/). Please
  complete the 'Data or Specimens Only Research' course and please provide proof
  via the verification URL, which takes the form
  https://www.citiprogram.org/verify/?XXXXXX. You agree to not use the model to
  conduct experiments that cause harm to human subjects.
extra_gated_fields:
  Full Name: text
  Email: text
  Affiliation: text
  CITI Certification Verification URL: text

ICD2BERT

Pretrained model on electronic health records on the MIMIC-IV dataset using a masked language modeling (MLM) objective.

Quickstart

from transformers import BertForMaskedLM, BertTokenizer

model = BertForMaskedLM.from_pretrained(
    "PredLMed/ICD2BERT",
    subfolder="model",
)

tokenizer = BertTokenizer.from_pretrained(
    "PredLMed/ICD2BERT",
    subfolder="tokenizer",
)


def encode(icd_sequence: str, model: BertForMaskedLM, tokenizer: BertTokenizer):
    inputs = tokenizer(
        icd_sequence,
        return_tensors="pt",
        truncation=True,
        padding=True,
        max_length=512,
    )

    outputs = model(**inputs, output_hidden_states=True)

    embedding = outputs.hidden_states[-1][:, 0, :]
    return embedding


icd_sequence = (
    [
        "d_10_T1404 d_10_I82532 d_10_Z7901 d_10_Z86711 d_10_S0628 d_10_I872 d_10_F329 d_10_E669 d_10_Z6834 d_10_Z590 ",
        "p_09_5122 d_09_57400 d_09_V6441 d_09_78829 d_09_53081 ",
    ],
)

embedding = encode(icd_sequence, model=model, tokenizer=tokenizer)
print(embedding[0])
print(embedding[1])