| --- |
| license: cc-by-2.0 |
| datasets: |
| - allenai/s2orc |
| language: |
| - en |
| pipeline_tag: token-classification |
| --- |
| Another name for this model is sciDeBERta v2[1]. |
| This model is trained from scratch using S2ORC dataset(260GB), which include abstract, body text of papers, on DeBERTa v2. |
| This model achieves the SOTA in NET of SciERC dataset. |
| From this model, MediBioDeBERTa, which continuously leaned from scidebert v2. to medibiodeberta using the data from the domain (bio, medical, chemistry domain data) |
| and additional intermediate fine-tuning for specific blurb benchmark tasks, achieve the 11 rank in the BLURB benchmark. |
|
|
| [1] Eunhui Kim, Yuna Jeong, Myung-seok Choi, "MediBioDeBERTa: BioMedical Language Model with Continous Learning and Intermediate Fine-Tuning, Dec. 2023, IEEE Access" |
