Commit History
-
Update README.md
Jonghyun Lee7e55a0d -
Update README.md
Jonghyun Lee3c47826 -
Update README.md
Jonghyun Lee9129738
-
Update README.md
Jonghyun Lee8d08d2f -
Create README.md
Jonghyun Lee# Molecular BERT Pretrained Using ChEMBL Database This model has been pretrained based on the methodology outlined in the paper [Pushing the Boundaries of Molecular Property Prediction for Drug Discovery with Multitask Learning BERT Enhanced by SMILES Enumeration](https://spj.science.org/doi/10.34133/research.0004). While the original model was initially trained using custom code, it has been adapted for use within the Hugging Face Transformers framework in this project. ## Model Details The model architecture utilized is based on BERT. Here are the key configuration details: ``` BertConfig( vocab_size=len(tokenizer_pretrained.vocab), hidden_size=256, num_hidden_layers=8, num_attention_heads=8, intermediate_size=1024, hidden_act="gelu", hidden_dropout_prob=0.1, attention_probs_dropout_prob=0.1, max_position_embeddings=max_seq_len, type_vocab_size=1, pad_token_id=tokenizer_pretrained.vocab["[PAD]"], position_embedding_type="absolute" ) …2d62678 -
add tokenizer
bb74661 -
add model
3ca94a7 -
initial commit
0dd5a4b