Instructions to use fabikru/trainer with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use fabikru/trainer with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("fill-mask", model="fabikru/trainer")# pip install -U transformers accelerate # Load model directly from transformers import AutoTokenizer, AutoModelForMaskedLM tokenizer = AutoTokenizer.from_pretrained("fabikru/trainer") model = AutoModelForMaskedLM.from_pretrained("fabikru/trainer", device_map="auto") - Notebooks
- Google Colab
- Kaggle
new_fabikru/half-of-chembl-2025-randomized-smiles-cleaned_modern_smiles_bert_4.5M_character-level_MLM0.3
Browse files- README.md +43 -30
- config.json +4 -4
- model.safetensors +2 -2
- training_args.bin +1 -1
README.md
CHANGED
|
@@ -16,8 +16,8 @@ should probably proofread and complete it, then remove this comment. -->
|
|
| 16 |
|
| 17 |
This model is a fine-tuned version of [](https://huggingface.co/) on an unknown dataset.
|
| 18 |
It achieves the following results on the evaluation set:
|
| 19 |
-
- Loss: 0.
|
| 20 |
-
- Accuracy: 0.
|
| 21 |
|
| 22 |
## Model description
|
| 23 |
|
|
@@ -36,7 +36,7 @@ More information needed
|
|
| 36 |
### Training hyperparameters
|
| 37 |
|
| 38 |
The following hyperparameters were used during training:
|
| 39 |
-
- learning_rate: 0.
|
| 40 |
- train_batch_size: 256
|
| 41 |
- eval_batch_size: 256
|
| 42 |
- seed: 42
|
|
@@ -50,33 +50,46 @@ The following hyperparameters were used during training:
|
|
| 50 |
|
| 51 |
| Training Loss | Epoch | Step | Validation Loss | Accuracy |
|
| 52 |
|:-------------:|:-------:|:-----:|:---------------:|:--------:|
|
| 53 |
-
| No log | 0 | 0 | 4.
|
| 54 |
-
| 0.
|
| 55 |
-
| 0.
|
| 56 |
-
| 0.
|
| 57 |
-
| 0.
|
| 58 |
-
| 0.
|
| 59 |
-
| 0.
|
| 60 |
-
| 0.
|
| 61 |
-
| 0.
|
| 62 |
-
| 0.
|
| 63 |
-
| 0.
|
| 64 |
-
| 0.
|
| 65 |
-
| 0.
|
| 66 |
-
| 0.
|
| 67 |
-
| 0.
|
| 68 |
-
| 0.
|
| 69 |
-
| 0.
|
| 70 |
-
| 0.
|
| 71 |
-
| 0.
|
| 72 |
-
| 0.
|
| 73 |
-
| 0.
|
| 74 |
-
| 0.
|
| 75 |
-
| 0.
|
| 76 |
-
| 0.
|
| 77 |
-
| 0.
|
| 78 |
-
| 0.
|
| 79 |
-
| 0.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 80 |
|
| 81 |
|
| 82 |
### Framework versions
|
|
|
|
| 16 |
|
| 17 |
This model is a fine-tuned version of [](https://huggingface.co/) on an unknown dataset.
|
| 18 |
It achieves the following results on the evaluation set:
|
| 19 |
+
- Loss: 0.1940
|
| 20 |
+
- Accuracy: 0.9309
|
| 21 |
|
| 22 |
## Model description
|
| 23 |
|
|
|
|
| 36 |
### Training hyperparameters
|
| 37 |
|
| 38 |
The following hyperparameters were used during training:
|
| 39 |
+
- learning_rate: 0.008141
|
| 40 |
- train_batch_size: 256
|
| 41 |
- eval_batch_size: 256
|
| 42 |
- seed: 42
|
|
|
|
| 50 |
|
| 51 |
| Training Loss | Epoch | Step | Validation Loss | Accuracy |
|
| 52 |
|:-------------:|:-------:|:-----:|:---------------:|:--------:|
|
| 53 |
+
| No log | 0 | 0 | 4.3643 | 0.0040 |
|
| 54 |
+
| 0.5212 | 0.4302 | 1953 | 0.4259 | 0.8526 |
|
| 55 |
+
| 0.3855 | 0.8604 | 3906 | 0.3514 | 0.8772 |
|
| 56 |
+
| 0.3391 | 1.2905 | 5859 | 0.3167 | 0.8893 |
|
| 57 |
+
| 0.3159 | 1.7207 | 7812 | 0.2958 | 0.8961 |
|
| 58 |
+
| 0.2969 | 2.1509 | 9765 | 0.2806 | 0.9011 |
|
| 59 |
+
| 0.2842 | 2.5811 | 11718 | 0.2707 | 0.9045 |
|
| 60 |
+
| 0.2741 | 3.0112 | 13671 | 0.2610 | 0.9077 |
|
| 61 |
+
| 0.2652 | 3.4414 | 15624 | 0.2548 | 0.9097 |
|
| 62 |
+
| 0.2596 | 3.8716 | 17577 | 0.2476 | 0.9123 |
|
| 63 |
+
| 0.253 | 4.3018 | 19530 | 0.2441 | 0.9135 |
|
| 64 |
+
| 0.2511 | 4.7319 | 21483 | 0.2398 | 0.9145 |
|
| 65 |
+
| 0.2456 | 5.1621 | 23436 | 0.2364 | 0.9158 |
|
| 66 |
+
| 0.2415 | 5.5923 | 25389 | 0.2319 | 0.9173 |
|
| 67 |
+
| 0.2389 | 6.0225 | 27342 | 0.2290 | 0.9185 |
|
| 68 |
+
| 0.236 | 6.4526 | 29295 | 0.2274 | 0.9192 |
|
| 69 |
+
| 0.2323 | 6.8828 | 31248 | 0.2240 | 0.9202 |
|
| 70 |
+
| 0.2296 | 7.3130 | 33201 | 0.2226 | 0.9206 |
|
| 71 |
+
| 0.2282 | 7.7432 | 35154 | 0.2200 | 0.9216 |
|
| 72 |
+
| 0.2254 | 8.1733 | 37107 | 0.2165 | 0.9228 |
|
| 73 |
+
| 0.2221 | 8.6035 | 39060 | 0.2157 | 0.9230 |
|
| 74 |
+
| 0.2216 | 9.0337 | 41013 | 0.2144 | 0.9235 |
|
| 75 |
+
| 0.2204 | 9.4639 | 42966 | 0.2126 | 0.9243 |
|
| 76 |
+
| 0.2172 | 9.8941 | 44919 | 0.2103 | 0.9249 |
|
| 77 |
+
| 0.2149 | 10.3242 | 46872 | 0.2084 | 0.9254 |
|
| 78 |
+
| 0.2142 | 10.7544 | 48825 | 0.2091 | 0.9254 |
|
| 79 |
+
| 0.213 | 11.1846 | 50778 | 0.2065 | 0.9261 |
|
| 80 |
+
| 0.2117 | 11.6148 | 52731 | 0.2054 | 0.9267 |
|
| 81 |
+
| 0.2118 | 12.0449 | 54684 | 0.2035 | 0.9274 |
|
| 82 |
+
| 0.2095 | 12.4751 | 56637 | 0.2042 | 0.9272 |
|
| 83 |
+
| 0.2083 | 12.9053 | 58590 | 0.2024 | 0.9277 |
|
| 84 |
+
| 0.2069 | 13.3355 | 60543 | 0.2027 | 0.9278 |
|
| 85 |
+
| 0.2071 | 13.7656 | 62496 | 0.2004 | 0.9283 |
|
| 86 |
+
| 0.2063 | 14.1958 | 64449 | 0.1991 | 0.9289 |
|
| 87 |
+
| 0.2043 | 14.6260 | 66402 | 0.1998 | 0.9284 |
|
| 88 |
+
| 0.2054 | 15.0562 | 68355 | 0.1972 | 0.9295 |
|
| 89 |
+
| 0.2026 | 15.4863 | 70308 | 0.1973 | 0.9294 |
|
| 90 |
+
| 0.2024 | 15.9165 | 72261 | 0.1956 | 0.9301 |
|
| 91 |
+
| 0.2013 | 16.3467 | 74214 | 0.1952 | 0.9304 |
|
| 92 |
+
| 0.2007 | 16.7769 | 76167 | 0.1945 | 0.9305 |
|
| 93 |
|
| 94 |
|
| 95 |
### Framework versions
|
config.json
CHANGED
|
@@ -17,10 +17,10 @@
|
|
| 17 |
"global_attn_every_n_layers": 1,
|
| 18 |
"global_rope_theta": 160000.0,
|
| 19 |
"hidden_activation": "gelu",
|
| 20 |
-
"hidden_size":
|
| 21 |
"initializer_cutoff_factor": 2.0,
|
| 22 |
"initializer_range": 0.02,
|
| 23 |
-
"intermediate_size":
|
| 24 |
"local_attention": 128,
|
| 25 |
"local_rope_theta": 10000.0,
|
| 26 |
"max_position_embeddings": 502,
|
|
@@ -29,8 +29,8 @@
|
|
| 29 |
"model_type": "modernbert",
|
| 30 |
"norm_bias": false,
|
| 31 |
"norm_eps": 1e-05,
|
| 32 |
-
"num_attention_heads":
|
| 33 |
-
"num_hidden_layers":
|
| 34 |
"pad_token_id": 1,
|
| 35 |
"reference_compile": true,
|
| 36 |
"repad_logits_with_grad": false,
|
|
|
|
| 17 |
"global_attn_every_n_layers": 1,
|
| 18 |
"global_rope_theta": 160000.0,
|
| 19 |
"hidden_activation": "gelu",
|
| 20 |
+
"hidden_size": 256,
|
| 21 |
"initializer_cutoff_factor": 2.0,
|
| 22 |
"initializer_range": 0.02,
|
| 23 |
+
"intermediate_size": 384,
|
| 24 |
"local_attention": 128,
|
| 25 |
"local_rope_theta": 10000.0,
|
| 26 |
"max_position_embeddings": 502,
|
|
|
|
| 29 |
"model_type": "modernbert",
|
| 30 |
"norm_bias": false,
|
| 31 |
"norm_eps": 1e-05,
|
| 32 |
+
"num_attention_heads": 4,
|
| 33 |
+
"num_hidden_layers": 8,
|
| 34 |
"pad_token_id": 1,
|
| 35 |
"reference_compile": true,
|
| 36 |
"repad_logits_with_grad": false,
|
model.safetensors
CHANGED
|
@@ -1,3 +1,3 @@
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
-
oid sha256:
|
| 3 |
-
size
|
|
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:788ba3089867da43cd59920d146c37ea24e323d7d71dceafed99a689abf07c7c
|
| 3 |
+
size 18195880
|
training_args.bin
CHANGED
|
@@ -1,3 +1,3 @@
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
-
oid sha256:
|
| 3 |
size 5905
|
|
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:44106c6ba5dc86ae6f86209e5f3e3e6c5e4acce8d60d4a87d0cbc225634500a1
|
| 3 |
size 5905
|