Update README.md

README.md

@@ -31,9 +31,60 @@ nor has it been tested on other downstream tasks.
 # Architecture description
 TODO: get model architecture like on the model card of the previous model
 
-Don't forget to describe that during pretraining, the --mlm-prob parameter has been set to 0.4, in accordance to [paper](INSERT LINK)
-
-Parameters used: epochs=19, eval_steps=10000, batch_size=512, gradient_accumulation_steps=1, mlm_prob=0.4, bf16=True, tf32=True
+Full architecture:
+`[
+  {
+    "idx": 0,
+    "name": "0",
+    "path": "0_ASMTransformerModule",
+    "type": "asmtransformers.models.asmsentencebert.ASMTransformerModule"
+  },
+  {
+    "idx": 1,
+    "name": "1",
+    "path": "1_Pooling",
+    "type": "sentence_transformers.sentence_transformer.modules.pooling.Pooling"
+  }
+]`
+
+Parameters used: 
+  `{
+    "architectures": [
+      "BertForMaskedLM"
+    ],
+    "attention_probs_dropout_prob": 0.1,
+    "classifier_dropout": null,
+    "hidden_act": "gelu",
+    "hidden_dropout_prob": 0.1,
+    "hidden_size": 768,
+    "initializer_range": 0.02,
+    "intermediate_size": 3072,
+    "layer_norm_eps": 1e-12,
+    "max_position_embeddings": 512,
+    "model_type": "bert",
+    "num_attention_heads": 12,
+    "num_hidden_layers": 12,
+    "pad_token_id": 0,
+    "position_embedding_type": "absolute",
+    "torch_dtype": "float32",
+    "transformers_version": "4.12.5",
+    "type_vocab_size": 2,
+    "use_cache": true,
+    "vocab_size": 6161
+  }`
+
+Preraining parameters used: 
+`{
+  "epochs"=19, 
+  "eval_steps"=10000, 
+  "batch_size"=512, 
+  "gradient_accumulation_steps"=1, 
+  "mlm_prob"=0.4, 
+  "bf16"=True, 
+  "tf32"=True
+  }`
+
+Note that during pretraining, the --mlm-prob parameter has been set to 0.4, in accordance to [paper](INSERT LINK)
 
 Estimated time it has cost to train:
 pretraining: 33 hours on TODO! 
@@ -110,3 +161,4 @@ but it's also very strict, which is why we used it combined with MRR.
 
 |model|ARM64 mrr|ARM64 acc|AMD64 mrr|AMD64 acc|RISC-V mrr|RISC-V acc|I386 mrr|I386 acc|crosslingual mrr|crosslingual acc|
 |-----|---------|---------|---------|---------|----------|----------|--------|--------|----------------|----------------|
+|Multilingual-ASMBERT|0.761845832197267|	0.685	|0.8481566008780298	|0.803	|0.6150729328823104	|0.514	|0.6600633426585386	|0.575	|0.6314267393170043	|0.53|