alexaapo commited on
Commit
ebf3726
·
verified ·
1 Parent(s): fd8a649

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +3 -3
README.md CHANGED
@@ -126,7 +126,7 @@ The data was processed into fixed-size chunks of 512 tokens, respecting document
126
 
127
  ### Pre-training
128
 
129
- The model was pre-trained from scratch for **150,000 steps** on 8x NVIDIA A100 40GB GPUs, using BFloat16 (`bf16`) mixed-precision for stability and speed. The training took approximately **25 hours and 7 minutes** to complete.
130
 
131
  The key hyperparameters used were:
132
  - **Learning Rate**: 2e-4 (0.0002) with a linear warmup of 9,000 steps
@@ -144,7 +144,7 @@ The key hyperparameters used were:
144
  The model achieved the following performance metrics:
145
  - **Final Training Loss**: 0.7479
146
  - **Final Evaluation Loss**: 0.69405
147
- - **Training Infrastructure**: 8x NVIDIA H100 80GB GPUs
148
  - **Total Training Steps**: 150,000
149
  - **Total Training Time**: 25 hours 7 minutes
150
  - **Train/Validation Split**: 95%/5%
@@ -222,7 +222,7 @@ The model shows stable convergence across both languages:
222
  - **Model Size**: ~125M parameters
223
  - **Architecture**: RoBERTa-base (12 layers, 12 attention heads)
224
  - **Languages**: Greek (el) + English (en)
225
- - **Training Time**: 25 hours 7 minutes on 8x A100 GPUs
226
  - **Dataset Size**: 26GB bilingual legal corpus
227
  - **Language Ratio**: 60.3% Greek, 39.7% English
228
  - **Memory Requirements**: Efficient base architecture suitable for production deployment
 
126
 
127
  ### Pre-training
128
 
129
+ The model was pre-trained from scratch for **150,000 steps** on 8x NVIDIA H100 GPUs, using BFloat16 (`bf16`) mixed-precision for stability and speed. The training took approximately **25 hours and 7 minutes** to complete.
130
 
131
  The key hyperparameters used were:
132
  - **Learning Rate**: 2e-4 (0.0002) with a linear warmup of 9,000 steps
 
144
  The model achieved the following performance metrics:
145
  - **Final Training Loss**: 0.7479
146
  - **Final Evaluation Loss**: 0.69405
147
+ - **Training Infrastructure**: 8x NVIDIA H100 GPUs
148
  - **Total Training Steps**: 150,000
149
  - **Total Training Time**: 25 hours 7 minutes
150
  - **Train/Validation Split**: 95%/5%
 
222
  - **Model Size**: ~125M parameters
223
  - **Architecture**: RoBERTa-base (12 layers, 12 attention heads)
224
  - **Languages**: Greek (el) + English (en)
225
+ - **Training Time**: 25 hours 7 minutes on 8x H100 GPUs
226
  - **Dataset Size**: 26GB bilingual legal corpus
227
  - **Language Ratio**: 60.3% Greek, 39.7% English
228
  - **Memory Requirements**: Efficient base architecture suitable for production deployment