|
Download README.md from Thunderbee/gptx_test: direct link, hf CLI and curl.
- Browser
- Download file 441 Bytes
-
https://huggingface.co/Thunderbee/gptx_test/resolve/main/README.md
- Command line
-
hf download hf://Thunderbee/gptx_test/README.md
-
curl -L -o README.md https://huggingface.co/Thunderbee/gptx_test/resolve/main/README.md
441 Bytes
| language: en | |
| tags: | |
| - pytorch | |
| - gpt2 | |
| - language-model | |
| pipeline_tag: text-generation | |
| # GPT-X Model | |
| This model was trained using the GPT-X framework. | |
| ## Model Architecture | |
| - Layers: 12 | |
| - Attention Heads: 12 | |
| - Hidden Size: 768 | |
| - Vocabulary Size: 50257 | |
| - Maximum Sequence Length: 1024 | |
| - Model Type: base | |
| ## Training Details | |
| - Batch Size: 524288 | |
| - Learning Rate: 0.0006 | |
| - Weight Decay: 0.0 | |
| - Mixed Precision: True | |
| - Optimizer: muon | |