| # Custom Urdu LLM | |
| This is a custom transformer-based Large Language Model for Urdu. | |
| ## Model Details | |
| - **Architecture:** Transformer (GPT-based) | |
| - **Framework:** PyTorch | |
| - **Tokenizer:** SentencePiece | |
| - **Hyperparameters:** | |
| - Vocabulary Size: 20,000 | |
| - Embedding Size: 768 | |
| - Attention Heads: 12 | |
| - Layers: 12 | |
| - Dropout: 0.2 | |
| ## Usage | |
| ```python | |
| from transformers import AutoModel, AutoTokenizer | |
| model = AutoModel.from_pretrained("AliMuhammad73/testing-model") | |
| tokenizer = AutoTokenizer.from_pretrained("AliMuhammad73/testing-model") | |
| prompt = <prompt in urdu> | |
| inputs = tokenizer(prompt, return_tensors="pt") | |
| output = model.generate(inputs.input_ids, max_new_tokens=tokens_to_generate) | |
| print(tokenizer.decode(output[0])) | |
| ``` | |
| --- | |
| license: apache-2.0 | |
| --- | |