Custom Urdu LLM
This is a custom transformer-based Large Language Model for Urdu.
Model Details
- Architecture: Transformer (GPT-based)
- Framework: PyTorch
- Tokenizer: SentencePiece
- Hyperparameters:
- Vocabulary Size: 20,000
- Embedding Size: 768
- Attention Heads: 12
- Layers: 12
- Dropout: 0.2
Usage
from transformers import AutoModel, AutoTokenizer
model = AutoModel.from_pretrained("AliMuhammad73/testing-model")
tokenizer = AutoTokenizer.from_pretrained("AliMuhammad73/testing-model")
prompt = <prompt in urdu>
inputs = tokenizer(prompt, return_tensors="pt")
output = model.generate(inputs.input_ids, max_new_tokens=tokens_to_generate)
print(tokenizer.decode(output[0]))