| --- |
| license: mit |
| library_name: transformers |
| pipeline_tag: text-generation |
| language: |
| - en |
| tags: |
| - testing |
| - random-initialization |
| - gpt2 |
| --- |
| # Tiny random GPT-2 for testing |
|
|
| This model is randomly initialized and **has not been trained**. It is for |
| testing upload, download, tokenization, and model loading only. Its output is |
| not meaningful and it is not suitable for real language tasks or benchmarking. |
| No pretrained model weights or training datasets were used. |
|
|
| Architecture: 1 GPT-2 layer, 1 attention head, 16 hidden dimensions, |
| 32 vocabulary tokens, and a maximum context length of 64 tokens. |
| Parameter count: 3792. |
|
|
| ## Usage |
| ```python |
| from transformers import AutoTokenizer, AutoModelForCausalLM |
| |
| repo = "ruhook/test-ruhook" |
| tokenizer = AutoTokenizer.from_pretrained(repo) |
| model = AutoModelForCausalLM.from_pretrained(repo) |
| inputs = tokenizer("hello world", return_tensors="pt") |
| outputs = model.generate(**inputs, max_new_tokens=5, do_sample=False) |
| print(tokenizer.decode(outputs[0], skip_special_tokens=True)) |
| ``` |
|
|
| Validated locally with Python 3, torch 2.2.2 and transformers 4.46.3. |
| The toy word-level tokenizer maps words outside its small vocabulary to UNK. |
|
|
