|
|
| --- |
| license: apache-2.0 |
| datasets: |
| - roneneldan/TinyStories |
| language: |
| - en |
| library_name: transformers |
| tags: |
| - text-generation-inference |
| pipeline_tag: text-generation |
| --- |
| |
| # 🧠 Pragya Preview (प्रज्ञा) |
|
|
| > **Note from the Creator:** |
| > *"This is my message to all: **Pragya Preview** is a small edition/prototype of **Pragya** (Sanskrit for wisdom/intelligence). It was trained on a free Google Colab T4 GPU in a single 1-hour session. I hope you will love it!"* |
|
|
| --- |
|
|
| ## Model Summary |
|
|
| **Pragya Preview** is an experimental, lightweight language model pretrained on the [TinyStories](https://huggingface.co/datasets/roneneldan/TinyStories) dataset. Built as a proof-of-concept under strict compute limits (1-hour single T4 GPU run), it demonstrates rapid baseline convergence and efficient short-form narrative generation. |
|
|
| - **Developer:** Arush Kumar and Pragya's team |
| - **Model Type:** Causal Language Model using Grouped Query Attention(Decoder-only Transformer) |
| - **Language:** English (`en`) |
| - **Base Dataset:** `roneneldan/TinyStories` |
| - **License:** Apache 2.0 |
| - **Primary Task:** Text Generation / Short Story Completion |
| - **Note**: Model is not Instruct tuned and is not trained completely |
| --- |
| ## Model is open weight |
| - weights are available in .pt and .onnx |
| ## Intended Uses & Scope |
|
|
| ### Recommended Use |
| - Fast, low-latency text generation experiments on consumer hardware or CPU environments. |
| - Educational demonstrations and prototype benchmarking for resource-constrained training. |
|
|
| ### Limitations |
| - **Capacity:** Trained during a brief 1-hour session, so depth and complex reasoning are limited. |
| - **Scope:** Tailored primarily for simple, coherent story completion using basic vocabulary. |
| - **Factual Knowledge:** Does not possess general world knowledge, advanced coding skills, or multi-turn reasoning abilities. |
|
|
| --- |
|
|
| ## Quickstart & Usage |
|
|
| ### 1. Installation |
|
|
| Ensure you have PyTorch and Hugging Face `transformers` installed: |
|
|
| ```bash |
| pip install torch transformers |
| |
| ``` |
|
|
| ### 2. Inference Code |
|
|
| ```python |
| import torch |
| from transformers import AutoTokenizer, AutoModelForCausalLM |
| |
| # Set your Hugging Face model repository ID |
| model_id = "ArushBuilds/Pragya-Preview" |
| |
| # Load tokenizer and model |
| tokenizer = AutoTokenizer.from_pretrained(model_id) |
| model = AutoModelForCausalLM.from_pretrained( |
| model_id='ArushBuilds/Pragya-Preview', |
| torch_dtype=torch.float16 if torch.cuda.is_available() else torch.float32, |
| device_map="auto" |
| ) |
| |
| # Define your prompt |
| prompt = "Once upon a time, " |
| inputs = tokenizer(prompt, return_tensors="pt").to(model.device) |
| |
| # Generate text |
| outputs = model.generate( |
| **inputs, |
| max_new_tokens=100, |
| temperature=0.7, |
| top_p=0.9, |
| do_sample=True, |
| pad_token_id=tokenizer.eos_token_id |
| ) |
| |
| story = tokenizer.decode(outputs[0], skip_special_tokens=True) |
| print("\n--- Generated Story ---") |
| print(story) |
| |
| ``` |
|
|
| --- |
|
|
| ## Training Setup |
|
|
| | Parameter | Configuration | |
| | --- | --- | |
| | **Hardware** | 1x NVIDIA T4 GPU (Google Colab Free Tier) | |
| | **Training Duration** | ~1 hour | |
| | **Dataset** | `roneneldan/TinyStories` | |
| | **Framework** | PyTorch + Hugging Face Transformers | |
|
|
| --- |
|
|
| ## Citation & Acknowledgments |
|
|
| If you use or reference **Pragya Preview**, please credit the underlying dataset: |
|
|
| ```bibtex |
| @article{eldan2023tinystories, |
| title={TinyStories: How Small Can Language Models Be and Still Speak Coherent English?}, |
| author={Eldan, Ronen and Li, Yuanzhi}, |
| journal={arXiv preprint arXiv:2305.07759}, |
| year={2023} |
| } |
| |
| ``` |
|
|
|
|
|
|
|
|
