---
license: apache-2.0
datasets:
- roneneldan/TinyStories
language:
- en
library_name: transformers
tags:
- text-generation-inference
pipeline_tag: text-generation
---

# 🧠 Pragya Preview (प्रज्ञा)

> **Note from the Creator:**  
> *"This is my message to all: **Pragya Preview** is a small edition/prototype of **Pragya** (Sanskrit for wisdom/intelligence). It was trained on a free Google Colab T4 GPU in a single 1-hour session. I hope you will love it!"*

---

## Model Summary

**Pragya Preview** is an experimental, lightweight language model pretrained on the [TinyStories](https://huggingface.co/datasets/roneneldan/TinyStories) dataset. Built as a proof-of-concept under strict compute limits (1-hour single T4 GPU run), it demonstrates rapid baseline convergence and efficient short-form narrative generation.

- **Developer:** Arush Kumar and Pragya's team
- **Model Type:** Causal Language Model using Grouped Query Attention(Decoder-only Transformer)
- **Language:** English (`en`)
- **Base Dataset:** `roneneldan/TinyStories`
- **License:** Apache 2.0
- **Primary Task:** Text Generation / Short Story Completion
- **Note**: Model is not Instruct tuned and is not trained completely
---
## Model is open weight
- weights are available in .pt and .onnx 
## Intended Uses & Scope

### Recommended Use
- Fast, low-latency text generation experiments on consumer hardware or CPU environments.
- Educational demonstrations and prototype benchmarking for resource-constrained training.

### Limitations
- **Capacity:** Trained during a brief 1-hour session, so depth and complex reasoning are limited.
- **Scope:** Tailored primarily for simple, coherent story completion using basic vocabulary.
- **Factual Knowledge:** Does not possess general world knowledge, advanced coding skills, or multi-turn reasoning abilities.

---

## Quickstart & Usage

### 1. Installation

Ensure you have PyTorch and Hugging Face `transformers` installed:

```bash
pip install torch transformers

```

### 2. Inference Code

```python
import torch
from transformers import AutoTokenizer, AutoModelForCausalLM

# Set your Hugging Face model repository ID
model_id = "ArushBuilds/Pragya-Preview"

# Load tokenizer and model
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(
    model_id='ArushBuilds/Pragya-Preview',
    torch_dtype=torch.float16 if torch.cuda.is_available() else torch.float32,
    device_map="auto"
)

# Define your prompt
prompt = "Once upon a time, "
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)

# Generate text
outputs = model.generate(
    **inputs,
    max_new_tokens=100,
    temperature=0.7,
    top_p=0.9,
    do_sample=True,
    pad_token_id=tokenizer.eos_token_id
)

story = tokenizer.decode(outputs[0], skip_special_tokens=True)
print("\n--- Generated Story ---")
print(story)

```

---

## Training Setup

| Parameter | Configuration |
| --- | --- |
| **Hardware** | 1x NVIDIA T4 GPU (Google Colab Free Tier) |
| **Training Duration** | ~1 hour |
| **Dataset** | `roneneldan/TinyStories` |
| **Framework** | PyTorch + Hugging Face Transformers |

---

## Citation & Acknowledgments

If you use or reference **Pragya Preview**, please credit the underlying dataset:

```bibtex
@article{eldan2023tinystories,
  title={TinyStories: How Small Can Language Models Be and Still Speak Coherent English?},
  author={Eldan, Ronen and Li, Yuanzhi},
  journal={arXiv preprint arXiv:2305.07759},
  year={2023}
}

```