README.md

metadata

language: en
license: mit
tags:
  - language-model
  - pytorch
  - rnn
  - text-generation
  - gru
  - tiny-stories
  - bpe-tokenizer
datasets:
  - aditya-6122/tinystories-custom-dataset-18542-v2-test
pipeline_tag: text-generation
widget:
  - text: Once there was a lonely robot.
    example_title: Robot Story
    output:
      text: >
        He was very lonely and lonely. He wanted to be friends with everyone. He
        thought he would never be lonely. One day, he decided to take a walk. He
        walked up to the robot and said, "Hello, robot! Can I be your friend?"
        The robot was very happy and said, "Yes, I would like that. I would like
        to be your friend." The robot was very happy. He had been given a friend
        to the robot. He was so happy to have a friend. He would always be
        friends with the robot. The robot was very happy. He had made a new
        friend. He was no longer lonely. He had a friend who could talk and play
        with him. The robot was so happy to have a friend. They played together
        every day and were never lonely again.
  - text: A child found a mysterious key.
    example_title: Mystery Key
    output:
      text: >
        The child was very curious and wanted to open it. He asked his mom for
        help, but her mom said no. She said that if he could open the lock, the
        key would lock the door open. The child was sad and he didn't know what
        to do. He asked his mom for help. She said, "Let's try to unlock the
        door. It will be open and lock the door open." So the child locked the
        door and locked the door. He was so excited to open it and see what was
        inside. He opened the door and saw the key. He opened it and inside was
        a big, bouncy ball. He was so excited and he ran to the ball. He opened
        the door and saw the ball inside. He was so excited! He ran around the
        house with the key and the ball bounced around. He was so happy! The
        child was so excited that he ran around the house with his key. He ran
        around the garden, playing with the key and the ball. He had so much fun
        that he forgot all about the key. The moral of the story is that it is
        important to be social and obey the rules.
  - text: In a world where time stops at night.
    example_title: Time Stops
    output:
      text: >
        The stars were twinkling and the stars were twinkling.  One day, a
        little girl named Lily came to visit the house. She saw the stars and
        the moon. She asked her mom, "What is that?" Her mom smiled and said,
        "That's a star. It is called a peony. It has stars and lights and people
        who are very happy. Would you like to try?" Lily nodded and said, "Yes
        please!" She grabbed a bright blue star and a bright yellow star. She
        was so proud of her star. The next day, Lily went to the park with her
        mom. She saw the stars and the moon. She was so happy. She ran around
        the house with her mom and dad. They had a wonderful time. 

Tiny-Stories-GRU-LanguageModel-ByteLevelEncoding

Model Details

Model Description

This is a custom language model trained on a dataset of short stories, designed for text generation tasks.

Architecture

Model Sources

• Repository: Aditya6122/BuildingLanguageModel-TinyStories

Uses

Direct Use

This model can be used for generating short stories and text completion tasks.

Downstream Use

Fine-tune the model on specific domains for specialized text generation.

Out-of-Scope Use

Not intended for production use without further validation.

Training Details

Training Data

The model was trained on the aditya-6122/tinystories-custom-dataset-18542-v2-test dataset.

Training Procedure

• Training Regime: Standard language model training with cross-entropy loss
• Epochs: 5
• Batch Size: 128
• Learning Rate: 0.001
• Optimizer: Adam (assumed)
• Hardware: Apple Silicon MPS (if available) or CPU

Tokenizer

The model uses the aditya-6122/tinystories-tokenizer-vb-18542-byte_level_bpe-v3-test tokenizer.

Model Architecture

• Architecture Type: RNN-based language model with GRU cells
• Embedding Dimension: 512
• Hidden Dimension: 1024
• Vocabulary Size: 18542
• Architecture Diagram: See model_arch.jpg for visual representation

Files

• model.bin: The trained model weights in PyTorch format.
• tokenizer.json: The tokenizer configuration.
• model_arch.jpg: Architecture diagram showing the GRU model structure.

How to Use

Since this is a custom model, you'll need to load it using the provided code:

import torch
from your_language_model import LanguageModel  # Replace with actual import
from tokenizers import Tokenizer

# Load tokenizer
tokenizer = Tokenizer.from_file("tokenizer.json")

# Load model
vocab_size = tokenizer.get_vocab_size()
model = LanguageModel(vocab_size=vocab_size, embedding_dimension=512, hidden_dimension=1024)
model.load_state_dict(torch.load("model.bin"))
model.eval()

# Generate text
input_text = "Once upon a time"

# Tokenize and generate [Add your Generation Logic]

Limitations

• This is a basic RNN model and may not perform as well as transformer-based models.
• Trained on limited data, may exhibit biases from the training dataset.
• Not optimized for production deployment.

Ethical Considerations

Users should be aware of potential biases in generated text and use the model responsibly.

Citation

If you use this model, please cite:

@misc{vanilla-rnn-gru-like},
  title={Tiny-Stories-GRU-LanguageModel-ByteLevelEncoding},
  author={Aditya Wath},
  year={2024},
  publisher={Hugging Face},
  url={https://huggingface.co/aditya-6122/Tiny-Stories-GRU-LanguageModel-ByteLevelEncoding}
}