new-word-teaching: trained models

Fine-tuned copies of the TinyStories models. Each one was trained for 300 updates to learn two invented words, "vekta" and "nolta", that stand for two words the model already knows.

Files

Each run has one file, final_model_fp16.pt: the model's weights after training, stored as 16-bit floats.

Folder Base model How the invented words were taught
taught-by-use/ TinyStories-33M Stories with the invented word in place of the old word
taught-by-short-use-sentences/ TinyStories-33M Only the sentences of those stories that contain the word
taught-by-definition/ TinyStories-33M Sentences that state the meaning, such as "A vekta is a ball."
taught-by-definitions-inside-stories/ TinyStories-33M The unaltered story with a definition sentence after each mention of the old word
size-sweep/tinystories-<size>-taught-by-use/ TinyStories-1M, 3M, 8M, 28M As taught-by-use/
size-sweep/tinystories-<size>-taught-by-definition/ TinyStories-1M, 3M, 8M, 28M As taught-by-definition/

Run folders are named by mapping and random seed:

Run folder vekta stands for nolta stands for
linked_seed<n> ball book
swapped_seed<n> book ball
unlinked_seed<n> car hat
unlinked_swapped_seed<n> hat car

Load a model

import torch
from huggingface_hub import hf_hub_download
from transformers import AutoModelForCausalLM, AutoTokenizer

base = "roneneldan/TinyStories-33M"
tok = AutoTokenizer.from_pretrained(base)
model = AutoModelForCausalLM.from_pretrained(base)
path = hf_hub_download("vein05/new-word-teaching", "taught-by-use/linked_seed0/final_model_fp16.pt")
state = torch.load(path, map_location="cpu")
model.load_state_dict({k: v.float() for k, v in state.items()})
Downloads last month

-

Downloads are not tracked for this model. How to track
Inference Providers NEW
This model isn't deployed by any Inference Provider. ๐Ÿ™‹ Ask for provider support

Model tree for vein05/new-word-teaching

Finetuned
(2)
this model