{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "# NoTokenLM-Gen-5.5-Preview — Interactive Playground\n", "\n", "A 169-million-parameter, byte-level, tokenizer-free language model. No subword vocabulary, no BPE — just raw UTF-8 bytes in, raw UTF-8 bytes out. Internally the bytes are grouped into entropy-based patches (≈ 6 bytes each) and processed by a hybrid RWKV-7 / attention trunk.\n", "\n", "> ⚠️ **This is an early preview** — a mid-training checkpoint (step 39,350, ≈ 42 % of the planned run). It writes fluent-looking English and French, but its content is often wrong or incoherent, it is a base model (it continues text, it does not answer questions or follow instructions) and it has no benchmark results yet. See the [model card](https://huggingface.co/VeroLM/NoTokenLM-Gen-5.5-Preview) for samples and limitations.\n", "\n", "This notebook lets you:\n", "- Load **NoTokenLM-Gen-5.5 (Preview)** directly from the Hugging Face Hub\n", "- Generate text from your own prompt\n", "- Control **temperature**, **top-k**, **top-p** and **max new bytes** with an interactive panel\n", "- Re-run generation as many times as you like without reloading the model\n", "\n", "Works on **Google Colab**, **Kaggle Notebooks** and any standard Jupyter environment. A GPU is not required; generation is a sequence of small per-byte steps, so a 300-byte sample takes roughly 20–60 seconds on CPU." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## 1. Setup\n", "\n", "Installs the required libraries. Safe to re-run — already-satisfied packages are skipped.\n", "\n", "Note: this intentionally does **not** force-upgrade `torch`. Colab and Kaggle ship a pre-matched `torch`/`torchvision` pair; upgrading only `torch` can break `transformers` imports. If you hit such an error, use `Runtime → Restart session` and re-run from the top." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "%pip install -q -U \"transformers>=4.44.0\" ipywidgets safetensors huggingface_hub" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## 2. Load the model\n", "\n", "Set `REPO_ID` if you point at a different repo (a fork or a local folder). By default this pulls **NoTokenLM-Gen-5.5-Preview** from the Hugging Face Hub (about 0.7 GB).\n", "\n", "`trust_remote_code=True` is required — the model ships its own architecture code (byte encoder, entropy patching, RWKV-7 trunk, ROSA) alongside the weights. The model runs in float32." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "import torch\n", "from transformers import AutoModelForCausalLM\n", "\n", "REPO_ID = \"VeroLM/NoTokenLM-Gen-5.5-Preview\" # change this if you're loading a different repo/fork or a local folder\n", "\n", "device = \"cuda\" if torch.cuda.is_available() else \"cpu\"\n", "print(f\"Loading model from '{REPO_ID}' ...\")\n", "model = AutoModelForCausalLM.from_pretrained(REPO_ID, trust_remote_code=True)\n", "model.eval()\n", "\n", "try:\n", " model.to(device)\n", " model.generate(prompt=\"Hi\", max_new_tokens=2, seed=0) # smoke test on the chosen device\n", "except Exception as e: # fall back to CPU if the GPU path misbehaves\n", " print(f\"Could not use '{device}' ({type(e).__name__}: {e}); falling back to CPU.\")\n", " device = \"cpu\"\n", " model.to(device)\n", "\n", "n_params = sum(p.numel() for p in model.parameters())\n", "print(f\"Model loaded: {n_params:,} parameters on {device} (training step {model.config.training_step}).\")" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## 3. Generation function\n", "\n", "A thin wrapper around the model's own `generate()` method. The model works on bytes, so `max_new_tokens` counts **bytes** (most characters are 1 byte, accented letters 2, many other scripts 3–4) and decoding uses `errors=\"replace\"`, because a generation can legitimately end in the middle of a multi-byte character. Byte `0x00` is the model's *end-of-document* marker: by default generation stops when the model emits it, and the marker is dropped from the returned text." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "def run_generation(prompt: str, max_new_tokens: int, temperature: float, top_k: int = 0, top_p: float = 1.0,\n", " do_sample: bool = True, seed: int | None = None, streamer=None):\n", " \"\"\"Generate a continuation for `prompt` and return the full text (prompt + completion).\"\"\"\n", " if not prompt:\n", " prompt = \" \" # avoid feeding a fully empty byte sequence\n", "\n", " input_ids = torch.tensor([list(prompt.encode(\"utf-8\"))], dtype=torch.long)\n", "\n", " output_ids = model.generate(\n", " input_ids,\n", " max_new_tokens=max_new_tokens,\n", " temperature=temperature,\n", " top_k=top_k if top_k > 0 else None,\n", " top_p=top_p,\n", " do_sample=do_sample,\n", " seed=seed,\n", " streamer=streamer,\n", " )\n", "\n", " byte_values = [b for b in output_ids[0].tolist() if 0 < b <= 255] # 0x00 = end-of-document marker\n", " return bytes(byte_values).decode(\"utf-8\", errors=\"replace\")\n", "\n", "\n", "# Quick sanity check\n", "print(run_generation(\"Once upon a time,\", max_new_tokens=40, temperature=0.8, seed=0))" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## 4. Interactive generation panel\n", "\n", "Adjust the prompt, temperature, top-k, top-p and length below, then click **Generate**. Re-click any time — the model stays loaded in memory. The text appears progressively while it is being generated.\n", "\n", "- **Temperature** — lower (e.g. 0.5) is safer and more repetitive; higher (e.g. 1.0+) is more varied but drifts into incoherence sooner. The README samples use **0.8**.\n", "- **Top-k** — restricts sampling to the k most likely next bytes at each step (0 = off).\n", "- **Top-p** — nucleus sampling: keep the smallest set of bytes whose probability adds up to p (1.0 = off).\n", "- **Max new bytes** — how many new bytes to generate.\n", "- **Greedy (no sampling)** — always picks the single most likely next byte (temperature/top-k/top-p are ignored). Deterministic, but very repetitive.\n", "- **Seed** — a fixed integer for reproducible output, or -1 for a different result every time.\n", "\n", "This is a **base model**: write the *beginning* of a text (an article opening, a story, a function signature) and let it continue. Questions and instructions are not answered." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "import ipywidgets as widgets\n", "from IPython.display import display, clear_output\n", "\n", "prompt_box = widgets.Textarea(\n", " value=\"Once upon a time, there was a little rabbit named Milo who\",\n", " placeholder=\"Type the beginning of a text...\",\n", " description=\"Prompt:\",\n", " layout=widgets.Layout(width=\"100%\", height=\"90px\"),\n", " style={\"description_width\": \"100px\"},\n", ")\n", "\n", "temperature_slider = widgets.FloatSlider(\n", " value=0.8, min=0.05, max=1.5, step=0.05,\n", " description=\"Temperature:\",\n", " style={\"description_width\": \"100px\"},\n", " layout=widgets.Layout(width=\"400px\"),\n", " readout_format=\".2f\",\n", ")\n", "\n", "top_k_slider = widgets.IntSlider(\n", " value=0, min=0, max=200, step=1,\n", " description=\"Top-k (0=off):\",\n", " style={\"description_width\": \"100px\"},\n", " layout=widgets.Layout(width=\"400px\"),\n", ")\n", "\n", "top_p_slider = widgets.FloatSlider(\n", " value=1.0, min=0.1, max=1.0, step=0.05,\n", " description=\"Top-p:\",\n", " style={\"description_width\": \"100px\"},\n", " layout=widgets.Layout(width=\"400px\"),\n", " readout_format=\".2f\",\n", ")\n", "\n", "max_tokens_slider = widgets.IntSlider(\n", " value=300, min=10, max=1024, step=10,\n", " description=\"Max new bytes:\",\n", " style={\"description_width\": \"100px\"},\n", " layout=widgets.Layout(width=\"400px\"),\n", ")\n", "\n", "greedy_checkbox = widgets.Checkbox(\n", " value=False,\n", " description=\"Greedy (no sampling)\",\n", " style={\"description_width\": \"initial\"},\n", ")\n", "\n", "seed_box = widgets.IntText(\n", " value=-1,\n", " description=\"Seed (-1 = random):\",\n", " style={\"description_width\": \"140px\"},\n", " layout=widgets.Layout(width=\"260px\"),\n", ")\n", "\n", "generate_button = widgets.Button(\n", " description=\"Generate\",\n", " button_style=\"primary\",\n", " icon=\"magic\",\n", ")\n", "\n", "output_area = widgets.Output()\n", "\n", "def on_generate_clicked(_):\n", " generate_button.disabled = True\n", " try:\n", " with output_area:\n", " clear_output(wait=True)\n", " seed = None if seed_box.value < 0 else seed_box.value\n", " prompt = prompt_box.value or \" \"\n", " produced = bytearray(prompt.encode(\"utf-8\"))\n", " print(\"Generating...\\n\")\n", "\n", " def show_progress(b): # called with every new byte\n", " if b != 0:\n", " produced.append(b)\n", " if len(produced) % 12 == 0:\n", " clear_output(wait=True)\n", " print(bytes(produced).decode(\"utf-8\", errors=\"replace\"))\n", "\n", " text = run_generation(\n", " prompt=prompt,\n", " max_new_tokens=max_tokens_slider.value,\n", " temperature=temperature_slider.value,\n", " top_k=top_k_slider.value,\n", " top_p=top_p_slider.value,\n", " do_sample=not greedy_checkbox.value,\n", " seed=seed,\n", " streamer=show_progress,\n", " )\n", " clear_output(wait=True)\n", " print(text)\n", " finally:\n", " generate_button.disabled = False\n", "\n", "generate_button.on_click(on_generate_clicked)\n", "\n", "controls = widgets.VBox([\n", " prompt_box,\n", " widgets.HBox([temperature_slider, top_k_slider]),\n", " widgets.HBox([top_p_slider, max_tokens_slider]),\n", " widgets.HBox([seed_box, greedy_checkbox]),\n", " generate_button,\n", " output_area,\n", "])\n", "\n", "display(controls)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## 5. Try the model-card prompts (optional)\n", "\n", "The prompts below are the ones used for the samples in the model card (temperature 0.8, up to 300 bytes). Your outputs will differ from the card's — sampling is random, and a different seed gives different text — but you should see the same kind of text: fluent form, unreliable facts. Each prompt takes a little while on CPU, so pick one or two." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "CARD_PROMPTS = {\n", " \"wikipedia\": \"Photosynthesis is\",\n", " \"history\": \"The French Revolution began in 1789 when\",\n", " \"story\": \"Once upon a time, there was a little rabbit named Milo who\",\n", " \"textbook\": \"Chapter 3: Introduction to Fractions\\n\\nIn this chapter, we will learn\",\n", " \"python\": \"def is_palindrome(s: str) -> bool:\\n\",\n", " \"french\": \"La Révolution française est une période de l'histoire de France qui\",\n", " \"facts\": \"The capital of France is\",\n", " \"pattern\": \"apple, banana, cherry, apple, banana,\",\n", " \"q_and_a\": \"Question: What is the capital of Italy?\\nAnswer:\",\n", "}\n", "\n", "for name in [\"history\", \"python\"]: # edit this list to try others\n", " print(f\"=== {name} ===\")\n", " print(run_generation(CARD_PROMPTS[name], max_new_tokens=300, temperature=0.8, seed=42))\n", " print()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## 6. Batch generation (optional)\n", "\n", "Generate several completions of the same prompt — useful for comparing sampling settings or for seeing the range of what the model produces." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "def batch_generate(prompt: str, n: int = 3, max_new_tokens: int = 120, temperature: float = 0.8, top_k: int = 0, top_p: float = 1.0):\n", " for i in range(n):\n", " text = run_generation(prompt, max_new_tokens=max_new_tokens, temperature=temperature, top_k=top_k, top_p=top_p)\n", " print(f\"[{i + 1}] {text}\\n{'-' * 60}\")\n", "\n", "# Example — feel free to edit the arguments and re-run\n", "batch_generate(\"Deep in the forest, Jack found a shell\", n=3, max_new_tokens=120, temperature=0.8)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## About this model\n", "\n", "**NoTokenLM-Gen-5.5 (Preview)** is a ~169M-parameter byte-level model: byte embeddings plus hashed n-gram and suffix-match (ROSA) features, a small local encoder, entropy-based patching, a 20-layer hybrid RWKV-7 / gated-attention trunk working on patches, and a local byte decoder. It was trained from scratch on an English-heavy text mix (educational and high-quality web text, Wikipedia, synthetic textbooks, stories, math, code) with some French. This checkpoint is from step 19,950 of a planned ~95k-step run.\n", "\n", "It is **not** meant for factual questions, math, reasoning, code you intend to run, or conversation. See the full [model card on Hugging Face](https://huggingface.co/VeroLM/NoTokenLM-Gen-5.5-Preview) for sample outputs and an honest breakdown of what it can and can't do." ] } ], "metadata": { "colab": { "provenance": [], "name": "NoTokenLM-Gen-5.5-Preview-Playground.ipynb" }, "kernelspec": { "display_name": "Python 3", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.10.12" }, "accelerator": "GPU" }, "nbformat": 4, "nbformat_minor": 5 }