Upload README.md with huggingface_hub

README.md

@@ -1,58 +1,30 @@
----
-language:
-- en
-license: mit
-tags:
-- bitnet
-- 1.58-bit
-- ternary
-- tinystories
-- edge-device
-datasets:
-- roneneldan/TinyStories
----
-
-# MiniLM (BitNet-TinyStories-V2) (3.9 MB)
-
-This is an ultra-compressed **1.58-bit** language model trained entirely from scratch on the `TinyStories` dataset. 
-
-It implements the **BitNet (1.58b)** architecture, where all internal Linear layers are heavily quantized into ternary weights (`-1, 0, 1`). This version uses **Weight Tying**, allowing it to achieve a deep 12-Layer architecture while staying under a 4MB footprint!
-
-## Model Details
-- **Architecture:** BitNet (1.58b)
-- **Parameters:** ~21 Million 
-- **Layers:** 12 (Tied)
-- **Precision:** 1.58-bit (Ternary) for internal weights
-- **File Size:** 3.96 MB
-- **Tokenizer:** `arnir0/Tiny-LLM` SentencePiece (32,000 vocab size)
-- **Dataset:** `roneneldan/TinyStories`
-- **Validation Perplexity:** 23.7
-
-## Usage
-
-Because this model uses a highly customized ternary architecture, it cannot be loaded using standard HuggingFace `AutoModel`. You must use the `BitGPT` class implementation.
-
-```python
-import torch
-from transformers import AutoTokenizer
-from bitnet_test import BitGPT
-
-# 1. Load Tokenizer
-tokenizer = AutoTokenizer.from_pretrained("arnir0/Tiny-LLM")
-
-# 2. Initialize Model
-model = BitGPT(vocab_size=len(tokenizer), embed_dim=256, num_layers=12, num_heads=4, tie_weights=True)
-
-# 3. Load 1.58-bit Weights
-model.load_state_dict(torch.load("bitnet_tied.pt", map_location="cpu"))
-model.eval()
-
-# 4. Generate Text
-prompt = "Once upon a time, there was a tiny cat named"
-input_ids = tokenizer.encode(prompt, return_tensors="pt")
-
-# ... Run standard auto-regressive generation loop
+# MiniLM
+
+MiniLM is a fully-functional 1.58-bit (Ternary) Large Language Model engineered entirely from scratch, built explicitly for edge devices.
+
+The base model is a 12-Layer Transformer that fits inside **6.00 MB**.
+
+Because it is so small, it can be combined with "Side-Car" LoRAs (Low-Rank Adaptations) to perform complex logic (like parsing natural language into exact JSON structures) entirely offline.
+
+## Repository Contents
+- `app.py`: The interactive Streamlit Web UI. Run `streamlit run app.py` to start.
+- `model.py`: The custom 1.58-bit PyTorch architecture.
+- `lora.py`: The `BitLoraLinear` side-car architecture.
+- `train_lora_dynamic.py`: A script to easily train 1MB LoRAs using your own JSON datasets.
+- `ARCHITECTURE.md`: A deep-dive into how 1.58-bit quantization and weight-tying work.
+- `LORA_GUIDE.md`: How to build and train side-car LoRAs.
+- `API_USAGE.md`: Example code for loading the model and generating text.
+
+## Installation
+
+```bash
+python3 -m venv .venv
+source .venv/bin/activate
+pip install -r requirements.txt
 ```
 
-## Intended Use
-This model is intended purely as a research demonstration of the viability of 1.58-bit LLMs on edge devices. Because it was trained exclusively on the TinyStories dataset, it is completely incapable of performing complex reasoning, answering factual questions, or following instructions. It will only generate children's storybooks.
+## Running the Web UI
+
+```bash
+streamlit run app.py
+```