--- language: - en license: mit tags: - micro-distillery - bigram - grpo - vae - distilled - smollm2 - qwen2.5 - olmo - tinyllama library_name: micro-distillery base_model: - HuggingFaceTB/SmolLM2-135M-Instruct - Qwen/Qwen2.5-0.5B - TinyLlama/TinyLlama-1.1B-Chat-v1.0 ---
-- ▄▖▄▖▄▖▄▖▄ -- ▛▖▞▌▐ ▌ ▙▘▌▌▌▌ -- ▌▝ ▌▟▖▙▖▌▌▙▌▙▘ v2
# MICROD v2 This model was trained from scratch using a combination of: - **GRPO** (Group Relative Policy Optimization) for character-level trajectory training - **VAE filtering** to keep only high-quality trajectories - **4x quantum harmonic oscillator** superposition for deterministic seeding It is a compact character-level bigram model - no transformer weights, just transition probabilities. Runs on any machine with Python 3.8+. This Model was made on the fly with: Micro Distillery located in the /app/ folder. A single file html PWA that pulls from fanned api to create custom models. ## CHANGE LOG - Updated 10/23/2026 Small UX + correctness maintenance release. No breaking changes. No new features. ### Fixed - **Export modal selection drift on filter change** — toggling an export checkbox (e.g. unchecking "Include diagrams") rebuilt the filtered file list but preserved the *array index* of the previously-selected file. Because the filtered list shrinks, that index pointed to a **different file**, so the selection highlight and the preview pane silently jumped to an unrelated file. Selection is now tracked by **filename** (`selectedPreviewName`) instead of by array index, so it follows the same file across filter changes. Falls back to the first visible file when the selected file is filtered out, and re-syncs on `buildFiles()` (which rebuilds `modelFiles`). - **`renderFileList` empty-state** — when the filtered list is empty, the preview pane now explicitly shows `"No files selected."` rather than leaving stale content from a prior render. (Cosmetic, but consistent with the selection-drift fix above.) ### Changed - **Version string bumped** `3.1.2 → 3.1.3` in three places to stay consistent: - `brand-version` in the header - `modelJson.version` emitted in the export bundle - boot log line (`Micro Distillery v3.1.3 initialized.`) - **`reset()`** now clears the new `selectedPreviewName` field alongside `selectedPreviewIndex` (which is retained for backward compatibility with any external code reading it). ### Notes - No public API surface changed. `selectedPreviewName` is a new internal field on `Engine`; external callers continue to work unchanged. - All prior regression checks still pass: `reset()` clears cache + modelFiles, `setupExportOptions()` runs in `boot()`, `renderFileList` guards empty arrays, `updateCache` is bounded by `maxSize=512`, and `downloadBundle` still validates `JSZip`/`saveAs`. --- ## Reference Architectures The bundle metadata references these modern (2025-2026) open models as inspiration sources. The actual model is a distilled bigram - not a transformer checkpoint - and does not load via `AutoModelForCausalLM`. - [SmolLM2-135M](https://huggingface.co/HuggingFaceTB/SmolLM2-135M) (HuggingFaceTB) - [Qwen2.5-0.5B](https://huggingface.co/Qwen/Qwen2.5-0.5B) (Qwen) - [OLMo-2-1B](https://huggingface.co/allenai/OLMo-2-0425-1B) (AllenAI) - [TinyLlama-1.1B](https://huggingface.co/TinyLlama/TinyLlama-1.1B-Chat-v1.0) (TinyLlama) ## Model Details - **Format**: micro-distill-bigram-v1 - **Version**: 3.1.2 - **Reference arch**: tiny-bigram - **Vocab size**: 27 tokens - **Transition states**: 27 - **Harmonic seed**: 1251963776 - **Created**: 2026-10-23T16:28:35.871Z ## 2 Test Wrappers Included ```bash cd /the folder you have the files in/ # 1. Run the test harness chmod +x test.sh run.sh ./test.sh # 2. Or chat directly python3 test.py ``` ## Enhance test.sh / run.sh to Automate the Suite Update your bash test harness script to systematically validate predictions, catch empty states, and verify output stability across your custom files. ## How to Add Data & Enhance the Tests Because microd_v2 is an ultra-compact character model trained on Andrej Karpathy's names.txt corpus, you can directly enhance its test coverage by feeding it alternative datasets or updating the wrapper code. ## How the Quantum Oscillators Work for Next Tokens The "4x quantum harmonic oscillator superposition" in microd_v2 does not process token prediction step-by-step during inference. Instead, it functions strictly at the end of the training pipeline to establish a deterministic seed. Component Operational Role Simulated Superposition The mathematical engine runs an algorithmic simulation of 4 waves/oscillators with assigned frequencies (ω = 1, φ, 2, π). Wavefunction Collapse When training finishes, these overlapping wave equations are collapsed down to a single integer value (the Harmonic seed: 1251963776). Deterministic Initialization This seed locks in the pseudorandom states needed to output reproducible transition probabilities inside model.json. ## Files in the Bundle | File | Description | |------|-------------| | `model.json` | The trained bigram model (all weights) | | `config.json` | HuggingFace-style metadata config | | `vocab.json` | Character vocabulary | | `tokenizer_config.json` | Tokenizer metadata | | `knowledge_graph.json` | 2D graph used to seed the model | | `training_logs.json` | Metrics + quantum measurements | | `test.py` | Interactive chat wrapper | | `test.sh` | Test harness | | `test_readme.md` | Testing instructions | | `run.sh` | Minimal launcher | | `to_parquet.py` | Convert to parquet | | `diagrams/pipeline.svg` | Training pipeline diagram | | `diagrams/architecture.svg` | Model architecture diagram | ## Training Pipeline 1. **Fetch** - pulls corpora and model cards from HuggingFace + GitHub (2026 open sources) 2. **Knowledge graph** - builds a character-transition graph, laid out in 2D 3. **Seed** - initializes bigram transition counts from the corpus 4. **GRPO** - samples trajectories, scores with a name heuristic, boosts high-advantage transitions 5. **VAE filter** - retains only high-probability trajectories 6. **Quantum seed** - 4 harmonic oscillators (w = 1, phi, 2, pi) collapse to a deterministic seed 7. **Export** - produces a HuggingFace-ready .zip bundle ## Data Sources This bundle was produced by fetching from these free, open endpoints: - `raw.githubusercontent.com/karpathy/makemore/master/names.txt` (training corpus) - `huggingface.co/api/models?search=smollm2` (model reference list) - `huggingface.co/api/models?search=qwen2.5-0.5b` (model reference list) - `huggingface.co/HuggingFaceTB/SmolLM2-135M/raw/main/README.md` (model card) - `huggingface.co/allenai/OLMo-2-0425-1B/raw/main/README.md` (model card) - `api.github.com/repos/karpathy/makemore` (repo metadata) ## License MIT Open Sourced