| --- |
| language: |
| - en |
| license: mit |
| tags: |
| - micro-distillery |
| - bigram |
| - grpo |
| - vae |
| - distilled |
| - smollm2 |
| - qwen2.5 |
| - olmo |
| - tinyllama |
| library_name: micro-distillery |
| base_model: |
| - HuggingFaceTB/SmolLM2-135M-Instruct |
| - Qwen/Qwen2.5-0.5B |
| - TinyLlama/TinyLlama-1.1B-Chat-v1.0 |
| --- |
| |
| <div style="background:#000; color:#fff; font-family:monospace; white-space:pre; padding:2rem; display:flex; justify-content:center; line-height:1.2;"> |
| -- βββββββββ -- |
| βββββ β ββββββ -- |
| ββ βββββββββββ v2 |
| </div> |
|
|
|
|
|
|
| # MICROD v2 |
|
|
| This model was trained from scratch using a combination of: |
|
|
| - **GRPO** (Group Relative Policy Optimization) for character-level trajectory training |
| - **VAE filtering** to keep only high-quality trajectories |
| - **4x quantum harmonic oscillator** superposition for deterministic seeding |
|
|
| It is a compact character-level bigram model - no transformer weights, |
| just transition probabilities. Runs on any machine with Python 3.8+. |
|
|
| This Model was made on the fly with: Micro Distillery located in the /app/ folder. A single file html PWA that pulls from fanned api to create custom models. |
|
|
| ## CHANGE LOG - Updated 10/23/2026 |
|
|
| Small UX + correctness maintenance release. No breaking changes. No new features. |
|
|
| ### Fixed |
|
|
| - **Export modal selection drift on filter change** β toggling an export checkbox |
| (e.g. unchecking "Include diagrams") rebuilt the filtered file list but preserved the |
| *array index* of the previously-selected file. Because the filtered list shrinks, that |
| index pointed to a **different file**, so the selection highlight and the preview pane |
| silently jumped to an unrelated file. Selection is now tracked by **filename** |
| (`selectedPreviewName`) instead of by array index, so it follows the same file across |
| filter changes. Falls back to the first visible file when the selected file is filtered |
| out, and re-syncs on `buildFiles()` (which rebuilds `modelFiles`). |
|
|
| - **`renderFileList` empty-state** β when the filtered list is empty, the preview pane now |
| explicitly shows `"No files selected."` rather than leaving stale content from a prior |
| render. (Cosmetic, but consistent with the selection-drift fix above.) |
|
|
| ### Changed |
|
|
| - **Version string bumped** `3.1.2 β 3.1.3` in three places to stay consistent: |
| - `brand-version` in the header |
| - `modelJson.version` emitted in the export bundle |
| - boot log line (`Micro Distillery v3.1.3 initialized.`) |
|
|
| - **`reset()`** now clears the new `selectedPreviewName` field alongside `selectedPreviewIndex` |
| (which is retained for backward compatibility with any external code reading it). |
|
|
| ### Notes |
|
|
| - No public API surface changed. `selectedPreviewName` is a new internal field on `Engine`; |
| external callers continue to work unchanged. |
| - All prior regression checks still pass: `reset()` clears cache + modelFiles, `setupExportOptions()` |
| runs in `boot()`, `renderFileList` guards empty arrays, `updateCache` is bounded by `maxSize=512`, |
| and `downloadBundle` still validates `JSZip`/`saveAs`. |
|
|
| --- |
|
|
| ## Reference Architectures |
|
|
| The bundle metadata references these modern (2025-2026) open models as |
| inspiration sources. The actual model is a distilled bigram - not a |
| transformer checkpoint - and does not load via `AutoModelForCausalLM`. |
|
|
| - [SmolLM2-135M](https://huggingface.co/HuggingFaceTB/SmolLM2-135M) (HuggingFaceTB) |
| - [Qwen2.5-0.5B](https://huggingface.co/Qwen/Qwen2.5-0.5B) (Qwen) |
| - [OLMo-2-1B](https://huggingface.co/allenai/OLMo-2-0425-1B) (AllenAI) |
| - [TinyLlama-1.1B](https://huggingface.co/TinyLlama/TinyLlama-1.1B-Chat-v1.0) (TinyLlama) |
|
|
| ## Model Details |
|
|
| - **Format**: micro-distill-bigram-v1 |
| - **Version**: 3.1.2 |
| - **Reference arch**: tiny-bigram |
| - **Vocab size**: 27 tokens |
| - **Transition states**: 27 |
| - **Harmonic seed**: 1251963776 |
| - **Created**: 2026-10-23T16:28:35.871Z |
|
|
| ## 2 Test Wrappers Included |
|
|
| ```bash |
| |
| cd /the folder you have the files in/ |
| |
| # 1. Run the test harness |
| chmod +x test.sh run.sh |
| ./test.sh |
| |
| # 2. Or chat directly |
| python3 test.py |
| ``` |
|
|
| ## Enhance test.sh / run.sh to Automate the Suite |
| Update your bash test harness script to systematically validate predictions, catch empty states, and verify output stability across your custom files. |
|
|
| ## How to Add Data & Enhance the Tests |
|
|
| Because microd_v2 is an ultra-compact character model trained on Andrej Karpathy's names.txt corpus, you can directly enhance its test coverage by feeding it alternative datasets or updating the wrapper code. |
| |
| |
| ## How the Quantum Oscillators Work for Next Tokens |
| |
| The "4x quantum harmonic oscillator superposition" in microd_v2 does not process token prediction step-by-step during inference. Instead, it functions strictly at the end of the training pipeline to establish a deterministic seed. |
|
|
| Component Operational Role |
| Simulated Superposition The mathematical engine runs an algorithmic simulation of 4 waves/oscillators with assigned frequencies (Ο = 1, Ο, 2, Ο). |
| Wavefunction Collapse When training finishes, these overlapping wave equations are collapsed down to a single integer value (the Harmonic seed: 1251963776). |
| Deterministic Initialization This seed locks in the pseudorandom states needed to output reproducible transition probabilities inside model.json. |
|
|
|
|
| ## Files in the Bundle |
|
|
| | File | Description | |
| |------|-------------| |
| | `model.json` | The trained bigram model (all weights) | |
| | `config.json` | HuggingFace-style metadata config | |
| | `vocab.json` | Character vocabulary | |
| | `tokenizer_config.json` | Tokenizer metadata | |
| | `knowledge_graph.json` | 2D graph used to seed the model | |
| | `training_logs.json` | Metrics + quantum measurements | |
| | `test.py` | Interactive chat wrapper | |
| | `test.sh` | Test harness | |
| | `test_readme.md` | Testing instructions | |
| | `run.sh` | Minimal launcher | |
| | `to_parquet.py` | Convert to parquet | |
| | `diagrams/pipeline.svg` | Training pipeline diagram | |
| | `diagrams/architecture.svg` | Model architecture diagram | |
|
|
| ## Training Pipeline |
|
|
| 1. **Fetch** - pulls corpora and model cards from HuggingFace + GitHub (2026 open sources) |
| 2. **Knowledge graph** - builds a character-transition graph, laid out in 2D |
| 3. **Seed** - initializes bigram transition counts from the corpus |
| 4. **GRPO** - samples trajectories, scores with a name heuristic, boosts high-advantage transitions |
| 5. **VAE filter** - retains only high-probability trajectories |
| 6. **Quantum seed** - 4 harmonic oscillators (w = 1, phi, 2, pi) collapse to a deterministic seed |
| 7. **Export** - produces a HuggingFace-ready .zip bundle |
|
|
|
|
| ## Data Sources |
|
|
| This bundle was produced by fetching from these free, open endpoints: |
|
|
| - `raw.githubusercontent.com/karpathy/makemore/master/names.txt` (training corpus) |
| - `huggingface.co/api/models?search=smollm2` (model reference list) |
| - `huggingface.co/api/models?search=qwen2.5-0.5b` (model reference list) |
| - `huggingface.co/HuggingFaceTB/SmolLM2-135M/raw/main/README.md` (model card) |
| - `huggingface.co/allenai/OLMo-2-0425-1B/raw/main/README.md` (model card) |
| - `api.github.com/repos/karpathy/makemore` (repo metadata) |
|
|
| ## License |
|
|
| MIT Open Sourced |
