README.md

---
language:
- en
license: mit
tags:
- micro-distillery
- bigram
- grpo
- vae
- distilled
- smollm2
- qwen2.5
- olmo
- tinyllama
library_name: micro-distillery
base_model:
- HuggingFaceTB/SmolLM2-135M-Instruct
- Qwen/Qwen2.5-0.5B
- TinyLlama/TinyLlama-1.1B-Chat-v1.0
---

<div style="background:#000; color:#fff; font-family:monospace; white-space:pre; padding:2rem; display:flex; justify-content:center; line-height:1.2;">
--  ▄▖▄▖▄▖▄▖▄ --
▛▖▞▌▐ ▌ ▙▘▌▌▌▌ --
▌▝ ▌▟▖▙▖▌▌▙▌▙▘ v2
</div>



# MICROD v2

This model was trained from scratch using a combination of:

- **GRPO** (Group Relative Policy Optimization) for character-level trajectory training
- **VAE filtering** to keep only high-quality trajectories
- **4x quantum harmonic oscillator** superposition for deterministic seeding

It is a compact character-level bigram model - no transformer weights,
just transition probabilities. Runs on any machine with Python 3.8+.

This Model was made on the fly with: Micro Distillery located in the /app/ folder. A single file html PWA that pulls from fanned api to create custom models.

## CHANGE LOG - Updated 10/23/2026 

Small UX + correctness maintenance release. No breaking changes. No new features.

### Fixed

- **Export modal selection drift on filter change** — toggling an export checkbox
  (e.g. unchecking "Include diagrams") rebuilt the filtered file list but preserved the
  *array index* of the previously-selected file. Because the filtered list shrinks, that
  index pointed to a **different file**, so the selection highlight and the preview pane
  silently jumped to an unrelated file. Selection is now tracked by **filename**
  (`selectedPreviewName`) instead of by array index, so it follows the same file across
  filter changes. Falls back to the first visible file when the selected file is filtered
  out, and re-syncs on `buildFiles()` (which rebuilds `modelFiles`).

- **`renderFileList` empty-state** — when the filtered list is empty, the preview pane now
  explicitly shows `"No files selected."` rather than leaving stale content from a prior
  render. (Cosmetic, but consistent with the selection-drift fix above.)

### Changed

- **Version string bumped** `3.1.2 → 3.1.3` in three places to stay consistent:
  - `brand-version` in the header
  - `modelJson.version` emitted in the export bundle
  - boot log line (`Micro Distillery v3.1.3 initialized.`)

- **`reset()`** now clears the new `selectedPreviewName` field alongside `selectedPreviewIndex`
  (which is retained for backward compatibility with any external code reading it).

### Notes

- No public API surface changed. `selectedPreviewName` is a new internal field on `Engine`;
  external callers continue to work unchanged.
- All prior regression checks still pass: `reset()` clears cache + modelFiles, `setupExportOptions()`
  runs in `boot()`, `renderFileList` guards empty arrays, `updateCache` is bounded by `maxSize=512`,
  and `downloadBundle` still validates `JSZip`/`saveAs`.

---

## Reference Architectures

The bundle metadata references these modern (2025-2026) open models as
inspiration sources. The actual model is a distilled bigram - not a
transformer checkpoint - and does not load via `AutoModelForCausalLM`.

- [SmolLM2-135M](https://huggingface.co/HuggingFaceTB/SmolLM2-135M) (HuggingFaceTB)
- [Qwen2.5-0.5B](https://huggingface.co/Qwen/Qwen2.5-0.5B) (Qwen)
- [OLMo-2-1B](https://huggingface.co/allenai/OLMo-2-0425-1B) (AllenAI)
- [TinyLlama-1.1B](https://huggingface.co/TinyLlama/TinyLlama-1.1B-Chat-v1.0) (TinyLlama)

## Model Details

- **Format**: micro-distill-bigram-v1
- **Version**: 3.1.2
- **Reference arch**: tiny-bigram
- **Vocab size**: 27 tokens
- **Transition states**: 27
- **Harmonic seed**: 1251963776
- **Created**: 2026-10-23T16:28:35.871Z

## 2 Test Wrappers Included

```bash

cd /the folder you have the files in/

# 1. Run the test harness
chmod +x test.sh run.sh
./test.sh

# 2. Or chat directly
python3 test.py
```

## Enhance test.sh / run.sh to Automate the Suite
Update your bash test harness script to systematically validate predictions, catch empty states, and verify output stability across your custom files.

## How to Add Data & Enhance the Tests

Because microd_v2 is an ultra-compact character model trained on Andrej Karpathy's names.txt corpus, you can directly enhance its test coverage by feeding it alternative datasets or updating the wrapper code.


## How the Quantum Oscillators Work for Next Tokens

The "4x quantum harmonic oscillator superposition" in microd_v2 does not process token prediction step-by-step during inference. Instead, it functions strictly at the end of the training pipeline to establish a deterministic seed.

Component	Operational Role
Simulated Superposition	The mathematical engine runs an algorithmic simulation of 4 waves/oscillators with assigned frequencies (ω = 1, φ, 2, π).
Wavefunction Collapse	When training finishes, these overlapping wave equations are collapsed down to a single integer value (the Harmonic seed: 1251963776).
Deterministic Initialization	This seed locks in the pseudorandom states needed to output reproducible transition probabilities inside model.json.


## Files in the Bundle

| File | Description |
|------|-------------|
| `model.json` | The trained bigram model (all weights) |
| `config.json` | HuggingFace-style metadata config |
| `vocab.json` | Character vocabulary |
| `tokenizer_config.json` | Tokenizer metadata |
| `knowledge_graph.json` | 2D graph used to seed the model |
| `training_logs.json` | Metrics + quantum measurements |
| `test.py` | Interactive chat wrapper |
| `test.sh` | Test harness |
| `test_readme.md` | Testing instructions |
| `run.sh` | Minimal launcher |
| `to_parquet.py` | Convert to parquet |
| `diagrams/pipeline.svg` | Training pipeline diagram |
| `diagrams/architecture.svg` | Model architecture diagram |

## Training Pipeline

1. **Fetch** - pulls corpora and model cards from HuggingFace + GitHub (2026 open sources)
2. **Knowledge graph** - builds a character-transition graph, laid out in 2D
3. **Seed** - initializes bigram transition counts from the corpus
4. **GRPO** - samples trajectories, scores with a name heuristic, boosts high-advantage transitions
5. **VAE filter** - retains only high-probability trajectories
6. **Quantum seed** - 4 harmonic oscillators (w = 1, phi, 2, pi) collapse to a deterministic seed
7. **Export** - produces a HuggingFace-ready .zip bundle


## Data Sources

This bundle was produced by fetching from these free, open endpoints:

- `raw.githubusercontent.com/karpathy/makemore/master/names.txt` (training corpus)
- `huggingface.co/api/models?search=smollm2` (model reference list)
- `huggingface.co/api/models?search=qwen2.5-0.5b` (model reference list)
- `huggingface.co/HuggingFaceTB/SmolLM2-135M/raw/main/README.md` (model card)
- `huggingface.co/allenai/OLMo-2-0425-1B/raw/main/README.md` (model card)
- `api.github.com/repos/karpathy/makemore` (repo metadata)

## License

MIT Open Sourced