Hugging Face's logo

webxos
/
microd_v2

English
micro-distillery
custom
bigram
grpo
vae
distilled
smollm2
qwen2.5
olmo
tinyllama
Model card Files
xet
 Community
microd_v2 / README.md
webxos's picture
webxos
Update README.md
f6a0df5 verified
preview code
|
Raw History Blame Contribute Delete
7.08 kB
metadata 
language:
  - en
license: mit
tags:
  - micro-distillery
  - bigram
  - grpo
  - vae
  - distilled
  - smollm2
  - qwen2.5
  - olmo
  - tinyllama
library_name: micro-distillery
base_model:
  - HuggingFaceTB/SmolLM2-135M-Instruct
  - Qwen/Qwen2.5-0.5B
  - TinyLlama/TinyLlama-1.1B-Chat-v1.0
-- β–„β––β–„β––β–„β––β–„β––β–„ -- β–›β––β–žβ–Œβ– β–Œ β–™β–˜β–Œβ–Œβ–Œβ–Œ -- β–Œβ– β–Œβ–Ÿβ––β–™β––β–Œβ–Œβ–™β–Œβ–™β–˜ v2

MICROD v2

This model was trained from scratch using a combination of:

  • GRPO (Group Relative Policy Optimization) for character-level trajectory training
  • VAE filtering to keep only high-quality trajectories
  • 4x quantum harmonic oscillator superposition for deterministic seeding

It is a compact character-level bigram model - no transformer weights, just transition probabilities. Runs on any machine with Python 3.8+.

This Model was made on the fly with: Micro Distillery located in the /app/ folder. A single file html PWA that pulls from fanned api to create custom models.

CHANGE LOG - Updated 10/23/2026

Small UX + correctness maintenance release. No breaking changes. No new features.

Fixed

  • Export modal selection drift on filter change β€” toggling an export checkbox (e.g. unchecking "Include diagrams") rebuilt the filtered file list but preserved the array index of the previously-selected file. Because the filtered list shrinks, that index pointed to a different file, so the selection highlight and the preview pane silently jumped to an unrelated file. Selection is now tracked by filename (selectedPreviewName) instead of by array index, so it follows the same file across filter changes. Falls back to the first visible file when the selected file is filtered out, and re-syncs on buildFiles() (which rebuilds modelFiles).

  • renderFileList empty-state β€” when the filtered list is empty, the preview pane now explicitly shows "No files selected." rather than leaving stale content from a prior render. (Cosmetic, but consistent with the selection-drift fix above.)

Changed

  • Version string bumped 3.1.2 β†’ 3.1.3 in three places to stay consistent:

    • brand-version in the header
    • modelJson.version emitted in the export bundle
    • boot log line (Micro Distillery v3.1.3 initialized.)

  • reset() now clears the new selectedPreviewName field alongside selectedPreviewIndex (which is retained for backward compatibility with any external code reading it).

Notes

  • No public API surface changed. selectedPreviewName is a new internal field on Engine; external callers continue to work unchanged.
  • All prior regression checks still pass: reset() clears cache + modelFiles, setupExportOptions() runs in boot(), renderFileList guards empty arrays, updateCache is bounded by maxSize=512, and downloadBundle still validates JSZip/saveAs.

Reference Architectures

The bundle metadata references these modern (2025-2026) open models as inspiration sources. The actual model is a distilled bigram - not a transformer checkpoint - and does not load via AutoModelForCausalLM.

Model Details

  • Format: micro-distill-bigram-v1
  • Version: 3.1.2
  • Reference arch: tiny-bigram
  • Vocab size: 27 tokens
  • Transition states: 27
  • Harmonic seed: 1251963776
  • Created: 2026-10-23T16:28:35.871Z

2 Test Wrappers Included 


cd /the folder you have the files in/

# 1. Run the test harness
chmod +x test.sh run.sh
./test.sh

# 2. Or chat directly
python3 test.py

Enhance test.sh / run.sh to Automate the Suite

Update your bash test harness script to systematically validate predictions, catch empty states, and verify output stability across your custom files.

How to Add Data & Enhance the Tests

Because microd_v2 is an ultra-compact character model trained on Andrej Karpathy's names.txt corpus, you can directly enhance its test coverage by feeding it alternative datasets or updating the wrapper code.

How the Quantum Oscillators Work for Next Tokens

The "4x quantum harmonic oscillator superposition" in microd_v2 does not process token prediction step-by-step during inference. Instead, it functions strictly at the end of the training pipeline to establish a deterministic seed.

Component Operational Role Simulated Superposition The mathematical engine runs an algorithmic simulation of 4 waves/oscillators with assigned frequencies (Ο‰ = 1, Ο†, 2, Ο€). Wavefunction Collapse When training finishes, these overlapping wave equations are collapsed down to a single integer value (the Harmonic seed: 1251963776). Deterministic Initialization This seed locks in the pseudorandom states needed to output reproducible transition probabilities inside model.json.

Files in the Bundle

File Description
model.json The trained bigram model (all weights)
config.json HuggingFace-style metadata config
vocab.json Character vocabulary
tokenizer_config.json Tokenizer metadata
knowledge_graph.json 2D graph used to seed the model
training_logs.json Metrics + quantum measurements
test.py Interactive chat wrapper
test.sh Test harness
test_readme.md Testing instructions
run.sh Minimal launcher
to_parquet.py Convert to parquet
diagrams/pipeline.svg Training pipeline diagram
diagrams/architecture.svg Model architecture diagram

Training Pipeline

  1. Fetch - pulls corpora and model cards from HuggingFace + GitHub (2026 open sources)
  2. Knowledge graph - builds a character-transition graph, laid out in 2D
  3. Seed - initializes bigram transition counts from the corpus
  4. GRPO - samples trajectories, scores with a name heuristic, boosts high-advantage transitions
  5. VAE filter - retains only high-probability trajectories
  6. Quantum seed - 4 harmonic oscillators (w = 1, phi, 2, pi) collapse to a deterministic seed
  7. Export - produces a HuggingFace-ready .zip bundle

Data Sources

This bundle was produced by fetching from these free, open endpoints:

  • raw.githubusercontent.com/karpathy/makemore/master/names.txt (training corpus)
  • huggingface.co/api/models?search=smollm2 (model reference list)
  • huggingface.co/api/models?search=qwen2.5-0.5b (model reference list)
  • huggingface.co/HuggingFaceTB/SmolLM2-135M/raw/main/README.md (model card)
  • huggingface.co/allenai/OLMo-2-0425-1B/raw/main/README.md (model card)
  • api.github.com/repos/karpathy/makemore (repo metadata)

License

MIT Open Sourced