| # HAKO — Hybrid Attention Kohonen Orchestrator |
|
|
| Orchestrated / routed / cooperative hybrid-attention system fusing **GHSOM + |
| CPNN + BKN + DASOM** (Kohonen core), **MoE cross-attention**, |
| **Tree-GNN reasoning**, **chain recursion + beta fusion**, a |
| **Diffusion game-loop generative layer**, a **cyclic thinking orchestrator**, |
| and **Lyapunov-guarded auto-tuning** (Robbins–Monro Θ hyper-gradient, |
| EWC-QAC, CRB-gated growth, ABMO barrier, orthogonal ensemble, GP-PGO), |
| built on **frozen quantized sources**: AMD `Qwen2.5-0.5B int4 ONNX` and |
| AMD `granite-4.0-1b AWQ grp32 ONNX` (weights decomposed into |
| `int4_proto`, `int4_xi`, expert keys `K`, and 4 source-grounded adapters per |
| source, quantization preserved). |
|
|
| All mathematical statements (lemmas, theorems, propositions) and their proofs |
| live in the **module docstrings** (`hako/**.py`); `tests/test_math_properties.py` |
| verifies them numerically before any training or publication. |
|
|
| ## Layout |
| ``` |
| hako/ |
| config.py hardware probe, budgets, storage caps |
| memory_manager.py aggressive RAM/disk watchdog (P-MEM) |
| telemetry.py 5 JSONL streams (learning/routing/orchestration/diffusion/thinking) |
| checkpoint.py compressed state save/load |
| tokenizer/byte_bpe.py parallel byte-BPE, Lemma-1 exact sharded counts |
| sources/loader.py streaming HF fetch (token via env HF_TOKEN ONLY) |
| sources/decompose.py mandated geometry decomposition (P-DEC, T15) |
| sources/runtime.py frozen ONNX embedding runtime (T-EXH) |
| core/kmeanspp.py K-Means++ + Lloyd (T-KM) |
| core/plateau.py 2D plateau density (T-PLATEAU, Morse) |
| core/ghsom.py growing hierarchical SOM (T-GHSOM) |
| core/heads.py CPNN/BKN/DASOM heads (T-HEADS) |
| core/moe_attention.py Mechanism I (T-ATTVAR corrected proof) |
| core/tree_gnn.py Mechanism II (T-DIRICHLET + Brouwer) |
| core/chain.py chain recursion + beta fusion (T-CHAIN) |
| generative/diffusion.py DDPM + game curriculum (T-CURRICULUM) |
| orchestrator/cyclic.py 4-step thinking loop (T-ORCH termination bound) |
| orchestrator/router.py REINFORCE router + Gumbel-Softmax (T-ROUTER) |
| autotune/meta.py Robbins–Monro Θ controller (T-RM) |
| autotune/ewc_qac.py Fisher clip/smooth + QAC cubic (T-QAC) |
| autotune/growth.py CRB growth gate + N-lift (T-GROWTH) |
| autotune/abmo.py attention meta-optimizer + log barrier (T-ABMO) |
| autotune/ensemble.py orthogonal ensemble + PID (T-ENS, T-PID) |
| autotune/pgo.py GP surrogate + EI + L-BFGS (T-GP) |
| train/phase*.py phase pipelines |
| publish/push_hf.py token-gated publication to PowerMachine |
| tests/test_math_properties.py 15 numerical proof checks |
| run_all.py phase 0–4 runner (math suite gates everything) |
| ``` |
|
|
| ## Run |
| ```bash |
| pip install -r requirements.txt |
| export HF_TOKEN=... # used ONLY at runtime, never stored |
| python3 run_all.py # full pipeline (~1h budget) |
| python3 run_all.py --smoke # short validation pass |
| ``` |
|
|
| ## Telemetry |
| `telemetry/{learning,routing,orchestration,diffusion,thinking}.jsonl` — |
| append-only, crash-consistent JSONL (P-TEL). |
|
|
|
|
