File size: 3,145 Bytes
e0d12b7 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 | # HAKO — Hybrid Attention Kohonen Orchestrator
Orchestrated / routed / cooperative hybrid-attention system fusing **GHSOM +
CPNN + BKN + DASOM** (Kohonen core), **MoE cross-attention**,
**Tree-GNN reasoning**, **chain recursion + beta fusion**, a
**Diffusion game-loop generative layer**, a **cyclic thinking orchestrator**,
and **Lyapunov-guarded auto-tuning** (Robbins–Monro Θ hyper-gradient,
EWC-QAC, CRB-gated growth, ABMO barrier, orthogonal ensemble, GP-PGO),
built on **frozen quantized sources**: AMD `Qwen2.5-0.5B int4 ONNX` and
AMD `granite-4.0-1b AWQ grp32 ONNX` (weights decomposed into
`int4_proto`, `int4_xi`, expert keys `K`, and 4 source-grounded adapters per
source, quantization preserved).
All mathematical statements (lemmas, theorems, propositions) and their proofs
live in the **module docstrings** (`hako/**.py`); `tests/test_math_properties.py`
verifies them numerically before any training or publication.
## Layout
```
hako/
config.py hardware probe, budgets, storage caps
memory_manager.py aggressive RAM/disk watchdog (P-MEM)
telemetry.py 5 JSONL streams (learning/routing/orchestration/diffusion/thinking)
checkpoint.py compressed state save/load
tokenizer/byte_bpe.py parallel byte-BPE, Lemma-1 exact sharded counts
sources/loader.py streaming HF fetch (token via env HF_TOKEN ONLY)
sources/decompose.py mandated geometry decomposition (P-DEC, T15)
sources/runtime.py frozen ONNX embedding runtime (T-EXH)
core/kmeanspp.py K-Means++ + Lloyd (T-KM)
core/plateau.py 2D plateau density (T-PLATEAU, Morse)
core/ghsom.py growing hierarchical SOM (T-GHSOM)
core/heads.py CPNN/BKN/DASOM heads (T-HEADS)
core/moe_attention.py Mechanism I (T-ATTVAR corrected proof)
core/tree_gnn.py Mechanism II (T-DIRICHLET + Brouwer)
core/chain.py chain recursion + beta fusion (T-CHAIN)
generative/diffusion.py DDPM + game curriculum (T-CURRICULUM)
orchestrator/cyclic.py 4-step thinking loop (T-ORCH termination bound)
orchestrator/router.py REINFORCE router + Gumbel-Softmax (T-ROUTER)
autotune/meta.py Robbins–Monro Θ controller (T-RM)
autotune/ewc_qac.py Fisher clip/smooth + QAC cubic (T-QAC)
autotune/growth.py CRB growth gate + N-lift (T-GROWTH)
autotune/abmo.py attention meta-optimizer + log barrier (T-ABMO)
autotune/ensemble.py orthogonal ensemble + PID (T-ENS, T-PID)
autotune/pgo.py GP surrogate + EI + L-BFGS (T-GP)
train/phase*.py phase pipelines
publish/push_hf.py token-gated publication to PowerMachine
tests/test_math_properties.py 15 numerical proof checks
run_all.py phase 0–4 runner (math suite gates everything)
```
## Run
```bash
pip install -r requirements.txt
export HF_TOKEN=... # used ONLY at runtime, never stored
python3 run_all.py # full pipeline (~1h budget)
python3 run_all.py --smoke # short validation pass
```
## Telemetry
`telemetry/{learning,routing,orchestration,diffusion,thinking}.jsonl` —
append-only, crash-consistent JSONL (P-TEL).
|