hypneum-lab/micro-kiki-v4-sota

micro-kiki V4-SOTA — 35 domain-expert LoRA adapters for Qwen3.6-35B-A3B

35 per-domain LoRA adapters trained with mlx_lm (Apple MLX, BF16) on top of Qwen/Qwen3.6-35B-A3B (native MoE, 256 experts, ~3B active parameters per token). This is the canonical micro-kiki adapter set (April 2026). It supersedes micro-kiki-v35b (rank-8 pilot) and micro-kiki-v3 (pre-pivot MoE-LoRA, dead weights).

Produced by Clément Saillant (L'Electron Rare) for the Hypneum Lab dreamOfkiki research program. Runtime, routing, cognitive layer and configs: https://github.com/hypneum-lab/micro-kiki. Training execution scripts and the mlx-lm fork: https://github.com/ailiance/ailiance-mac-tuner (formerly L-electron-Rare/KIKI-Mac_tunner).

Recipe

All values below come from the per-domain adapter_config.json, config-<domain>.yaml and log-<domain>.txt files shipped in this repository.

Parameter	Value
Base model	Qwen/Qwen3.6-35B-A3B (BF16, local copy models/Qwen3.6-35B-A3B)
Framework	mlx_lm.lora (mlx-lm 0.31.x), fine_tune_type: lora
LoRA rank	16
LoRA scale (effective)	20.0 (hard-coded in the training driver)
LoRA alpha	16.0 — recorded but unused: mlx-lm LoRA layers read only rank and scale, so alpha has no effect
Dropout	0.0
Target layers	32 (last 32 of 40) for 31 domains; 20 for components, electronics, llm-ops, security
Target modules	mlx-lm default LoRA surface for this architecture (linear attention in_proj_*/out_proj, self_attn q/k/v/o, MoE switch_mlp and shared-expert projections). See the GitHub README "Adapter surface".
Trainable parameters	1 025.8 M (2.96 % of 34 660.6 M) for 32-layer adapters (from logs)
Iterations	per domain: 1000 (chat-fr, python), 500 (e.g. electronics, kicad-pcb, sql), 300 (components, llm-ops, security), others per adapter_config.json — not a uniform 200
Batch size	1, grad_accumulation_steps 1
Learning rate	1e-5, no schedule, optimizer adam
Max sequence length	1024 (512 for python); longer samples were truncated
Gradient checkpointing	true
Seed	0
Checkpoints	every 100 iterations (0000NNN_adapters.safetensors); adapters.safetensors is the final one

Adapter file sizes: 4 103 288 730 bytes (32 layers) and 2 564 555 362 bytes (20 layers). The 4 smaller adapters are smaller only because they target 20 layers — not because of smaller datasets or a lower rank.

Training data

Per-domain JSONL under data/micro-kiki/<domain>/ built by the micro-kiki data pipeline (489 K examples across 35 domains; mix of Claude/Codex CLI session traces, curated public datasets and teacher distillation — see the GitHub README and the dataset card). Exact per-domain example counts for V4: not recorded in this repository. The dataset repository clemsail/micro-kiki-v3-dataset was not publicly resolvable on 2026-10-25 (HTTP 401 anonymous); provenance details live in the GitHub repo.

Domains (35)

chat-fr, components, cpp, devops, docker, dsp, electronics, embedded, emc, freecad, html-css, iot, kicad-dsl, kicad-pcb, llm-ops, llm-orch, lua-upy, math, ml-training, music-audio, platformio, power, python, reasoning, rust, security, shell, spice, spice-sim, sql, stm32, typescript, web-backend, web-frontend, yaml-json.

About the shipped logs

• log-chat-fr.txt is a complete run: val loss 2.417 (iter 1) → 0.849 (iter 1000), train loss 0.876, 0.429 it/s, peak memory 114.33 GB.
• log-components.txt and log-security.txt are failed first attempts (Metal allocation errors at iteration 1, with a different config: seq 512 / 500 iters). The adapters were then retrained successfully with the configs recorded in adapter_config.json; the logs of those successful runs are not in this repository.
• log-llm-ops.txt and log-ml-training.txt only contain a Metal initialisation crash; the corresponding adapters were produced by later runs whose logs are not recorded.

Training host & cost

Item	Value
Host	Mac Studio M3 Ultra, 512 GB unified memory, macOS, MLX (BF16, no quantization)
Peak memory	~107 GB (project README); 114.33 GB logged for chat-fr
Metal settings	mx.set_memory_limit(460 GB), mx.set_cache_limit(32 GB) (required to avoid GPU hangs)
Wall-clock	chat-fr: 1000 iterations at 0.429 it/s ≈ 39 min (derived from the log). Other domains: not recorded
Training window	2026-04-19 → 2026-04-26
Energy / monetary cost	not recorded
Reproducibility	the training host has been powered off since 2026-10; re-running requires a comparable Apple Silicon machine

Evaluation

Measured (from the GitHub README, results/ of hypneum-lab/micro-kiki):

Metric	Value
chat-fr val loss	0.849 (base 2.417)
reasoning val loss	0.638
HumanEval pilot (pass@1)	base 0.80 → cpp adapter 0.90
HumanEval+ n=164, chat-fr adapter, max_tokens 384	pass@1 0.543 (chat-fr-164-cross)
Forgetting (angle)	chat-fr ↔ reasoning mean 79.4°; typescript ↔ python 66.6°; only 5 of 41 adapter artefacts pairwise checked
Adapter health	all 35 pass validate_adapter_health.py (all lora_B non-zero)

Caveats

• The adapters have not been re-evaluated with a comparative benchmark since training finished (last bench-complete.json predates the end of V4 training).
• On the same base model, the sibling ailiance family of Qwen3.6-35B-A3B LoRA adapters (curriculum-trained, r32, 16 layers) was benchmarked in June 2026 with iact-bench: no adapter improved over the base model except html-css, and the base alone was routed in production afterwards. Those are different adapters from this set, but the result is the closest available evidence and it does not support a general "adapters beat the base" claim for this recipe. Treat these adapters as research artefacts, not as a validated improvement.
• The project README notes that only 3 of 10 probed domains showed measurable improvement over the base model.

Limitations

• Research artefacts; no safety tuning beyond the base model.
• Sequences > 1024 tokens were truncated during training.
• Not designed for medical, legal or financial advice; outputs may be wrong or fabricated.
• Serving: Q4_K_M base recommended; at most 4 adapters active simultaneously.
• DoRA is not supported on this MoE architecture.

Usage (MLX)

from mlx_lm import load, generate

model, tokenizer = load("Qwen/Qwen3.6-35B-A3B", adapter_path="./math")
print(generate(model, tokenizer, prompt="Prove the Pythagorean theorem.", max_tokens=512))

Citation

@software{saillant_microkiki_2026,
  author    = {Saillant, Clément},
  title     = {micro-kiki: 35 Domain-Expert LoRA Adapters with Cognitive Layer on Qwen MoE},
  year      = {2026},
  publisher = {Zenodo},
  doi       = {10.5281/zenodo.19666759},
  url       = {https://doi.org/10.5281/zenodo.19666759}
}

Concept DOI 10.5281/zenodo.19666759 (resolves to the latest version; v0.3.0 is 10.5281/zenodo.19666760). Parent pre-registration: OSF 10.17605/OSF.IO/Q6JYN.

Provenance

• Code, configs, validators, eval results: https://github.com/hypneum-lab/micro-kiki
• Training execution (MLX scripts, status table): https://github.com/ailiance/ailiance-mac-tuner
• Base model: https://huggingface.co/Qwen/Qwen3.6-35B-A3B (Apache-2.0)
• Related: hypneum-lab/micro-kiki-router-v4 (domain router), hypneum-lab/micro-kiki-v35b (rank-8 pilot)

EU AI Act transparency (Regulation EU 2024/1689)

Field	Value
Provider	Clément Saillant / L'Electron Rare, for Hypneum Lab
Artefact type	LoRA adapter weights only; base model unchanged
Base model	Qwen/Qwen3.6-35B-A3B (upstream licence applies separately)
Training data origin	project corpus (CLI session traces, curated public datasets, teacher distillation) — see GitHub provenance
Intended use	technical assistance (electronics, embedded, code, FR/EN chat) in research settings
Out of scope	healthcare, legal, safety-critical autonomous decisions, malicious code
Risk classification	limited risk — Article 50 transparency obligations
Contact	clement@saillant.cc

You are using an AI model. Outputs may be inaccurate, biased or fabricated; verify independently.