docs/blueprints/mindXtrain2.md

# mindXtrain — GLM-5.1, aGLM Lineage & Training-Framework Master Reference

> Foundational technical brief prepared for Gregory ("codephreak" / Professor Codephreak), BANKON / PYTHAI / DELTAVERSE, May 2026. Apache 2.0 redistribution target. Python ≥ 3.12. Podman, OpenBSD vmm, Foundry. Flat snake_case, cypherpunk2048 standard.

This document is the working reference for the mindXtrain project — the training framework that will produce aGLM v2 derivatives consumable by automindX v2 and the cognitive API at `mindx.pythai.net`. It corrects three premise errors in the original research brief, executes the full technical analysis the brief requested, and ends with a concrete construction plan and a non-trivial recommendation: **target Qwen3.5 as the primary base and treat GLM-5.1 as a premium specialist track**, on rigorous evidence laid out below. None of this is hand-waved; every figure is sourced and every disagreement between sources is flagged.

## Three corrections to the brief, before anything else

The first correction is that **the GLM-5.1 "family" the brief assumed does not exist**. There is no GLM-5.1-Air, no GLM-5.1-Flash, no GLM-5.1-AirX, no GLM-5.1-Plus, and no GLM-5.1V. The Hugging Face collection at `zai-org` contains exactly two artifacts — `zai-org/GLM-5.1` (BF16 weights) and `zai-org/GLM-5.1-FP8` (native FP8 quantization). The Z.AI documentation sidebar lists a single GLM-5.1 entry. The Ollama library exposes one tag, `glm-5.1:cloud`, which routes to Z.AI's hosted endpoint rather than running locally. Earlier-generation tiers (GLM-4.5-Air, GLM-4.5-Flash, GLM-4.7-Flash, GLM-4.7-FlashX) remain accessible on the Z.AI API but do not share GLM-5.1's `glm_moe_dsa` architecture and are not part of the same training generation. The "Air/Flash" cost-tier idea from GLM-4.5 was deliberately abandoned for GLM-5.1: Z.AI shipped one flagship designed to do everything, plus an FP8 quantization to make it deployable on a single 8×B200 node. Any planning that depends on a smaller native GLM-5.1 variant has to either use community quantizations, fall back to GLM-4.5-Air for the cheap tier, or pick a different model family for the lower rungs of the size ladder.

The second correction is that **the "GLM" inside aGLM is not Zhipu GLM**. Reading `pythaiml/automindx/aglm.py` and the `autoGLM/README-md` concept document together makes this unambiguous: aGLM stands for *Autonomous General Learning Model* (also rendered *Autonomous General Learning Machine* — both are author-sanctioned). The actual model loaded by `aglm.py` in its current canonical form is `TheBloke/llama2-7b-chat-codeCherryPop-qLoRA-GGML`, a Llama-2 GGML-era quant — not a ChatGLM or GLM-4 weight. The acronym collision with Zhipu's separate "AutoGLM" research project (`zai-org/Open-AutoGLM`, the AutoGLM-Phone-9B paper) is a permanent disambiguation hazard the v2 README must address up front. mindXtrain v2's job is therefore *not* to "wrap GLM-5.1 inside aglm.py" in the literal sense; it is to upgrade the aGLM runtime so that one of the model backends it can dispatch to is GLM-5.1 (or Qwen3.5, or any modern base), while preserving the Codephreak persona, agenda-conditioning, four-axis decomposition, and JSON-on-disk memory pattern that constitute aGLM's identity.

The third correction is that **`huggingface/ml-intern` is not a training framework**. The repository, first published around 19–21 April 2026 by Aksel Joonas Reedi and the HF AI-Agents team, is a Claude-Code-style autonomous coding agent pre-wired to Hugging Face Hub, Papers, Datasets, Jobs, and Spaces. Its `pyproject.toml` declares `name = "hf-agent"` at version 0.1.0 and lists `huggingface-hub>=1.0.1`, `litellm>=1.83.0`, `fastmcp>=3.2.0`, `pydantic>=2.12.3`, and `whoosh>=2.7.4` — but it does **not** depend on `transformers`, `peft`, `trl`, `accelerate`, `lighteval`, or `evaluate`. Distributed training is delegated entirely: ml-intern's `agent/tools/hf_jobs.py` writes a TRL-or-Transformers script in-context, submits it to a Hugging Face Job at the chosen flavor (`gpu-h100`, `gpu-a100`, etc.), polls the logs, and feeds them back into the LLM context. There is no Trainer abstraction inside ml-intern. There is also, as of late April 2026, **no LICENSE file** — issue #41 in the ml-intern repo is the open ticket asking HF to confirm; the third-party `mudler/universal-ml-intern` port assumes Apache 2.0, but that is an assumption, not an attached license. mindXtrain cannot vendor ml-intern code yet. What it **can** do — and what this document recommends — is study and reimplement five specific patterns from ml-intern (the unified `ToolRouter`, the bounded ReAct loop with doom-loop detector, the 170k-token auto-compacting `ContextManager` with Claude-Code-JSONL trajectory upload, the approval-required tool flag with live USD pricing, and the three-phase Research → Plan → Implement system prompt). These patterns belong in the *operator layer* above mindXtrain's training core, not inside it.

With those corrections in place, the rest of this document is exhaustive on the technical substance.

## Part 1 — GLM-5.1 in full technical depth

GLM-5.1 was announced via the Z.AI blog post *"GLM-5.1: Towards Long-Horizon Tasks"* (`https://z.ai/blog/glm-5.1`) and released on 7 April 2026. It is not a new pretraining run; it is a post-training refresh of GLM-5, sharing the same `glm_moe_dsa` architecture and the same paper, *"GLM-5: from Vibe Coding to Agentic Engineering"* (arXiv 2602.15763, lead author Aohan Zeng, 185-author roster). The differentiator that justifies the .1 bump, per the Z.AI blog and the model card on `huggingface.co/zai-org/GLM-5.1`, is asynchronous agent-RL training "emphasizing hundreds of rounds and thousands of tool calls" — agentic-trajectory reinforcement learning with verifiable rewards on real engineering tasks (the Z.AI team cites optimizing a vector database to 21,500 QPS over 600+ iterations and 6,000 tool calls, and KernelBench Level 3 producing a 3.6× geometric-mean speedup vs `torch.compile max-autotune`'s 1.49× over thousands of optimization rounds). The marketing tagline — "can work autonomously on a single task for up to 8 hours" — is a deliberate framing of where the model's headroom is.

### Parameter accounting and shape

GLM-5.1 is a Mixture-of-Experts decoder of total capacity ~754 B parameters with ~40.8 B active per token. The 754 B figure is what the Hugging Face model card metadata reports; Lambda's deployment guide and the GLM-5 GitHub README report 744 B; the gap (~10 B) is reconciled by the static reference site `glm51.si5.pl` — built directly from the merged `transformers/models/glm_moe_dsa/{configuration,modular,modeling}_glm_moe_dsa.py` — as MTP head plus fp32 buffers plus per-checkpoint scale tensors. Active-per-token is consistent across all sources at "~40 B".

The model has 78 decoder layers. The first three are dense; the remaining seventy-five are MoE — the configuration object encodes this as `mlp_layer_types = ["dense"] * 3 + ["sparse"] * 75`. Hidden size `d_model` is 6,144. Vocabulary is 154,880 tokens (up from 151,552 in GLM-4.6); embeddings and lm_head are untied (`tie_word_embeddings=False`), so each is a 6,144 × 154,880 matrix of about 951.4 M parameters, contributing roughly 1.9 B parameters to the total just for the input/output projections. Native context length is 202,752 tokens — about 200 K — with no YaRN or NTK extrapolation in the released config. RoPE is the NeoX/Llama split-half variant (rotate-half), applied only to the 64-dimensional "rope" subspace via `attribute_map = {"head_dim": "qk_rope_head_dim"}`; `rope_theta` is configurable in `rope_parameters` and `partial_rotary_factor` is honored. Crucially, GLM-5.1 explicitly **removed** the interleaved-RoPE path GLM-5 inherited from DeepSeek V3 — `rope_interleave` raises `AttributeError`. Any weight-conversion script written for GLM-5 that assumes the DeepSeek V3 RoPE will silently break on GLM-5.1.

Normalization is RMSNorm pre-norm, weight-only, ε = 1e-5 throughout the body and inside the attention's `q_a_layernorm` and `kv_a_layernorm`. The lone exception is the DSA Indexer's `k_norm`, which is a standard `nn.LayerNorm` with ε = 1e-6 — a deliberate departure to match the DeepSeek V3.2 reference implementation. Activations are SiLU inside a SwiGLU GLU MLP (`gate_proj`, `up_proj`, `down_proj`, no biases). Attention bias is False everywhere. The FP8 escapes are explicit: `_keep_in_fp32_modules = ["indexer.weights_proj"]` and `_keep_in_fp32_modules_strict = ["e_score_correction_bias"]`.

### Attention: MLA stacked with DSA

Every layer combines two attention innovations. The first is Multi-head Latent Attention from DeepSeek V2/V3: 64 query heads, 64 KV heads (in MLA, KV are produced from a low-rank latent and expanded per head, so `num_key_value_groups` is 1 and `repeat_kv` is a no-op), a query LoRA rank of 2,048 (raised from 768 in GLM-5 because the q-latent must now also feed the new DSA Indexer), a KV LoRA rank of 512, an `nope` content slice of 192 dimensions per head and a `rope` positional slice of 64 dimensions per head for total `qk_head_dim = 256`, and a value head dimension of 256. The latent KV cache is therefore 512 + 64 = 576 elements per token per layer, or roughly 89.9 KB per token over the 78 layers; the *expanded* KV cache that vanilla Transformers materializes is 64 × (256 + 256) = 32,768 elements per token per layer, about 5.13 MB per token. That ratio is the entire reason MLA exists.

The second innovation is the DeepSeek Sparse Attention Indexer, the headline change from GLM-5. The Indexer has 32 heads of 128 dimensions each, takes the post-`q_a_layernorm` query latent for queries, reads raw `hidden_states` through its own `wk` projection for keys, computes scores as `Σ_h weights[s,h] · ReLU(softmax_scale · q[s,h] · k[t])` in fp32, and selects `index_topk = 2,048` keys per query. The Indexer adds about 9.4 M parameters per layer (~5.4% of the layer's attention weight) and maintains its own KV cache outside the standard `DynamicCache` as a layer-local `_cached_keys` tensor (~256 B per token in bf16). The flash-mla kernel from `kernels-community/flash-mla` reads the `topk_indices` kwarg directly to skip masked positions; the eager and SDPA paths instead materialize a full `[B, S, T]` `-inf` mask and `scatter_` zeros at the indexer-selected columns. The point of the Indexer is that it makes per-token attention compute **independent of sequence length** beyond about 2 K, which is what makes 200 K-token decode tractable at 754 B-parameter scale. Note that DSA does *not* shrink the KV cache — it shrinks compute. KV cache memory at long context is still dominated by MLA's latent representation.

### MoE block

The 75 sparse layers carry 256 routed experts plus 1 always-on shared expert with intermediate size 2,048; top-k routing selects 8 routed experts per token, so 8 + 1 = 9 experts are active at any moment. The dense FFN in layers 0–2 has intermediate size 12,288 (about 2× hidden). Router scoring uses sigmoid (not softmax) in fp32, with auxiliary-loss-free balancing via a per-expert `e_score_correction_bias` that is added for *selection only*, not for weighting — the DeepSeek V3 trick. `routed_scaling_factor` is 2.5 (raised from 1.8 in GLM-5), multiplied into the L1-normalized top-k sigmoid scores before they are added to the residual. The shared-expert path is added without the routing scale, so it acts as a constant baseline. The `n_group`/`topk_group` fields are 1/1, which collapses the multi-group routing back to plain top-8 over all 256 experts. Per-MoE-layer capacity is 256 × 37.75 M (3 × 6144 × 2048) + 1 × 37.75 M + 1.57 M router ≈ 9.70 B; per-MoE-layer active is 8 × 37.75 M + 37.75 M ≈ 341.3 M FFN + 174.4 M attention ≈ 515.7 M per token per layer. Aggregate: ~743.6 B capacity, ~40.8 B active, matching the official "754 B / ~40 B" within the noted reconciliation gap.

### MTP head, tokenizer, chat template

GLM-5.1 ships a Multi-Token Prediction head for speculative decoding, DeepSeek V3 style. The Hugging Face transformers integration uses `_keys_to_ignore_on_load_unexpected = [r"model\.layers\.78.*"]` to admit a layer-78 placeholder for the MTP head; the main body has 78 layers indexed 0–77. vLLM exposes MTP via `--speculative-config.method mtp --speculative-config.num_speculative_tokens 3`; SGLang uses EAGLE for the same role. The tokenizer is a SentencePiece-derived BPE with 154,880 vocabulary entries; special tokens include `<|system|>`, `<|user|>`, `<|assistant|>`, `<|observation|>`, `<|tool|>`, plus thinking delimiters and tool-call tokens. vLLM's `--tool-call-parser glm47 --reasoning-parser glm45` flags select the right parser pair. **Thinking mode is enabled by default** in GLM-5.1 — a behavioral change from GLM-5 — and is disabled with `chat_template_kwargs: {"enable_thinking": false}`. The `chat_template.jinja` (~4.67 kB) supports Claude-style deferred tool loading via `defer_loading=True`, which puts tool schemas into the tool *result* messages rather than the system prompt, and accepts both `List[tool]` and `List[tool.function]` shapes for SGLang compatibility. There are two thinking flavors: *Interleaved Thinking* (default, general chat) and *Interleaved + Preserved Thinking* for agentic workflows like Claude Code, Roo Code, and Kilo Code, toggled via the `enable_thinking` and `clear_thinking` chat-template kwargs.

### Training methodology, with explicit gaps

Pretraining used 28.5 trillion tokens, up from 23 T for GLM-4.5. The English:Chinese:other split is not disclosed in the model card or the extractable arXiv abstract; the tokenizer is jointly built over English and Chinese with code and tool tokens, and the Hugging Face metadata tags the model with both `English` and `Chinese`. One secondary blog (`aimadetools.com`) claims pretraining was done on 100,000 Huawei Ascend 910B chips with zero NVIDIA dependency; this is **not** corroborated by the arXiv paper, the model card, or the Z.AI blog and should be treated as unverified reporting, although the parallel choice of `xLLM` (JD's Ascend-aware serving stack) as a first-class deployment target lends it some weight. Total pretraining FLOPs are not disclosed.

Post-training is described in the arXiv 2602.15763 abstract — the only programmatically extractable text from the paper at the time of this research; the PDF body is not machine-readable in current archives — as using an asynchronous reinforcement-learning infrastructure called `slime`, open-sourced at `github.com/THUDM/slime`. `slime` decouples generation from training so that fine-grained RL iterations can run without blocking the trainer. The asynchronous agent-RL algorithms are said to "improve RL quality, enabling the model to learn from complex, long-horizon interactions more effectively." For GLM-5.1 specifically, the differentiator is that this RL was run with verifiable rewards over real engineering tasks at "hundreds of rounds and thousands of tool calls" depth. The choice of PPO vs GRPO vs DPO, the SFT data mix, the reward-model architecture, the cold-start reasoning data, and the source of any distillation are **not disclosed**. mindXtrain's RL track will therefore have to either trust `slime` as the operational primitive (it is GPL-3.0 / Apache 2.0 / MIT compatible — verify per repo file) or stand up its own GRPO/DPO loops via TRL.

### Benchmarks: vendor-claimed vs independent

Z.AI's published numbers for GLM-5.1 are deliberately concentrated on long-horizon agentic and engineering benchmarks — SWE-Bench Pro 58.4 (claimed SOTA, 0.7 points above GPT-5.4's 57.7 and 1.1 above Claude Opus 4.6's 57.3), Terminal-Bench 2.0 63.5 with Terminus-2 / 66.5 with Claude Code (vs Gemini 3.1 Pro at 68.5), CyberGym 68.7 (claimed SOTA over Claude Opus 4.6 at 66.6), BrowseComp 68.0 (claimed SOTA), MCP-Atlas 71.8, τ³-Bench 70.6, Tool-Decathlon 40.7, Vending Bench 2 final balance \$5,634, AIME 2026 95.3, HMMT February 2026 82.6, GPQA-Diamond 86.2, HLE 31.0 / HLE w/ Tools 52.3. Z.AI explicitly did not publish numbers for MMLU, MMLU-Pro, CMMLU, C-Eval, BBH, MATH-500, GSM8K, HumanEval, MBPP, LiveCodeBench, BigCodeBench, BFCL v3, AgentBench, GAIA, vanilla SWE-bench, IFEval, Arena-Hard, RULER, or NIAH. If mindXtrain needs any of those, they have to be re-run locally.

The independent picture is more mixed. Artificial Analysis's Intelligence Index v4.0 places GLM-5.1 (Reasoning) at 51 and GLM-5.1 (Non-Reasoning) at 44 — well above the open-weight reasoning median of 29 but a clear step below GPT-5.4 / Gemini 3.1 Pro / Claude Opus 4.6, which sit closer to 65–75. AAII v4.0 evaluation consumed about 110 M output tokens (vs a 40 M peer median, "very verbose") at \$543.95 total. BenchLM's *provisional* leaderboard ranks GLM-5.1 #14 of 115 models with an overall 83 — but its *verified* leaderboard, which only counts source-attached scores, places GLM-5.1 #21 of 23. That gap is the cleanest signal that vendor-reported scores are running ahead of independently re-run scores. LiveBench had not yet ingested GLM-5.1 in the captures available at the time of research; LMSYS Chatbot Arena had not surfaced a GLM-5.1 ELO yet (GLM-5 base sits around 1451 and was reported as the #1 open model in the GLM-5 paper); HuggingFace Open LLM Leaderboard v2 cannot evaluate models of this size under its compute budget. The Scale AI SEAL leaderboard hosts the SWE-Bench Pro 58.4 number with the asterisk denoting self-submission and no independent re-run as of this report. The fair characterization is that GLM-5.1 is genuinely a top-tier open-weights model that sits within roughly 5–10% of the absolute frontier closed models on most coding benchmarks, narrowly leads on SWE-Bench Pro (self-reported), and is the strongest open agentic-coding model available in May 2026 — but the Z.AI marketing claim of "outperforming GPT-5.4, Claude Opus 4.6, and Gemini 3.1 Pro" is true on a single benchmark and false on most others. Cherry-picked headline.

### Inference characteristics and VRAM math worked out

Weight memory at 754 B parameters: BF16 ≈ 1,508 GB (what `zai-org/GLM-5.1` ships, marked "BF16 · F32" in the HF metadata); native FP8 ≈ 754 GB (what `zai-org/GLM-5.1-FP8` ships — the on-disk checkpoint is 756 GB across 142 safetensors shards of about 5.36 GB each); INT4 ≈ 377 GB (community quants, currently one such model listed under HF "Quantizations"); INT8 ≈ 754 GB (rare — not commonly distributed). GGUF Q4_K_M / Q5_K_M / Q6_K / Q8_0 are **not available** because llama.cpp does not yet ship the `glm_moe_dsa` architecture; Unsloth has shipped Dynamic 2.0 GGUF quants in the UD-IQ2_M (~241 GB) through UD-Q8_0 range at `huggingface.co/unsloth/GLM-5.1-GGUF` by carrying a ggml-org/llama.cpp build, but the upstream PR numbers landing the architecture were not surfaced in the research pass and the architecture is closely related to GLM-4.5/4.6.

KV cache at batch 1 in BF16, expanded form: about 3.7 GB at 4 K tokens, 30 GB at 32 K, 120 GB at 128 K, 187 GB at 200 K. In MLA-compressed (latent) form: 0.36 GB at 4 K, 2.9 GB at 32 K, 11.5 GB at 128 K, 18 GB at 200 K. Total VRAM for FP8 weights plus KV at 128 K context: 754 + 120 = 874 GB (expanded) or 754 + 11.5 = 765 GB (MLA-compressed). For INT4 + MLA-compressed KV: 377 + 5.7 = 383 GB. With DSA active on a ~2 K window, FP8 + 0.18 GB = 754 GB. The official Lambda minimum is **a single 8×B200 (HGX B200) node** to load the FP8 model with usable context. **An RTX 4090 24 GB cannot run GLM-5.1 at any quantization.** **An M3 Ultra 192 GB unified-memory Mac cannot run the full model at any quantization** — the FP8 weights alone are 754 GB; in principle the INT4 quant would fit if MLX support existed, but `ml-explore/mlx-lm` issue #879 ("Add model support for GLM-5 (`glm_moe_dsa` architecture)") was still open at research time. **An M2 Max 96 GB cannot run any usable form of GLM-5.1.** A100 80 GB requires roughly 10× to load FP8.

Throughput, from Lambda's official benchmarks at 8 K input / 2 K output and 32 concurrent users: SGLang 0.5.10 on 1× HGX B200 produces 1,345.4 output tokens/s, 42.0 per-user, 6,727.2 total, with mean TTFT 1,073 ms and mean inter-token latency 58.6 ms. vLLM 0.19.0 on the same hardware produces 1,265.4 / 39.5 / 6,327.2 tokens/s with TTFT 1,317 ms, ITL 57.8 ms. Artificial Analysis's median across providers is 53.7 tok/s output with TTFT 1.73 s — slower than the open-weight median of 56.3 tok/s in this scale class.

The recommended decoding parameters from the Z.AI model card are: temperature 1.0, top_p 0.95, max generation 131,072 for default and general use; temperature 0.7, top_p 1.0, max 16,384, with `enable_thinking=true` and `clear_thinking=false` for Terminal-Bench / coding-agent flows; temperature 0, max 16,384, Interleaved + Preserved thinking for τ²-Bench-style pure tool-use flows.

### Framework support matrix

Hugging Face Transformers requires version 5.3.0 or later — the `glm_moe_dsa` architecture is not in transformers 4.x. (Z.AI's README states "v0.5.3+", which is a typographical error for v5.3.0+; Lambda's deployment guide uses the corrected version.) `trust_remote_code` is no longer required after the merge. vLLM supports GLM-5.1 from 0.19.0 with custom Docker images at `vllm/vllm-openai:glm51` and `vllm/vllm-openai:glm51-cu130` (CUDA 13+); the recipes URL is `https://docs.vllm.ai/projects/recipes/en/latest/GLM/GLM5.html`. There is a known caveat: tool-calling combined with MTP-enabled speculative decoding requires the vLLM main branch rather than the 0.19.0 release. SGLang requires v0.5.10 (not 0.5.10rc0, which has a known flashmla bug fixed in the 0.5.10 release). xLLM v0.8.0+ supports the Ascend NPU path. KTransformers v0.5.3+ supports CPU-offload-plus-GPU hybrid. Ollama supports `glm-5.1:cloud` (cloud-only) as of research time; an open issue at `github.com/ollama/ollama/issues/15412` tracks offline support. A community fork at `ollama.com/frob/glm-5.1` imports `unsloth/GLM-5.1-GGUF` and requires a patched Ollama build with PR #14864 applied; the maintainer notes tool-calling is poor on this fork pending an Ollama PARSER. **Important known bug:** Unsloth's discussion thread on `huggingface.co/unsloth/GLM-5.1-GGUF/discussions/4` warns that CUDA 13.2 produces gibberish or breaks tool calling on Gemma 4 and GLM-5.1; NVIDIA had not issued a fix at research time. Use CUDA 13.0 or 13.1 for any GGUF deployment.

A canonical vLLM launch and a canonical SGLang launch, copy-pasteable:

```bash
vllm serve zai-org/GLM-5.1-FP8 \
  --tensor-parallel-size 8 \
  --max-model-len 202752 \
  --max-num-seqs 64 \
  --speculative-config.method mtp \
  --speculative-config.num_speculative_tokens 3 \
  --tool-call-parser glm47 \
  --reasoning-parser glm45 \
  --enable-auto-tool-choice \
  --chat-template-content-format=string \
  --served-model-name glm-5.1-fp8
```

```bash
SGLANG_ENABLE_SPEC_V2=1 \
sglang serve \
  --model-path zai-org/GLM-5.1-FP8 \
  --tp-size 8 \
  --tool-call-parser glm47 \
  --reasoning-parser glm45 \
  --speculative-algorithm EAGLE \
  --speculative-num-steps 3 \
  --speculative-eagle-topk 1 \
  --speculative-num-draft-tokens 4 \
  --mem-fraction-static 0.85 \
  --served-model-name glm-5.1-fp8
```

### API access

The English portal is `https://api.z.ai/api/paas/v4/`, with OpenAI-compatible chat completions at `POST /chat/completions`, `Authorization: Bearer <key>`, keys managed at `https://z.ai/manage-apikey/apikey-list`. The Chinese portal is `bigmodel.cn`. The model ID is `glm-5.1`; siblings include `glm-5`, `glm-5-turbo`, `glm-4.7`, `glm-4.7-flash`, `glm-4.7-flashx`, `glm-4.6`, `glm-4.5`, `glm-4.5-air`, `glm-4.5-airx`, `glm-4.5-x`, and `glm-4.5-flash`. The Python SDK is `pip install zai-sdk` (≥0.2.2); Java is Maven `ai.z.openapi:zai-sdk:0.3.3`; any OpenAI-compatible client with `base_url="https://api.z.ai/api/paas/v4/"` works. Pricing for GLM-5.1 in USD per 1 M tokens is \$1.40 in / \$0.26 cached input / \$4.40 out, with cache storage free for a limited time. Third-party reseller pricing varies: OpenRouter `z-ai/glm-5.1` is \$1.05 / \$3.50 (output capped at 65,535 tokens); Requesty / Fireworks match the official \$1.40 / \$4.40 (max output 25,000, 202 K context with prompt caching "up to 90%"); Inworld / DeepInfra match OpenRouter at \$1.05 / \$3.50; the Artificial Analysis median is \$1.40 / \$4.40. Feature flags supported on the native API include OpenAI-compatible `tools` arrays (with Claude-style deferred tool loading), MCP integration, structured JSON output, streaming including tool-streaming output, prefix/context caching, the `thinking: {"type": "enabled" | "disabled"}` toggle, and a built-in web search tool at \$0.01 per use. Vision input is **not** supported on `glm-5.1` — use the sibling `glm-5v-turbo`. There is no free tier on `glm-5.1` itself; the free tier exists on the Flash variants of earlier generations.

### License — the most important section for BANKON

GLM-5.1 weights ship under the **MIT License** on Hugging Face. The model card YAML for both `zai-org/GLM-5.1` and `zai-org/GLM-5.1-FP8` declares `license: mit`; the Unsloth GGUF mirror at `huggingface.co/unsloth/GLM-5.1-GGUF` does the same; Wikipedia summarizes Z.ai's policy as "released under the free and open-source MIT License since July 2025." The Hugging Face license-label "mit" maps in HF's taxonomy to the canonical SPDX MIT text — HF rejects uploads that declare `license: mit` if the LICENSE file deviates. Treat this as high-confidence even though the literal raw bytes of `github.com/zai-org/GLM-5/blob/main/LICENSE` returned 429s during the research pass. (Note: there is a code-vs-weights split — the GitHub repo header for `zai-org/GLM-5` reads "Apache License 2.0" in the search index sidebar, which means the *code* in the repo is Apache 2.0 while the *weights* on Hugging Face are MIT. This is the same pattern Z.ai used for GLM-4.5 and GLM-4.6.)

Verbatim, the operative MIT text for the weights is:

```
MIT License

Copyright (c) 2026 Z.ai (or "ZHIPU AI" as published)

Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHER WISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
```

Clause-by-clause: **commercial use is unrestricted**; the grant explicitly covers "use, copy, modify, merge, publish, distribute, sublicense, and/or sell." **Redistribution is unrestricted** subject to preserving the copyright notice and the permission text. **Derivative works can be released under any license BANKON chooses**, including Apache 2.0 — MIT is permissive, non-copyleft, and does not propagate. **Attribution is the MIT notice preservation, nothing more** — no name-prefix requirement, no "Powered by GLM" badge, no UI credit. **There are no field-of-use restrictions on the weights** — no military prohibition, no surveillance carve-out, no critical-infrastructure exclusion, no biosecurity rider. There is no patent grant (this is MIT's standard weakness vs Apache 2.0; patent rights are at most implied), no trademark grant (so do not use "GLM", "Zhipu", or "Z.ai" marks in BANKON product names in ways that suggest endorsement — naming the derivative `aGLM-BANKON` is borderline, and a more defensible product mark in user-facing UI is something like "MindX" with a model-card technical name of `aGLM-BANKON`), no AUP, no MAU thresholds, and no geographic restrictions encoded in the license itself. Z.ai is on the U.S. Commerce Department Entity List as of January 2025 — that is a sanctions/export-controls matter affecting U.S. companies *transacting with* Z.ai as a corporate counterparty, not a license restriction encoded in the MIT text; the consensus practice is that downloading MIT-licensed open weights is not Entity-List-restricted activity, but BANKON should confirm with counsel.

The hosted-API regime is separate. `https://docs.z.ai/legal-agreement/terms-of-use` governs `api.z.ai`, `BigModel.cn`, and `chat.z.ai` and contains AUP-style language including a prohibition on using outputs "for the development, training, labeling, fine-tuning, optimization, iteration, or similar activities related to external models" or to "develop, train, or enhance algorithms or models that compete with us"; restrictions on services "requiring subject qualification, including but not limited to medical services, legal services" plus "any decision-making behavior, operation of critical infrastructure, transportation technologies, heavy machinery"; and a clause stating "If you suffer damage after training, fine-tuning and development and claim that we should bear the responsibility, you shall prove that the damage is unrelated to your training, fine-tuning and development, otherwise we shall be exempted from liability for the damage." These restrictions are **API-only** and do not bind BANKON if it self-hosts the open weights.

The compliance recipe for BANKON is therefore: pull the FP8 weights from `huggingface.co/zai-org/GLM-5.1-FP8`, self-host on owned/leased GPUs (vLLM or SGLang on a single 8×B200 or two 8×H100s), and avoid touching `api.z.ai` for any production path that would create a contractual nexus. For the `aGLM-BANKON` derivative published under Apache 2.0, ship a `LICENSE` file containing the Apache 2.0 text, ship a `NOTICE` file containing the verbatim upstream MIT notice ("Copyright (c) 2026 Z.ai. Licensed under the MIT License. See LICENSE-MIT-upstream for full text."), ship `LICENSE-MIT-upstream` with the MIT text and Z.ai's copyright assertion, and stamp the model card with "Copyright (c) 2026 BANKON — all rights reserved. Fine-tuned from `zai-org/GLM-5.1`, originally licensed under the MIT License (Copyright (c) 2026 Z.ai)." Dual-licensing is permitted; there is no copyleft pull-through. The derivative does not have to inherit the MIT licence — only the upstream notice must be preserved.

Compared to peers: Qwen3 ships under Apache 2.0 across the entire family — equivalent for redistribution, slightly stronger on patent posture, slightly heavier on NOTICE compliance overhead. DeepSeek V3.2 / R1 are MIT for both code and weights — equivalent. Phi-4 / Phi-4-mini are MIT — equivalent. Llama 3.3 / Llama 4 ship under the Llama Community License with the 700 M MAU clause, the "Built with Llama" name-prefix and badge requirement, and the AUP at `llama.meta.com/llama3_3/use-policy` incorporated by reference — *incompatible* with clean Apache 2.0 downstream redistribution. Gemma 1–3 carried the Gemma Terms of Use with a Prohibited Use Policy that propagates to downstream redistributors — restrictive — but **Gemma 4** (April 2026) flipped to Apache 2.0, becoming compatible. Mistral's flagship line (Mistral Large, Codestral, Mistral Small) used the Mistral Research License (non-commercial) until **Mistral Large 3** (December 2025), which moved to Apache 2.0 along with Ministral 3 and Mistral Small 4. Cohere Command A is CC-BY-NC 4.0 with an Acceptable Use Addendum — non-commercial only — *incompatible*. Falcon 3 ships under TII Falcon License 2.0, Apache-2.0-derived but with an enforceable AUP — most legal teams will treat this as restrictive.

## Part 2 — autoGLM and aGLM lineage as foundational context

The pythaiml/automindx repository at `https://github.com/pythaiml/automindx` (id 686099738, network root for the GATERAGE fork) is 84.0% Python, 13.3% Shell, 2.7% Dockerfile, with 23 commits on `main`, 2 stars, 4 forks, and a flat layout: `aglm.py`, `automind.py`, `memory.py`, `uiux.py`, paired with `algm.md`, `automind.md`, `memory.md`, `uiux.md`, plus `4096chunk.md`, `INSTALL.md`, `DOCUMENTATION.md`, `Dockerfile`, `LICENSE`, `README.md`, `automindx.install`, `chunk4096.py`, `hfUIUX.py`, `hfapp.py`, `hfmemory.py`, and `requirements.txt`. The README declares the project's core composition as `codephreak = uiux.py + memory.py + automind.py + aglm.py` and states the persona explicitly: *"Professor Codephreak is an expert in machine learning, computer science and computer programming."* The runtime entry point is `python3 uiux.py --model_name="TheBloke/llama2-7b-chat-codeCherryPop-qLoRA-GGML" --tokenizer_name="TheBloke/llama2-7b-chat-codeCherryPop-qLoRA-GGML" --model_type="ggml" --save_history --file_name="llama-2-7b-chat-codeCherryPop.ggmlv3.q4_1.bin"`. The license is GPL-3.0 by inheritance (the GATERAGE/aglm fork is explicitly GPL-3.0; the wider codephreak ecosystem — easyAGI, RAGE, MASTERMIND — is uniformly GPL-3.0; the README footer asserts "MASTERMIND (c) codephreak GPLv3 2024"); the LICENSE bytes were not retrievable in the research pass but the inheritance is unambiguous.

`aglm.py` itself is a Hugging Face Transformers wrapper, not an agent. The imports — reconstructed from the Hugging Face `aGLM` org card, which is an authoritative author paraphrase — are `os`, `glob`, `ujson`, `psutil`, `transformers.AutoModelForCausalLM`, `transformers.AutoTokenizer`, and `automind.format_to_llama_chat_style`. The single class is `LlamaModel(model_name, models_folder)`, with methods `initialize_model()` (loads tokenizer and model from `models_folder + model_name` via `AutoTokenizer.from_pretrained` and `AutoModelForCausalLM.from_pretrained`) and `generate_contextual_output(conversation_context)` (which formats via `format_to_llama_chat_style`, tokenizes, runs `model.generate(...)`, and decodes). Module-level helpers are `determine_batch_size()` (uses `psutil.virtual_memory()` against a hard-coded `MAX_MEMORY_USAGE` to decide how many memory-file JSONs to load per batch) and `main()` (globs `memory/*.json`, batches via `determine_batch_size()`, reads with `ujson`, builds `conversation_context`, calls `LlamaModel.generate_contextual_output(context)`, prints). There is no async, no asyncio, no LangChain, no SuperAGI, no AutoGen, no `openai`, no `anthropic`, no `chromadb`, no `faiss`, no `pgvector`. Memory is `glob`-walked from `*.json` files via `ujson`; persistence happens via `memory.save_conversation_memory(...)` writing timestamped JSON files.

The patterns that mindXtrain v2 must preserve are precise. First, the **four-axis decomposition**: `uiux.py` (interface) plus `memory.py` (persistence) plus `automind.py` (prompt/format) plus `aglm.py` (model). Second, the **`.py` paired with `.md` discipline** — every Python module has a sibling Markdown documentation file colocated. Third, **shallow flat class hierarchy**: `LlamaModel`, `DialogEntry`, `EasyAGI`, `AGI`, `LogicTables`, `SocraticReasoning`, `SelfHealingSystem`, `BDI`, `Memory` — no mixins, no ABCs, no Protocols; one responsibility each, instantiated once. Fourth, **synchronous default surface** — `LlamaModel.generate_contextual_output` is sync, `EasyAGI.main_loop` is sync, the whole stack is a blocking REPL or batch. Fifth, **named-class registry dispatched by string key** for tool-like backends (`GPT4o`, `GroqModel`, `OllamaModel` selected by `APIManager` in easyAGI). Sixth, **append-only JSON-on-disk memory with batch-glob replay** ordered by filename timestamp. Seventh, **`psutil`-driven memory budgeting** generalizable to a `ResourceBudget` helper used identically by training, eval, and inference. Eighth, **CLI flags with no config file** — argparse in `uiux.py`, with `automindx.install` shell-script baking the canonical invocation. Ninth, **persona-as-agenda**: the system prompt isn't a role description but contains an explicit *agenda* (the model is told "your job is to build the automindx deployment environment"). The agenda string must remain a first-class field in v2, not baked into a string.

The gaps mindXtrain must fill are equally precise. The hard-coded Llama-2 chat assumption via `format_to_llama_chat_style` must be replaced by a `ChatTemplate` abstraction that picks the right template for Llama-3, Mistral, Qwen, GLM-4, GLM-5.1, and any future base, falling back to `tokenizer.apply_chat_template`. There is no multi-model registry — `LlamaModel` is one class for one family, with a separate dual-path GGML loader implicit in `uiux.py`'s `--model_type="ggml"` branch; v2 needs `class ModelRegistry` with `register(name, factory)` and `get(name)`, supporting backends `hf-transformers`, `llama-cpp-python`, `ollama`, `vllm`, `openai`, `anthropic`, `groq`, and the Z.ai API. The `autoGLM/litellm` fork already foreshadows this; wire it. There is no fine-tuning support — `aglm.py` does inference only — and the `autoGLM/levanter` fork (Apache 2.0, JAX + named tensors) was added as the latent training rail but never integrated. There is no evaluation harness, no observability, no memory layer beyond JSON-on-disk, no tokenizer-aware truncation (the `chunk4096.py` 4096-char ceiling is a workaround for context-window saturation), no session/concurrency primitives, no agent loop in `aglm.py`, no prompt-as-data declarative persona files, no CI, and no tests. Each gap is a real technical debt item, not a feature wishlist.

The wider org context matters because it dictates v2's integration surface. Under `autoGLM/`, the active source repos are `autoGLM/easyAGI` (Python, GPL-3.0, the openmindx → easyAGI point-of-departure stack with modules `EasyAGI`, `AGI`, `LogicTables`, `Reasoning`, `SocraticReasoning`, `SelfHealingSystem`, `Memory`, `GPT4o`, `GroqModel`, `OllamaModel`, `APIManager`, `BDI`); `autoGLM/funAGI` (the first "working" instance with `EasyAGI.main_loop`, archived as the canonical reference); `autoGLM/automindx` (org-level mirror of the canonical `pythaiml/automindx`); and `autoGLM/README-md`, the canonical aGLM concept document defining the architecture as supervised+unsupervised learning with subsystems RAGE (retrieval-augmented memory), machine dreaming, MASTERMIND (logic+prediction), blockchain-anchored knowledge "THOTs" (Theories of Hypothetical Output Trajectories) on decentralized storage, and Continuous Adaptation and Optimization (auto-tuning / self-healing). Forked-in tooling includes `autoGLM/RAGE` (GPL-3.0, retrieval engine), `autoGLM/imaginarium` (TypeScript, NLP UI), `autoGLM/pgvectorscale` (Rust, PostgreSQL-license, the intended long-term-memory backend), `autoGLM/litellm` (the OpenAI-format multi-LLM router), `autoGLM/levanter` (Apache-2.0, JAX-based scalable training — *the latent fine-tuning rail that was never wired into aglm.py*), and `autoGLM/anything-llm` (the desktop/Docker RAG+agent UI candidate). The most complete public surface of the aGLM concept lives at `GATERAGE/aglm`, which is a fork of `pythaiml/automindx` plus MASTERMIND modules (`prediction.py`, `nonmonotonic.py`, `socratic.py`, `reasoning.py`, `logic.py`, `epistemic.py`, `autonomize.py`, `bdi.py`, `terminai.py`, `terminai_module.py`, `SimpleCoder.py`, `model_handler.py`, `controller.py`, `config.json`, `config.py`, `main.py`, plus seventeen numbered `UIUX*.py` evolution snapshots) and is flagged in its own README as "currently BROKEN and useful as reference point of aGLM MASTERMIND and RAGE for modular component display only."

The integration topology is therefore: mindX (`github.com/abaracadabra/mindX`, augmentic-intelligence orchestration, public face `mindx.pythai.net`) is the consumer of the aGLM v2 inference API; PYTHAI hosts the canonical `automindx`, `funAGI`, `pgvectorscale` and the domain anchors `ai.pythai.net`, `gpt.pythai.net`, `rage.pythai.net`, `bankon.pythai.net`, `agenticplace.pythai.net`; MASTERMIND is the rational-controller layer above aGLM (intended composition `MASTERMIND(aGLM, RAGE, BDI)`); RAGE / GATERAGE provides retrieval-augmented memory; DELTAVERSE supplies decentralized/metaDAO settlement; BANKON supplies banking-OS infrastructure (`bankonOS`, `BANKONPYTHAI` with the Algorand ASA `203977300`, the ERC-8004 Identity Registry at `0x8004A169FB4a3325136EB29fA0ceB6D2e539a432` and Reputation Registry at `0x8004BAa17C55a88189AE136b182e5fdA19dE9b63`); Lighthouse Storage / Filecoin is the decentralized-knowledge-storage target named in `autoGLM/README-md`; and `cypherpunk2048/x402` provides the HTTP-402 micropayment protocol for paid inference settlement. mindXtrain produces `aGLM-BANKON` checkpoints, anchors their provenance via ERC-8004 attestations and Lighthouse-Filecoin CIDs, and ships them into automindX v2 which is consumed by mindX at `mindx.pythai.net`.

## Part 3 — `huggingface/ml-intern` patterns to adopt for the operator layer

The five highest-leverage patterns from ml-intern, all of them in the *operator* layer above the trainer, are these.

The **`ToolRouter` and `ToolSpec` dispatcher** (`agent/core/tools.py`, ≈230 LOC) unifies built-in async handlers, MCP JSON-RPC, and OpenAPI specs behind one OpenAI-compatible tool schema, with a deny-list (`{"hf_jobs", "hf_doc_search", "hf_doc_fetch", "hf_whoami"}`) that prevents MCP-supplied tools from shadowing optimized built-ins, and graceful MCP-error capture that returns errors as strings rather than bubbling exceptions. The dataclass is `@dataclass class ToolSpec: name: str; description: str; parameters: dict; handler: Callable | None; needs_approval: bool = False`. mindXtrain's heterogeneous backends — start-run, eval-checkpoint, push-to-hub, deploy-to-API, anchor-to-IPFS, mint-ERC-8004-attestation — map cleanly onto this single surface.

The **bounded ReAct loop with explicit doom-loop detector** (`agent/agent_loop.py`, `submission_loop` and `Handlers.run_agent`) caps autonomous iterations at 300 and runs `DoomLoopDetector.observe(resp.tool_calls)` after each LLM turn; when repeated tool-call patterns (same tool, same args N times) are detected, a corrective system message is injected into the next call to break the cycle. Without this, autonomous training agents wedge inside the first hour of a long run.

The **`ContextManager` with 170 K auto-compaction and Claude-Code-JSONL trajectory upload** (`agent/core/context.py`) handles two responsibilities: it triggers `MessageCompactor` summarization when the message buffer crosses ~170 K tokens, and it serializes every session in Claude-Code JSONL format and pushes it to a private Hugging Face Dataset (`{username}/ml-intern-sessions`). HF's Agent Trace Viewer auto-renders these. For mindXtrain, the JSONL format is the right schema for an auditable, replayable, RFT-corpus-ready training-record format; just swap the storage backend from HF Dataset to a `StorageProvider` interface that writes equally to local FS, HF Datasets, IPFS, or Lighthouse-Filecoin.

The **approval-required tool flag with live USD pricing** combines a per-`ToolSpec` `needs_approval: bool` flag with a generic `await session.await_approval(tc)` flow wired to interactive CLI prompt, web "approve in browser" button, and Slack interactive approval. Live USD/hour pricing is surfaced at the prompt before the user approves any paid operation. For mindXtrain, this is exactly the right ergonomics for "agent picks GPU flavor → user sees \$/hour → user confirms → job runs → logs streamed back into context."

The **three-phase Research → Plan & Validate → Implement system prompt** (`agent/prompts/system_prompt_v3.yaml`) is the load-bearing discipline document. It forces a research phase (read papers, fetch documentation, enumerate options), a plan-and-validate phase (does this dataset exist, does this tokenizer match, does this flavor have the VRAM), and only then an implement phase. The single biggest reason ml-intern's GPQA demo completes in under ten hours is that the LLM is *not allowed* to call paid tools without first emitting and getting approval on a plan. mindXtrain's training agent must inherit this prompt verbatim, with the validation checklist extended to include base-model-hash-pinning, dataset-CID-pinning, and tokenizer-vocab-hash matching against the dataset's tokenized cache.

What ml-intern does **not** provide and mindXtrain must add: decentralized storage (no Filecoin/Lighthouse abstraction — everything goes to HF Hub); blockchain provenance (no on-chain attestation, no signed run-manifests, no Merkle-rooted training-data commitments); first-class hyperparameter sweeps (no Optuna, no Ray Tune — ablations are LLM-driven re-launches); BFCL and agentic-trajectory evals (the `eval` extra ships `inspect-ai>=0.3.149` but BFCL is not first-class); auto-populated model cards from training runs (the agent can be prompted to write a card but there is no card-templating that reads `TrainingArguments` + eval JSON); a hot-swap deployment plane to a live API (push-to-hub exists, but atomic API-level model swap with canary and rollback does not); a checkpoint registry with diffing and promotion; a typed `TrainingRun` Pydantic record as the canonical unit of provenance. All of these go into the mindXtrain trainer and observability layers, not the agent layer.

## Part 4 — mindXtrain technical specification

mindXtrain is the production-grade training framework that produces `aGLM-BANKON-*` derivatives. It is built on the cypherpunk2048 standard (no proprietary lock-in, Podman over Docker, OpenBSD vmm over VirtualBox, Foundry as canonical Solidity test framework, flat snake_case layout, Python ≥ 3.12, Apache 2.0 license with `(c) 2026 BANKON — all rights reserved` plus upstream MIT NOTICE preservation). The framework has three layers: the **trainer core** (Trainer + TRL + PEFT + Accelerate + Transformers, hand-built), the **operator layer** (ml-intern-pattern-derived agent runtime, reimplemented), and the **provenance layer** (Lighthouse-Filecoin storage, ERC-8004 attestation, Pydantic `TrainingRun` records).

### Repository layout

```
mindxtrain/
├── pyproject.toml                 # name="mindxtrain", py>=3.12, Apache-2.0
├── LICENSE                        # Apache 2.0
├── NOTICE                         # BANKON copyright + upstream MIT (Z.ai for GLM-5.1, Apache for Qwen3.5)
├── LICENSE-MIT-upstream-glm51     # verbatim Z.ai MIT notice
├── README.md
├── CHANGELOG.md
├── Containerfile                  # Podman, not Dockerfile
├── compose.yaml                   # podman-compose
├── mindxtrain/
│   ├── __init__.py
│   ├── config/                    # JSON config with ${ENV} interpolation, ml-intern style
│   │   ├── train_default.json
│   │   ├── eval_default.json
│   │   ├── deploy_default.json
│   │   └── schema.py              # Pydantic schemas
│   ├── data/                      # data pipeline
│   │   ├── curate.py              # source → raw
│   │   ├── dedupe.py              # MinHash + exact-match
│   │   ├── filter.py              # quality, language, toxicity
│   │   ├── tokenize.py            # tokenizer-aware
│   │   ├── pack.py                # sequence packing
│   │   ├── synth.py               # synthetic data via GLM-5.1 / Qwen3.5
│   │   └── verify.py              # hash + manifest
│   ├── models/                    # model registry
│   │   ├── registry.py            # ModelRegistry, register/get
│   │   ├── chat_template.py       # ChatTemplate abstraction
│   │   ├── glm51.py               # backend
│   │   ├── qwen35.py              # backend
│   │   ├── deepseek_v32.py        # backend
│   │   ├── mistral3.py            # backend
│   │   └── phi4_mini.py           # backend
│   ├── train/                     # trainer core
│   │   ├── sft.py                 # full SFT + LoRA + QLoRA
│   │   ├── dpo.py                 # DPO via TRL
│   │   ├── grpo.py                # GRPO via TRL
│   │   ├── rlhf.py                # PPO via TRL
│   │   ├── tool_use.py            # BFCL-style tool-trajectory training
│   │   ├── distributed.py         # accelerate / FSDP / DeepSpeed config builders
│   │   └── callbacks.py           # eval-during-training, checkpoint mgmt
│   ├── eval/                      # eval harness
│   │   ├── lighteval_adapter.py
│   │   ├── inspect_ai_adapter.py
│   │   ├── bfcl.py                # BFCL v3 / v4
│   │   ├── persona_regression.py  # Codephreak voice tests
│   │   ├── agenda_regression.py   # agenda-conditioning tests
│   │   ├── tau_bench.py
│   │   └── card.py                # auto model card from run
│   ├── operator/                  # ml-intern-pattern-derived
│   │   ├── tool_router.py         # ToolRouter + ToolSpec
│   │   ├── agent_loop.py          # bounded ReAct, doom-loop
│   │   ├── context.py             # 170k compaction
│   │   ├── trajectory.py          # JSONL writer
│   │   ├── approval.py            # CLI/web/Slack approval flow
│   │   └── prompts/
│   │       ├── system_v1.yaml     # Research → Plan → Implement
│   │       └── codephreak.yaml    # persona + agenda, prompt-as-data
│   ├── storage/                   # provenance layer
│   │   ├── provider.py            # StorageProvider interface
│   │   ├── local_fs.py            # always-available fallback
│   │   ├── hf_hub.py
│   │   ├── lighthouse.py          # Lighthouse Storage / Filecoin
│   │   └── ipfs.py                # raw IPFS
│   ├── provenance/                # blockchain anchoring
│   │   ├── manifest.py            # Pydantic TrainingRun record
│   │   ├── erc8004.py             # Identity + Reputation registry attestation
│   │   ├── algorand.py            # BANKON ASA 203977300 hooks
│   │   └── x402.py                # HTTP-402 micropayment for paid inference
│   ├── deploy/                    # automindX v2 integration
│   │   ├── registry.py            # model-version registry
│   │   ├── hot_swap.py            # atomic swap with canary
│   │   ├── ab_test.py             # A/B traffic split
│   │   └── api_client.py          # mindx.pythai.net OpenAI-compat client
│   ├── budget/                    # ResourceBudget (psutil-derived from aGLM)
│   │   └── resource.py
│   └── cli/                       # snake_case entries
│       └── main.py                # mindxtrain.cli.main:cli
├── contracts/                     # Foundry, for ERC-8004 hooks
│   ├── foundry.toml
│   ├── src/
│   ├── test/
│   └── script/
├── ops/
│   ├── containerfiles/            # Podman build files per role
│   ├── compose/                   # podman-compose stacks
│   ├── vmm/                       # OpenBSD vmm vm definitions
│   └── gensyn/                    # Gensyn distributed-training configs
├── tests/                         # pytest, persona regression
├── docs/                          # .py↔.md colocated where reasonable
└── scripts/                       # dev helpers
```

### Hardware feasibility — VRAM math worked out per target

For **single H100 80 GB**, GLM-5.1 is impossible at any quantization (754 GB FP8 weights alone). Realistic targets are Qwen3-32B in BF16 (~64 GB weights + ~6 GB KV at 8 K + ~4 GB activations + grad + optimizer for inference, but training requires ZeRO offload), Qwen3-7B in BF16 with full LoRA (14 GB weights + LoRA adapters ~200 MB + Adam state ~28 GB for FP32 master + 14 GB grads — tight, use QLoRA), Phi-4-mini in BF16 with full SFT (~7.6 GB weights + 15.2 GB grads + 30.4 GB Adam states ≈ 53 GB, fits with margin), Gemma 4 31B with QLoRA. Active ~10B-class MoEs like Qwen3.5-35B-A3B fit in inference at INT4 (~17.5 GB weights + 8 GB KV + activations ≈ 30 GB), and QLoRA fine-tuning is feasible at 16K context.

For **8× H100 80 GB cluster** (640 GB aggregate), GLM-5.1-FP8 is borderline — 754 GB weights does not fit even with ZeRO-3 splitting unless KV is compressed via MLA and you accept 16-bit gradient checkpointing with CPU offload (KTransformers-style); the realistic posture is "inference yes via FP8 MLA-compressed, training no." Qwen3-235B-A22B in BF16 fits trivially for inference (~470 GB weights) and is trainable via FSDP+ZeRO-3 with QLoRA (active 22 B → adapter math is reasonable). Qwen3.5-122B-A10B in BF16 (~244 GB) is comfortable for both inference and full SFT. Mistral Large 3 (675 B / 41 B-A) at FP8 is comparable to GLM-5.1 — borderline. The 8×H100 sweet spot for full SFT is in the 32B–122B-active range.

For **single A100 80 GB**, treat as a slightly slower H100 with the same memory ceiling. GLM-5.1 still impossible. Qwen3-32B QLoRA feasible. Same VRAM math as H100.

For **8× A100 80 GB**, identical capacity to 8× H100 (640 GB) but lower throughput; same model ceilings.

For **RTX 4090 24 GB**, GLM-5.1 impossible. Qwen3-7B QLoRA feasible (4-bit weights ~3.5 GB + LoRA ~200 MB + KV at 4 K ~1 GB + grads + Adam ≈ 18 GB). Phi-4-mini full SFT feasible (BF16, ~16 GB total). Qwen3-1.7B and Qwen3-0.6B full SFT comfortable. Edge fine-tunes only.

For **Apple Silicon M3 Ultra 192 GB unified memory** via MLX, GLM-5.1 impossible until `mlx-lm` issue #879 lands and INT4 quants become available (then INT4 + MLA-compressed KV ≈ 383 GB at 128K context — *still* doesn't fit). Qwen3-32B BF16 fits with margin (~64 GB + KV). Qwen3.5-122B-A10B in INT4 (~30 GB weights) fits comfortably for inference; QLoRA fine-tuning works via MLX-LM's PEFT support. Mistral Large 3 at INT4 borderline (~169 GB + KV). M2 Max 96 GB is more constrained: Qwen3-32B INT4 (~16 GB) + LoRA fine-tune is the realistic ceiling.

For **Gensyn distributed training** (the \$5K hackathon track), the framework's posture is to use Gensyn's RL Swarm SDK over the WAN training fabric for the agentic-trajectory RL phase only — pretraining and SFT happen on owned/leased H100/A100 clusters; the Gensyn integration enters at the GRPO stage where many small rollout workers (Qwen3-7B / Phi-4-mini scale) generate trajectories that the central trainer aggregates. This both fits the Gensyn programming model and exercises the BANKON x402 micropayment rail when each rollout worker is paid per accepted trajectory.

### Model size selection logic for mindX cognitive API

The mindX cognitive API at `mindx.pythai.net` should serve a tiered family, not a single flagship. The recommended composition: **edge tier** is `aGLM-BANKON-edge` derived from Phi-4-mini (3.8 B, MIT, BFCL 70.3) for sub-second-latency tool-call routing and on-device deployments; **mid tier** is `aGLM-BANKON-mid` derived from Qwen3-7B or Qwen3.5-Flash for the bulk of agentic traffic where per-token cost matters; **flagship tier** is `aGLM-BANKON-flag` derived from Qwen3.5-122B-A10B for hard agentic flows where 22 B-class active capacity isn't enough but 41 B-class is overkill; **specialist tier** is `aGLM-BANKON-spec` derived from GLM-5.1-FP8 for the long-horizon SWE / 8-hour-autonomous-session use case where GLM-5.1's SWE-Bench Pro 58.4 dominance and 200K-context DSA are decisive. Routing between tiers happens at the operator layer based on task classification (tool-call routing → edge; chat with tools → mid; complex agentic flow → flagship; long-horizon SWE → specialist).

### Data pipeline

The dataset construction pipeline is a Pydantic-typed DAG: `curate` (pull from source — HF Datasets, Common Crawl-derived, codephreak conversation history, mindX session logs) produces `raw/`; `dedupe` runs MinHash near-duplicate detection plus exact-match removal producing `deduped/`; `filter` applies language detection (`fasttext`), quality classifiers (Cosmopedia-style), and toxicity filtering producing `filtered/`; `tokenize` runs the target tokenizer with vocab-hash recorded into the manifest producing `tokenized/`; `pack` does sequence packing to the target context length producing `packed/`; `synth` is the optional synthetic-data generation step that uses GLM-5.1 (specialist tier) or Qwen3.5-122B-A10B (flagship tier) to generate domain-specific tool-use trajectories, persona-conditioned dialogues, and edge-case examples (the ml-intern healthcare demo's "generate 1,100 synthetic edge cases and upsample 50×" pattern is the template); `verify` produces a manifest containing every artifact's BLAKE3 hash, the Lighthouse-Filecoin CID, the source URLs, and the generation parameters. The whole pipeline is a single `mindxtrain.data.run(config)` call that produces a `DatasetManifest` Pydantic record consumed by `train`.

### Evaluation harness

The eval harness has three sub-layers: standard benchmarks via `lighteval` and `inspect-ai` adapters (MMLU-Pro, GPQA-Diamond, AIME, MATH, HumanEval, MBPP, IFEval); agentic benchmarks via custom adapters (BFCL v3/v4 — full Berkeley harness wired in, τ²-Bench, τ³-Bench, AgentBench, GAIA, MCP-Atlas where dataset is public, SWE-Bench Verified — Pro is gated behind Scale AI submission); and the **persona-and-agenda regression suite**, which is unique to mindXtrain and irreplaceable. The persona suite tests "does the model still call itself codephreak", "does the model preserve the Professor Codephreak ML/CS/programming domain", "does the model stay agenda-conditioned when given a multi-step build task", "does the model emit the correct chat-template tokens for the target backend", and "does the model degrade gracefully when the agenda is unfulfillable." Regression detection is automatic: every checkpoint runs the full suite, scores are diffed against the baseline (the most recent green checkpoint), and any score regression beyond a configurable tolerance halts the deployment pipeline. Model card auto-generation reads the `TrainingRun` manifest plus eval JSONs and emits a Hugging Face-compatible `README.md` with full provenance.

### Training pipeline configurations — copy-pasteable

A canonical Qwen3.5-122B-A10B SFT-LoRA configuration in TRL/PEFT, expressible directly in the mindXtrain JSON config:

```json
{
  "run_id": "aGLM-BANKON-flag-sft-001",
  "base_model": {
    "id": "Qwen/Qwen3.5-122B-A10B",
    "revision": "main",
    "license": "apache-2.0",
    "vocab_hash": "blake3:..."
  },
  "tokenizer": {"chat_template": "qwen3"},
  "dataset": {
    "manifest_cid": "lighthouse://bafy...codephreak-sft-v3",
    "format": "jsonl-chatml",
    "max_seq_len": 16384,
    "packing": true
  },
  "trainer": {
    "type": "sft",
    "framework": "trl",
    "lora": {"r": 64, "alpha": 128, "dropout": 0.05,
             "target_modules": ["q_proj","k_proj","v_proj","o_proj",
                                "gate_proj","up_proj","down_proj"]},
    "qlora": {"bits": 4, "compute_dtype": "bfloat16",
              "double_quant": true, "quant_type": "nf4"}
  },
  "optim": {
    "optimizer": "paged_adamw_32bit",
    "lr": 2e-5, "lr_scheduler": "cosine", "warmup_ratio": 0.03,
    "weight_decay": 0.0, "max_grad_norm": 1.0,
    "epochs": 3, "global_batch": 64, "micro_batch": 1,
    "grad_accum": 8, "grad_checkpointing": true
  },
  "distributed": {
    "strategy": "fsdp", "shard": "full", "mixed_precision": "bf16",
    "cpu_offload": false
  },
  "callbacks": {
    "eval_steps": 200, "save_steps": 500,
    "eval_suites": ["bfcl_v4","persona_regression","agenda_regression"],
    "stop_on_regression": true
  },
  "storage": {"provider": "lighthouse",
              "checkpoint_dir": "lighthouse://aGLM-BANKON-flag/sft-001/"},
  "provenance": {"erc8004_attest": true,
                 "x402_settlement": false,
                 "algorand_asa": 203977300}
}
```

For DPO, swap `trainer.type` to `dpo`, point the dataset at a preference manifest with `chosen`/`rejected` pairs, drop the LR to `5e-7`, set `beta: 0.1`, and target the same LoRA modules — TRL's `DPOTrainer` consumes this directly.

For GRPO over agentic trajectories, `trainer.type: "grpo"` with `reward_funcs: ["bfcl_pass","tau_bench_score","persona_consistency"]`, `num_generations: 8`, `temperature: 0.9`, `max_prompt_length: 8192`, `max_completion_length: 8192`. The reward functions are pluggable Python callables registered in `mindxtrain.train.grpo.reward_registry`.

## Part 5 — automindX v2 integration

automindX v2 is the consumer of mindXtrain-produced `aGLM-BANKON-*` checkpoints. The upgrade path from `pythaiml/automindx/aglm.py` to v2 is mechanical given the gap analysis. The single class `LlamaModel(model_name, models_folder)` becomes the package `automindx.models.{Backend}` with backends `HfTransformersBackend`, `LlamaCppBackend`, `OllamaBackend`, `VllmBackend`, `OpenAiCompatBackend`, `ZaiBackend`, `AnthropicBackend`, `GroqBackend` — registered in `automindx.models.registry.ModelRegistry`, dispatched by string key, picking the right `automindx.chat.ChatTemplate` for the backend and falling back to `tokenizer.apply_chat_template`. The synchronous `generate_contextual_output` becomes `generate_contextual_output(context: ConversationContext) -> Generation`, with an `agenerate_contextual_output` async sibling — sync default surface is preserved so existing call sites work unchanged. The JSON-on-disk memory becomes a `MemoryStore` interface with backends `JsonFsBackend` (default, never break), `PgVectorScaleBackend`, `LighthouseFilecoinBackend`. The 4096-character ceiling (`chunk4096.py`) is replaced by tokenizer-aware truncation plus sliding-window summarization for long contexts; default ctx 8K–128K depending on backend. The hard-coded persona becomes `prompts/codephreak.yaml` with the persona, the agenda field as a first-class slot, and the chat-template-token mapping declared.

The model registry, versioning, and hot-swap mechanics: `automindx.deploy.Registry` is a content-addressed registry where every `aGLM-BANKON-*` checkpoint is identified by the BLAKE3 hash of its safetensors plus its `TrainingRun` manifest CID. The registry tracks `current`, `canary`, and `rollback` pointers per tier (edge, mid, flagship, specialist). Hot-swap is atomic: `automindx.deploy.HotSwap.promote(tier, run_id)` flips the `current` pointer after running the persona-regression and agenda-regression suites against the live API harness; on failure, it auto-rollbacks. A/B testing is a traffic split at the `automindx.api.Router` layer: 95% to `current`, 5% to `canary`, with both branches logged as Claude-Code-JSONL trajectories to Lighthouse for offline statistical comparison via `automindx.eval.ab_compare(run_a, run_b, metric)`. The mindx.pythai.net public API exposes an OpenAI-compatible `/v1/chat/completions` plus a mindX-native `/v1/agentic` that takes an agenda field directly and returns a session ID plus streaming events. Every accepted request is provenance-linked: the model checkpoint hash, the run-id, the registry's ERC-8004 attestation, and (for paid tiers) the x402 micropayment receipt are all included in response headers.

## Part 6 — comparative due diligence: the verdict

The brief asked for a comparison of GLM-5.1 against Qwen3 (235B / 72B / 32B / 14B / 7B / 4B / 1.7B / 0.6B), Llama 3.3 70B, DeepSeek V3.1, Mistral Large 2, Gemma 3, and Phi-4. The May-2026 reality has moved beyond several of those: Qwen3.5 (Flash, 27B-dense, 35B-A3B, 122B-A10B) ships and outperforms Qwen3-235B on multiple axes; Llama 4 (Scout/Maverick, April 2025) ships under the *Llama 4 Community License* with the 700M MAU clause and EU AUP; DeepSeek V3.2 / V3.2-Speciale (December 2025) ships under MIT; Mistral Large 3 + Ministral 3 (December 2025) ships under Apache 2.0; Gemma 4 (April 2026) ships under Apache 2.0 — the single biggest license unlock of 2026; Phi-4 / Phi-4-mini ships under MIT.

Apache-2.0 redistribution requires the upstream to be permissive: Apache 2.0, MIT, or BSD-class. That filter cleanly admits **GLM-5.1, Qwen3, Qwen3.5, DeepSeek V3.2, Mistral Large 3 + Ministral 3 + Mistral Small 4, Gemma 4, Phi-4, Phi-4-mini, and Yi-1.5**, and cleanly rejects **Llama 3.3, Llama 4, Cohere Command A** (CC-BY-NC, non-commercial), **Falcon 3** (TII Falcon 2.0 with AUP — most counsel will flag this as restrictive), and **Yi-Lightning** (proprietary API-only). That kills five of the ten brief candidates before benchmarks are scored.

The verdict, on the dimensions of license-compatibility, agentic capability, tool-use quality, fine-tunability, ecosystem support, performance-per-parameter, multilingual balance, and size-ladder coverage, is that **mindXtrain should target Qwen3.5 as the primary base and treat GLM-5.1 as a premium specialist track**. Six reasons.

First, license strength is symmetric. MIT and Apache 2.0 are functionally equivalent for downstream Apache 2.0 redistribution; both pass cleanly. There is no license-based reason to prefer GLM-5.1.

Second, GLM-5.1's wins are real but narrow. SWE-Bench Pro 58.4 is genuinely SOTA. Terminal-Bench 2.0, MCP-Atlas, BrowseComp, CyberGym, AIME 2026 95.3, GPQA-Diamond 86.2 are all top-tier. But for tool-use plus function-calling plus web automation — the brief's stated agentic spec — GLM-5.1 is overkill. The 8-hour-autonomous-session capability is for software-engineering agents specifically.

Third, fine-tunability gap. GLM-5.1 was 27 days old at research time; Axolotl recipes are still maturing, LLaMA-Factory support is partial, Unsloth has GGUF but full LoRA/QLoRA flows are not battle-tested. Qwen3 and Qwen3.5 have day-one PEFT, Axolotl, LLaMA-Factory, Unsloth, and TRL support with hundreds of teams' worth of exercised DPO and GRPO recipes.

Fourth, per-parameter economics. GLM-5.1 at 754B/40B-A requires 4–8×H100 to *serve* and ZeRO-3 territory to fine-tune. Qwen3.5-122B-A10B has roughly comparable intelligence at one-quarter the active footprint and runs on 2×H100 comfortably; Qwen3.5-35B-A3B with 3B active reportedly exceeds Qwen3-235B-A22B. The agentic fine-tune sweet spot for mindXtrain is the 30B–122B-active class.

Fifth, size ladder. mindXtrain needs a *family*, not a single model — flagship for hard flows, mid-tier for production traffic, edge for cost-or-latency-sensitive deployments. Only Qwen has a contiguous Apache-2.0 family from 0.6B → 235B → 235B-Instruct-2507 → 235B-Thinking-2507 → 3.5 medium series. GLM-5.1 is one model. Family completeness alone tips this toward Qwen.

Sixth, BFCL leadership lives in the Qwen lineage. Qwen3-32B hits BFCL v3 75.7%; Qwen3.5-122B-A10B reaches BFCL v4 72.2%; the GLM-4.5 ancestor leads at 76.7%, but GLM-5.1 has not officially submitted to BFCL. For a function-calling-first agentic framework, Qwen has more *demonstrated* surface.

The recommendation, restated in concrete pin-the-version-and-go form: target Qwen3.5-Flash for the edge tier or Phi-4-mini if BFCL-at-tiny-size matters more than open-Apache-only purity (Phi-4-mini is MIT, equivalent for redistribution); target Qwen3-7B for mid-tier; target Qwen3.5-122B-A10B for flagship; maintain GLM-5.1-FP8 as the specialist track for long-horizon SWE flows where 8-hour autonomous sessions and SWE-Bench Pro 58.4 are decisive; keep DeepSeek V3.2 as the reasoning-with-tool-use track backup if Qwen3.5-Thinking-class falls short on internal evals; keep Mistral Large 3 + Ministral 3 as the EU-jurisdictional secondary track; keep Gemma 4 as a watch-list option once vLLM kernel optimization and Axolotl recipes mature (the early-release speed and stability issues should clear within 60–90 days); do not base anything on Llama 3.3, Llama 4, Cohere Command A, Falcon 3, or Yi-Lightning regardless of benchmark performance.

## Conclusion: provenance chain and execution order

The end-to-end provenance chain is: **upstream open-weight base** (Qwen3.5-122B-A10B Apache 2.0 for primary, GLM-5.1-FP8 MIT for specialist) → **mindXtrain training pipeline** (data DAG with Lighthouse-anchored manifests, SFT + LoRA + DPO + GRPO + tool-use trajectory training, full eval harness with persona regression) → **`aGLM-BANKON-*` checkpoint** (Apache 2.0, NOTICE preserves upstream MIT / Apache, BLAKE3-hashed and Lighthouse-Filecoin-CIDed) → **ERC-8004 attestation** (Identity Registry `0x8004A169...` and Reputation Registry `0x8004BAa1...` on EVM via Foundry-tested contract calls, BANKON ASA `203977300` on Algorand for settlement) → **automindX v2 model registry** (content-addressed, hot-swap with canary and rollback, persona-and-agenda regression gates) → **mindx.pythai.net cognitive API** (OpenAI-compatible `/v1/chat/completions` plus mindX-native `/v1/agentic`, x402 micropayment for paid tiers, full provenance in response headers).

Execution order, the boring critical-path version: first, lock the license posture — pull `zai-org/GLM-5.1-FP8` to a self-hosted store, write the Apache-2.0 + MIT-NOTICE compliance bundle for the BANKON derivative, draft the `mindx.pythai.net` Terms of Service that does *not* inherit Z.ai's AUP. Second, stand up the Qwen3.5-122B-A10B fine-tuning rail end-to-end on whatever 8×H100 / 8×A100 capacity is available, validate it with a deliberately-trivial Codephreak-persona SFT to exercise every callback, every storage backend, every regression gate. Third, port `pythaiml/automindx/aglm.py` to `automindx.models.HfTransformersBackend` with the `ChatTemplate` abstraction, preserving the persona-as-agenda discipline and the four-axis decomposition, in a v2 branch that the v1 callers can opt into without breaking. Fourth, reimplement the five ml-intern operator patterns into `mindxtrain.operator.*` once HF posts the LICENSE on `huggingface/ml-intern` (issue #41), or right now if the patterns are clean-room reimplemented from the public API description without copying source. Fifth, wire the storage and provenance layers — Lighthouse-Filecoin first (CIDs anchored in `TrainingRun` manifests), ERC-8004 attestation second (Foundry-tested contract calls from `mindxtrain.provenance.erc8004`), x402 settlement last. Sixth, take the resulting `aGLM-BANKON-flag-001` to the Gensyn distributed-RL track for the GRPO trajectory phase, exercising the WAN training fabric and the x402 per-trajectory micropayment rail simultaneously.

The framing that matters for everything downstream: GLM-5.1 is a remarkable model, the strongest open-weights agentic-engineering base in May 2026, and a serious specialist track for mindXtrain. But it is not, on the rigorous evidence, the right *primary* base for a multi-tier Apache-2.0-redistributable agentic family. The right primary is Qwen3.5. Building the framework with that priority pair locked in — Qwen3.5 primary, GLM-5.1 specialist — is what makes mindXtrain a production-grade training framework rather than a single-model wrapper, and what gives `aGLM-BANKON` the family completeness it needs to serve every cell of the mindX cognitive API at `mindx.pythai.net`.