mindXtrain / docs /NAV.md
Gregory-L's picture
v1.0.4 — close the limitations 1.0.3 listed, and correct the ones it overstated
74da4f7 verified
|
Raw History Blame Contribute Delete
21.4 kB
# mindXtrain Documentation Index
Every doc lives in `docs/`. The only Markdown at the repo root is `README.md` (entry
point) plus `CLAUDE.md` / `AGENTS.md` (agent-tooling entrypoints, required at root).
This is the **detailed table of contents** for the whole `docs/` tree: every doc gets a
one-line hook and a nested list of its sections, each deep-linked to the heading. Skim the
reading paths first, then jump straight to the section you need.
## Reading paths
- **New here?** → [Quickstart](quickstart.md) → [Architecture](architecture.md) → [CLI reference](cli.md).
- **Running the demo / operating a real run?** → [HANDOFF.md](HANDOFF.md) → [CLI](cli.md) → [YAML schema](yaml_schema.md).
- **Contributing code?** → [Development workflow](development.md) → [Architecture](architecture.md) → [Actualization status](actualization_status.md).
- **Driving the UI?** → [Coach UI](coach.md) → [dcoach](dcoach.md) → [Governance](governance.md).
- **Researching the space?** → [Decentralized training deep-dive](decentralized-training-deep-dive-2026.md) → [LLM training-stack landscape](mindxtrain-llm-training-landscape-2026.md) → [Autoresearch strategy](mindXtrain_autoresearch_strategy.md).
---
## Getting started
### [Quickstart](quickstart.md)
Install (with optional-dep groups), init, dry-run autotune, and a 30-second CPU-only tour.
- [Prerequisites](quickstart.md#prerequisites)
- [Install — base](quickstart.md#install--base)
- [Install — with extras](quickstart.md#install--with-extras)
- [30-second tour (CPU-only)](quickstart.md#30-second-tour-cpu-only)
- [What you have on a CPU (base install)](quickstart.md#what-you-have-on-a-cpu-base-install)
- [On the MI300X (operator path)](quickstart.md#on-the-mi300x-operator-path)
- [Useful one-liners](quickstart.md#useful-one-liners)
- [Next steps](quickstart.md#next-steps)
### [HANDOFF.md](HANDOFF.md)
The 11-step operator checklist: local setup → MI300X provision → train/eval/quantize → publish → contracts → deploy.
- [1. Local setup (no GPU; 10 minutes)](HANDOFF.md#1-local-setup-no-gpu-10-minutes)
- [2. Provision the MI300X droplet (sign-up + 30 min)](HANDOFF.md#2-provision-the-mi300x-droplet-sign-up--30-min)
- [3. Install heavyweight deps inside the container](HANDOFF.md#3-install-heavyweight-deps-inside-the-container)
- [4. Run the autotune probe (real, ~60 s)](HANDOFF.md#4-run-the-autotune-probe-real-60-s)
- [5. Train + eval + quantize](HANDOFF.md#5-train--eval--quantize--2-hours-total-for-the-demo-recipe)
- [6. Build the manifest + verify](HANDOFF.md#6-build-the-manifest--verify)
- [7. Publish (HF Hub + Lighthouse + mindX register)](HANDOFF.md#7-publish-hf-hub--lighthouse--mindx-register)
- [8. Deploy contracts (optional)](HANDOFF.md#8-deploy-contracts-optional)
- [9. Serve the model + wire the production URL](HANDOFF.md#9-serve-the-model--wire-the-production-url)
- [10. Publish & demo](HANDOFF.md#10-publish--demo)
- [11. Quality gates (run before every push)](HANDOFF.md#11-quality-gates-run-before-every-push)
- [What's still TODO](HANDOFF.md#whats-still-todo) · [Quick reference](HANDOFF.md#quick-reference)
---
## Architecture & invariants
### [Architecture](architecture.md)
The 5-layer single-package layout, the MI300X invariants, and the end-to-end data flow.
- [The five conceptual layers](architecture.md#the-five-conceptual-layers)
- [Autotune is the spine](architecture.md#autotune-is-the-spine)
- [MI300X-specific invariants (non-negotiable)](architecture.md#mi300x-specific-invariants-non-negotiable)
- [End-to-end data flow](architecture.md#end-to-end-data-flow)
- [Model strategy](architecture.md#model-strategy-per-mindxtrain2md-part-6)
- [Actualization status](architecture.md#actualization-status)
- [What lives outside the Python tree](architecture.md#what-lives-outside-the-python-tree)
### [Development workflow](development.md)
Toolchain, optional-deps, the lazy-import pattern, training lanes, and how to add recipes/backends/methods.
- [Toolchain](development.md#toolchain)
- [Optional dependency groups](development.md#optional-dependency-groups)
- [Lazy-import pattern](development.md#lazy-import-pattern)
- [The standard local cycle](development.md#the-standard-local-cycle)
- [Repository layout](development.md#repository-layout)
- [Reuse boundaries](development.md#reuse-boundaries)
- [Invariants](development.md#invariants)
- [Training lanes (CPU / local-GPU / MI300X)](development.md#training-lanes-cpu--local-gpu--mi300x)
- [Adding a new recipe](development.md#adding-a-new-recipe)
- [Adding a new training backend](development.md#adding-a-new-training-backend)
- [Adding a new model backend (operator)](development.md#adding-a-new-model-backend-operator)
- [Adding a new training method](development.md#adding-a-new-training-method)
- [Adding a new optional-dep group](development.md#adding-a-new-optional-dep-group)
- [Adding a new doc](development.md#adding-a-new-doc)
- [Live training UI](development.md#live-training-ui) — [Routes](development.md#routes) · [Invariants](development.md#invariants) · [Frontend](development.md#frontend) · [Why not Selenium / WebSocket / Streamlit](development.md#why-not-selenium--websocket--streamlit)
- [Common debugging](development.md#common-debugging)
- [What not to commit](development.md#what-not-to-commit)
### [Actualization status](actualization_status.md)
Per-module map of what's real vs. needs `--extra` vs. CPU-active / GPU-pending / stub.
- [v1.0.0 production-readiness audit](actualization_status.md#v100-production-readiness-objective-audit-2026-06-11)
- [Headline numbers](actualization_status.md#headline-numbers)
- [What `uv sync` (no extras) gives you](actualization_status.md#what-uv-sync-no-extras-gives-you)
- [What the optional-dep groups unlock](actualization_status.md#what-the-optional-dep-groups-unlock)
- [Per-subpackage status](actualization_status.md#per-subpackage-status) — [cli](actualization_status.md#mindxtraincli) · [config](actualization_status.md#mindxtrainconfig) · [data](actualization_status.md#mindxtraindata) · [models](actualization_status.md#mindxtrainmodels) · [train](actualization_status.md#mindxtraintrain) · [eval](actualization_status.md#mindxtraineval) · [autotune](actualization_status.md#mindxtrainautotune) · [operator](actualization_status.md#mindxtrainoperator) · [storage](actualization_status.md#mindxtrainstorage) · [provenance](actualization_status.md#mindxtrainprovenance) · [deploy](actualization_status.md#mindxtraindeploy) · [budget](actualization_status.md#mindxtrainbudget)
- [What stays as `NotImplementedError`](actualization_status.md#what-stays-as-notimplementederror)
- [Test coverage](actualization_status.md#test-coverage) · [See also](actualization_status.md#see-also)
### [Autotune deep-dive](autotune.md)
The 60-second AOT probe — the differentiator. Probe taxonomy, the `AutotunePlan`, and how training consumes it.
- [The AOT-only discipline](autotune.md#the-aot-only-discipline)
- [The probe taxonomy](autotune.md#the-probe-taxonomy) — [attention_probe (CK vs Triton)](autotune.md#1-attention_probe--ck-vs-triton-sdpa) · [gemm_probe (hipBLASLt)](autotune.md#2-gemm_probe--hipblaslt-heuristic) · [rccl_probe (collective bw)](autotune.md#3-rccl_probe--collective-bandwidth)
- [The `AutotunePlan` schema](autotune.md#the-autotuneplan-schema)
- [How the training layer consumes the plan](autotune.md#how-the-training-layer-consumes-the-plan)
- [Dry-run / CI path](autotune.md#dry-run--ci-path)
- [Implementation budget](autotune.md#day-2-implementation-budget-target-30-minutes-per-probe)
- [Where the demo wow-moment lives](autotune.md#where-the-demo-wow-moment-lives)
---
## Coach UI & training workflow
### [Coach UI](coach.md)
The interactive `/coach/` operator UI: create-script, live-training diagnostics, verifiable receipt, streaming chat, Modelfile builder.
- [Why it exists](coach.md#why-it-exists) · [Boot](coach.md#boot) · [Layout](coach.md#layout) · [Routes](coach.md#routes)
- [Live training diagnostics](coach.md#live-training-diagnostics)
- [Verifiable receipt card](coach.md#verifiable-receipt-card)
- [Create script + imprint (actor / persona / script)](coach.md#create-script--imprint-actor--persona--script)
- [Create script — personas + skills](coach.md#create-script--personas--skills)
- [Build an Ollama Modelfile (separate window)](coach.md#build-an-ollama-modelfile-separate-window)
- [The core storyboard](coach.md#the-core-storyboard) · [Demo storyboard](coach.md#demo-storyboard)
- [Dependencies](coach.md#dependencies) · [Tests](coach.md#tests) · [Customizing for the demo](coach.md#customizing-for-the-demo)
- [Streaming chat + ollama controls (Try the model)](coach.md#streaming-chat--ollama-controls-try-the-model)
### [dcoach](dcoach.md)
`/coach/dcoach`: prove a CPU-trained model recalls its training (imprint → classroom → boardroom → autotune feedback), clean-room eval tools, and how mindXtrain fits decentralized training.
- [The proof loop](dcoach.md#the-proof-loop)
- [Clean-room eval tools](dcoach.md#clean-room-eval-tools)
- [Prompt tools — test cheap, promote if it wins](dcoach.md#prompt-tools--test-cheap-promote-if-it-wins)
- [How mindXtrain fits decentralized training](dcoach.md#how-mindxtrain-fits-decentralized-training)
- [Why this matters](dcoach.md#why-this-matters)
### [Governance](governance.md)
classroom (graduation) / boardroom (any-N consensus) / dojo (prime-N dispute settlement), model-backed deliberation.
- [The model](governance.md#the-model) · [Flow](governance.md#flow) · [Why prime](governance.md#why-prime)
- [Model-backed deliberation](governance.md#model-backed-deliberation)
- [Coach surface](governance.md#coach-surface) · [Tests](governance.md#tests)
---
## Reference
### [CLI reference](cli.md)
Every `mindxtrain` verb with synopsis, options, and exit codes.
- [Global options](cli.md#global-options)
- [`init` — scaffold a YAML](cli.md#init--scaffold-a-yaml)
- [`bench` — 60-second AOT autotune probe](cli.md#bench--run-the-60-second-aot-autotune-probe)
- [`train` — dispatch a training run](cli.md#train--dispatch-a-training-run)
- [`dataset prep` — run the dataset pipeline](cli.md#dataset-prep--run-the-dataset-pipeline)
- [`eval` — run lm-evaluation-harness](cli.md#eval--run-lm-evaluation-harness)
- [`quantize` — Quark FP8 / MXFP4](cli.md#quantize--quark-fp8--mxfp4)
- [`serve` — print the vLLM-ROCm launch command](cli.md#serve--print-the-vllm-rocm-launch-command)
- [`publish` — push to HF + Lighthouse + register](cli.md#publish--push-to-hf--lighthouse--register)
- [`receipt` — verify a provenance manifest](cli.md#receipt--verify-a-provenance-manifest)
- [Exit-code summary](cli.md#exit-code-summary) · [Where the verbs live](cli.md#where-the-verbs-live)
### [YAML schema](yaml_schema.md)
Every field of the 10-section `XTrainConfig`.
- [`meta`](yaml_schema.md#meta) · [`hardware`](yaml_schema.md#hardware) · [`autotune`](yaml_schema.md#autotune) · [`model`](yaml_schema.md#model) · [`data`](yaml_schema.md#data)
- [`train`](yaml_schema.md#train) — [`train.method` (discriminated union)](yaml_schema.md#trainmethod-discriminated-union)
- [`eval`](yaml_schema.md#eval) · [`quantize`](yaml_schema.md#quantize) · [`serve`](yaml_schema.md#serve) · [`publish`](yaml_schema.md#publish) · [`receipt`](yaml_schema.md#receipt)
- [How the schema is enforced](yaml_schema.md#how-the-schema-is-enforced)
### [Benchmarks](benchmarks.md)
Target metrics + framework comparison.
- [Hero workload](benchmarks.md#hero-workload)
- [H100 cost baseline](benchmarks.md#h100-cost-baseline)
- [Framework comparison](benchmarks.md#framework-comparison-the-readme-differentiator)
- [Capturing the numbers](benchmarks.md#capturing-the-numbers)
- [Regression detection](benchmarks.md#regression-detection)
- [What's not measured (yet)](benchmarks.md#whats-not-measured-yet)
### [CHANGELOG](CHANGELOG.md)
Version history (current: v1.0.4).
- [[Unreleased]](CHANGELOG.md#unreleased)
- [[1.0.4] — 2026-09-16](CHANGELOG.md#104--2026-09-16)
- [[1.0.3] — 2026-09-15](CHANGELOG.md#103--2026-09-15)
- [[1.0.2] — 2026-09-15](CHANGELOG.md#102--2026-09-15)
- [[1.0.1] — 2026-09-15](CHANGELOG.md#101--2026-09-15)
- [[1.0.0] — 2026-06-11](CHANGELOG.md#100--2026-06-11)
- [[0.1.0] — 2026-05-06](CHANGELOG.md#010--2026-05-06)
### [LICENSE-NOTICE](LICENSE-NOTICE.md)
Apache-2.0 + MIT-compatibility statement.
---
## Decentralized-training landscape (2026)
### [Decentralized training deep-dive](decentralized-training-deep-dive-2026.md)
Prime Intellect / Nous Psyche / Gensyn / Templar / Pluralis, the DiLoCo/SparseLoCo algorithms, verification (TOPLOC / Verde / Gauntlet), and where mindXtrain fits.
- [TL;DR](decentralized-training-deep-dive-2026.md#tldr)
- [1. The Algorithms](decentralized-training-deep-dive-2026.md#1-the-algorithms-how-training-escaped-the-datacenter) — [DiLoCo family](decentralized-training-deep-dive-2026.md#diloco-family-data-parallel-low-communication) · [Momentum-decoupling](decentralized-training-deep-dive-2026.md#momentum-decoupling-nous-lineage) · [Model-parallel over WAN](decentralized-training-deep-dive-2026.md#model-parallel-over-wan-pluralis) · [Other lineages](decentralized-training-deep-dive-2026.md#other-lineages)
- [2. The Networks](decentralized-training-deep-dive-2026.md#2-the-networks-whos-actually-training-what) — [Prime Intellect](decentralized-training-deep-dive-2026.md#prime-intellect--the-open-superintelligence-stack) · [Nous / Psyche](decentralized-training-deep-dive-2026.md#nous-research--psyche--distro-on-solana) · [Gensyn](decentralized-training-deep-dive-2026.md#gensyn--verification-first-ml-compute-protocol) · [Templar](decentralized-training-deep-dive-2026.md#templar--bittensor-sn3--the-permissionless-proof) · [Pluralis](decentralized-training-deep-dive-2026.md#pluralis-research--protocol-learning-model-parallel) · [Others](decentralized-training-deep-dive-2026.md#others-worth-tracking)
- [3. Verification: The Trust Layer](decentralized-training-deep-dive-2026.md#3-verification-the-trust-layer)
- [4. Why RL Is the Decentralization Sweet Spot](decentralized-training-deep-dive-2026.md#4-why-rl-is-the-decentralization-sweet-spot)
- [5. Economics & Crypto Integration](decentralized-training-deep-dive-2026.md#5-economics--crypto-integration)
- [6. Hardware & Network Realities](decentralized-training-deep-dive-2026.md#6-hardware--network-realities)
- [7. Critical Assessment & Open Problems](decentralized-training-deep-dive-2026.md#7-critical-assessment--open-problems)
- [8. Practical Integration for mindXtrain / PYTHAI](decentralized-training-deep-dive-2026.md#8-practical-integration-for-mindxtrain--pythai)
- [Network Comparison Table](decentralized-training-deep-dive-2026.md#network-comparison-table) · [Key Papers Index](decentralized-training-deep-dive-2026.md#key-papers-index) · [Caveats](decentralized-training-deep-dive-2026.md#caveats)
### [LLM training-stack landscape](mindxtrain-llm-training-landscape-2026.md)
The open-source training/eval/quantize stack survey anchored on mindXtrain.
- [TL;DR](mindxtrain-llm-training-landscape-2026.md#tldr)
- [1. The mindXtrain Point of Departure](mindxtrain-llm-training-landscape-2026.md#1-the-mindxtrain-point-of-departure)
- [2. Open-Source Training Frameworks](mindxtrain-llm-training-landscape-2026.md#2-open-source-training-frameworks)
- [3. Automated / Autonomous Training Pipelines](mindxtrain-llm-training-landscape-2026.md#3-automated--autonomous-training-pipelines)
- [4. Training-as-a-Service](mindxtrain-llm-training-landscape-2026.md#4-training-as-a-service-centralized-and-decentralized) — [4a. Verification software](mindxtrain-llm-training-landscape-2026.md#4a-verification-software-verifiable-training--inference--links-and-source)
- [5. Data Curation](mindxtrain-llm-training-landscape-2026.md#5-data-curation)
- [6. LoRA/Adapter Ecosystem, Weights, Formats, Quantization](mindxtrain-llm-training-landscape-2026.md#6-loraadapter-ecosystem-weights-formats-and-quantization)
- [7. AOT Compilation, Reproducibility, Licensing](mindxtrain-llm-training-landscape-2026.md#7-aot-compilation-reproducibility-licensing)
- [The End-to-End Pipeline](mindxtrain-llm-training-landscape-2026.md#the-end-to-end-pipeline) · [Recommendations](mindxtrain-llm-training-landscape-2026.md#recommendations) · [Caveats](mindxtrain-llm-training-landscape-2026.md#caveats)
### [Autoresearch strategy](mindXtrain_autoresearch_strategy.md)
The research-harness-as-a-wrapper design: attempt contract, git guard, search loop, on-chain champion provenance, parallel researchers.
- [0. Thesis](mindXtrain_autoresearch_strategy.md#0-thesis)
- [1. The four disciplines being imported](mindXtrain_autoresearch_strategy.md#1-the-four-disciplines-being-imported)
- [2. Architecture: the research harness as a wrapper](mindXtrain_autoresearch_strategy.md#2-architecture-the-research-harness-as-a-wrapper) — [attempt contract](mindXtrain_autoresearch_strategy.md#21-the-attempt-contract-discipline-2) · [git guard](mindXtrain_autoresearch_strategy.md#22-the-git-guard-disciplines-1--3) · [attempt runner](mindXtrain_autoresearch_strategy.md#23-the-attempt-runner-discipline-1) · [search loop](mindXtrain_autoresearch_strategy.md#24-the-search-loop-keepreset)
- [3. Provenance: every champion is verifiable](mindXtrain_autoresearch_strategy.md#3-provenance-every-champion-is-verifiable-and-anchored) — [off-chain](mindXtrain_autoresearch_strategy.md#31-off-chain-exists-today-just-call-it) · [on-chain](mindXtrain_autoresearch_strategy.md#32-on-chain-existing-contract-handles-per-run-add-a-sibling-for-the-search-tree) · [settlement](mindXtrain_autoresearch_strategy.md#33-settlement-existing-x402-optional-gate)
- [4. Foundry + mainnet](mindXtrain_autoresearch_strategy.md#4-foundry--mainnet-per-standing-convention) — [test](mindXtrain_autoresearch_strategy.md#41-test-contractstestmindxautoresearchregistrytsol) · [deploy](mindXtrain_autoresearch_strategy.md#42-deploy-extend-the-existing-deployssol)
- [5. Operator + CLI + parallel researchers](mindXtrain_autoresearch_strategy.md#5-operator--cli--parallel-researchers) — [CLI verb](mindXtrain_autoresearch_strategy.md#51-cli-verb-typer-matches-mei_scoreimprint-house-style) · [operator route](mindXtrain_autoresearch_strategy.md#52-operator-route--durable-ledger-discipline-4) · [parallel researchers](mindXtrain_autoresearch_strategy.md#53-parallel-researchers-k8s)
- [6. Phased rollout](mindXtrain_autoresearch_strategy.md#6-phased-rollout) · [7. What this buys you](mindXtrain_autoresearch_strategy.md#7-what-this-buys-you)
### [Vercel AI SDK 6 deep-dive](<Vercel AI SDK 6_ A Framework-Agnostic Deep Dive (June 2026).md>)
The streaming/agent toolkit the Coach chat patterns after (clean-room, vanilla JS).
- [TL;DR](<Vercel AI SDK 6_ A Framework-Agnostic Deep Dive (June 2026).md#tldr>) · [Key Findings](<Vercel AI SDK 6_ A Framework-Agnostic Deep Dive (June 2026).md#key-findings>)
- [Architecture & Philosophy](<Vercel AI SDK 6_ A Framework-Agnostic Deep Dive (June 2026).md#1-architecture--philosophy>)
- [Detailed Capability List](<Vercel AI SDK 6_ A Framework-Agnostic Deep Dive (June 2026).md#2-detailed-capability-list>)
- [Clean-Room Setup](<Vercel AI SDK 6_ A Framework-Agnostic Deep Dive (June 2026).md#3-clean-room-setup-framework-agnostic>)
- [Code Examples (v6 API)](<Vercel AI SDK 6_ A Framework-Agnostic Deep Dive (June 2026).md#4-code-examples-current-v6-api>)
- [Ecosystem & Context](<Vercel AI SDK 6_ A Framework-Agnostic Deep Dive (June 2026).md#5-ecosystem--context>) · [Recommendations](<Vercel AI SDK 6_ A Framework-Agnostic Deep Dive (June 2026).md#recommendations>) · [Caveats](<Vercel AI SDK 6_ A Framework-Agnostic Deep Dive (June 2026).md#caveats>)
---
## Source briefs (`blueprints/`)
The frozen design briefs the project was built against — historical specification; for
current state read the docs above. **Do not edit.**
- [`blueprints/mindXtrain.md`](blueprints/mindXtrain.md) — operating brief; three-track pitch, module design, day-by-day execution.
- [`blueprints/mindXtrain2.md`](blueprints/mindXtrain2.md) — technical master reference; GLM-5.1 depth, aGLM lineage, canonical Part 4 repository layout, training configs.
- [`blueprints/mindXtrain_ Production Blueprint for the AMD and lablab.ai Hackathon.md`](<blueprints/mindXtrain_ Production Blueprint for the AMD and lablab.ai Hackathon.md>) — pinned reference stack, Qwen3 targeting, repo skeleton, critical code snippets.
- PDFs (read-only design source): `blueprints/mindXtrain Framework_ GLM-5.1, aGLM Lineage, and Qwen3.5 Primary Base Strategy.pdf`, `blueprints/mindXtrain_ Production Blueprint ….pdf`, `blueprints/Winning the AMD x lablab.ai Developer Hackathon ….pdf`.
## Build-in-public posts (`posts/`)
Dated launch-narrative posts and their rendered HTML — historical, not product docs.
- [`posts/README.md`](posts/README.md) — index + cross-platform posting notes.
- [`posts/day1_scaffold.md`](posts/day1_scaffold.md) · [`posts/day2_autotune.md`](posts/day2_autotune.md) · [`posts/day5_demo.md`](posts/day5_demo.md)
- Rendered: `posts/rendered/{about,day1,day2,day5}.html`.
## On-chain
- [`contracts/README.md`](../contracts/README.md) — Foundry workspace for the immutable run-receipt registry + x402 receiver.
</content>
</invoke>