| # Changelog |
|
|
| All notable changes to **mindxtrain** are documented in this file. The format |
| follows [Keep a Changelog](https://keepachangelog.com/en/1.1.0/), and this |
| project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html). |
|
|
| ## [Unreleased] |
|
|
| ### Added |
|
|
| - **bankml 0.3.6 penalties.** `serve --to bankml` takes `repeat_penalty`, `repeat_last_n`, |
| `presence_penalty` and `frequency_penalty` in the Modelfile when `bankml version` is 0.3.6 or |
| later (bankml's O2), and refuses them with the reason before; `--bankml-system`, |
| `--bankml-param KEY=VALUE` and `--bankml-stop` layer a persona and its sampling over the pin. |
| The backend sends sampling fields from `options=` / `MINDXTRAIN_BANKML_OPTIONS`, only when set. |
| - **Byte-identical conversion.** The merged directory is converted through a link named after the |
| tag, so llama.cpp's `general.name` is the tag's and gen39 converts to `6b64c748…`, the GGUF mindX |
| pins (through `merged/` it was "Merged" and a different sha256). |
| - **`MINDXTRAIN_CHAT_MODEL`, `MINDXTRAIN_JUDGE_MODEL`, `MINDXTRAIN_CHAT_OPTIONS`.** The panel and |
| the judges named `llama3.2` when given no model; a bankml node serves none. Read at call time. |
| `classroom(use_judge=True)` without a model now uses the default judge instead of skipping it. |
| - **`docs/install.md`**: deploying bankml 0.3.6 and mindXtrain's bankml lane to the mindX VPS. |
|
|
| - **bankml, the verified CPU engine** ([github.com/cryptoAGI/bankml](https://github.com/cryptoAGI/bankml), |
| [`docs/bankml.md`](bankml.md)), reached over HTTP and its CLI only — nothing vendored. |
| - `operator/backends/bankml.py`: `@register_backend("bankml")`, `MINDXTRAIN_BANKML_BASE_URL` |
| (default `http://127.0.0.1:18093/v1`). Keeps bankml's receipt (`ChatResponse.receipt`, new |
| optional field; `last_receipt` for streams, parsed from the final SSE receipt event). HTTP 400 |
| → `BankmlRefusal(reason)`, never retried with altered parameters. |
| - Operator wiring: `backend_kwargs(name)` replaces the duplicated per-backend branches in |
| `operator/app.py` and `coach/api.py`; health probes `GET /bankml` and `/v1/models`; |
| auto-detect tries bankml after ollama and vllm, so existing hosts resolve as before; the |
| governance panel's env chain gains `MINDXTRAIN_BANKML_BASE_URL` last, only when set. |
| - `deploy/bankml_push.py` and `mindxtrain serve --to bankml [--bankml-bin] [--bankml-convert]`: |
| merge → Modelfile through `bankml_sanitize` (refuses `ADAPTER`, a foreign `TEMPLATE`, |
| penalties, mirostat, `typical_p`, resource options, rather than dropping them) → |
| `bankml create`; records the model sha256; refuses quantized configs and non-Llama |
| architectures; detects a bankml without `create`/`convert` (pre-0.3.5) as `bankml_too_old`. |
| - `eval/imprint_bankml.py` and `mindxtrain imprint-bankml`: seeded, unpenalised greedy probes with |
| a receipt per utterance, scored by `score_imprint`, tagged `<scorer>/bankml-greedy` and marked |
| **not comparable** with the canonical 1.3-penalty gate. |
| - MEI `InferenceEngineIdent.name` accepts `"bankml"`; the Gradio Serve room offers `bankml`. |
| |
| - **`serve --to vllm|sglang` are real deploy targets** ([`docs/serve.md`](serve.md)), no longer |
| print-only. `deploy/openai_server_push.py`: `launch_openai_server` serves a LoRA natively over |
| the base (vLLM `--enable-lora --lora-modules`, SGLang `--enable-lora --lora-paths`) or merged |
| (`--merge`, `merge_lora_adapter`), launches detached (own session; `server.log`, `server.pid`, |
| `launch.json` under `out/runs/<run>/serve/<to>/`), waits for `/v1/models` (+ vLLM `/health`), |
| and optionally swaps mindX's fallback (`provider="vllm"|"sglang"`); `stop_openai_server` SIGTERMs |
| the process group and verifies it is gone. CPU hosts get bfloat16, `VLLM_CPU_KVCACHE_SPACE`, |
| SGLang `--device cpu`; quantized checkpoints and `tensor_parallel > 1` are refused without a |
| GPU; a missing server is reported with its install hint. Every outcome is a |
| `ServerLaunchResult` / `ServerStopResult`. New CLI options: `--dry-run`, `--stop`, `--merge`, |
| `--host`, `--port`, `--dtype`, `--server-bin`, `--server-arg`, `--ready-timeout`, |
| `--cpu-kvcache-gib`. The Gradio Serve room offers `sglang`. vLLM / SGLang are reached as a |
| subprocess and over HTTP only. |
|
|
| ### Changed |
|
|
| - `ui/console.py` omits penalty / mirostat options left at the engine's own default, so bankml |
| accepts default console requests. For Ollama a Modelfile's `repeat_penalty` now applies where |
| the console used to override it with 1.1. |
| - `hf.extension.publish_generation(repeat_penalty=…)`: `None` publishes a Modelfile without the |
| penalty line (default unchanged, 1.3); the Modelfile text is `published_modelfile()`. |
| - `cli imprint`: the script-probe extraction is `_script_probes`, shared with `imprint-bankml`. |
|
|
| ## [1.0.4] — 2026-09-16 |
|
|
| ### Added |
|
|
| - **The Gradio surface is tested.** `mindxtrain ui` had no coverage at all — gradio is an |
| optional extra, so a base install cannot import the module, and every release note listed the |
| UI as untested. `tests/test_ui_app_builds.py` constructs the whole Blocks tree (which |
| exercises every component definition in `ui/app.py`), checks the rooms the docs promise, |
| asserts the surface version matches the package, and confirms the PEP 562 lazy export |
| resolves. It skips cleanly where the extra is absent. The UI was also launched for real |
| under gradio 6.27.0 and served its page and config. |
|
|
| ### Fixed |
|
|
| - **The Hub tests asserted the missing-dependency path by trusting the environment.** They |
| passed on a bare install and inverted silently the moment the `ui` extra arrived, because |
| gradio depends on huggingface-hub — three tests failed the first time they met an |
| environment that had it. Absence is now *forced* (`sys.modules[name] = None` makes the |
| import raise), so the contract holds whatever is installed, and it is verified both ways. |
| Introduced in 1.0.2, where the note "tested for real rather than mocked" was true only of a |
| bare install. |
|
|
| ### Changed |
|
|
| - **`httpx2` added to the dev group.** starlette 1.6 deprecates driving its `TestClient` with |
| `httpx`; 24 test modules use `TestClient`, so the dependency moves rather than the tests. |
| The deprecation is gone; the two warnings that remain are upstream typer → click 9.0. |
|
|
| ### Notes — a correction to v1.0.3, and what is now verified |
|
|
| **v1.0.3 overstated its own limitations.** It listed "every write endpoint" and "SSE under a |
| real producer" as untested. That was true of the dependency-refresh *probes* run for that |
| release; it was **not** true of the project. The suite already covers the coach's write |
| endpoints — bench, compile, cost, eval/prompt, modelfile build and create, ollama start and |
| stop, boardroom, dojo, datasets, mei promote, classroom, autotune feedback, dcoach, chat |
| stream — and SSE under a real producer, in `tests/test_runs_sse.py` and a dozen |
| `test_coach_*_api.py` modules, including buffer replay on late subscribe, log-kind filtering, |
| ingest and cancel. All of it passed on the refreshed lock. A new test module was written for |
| those endpoints before that was checked; every endpoint turned out to be covered already and |
| the in-stream error contract already asserted, so it was deleted rather than added as |
| duplicate coverage. |
|
|
| **The training path is now verified under the refreshed lock** (torch 2.11.0, transformers |
| 5.8.0, trl 1.3.0, peft 0.19.1, alongside starlette 1.6.0 and the pydantic downgrade). The |
| dream-corpus loop was run end to end on CPU — corpus read, tokenized, SFT executed |
| (loss 3.938, mean_token_accuracy 0.370, 512 tokens), and a real 22 MB LoRA checkpoint written |
| with `adapter_model.safetensors`, `adapter_config.json`, tokenizer and chat template. With the |
| ml extra installed the suite is **767 passed** (the two tokenizer tests stop skipping). |
|
|
| **Still not tested, and not claimed:** |
|
|
| - **MI300X / ROCm, multi-GPU, and vLLM serving.** No such hardware here; the CPU lane is what |
| was exercised. |
| - **A full-length run.** The shipped 32-sample smoke recipe projects ~70 minutes on this |
| shared box (8.7 min/step at 50% CPU), so the loop was closed with a reduced variant — 4 |
| samples at seq 128, the same code path, fewer tokens. Multi-hour and multi-day behaviour is |
| unchanged and unverified here. |
| - **The `imprint` → `serve --to ollama` tail**, which needs a running daemon. |
|
|
| ## [1.0.3] — 2026-09-15 |
|
|
| ### Changed |
|
|
| - **`uv.lock` refreshed against the current `pyproject.toml`.** The lock had drifted, so every |
| `uv run` on a node re-resolved before doing anything — about 2.5 minutes per invocation, |
| paid by each bridge probe. 296 → 306 packages; 23 changed. The notable moves are |
| **starlette 0.52.1 → 1.6.0** (a major version, under FastAPI 0.136.1, which accepts it), |
| **huggingface-hub 1.14.0 → 1.31.0**, and **gradio 6.27.0** entering the lock for the first |
| time because `all` now includes the `ui` extra. |
| - Three packages move **backwards** — **pydantic 2.13.3 → 2.12.5**, pydantic-core 2.46.3 → |
| 2.41.5, inspect-ai 0.3.219 → 0.3.145. Nothing in the project requires the older versions; |
| only `pydantic>=2.9` is declared, so these are the resolver's choice of a consistent set, |
| not a constraint. They are called out because a downgrade arriving inside a routine refresh |
| is worth seeing rather than discovering later. |
|
|
| ### Notes — what was tested, and what was not |
|
|
| The refreshed lock was installed into a **separate environment**, leaving the working one |
| intact, and the same probes were run against both stacks so the comparison is like-for-like. |
|
|
| **Tested, identical on both stacks:** the full suite (**765 passed, 2 skipped**); the coach |
| booted under real uvicorn exactly as production runs it (`mindxtrain.operator.app:app`), and |
| **37 read-only probes returned identical status codes and identical response sizes** — every |
| page, the JSON API, the static mount, the OpenAPI schema (64 paths), the diagnostics surface, |
| the path-traversal guard still answering 400, and **SSE** (`text/event-stream`, bytes flowing, |
| and the 404 path returning without hanging). **Zero server errors** in either run. |
|
|
| **Not tested — do not read this release as clearing them:** |
|
|
| - **Every write endpoint.** Nothing that launches a run, builds or creates a Modelfile, |
| starts or stops Ollama, convenes the boardroom, settles the dojo, promotes an MEI score, or |
| pushes to GitHub / a droplet was exercised; they need a model, a daemon, or they do real |
| work. `/v1/chat/completions` and `/v1/agentic` are likewise untested. |
| - **SSE under a real producer.** The stream was checked for content type and first bytes on an |
| empty run, not for a long live stream with a training job writing into it. |
| - **The training path.** The `ml` extra (torch, transformers) is not installed on the machine |
| this was verified on, so `train` / `imprint` / `serve` were not run against the new lock — |
| and huggingface-hub, which moved two minor versions, sits under transformers' download path. |
| - **The Gradio UI.** gradio 6.27.0 enters the lock here for the first time; `mindxtrain ui` |
| was not started. |
|
|
| **One new warning, test-only.** starlette 1.6 deprecates using `httpx` with its `TestClient` |
| and asks for `httpx2`; the suite emits it three times and passes regardless. Nothing in the |
| serving path is affected — production does not import `TestClient` — but the test dependency |
| will need moving before a starlette release removes the shim. |
|
|
| **Operational caveat.** `uv run` syncs the project environment, so moving a node's checkout to |
| this commit means the next invocation rewrites that node's venv — a starlette major bump, a |
| huggingface-hub bump and a pydantic downgrade — **underneath whatever is running there**. On a |
| node with a training run in flight, take it in a maintenance window, not by surprise. |
|
|
| ## [1.0.2] — 2026-09-15 |
|
|
| ### Added |
|
|
| - **mindXtrain can use Hugging Face from the command line.** The `mindxtrain/hf/` extension |
| existed and was complete — `account`, `pull_base`, `warm`, `publish_generation`, |
| `push_dataset`, `lineage`, `push_space` — but nothing outside a Python import could reach |
| it: there was no CLI verb, so on a node the Hub was effectively unavailable. New |
| `mindxtrain hf` group: `whoami · pull · warm · publish · lineage · dataset · space`. |
| Each prints the extension's own result dict and **exits nonzero when it reports |
| `ok: false`**, so the ascent loop can warm a base, check `$?`, and refuse to start a |
| three-hour run that would otherwise discover the missing base at the end of it. |
| - **Tests for the Hub extension**, which had none: token precedence across the three |
| accepted environment variables, the missing-dependency path (genuinely missing here, not |
| mocked), the `RepoFolder` trap that makes a scan report an empty repo, the guards that |
| must not need a network, and the CLI wiring including the nonzero exit. |
|
|
| ### Changed |
|
|
| - **`huggingface-hub` moved out of the `chain` extra into its own `hf` extra.** Reaching the |
| Hub used to mean installing `web3` and the Algorand SDK, because the Hub was filed under |
| "chain". It is not a chain. `chain` now pulls `mindxtrain[hf]` so existing installs keep |
| working, `all` includes it, and every "not installed" message names `--extra hf` — the |
| extra that actually installs it — instead of sending the reader to the blockchain one. |
| - The `hf` module is ruff clean under the house config. |
|
|
| ### Fixed |
|
|
| - **`push_space` captured the upload's commit receipt and discarded it**, while its two |
| sibling publishers both return it. A Space push now reports its commit like everything |
| else, so what was pushed is identifiable afterwards. |
| |
| ## [1.0.1] — 2026-09-15 |
| |
| |
| ### Added |
| |
| - **autoresearch — the keep/reset search, shipped whole.** The `research` command had |
| been in the CLI since the Hugging Face fork, but `mindxtrain/research/` was never |
| committed, so `mindxtrain research` raised `ModuleNotFoundError` on every checkout but |
| the author's. The package now ships: the declarative `AttemptContract`, the git fence |
| (commit-before-measure · editable allowlist · dedup-by-commit), the bounded attempt |
| runner with process-group teardown, the durable append-only ledger, the keep/reset |
| search loop, the Base anchor + `mindx_autoresearch_registry.sol`, the coach's read-only |
| observer page (`/coach/research`), the k8s researcher job, and the `attempt_diff` hash |
| in the provenance manifest — so the *edit* that earned a promotion is hashed into the |
| receipt, not just the resulting weights. |
| - **The Console and Science rooms' own modules.** `ui/console.py` (Ollama chat, token |
| accounting, error explanations that say what to run) and `ui/reasoning.py` (the claims |
| / Socratic-questions / contradictions pass, aGLM when importable and an in-house |
| fallback otherwise), both previously uncommitted, both now covered by tests that need |
| no daemon, GPU or optional dependency. |
|
|
| - **dcoach page + prompt-tools + decentralized panel.** New `/coach/dcoach` workflow |
| page: one-click **Imprint & Prove** (persona/skills + advanced toggles) streams the |
| proof loop live (`POST /coach/api/dcoach/run`, SSE) and shows the classroom before/after |
| recall, boardroom verdict, and autotune-feedback next-params. A read-only |
| **decentralized-network panel** (`GET /coach/api/decentralized`) maps Prime Intellect / |
| Templar SN3 / Nous Psyche / Gensyn / Pluralis to how mindXtrain fits (AOT-only ⇒ Verde- |
| compatible; BLAKE3 receipt; x402; AgenticPlace). New `/coach/prompts` **prompt-tools** |
| page: craft a system prompt + few-shot demos → test against a base model (no training) → |
| evaluate (`POST /coach/api/eval/prompt`) → **make permanent** via Modelfile. Docs: |
| [dcoach.md](dcoach.md). |
| - **dcoach proof loop + clean-room eval tools.** Prove a CPU-trained model recalls |
| its training: `governance.proof_loop.run_proof_loop` chains compose-script → imprint- |
| train → probe before/after → **classroom** (`governance.classroom.evaluate_classroom`) |
| → **boardroom** decides → **autotune feedback** (`autotune.feedback`, nudges the next |
| run's params). New clean-room LlamaIndex-style evaluators (`eval.llama_evals`: |
| Semantic-Similarity / Correctness / Pairwise / Guideline — reuse existing embeddings + |
| the governance chat-judge). Endpoints `POST /coach/api/classroom/evaluate` and |
| `POST /coach/api/autotune/feedback`. |
|
|
| - **Streaming chat + ollama controls** (Coach "Try the model" card). Responses now |
| **stream token-by-token** (the AI-SDK text-stream pattern over SSE) via |
| `POST /coach/api/chat/stream`, with a **model picker** (local models first, no more |
| silently-broken `:cloud` default) and **start/stop/status** for the local ollama |
| server (`/coach/api/ollama/{status,start,stop}`). Fixes the chat that returned no |
| visible response. Reference: `docs/Vercel AI SDK 6_ … .md`. |
|
|
| - **Ollama Modelfile builder** (`mindxtrain.deploy.modelfile`) — render a valid |
| `Modelfile` from a typed `ModelfileSpec` (every instruction: FROM/SYSTEM/TEMPLATE/ |
| ADAPTER/LICENSE/MESSAGE/REQUIRES + the full PARAMETER catalogue). A standalone Coach |
| window at `/coach/modelfile` exposes a toggle + input for every instruction and |
| parameter; `POST /coach/api/modelfile/{build,create}` render it and run `ollama create`. |
| - **Default personas + skills** (`mindxtrain.data.personas`) — built-in personas |
| (codephreak / assistant / mentor) and toggleable **skills** (software engineer, platform |
| architect, bash, solidity) that mix in-domain exchanges into a script. The Create-script |
| card gains a persona picker + skill toggles; `derive_training_params` auto-tunes CPU |
| imprint epochs/grad_accum from the dataset size. |
| |
| - **Governance layer** (`mindxtrain.governance`) — classroom / boardroom / dojo, |
| a clean-room reimplementation of openmindx/openmind's Boardroom-consensus + |
| Dojo-evaluation behaviour. The **classroom** graduates an actor on its imprint; |
| the **boardroom** (any N role-based members) convenes on the promotion motion → |
| approved / rejected / disputed; the **dojo** settles a dispute with a panel of a |
| **prime number** of judges (odd prime ≥ 3 → no tie, always resolves). See |
| [`docs/governance.md`](governance.md). |
| - **Model-backed boardroom/dojo** (`governance.panel`) — members + judges |
| deliberate with real models over any OpenAI-compatible backend |
| (`deliberate`, `model_ballot`, `model_judge_ballot`). Coach **Boardroom** card + |
| `GET /coach/api/boardroom/presets`, `POST /coach/api/boardroom/convene`, |
| `POST /coach/api/dojo/settle` (model deliberation runs off the event loop). |
|
|
| ### Fixed |
|
|
| - **The default editor corrupted any editable that was not a flat TOML.** Its docstring |
| promised a no-op; the code stringified a `[table]` into `trainer = "{'lr': 0.001}"`, |
| then committed and measured that. It now checks flatness and declines, and |
| `_write_flat_toml` refuses non-scalars instead of repr-ing them. Strings escape through |
| `json.dumps`, so a value holding a quote or a backslash can no longer emit broken TOML. |
| - **A failed attempt stranded HEAD on a loser.** When an attempt raised after its edit was |
| committed, the loop propagated the error and left the branch on a commit it had never |
| scored — so the next run's baseline read the loser. The champion is restored on the way |
| out. The restore is `reset --hard`, so it is taken only while the fence still passes: if |
| the tree is dirty outside the allowlist that work belongs to someone else, and it is left |
| exactly where it is. |
| - **`mindxtrain.ui` demanded gradio for stdlib-only code.** The package eagerly imported |
| `.app`, so the optional `ui` extra was needed to reach `console`, `reasoning` or |
| `parse_log` — none of which import anything beyond the standard library. The |
| Gradio-backed exports (`build`, `main`, `VERSION`) resolve lazily via PEP 562; a box that |
| cannot spare the dependency can still parse a training log and talk to the daemon. |
| - **The contradiction check was asymmetric**, though documented as a truth-table check: |
| it only fired when the negative claim happened to be written first. |
|
|
| - **Imprint recall now measured under the trained conditioning.** `probe_recall` accepts an |
| optional `system` prompt and the proof loop passes the persona's system prompt to both the |
| before and after probes — matching the system turn the script rows train under. Without it |
| the adapter was probed out of its learned distribution, understating (even inverting) the |
| imprint; the CPU proof loop now reports a strong positive Δ (e.g. 0.066→0.247) and an |
| APPROVED verdict. |
|
|
| ### Changed |
|
|
| - New and touched modules are ruff clean under the house config (unused imports, `typing` |
| modernisation). Pre-existing debt elsewhere is left alone rather than folded into this |
| release. |
|
|
| - **Removed hackathon framing from the public surfaces.** README, `pyproject.toml` |
| description, the package docstring, `CLAUDE.md`, and the active docs (autotune / |
| architecture / development / coach / yaml_schema / HANDOFF / NAV) no longer frame |
| mindXtrain as a hackathon submission; deleted `docs/HACKATHON.md` and |
| `docs/hackathon_submission.md`. The frozen `docs/blueprints/` and dated dev-blog |
| posts are preserved unchanged as the historical record. README now leads with the |
| dcoach proof loop. |
|
|
| ## [1.0.0] — 2026-06-11 |
|
|
| First production release. **CPU training is active end-to-end**; the GPU |
| (MI300X / consumer) path is code-complete and validated on CPU dry-run + unit |
| tests, pending real ROCm hardware to execute. Honest per-module status is in |
| [`docs/actualization_status.md`](actualization_status.md). |
|
|
| ### Added (1.0.0) |
|
|
| - **Device-aware local-GPU lane** (`trl_local`, |
| `mindxtrain.train.backend_trl_cpu.run_trl_local`). Auto-detects a consumer GPU |
| (CUDA or ROCm Radeon, bf16/fp16) and falls back to CPU; `trl_cpu` is the |
| force-CPU wrapper. Recipe `mindx_fallback_qwen3_1_5b_local`. Honest limit: |
| integrated Vega/`gfx90c` APUs are unsupported and fall back to CPU. |
| - **Verifiable training receipt.** Every operator/CPU run emits `manifest.json` |
| binding the frozen `AutotunePlan` hash to the checkpoint + config hashes |
| (`provenance.manifest.emit_receipt_for_run`); re-verified via `mindxtrain |
| receipt`, the `GET /coach/api/receipt/{run_id}` endpoint, and a Coach |
| "Verifiable receipt" card. Optional x402 metering gate on `/v1/training/jobs`. |
| - **Actor / persona / script + imprint.** Author a training *script* for an |
| *actor* (`mindxtrain.data.scripts`), ingest as `source: local`, and measure the |
| persona **imprint** by recall before/after training |
| (`mindxtrain.eval.imprint`, `mindxtrain imprint`, |
| `POST /coach/api/imprint/score`). Coach **Create script** card + |
| `POST/GET /coach/api/datasets`. Recipe `mindx_persona_imprint_local`. |
| - **mindX dream-cycle trigger** (`deploy.api_client.trigger_dream_ingestion`) — |
| hands an imprinted actor to mindX's `machine.dream` 8hr cycle via HTTP or an |
| inbox drop (clean-room: a pointer, never mindX code). `mindxtrain imprint |
| --trigger-dream`. |
| - **Coach live-training diagnostics** — accurate depiction with accordion |
| compression: rolling loss-chart window, per-step metrics + log accordions with |
| honest "showing last N" counts, system-metric sparklines, chat re-probe. |
| - **Autotune on real hardware** — runtime GPU-count autodetect (`rccl_probe`), |
| GEMM microbenchmark with timings (`gemm_probe`), `serve --to sglang`. |
| - **Clean-room policy** codified in `CLAUDE.md` + `AGENTS.md`: reimplement/adapt |
| locally, never copy mindX/external source bytes. |
|
|
| ### Added (pre-1.0 foundation) |
|
|
| - **mindX self-training loop.** `mindx_dreams` dataset source adapter walks |
| `<mindx>/data/memory/ltm/*/*_training.jsonl`, deduplicates by content hash, |
| and yields OpenAI-chat rows. Pure stdlib; no GPU/heavy deps to load |
| (`mindxtrain.data.sources.mindx_dreams`). |
| - **CPU training lane** (`mindxtrain.train.backend_trl_cpu.run_trl_cpu`). |
| Real TRL SFT/LoRA on CPU, produces a real HF-format checkpoint compatible |
| with `quantize`/`receipt`/`publish`. Wired via `TrainingBackend = "trl_cpu"`. |
| - **Public training-jobs API** at `/v1/training/jobs` with bearer auth |
| (`MINDXTRAIN_API_KEY`). Versioned facade over the same `RunRegistry` Coach |
| uses, so mindX agents and external clients become peer dispatchers |
| (`mindxtrain.operator.training_api`). |
| - **mindX fallback-swap caller** (`mindxtrain.deploy.api_client.swap_mindx_fallback_model`). |
| PATCHes mindX's `/v1/config/fallback-model` so a freshly published HF Hub |
| checkpoint becomes mindX's active default without a source edit. Pairs with |
| the mindX-side endpoint shipped in AgenticPlace/mindX@ad8193ea3. |
| - Two new YAML recipes: `mindx_fallback_qwen3_1_5b_sft_lora` (Qwen3-1.5B |
| LoRA on MI300X, the production target) and `mindx_fallback_qwen3_1_5b_cpu_smoke` |
| (SmolLM2-135M CPU smoke). |
| - `.env.example`: `MINDXTRAIN_API_KEY` and `MINDXTRAIN_MINDX_HOME`. |
|
|
| ### Changed |
|
|
| - Schema: `DataSource` extends to include `"mindx_dreams"`; `DataCfg.hf_id` |
| is now defaultable with a `model_validator` requiring it only for |
| `source: "hf"`; `DataCfg.path` added (used by `local` + `mindx_dreams`). |
| - `HardwareCfg.gpus: Literal[0, 1, 8]` — `0` = CPU lane. |
| - Production URL flipped from `mindx.pythai.net/hackathon` to |
| `mindx.pythai.net/coach`. Hackathon-era references preserved in |
| `HACKATHON.md` and the build-in-public posts for the historical record. |
| - CI: ruff scoped to `mindxtrain/ tests/`; mypy points at the canonical |
| `mindxtrain/config mindxtrain/provenance` (fixing stale paths). Container |
| build added on push-to-main; GHCR publish on tag. |
|
|
| ### Notes |
|
|
| - Adapter smoke against the real corpus saw 1051 unique rows in |
| `/home/hacker/mindX/data/memory` (corpus snapshot 2026-05-14). |
|
|
| ## [0.1.0] — 2026-05-06 |
|
|
| Initial public release. Submitted to the AMD × lablab.ai Developer Hackathon |
| (build window May 4–10 2026, on-site finale May 9–10 in San Francisco). |
|
|
| ### Added |
|
|
| - Single-package canonical layout per `docs/blueprints/mindxtrain2.md` §Part 4: |
| `mindxtrain/{cli,config,data,models,train,eval,autotune,operator,storage,provenance,deploy,budget}`. |
| - CLI with 8 verbs: `init`, `bench`, `train`, `eval`, `quantize`, `serve`, |
| `publish`, `receipt` (`mindxtrain.cli.main`). |
| - 60-second AOT autotune probe for AMD MI300X — the hackathon differentiator. |
| CK vs Triton attention selection + hipBLASLt GEMM heuristic + RCCL config |
| (`mindxtrain.autotune.{benchmark,attention_probe,gemm_probe,rccl_probe,plan}`). |
| - Pydantic v2 `XTrainConfig` with discriminated union over 9 training methods |
| (full, lora, qlora, dpo, orpo, grpo, gspo, kto, cpt) (`mindxtrain.config.schema`). |
| - 12 YAML training recipes covering Qwen3.5/Qwen3.6/Instella across |
| SFT-LoRA, full-FSDP, DPO, ORPO, GRPO, CPT, and VL. |
| - Axolotl YAML compiler; alt backends (Unsloth, torchtune, Primus) wired as |
| dispatch stubs (`mindxtrain.train`). |
| - Provenance manifest with BLAKE3 content addressing, ROCm/git/gfx capture, |
| and on-chain pointers (ERC-7857 INFT, Algorand ASA, ERC-8004 attestation) |
| (`mindxtrain.provenance`). |
| - Operator FastAPI app exposing `/v1/chat/completions`, `/v1/agentic`, and |
| the interactive `/coach/` UI (`mindxtrain.operator`). |
| - Pluggable inference backends: vLLM, OpenAI-compatible |
| (`mindxtrain.operator.backends`). |
| - Pluggable storage providers: local fs, HF Hub, Lighthouse, IPFS |
| (`mindxtrain.storage`). |
| - Foundry contracts for ERC-8004 attestation registry (`contracts/`). |
| - Containerfiles + compose + k8s manifests for MI300X (`ops/`). |
|
|
| ### Notes |
|
|
| - This release replaces the earlier 3-package layout |
| (`mindXtrain/`, `automindXtrain/`, `custmodel/`) with a single ordered |
| package `mindxtrain/`. Old import paths (`xtrain.*`, `automindx.*`, |
| `custmodel.*`) are not preserved — adopters must rewrite to `mindxtrain.*`. |
| - Most operator and trainer surfaces ship as honest minimal stubs that raise |
| `NotImplementedError` for paths not yet implemented in the hackathon scope. |
| The autotune probe, config schema, manifest, recipes, and Axolotl compiler |
| are the production-ready paths. |
|
|
| [0.1.0]: https://example.invalid/mindxtrain/releases/tag/v0.1.0 |
|
|
