docs/install.md

Install on the mindX VPS: bankml 0.3.6 and mindXtrain's bankml lane

This guide upgrades the mindX node (mindx.pythai.net, 168.231.126.58) in eight steps:

1. bankml 0.3.5 → 0.3.6 (repetition penalties, token-identical to llama-server)
2. gen39 re-layered so it answers in words
3. mindXtrain → the Hub's main (the bankml serve target, backend and judges)
4. mindX pointed at bankml for mindXtrain's chat, judges and teacher

Every block is meant to be copied whole and pasted into a root shell on the VPS. Each step ends with a check; do not go on until it passes. Roll back undoes any step.

What you need: root SSH to the VPS, about 15 minutes, and 1 GB free on /. Restarting bankml makes mindX's default model unavailable for a few seconds; restarting mindx takes 3–4 minutes before /health answers again.

---

0. Log in and look before you touch

ssh root@168.231.126.58

df -h / | tail -1
systemctl is-active bankml mindx
curl -s 127.0.0.1:18093/bankml | head -c 200; echo
/home/mindx/bankml-0.3.5/bankml --version

You should see both services active, a "verdict": "play" from bankml, and bankml 0.3.5.

Check that no training run is going (a mindX restart in step 7 would end it):

pgrep -af 'mindxtrain (train|ascend)' || echo "no training running"

1. Back up what you will change

TS=$(date -u +%Y%m%dT%H%M%SZ); mkdir -p /root/deploy_backup_bankml036_$TS
cp /etc/systemd/system/bankml.service /home/mindx/mindX/.env /root/deploy_backup_bankml036_$TS/
echo "/root/deploy_backup_bankml036_$TS" | tee /root/LAST_BANKML036_BACKUP
ls -la /root/deploy_backup_bankml036_$TS

2. Build bankml 0.3.6

bankml has no dependencies to fetch; it builds with the mindx user's cargo in a few minutes on the VPS's two cores. Build from the v0.3.6 tag once it is published; until then, from main.

sudo -u mindx bash -lc 'cd /home/mindx && rm -rf bankml-src-0.3.6 && git clone --depth 1 --branch v0.3.6 https://github.com/cryptoAGI/bankml bankml-src-0.3.6 || git clone --depth 1 https://github.com/cryptoAGI/bankml bankml-src-0.3.6'

sudo -u mindx bash -lc 'cd /home/mindx/bankml-src-0.3.6 && nice -n 19 cargo build --release 2>&1 | tail -3'

sudo -u mindx mkdir -p /home/mindx/bankml-0.3.6
sudo -u mindx cp /home/mindx/bankml-src-0.3.6/target/release/bankml /home/mindx/bankml-0.3.6/bankml
/home/mindx/bankml-0.3.6/bankml --version

Check: it prints bankml 0.3.6 (or later). If it prints 0.3.5, the tag is not out yet and main has not moved past it either: stop here, nothing has changed on the running node.

3. Point the service at 0.3.6 and restart bankml

sed -i 's#/home/mindx/bankml-0\.3\.5/bankml#/home/mindx/bankml-0.3.6/bankml#' /etc/systemd/system/bankml.service
grep ExecStart /etc/systemd/system/bankml.service

systemctl daemon-reload && systemctl restart bankml && sleep 5 && systemctl is-active bankml

curl -s 127.0.0.1:18093/bankml | grep -o '"bankml": *"[^"]*"'

Check: "bankml": "0.3.6". Now prove the penalties are honoured (0.3.5 answered HTTP 400 to this request):

curl -s -o /dev/null -w '%{http_code}\n' 127.0.0.1:18093/v1/chat/completions -H 'Content-Type: application/json' -d '{"model":"mindx-gen39-f16","messages":[{"role":"user","content":"Who are you?"}],"max_tokens":16,"repeat_penalty":1.3}'

Check: 200.

4. Layer gen39 so it answers in words

A 135M generation on its own repeats itself (,,,,). A layer over the pinned GGUF adds its persona, its stop string and a 1.3 repeat penalty; the weights are not copied.

cat > /home/mindx/models/Modelfile.mindx-gen39 <<'MF'
FROM mindx-gen39-f16
SYSTEM """You are mindX, generation 39."""
PARAMETER stop "<|im_end|>"
PARAMETER num_ctx 2048
PARAMETER repeat_penalty 1.3
MF
chown mindx:mindx /home/mindx/models/Modelfile.mindx-gen39

sudo -u mindx /home/mindx/bankml-0.3.6/bankml create mindx-gen39 -f /home/mindx/models/Modelfile.mindx-gen39 --registry /home/mindx/models/registry

curl -s 127.0.0.1:18093/v1/chat/completions -H 'Content-Type: application/json' -d '{"model":"mindx-gen39","messages":[{"role":"user","content":"Who are you?"}],"max_tokens":48}' | python3 -c 'import json,sys; print(json.load(sys.stdin)["choices"][0]["message"]["content"])'

Check: an answer in words, not a run of commas. (mindx-gen39-f16 is still the bare pin; mindx-gen39 is now the layer. One thread on the VPS: give it 10–30 seconds.)

5. Bring mindXtrain to the Hub's main

mindXtrain moved from GitHub (archived) to huggingface.co/PYTHAI/mindXtrain. The checkout on the VPS still tracks GitHub at v1.0.2 and has a local uv.lock edit, which is set aside first.

cd /home/mindx/mindXtrain
sudo -u mindx git stash push -m "pre-bankml uv.lock" -- uv.lock || true
sudo -u mindx git remote add hf https://huggingface.co/PYTHAI/mindXtrain 2>/dev/null || true
sudo -u mindx GIT_LFS_SKIP_SMUDGE=1 git fetch hf main
sudo -u mindx git checkout -B main hf/main
sudo -u mindx git log --oneline -1

Install it into the existing virtualenv. --inexact keeps the extras already installed there (torch, peft, trl), which a plain uv sync would remove:

cd /home/mindx/mindXtrain && sudo -u mindx /home/mindx/.local/bin/uv sync --inexact --extra ml 2>&1 | tail -2

cd /home/mindx/mindXtrain && sudo -u mindx .venv/bin/mindxtrain serve --help | grep -c 'bankml'

Check: a number above 0 (the --to bankml options are there).

6. Tell mindX's mindXtrain to use bankml

mindX starts mindXtrain with its own environment (/home/mindx/mindX/.env). Append the bankml lane (Bonsai-8B is the model on the VPS that is good enough to judge and teach; gen39 is too small for that):

cat >> /home/mindx/mindX/.env <<'ENV'

# mindXtrain on bankml (docs/install.md in PYTHAI/mindXtrain)
MINDXTRAIN_BANKML_BASE_URL=http://127.0.0.1:18093/v1
MINDXTRAIN_CHAT_MODEL=bonsai-8b-q1_0
MINDXTRAIN_JUDGE_MODEL=bonsai-8b-q1_0
MINDXTRAIN_TEACHER_BASE_URL=http://127.0.0.1:18093/v1
MINDXTRAIN_TEACHER_MODEL=bonsai-8b-q1_0
ENV
grep -n 'MINDXTRAIN_' /home/mindx/mindX/.env

MINDXTRAIN_BACKEND=bankml is deliberately not set: with it, the panel would use bankml even where an MINDXTRAIN_OPENAI_BASE_URL is configured. Add it only if bankml should be the only chat endpoint.

7. Restart mindX and confirm

Look at what is inside mindX's cgroup first; a restart ends everything listed there:

for p in $(cat /sys/fs/cgroup/system.slice/mindx.service/cgroup.procs); do ps -o pid=,args= -p $p; done | cut -c1-120

systemctl restart mindx && echo "restarted at $(date -u +%T); /health answers in 3-4 minutes"

until curl -s -o /dev/null -w '%{http_code}' 127.0.0.1:8000/health | grep -q 200; do sleep 15; done; echo "mindX is up"

tr '\0' '\n' < /proc/$(systemctl show -p MainPID --value mindx)/environ | grep MINDXTRAIN_

Check: the five MINDXTRAIN_ lines from step 6.

8. Serving the next generation

When a run finishes, put it on bankml (merge, pin byte-identically, layer) in one command. Use the run's config and tag:

cd /home/mindx/mindXtrain && sudo -u mindx BANKML_FORKS=/home/mindx/models/registry .venv/bin/mindxtrain serve /path/to/run.yaml --to bankml --tag mindx-gen82 --bankml-bin /home/mindx/bankml-0.3.6/bankml --bankml-system "You are mindX, generation 82." --bankml-param repeat_penalty=1.3 --bankml-param num_ctx=2048 --bankml-stop '<|im_end|>'

It prints the model's sha256; curl -s 127.0.0.1:18093/api/tags then lists mindx-gen82.

---

Rollback

Each line undoes one step; run only the ones you need.

B=$(cat /root/LAST_BANKML036_BACKUP); echo "restoring from $B"

bankml back to 0.3.5 (steps 2–3):

cp $B/bankml.service /etc/systemd/system/bankml.service && systemctl daemon-reload && systemctl restart bankml && /home/mindx/bankml-0.3.5/bankml --version

gen39's layer (step 4), so mindx-gen39 is the bare pin's alias again:

rm -f /home/mindx/models/registry/mindx-gen39.MODEL.json

mindX's environment (step 6), then restart mindX as in step 7:

cp $B/.env /home/mindx/mindX/.env && chown mindx:mindx /home/mindx/mindX/.env

mindXtrain back to the GitHub checkout (step 5):

cd /home/mindx/mindXtrain && sudo -u mindx git checkout -B main github/main && sudo -u mindx git stash pop || true

Troubleshooting

you see	it means	do
cargo: command not found	the build ran outside mindx's login shell	keep the sudo -u mindx bash -lc '…' form
step 3 answers 400	bankml is still 0.3.5	grep ExecStart /etc/systemd/system/bankml.service, then systemctl daemon-reload && systemctl restart bankml
bankml create … refuse	the Modelfile asks for something bankml does not reproduce	the message names it; penalties need 0.3.6
git fetch hf fails	the Hub was unreachable	curl -sI https://huggingface.co/PYTHAI/mindXtrain, then retry
/health never turns 200	mindX did not start	journalctl -u mindx -n 50 --no-pager