Hugging Face's logo

khursheed
/
VibeGame-8B

Text Generation
Transformers
Safetensors
qwen2
godot
gdscript
qlora
game-development
conversational
text-generation-inference
Model card Files
xet
 Community
VibeGame-8B / code /docs /training.md
khursheed's picture
khursheed
Release pipeline source, tests, notebook, and licenses
ae36cfb verified
preview code
|
Raw History Blame Contribute Delete
20.7 kB
# Training and export on a free T4
VibeGame-8B is the project name. The selected model is **Qwen2.5-Coder-7B-Instruct**, with 7.61B parameters. We chose the code-specialized base because Unsloth distributes an NF4 checkpoint and links a free-T4 Qwen training notebook. We did not run a Qwen3 comparison or establish which base produces better games.
The self-contained [Colab notebook](../notebooks/VibeGame-8b.ipynb) embeds the project source, original seed candidates, frozen benchmark, reference corpus, source licenses, and separate teacher prompts. Rebuild it with `python scripts/build_notebook.py` after changing those files. It has no unpublished GitHub dependency, notebook outputs, or credentials. Every code cell is valid Python.
The notebook is designed for a **free Tesla T4** in Colab. No script purchases compute, invokes paid inference, or enables an endpoint. Python can check that the GPU is a T4; it cannot inspect the Colab account subscription. Google's free allocations vary and are not guaranteed to last 12 hours. A new session does not reset exhausted quota. This run finished training on a free T4. After the free GPU allowance was exhausted, the project owner purchased Colab Pro on October 4, 2026 and authorized resuming evaluation on a Pro T4. That partial T4 experiment is now stopped. An A100 is running a separate full evaluation after passing its training-prompt capacity/budget check. CPU-only recovery was stopped at the owner's request. Paid evaluation sessions must be disclosed alongside their measurements; they do not establish that the entire experiment finished on the free tier.
The default pipeline admits original complete-project candidates generated by the in-session Codex teacher, alongside the nine bootstrap seeds. Each candidate carries authorship provenance and must pass Godot import and a full 60-second clean run. No seed-only final training is silently substituted if the expansion candidates fail.
The optional expansion script samples Qwen2.5-Coder-14B-Instruct in 4-bit on the free T4. Its actual pilot loaded in about 10.5GB GPU RAM, but produced basic Godot syntax and asset-reference mistakes, so the failed outputs were discarded and generation was stopped. The revised sampler generates a gameplay script and assembles original fixed project/scene files; it never repairs model evaluation outputs. All attempts persist on Drive, including failures. In the notebook, `LOCAL_TEACHER_ATTEMPTS=0` uses the embedded strong-teacher candidates without rerunning the weak pilot. Set it explicitly to add local-teacher candidates, which must pass the same checks.
## Exact training recipe
| Setting | Value |
| --- | --- |
| Training weights | `unsloth/Qwen2.5-Coder-7B-Instruct-bnb-4bit` |
| Pinned revision | `4858886f896bd05db823557fc5cbc4ac28342af7` |
| Quantization | 4-bit NF4, double quantization |
| Compute | fp16; T4 does not use bf16 training |
| LoRA | rank 32, alpha 64, dropout 0, no bias |
| Targets | q/k/v/o projections and gate/up/down MLP projections |
| Gradient checkpointing | Unsloth |
| Context | 4096 tokens maximum |
| Batch | 1 × gradient accumulation 16 |
| Optimizer | 8-bit AdamW, learning rate 2e-4 |
| Schedule | cosine, 3% warmup, weight decay 0.01 |
| Epochs | 2 by default; 3 allowed |
| Loss | Assistant response tokens only |
| Evaluation loss | Development split, once per epoch and after final training |
| Blind benchmark | Exactly 500 frozen prompts, never used for loss or checkpoint selection |
The trainer first rejects duplicate prompts across train/dev/blind and identical project outputs across train/dev. Dataset construction also performs token-shingle near-duplicate rejection. A validation manifest must prove every training/development output was admitted by the Godot harness. Tokenization checks the exact Qwen chat prefix. A response over the context limit is rejected as a whole project and recorded in `token_filter.json`; no partial project is silently trained.
Run in Colab:
```sh
python scripts/train.py \
--train /content/drive/MyDrive/VibeGame-8B/data/train.jsonl \
--dev /content/drive/MyDrive/VibeGame-8B/data/dev.jsonl \
--blind-prompts /content/drive/MyDrive/VibeGame-8B/data/eval_prompts.jsonl \
--output /content/drive/MyDrive/VibeGame-8B/runs/teacher-v2
```
## Surviving disconnects
All training checkpoints are written directly to Drive, every 100 optimizer updates and at each epoch. Each includes the adapter, optimizer, scheduler, random-number-generator state, Trainer state, tokenizer, package versions, and input hashes. `COMPLETE.json` is written last with checksums. Resume selects the newest intact complete checkpoint, ignoring incomplete/corrupt writes. No checkpoint is automatically deleted.
A 10.5-hour per-invocation timer requests a save at an optimizer-step boundary and stops training with exit code 75. Signals request the same save. A VM killed without warning can still lose work since the last checkpoint; no code can guarantee a final write after the VM is gone. Rerun with the same data, software versions, and run directory to continue. Changed inputs or training versions require restoring the old environment or selecting a new run directory.
The final adapter is only written with a `TRAINING_COMPLETE.json` marker after the configured total training steps finish. A session-limited partial adapter is not relabeled as a completed release. `training_metrics.json`, `run_manifest.json`, package versions, and validation evidence stay on Drive.
Notebook setup preserves the first `RUN/packages.txt` rather than replacing it. Later setups write `packages-session-<UTC>-<number>.txt` with exclusive creation, so even two runs in the same second retain separate snapshots. If the original session was started outside the notebook, `packages-original.txt` and `runtime-package-versions.json` can preserve that initial evidence; the latter accepts a version mapping or a `versions` mapping inside an object.
Before installing on a fresh VM, setup builds exact constraints for a small allowlist of training/inference packages. It starts from `packages-original.txt` (or the initial `packages.txt`), applies `runtime-package-versions.json`, then the actual `run_manifest.json` version map. A completed adapter's saved evaluation manifest takes priority for its recorded `generation.packages` and `generation.torch_version`, because continuing evaluation may require a different environment from the earlier training run. Conflicting training/evaluation versions are rejected while training remains incomplete. A saved CUDA wheel suffix such as `+cu130` selects PyTorch's matching official wheel index. Setup verifies the selected installed versions in a fresh Python subprocess; it does not execute or reinstall arbitrary full-freeze entries or VCS URLs.
The recorded initial run used Unsloth 2026.9.14, Unsloth Zoo 2026.9.9, Transformers 4.56.2, Torch 2.11.0+cu130, PEFT 0.21.0, Datasets 4.3.0, bitsandbytes 0.50.2, Accelerate 1.15.0, and TRL 0.24.0. These saved versions, rather than a later package resolver's choices, guide a resumed setup. The converter separately requires Torch 2.11.0 and keeps its own dependency overlay.
This is not an exact recreation of an entire Colab machine: Python version, NVIDIA driver, system libraries, compiler kernels, and unselected transitive dependencies may change, and old wheels may become unavailable or incompatible. Setup does not weaken the existing training/evaluation resume guards if versions, runtime hashes, or protocol metadata still differ. Model jobs start in fresh subprocesses; after changing packages, old imports in the notebook kernel may require a Python-session restart for interactive use. Keep the complete original run folder and the original notebook when moving to a new free VM.
## Merging within normal host RAM
Loading a whole fp16 7.61B model into 12GB system RAM cannot work. The export script instead maps one source tensor at a time, adds that tensor's LoRA delta in float32 row chunks, rounds it to fp16, and writes one safetensors shard per tensor. A resumable manifest hashes each shard. The merge uses the original, unquantized Qwen base revision `c03e6d358207e414f1eca0bb1891e29f1db0e242`, as is conventional for QLoRA adapter merging. This changes the effective base from the 4-bit training weights; adapter metrics must not be reported as merged-model metrics.
For GGUF, the script uses llama.cpp revision `11fe02151f79c41d0d4af7da708755d73b9c0da6`. Its converter uses lazy tensor loading with `--use-temp-file`, then `llama-quantize` produces Q4_K_M on CPU. An Ollama Modelfile preserves the Qwen chat delimiters, 4096-token context, and the evaluation decoding defaults. Finite-value and shape checks catch invalid tensor merges; a small numerical test compares chunked and dense LoRA arithmetic. These checks do not prove full-model memory use, generation quality, or GGUF equivalence. A full T4 export must be measured.
The converter dependencies live in a separate virtual environment that inherits the runtime's installed Torch 2.11.0 build. Its Transformers, NumPy, and Protobuf requirements never replace the active evaluation environment. The GGUF manifest records the converter's exact package versions. Merge math, compilation, and quantization each use one CPU thread; quantization has a 256MiB row-buffer budget instead of upstream's 8192MiB default. The merge also avoids a second copy of its largest 1.01GiB tensor.
Export can overlap GPU evaluation because it uses CPU memory and disk, but shared CPU and disk load affects the recorded game-generation timings. For overlap, start only after all evaluation workers have loaded and at least 6GiB host RAM remains available, then monitor real memory use. The 4GiB script minimum is a sequential-export preflight, not a measured concurrency guarantee. Stop the export subprocess if memory becomes tight; tensor merge progress is resumable. Record any overlap when reporting latency. About 54GB free VM disk can cover the roughly 46GB peak model staging, provided other downloads do not consume the remaining space.
```sh
python scripts/export_model.py \
--adapter /content/drive/MyDrive/VibeGame-8B/runs/teacher-v2/adapter \
--output /content/vibegame-exports
```
The fp16 model alone is about 15.2GB in decimal units, before checkpoints or other files. It cannot fit alongside them in a default empty 15GB Drive. Only the checkpoints/final adapter are required to persist there. Export artifacts use ephemeral VM disk by default, with at least 4GiB available host RAM and conservative per-stage 34GiB free-disk checks. The script removes its own disposable original-base cache after a verified merge, then uses temporary disk for GGUF conversion. It does not purchase storage or delete unrelated files. Back up the exports privately before disconnect, or reconstruct them from the persistent adapter later.
## Restoring a completed private export backup
The current run's private backups completed at **13:30 UTC on October 4**: 353 merged-model repository files and six GGUF repository files were verified. Before staging, all 339 recovered FP16 shard hashes and the completion metadata matched the original Drive evidence. Both repositories remain private with empty inference-provider mappings; this is an export backup, not the final public release.
After `stage_exports.py` finishes both private uploads, retain its complete staging receipt plus `export-evidence/merged-fp16/MERGE_COMPLETE.json`, `export-evidence/merged-fp16/merge_progress.json`, and `export-evidence/gguf/GGUF_COMPLETE.json` on Drive beside the completed adapter. A new free Colab VM can recover those exact exports without another merge or quantization:
```sh
python scripts/restore_exports.py \
--receipt /content/drive/MyDrive/VibeGame-8B/runs/teacher-v2/export-staging-receipt.json \
--evidence /content/drive/MyDrive/VibeGame-8B/runs/teacher-v2/export-evidence \
--adapter /content/drive/MyDrive/VibeGame-8B/runs/teacher-v2/adapter \
--output /content/vibegame-exports
```
Use the actual run and receipt paths. The helper requests a hidden token, or accepts an inherited `HF_TOKEN`; it never logs in, saves the token, changes repository visibility, or calls inference. It requires both repositories to remain private with empty provider mappings. Downloads are pinned to the immutable commits recorded in the receipt, with exact remote inventory checks and local SHA-256 verification. The adapter, original base revision, tensor index, Drive merge evidence, and GGUF receipt must agree. The final completion markers are copied from verified Drive evidence only after both exports finish. The global export lock prevents a continuation worker from rebuilding exports while recovery is running.
Downloads run one file at a time through the Hub's streaming HTTP path, without a second full central-cache copy. Preflight reserves the missing bytes plus the largest missing file and 2GiB of disk, and requires 2GiB available host RAM. Interrupted downloads can be resumed; already verified files are reused. An existing file with different contents stops recovery rather than being overwritten. The original staging receipt is preserved, and a separate `export-restore.json` on Drive records progress without credential details. Recovery requires a completed receipt for both exports; a partial staging upload or a repository already made public is not accepted by this private-backup helper. Evaluation records and decoding settings are untouched.
## Export recovery after a disconnected session
If private staging never completed, `restore_exports.py` cannot recover missing uploads. Keep the completed adapter, original export evidence, and saved GGUF on Drive. Continue in the authorized T4 runtime. Restore the recorded export dependencies, including **Torch 2.11.0+cu130** and **safetensors 0.8.0**, then rebuild only FP16 into a fresh recovery directory. Do not reuse a reconstruction that merely passes its own manifest:
```sh
python scripts/export_model.py \
--adapter /content/drive/MyDrive/VibeGame-8B/runs/teacher-v2/adapter \
--output /content/vibegame-exports \
--merge-only
```
Before reusing the earlier GGUF, compare the rebuilt `merge_progress.json` against Drive's original `export-evidence/merged-fp16/merge_progress.json`: the merge signature, all **339** tensor names and shard filenames must agree, and each rebuilt shard's SHA-256 must match its original recorded hash. Also require the rebuilt `MERGE_COMPLETE.json` to equal the GGUF receipt's `source_merge`, and verify the saved GGUF's size and SHA-256 against `GGUF_COMPLETE.json`. Stop on any mismatch; do not replace the original evidence. An earlier October 4 CPU-only reconstruction completed all 339 shards, but 25 file hashes differed from the original export. That CPU-only session was stopped at the owner's request. The private upload guard stopped it. A fresh recovery on the Pro T4 host with the recorded Torch 2.11.0+cu130 and safetensors 0.8.0 completed at 13:14 UTC: all 339 shard hashes and completion metadata matched the original export. This establishes export identity; the earlier CPU-build mismatch remains unexplained. This did not change the saved training adapter or original GGUF, and the cause of the file differences has not yet been established.
Reuse Drive's `exports/gguf` directory, including its original `GGUF_COMPLETE.json` and `Modelfile`; another quantization is unnecessary. Once the comparisons pass, stage both exports privately:
```sh
python scripts/stage_exports.py \
--adapter /content/drive/MyDrive/VibeGame-8B/runs/teacher-v2/adapter \
--merged /content/vibegame-exports/merged-fp16 \
--original-merge-evidence /content/drive/MyDrive/VibeGame-8B/runs/teacher-v2/export-evidence/merged-fp16 \
--gguf /content/drive/MyDrive/VibeGame-8B/runs/teacher-v2/exports/gguf \
--output /content/drive/MyDrive/VibeGame-8B/runs/teacher-v2/export-staging-receipt.json
```
Use a fine-grained token with read access to your own repositories and **Create repos and write to repos created by this token** (`repo.contribute.write`). Leave inference, endpoints, billing, and jobs permissions unchecked. Supply it through hidden input or an inherited in-memory environment. The scripts rely on API authorization rather than requiring the broader `repo.write` label; later visibility changes must also be authorized. Staging checks both artifacts, preserves upstream licenses, verifies remote hashes and empty provider mappings, and leaves both repositories private. It does not complete GPU evaluation or guarantee publication: the full 500-prompt evidence and verified demo remain required.
## Evaluation and publication
The owner later purchased Colab Pro. An optional `scripts/evaluate_pro.py` entry point can run a **separate full A100 benchmark** after export backups are verified. It requires an actual A100 plus the compute-unit rate and remaining balance observed in Colab. It preserves `evaluation/` as the partial T4 experiment and writes only to `evaluation-a100-v1/`. The adapter, tokenizer, prompts, greedy decoding, 3072-token cap, batch of eight, and sample slots 0/200/400 stay fixed. Results from the two hardware runs are reported separately; they are never pooled by choosing the better output for each prompt. On October 4 an A100 was allocated after selection at 14:06 UTC. Its full batch-eight capacity check took 271.711 seconds and passed the observed budget check; the separate evaluation is running. The stopped T4 snapshot has 72 recorded results, 64 saved generations, one compile/clean-run success and no infrastructure failures; see `docs/evaluation.md` for its limited timing evidence. No final model rates are available yet.
Before reading new blind outputs, this runner records the protocol and measures capacity using four prompts from the exact admitted training split. It projects the full evaluation cost from that measurement, with a generation margin, separate validation allowance, and a balance reserve. It stops if the observed balance cannot cover that estimate. This does not purchase compute or guarantee Colab's future rate or availability. The notebook's default training path still requires a T4.
The blind evaluation performs one generation per prompt, checks import, and runs each accepted project for 60 wall-clock seconds. It records original responses, errors, runtime logs, timing, and model/dataset hashes on Drive. Sequential validation alone takes at least 8 hours 20 minutes for 500 clean runs, excluding generation, installation, and imports. Four CPU validation workers reduce that theoretical runtime floor to 2 hours 5 minutes. GPU generation uses a batch size selected by capacity checks on training prompts before blind evaluation. Each prompt receives the actual whole-batch call time and validation queue time; timings are never divided by batch size. The pipeline therefore does not promise training plus complete evaluation inside one free session. Evaluation resumes with immutable per-prompt records.
Publication checks complete training, actual LoRA tensors, a complete fp16 tensor index, GGUF magic/hash, admitted dataset outputs, and all 500 evaluation records tied to that exact adapter. It publishes four repositories, preserving source licenses and honest measurements, without logging in to a persistent token store. The write token enters through hidden input or an inherited in-memory environment. Provider mappings are inspected before and after uploads. `inference: false` hides a widget; it cannot disable account billing. A final account-settings inspection must verify spending/provider settings separately. No provider or endpoint is created by the publisher.
## Primary sources
- [Unsloth's Qwen2.5-Coder-7B NF4 model and free-T4 notebook links](https://huggingface.co/unsloth/Qwen2.5-Coder-7B-Instruct-bnb-4bit)
- [Qwen's original model](https://huggingface.co/Qwen/Qwen2.5-Coder-7B-Instruct)
- [Unsloth Colab installation](https://unsloth.ai/docs/get-started/install/google-colab)
- [Unsloth export memory troubleshooting](https://docs.unsloth.ai/basics/saving-and-using-models/troubleshooting)
- [Colab resource limits and availability](https://research.google.com/colaboratory/faq.html)
- [llama.cpp converter's lazy/temp-file options](https://github.com/ggml-org/llama.cpp/blob/11fe02151f79c41d0d4af7da708755d73b9c0da6/convert_hf_to_gguf.py)
Source support was checked on October 3, 2026. Package versions and actual runtime measurements are recorded separately; support documentation is not a successful training run.