PocketVision Chat & Code — version 2 preparation

This version accepts text and produces text. It focuses on ordinary conversation, explaining code, generating functions, and debugging. There is no image input, vision encoder, image processor, or ChartQA training in the new pipeline.

Status: source and training recipe published privately at gamedevelop/pocketvision-chat-code; no new model trained or deployed. The original gamedevelop/pocketvision repository was deleted at the owner's request. Its local SmolVLM/ChartQA weights and ZIP backup remain available and are incompatible with this version. The local app shows an interface preview without downloading the new base weights. It does not silently run the old model under the new name.

Model and training data

Base: Qwen/Qwen3-4B-Instruct-2507, Apache-2.0, pinned to cdbee75f17c01a7cc42f958dc650907174af0554. This is an instruction-tuned text model; its model card includes chat and coding evaluations.
Data: HuggingFaceTB/smoltalk, pinned to 5feaf2fd3ffca7c237fc38d1861bc30365d48ffa.
Pilot: 12,000 self-oss-instruct programming examples and 8,000 smol-magpie-ultra conversation examples. The coding source primarily covers Python. JavaScript and other languages need additional sources and tests.
Attribution for the coding subset: BigCode, self-oss-instruct-sc2-exec-filter-50k, Open Data Commons Attribution License (ODC-BY). The source describes its responses as execution-validated; this project does not rerun or independently certify them. Smol-Magpie-Ultra is Apache-2.0. Dependency and underlying source terms still apply.
Separate validation and test sets: 128 examples from each source per split. All three derived splits come from the sources' official training split. Official test data is unused. Identical normalized opening questions are grouped and deduplicated globally. This does not establish novel or semantically disjoint data relative to the pretrained base.
Examples longer than 2,048 tokens are skipped, preserving complete code/answers. Only assistant answer and end-of-turn tokens contribute to loss.
One epoch of 4-bit NF4 QLoRA, rank 16, all linear projections, learning rate 5e-5, microbatch 1, accumulation 16. This is a starting recipe, not validated evidence of improved model quality or a guarantee that it fits every GPU.

Local interface preview

Use the existing environment and run python app.py. With no CUDA GPU, it displays the text-only interface with sending disabled. No new model weights are fetched. The original visual UI is preserved as vision_app.py for reproducibility.

Train on a remote Linux GPU

Use a Colab GPU runtime or a Hugging Face Job. A 24 GB GPU is a reasonable starting target, subject to an actual memory smoke test. None has been rented or started. Run these commands on the cloud machine, from the unpacked source directory:

python -m pip install torch==2.9.1 --index-url https://download.pytorch.org/whl/cu128
python -m pip install -r chatcode/requirements-cloud.txt
python -m pip check
python -m chatcode.data --output /checkpoints/chatcode-data
python -m chatcode.train --data /checkpoints/chatcode-data --output /checkpoints/smoke --max-steps 25

/checkpoints must be your mounted persistent bucket/volume or Drive directory. Do not use a transient runtime directory as the only copy: Jobs discard local disk when they end, and notebook sessions can be terminated. Checkpoints include optimizer, scheduler, trainer and random state; they are saved every 25 steps.

First inspect the smoke run's GPU memory and steps/second. Estimate cost and agree on a run limit before launching the full run. A 25-step smoke run is still paid compute when using a paid GPU. Model/data downloads and setup also take time. Once that succeeds, run the full pilot in a new output directory:

python -m chatcode.train --data /checkpoints/chatcode-data --output /checkpoints/pilot

Resume an interrupted run with the same configuration and data:

python -m chatcode.train --data /checkpoints/chatcode-data --output /checkpoints/pilot --resume-from-checkpoint /checkpoints/pilot/checkpoint-N

The script explicitly refuses CPU training. It writes the selected adapter, tokenizer, provenance, training report, and candidate model card under adapter/. It does not upload automatically, launch cloud hardware, or change the existing Hub repository. There is no new dataset or GPU training run in this delivery.

Evaluate and release

Validation loss selects a checkpoint; it does not measure whether generated programs work. Before releasing, compare the base and candidate on the untouched derived test split and a separate set of conversation, instruction-following, debugging, and coding tasks. Record every prompt, output, model revision, and generation setting. Use tests for coding correctness and inspect chat quality for regressions. The source bundle does not include a sandboxed code-execution evaluator; never automatically execute arbitrary training data or generated programs on a machine with credentials. Use an isolated evaluation environment.

Publish a tested candidate to a new private repository or a distinct version branch; the old vision adapter must not be relabeled as a chat/code model. Preserve the base revision and dataset attribution. No quality improvement is claimed yet.

For a regular cloud GPU demo, install the cloud dependencies, set POCKETCHAT_ADAPTER to the compatible local adapter directory or Hub repository, and run python app.py. Without an adapter it explicitly identifies itself as the pretrained Qwen base. On Spaces set SPACE_ID normally via the platform. ZeroGPU needs a separate startup/decorator integration and compatible PyTorch environment; this ordinary GPU app is not yet a verified ZeroGPU deployment.

Package

python -m chatcode.package creates dist/pocketvision-chat-code-source.zip. It includes only the active app, text pipeline, configuration, documentation, and licenses. It excludes old weights, datasets, caches, environments, and credentials. Project source remains GPL-3.0-only; original Qwen weights remain Apache-2.0.

Sources: QLoRA with PEFT, Hugging Face training Jobs.