README.md

metadata

language:
  - en
license: gpl-3.0
tags:
  - code
  - text-only
  - training-recipe
  - work-in-progress

PocketVision Chat, Code & Writing

Status: source project and cloud training recipe only. No custom model weights or adapter have been trained or uploaded for this version.

This project focuses on conversation, programming and source-based writing: explaining code, generating functions, debugging, neutral reference articles, and research outlines. Image upload and image understanding have been removed from the active application.

This repository is not yet a downloadable fine-tuned model or a running chat service. The supplied app uses the pretrained base until a compatible trained adapter is supplied. On a computer without a CUDA GPU it displays an interface preview and does not download the larger base weights.

Selected base model

Qwen/Qwen3-4B-Instruct-2507, Apache-2.0. Pinned revision: cdbee75f17c01a7cc42f958dc650907174af0554.

The Qwen weights are supplied by the original publisher and are not re-uploaded here. The original PocketVision SmolVLM/ChartQA adapter cannot be used with this architecture.

Planned training

The first proposed cloud experiment uses 20,000 examples from HuggingFaceTB/smoltalk, pinned to 5feaf2fd3ffca7c237fc38d1861bc30365d48ffa:

• 50% self-oss-instruct: programming, primarily Python. Original source: BigCode, self-oss-instruct-sc2-exec-filter-50k, ODC-BY.
• 30% smol-magpie-ultra: conversation and instruction following, Apache-2.0.
• 10% smol-rewrite: writing and editing, Apache-2.0.
• 10% smol-summarize: source summarization, Apache-2.0.
• 20 additional authored fictional article-format examples (12 train, 4 validation, 4 test); source packets stay in one split.
• Separate derived validation and test splits; duplicate opening prompts are removed.
• Assistant-only loss; 4-bit NF4 QLoRA, rank 16; one epoch as a starting experiment.

These are planned settings, not a description of completed training. Synthetic answers may contain errors. The base may already have seen these sources. The programming pilot does not establish broad coverage of other languages.

Files and cloud workflow

• app.py and chat.py: text-only web and CLI entry points.
• chatcode/: runtime, data preparation, cloud trainer, tests, configuration, dependencies, and the detailed guide.
• Cloud training guide.
• Cloud training notebook (saved outputs are a local dry run only).
• The existing pocketvision-chat-code-source.zip is the original source snapshot before writing support. Use the current source files or the notebook's pinned snapshot download for this version.

Articles and Wikipedia preparation

Reference article mode uses supplied source excerpts to draft a short neutral article with a lead, sections, references and research gaps. It rejects missing/unknown citation IDs and builds reference URLs only from supplied metadata. It does not fetch source URLs or verify factual entailment; each claim still requires human source review.

Wikipedia outline mode returns questions and source-review tasks for a human editor. English Wikipedia's current guideline restricts LLM-generated article prose, so reference drafts are intended for other writing contexts. Nothing is automatically published to Wikipedia.

The Wikimedia Wikipedia corpus is not included in the default mix: its cleaned articles have references removed and are not source-grounded instruction examples. The small authored examples use fictional practice topics and reserved URLs, not genuine research references.

Run the following on a remote Linux CUDA GPU, from the source directory:

python -m pip install torch==2.9.1 --index-url https://download.pytorch.org/whl/cu128
python -m pip install -r chatcode/requirements-cloud.txt
python -m pip check
python -m chatcode.data --output /checkpoints/chatcode-data
python -m chatcode.train --data /checkpoints/chatcode-data --output /checkpoints/smoke --max-steps 25

Use mounted persistent storage for /checkpoints. Inspect the smoke run before estimating and launching a full experiment. No paid hardware is started by uploading this repository. The trainer refuses CPU training and does not upload automatically.

Verification and limits

Offline checks cover assistant-only labels, source-packet separation, malformed and invented references, text-only validation, CPU refusal before model download, data integrity, and a forward/backward check with a tiny randomly initialized Qwen architecture. These checks do not train or evaluate the selected 4B model. The Colab notebook's Python cells were executed only in dry-run mode; its full GPU path is unverified.

The cloud dependencies and full GPU training run still need a cloud smoke test. No chat-quality or programming-accuracy improvement is claimed. A trained candidate needs separate answer review and isolated code tests before release. This repository does not include a sandboxed code-execution evaluator.

Licenses and attribution

Project source: GPL-3.0-only (LICENSE). Original Qwen base: Apache-2.0 (LICENSE-BASE-MODEL.txt). Dataset and dependency licenses remain applicable. See the training guide and configuration for provenance. No pretrained base weights, raw datasets, credentials, or original vision adapter are included.