Update chatcode/README.md

b605310 verified 3 days ago

10.3 kB

# PocketVision Chat, Code & Writing — source preparation

This version accepts text and produces text. It focuses on ordinary conversation,
explaining code, generating functions, debugging, and source-based article writing. There is no image input,
vision encoder, image processor, or ChartQA training in the new pipeline.

**Status:** source and training recipe published privately at
[gamedevelop/pocketvision-chat-code](https://huggingface.co/gamedevelop/pocketvision-chat-code);
no new model trained or deployed. The original `gamedevelop/pocketvision` repository
was deleted at the owner's request. Its local SmolVLM/ChartQA weights and ZIP backup
remain available and are incompatible with this version.
The local app shows an interface preview without downloading the new base weights.
It does not silently run the old model under the new name.

## Model and training data

- Base: [Qwen/Qwen3-4B-Instruct-2507](https://huggingface.co/Qwen/Qwen3-4B-Instruct-2507),
  Apache-2.0, pinned to `cdbee75f17c01a7cc42f958dc650907174af0554`.
  This is an instruction-tuned text model; its model card includes chat and coding evaluations.
- Data: [HuggingFaceTB/smoltalk](https://huggingface.co/datasets/HuggingFaceTB/smoltalk),
  pinned to `5feaf2fd3ffca7c237fc38d1861bc30365d48ffa`.
- Planned Hub mix: 10,000 `self-oss-instruct` programming examples, 6,000
  `smol-magpie-ultra` conversation examples, 2,000 `smol-rewrite` writing/editing
  examples, and 2,000 `smol-summarize` source summarization examples. The coding source primarily covers
  Python. JavaScript and other languages need additional sources and tests.
- Attribution for the coding subset: BigCode,
  [self-oss-instruct-sc2-exec-filter-50k](https://huggingface.co/datasets/bigcode/self-oss-instruct-sc2-exec-filter-50k),
  Open Data Commons Attribution License (ODC-BY). The source describes its responses
  as execution-validated; this project does not rerun or independently certify them.
  Smol-Magpie-Ultra, Smol-rewrite, and Smol-summarize are Apache-2.0.
  Dependency and underlying source terms still apply.
- Separate Hub validation and test sets: 128 examples from each of four sources per split (512 each).
  All three derived splits come from the sources' official **training** split.
  Official test data is unused. Identical normalized opening questions are grouped
  and deduplicated globally. This does not establish novel or semantically disjoint
  data relative to the pretrained base.
- An additional 20 project-authored examples teach the article JSON and source-review
  tasks: 12 training, 4 validation, 4 test. They use **fictional practice topics and
  reserved example.org URLs**, not real references. Both tasks for a source packet
  stay in the same split, enforced again when the trainer loads data. These small
  examples teach format; they are not a large factual corpus or evidence of quality.
  The final default counts are 20,012 train and 516 each validation/test.
- Examples longer than 2,048 tokens are skipped, preserving complete code/answers.
  Only assistant answer and end-of-turn tokens contribute to loss.
- One epoch of 4-bit NF4 QLoRA, rank 16, all linear projections, learning rate
  5e-5, microbatch 1, accumulation 16. This is a starting recipe, not validated
  evidence of improved model quality or a guarantee that it fits every GPU.

## Local interface preview

Use the existing environment and run `python app.py`. With no CUDA GPU, it displays
the text-only interface with sending disabled. No new model weights are fetched.
The original visual UI is preserved as `vision_app.py` for reproducibility.

## Article writing and Wikipedia preparation

The **Articles & research** tab has two tasks. Reference article produces a short,
neutral article with a lead, sections, references and research gaps. Wikipedia
outline and source review produces section questions and source-review tasks for
a human editor, without article prose. Paste titles, URLs and relevant excerpts;
the app does not fetch websites or search the web.

The model returns structured text with source IDs. Application code rejects unknown
or absent IDs and renders URLs only from the supplied metadata. Incomplete JSON,
uncited paragraphs and model-added raw citation markup are rejected. This is **not
a factual-entailment or source-reliability checker**. A valid ID can still point to
an excerpt that does not support the claim, so every draft requires human review.
No articles are uploaded to Wikipedia by this app.

English Wikipedia's [current guideline](https://en.wikipedia.org/wiki/Wikipedia:Writing_articles_with_large_language_models)
prohibits LLM-generated/re-written article prose, with limited copyediting and
translation exceptions. The reference-article task is for other writing contexts;
use the planning task for Wikipedia assistance.

The [Wikimedia Wikipedia dataset](https://huggingface.co/datasets/wikimedia/wikipedia)
contains cleaned article text with references removed. It is not in the default
mix: its raw text is not a ready-made source-grounded instruction dataset. Any later
use needs article-level provenance and the corpus's CC-BY-SA/GFDL terms recorded.
Wikipedia articles should not themselves be cited as evidence for new Wikipedia
article prose; consult their original reliable sources.

## Colab notebook

[cloud_training.ipynb](cloud_training.ipynb) is a cloud training walkthrough.
Upload it to Colab, select a free GPU if available, add your own existing read token
in Colab Secrets, and follow its setup. It downloads an immutable snapshot of source
files and keeps checkpoints in your chosen persistent Drive folder.

The saved notebook was structurally checked, its Python cells compiled, and all
cells executed **only with `DRY_RUN=True`**. `nbformat` and a Colab GPU runtime are
not available in this local environment; a full cloud run remains unverified.
Set `DRY_RUN=False` for cloud setup and separately enable `RUN_TRAINING` to train.
No subscription, GPU job or model publication is started by this preparation.
## Train on a remote Linux GPU
Use a Colab GPU runtime or a Hugging Face Job. A 24 GB GPU is a reasonable starting
target, subject to an actual memory smoke test. None has been rented or started.
Run these commands **on the cloud machine**, from the unpacked source directory:
```bash
python -m pip install torch==2.9.1 --index-url https://download.pytorch.org/whl/cu128
python -m pip install -r chatcode/requirements-cloud.txt
python -m pip check
python -m chatcode.data --output /checkpoints/chatcode-data
python -m chatcode.train --data /checkpoints/chatcode-data --output /checkpoints/smoke --max-steps 25
```
`/checkpoints` must be your mounted **persistent** bucket/volume or Drive directory.
Do not use a transient runtime directory as the only copy: Jobs discard local disk
when they end, and notebook sessions can be terminated. Checkpoints include optimizer,
scheduler, trainer and random state; they are saved every 25 steps.
First inspect the smoke run's GPU memory and steps/second. Estimate cost and agree
on a run limit before launching the full run. A 25-step smoke run is still paid
compute when using a paid GPU. Model/data downloads and setup also take time.
Once that succeeds, run the full pilot in a new output directory:
```bash
python -m chatcode.train --data /checkpoints/chatcode-data --output /checkpoints/pilot
```
Resume an interrupted run with the same configuration and data:
```bash
python -m chatcode.train --data /checkpoints/chatcode-data --output /checkpoints/pilot --resume-from-checkpoint /checkpoints/pilot/checkpoint-N
```
The script explicitly refuses CPU training. It writes the selected adapter,
tokenizer, provenance, training report, and candidate model card under `adapter/`.
It does not upload automatically, launch cloud hardware, or change the existing Hub
repository. There is no new dataset or GPU training run in this delivery.
## Evaluate and release
Validation loss selects a checkpoint; it does not measure whether generated programs
work. Before releasing, compare the base and candidate on the untouched derived test
split and a separate set of conversation, instruction-following, debugging, and
coding tasks. Record every prompt, output, model revision, and generation setting.
Use tests for coding correctness and inspect chat quality for regressions.
The source bundle does **not** include a sandboxed code-execution evaluator;
never automatically execute arbitrary training data or generated programs on a
machine with credentials. Use an isolated evaluation environment.
`python -m chatcode.evaluate_articles --adapter /path/to/adapter --output /persistent/article-evaluation.json`
compares base and adapter formatting/citation-ID acceptance on the four held-out
fictional behavior cases. It is cloud-GPU only and retains generated outputs for
human review. Its acceptance rate is not factual accuracy, article quality, or
Wikipedia suitability. Also evaluate real source packets and chat/code regressions.
Publish a tested candidate to a new private repository or a distinct version branch;
the old vision adapter must not be relabeled as a chat/code model. Preserve the
base revision and dataset attribution. No quality improvement is claimed yet.
For a regular cloud GPU demo, install the cloud dependencies, set
`POCKETCHAT_ADAPTER` to the compatible local adapter directory or Hub repository,
and run `python app.py`. Without an adapter it explicitly identifies itself as the
pretrained Qwen base. On Spaces set `SPACE_ID` normally via the platform.
ZeroGPU needs a separate startup/decorator integration and compatible PyTorch
environment; this ordinary GPU app is not yet a verified ZeroGPU deployment.
## Package
`python -m chatcode.package` creates `dist/pocketvision-chat-code-writing-source.zip`.
It includes only the active app, text pipeline, configuration, documentation,
and licenses. It excludes old weights, datasets, caches, environments, and credentials.
Project source remains GPL-3.0-only; original Qwen weights remain Apache-2.0.
Sources: [QLoRA with PEFT](https://huggingface.co/docs/peft/developer_guides/quantization),
[Hugging Face training Jobs](https://huggingface.co/docs/hub/jobs-training).

	# PocketVision Chat, Code & Writing — source preparation

	This version accepts text and produces text. It focuses on ordinary conversation,
	explaining code, generating functions, debugging, and source-based article writing. There is no image input,
	vision encoder, image processor, or ChartQA training in the new pipeline.

	Status: source and training recipe published privately at
	[gamedevelop/pocketvision-chat-code](https://huggingface.co/gamedevelop/pocketvision-chat-code);
	no new model trained or deployed. The original `gamedevelop/pocketvision` repository
	was deleted at the owner's request. Its local SmolVLM/ChartQA weights and ZIP backup
	remain available and are incompatible with this version.
	The local app shows an interface preview without downloading the new base weights.
	It does not silently run the old model under the new name.

	## Model and training data

	- Base: [Qwen/Qwen3-4B-Instruct-2507](https://huggingface.co/Qwen/Qwen3-4B-Instruct-2507),
	Apache-2.0, pinned to `cdbee75f17c01a7cc42f958dc650907174af0554`.
	This is an instruction-tuned text model; its model card includes chat and coding evaluations.
	- Data: [HuggingFaceTB/smoltalk](https://huggingface.co/datasets/HuggingFaceTB/smoltalk),
	pinned to `5feaf2fd3ffca7c237fc38d1861bc30365d48ffa`.
	- Planned Hub mix: 10,000 `self-oss-instruct` programming examples, 6,000
	`smol-magpie-ultra` conversation examples, 2,000 `smol-rewrite` writing/editing
	examples, and 2,000 `smol-summarize` source summarization examples. The coding source primarily covers
	Python. JavaScript and other languages need additional sources and tests.
	- Attribution for the coding subset: BigCode,
	[self-oss-instruct-sc2-exec-filter-50k](https://huggingface.co/datasets/bigcode/self-oss-instruct-sc2-exec-filter-50k),
	Open Data Commons Attribution License (ODC-BY). The source describes its responses
	as execution-validated; this project does not rerun or independently certify them.
	Smol-Magpie-Ultra, Smol-rewrite, and Smol-summarize are Apache-2.0.
	Dependency and underlying source terms still apply.
	- Separate Hub validation and test sets: 128 examples from each of four sources per split (512 each).
	All three derived splits come from the sources' official training split.
	Official test data is unused. Identical normalized opening questions are grouped
	and deduplicated globally. This does not establish novel or semantically disjoint
	data relative to the pretrained base.
	- An additional 20 project-authored examples teach the article JSON and source-review
	tasks: 12 training, 4 validation, 4 test. They use **fictional practice topics and
	reserved example.org URLs**, not real references. Both tasks for a source packet
	stay in the same split, enforced again when the trainer loads data. These small
	examples teach format; they are not a large factual corpus or evidence of quality.
	The final default counts are 20,012 train and 516 each validation/test.
	- Examples longer than 2,048 tokens are skipped, preserving complete code/answers.
	Only assistant answer and end-of-turn tokens contribute to loss.
	- One epoch of 4-bit NF4 QLoRA, rank 16, all linear projections, learning rate
	5e-5, microbatch 1, accumulation 16. This is a starting recipe, not validated
	evidence of improved model quality or a guarantee that it fits every GPU.

	## Local interface preview

	Use the existing environment and run `python app.py`. With no CUDA GPU, it displays
	the text-only interface with sending disabled. No new model weights are fetched.
	The original visual UI is preserved as `vision_app.py` for reproducibility.

	## Article writing and Wikipedia preparation

	The Articles & research tab has two tasks. Reference article produces a short,
	neutral article with a lead, sections, references and research gaps. Wikipedia
	outline and source review produces section questions and source-review tasks for
	a human editor, without article prose. Paste titles, URLs and relevant excerpts;
	the app does not fetch websites or search the web.

	The model returns structured text with source IDs. Application code rejects unknown
	or absent IDs and renders URLs only from the supplied metadata. Incomplete JSON,
	uncited paragraphs and model-added raw citation markup are rejected. This is **not
	a factual-entailment or source-reliability checker**. A valid ID can still point to
	an excerpt that does not support the claim, so every draft requires human review.
	No articles are uploaded to Wikipedia by this app.

	English Wikipedia's [current guideline](https://en.wikipedia.org/wiki/Wikipedia:Writing_articles_with_large_language_models)
	prohibits LLM-generated/re-written article prose, with limited copyediting and
	translation exceptions. The reference-article task is for other writing contexts;
	use the planning task for Wikipedia assistance.

	The [Wikimedia Wikipedia dataset](https://huggingface.co/datasets/wikimedia/wikipedia)
	contains cleaned article text with references removed. It is not in the default
	mix: its raw text is not a ready-made source-grounded instruction dataset. Any later
	use needs article-level provenance and the corpus's CC-BY-SA/GFDL terms recorded.
	Wikipedia articles should not themselves be cited as evidence for new Wikipedia
	article prose; consult their original reliable sources.

	## Colab notebook

	[cloud_training.ipynb](cloud_training.ipynb) is a cloud training walkthrough.
	Upload it to Colab, select a free GPU if available, add your own existing read token
	in Colab Secrets, and follow its setup. It downloads an immutable snapshot of source
	files and keeps checkpoints in your chosen persistent Drive folder.

	The saved notebook was structurally checked, its Python cells compiled, and all
	cells executed only with `DRY_RUN=True`. `nbformat` and a Colab GPU runtime are
	not available in this local environment; a full cloud run remains unverified.
	Set `DRY_RUN=False` for cloud setup and separately enable `RUN_TRAINING` to train.
	No subscription, GPU job or model publication is started by this preparation.

	## Train on a remote Linux GPU

	Use a Colab GPU runtime or a Hugging Face Job. A 24 GB GPU is a reasonable starting
	target, subject to an actual memory smoke test. None has been rented or started.
	Run these commands on the cloud machine, from the unpacked source directory:

	```bash
	python -m pip install torch==2.9.1 --index-url https://download.pytorch.org/whl/cu128
	python -m pip install -r chatcode/requirements-cloud.txt
	python -m pip check
	python -m chatcode.data --output /checkpoints/chatcode-data
	python -m chatcode.train --data /checkpoints/chatcode-data --output /checkpoints/smoke --max-steps 25
	```

	`/checkpoints` must be your mounted persistent bucket/volume or Drive directory.
	Do not use a transient runtime directory as the only copy: Jobs discard local disk
	when they end, and notebook sessions can be terminated. Checkpoints include optimizer,
	scheduler, trainer and random state; they are saved every 25 steps.

	First inspect the smoke run's GPU memory and steps/second. Estimate cost and agree
	on a run limit before launching the full run. A 25-step smoke run is still paid
	compute when using a paid GPU. Model/data downloads and setup also take time.
	Once that succeeds, run the full pilot in a new output directory:

	```bash
	python -m chatcode.train --data /checkpoints/chatcode-data --output /checkpoints/pilot
	```

	Resume an interrupted run with the same configuration and data:

	```bash
	python -m chatcode.train --data /checkpoints/chatcode-data --output /checkpoints/pilot --resume-from-checkpoint /checkpoints/pilot/checkpoint-N
	```

	The script explicitly refuses CPU training. It writes the selected adapter,
	tokenizer, provenance, training report, and candidate model card under `adapter/`.
	It does not upload automatically, launch cloud hardware, or change the existing Hub
	repository. There is no new dataset or GPU training run in this delivery.

	## Evaluate and release

	Validation loss selects a checkpoint; it does not measure whether generated programs
	work. Before releasing, compare the base and candidate on the untouched derived test
	split and a separate set of conversation, instruction-following, debugging, and
	coding tasks. Record every prompt, output, model revision, and generation setting.
	Use tests for coding correctness and inspect chat quality for regressions.
	The source bundle does not include a sandboxed code-execution evaluator;
	never automatically execute arbitrary training data or generated programs on a
	machine with credentials. Use an isolated evaluation environment.

	`python -m chatcode.evaluate_articles --adapter /path/to/adapter --output /persistent/article-evaluation.json`
	compares base and adapter formatting/citation-ID acceptance on the four held-out
	fictional behavior cases. It is cloud-GPU only and retains generated outputs for
	human review. Its acceptance rate is not factual accuracy, article quality, or
	Wikipedia suitability. Also evaluate real source packets and chat/code regressions.

	Publish a tested candidate to a new private repository or a distinct version branch;
	the old vision adapter must not be relabeled as a chat/code model. Preserve the
	base revision and dataset attribution. No quality improvement is claimed yet.

	For a regular cloud GPU demo, install the cloud dependencies, set
	`POCKETCHAT_ADAPTER` to the compatible local adapter directory or Hub repository,
	and run `python app.py`. Without an adapter it explicitly identifies itself as the
	pretrained Qwen base. On Spaces set `SPACE_ID` normally via the platform.
	ZeroGPU needs a separate startup/decorator integration and compatible PyTorch
	environment; this ordinary GPU app is not yet a verified ZeroGPU deployment.

	## Package

	`python -m chatcode.package` creates `dist/pocketvision-chat-code-writing-source.zip`.
	It includes only the active app, text pipeline, configuration, documentation,
	and licenses. It excludes old weights, datasets, caches, environments, and credentials.
	Project source remains GPL-3.0-only; original Qwen weights remain Apache-2.0.

	Sources: [QLoRA with PEFT](https://huggingface.co/docs/peft/developer_guides/quantization),
	[Hugging Face training Jobs](https://huggingface.co/docs/hub/jobs-training).