Update README.md
Browse files
README.md
CHANGED
|
@@ -9,11 +9,11 @@ tags:
|
|
| 9 |
- work-in-progress
|
| 10 |
---
|
| 11 |
|
| 12 |
-
# PocketVision Chat &
|
| 13 |
|
| 14 |
**Status: source project and cloud training recipe only. No custom model weights or adapter have been trained or uploaded for this version.**
|
| 15 |
|
| 16 |
-
This project focuses on
|
| 17 |
|
| 18 |
This repository is not yet a downloadable fine-tuned model or a running chat service. The supplied app uses the pretrained base until a compatible trained adapter is supplied. On a computer without a CUDA GPU it displays an interface preview and does not download the larger base weights.
|
| 19 |
|
|
@@ -28,8 +28,11 @@ The Qwen weights are supplied by the original publisher and are not re-uploaded
|
|
| 28 |
|
| 29 |
The first proposed cloud experiment uses 20,000 examples from [HuggingFaceTB/smoltalk](https://huggingface.co/datasets/HuggingFaceTB/smoltalk), pinned to `5feaf2fd3ffca7c237fc38d1861bc30365d48ffa`:
|
| 30 |
|
| 31 |
-
-
|
| 32 |
-
-
|
|
|
|
|
|
|
|
|
|
| 33 |
- Separate derived validation and test splits; duplicate opening prompts are removed.
|
| 34 |
- Assistant-only loss; 4-bit NF4 QLoRA, rank 16; one epoch as a starting experiment.
|
| 35 |
|
|
@@ -40,7 +43,16 @@ These are planned settings, not a description of completed training. Synthetic a
|
|
| 40 |
- `app.py` and `chat.py`: text-only web and CLI entry points.
|
| 41 |
- `chatcode/`: runtime, data preparation, cloud trainer, tests, configuration, dependencies, and the detailed guide.
|
| 42 |
- [Cloud training guide](chatcode/README.md).
|
| 43 |
-
- [
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 44 |
|
| 45 |
Run the following on a remote Linux CUDA GPU, from the source directory:
|
| 46 |
|
|
@@ -56,10 +68,10 @@ Use mounted persistent storage for `/checkpoints`. Inspect the smoke run before
|
|
| 56 |
|
| 57 |
## Verification and limits
|
| 58 |
|
| 59 |
-
|
| 60 |
|
| 61 |
The cloud dependencies and full GPU training run still need a cloud smoke test. No chat-quality or programming-accuracy improvement is claimed. A trained candidate needs separate answer review and isolated code tests before release. This repository does not include a sandboxed code-execution evaluator.
|
| 62 |
|
| 63 |
## Licenses and attribution
|
| 64 |
|
| 65 |
-
Project source: GPL-3.0-only (`LICENSE`). Original Qwen base: Apache-2.0 (`LICENSE-BASE-MODEL.txt`). Dataset and dependency licenses remain applicable. See the training guide and configuration for provenance. No pretrained base weights, raw datasets, credentials, or original vision adapter are included.
|
|
|
|
| 9 |
- work-in-progress
|
| 10 |
---
|
| 11 |
|
| 12 |
+
# PocketVision Chat, Code & Writing
|
| 13 |
|
| 14 |
**Status: source project and cloud training recipe only. No custom model weights or adapter have been trained or uploaded for this version.**
|
| 15 |
|
| 16 |
+
This project focuses on conversation, programming and source-based writing: explaining code, generating functions, debugging, neutral reference articles, and research outlines. Image upload and image understanding have been removed from the active application.
|
| 17 |
|
| 18 |
This repository is not yet a downloadable fine-tuned model or a running chat service. The supplied app uses the pretrained base until a compatible trained adapter is supplied. On a computer without a CUDA GPU it displays an interface preview and does not download the larger base weights.
|
| 19 |
|
|
|
|
| 28 |
|
| 29 |
The first proposed cloud experiment uses 20,000 examples from [HuggingFaceTB/smoltalk](https://huggingface.co/datasets/HuggingFaceTB/smoltalk), pinned to `5feaf2fd3ffca7c237fc38d1861bc30365d48ffa`:
|
| 30 |
|
| 31 |
+
- 50% `self-oss-instruct`: programming, primarily Python. Original source: BigCode, [self-oss-instruct-sc2-exec-filter-50k](https://huggingface.co/datasets/bigcode/self-oss-instruct-sc2-exec-filter-50k), ODC-BY.
|
| 32 |
+
- 30% `smol-magpie-ultra`: conversation and instruction following, Apache-2.0.
|
| 33 |
+
- 10% `smol-rewrite`: writing and editing, Apache-2.0.
|
| 34 |
+
- 10% `smol-summarize`: source summarization, Apache-2.0.
|
| 35 |
+
- 20 additional authored fictional article-format examples (12 train, 4 validation, 4 test); source packets stay in one split.
|
| 36 |
- Separate derived validation and test splits; duplicate opening prompts are removed.
|
| 37 |
- Assistant-only loss; 4-bit NF4 QLoRA, rank 16; one epoch as a starting experiment.
|
| 38 |
|
|
|
|
| 43 |
- `app.py` and `chat.py`: text-only web and CLI entry points.
|
| 44 |
- `chatcode/`: runtime, data preparation, cloud trainer, tests, configuration, dependencies, and the detailed guide.
|
| 45 |
- [Cloud training guide](chatcode/README.md).
|
| 46 |
+
- [Cloud training notebook](chatcode/cloud_training.ipynb) (saved outputs are a local dry run only).
|
| 47 |
+
- The existing `pocketvision-chat-code-source.zip` is the original source snapshot before writing support. Use the current source files or the notebook's pinned snapshot download for this version.
|
| 48 |
+
|
| 49 |
+
## Articles and Wikipedia preparation
|
| 50 |
+
|
| 51 |
+
Reference article mode uses supplied source excerpts to draft a short neutral article with a lead, sections, references and research gaps. It rejects missing/unknown citation IDs and builds reference URLs only from supplied metadata. It does not fetch source URLs or verify factual entailment; each claim still requires human source review.
|
| 52 |
+
|
| 53 |
+
Wikipedia outline mode returns questions and source-review tasks for a human editor. English Wikipedia's [current guideline](https://en.wikipedia.org/wiki/Wikipedia:Writing_articles_with_large_language_models) restricts LLM-generated article prose, so reference drafts are intended for other writing contexts. Nothing is automatically published to Wikipedia.
|
| 54 |
+
|
| 55 |
+
The Wikimedia Wikipedia corpus is not included in the default mix: its cleaned articles have references removed and are not source-grounded instruction examples. The small authored examples use fictional practice topics and reserved URLs, not genuine research references.
|
| 56 |
|
| 57 |
Run the following on a remote Linux CUDA GPU, from the source directory:
|
| 58 |
|
|
|
|
| 68 |
|
| 69 |
## Verification and limits
|
| 70 |
|
| 71 |
+
Offline checks cover assistant-only labels, source-packet separation, malformed and invented references, text-only validation, CPU refusal before model download, data integrity, and a forward/backward check with a tiny randomly initialized Qwen architecture. These checks do not train or evaluate the selected 4B model. The Colab notebook's Python cells were executed only in dry-run mode; its full GPU path is unverified.
|
| 72 |
|
| 73 |
The cloud dependencies and full GPU training run still need a cloud smoke test. No chat-quality or programming-accuracy improvement is claimed. A trained candidate needs separate answer review and isolated code tests before release. This repository does not include a sandboxed code-execution evaluator.
|
| 74 |
|
| 75 |
## Licenses and attribution
|
| 76 |
|
| 77 |
+
Project source: GPL-3.0-only (`LICENSE`). Original Qwen base: Apache-2.0 (`LICENSE-BASE-MODEL.txt`). Dataset and dependency licenses remain applicable. See the training guide and configuration for provenance. No pretrained base weights, raw datasets, credentials, or original vision adapter are included.
|