gamedevelop commited on
Commit
ad37fae
·
verified ·
1 Parent(s): 64d9242

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +19 -7
README.md CHANGED
@@ -9,11 +9,11 @@ tags:
9
  - work-in-progress
10
  ---
11
 
12
- # PocketVision Chat & Code
13
 
14
  **Status: source project and cloud training recipe only. No custom model weights or adapter have been trained or uploaded for this version.**
15
 
16
- This project focuses on text conversation and programming: explaining code, generating functions, and debugging. Image upload and image understanding have been removed from the active application.
17
 
18
  This repository is not yet a downloadable fine-tuned model or a running chat service. The supplied app uses the pretrained base until a compatible trained adapter is supplied. On a computer without a CUDA GPU it displays an interface preview and does not download the larger base weights.
19
 
@@ -28,8 +28,11 @@ The Qwen weights are supplied by the original publisher and are not re-uploaded
28
 
29
  The first proposed cloud experiment uses 20,000 examples from [HuggingFaceTB/smoltalk](https://huggingface.co/datasets/HuggingFaceTB/smoltalk), pinned to `5feaf2fd3ffca7c237fc38d1861bc30365d48ffa`:
30
 
31
- - 60% `self-oss-instruct`: programming, primarily Python. Original source: BigCode, [self-oss-instruct-sc2-exec-filter-50k](https://huggingface.co/datasets/bigcode/self-oss-instruct-sc2-exec-filter-50k), ODC-BY.
32
- - 40% `smol-magpie-ultra`: conversation and instruction following, Apache-2.0.
 
 
 
33
  - Separate derived validation and test splits; duplicate opening prompts are removed.
34
  - Assistant-only loss; 4-bit NF4 QLoRA, rank 16; one epoch as a starting experiment.
35
 
@@ -40,7 +43,16 @@ These are planned settings, not a description of completed training. Synthetic a
40
  - `app.py` and `chat.py`: text-only web and CLI entry points.
41
  - `chatcode/`: runtime, data preparation, cloud trainer, tests, configuration, dependencies, and the detailed guide.
42
  - [Cloud training guide](chatcode/README.md).
43
- - [Download the complete source bundle](pocketvision-chat-code-source.zip).
 
 
 
 
 
 
 
 
 
44
 
45
  Run the following on a remote Linux CUDA GPU, from the source directory:
46
 
@@ -56,10 +68,10 @@ Use mounted persistent storage for `/checkpoints`. Inspect the smoke run before
56
 
57
  ## Verification and limits
58
 
59
- Five small offline checks passed: assistant-only labels and padding, text-only record validation, CPU refusal before model download, data manifest integrity, and a forward/backward check with a tiny randomly initialized Qwen architecture. These checks do not train or evaluate the selected 4B model.
60
 
61
  The cloud dependencies and full GPU training run still need a cloud smoke test. No chat-quality or programming-accuracy improvement is claimed. A trained candidate needs separate answer review and isolated code tests before release. This repository does not include a sandboxed code-execution evaluator.
62
 
63
  ## Licenses and attribution
64
 
65
- Project source: GPL-3.0-only (`LICENSE`). Original Qwen base: Apache-2.0 (`LICENSE-BASE-MODEL.txt`). Dataset and dependency licenses remain applicable. See the training guide and configuration for provenance. No pretrained base weights, raw datasets, credentials, or original vision adapter are included.
 
9
  - work-in-progress
10
  ---
11
 
12
+ # PocketVision Chat, Code & Writing
13
 
14
  **Status: source project and cloud training recipe only. No custom model weights or adapter have been trained or uploaded for this version.**
15
 
16
+ This project focuses on conversation, programming and source-based writing: explaining code, generating functions, debugging, neutral reference articles, and research outlines. Image upload and image understanding have been removed from the active application.
17
 
18
  This repository is not yet a downloadable fine-tuned model or a running chat service. The supplied app uses the pretrained base until a compatible trained adapter is supplied. On a computer without a CUDA GPU it displays an interface preview and does not download the larger base weights.
19
 
 
28
 
29
  The first proposed cloud experiment uses 20,000 examples from [HuggingFaceTB/smoltalk](https://huggingface.co/datasets/HuggingFaceTB/smoltalk), pinned to `5feaf2fd3ffca7c237fc38d1861bc30365d48ffa`:
30
 
31
+ - 50% `self-oss-instruct`: programming, primarily Python. Original source: BigCode, [self-oss-instruct-sc2-exec-filter-50k](https://huggingface.co/datasets/bigcode/self-oss-instruct-sc2-exec-filter-50k), ODC-BY.
32
+ - 30% `smol-magpie-ultra`: conversation and instruction following, Apache-2.0.
33
+ - 10% `smol-rewrite`: writing and editing, Apache-2.0.
34
+ - 10% `smol-summarize`: source summarization, Apache-2.0.
35
+ - 20 additional authored fictional article-format examples (12 train, 4 validation, 4 test); source packets stay in one split.
36
  - Separate derived validation and test splits; duplicate opening prompts are removed.
37
  - Assistant-only loss; 4-bit NF4 QLoRA, rank 16; one epoch as a starting experiment.
38
 
 
43
  - `app.py` and `chat.py`: text-only web and CLI entry points.
44
  - `chatcode/`: runtime, data preparation, cloud trainer, tests, configuration, dependencies, and the detailed guide.
45
  - [Cloud training guide](chatcode/README.md).
46
+ - [Cloud training notebook](chatcode/cloud_training.ipynb) (saved outputs are a local dry run only).
47
+ - The existing `pocketvision-chat-code-source.zip` is the original source snapshot before writing support. Use the current source files or the notebook's pinned snapshot download for this version.
48
+
49
+ ## Articles and Wikipedia preparation
50
+
51
+ Reference article mode uses supplied source excerpts to draft a short neutral article with a lead, sections, references and research gaps. It rejects missing/unknown citation IDs and builds reference URLs only from supplied metadata. It does not fetch source URLs or verify factual entailment; each claim still requires human source review.
52
+
53
+ Wikipedia outline mode returns questions and source-review tasks for a human editor. English Wikipedia's [current guideline](https://en.wikipedia.org/wiki/Wikipedia:Writing_articles_with_large_language_models) restricts LLM-generated article prose, so reference drafts are intended for other writing contexts. Nothing is automatically published to Wikipedia.
54
+
55
+ The Wikimedia Wikipedia corpus is not included in the default mix: its cleaned articles have references removed and are not source-grounded instruction examples. The small authored examples use fictional practice topics and reserved URLs, not genuine research references.
56
 
57
  Run the following on a remote Linux CUDA GPU, from the source directory:
58
 
 
68
 
69
  ## Verification and limits
70
 
71
+ Offline checks cover assistant-only labels, source-packet separation, malformed and invented references, text-only validation, CPU refusal before model download, data integrity, and a forward/backward check with a tiny randomly initialized Qwen architecture. These checks do not train or evaluate the selected 4B model. The Colab notebook's Python cells were executed only in dry-run mode; its full GPU path is unverified.
72
 
73
  The cloud dependencies and full GPU training run still need a cloud smoke test. No chat-quality or programming-accuracy improvement is claimed. A trained candidate needs separate answer review and isolated code tests before release. This repository does not include a sandboxed code-execution evaluator.
74
 
75
  ## Licenses and attribution
76
 
77
+ Project source: GPL-3.0-only (`LICENSE`). Original Qwen base: Apache-2.0 (`LICENSE-BASE-MODEL.txt`). Dataset and dependency licenses remain applicable. See the training guide and configuration for provenance. No pretrained base weights, raw datasets, credentials, or original vision adapter are included.