File size: 11,069 Bytes
5b8b17f dc41909 5b8b17f | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 | {
"nbformat": 4,
"nbformat_minor": 5,
"metadata": {
"kernelspec": {
"display_name": "Python 3",
"language": "python",
"name": "python3"
},
"language_info": {
"name": "python",
"version": "3.12"
},
"pocketvision_validation": "local_dry_run_only_no_cloud_execution"
},
"cells": [
{
"cell_type": "markdown",
"id": "cell-00",
"metadata": {},
"source": "# PocketVision: cloud training for chat, code and writing\n\n## Goal\nRun a small QLoRA smoke experiment on a **Colab cloud GPU**, then inspect a candidate before scaling up.\nThe PC only displays this notebook. No trained 4B weights are included.\n\n**Validation status:** the saved outputs show a local dry run only. Cloud installation, source authentication,\nGPU memory, training, and model quality remain untested. Free Colab GPU availability is not guaranteed.\nChoose a free runtime yourself; this notebook does not purchase compute or alter subscriptions.\n\nEnglish Wikipedia restricts AI-generated article prose. Use the reference-article task for other contexts,\nand the outline/source-review task for Wikipedia preparation.\n[Current guideline](https://en.wikipedia.org/wiki/Wikipedia:Writing_articles_with_large_language_models).\n"
},
{
"cell_type": "markdown",
"id": "cell-01",
"metadata": {},
"source": "## Setup\n1. Upload this notebook to Colab and select a GPU runtime.\n2. Set `DRY_RUN = False` below. Set `RUN_TRAINING = True` only when ready to use the allocated GPU.\n3. Store an existing Hugging Face **read token** in Colab Secrets as `HF_TOKEN` and grant this notebook access.\n The source repository is private. Do not paste a token into a code cell or chat.\n4. The Drive step asks you to mount Google Drive. This grants notebook code access to the mounted files;\n only the named PocketVision folder is used here for checkpoints.\n"
},
{
"cell_type": "code",
"id": "cell-02",
"metadata": {},
"source": "from pathlib import Path\nimport json\nimport os\nimport subprocess\nimport sys\n\nDRY_RUN = True\nRUN_TRAINING = False\nSOURCE_REPO = \"gamedevelop/pocketvision-chat-code\"\nSOURCE_REVISION = None # None resolves and records one immutable commit before downloading.\nPILOT_ROWS = 200 # Dependency/memory smoke test. Use 20000 only after reviewing the smoke run.\nHOLDOUT_PER_SOURCE = 2\nMAX_STEPS = 25\nRUN_NAME = \"writing-smoke-01\" # Change for a fresh run; existing output is never overwritten.\nRESUME_CHECKPOINT = None # Or an existing checkpoint-N directory from the same run.\ncloud_python = Path(\"/content/pocketvision-venv/bin/python\")\nsource_dir = None\nrun_dir = None\nprint(\"Dry-run validation only; no cloud operations.\" if DRY_RUN else \"Cloud setup enabled; training still needs RUN_TRAINING=True.\")\n",
"execution_count": 1,
"outputs": [
{
"output_type": "stream",
"name": "stdout",
"text": "Dry-run validation only; no cloud operations.\n"
}
]
},
{
"cell_type": "markdown",
"id": "cell-03",
"metadata": {},
"source": "### 1. Install an isolated cloud environment\nInstallation uses a separate environment, preserving the notebook kernel's packages.\nThe next check requires a CUDA GPU before any model weights are loaded.\n"
},
{
"cell_type": "code",
"id": "cell-04",
"metadata": {},
"source": "if not DRY_RUN:\n subprocess.run([sys.executable, \"-m\", \"venv\", str(cloud_python.parent.parent)], check=True)\n subprocess.run([str(cloud_python), \"-m\", \"pip\", \"install\", \"torch==2.9.1\",\n \"--index-url\", \"https://download.pytorch.org/whl/cu128\"], check=True)\n subprocess.run([sys.executable, \"-m\", \"pip\", \"install\", \"huggingface-hub==0.36.2\"], check=True)\n subprocess.run([str(cloud_python), \"-c\",\n \"import torch; assert torch.cuda.is_available(), 'Select a Colab GPU runtime'; print(torch.cuda.get_device_name(0))\"], check=True)\nelse:\n print(\"Skipped: dependency installation and GPU allocation checks.\")\n",
"execution_count": 2,
"outputs": [
{
"output_type": "stream",
"name": "stdout",
"text": "Skipped: dependency installation and GPU allocation checks.\n"
}
]
},
{
"cell_type": "markdown",
"id": "cell-05",
"metadata": {},
"source": "### 2. Download only the versioned source\nThe commit is resolved once and logged; pretrained weights and source ZIPs are excluded.\nThe model and datasets have their own pinned revisions in the configuration.\n"
},
{
"cell_type": "code",
"id": "cell-06",
"metadata": {},
"source": "if not DRY_RUN:\n from google.colab import userdata\n from huggingface_hub import HfApi, snapshot_download\n os.environ[\"HF_TOKEN\"] = userdata.get(\"HF_TOKEN\")\n revision = HfApi().model_info(SOURCE_REPO, revision=SOURCE_REVISION or \"main\").sha\n source_dir = Path(snapshot_download(SOURCE_REPO, revision=revision, allow_patterns=[\n \"app.py\", \"chat.py\", \"LICENSE*\", \"chatcode/*.py\", \"chatcode/*.json\", \"chatcode/*.txt\", \"chatcode/*.md\"]))\n assert (source_dir / \"chatcode/articles.py\").is_file(), \"This source commit predates article support.\"\n subprocess.run([str(cloud_python), \"-m\", \"pip\", \"install\", \"-r\",\n str(source_dir / \"chatcode/requirements-cloud.txt\")], check=True)\n subprocess.run([str(cloud_python), \"-m\", \"pip\", \"check\"], check=True)\n print(\"Source commit:\", revision)\nelse:\n print(\"Skipped: private Hub download and authentication. No token requested.\")\n",
"execution_count": 3,
"outputs": [
{
"output_type": "stream",
"name": "stdout",
"text": "Skipped: private Hub download and authentication. No token requested.\n"
}
]
},
{
"cell_type": "markdown",
"id": "cell-07",
"metadata": {},
"source": "### 3. Use persistent storage\nColab's runtime disk is temporary. Drive contains the data manifest, optimizer checkpoints and candidate.\nOnly this new run folder is created; no existing files are deleted.\n"
},
{
"cell_type": "code",
"id": "cell-08",
"metadata": {},
"source": "if not DRY_RUN:\n from google.colab import drive\n drive.mount(\"/content/drive\")\n checkpoint_root = Path(\"/content/drive/MyDrive/PocketVision-writing\")\n run_dir = checkpoint_root / RUN_NAME\n if run_dir.exists() and any(run_dir.iterdir()) and not RESUME_CHECKPOINT:\n raise FileExistsError(\"Choose a fresh RUN_NAME or explicitly resume an existing checkpoint.\")\n run_dir.mkdir(parents=True, exist_ok=True)\n (run_dir / \"run-info.json\").write_text(json.dumps({\"source_repo\": SOURCE_REPO,\n \"source_revision\": revision, \"pilot_rows\": PILOT_ROWS, \"max_steps\": MAX_STEPS}, indent=2))\n print(\"Persistent run directory:\", run_dir)\nelse:\n print(\"Skipped: Drive authorization and persistent cloud writes.\")\n",
"execution_count": 4,
"outputs": [
{
"output_type": "stream",
"name": "stdout",
"text": "Skipped: Drive authorization and persistent cloud writes.\n"
}
]
},
{
"cell_type": "markdown",
"id": "cell-09",
"metadata": {},
"source": "## Steps\n### 4. Prepare data and run the capped experiment\nThis cell streams selected chat, coding, rewriting and summarization data, adds the small authored behavior\npackets, and runs 25 optimizer steps. It may consume significant GPU time even on a small sample.\nThe full 20,000-example experiment is a separate decision after measuring steps/second and memory.\n"
},
{
"cell_type": "code",
"id": "cell-10",
"metadata": {},
"source": "if not DRY_RUN and RUN_TRAINING:\n data_dir = run_dir / \"data\"\n if not (data_dir / \"manifest.json\").is_file():\n subprocess.run([str(cloud_python), \"-m\", \"chatcode.data\", \"--output\", str(data_dir),\n \"--train\", str(PILOT_ROWS), \"--holdout-per-source\", str(HOLDOUT_PER_SOURCE)], cwd=source_dir, check=True)\n command = [str(cloud_python), \"-m\", \"chatcode.train\", \"--data\", str(data_dir),\n \"--output\", str(run_dir / \"training\"), \"--max-steps\", str(MAX_STEPS)]\n if RESUME_CHECKPOINT:\n command.extend([\"--resume-from-checkpoint\", str(RESUME_CHECKPOINT)])\n subprocess.run(command, cwd=source_dir, check=True)\nelse:\n print(\"Skipped: dataset preparation and weight training. No model weights downloaded.\")\n",
"execution_count": 5,
"outputs": [
{
"output_type": "stream",
"name": "stdout",
"text": "Skipped: dataset preparation and weight training. No model weights downloaded.\n"
}
]
},
{
"cell_type": "markdown",
"id": "cell-11",
"metadata": {},
"source": "## Checks\n### 5. Compare base and candidate\nThe automated article check measures JSON/citation-ID validity on four held-out fictional cases.\nIt does **not** measure factual accuracy, source reliability, or Wikipedia suitability. Inspect all saved\noutputs, and separately test chat and code regressions. Do not execute generated programs outside a sandbox.\n"
},
{
"cell_type": "code",
"id": "cell-12",
"metadata": {},
"source": "if not DRY_RUN and RUN_TRAINING:\n candidate_dir = run_dir / \"training/adapter\"\n subprocess.run([str(cloud_python), \"-m\", \"chatcode.evaluate_articles\", \"--adapter\", str(candidate_dir),\n \"--output\", str(run_dir / \"article-evaluation.json\")], cwd=source_dir, check=True)\n print(\"Candidate and evaluation saved under:\", run_dir)\nelse:\n print(\"Skipped: model evaluation. No accuracy or quality result is claimed.\")\n",
"execution_count": 6,
"outputs": [
{
"output_type": "stream",
"name": "stdout",
"text": "Skipped: model evaluation. No accuracy or quality result is claimed.\n"
}
]
},
{
"cell_type": "markdown",
"id": "cell-13",
"metadata": {},
"source": "## Next Steps\nReview the cloud smoke run before raising `PILOT_ROWS`, holdout sizes or run duration. Training data changes\nrequire a new run directory; do not resume a checkpoint against different data.\nThe source recipe alone does not improve weights. Compare factual support and neutrality with the base,\nand check programming and conversation regressions before publishing a candidate.\nNo upload or Wikipedia publication is performed by this notebook. A later Hub upload needs a separate\nwrite-scoped credential and a reviewed candidate. Do not share notebook outputs containing credentials.\n"
}
]
} |