{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "# 🕵️‍♂️ Global Syndicate Taskforce: RL Training Loop\n", "\n", "**Training a model to overcome 'Collective Delusion' using OpenEnv, TRL (GRPO), and Unsloth.**\n", "\n", "This notebook represents the core pipeline of our project. Current LLMs suffer from 'sycophancy'—if another agent tells them something, they believe it. In our Anti-Money Laundering environment, an LLM must learn to stop trusting hallucinating Tier 1 Analysts, interrogate Bank Liaisons with correct policy mandates, and construct an airtight evidence chain for the Legal Officer.\n", "\n", "### 🏆 Our Stack:\n", "- **Environment:** OpenEnv (FastAPI)\n", "- **Base Model:** `unsloth/Qwen2.5-3B-Instruct`\n", "- **RL Framework:** TRL (GRPO - Group Relative Policy Optimization)\n", "- **Compiler:** Unsloth (for extremely fast, memory-efficient LoRA adapters)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## ⚙️ 1. Local Jupyter / Mac Setup\n", "This notebook is now configured for **local Jupyter on Mac** (no Colab required).\n", "\n", "It assumes you are running from the repository root. We install the minimal dependencies for the Mac training path (`train_grpo_mac.py`) and plotting artifacts for submission." ] }, { "cell_type": "code", "execution_count": 1, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Using project root: /Users/md.kamranalam/Programming/hackathon/global_syndicate_taskforce_env\n", "/Users/md.kamranalam/Programming/hackathon/global_syndicate_taskforce_env/.venv/bin/python: No module named pip\n", "Note: you may need to restart the kernel to use updated packages.\n", "/Users/md.kamranalam/Programming/hackathon/global_syndicate_taskforce_env/.venv/bin/python: No module named pip\n", "Note: you may need to restart the kernel to use updated packages.\n", "✅ Environment setup complete.\n" ] } ], "source": [ "from pathlib import Path\n", "import os\n", "\n", "# Auto-detect repository root from current directory or its parents\n", "cwd = Path.cwd().resolve()\n", "candidates = [cwd] + list(cwd.parents)\n", "project_root = None\n", "for p in candidates:\n", " if (p / \"server\").exists() and (p / \"train_grpo_mac.py\").exists() and (p / \"pyproject.toml\").exists():\n", " project_root = p\n", " break\n", "\n", "if project_root is None:\n", " raise FileNotFoundError(\"Could not locate project root. Open notebook inside the project tree.\")\n", "\n", "os.chdir(project_root)\n", "print(f\"Using project root: {project_root}\")\n", "\n", "# Local Jupyter dependency install (Mac-friendly)\n", "%pip install -q -U pip\n", "%pip install -q torch transformers datasets trl peft requests openenv-core fastapi uvicorn matplotlib\n", "\n", "print(\"✅ Environment setup complete.\")" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## 🌍 2. Booting the OpenEnv API Server\n", "The environment is decoupled from training and exposed through FastAPI.\n", "\n", "This cell starts `uvicorn` in the current Jupyter session and verifies `/health` so we know training can interact with the environment." ] }, { "cell_type": "code", "execution_count": 2, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Starting OpenEnv server...\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "INFO: Started server process [62942]\n", "INFO: Waiting for application startup.\n", "INFO: Application startup complete.\n", "INFO: Uvicorn running on http://0.0.0.0:8000 (Press CTRL+C to quit)\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Testing /health endpoint...\n", "INFO: 127.0.0.1:58733 - \"GET /health HTTP/1.1\" 200 OK\n", "Health status: 200\n", "Testing /step endpoint...\n", "INFO: 127.0.0.1:58734 - \"POST /step HTTP/1.1\" 200 OK\n", "Step status: 200\n", "Step response: {'observation': {'transaction_id': 'TX-1024', 'amount': 1200.0, 'verified_facts': [], 'system_alerts': 'Analyst Report: Transaction appears CLEAN_VERIFIED.'}, 'reward': -0.05, 'done': False}\n", "✅ Server is ready for training.\n" ] } ], "source": [ "import subprocess\n", "import time\n", "import requests\n", "import sys\n", "\n", "# Best effort cleanup of previous server\n", "subprocess.run([\"pkill\", \"-f\", \"uvicorn\"], check=False)\n", "\n", "print(\"Starting OpenEnv server...\")\n", "server_process = subprocess.Popen(\n", " [sys.executable, \"-m\", \"uvicorn\", \"server.app:app\", \"--host\", \"0.0.0.0\", \"--port\", \"8000\"]\n", ")\n", "\n", "time.sleep(4)\n", "\n", "print(\"Testing /health endpoint...\")\n", "health = requests.get(\"http://127.0.0.1:8000/health\", timeout=5)\n", "print(\"Health status:\", health.status_code)\n", "\n", "print(\"Testing /step endpoint...\")\n", "step = requests.post(\n", " \"http://127.0.0.1:8000/step\",\n", " json={\"action\": {\"target_actor\": \"Tier_1_Analyst\", \"operation\": \"fetch_triage_report\"}},\n", " timeout=5,\n", ")\n", "print(\"Step status:\", step.status_code)\n", "print(\"Step response:\", step.json())\n", "\n", "assert health.status_code == 200 and step.status_code == 200, \"Server failed smoke test\"\n", "print(\"✅ Server is ready for training.\")\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## 🧠 3. Execute the GRPO Training Loop (Mac/Jupyter Path)\n", "For local Mac testing, run the lightweight training script first (`train_grpo_mac.py`).\n", "\n", "This gives you a reproducible baseline run, adapter artifacts, and trainer logs you can plot.\n", "\n", "Once you have access to stronger GPU, run `train_grpo_unsloth.py` for full-scale training." ] }, { "cell_type": "code", "execution_count": 3, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Loading Tiny Model for Mac...\n", "[transformers] `torch_dtype` is deprecated! Use `dtype` instead!\n", "Loading weights: 100%|███████████████████████| 290/290 [00:01<00:00, 285.40it/s]\n", "Starting GRPO Training on Mac...\n", "[transformers] The tokenizer has new PAD/BOS/EOS tokens that differ from the model config and generation config. The model config and generation config were aligned accordingly, being updated with the tokenizer's values. Updated tokens: {'bos_token_id': None, 'pad_token_id': 151643}.\n", " 0%| | 0/20 [00:00" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAxYAAAG4CAYAAADYN3EQAAAAOnRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjEwLjksIGh0dHBzOi8vbWF0cGxvdGxpYi5vcmcvJkbTWQAAAAlwSFlzAAAPYQAAD2EBqD+naQAAPHpJREFUeJzt3Qd8VFX+//9PIJIAUpUWepOioAIugiC4VEWp0kQBRVxYdWmKIAqiFCmylF0FG6BgQ6q4UqQIsggoZQUxUqWEEoWQ0CGZ3+Nzvv87/0mf5CRMbvJ6Ph5DmDt3Zu6cOTe573vKDfJ4PB4BAAAAAAu5bJ4MAAAAAAQLAAAAABmCFgsAAAAA1ggWAAAAAKwRLAAAAABYI1gAAAAAsEawAAAAAGCNYAEAAADAGsECAAAAgDWCBQDghjl8+LAEBQXJnDlzcnSpHz16VEJDQ2XTpk2B3pQs79q1a1K2bFl5++23A70pAFJBsADgaocOHZLnnntObrvtNsmXL5+51axZU5599ln53//+F2/d1157zRzUOrebbrpJKlSoIP/4xz8kKioq0WvrY77rFy9eXBo3biyLFy9OtK7H45GPP/5Y7r//filcuLDZjlq1asnrr78uFy5c8OuzpHX7coJTp07JCy+8INWrVzdlmj9/fqlbt66MGTPG1WWi9aJ+/fpy3333eZf17t3bfO8FCxaUS5cuJXrOvn37vHVj8uTJN3R7mzZtGq9u5s2bV2rXri1Tp06VuLi4TH1v3Q8GDx4sY8eOlcuXL2fqewGwE2z5fAAImOXLl0vXrl0lODhYevToIXfeeafkypVLfv31V1m0aJG88847JniUL18+3vN0+c0332wO+NesWSMzZsyQ7du3y/fff5/oPe666y4ZMmSI+X9ERITMmjVLOnbsaF6jX79+ZnlsbKw89thj8sUXX5jgoQFBD4I3btwoo0ePlgULFsi3334rJUqU8OtzpWX7srNt27bJQw89JOfPn5fHH3/cBAr1448/yptvvikbNmyQVatWidtERkbK3LlzzS0hrcsXL16Ur776Srp06RLvsfnz55tWjkAdXJcpU0bGjx9v/v/HH3/IJ598IoMGDTKfRw/6M9OTTz4pw4YNM+/51FNPZep7AbDgAQAX2r9/vyd//vyeGjVqeCIiIhI9fu3aNc+0adM8R44c8S4bNWqUR3/tRUZGxlu3a9euZvmWLVviLS9fvrynTZs28ZadOHHCvO9tt93mXTZu3Djz/BdeeCHRdixbtsyTK1cuT+vWrVP9TGndvqzqwoULyT526NAh81lmz56d4mucPXvWU7p0aU+JEiU8e/fuTfT4yZMnPW+88UaGbO/58+c9N9KUKVM8efPm9cTExMRb3qtXL1O3WrZs6Wnfvn2i51WtWtXTqVMnU36TJk26gVvs8TRp0sRz++23x1t26dIls48UKFDAc/369UzfhocfftjTuHHjTH8fAOlHVygArjRx4kRzRn/27NlSqlSpJM/8ahci7ZudGm1lUAcOHEh13ZIlS0qNGjVMS4jSLiuTJk0yXbGcs7m+HnnkEenVq5esWLFCfvjhBz8/nX/bt2XLFmndurUUKlTItJA0adIkXp997Qqm3VaWLVvmXfbTTz+ZZXXq1In3Wg8++KDpmuNYunSptGnTRsLCwiQkJEQqV64sb7zxhmmdSdhF5o477jCvq93AdDtefvll85h2VdLuPbp92j1My8Hf7kvaMnT8+HGZMmWK6QaVkLb+vPLKK977+pm0pSgh7Uqm2+DQsR267nfffSd///vfTfc2PRP/5ZdfepcntS362O7du73LtFXs0UcflaJFi5pWhHr16sUr55QsWbLElLW2SiVFW7+++eabeGWlrTfaFUofS+jMmTOmu5h2vdPX1K5U+n3u2rUr0bra2qHlpPVVt1v3HW2B86fuJ6TPv+eeeyQmJkZOnz6d6hiahN+R0/Vv//795jvSOqJ1RVsntNUmoRYtWphWO/28ALImggUA13aDqlKlSryD4fTSgyFVpEgRvwaS6sDbW265xdzXA52zZ8+aAz4NM0np2bOnd5szavvWrl1rDuSjo6Nl1KhRMm7cOHMg+te//lW2bt1q1tEDfj1Y0y5DDu2epd3F9KBTn6u0j/x///tf83oOPTDUg1Tt2z5t2jTTDWnkyJGmO0pCf/75pzmQ1W5j2uf+gQceMGNO2rVrZ8adaDcmHRNx7NgxEy78oQfp2o9fD94zg4aKX375xfuZNETp59XubAl9/vnncvvtt5vyVHv27JF7771X9u7da5771ltvmbEf7du3T3L8TcL6oyEhYbDzpQf6esCt3fkc2gVIA1ZSzzt48KAJKw8//LAJYi+++KL8/PPPJmhq9z2HhkJdR7vn6fep2z1gwAA5d+5cvNCUFk6Q0HqWXtrlS8OJBnP9v9Y93caEdJu1XmldBZBFWbR2AEBAnDt3znQHSaq7iHah0a5Ezu3ixYuJuhqFh4ebxw4fPuz58MMPTbeUYsWKJerCo908tFuK81q7du3ydOvWzbzG888/b9aZOnWqub948eJkt/fMmTNmnY4dO6b4ufzdvri4ONMtplWrVub/Dv2sFStW9LRo0cK7TLty/eUvf/He123QW+7cuT3ffPONWbZ9+3bzvkuXLo33Wgn97W9/8+TLl89z+fLleF1k9LkzZ86Mt+6SJUvM8okTJ3qXaXcZ7criT1eoIkWKeO68806Pv/Q1tfwS0u9Quxg59H113UaNGiXqvtO9e3dP8eLF4y3Xrm/ale3111/3LmvWrJmnVq1a8cpBv4eGDRua7yW1Lnz6/jNmzEj0mNMVSj366KPmfVRsbKynZMmSntGjR3u7kvl2hdLt0HV86XohISHxtlvrkj5Xu2Il5FuPkqLfc/Xq1b37wq+//up58cUXzev5dhdMqatbwu/Iqe9PPfVUvPU6dOjgueWWWxI9X7s86voTJkxIcVsBBA4tFgBcxznTnlRXEu2aU6xYMe/t3//+d6J1qlWrZh7TbjI6EFRbPrTriXbjSUgHBzuvpYPDdSD2E088IRMmTDCP65lWVaBAgWS313nM2e7UpLZ9O3fu9HaL0dYCHUirN+0a1qxZM9NC4czUo92odOC3MzOVtrDogGhtXdDWC6U/9axzo0aNvNugrQUO/Yz6+vpa2kVFuwH50q5S2n3F13/+8x/TgtO/f3/vsty5c8vzzz/vVxloWaVUprb69u1rtseXTgSgXXrWr1/vXaZdpLQs9TGl3XC0tcg5y+6UvX4PrVq1Mt+LduFKjq7nT+uYfre6HSdPnjTvpz+T6gbllL+2QjmtEvoeum9oPdLv3rFw4UK59dZbk/wO9PtPjX7vzr6grSfaBbBt27bWUwc7kyA4tJ7pZ0i4vzhlpuUNIGtiVigAruMccOpsQUn1h9cDPp2mVLvgJEUPsLQfus5mM336dDNewvdA2pd2tdJuPHrgpQf2Or7Ct9uHsy1OwEiKP+EjLdunB68qpW5F2r1FD8T0IO369euyefNmM95ED5x1mXbn8Q0WOkWvjhdw6OM6hkEPahMe4Olr+ypdurTkyZMn3rLff//d9N9PGP70YNcf+vlTKlNbFStWTLTMGa+iXZ80oCn9v4YwHZOgdDyAnnx/9dVXzS0pWsZaJin5vxP4ydPwp/VF31+DpI5l0IDpdIvzpcFHu6vpdR60rviOg3G67CkdR6Hln1yXvdRo0H3vvffM++lr6UxQWkd1rIWNcuXKJRkgtIuh1oOEZeZPCAIQGAQLAK6jB3960JpUv3BnzEVSB2AOHUugZ26dwdU66FWnq9UByM6ZX4eu17x582RfS4OGM1Ba+9gnxbmehh68+yO17XNaI/SMsR70JsU5oNdBxXrgp60YegCng5X1IFnDhR6IXrlyxQSLDh06eJ+rYzW0f74e1On1FnTgtr6Gnv1+6aWXEl23ILlQZkPPiOsB9dWrVxOFlrRIONg8pW3WM//OOAktGw2nOhhex684nM+ug6W1hSIpGgCS4xzo60FzSnRbdKyFTkmrYyiSGpju0O3TkKOtWzrAXgOi1pOBAwdm6DUmdByJ776g1+DQMR86WF8DcEoH/cl9Dyphy1Fy4cspM2ffAJD1ECwAuJIOtn3//ffNQOW//OUv6X4dPQDXwc/alUcH7nbr1i1Nz9fuQ9qCoYNrR4wYkeRB0kcffWR+6sDZjNg+PdBXeuCfUuhRelCu5aPhQYOFM8OU/tRQoddG0ANo34Hb2gVHu6Lo4GHf5c5MWP7Qa4foNTi0Vcm31SI8PNyv52ug0lYWbb3p3r17quvrWe6EM05pKDlx4oSkhXZ50oN53XYdnK0Ht043KFWpUiXvRdtSK/uk6HegocafstSuTx9++KEJCSnVS+2upQPmP/jgg3jLtTx8D8K13uhMYjqAXLffll4gT1sFtZVQg5Z+Nqe1IeF3oS1Ytpwyc8I8gKyHMRYAXGno0KGma5KepdUD47R2NfGlrQE65agzbiItdBv0oEoPmDVYJPT111+bPuh6dltnEkqPhNuns+PoQaJefTmp7mDaPcWXhgg9oFy3bp03WOgBpx6gOa/pLFdOOPItQz1I17P4/tKuPNoFSy/253vWWi/252+/e22V0osT/vbbb0l2N9Iuag4tD9/Zr9S7776b4pnypGhY0DP+2gVJbxrKfLtNaYuPjuPRg+mkQkvCsk9ID+i1FUkv8pcaDQvaAvGvf/3LTHOcHP2+EtZ3HQuUcKxHp06dzPgEfT2b/SXhfqhBRWejcsKu1q2E30Va6k5ynKmSGzRoYP1aADIHLRYAXKlq1aqmlUDPZmu/cefK23qApGc29TE906sH5KnRgz2ddlOn6dTrTWhf+7TQKUd37NhhDtL1LLsewOlZaR0oPW/ePHMAn9RVlv2V1PZpa41O8arToGprhvbp1wNJDQ96cKdXbnZoaND+8DpNrm+A0NYIPUDWvvO+5dSwYUNz5lnHcOi1QPRgTqeNTcvBp7Y4aFcZLRvtlqbdwLQFJOH4jOTo+2uXJGegue+Vt7VL1qeffhrvAPPpp582YUTLXq93oNPprly5Ms3dZrSstQvSZ599Zga8a3hLSCcE0JYq7aKmg8C1FUPDrX73OqVuUteP8KXT8GoI1bErvmMIEtL663utjuRoS5h2WdN6oN+dTjWrLVFO64rvtMfaeqZTCGtLn9YF/Yx6VXidfle3K630e9XvSOujdsfSrl76XeiV0fWnhigNGUmFw7RavXq1qVO+40YAZDEBnJEKAKzp9J39+/f3VKlSxRMaGmqmZtVpMfv16+fZuXOnX1e2dqawLVSokJlWM6UrbydHp/vUKTbvu+8+T8GCBc226JWKdYpQf6/snNbt27Fjh5k6Vqfm1KlFdXu7dOniWbNmTbznRkdHm+llE14hed68eeb9nnjiiUTvt2nTJs+9995ryjMsLMwzdOhQz8qVK83669atS/GKzI4///zTvLaWh267/l+32Z/pZn2nGB00aJC50rmWqU53W7duXc/YsWNNmfiW/0svveS59dZbzTo6Fa/WjeSmm922bVuy77l69WqzTlBQkOfo0aNJrnPgwAFPz549zTSwN910k7lKuF4Z+ssvv0z1M506dcoTHBzs+fjjj5OdbjY5yU03O2TIEE+pUqXM96V1cPPmzea78a0vzjTCI0aMMNMS63br9uvUtvp5UpLS97x+/fp4U8nqe/Tp08d851rntE6ePn062elmE9Z35zvSz+qIiory5MmTx/P++++nuJ0AAitI/wl0uAEAICfp06ePOYvvzMyFlOmFFydOnGhmo8qMyQIAZAyCBQAAN9iRI0fM7Fw6SFy79yB5OoZDx9BotzrtsgUg6yJYAAAAALDGrFAAAAAArBEsAAAAAFgjWAAAAACwRrAAAAAAYI0L5GWAuLg4iYiIkAIFCpgLSQEAAADZgV6ZIiYmRsLCwsyFO1NCsMgAGirKli2bES8FAAAAZDlHjx6VMmXKpLgOwSIDaEuFU+AFCxaUQLSYREZGSrFixVJNkqDsqHdZA/stZUe9cx/2W8ouJ9a76OhocwLdOd5NCcEiAzjdnzRUBCpYXL582bw3wYKyo965A/stZUe9cx/2W8ouJ9e7ID+6+3N6GwAAAIA1ggUAAAAAawQLAAAAANYIFgAAAACsESwAAAAAWCNYAAAAALBGsAAAAABgjWABAAAAwBrBAgAAAIA1ggUAAAAAawQLAAAAANYIFgAAAACsESwAAAAAWCNYAAAAALBGsAAAAABgjWABAAAAwBrBAgAAAIA1ggUAAAAAawQLAAAAANYIFgAAAACsESwAAAAAWCNYAAAAALBGsAAAAABgjWABAAAAwBrBAgAAAIA1ggUAAAAAawQLAAAAANYIFgAAAACsESwAAAAAWCNYAAAAALBGsAAAAABgjWABAAAAwBrBAgAAAIA1ggUAAAAAawQLAAAAANYIFgAAAACsESwAAAAAWCNYAAAAALBGsAAAAABgjWABAAAAwBrBAgAAAEDOCRZnzpyRHj16SMGCBaVw4cLSp08fOX/+fIrPOXDggHTo0EGKFStmntelSxc5depUovW+/vprqV+/vuTNm1eKFCki7du3z8RPAgAAAGQ/rgkWGir27Nkjq1evluXLl8uGDRvkmWeeSXb9CxcuSMuWLSUoKEjWrl0rmzZtkqtXr8ojjzwicXFx3vUWLlwoTzzxhDz55JOya9cus95jjz12gz4VAAAAkD0Eiwvs3btXVqxYIdu2bZN69eqZZTNmzJCHHnpIJk+eLGFhYYmeowHh8OHDsmPHDtNaoebOnWtaJDRoNG/eXK5fvy4DBgyQSZMmmRYQR82aNW/gpwMAAADczxUtFps3bzbdn5xQoTQY5MqVS7Zs2ZLkc65cuWJaK0JCQrzLQkNDzXO+//57c3/79u1y/Phxs+zuu++WUqVKyYMPPii7d+++AZ8KAAAAyD5c0WJx8uRJKV68eLxlwcHBUrRoUfNYUu69917Jnz+/vPTSSzJu3DjxeDwybNgwiY2NlRMnTph1Dh48aH6+9tprMmXKFKlQoYK89dZb0rRpU/ntt9/M6ycXWvTmiI6ONj+1i5VvN6sbRd9TP18g3tvtKDvKjrrnPuy3lB11z33Yb91bdml534AGCz3QnzBhQqrdoNJDB2wvWLBA+vfvL9OnTzetEt27d5c6deqY//sW1IgRI6RTp07m/7Nnz5YyZcqY5/7tb39L8rXHjx8vo0ePTrQ8MjJSLl++LDeafo5z586ZSud8NlB21Lusjf2WsqPeuQ/7LWWXE+tdTEyMO4LFkCFDpHfv3imuU6lSJSlZsqScPn063nIdH6EzReljydHB2zoz1B9//GFaOLQ7la6vr6m061PCMRXadUofP3LkSLKvO3z4cBk8eHC8FouyZct6Z58KRIXTbl/6/gQLyo565w7st5Qd9c592G8pu5xY70JDQ90RLLSA9JaaBg0aSFRUlPz0009St25ds0wHYGtB6zSxqbn11lu9z9GA0rZtW3NfX0uDRHh4uDRq1Mgsu3btmhn0Xb58+WRfT5/jO3bDoV92oA7stcIF8v3djLKj7Kh77sN+S9lR99yH/dadZZeW93TFUWiNGjWkdevW0rdvX9m6dauZ8em5556Tbt26eWeE0kHY1atXN487tFvTDz/8YFot5s2bJ507d5ZBgwZJtWrVzOPautCvXz8ZNWqUrFq1ygQM7TqldF0AAAAA2Wjwtpo/f74JE82aNTPJScdE6NgJh7Y0aDC4ePGid5ne125L2mVKB2brWAoNFr50qlntJqXXsrh06ZJpAdGWDZ2WFgAAAIB/gjw6EgRWdIxFoUKFzMCaQI2x0C5eOnMWXaEoO+qdO7DfUnbUO/dhv6XscmK9i07Dca4rukIBAAAAyNoIFgAAAACsESwAAAAAWCNYAAAAALBGsAAAAABgjWABAAAAwBrBAgAAAIA1ggUAAAAAawQLAAAAANYIFgAAAACsESwAAAAAWCNYAAAAALBGsAAAAABgjWABAAAAwBrBAgAAAIA1ggUAAAAAawQLAAAAANYIFgAAAACsESwAAAAAWCNYAAAAALBGsAAAAABgjWABAAAAwBrBAgAAAIA1ggUAAAAAawQLAAAAANYIFgAAAACsESwAAAAAECwAAAAABB4tFgAAAACsESwAAAAAWCNYAAAAALBGsAAAAABgjWABAAAAwBrBAgAAAIA1ggUAAAAAawQLAAAAANYIFgAAAACsESwAAAAAWCNYAAAAALBGsAAAAABgjWABAAAAwBrBAgAAAIA1ggUAAAAAawQLAAAAANYIFgAAAACsESwAAAAAWCNYAAAAALBGsAAAAABgjWABAAAAwBrBAgAAAIA1ggUAAAAAawQLAAAAANYIFgAAAACsESwAAAAA5JxgcebMGenRo4cULFhQChcuLH369JHz58+n+JwDBw5Ihw4dpFixYuZ5Xbp0kVOnTsVb57fffpN27drJrbfeatZp1KiRrFu3LpM/DQAAAJC9uCZYaKjYs2ePrF69WpYvXy4bNmyQZ555Jtn1L1y4IC1btpSgoCBZu3atbNq0Sa5evSqPPPKIxMXFedd7+OGH5fr162adn376Se68806z7OTJkzfokwEAAADuFywusHfvXlmxYoVs27ZN6tWrZ5bNmDFDHnroIZk8ebKEhYUleo4GicOHD8uOHTtMS4SaO3euFClSxISI5s2byx9//CH79u2TDz74QGrXrm3WefPNN+Xtt9+W3bt3S8mSJW/wJwUAAADcyRXBYvPmzab7kxMqlAaDXLlyyZYtW0x3p4SuXLliWitCQkK8y0JDQ81zvv/+e/P8W265RapVqyYfffSR1KlTx6w7a9YsKV68uNStWzfZ7dHX1psjOjra/NSWEN/WkBtF39Pj8QTkvd2OsqPsqHvuw35L2VH33If91r1ll5b3dUWw0G5JerDvKzg4WIoWLZpsl6V7771X8ufPLy+99JKMGzfOfCHDhg2T2NhYOXHihFlHg8e3334r7du3lwIFCpjQoe+jrSPaspGc8ePHy+jRoxMtj4yMlMuXL0sgvvBz586Zz6ifAZQd9S7rY7+l7Kh37sN+S9nlxHoXExPjjmChB/oTJkxItRtUeuiA7QULFkj//v1l+vTp5ovo3r27aZlwvhT9gp599lkTJjZu3Ch58+aV999/34zD0G5XpUqVSvK1hw8fLoMHD47XYlG2bFnvIPFAVDgNSfr+BAvKjnrnDuy3lB31zn3Ybym7nFjvQkND3REshgwZIr17905xnUqVKpmxDqdPn463XAdc60xRKY2D0MHbOjOUjqXQFg7tTqXr62sqHWuhA8HPnj3rDQQ6vkIHiOt4DA0+SdEuU75drBz6ZQfqwF4rXCDf380oO8qOuuc+7LeUHXXPfdhv3Vl2aXnPgAYLTV56S02DBg0kKirKzNrkjH3QUKAJrn79+qk+X6eSdZ6jAaVt27bm/sWLF5MsML3PeAUAAADAf644vV2jRg1p3bq19O3bV7Zu3WpmfHruueekW7du3hmhjh8/LtWrVzePO2bPni0//PCDabWYN2+edO7cWQYNGmQGbDuBRcdS9OrVS3bt2mWuafHiiy/KoUOHpE2bNgH7vAAAAIDbuGLwtpo/f74JE82aNTMtCp06dTJjJxzXrl2T8PBwbyuE0vs6HkK7TFWoUEFGjBhhgoVvS4YO1Nblf/3rX81r3H777bJ06VJzPQsAAAAA2SxY6AxQn3zySbKPa3DQwdi+9JoUekuJTmG7cuXKDNtOAAAAICdyRVcoAAAAAFkbwQIAAACANYIFAAAAAGsECwAAAADWCBYAAAAArBEsAAAAAFgjWAAAAACwRrAAAAAAYI1gAQAAAMAawQIAAACANYIFAAAAAGsECwAAAADWCBYAAAAArBEsAAAAAFgjWAAAAACwRrAAAAAAYI1gAQAAAMAawQIAAACANYIFAAAAAGsECwAAAADWCBYAAAAArBEsAAAAAFgjWAAAAACwRrAAAAAAYI1gAQAAAMAawQIAAACANYIFAAAAAGsECwAAAADWCBYAAAAArBEsAAAAAFgjWAAAAACwRrAAAAAAYI1gAQAAAMAawQIAAACANYIFAAAAAGvB/q44ePBgv190ypQp6d0eAAAAANk5WOzYsSPe/e3bt8v169elWrVq5v5vv/0muXPnlrp162b8VgIAAADIHsFi3bp18VokChQoIHPnzpUiRYqYZWfPnpUnn3xSGjdunDlbCgAAACB7jbF46623ZPz48d5QofT/Y8aMMY8BAAAAyFnSFSyio6MlMjIy0XJdFhMTkxHbBQAAACC7B4sOHTqYbk+LFi2SY8eOmdvChQulT58+0rFjx4zfSgAAAADZY4yFr5kzZ8oLL7wgjz32mFy7du3/Xig42ASLSZMmZfQ2AgAAAMhuwSI2NlZ+/PFHGTt2rAkRBw4cMMsrV64s+fPnz4xtBAAAAJDdgoVOKduyZUvZu3evVKxYUWrXrp05WwYAAAAge4+xuOOOO+TgwYMZvzUAAAAAck6w0GlldYzF8uXL5cSJE2aWKN8bAAAAgJwlXYO3H3roIfOzbdu2EhQU5F3u8XjMfR2HAQAAACDnSFew8L0KNwAAAACkK1g0adKEkgMAAABgFywcFy9elCNHjsjVq1fjLWemKAAAACBnSVewiIyMNFfe/uabb5J8nDEWAAAAQM6SrlmhBg4cKFFRUbJlyxbJmzevrFixQubOnStVq1aVZcuWZfxWAgAAAMh+LRZr166VpUuXSr169SRXrlxSvnx5adGihRQsWFDGjx8vbdq0yfgtBQAAAJC9WiwuXLggxYsXN/8vUqSI6RqlatWqJdu3b5fMMHbsWGnYsKHky5dPChcu7NdzdPrbkSNHSqlSpUzLSvPmzWXfvn3x1jlz5oz06NHDhCJ93T59+sj58+fFLWLjYmX94fWyeP9i81PvA9Q7ZFf8zgN1DzlJrMuO89LVYlGtWjUJDw+XChUqyJ133imzZs0y/585c6Y5iM8MOkC8c+fO0qBBA/nggw/8es7EiRNl+vTppptWxYoV5dVXX5VWrVrJL7/8IqGhoWYdDRV6kb/Vq1fLtWvXzNiRZ555Rj755BPJ6hbtXSQDVgyQY9HHvMvKFCwj01pPk441OgZ025B9Ue9A3UNOw+89UO/8E+TR0/ppNG/ePLl+/br07t1bfvrpJ2ndurU5858nTx6ZM2eOdO3aVTKLvr4zxiMl+rHCwsJkyJAh5irh6ty5c1KiRAnzGt26dZO9e/dKzZo1Zdu2baZbl9LxInoBwGPHjpnn+0OvNl6oUCHz+trycaN+yT36xaPikfhfX5D83wULv+zyJeHCD3FxcXL69GnTAqfd+kC9u1Goe2nD7zzqXaBQ96h7Ob3eRafhODddwSKpaWd//fVXKVeunNx6662SmfwNFgcPHpTKlSvLjh075K677op3DQ69P23aNPnwww9N8Dh79qz3cQ1M2pqxYMEC6dChQ5YMFtoMVmFahXgtFQkrXekCpWXP3/dI7ly5M3173H5wp135ihUrRrDwo97VfLumHI85nuTj1DvqXmah7mUcfudR9wKFupexv/O0h8qhAYduyHFeWo5z09UVSg/aK1Wq5L2v4x7q1KkjWcnJkyfNT22h8KX3ncf0pzNWxBEcHCxFixb1rpOUK1eumJtvgTs7jd4y23eHv0s2VChNt8dijkmhCYUyfVsA6h0Cjd95oO4hp/3OOxp91BwPNq3QNNPfLy3HtukKFlWqVJEyZcqYs/9NmzY1P3VZWg0bNkwmTJiQ4jraXal69eqSlejMV6NHj060XM98X758OdPfPzwiPNPfAwAAAFlXeES41MxXM9PfJyYmJnODxdGjR2X9+vXy3XffmQHSffv2NeMRNGA88MAD8vTTT/v1OtoNScdppMS3ZSQtSpYsaX6eOnUq3oByve90jdJ1tH+9L+0KpeNFnOcnZfjw4TJ48OB4LRZly5Y13WluRFeoaher+bXe192/lsblGmf69riZpvA//vjDdOFjjEXKNh7ZKG0+TX0qaeoddS+jUfcyDr/zqHuBQt3L+N951cKqJep5kxmcCY8yLViULl3azKakN6VTuOp0sPPnz5fPPvvM72ChB+J6yww6C5SGgzVr1niDhAYAvahf//79zX2dYUrHaugA9Lp163qv0aGVv379+sm+dkhIiLklpAemN+LgtEmFJqZv3fHo44kG9fj2vWtVpRVjLFKh3/WlPJekQGgBgkUqtD5R7zIOdc9/1D3qXaBQ96h7WbneNanQ5IYcu6TlPXKld7D2qlWr5OWXXzbXlqhdu7bs2rVLnnvuOVm0aJFkhiNHjsjOnTvNz9jYWPN/vflec0K7TC1evNj8PygoyAzyHjNmjLka+M8//yw9e/Y0LSvt27c369SoUcPMaKUtLlu3bpVNmzaZz6AzRvk7I1Qg6EAdnVLWd3YAh3N/auuphApQ75At8DsP1D3kJLldfJyXrmChF5J74oknzHgCHScRERFhZl/65z//Ke3atcv4rRQxF7q7++67ZdSoUSZM6P/19uOPP3rX0Wtr6Ih1x9ChQ+X5558316W45557zPN0OlnfJh1tZdFA0qxZMzPNbKNGjeTdd9+VrE6nGNOpxkoXLB1vuSZYppoF9Q7ZDb/zQN1DTtLRpcd56ZpuVs/4f//99+a6FTp427nddtttkhMF4joWvlOS6awAOoBH+9pps1hWTLBZFdcSSB/qHXUvUKh7dvidR90LFOqee3/n3bDrWPzvf/8zA7j1tnHjRjNVqwYMbQXISQIZLBQ7K2UXCNQ7yi9QqHuUHXXPfdhv3Vt2mX4dC0etWrXMLEpXr1413aJWrlwpn3/+eY4LFgAAAEBOl67YM2XKFGnbtq3ccsstZvakTz/91HSDWrhwobmWAwAAAICcJV0tFhok9JoVOii6cePGpnkEAAAAQM6VrmCxbdu2jN8SAAAAAK6V7hEgOlj78ccfNxeZO378uFn28ccfm9miAAAAAOQs6QoWOpaiVatWkjdvXnP9iitXrpjlOlp83LhxGb2NAAAAALJjsNCrWc+cOVPee+89uemmm7zL77vvPtm+fXtGbh8AAACA7Bos9ArX999/f6LlOog7KioqI7YLAAAAQHYPFiVLlpT9+/cnWq7jKypVqpQR2wUAAAAguweLvn37yoABA2TLli0SFBQkERER5qJ4Q4YMkf79+2f8VgIAAADIftPNDhs2zFxevFmzZnLx4kXTLSokJERefPFFefrppzN+KwEAAABkaelqsdBWihEjRsiZM2dk9+7d8sMPP5grbusYi4oVK2b8VgIAAADIPsFCp5UdPny41KtXz8wA9Z///Edq1qwpe/bskWrVqsm0adNk0KBBmbe1AAAAANzfFWrkyJEya9Ysad68ufz3v/+Vzp07y5NPPmlaLN566y1zP3fu3Jm3tQAAAADcHywWLFggH330kbRt29Z0gapdu7Zcv35ddu3aZbpHAQAAAMiZ0tQV6tixY1K3bl3z/zvuuMMM2NauT4QKAAAAIGdLU7CIjY2VPHnyeO8HBwfLzTffnBnbBQAAACC7doXyeDzSu3dv01KhLl++LP369ZP8+fPHW2/RokUZu5UAAAAAsk+w6NWrV7z7jz/+eEZvDwAAAIDsHixmz56deVsCAAAAIGddIA8AAAAAfBEsAAAAAFgjWAAAAACwRrAAAAAAYI1gAQAAAMAawQIAAACANYIFAAAAAGsECwAAAADWCBYAAAAArBEsAAAAAFgjWAAAAACwRrAAAAAAYI1gAQAAAMAawQIAAACANYIFAAAAAGsECwAAAADWCBYAAAAArBEsAAAAAFgjWAAAAACwRrAAAAAAYI1gAQAAAMAawQIAAACANYIFAAAAAGsECwAAAADWCBYAAAAArBEsAAAAAFgjWAAAAACwRrAAAAAAYI1gAQAAAMAawQIAAACANYIFAAAAAGsECwAAAAA5J1iMHTtWGjZsKPny5ZPChQv79RyPxyMjR46UUqVKSd68eaV58+ayb98+7+OHDx+WPn36SMWKFc3jlStXllGjRsnVq1cz8ZMAAAAA2Y9rgoUe7Hfu3Fn69+/v93MmTpwo06dPl5kzZ8qWLVskf/780qpVK7l8+bJ5/Ndff5W4uDiZNWuW7NmzR/75z3+adV9++eVM/CQAAABA9hMsLjF69Gjzc86cOX63VkydOlVeeeUVadeunVn20UcfSYkSJWTJkiXSrVs3ad26tbk5KlWqJOHh4fLOO+/I5MmTM+mTAAAAANmPa1os0urQoUNy8uRJ0/3JUahQIalfv75s3rw52eedO3dOihYteoO2EgAAAMgeXNNikVYaKpS2UPjS+85jCe3fv19mzJiRamvFlStXzM0RHR1tfmq3Kr3daPqe2kITiPd2O8qOsqPuuQ/7LWVH3XMf9lv3ll1a3jegwWLYsGEyYcKEFNfZu3evVK9ePdO35fjx46ZblI7j6Nu3b4rrjh8/3ts1y1dkZKR3/MaN/sK1pUUrXa5c2bYRKlNQdpQddc992G8pO+qe+7DfurfsYmJi3BEshgwZIr17905xHR33kB4lS5Y0P0+dOmVmhXLo/bvuuiveuhEREfLAAw+YWafefffdVF97+PDhMnjw4HgtFmXLlpVixYpJwYIFJRAVLigoyLw/wYKyo965A/stZUe9cx/2W8ouJ9a70NBQdwQLLSC9ZQadQlbDxZo1a7xBQgOAzg7lO7OUtlRoqKhbt67Mnj3bry8sJCTE3BLS5wbqwF4rXCDf380oO8qOuuc+7LeUHXXPfdhv3Vl2aXlP1xyFHjlyRHbu3Gl+xsbGmv/r7fz58951tMvU4sWLvV/AwIEDZcyYMbJs2TL5+eefpWfPnhIWFibt27f3hoqmTZtKuXLlzLgK7cqk4y+SG4MBAAAAwOWDt/VCd3PnzvXev/vuu83PdevWmXCgdKpY7YPmGDp0qFy4cEGeeeYZiYqKkkaNGsmKFSu8TTqrV682A7b1VqZMmXjvp/3YAAAAAPgnyMMRtDXtYqVT2WqoCdQYi9OnT0vx4sXpCkXZUe9cgv2WsqPeuQ/7LWWXE+tddBqOc13TFQoAAABA1kWwAAAAAGCNYAEAAADAGsECAAAAgDWCBQAAAABrBAsAAAAA1ggWAAAAAKwRLAAAAABYI1gAAAAAsEawAAAAAGCNYAEAAADAGsECAAAAgDWCBQAAAABrBAsAAAAA1ggWAAAAAKwRLAAAAABYI1gAAAAAsEawAAAAAGCNYAEAAADAGsECAAAAgDWCBQAAAABrBAsAAAAA1ggWAAAAAKwRLAAAAABYI1gAAAAAsEawAAAAAGCNYAEAAADAGsECAAAAgDWCBQAAAABrBAsAAAAA1ggWAAAAAAgWAAAAAAKPFgsAAAAA1ggWAAAAAKwRLAAAAABYI1gAAAAAsEawAAAAAGCNYAEAAADAGsECAAAAgDWCBQAAAABrBAsAAAAA1ggWAAAAAKwRLAAAAABYI1gAAAAAsEawAAAAAGCNYAEAAADAGsECAAAAgDWCBQAAAABrBAsAAAAA1ggWAAAAAKwRLAAAAABYI1gAAAAAsEawAAAAAGCNYAEAAAAg5wSLsWPHSsOGDSVfvnxSuHBhv57j8Xhk5MiRUqpUKcmbN680b95c9u3bl+S6V65ckbvuukuCgoJk586dGbz1AAAAQPbmmmBx9epV6dy5s/Tv39/v50ycOFGmT58uM2fOlC1btkj+/PmlVatWcvny5UTrDh06VMLCwjJ4qwEAAICcwTXBYvTo0TJo0CCpVauW360VU6dOlVdeeUXatWsntWvXlo8++kgiIiJkyZIl8db95ptvZNWqVTJ58uRM2noAAAAgewuWbOrQoUNy8uRJ0/3JUahQIalfv75s3rxZunXrZpadOnVK+vbta8KGdrPyh3ab0psjOjra/IyLizO3G03fU4NUIN7b7Sg7yo665z7st5Qddc992G/dW3Zped9sGyw0VKgSJUrEW673ncf0S+rdu7f069dP6tWrJ4cPH/brtcePH29aUBKKjIxMspvVjfjCz507Zz5PrlyuaYTKEig7yo665z7st5Qddc992G/dW3YxMTHuCBbDhg2TCRMmpLjO3r17pXr16pny/jNmzDCFNXz48DQ9T9cfPHhwvBaLsmXLSrFixaRgwYISiAqng871/QkWlB31zh3Ybyk76p37sN9Sdjmx3oWGhrojWAwZMsS0GKSkUqVK6XrtkiVLers66axQDr2vsz+ptWvXmm5RISEh8Z6rrRc9evSQuXPnJvnaun7C5yj9sgN1YK8VLpDv72aUHWVH3XMf9lvKjrrnPuy37iy7tLxnQIOFJi+9ZYaKFSuacLFmzRpvkNCWBZ0dyplZSmeMGjNmjPc5OrBbZ436/PPPzVgMAAAAANlsjMWRI0fkzJkz5mdsbKz3WhNVqlSRm2++2fxfu0zp+IcOHTqYZDdw4EATHKpWrWqCxquvvmqmlG3fvr1Zv1y5cvHew3mdypUrS5kyZW74ZwQAAADcyjXBQi9059s16e677zY/161bJ02bNjX/Dw8PN4NbfK9NceHCBXnmmWckKipKGjVqJCtWrEhTXzEAAAAA2ShYzJkzx9xSoqPlfWmrxeuvv25u/qhQoUKi1wAAAACQOkb6AgAAALBGsAAAAABgjWABAAAAwBrBAgAAAIA1ggUAAAAAawQLAAAAANYIFgAAAACsESwAAAAAWCNYAAAAALBGsAAAAABgjWABAAAAwBrBAgAAAIA1ggUAAAAAawQLAAAAANYIFgAAAACsESwAAAAAWCNYAAAAALBGsAAAAABgjWABAAAAwBrBAgAAAIA1ggUAAAAAawQLAAAAANYIFgAAAACsESwAAAAAWCNYAAAAALBGsAAAAABgjWABAAAAwBrBAgAAAIA1ggUAAAAAawQLAAAAANYIFgAAAACsESwAAAAAWCNYAAAAALBGsAAAAABgjWABAAAAwBrBAgAAAIA1ggUAAAAAawQLAAAAANYIFgAAAACsESwAAAAAWCNYAAAAALAWbP8S8Hg8phCio6MDUhhxcXESExMjoaGhkisXWZGyo965AfstZUe9cx/2W8ouJ9a76P/v+NY53k0JwSID6JetypYtmxEvBwAAAGS5491ChQqluE6Qx5/4gVSTZEREhBQoUECCgoICkiQ11Bw9elQKFix4w9/fzSg7yo665z7st5Qddc992G/dW3YaFTRUhIWFpdpiQotFBtBCLlOmjASaVjaCBWVHvXMX9lvKjnrnPuy3lF1Oq3eFUmmpcNAhHwAAAIA1ggUAAAAAawSLbCAkJERGjRplfoKyo965A/stZUe9cx/2W8qOepcyBm8DAAAAsEaLBQAAAABrBAsAAAAA1ggWAAAAAKwRLFxsw4YN8sgjj5gLluiF+ZYsWRLoTXKF8ePHyz333GMuaFi8eHFp3769hIeHB3qzXOOdd96R2rVre+fTbtCggXzzzTeB3ixXevPNN82+O3DgwEBviiu89tprprx8b9WrVw/0ZrnG8ePH5fHHH5dbbrlF8ubNK7Vq1ZIff/wx0JuV5VWoUCFRvdPbs88+G+hNy/JiY2Pl1VdflYoVK5o6V7lyZXnjjTfMBdfgH70wnf6NKF++vCnDhg0byrZt2ySr4gJ5LnbhwgW588475amnnpKOHTsGenNc47vvvjN/EDRcXL9+XV5++WVp2bKl/PLLL5I/f/5Ab16WpxeD1APiqlWrmj8Oc+fOlXbt2smOHTvk9ttvD/TmuYb+YZg1a5YJafCf1rFvv/3Wez84mD9j/jh79qzcd9998sADD5gTAcWKFZN9+/ZJkSJFqH5+7Kt6gOzYvXu3tGjRQjp37kzZpWLChAnmZJT+ndB9V4Psk08+aS629o9//IPy88PTTz9t6tzHH39sTiTPmzdPmjdvbo5ZSpcuLVkNs0JlE3r2ZPHixebsO9ImMjLStFxo4Lj//vspvnQoWrSoTJo0Sfr06UP5+eH8+fNSp04defvtt2XMmDFy1113ydSpUyk7P1ostGV2586dlFUaDRs2TDZt2iQbN26k7Czp2ePly5ebYKZ/e5G8hx9+WEqUKCEffPCBd1mnTp3MmXc9QEbKLl26ZHpXLF26VNq0aeNdXrduXXnwwQfN34+shq5QyPHOnTvnPThG2uhZvM8++8y0nmmXKPhHW8z0j4SedULa6MGcnrWrVKmS9OjRQ44cOUIR+mHZsmVSr149c5ZdT6Tcfffd8t5771F2aXT16lVzQKw9BQgVqdNuO2vWrJHffvvN3N+1a5d8//335qAYqdNeFfp3NjQ0NN5yDWZajlkRbcjI0eLi4szZJ+0icMcddwR6c1zj559/NkHi8uXLcvPNN5vWspo1awZ6s1xBg9j27duzdB/ZrKp+/foyZ84cqVatmpw4cUJGjx4tjRs3Nt0E9Kweknfw4EHTJWXw4MGm+6fWP+2KkidPHunVqxdF5ydtMYuKipLevXtTZn62lEVHR5uxULlz5zYHyWPHjjUnBZA6/b2mf2t1XEqNGjVM68+nn34qmzdvlipVqkhWRLCA5PQzx3pQklWTf1alB3baHUVbe7788ktzYKJdyQgXKTt69KgMGDBAVq9enegMFFLne5ZTx6Zo0NABjV988QXd8Pw4iaItFuPGjTP3tcVCf/fNnDmTYJEG2qVH66G2miF1um/Onz9fPvnkEzPGQv9u6Mk8LT8CrX90bIW2kOl4Cg1n2o22e/fu8tNPP0lWRLBAjvXcc8+ZfrI6u5YOSIb/9Cync7ZE+3rq2c9p06aZwchInv4hOH36tPnD4NAzeFoH//Wvf8mVK1fMHw74p3DhwnLbbbfJ/v37KbJUlCpVKlHw1zOgCxcupOz89Pvvv5uJAxYtWkSZ+enFF180rRbdunUz93UmMi1HnZ2RYOEfnUlLT9xpl2Nt/dF9uWvXrqY7aFbEGAvkODqTkYYK7b6zdu1aMw0e7M+G6kExUtasWTPTjUzP2jk3PYus3QL0/4SKtA+CP3DggPlDi5Rpd8+E02prv3dt8YF/Zs+ebcan+A6iRcouXrwouXLFP9TU33P6NwNpo7NW6u86neFt5cqVZjbGrIgWC5f/UfU9U3fo0CFzcKKDkMuVKxfQbcvq3Z+0WVZnWdD+iydPnjTLdfo7HRCFlA0fPtx0BdA6pvNra1muX7/e/KJDyrS+JRzLo38s9LoCjPFJ3QsvvGCu3aMHwxERETJq1ChzkKLdApCyQYMGmYG02hWqS5cusnXrVnn33XfNDanTA2ENFnqWnSmO/af7q46p0L8X2hVKpyWfMmWK6doD/+jfVj0hql2Q9ZhPW4F0zIpO25sleeBa69at0yvMJLr16tUr0JuWpSVVZnqbPXt2oDfNFZ566ilP+fLlPXny5PEUK1bM06xZM8+qVasCvVmu1aRJE8+AAQMCvRmu0LVrV0+pUqVM3StdurS5v3///kBvlmt89dVXnjvuuMMTEhLiqV69uufdd98N9Ca5xsqVK83fifDw8EBviqtER0eb32/lypXzhIaGeipVquQZMWKE58qVK4HeNNf4/PPPTbnp772SJUt6nn32WU9UVJQnq+I6FgAAAACsMcYCAAAAgDWCBQAAAABrBAsAAAAA1ggWAAAAAKwRLAAAAABYI1gAAAAAsEawAAAAAGCNYAEAAADAGsECAAAAgDWCBQAgYCIjI6V///5Srlw5CQkJkZIlS0qrVq1k06ZN5vGgoCBZsmQJ3xAAuEBwoDcAAJBzderUSa5evSpz586VSpUqyalTp2TNmjXy559/BnrTAABpFOTxeDxpfRIAALaioqKkSJEisn79emnSpEmixytUqCC///6793758uXl8OHD5v9Lly6V0aNHyy+//CJhYWHSq1cvGTFihAQHB3tbOt5++21ZtmyZef1SpUrJxIkT5dFHH+WLA4BMQlcoAEBA3HzzzeamXZ2uXLmS6PFt27aZn7Nnz5YTJ05472/cuFF69uwpAwYMMMFi1qxZMmfOHBk7dmy857/66qumRWTXrl3So0cP6datm+zdu/cGfToAyHlosQAABMzChQulb9++cunSJalTp45pudAAULt2bW/Lw+LFi6V9+/be5zRv3lyaNWsmw4cP9y6bN2+eDB06VCIiIrzP69evn7zzzjvede69917zHtqSAQDIeLRYAAACRlsUNAxol6XWrVubbkt68K8tEMnRFojXX3/d2+KhNw0n2qpx8eJF73oNGjSI9zy9T4sFAGQeBm8DAAIqNDRUWrRoYW7afenpp5+WUaNGSe/evZNc//z582Z8RceOHZN8LQBAYNBiAQDIUmrWrCkXLlww/7/pppskNjY23uPaohEeHi5VqlRJdMuV6///s/bDDz/Ee57er1Gjxg36FACQ89BiAQAICJ1StnPnzvLUU0+ZMRUFChSQH3/80cze1K5dO+/MUDr97H333Weuc6GzSI0cOVIefvhhc+0LneVJw4R2j9q9e7eMGTPG+/oLFiyQevXqSaNGjWT+/PmydetW+eCDD/i2ASCTMHgbABAQOhPUa6+9JqtWrZIDBw7ItWvXpGzZsiZsvPzyy5I3b1756quvZPDgwWaa2dKlS3unm125cqUZZ7Fjxw7TqlG9enXThUrHWjiDt//973+bGac2bNhgppudMGGCdOnShW8bADIJwQIAkO0kNZsUACBzMcYCAAAAgDWCBQAAAABrDN4GAGQ7Ho8n0JsAADkOLRYAAAAArBEsAAAAAFgjWAAAAACwRrAAAAAAYI1gAQAAAMAawQIAAACANYIFAAAAAGsECwAAAADWCBYAAAAAxNb/Azh96Oh9s7HrAAAAAElFTkSuQmCC", "text/plain": [ "
" ] }, "metadata": {}, "output_type": "display_data" }, { "name": "stdout", "output_type": "stream", "text": [ "✅ Created adapter archive: /Users/md.kamranalam/Programming/hackathon/global_syndicate_taskforce_env/grpo_trained_auditor_mac.zip\n", "✅ Plot artifacts ready: docs/reward_curve.png, docs/loss_curve.png\n" ] } ], "source": [ "import json\n", "import shutil\n", "from pathlib import Path\n", "import matplotlib.pyplot as plt\n", "\n", "state_file = Path(\"outputs_mac/checkpoint-10/trainer_state.json\")\n", "if not state_file.exists():\n", " checkpoints = sorted(Path(\"outputs_mac\").glob(\"checkpoint-*/trainer_state.json\"))\n", " if not checkpoints:\n", " raise FileNotFoundError(\"No trainer_state.json found in outputs_mac.\")\n", " state_file = checkpoints[-1]\n", "\n", "state = json.loads(state_file.read_text())\n", "log_history = state.get(\"log_history\", [])\n", "\n", "steps, losses = [], []\n", "reward_steps, rewards = [], []\n", "\n", "for row in log_history:\n", " step = row.get(\"step\")\n", " if step is None:\n", " continue\n", " if row.get(\"loss\") is not None:\n", " steps.append(step)\n", " losses.append(float(row[\"loss\"]))\n", " if row.get(\"reward\") is not None:\n", " reward_steps.append(step)\n", " rewards.append(float(row[\"reward\"]))\n", "\n", "Path(\"docs\").mkdir(exist_ok=True)\n", "\n", "plt.figure(figsize=(8, 4.5))\n", "plt.plot(steps, losses, marker=\"o\")\n", "plt.title(\"GRPO Loss Curve (Mac Run)\")\n", "plt.xlabel(\"Step\")\n", "plt.ylabel(\"Loss\")\n", "plt.grid(alpha=0.3)\n", "plt.tight_layout()\n", "plt.savefig(\"docs/loss_curve.png\", dpi=180)\n", "plt.show()\n", "\n", "plt.figure(figsize=(8, 4.5))\n", "plt.plot(reward_steps, rewards, marker=\"o\", color=\"green\")\n", "plt.title(\"GRPO Reward Curve (Mac Run)\")\n", "plt.xlabel(\"Step\")\n", "plt.ylabel(\"Reward\")\n", "plt.grid(alpha=0.3)\n", "plt.tight_layout()\n", "plt.savefig(\"docs/reward_curve.png\", dpi=180)\n", "plt.show()\n", "\n", "# Zip trained adapters for portability\n", "if Path(\"grpo_trained_auditor_mac\").exists():\n", " zip_path = shutil.make_archive(\"grpo_trained_auditor_mac\", \"zip\", \"grpo_trained_auditor_mac\")\n", " print(f\"✅ Created adapter archive: {zip_path}\")\n", "else:\n", " print(\"⚠️ Adapter folder grpo_trained_auditor_mac not found yet.\")\n", "\n", "print(\"✅ Plot artifacts ready: docs/reward_curve.png, docs/loss_curve.png\")" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## 🚀 5. Optional Full GPU Run (When Available)\n", "If you get access to stronger GPU later, run the upgraded Unsloth pipeline:\n", "\n", "`python train_grpo_unsloth.py --max-steps 120 --output-dir grpo_trained_auditor --plot-dir docs`\n", "\n", "This will train with the larger model path and auto-generate submission plots." ] }, { "cell_type": "code", "execution_count": 6, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Traceback (most recent call last):\n", " File \u001b[35m\"/Users/md.kamranalam/Programming/hackathon/global_syndicate_taskforce_env/train_grpo_unsloth.py\"\u001b[0m, line \u001b[35m13\u001b[0m, in \u001b[35m\u001b[0m\n", " from unsloth import FastLanguageModel\n", "\u001b[1;35mModuleNotFoundError\u001b[0m: \u001b[35mNo module named 'unsloth'\u001b[0m\n" ] } ], "source": [ "# Optional GPU-scale training command\n", "!python train_grpo_unsloth.py --max-steps 120 --output-dir grpo_trained_auditor --plot-dir docs" ] } ], "metadata": { "accelerator": "GPU", "colab": { "gpuType": "T4", "provenance": [] }, "kernelspec": { "display_name": ".venv", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.13.7" } }, "nbformat": 4, "nbformat_minor": 0 }