{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "# humanize-e: AI text -> human-sounding text (QLoRA fine-tune)\n", "\n", "**Kaggle settings before you run:** Accelerator = **GPU T4 x2** (or P100). Internet = **On** (to download the model) unless you attach the model as an Input.\n", "\n", "**How this works.** The Kaggle dataset is *not* paired (an AI text and a human text are about different things), so you can't train on it directly. Instead we use the **human texts as the target**, have an instruct model rewrite each one into stiff \"AI style\" (synthetic input), then train: `AI-style text -> original human text`. Every pair has the same meaning, so the model learns style only." ] }, { "cell_type": "code", "metadata": {}, "execution_count": null, "outputs": [], "source": [ "!pip install -q -U transformers accelerate peft bitsandbytes datasets scikit-learn" ] }, { "cell_type": "code", "metadata": {}, "execution_count": null, "outputs": [], "source": [ "import os, glob, random, gc\n", "import numpy as np, pandas as pd, torch\n", "from datasets import Dataset\n", "from tqdm.auto import tqdm\n", "\n", "SEED = 42\n", "random.seed(SEED); np.random.seed(SEED); torch.manual_seed(SEED)\n", "\n", "# ---- Base model: pick ONE ----\n", "# A) Internet ON: downloads from Hugging Face\n", "MODEL_NAME = \"Qwen/Qwen2.5-3B-Instruct\"\n", "# B) Internet OFF: Add Input -> Models -> \"Qwen2.5 3B Instruct\", then paste the folder that contains config.json\n", "# MODEL_NAME = \"/kaggle/input/qwen2.5/transformers/3b-instruct/1\"\n", "\n", "# ---- Dataset columns: check the printout in the next cell and edit if needed ----\n", "TEXT_COL, LABEL_COL = \"text_content\", \"label\"\n", "HUMAN_VALUE, AI_VALUE = \"human\", \"ai\" # use \"0\"/\"1\" etc. if labels are numeric\n", "\n", "N_PAIRS = 2000 # more = better, but synthetic generation takes longer\n", "MIN_WORDS, MAX_WORDS = 40, 180\n", "SMOKE_TEST = True # True = tiny 20-step run to check the loss. Set False for the real run.\n", "\n", "print(\"GPUs:\", torch.cuda.device_count(), [torch.cuda.get_device_name(i) for i in range(torch.cuda.device_count())])" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## 1. Load the dataset and inspect it" ] }, { "cell_type": "code", "metadata": {}, "execution_count": null, "outputs": [], "source": [ "csvs = glob.glob(\"/kaggle/input/**/*.csv\", recursive=True)\n", "print(csvs)\n", "CSV_PATH = csvs[0] # change if several CSVs are listed\n", "df = pd.read_csv(CSV_PATH)\n", "print(df.shape)\n", "print(df.columns.tolist())\n", "display(df.head(3))\n", "assert TEXT_COL in df.columns and LABEL_COL in df.columns, f\"Fix TEXT_COL / LABEL_COL. Columns are: {df.columns.tolist()}\"\n", "print(df[LABEL_COL].value_counts())" ] }, { "cell_type": "code", "metadata": {}, "execution_count": null, "outputs": [], "source": [ "is_human = df[LABEL_COL].astype(str).str.lower() == str(HUMAN_VALUE).lower()\n", "human = df.loc[is_human, TEXT_COL].dropna().astype(str).drop_duplicates()\n", "wc = human.str.split().str.len()\n", "human = human[(wc >= MIN_WORDS) & (wc <= MAX_WORDS)]\n", "print(\"usable human texts:\", len(human))\n", "assert len(human) > 0, \"No human texts found - check HUMAN_VALUE and the label values printed above.\"\n", "human_texts = human.sample(min(N_PAIRS, len(human)), random_state=SEED).tolist()\n", "print(\"using:\", len(human_texts))" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## 2. Load the model (4-bit)" ] }, { "cell_type": "code", "metadata": {}, "execution_count": null, "outputs": [], "source": [ "from transformers import AutoTokenizer, AutoModelForCausalLM, BitsAndBytesConfig\n", "\n", "tokenizer = AutoTokenizer.from_pretrained(MODEL_NAME)\n", "if tokenizer.pad_token is None:\n", " tokenizer.pad_token = tokenizer.eos_token\n", "\n", "bnb = BitsAndBytesConfig(\n", " load_in_4bit=True, bnb_4bit_quant_type=\"nf4\",\n", " bnb_4bit_compute_dtype=torch.float16, # T4/P100 have no bf16\n", " bnb_4bit_use_double_quant=True,\n", ")\n", "model = AutoModelForCausalLM.from_pretrained(MODEL_NAME, quantization_config=bnb, device_map=\"auto\")\n", "print(\"loaded\", MODEL_NAME)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## 3. Build synthetic pairs: human text -> \"AI-style\" version\n", "The model rewrites each human text in stiff, generic AI style. We then flip it: input = AI-style, target = the real human text." ] }, { "cell_type": "code", "metadata": {}, "execution_count": null, "outputs": [], "source": [ "AI_STYLE_SYS = (\n", " \"Rewrite the user's text the way a generic AI assistant would: formal, polished, balanced and \"\n", " \"slightly verbose, using phrases like 'it is important to note' and transitions like 'Moreover' or \"\n", " \"'Furthermore'. No slang, no typos, no contractions, no personal quirks. Keep exactly the same meaning \"\n", " \"and a similar length. Output ONLY the rewritten text.\"\n", ")\n", "\n", "@torch.no_grad()\n", "def to_ai_style(texts, bs=8, max_new_tokens=320):\n", " tokenizer.padding_side = \"left\"\n", " order = np.argsort([len(t) for t in texts]) # batch similar lengths together\n", " results = [None] * len(texts)\n", " for i in tqdm(range(0, len(texts), bs)):\n", " idx = order[i:i+bs]\n", " prompts = [tokenizer.apply_chat_template(\n", " [{\"role\": \"system\", \"content\": AI_STYLE_SYS}, {\"role\": \"user\", \"content\": texts[j]}],\n", " tokenize=False, add_generation_prompt=True) for j in idx]\n", " enc = tokenizer(prompts, return_tensors=\"pt\", padding=True, truncation=True, max_length=768).to(model.device)\n", " out = model.generate(**enc, max_new_tokens=max_new_tokens, do_sample=True, temperature=0.7,\n", " top_p=0.9, pad_token_id=tokenizer.pad_token_id)\n", " dec = tokenizer.batch_decode(out[:, enc[\"input_ids\"].shape[1]:], skip_special_tokens=True)\n", " for j, d in zip(idx, dec):\n", " results[j] = d.strip()\n", " return results\n", "\n", "PAIRS_CSV = \"/kaggle/working/pairs.csv\"\n", "if os.path.exists(PAIRS_CSV):\n", " pairs_df = pd.read_csv(PAIRS_CSV)\n", " print(\"loaded cached pairs:\", len(pairs_df))\n", "else:\n", " ai_versions = to_ai_style(human_texts)\n", " pairs_df = pd.DataFrame({\"ai_text\": ai_versions, \"human_text\": human_texts})\n", " pairs_df.to_csv(PAIRS_CSV, index=False)\n", " print(\"saved\", len(pairs_df))\n", "torch.cuda.empty_cache(); gc.collect()" ] }, { "cell_type": "code", "metadata": {}, "execution_count": null, "outputs": [], "source": [ "# Clean: drop empty / too short / unchanged generations\n", "pairs_df = pairs_df.dropna()\n", "pairs_df = pairs_df[pairs_df.ai_text.str.split().str.len() >= 20]\n", "pairs_df = pairs_df[pairs_df.ai_text.str.strip() != pairs_df.human_text.str.strip()]\n", "print(\"clean pairs:\", len(pairs_df))\n", "\n", "for _, r in pairs_df.head(2).iterrows():\n", " print(\"AI-STYLE:\", r.ai_text, \"\\nHUMAN :\", r.human_text, \"\\n\" + \"-\"*80)\n", "\n", "split = Dataset.from_pandas(pairs_df.reset_index(drop=True)).train_test_split(test_size=0.05, seed=SEED)\n", "train_ds, eval_ds = split[\"train\"], split[\"test\"]\n", "print(len(train_ds), len(eval_ds))" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## 4. Add LoRA and tokenize (loss only on the human completion)" ] }, { "cell_type": "code", "metadata": {}, "execution_count": null, "outputs": [], "source": [ "from peft import LoraConfig, get_peft_model, prepare_model_for_kbit_training\n", "\n", "model = prepare_model_for_kbit_training(model, use_gradient_checkpointing=True)\n", "lora_config = LoraConfig(\n", " r=32, lora_alpha=64, lora_dropout=0.05, bias=\"none\", task_type=\"CAUSAL_LM\",\n", " target_modules=[\"q_proj\", \"k_proj\", \"v_proj\", \"o_proj\", \"gate_proj\", \"up_proj\", \"down_proj\"], # Qwen/Llama names\n", ")\n", "model = get_peft_model(model, lora_config)\n", "model.print_trainable_parameters()" ] }, { "cell_type": "code", "metadata": {}, "execution_count": null, "outputs": [], "source": [ "INSTR = \"Rewrite the following text so it sounds natural and human-written. Keep the same meaning.\\n\\n{text}\"\n", "\n", "def build_prompt(text):\n", " return tokenizer.apply_chat_template(\n", " [{\"role\": \"user\", \"content\": INSTR.format(text=text)}], tokenize=False, add_generation_prompt=True)\n", "\n", "MAX_PROMPT, MAX_COMP = 512, 320\n", "tokenizer.padding_side = \"right\"\n", "\n", "def tokenize_fn(batch):\n", " out = {\"input_ids\": [], \"attention_mask\": [], \"labels\": []}\n", " for ai, hu in zip(batch[\"ai_text\"], batch[\"human_text\"]):\n", " p = tokenizer(build_prompt(ai), add_special_tokens=False)[\"input_ids\"][:MAX_PROMPT]\n", " c = tokenizer(hu, add_special_tokens=False)[\"input_ids\"][:MAX_COMP] + [tokenizer.eos_token_id]\n", " ids = p + c\n", " out[\"input_ids\"].append(ids)\n", " out[\"attention_mask\"].append([1] * len(ids))\n", " out[\"labels\"].append([-100] * len(p) + c) # prompt masked, completion learned\n", " return out\n", "\n", "tok_train = train_ds.map(tokenize_fn, batched=True, remove_columns=train_ds.column_names)\n", "tok_eval = eval_ds.map(tokenize_fn, batched=True, remove_columns=eval_ds.column_names)\n", "\n", "# Sanity check: labels must NOT be all -100\n", "ex = tok_train[0]\n", "n_learn = sum(l != -100 for l in ex[\"labels\"])\n", "print(\"tokens:\", len(ex[\"input_ids\"]), \"| learned tokens:\", n_learn)\n", "assert n_learn > 0\n", "print(tokenizer.decode([l for l in ex[\"labels\"] if l != -100])[:300])" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## 5. Train\n", "Run once with `SMOKE_TEST = True` (loss should be a real number and drop). Then set it to `False` and re-run this cell for the full run." ] }, { "cell_type": "code", "metadata": {}, "execution_count": null, "outputs": [], "source": [ "from transformers import TrainingArguments, Trainer, DataCollatorForSeq2Seq\n", "\n", "args = TrainingArguments(\n", " output_dir=\"/kaggle/working/humanize-e-ckpt\",\n", " num_train_epochs=2,\n", " max_steps=20 if SMOKE_TEST else -1,\n", " per_device_train_batch_size=2, per_device_eval_batch_size=2,\n", " gradient_accumulation_steps=8,\n", " learning_rate=2e-4, lr_scheduler_type=\"cosine\", warmup_ratio=0.05,\n", " weight_decay=0.01,\n", " fp16=True, # use bf16=True instead on L4/A100\n", " optim=\"paged_adamw_8bit\",\n", " gradient_checkpointing=True, gradient_checkpointing_kwargs={\"use_reentrant\": False},\n", " eval_strategy=\"steps\", eval_steps=50,\n", " save_strategy=\"steps\", save_steps=100, save_total_limit=2,\n", " logging_steps=5, group_by_length=False,\n", " remove_unused_columns=False, report_to=\"none\",\n", ")\n", "\n", "trainer = Trainer(\n", " model=model, args=args,\n", " train_dataset=tok_train, eval_dataset=tok_eval,\n", " data_collator=DataCollatorForSeq2Seq(tokenizer, padding=True, label_pad_token_id=-100),\n", ")\n", "trainer.train()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## 6. Save the LoRA adapter" ] }, { "cell_type": "code", "metadata": {}, "execution_count": null, "outputs": [], "source": [ "SAVE_DIR = \"/kaggle/working/humanize-e-adapter\"\n", "trainer.model.save_pretrained(SAVE_DIR)\n", "tokenizer.save_pretrained(SAVE_DIR)\n", "print(\"saved to\", SAVE_DIR)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## 7. Try it (same prompt template as training)" ] }, { "cell_type": "code", "metadata": {}, "execution_count": null, "outputs": [], "source": [ "model.eval()\n", "\n", "@torch.no_grad()\n", "def humanize(text, max_new_tokens=320, temperature=0.8):\n", " enc = tokenizer(build_prompt(text), return_tensors=\"pt\", add_special_tokens=False).to(model.device)\n", " out = model.generate(**enc, max_new_tokens=max_new_tokens, do_sample=True, temperature=temperature,\n", " top_p=0.9, repetition_penalty=1.05, use_cache=True, pad_token_id=tokenizer.pad_token_id)\n", " return tokenizer.decode(out[0][enc[\"input_ids\"].shape[1]:], skip_special_tokens=True).strip()\n", "\n", "tests = [\n", " \"In order to provide you with the most comprehensive solution, it is important to note that we have conducted extensive research on this topic.\",\n", " \"Moreover, the analysis reveals several important implications that should be carefully considered by all stakeholders.\",\n", "]\n", "for t in tests:\n", " print(\"AI :\", t); print(\"HUMAN:\", humanize(t), \"\\n\")" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## 8. Rough quality check with the Kaggle dataset as a detector\n", "We train a quick TF-IDF classifier on the dataset (AI vs human), **excluding the held-out eval texts**, and compare its \"AI probability\" before vs after humanizing. This is only a proxy. It does not predict how real detectors (GPTZero, Turnitin, etc.) will score the text." ] }, { "cell_type": "code", "metadata": {}, "execution_count": null, "outputs": [], "source": [ "from sklearn.feature_extraction.text import TfidfVectorizer\n", "from sklearn.linear_model import LogisticRegression\n", "\n", "d = df[[TEXT_COL, LABEL_COL]].dropna()\n", "d = d[~d[TEXT_COL].astype(str).isin(set(eval_ds[\"human_text\"]))]\n", "y = (d[LABEL_COL].astype(str).str.lower() == str(AI_VALUE).lower()).astype(int)\n", "vec = TfidfVectorizer(ngram_range=(1, 2), min_df=3, max_features=200000, sublinear_tf=True)\n", "clf = LogisticRegression(max_iter=1000).fit(vec.fit_transform(d[TEXT_COL].astype(str)), y)\n", "\n", "n = min(40, len(eval_ds))\n", "src = eval_ds[\"ai_text\"][:n]\n", "outs = [humanize(t) for t in tqdm(src)]\n", "p_before = clf.predict_proba(vec.transform(src))[:, 1].mean()\n", "p_after = clf.predict_proba(vec.transform(outs))[:, 1].mean()\n", "p_real = clf.predict_proba(vec.transform(eval_ds[\"human_text\"][:n]))[:, 1].mean()\n", "print(f\"P(AI) AI-style input : {p_before:.2f}\")\n", "print(f\"P(AI) model output : {p_after:.2f}\")\n", "print(f\"P(AI) real human text: {p_real:.2f}\")" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## 9. Load the adapter later (new notebook / your app)\n", "```python\n", "import torch\n", "from transformers import AutoTokenizer, AutoModelForCausalLM, BitsAndBytesConfig\n", "from peft import PeftModel\n", "\n", "BASE = \"Qwen/Qwen2.5-3B-Instruct\"\n", "tok = AutoTokenizer.from_pretrained(\"humanize-e-adapter\")\n", "base = AutoModelForCausalLM.from_pretrained(BASE, device_map=\"auto\",\n", " quantization_config=BitsAndBytesConfig(load_in_4bit=True, bnb_4bit_quant_type=\"nf4\", bnb_4bit_compute_dtype=torch.float16))\n", "model = PeftModel.from_pretrained(base, \"humanize-e-adapter\").eval()\n", "# then reuse build_prompt() + humanize() from section 7\n", "```\n", "Upload the adapter folder to Hugging Face with `huggingface_hub.upload_folder(...)`, or download it from `/kaggle/working`.\n", "\n", "**Tips:** if outputs look too similar to the input, raise `N_PAIRS` and train 3 epochs. If they ramble, lower the temperature to 0.6. Mix in other human styles (news, social, academic) by sampling evenly across the dataset's domain column if it has one." ] } ], "metadata": { "kernelspec": { "display_name": "Python 3", "language": "python", "name": "python3" }, "language_info": { "name": "python" } }, "nbformat": 4, "nbformat_minor": 4 }