Spaces:
Configuration error
Configuration error
File size: 16,959 Bytes
6595ce6 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340 341 342 343 344 345 346 347 348 349 350 351 352 353 354 355 356 357 358 359 360 361 362 363 364 365 366 367 368 369 370 371 372 373 374 375 376 377 378 379 380 381 382 383 384 385 386 387 388 389 390 391 392 393 394 395 396 397 398 399 400 401 402 403 404 405 406 407 408 409 410 411 412 413 | {
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# humanize-e: AI text -> human-sounding text (QLoRA fine-tune)\n",
"\n",
"**Kaggle settings before you run:** Accelerator = **GPU T4 x2** (or P100). Internet = **On** (to download the model) unless you attach the model as an Input.\n",
"\n",
"**How this works.** The Kaggle dataset is *not* paired (an AI text and a human text are about different things), so you can't train on it directly. Instead we use the **human texts as the target**, have an instruct model rewrite each one into stiff \"AI style\" (synthetic input), then train: `AI-style text -> original human text`. Every pair has the same meaning, so the model learns style only."
]
},
{
"cell_type": "code",
"metadata": {},
"execution_count": null,
"outputs": [],
"source": [
"!pip install -q -U transformers accelerate peft bitsandbytes datasets scikit-learn"
]
},
{
"cell_type": "code",
"metadata": {},
"execution_count": null,
"outputs": [],
"source": [
"import os, glob, random, gc\n",
"import numpy as np, pandas as pd, torch\n",
"from datasets import Dataset\n",
"from tqdm.auto import tqdm\n",
"\n",
"SEED = 42\n",
"random.seed(SEED); np.random.seed(SEED); torch.manual_seed(SEED)\n",
"\n",
"# ---- Base model: pick ONE ----\n",
"# A) Internet ON: downloads from Hugging Face\n",
"MODEL_NAME = \"Qwen/Qwen2.5-3B-Instruct\"\n",
"# B) Internet OFF: Add Input -> Models -> \"Qwen2.5 3B Instruct\", then paste the folder that contains config.json\n",
"# MODEL_NAME = \"/kaggle/input/qwen2.5/transformers/3b-instruct/1\"\n",
"\n",
"# ---- Dataset columns: check the printout in the next cell and edit if needed ----\n",
"TEXT_COL, LABEL_COL = \"text_content\", \"label\"\n",
"HUMAN_VALUE, AI_VALUE = \"human\", \"ai\" # use \"0\"/\"1\" etc. if labels are numeric\n",
"\n",
"N_PAIRS = 2000 # more = better, but synthetic generation takes longer\n",
"MIN_WORDS, MAX_WORDS = 40, 180\n",
"SMOKE_TEST = True # True = tiny 20-step run to check the loss. Set False for the real run.\n",
"\n",
"print(\"GPUs:\", torch.cuda.device_count(), [torch.cuda.get_device_name(i) for i in range(torch.cuda.device_count())])"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## 1. Load the dataset and inspect it"
]
},
{
"cell_type": "code",
"metadata": {},
"execution_count": null,
"outputs": [],
"source": [
"csvs = glob.glob(\"/kaggle/input/**/*.csv\", recursive=True)\n",
"print(csvs)\n",
"CSV_PATH = csvs[0] # change if several CSVs are listed\n",
"df = pd.read_csv(CSV_PATH)\n",
"print(df.shape)\n",
"print(df.columns.tolist())\n",
"display(df.head(3))\n",
"assert TEXT_COL in df.columns and LABEL_COL in df.columns, f\"Fix TEXT_COL / LABEL_COL. Columns are: {df.columns.tolist()}\"\n",
"print(df[LABEL_COL].value_counts())"
]
},
{
"cell_type": "code",
"metadata": {},
"execution_count": null,
"outputs": [],
"source": [
"is_human = df[LABEL_COL].astype(str).str.lower() == str(HUMAN_VALUE).lower()\n",
"human = df.loc[is_human, TEXT_COL].dropna().astype(str).drop_duplicates()\n",
"wc = human.str.split().str.len()\n",
"human = human[(wc >= MIN_WORDS) & (wc <= MAX_WORDS)]\n",
"print(\"usable human texts:\", len(human))\n",
"assert len(human) > 0, \"No human texts found - check HUMAN_VALUE and the label values printed above.\"\n",
"human_texts = human.sample(min(N_PAIRS, len(human)), random_state=SEED).tolist()\n",
"print(\"using:\", len(human_texts))"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## 2. Load the model (4-bit)"
]
},
{
"cell_type": "code",
"metadata": {},
"execution_count": null,
"outputs": [],
"source": [
"from transformers import AutoTokenizer, AutoModelForCausalLM, BitsAndBytesConfig\n",
"\n",
"tokenizer = AutoTokenizer.from_pretrained(MODEL_NAME)\n",
"if tokenizer.pad_token is None:\n",
" tokenizer.pad_token = tokenizer.eos_token\n",
"\n",
"bnb = BitsAndBytesConfig(\n",
" load_in_4bit=True, bnb_4bit_quant_type=\"nf4\",\n",
" bnb_4bit_compute_dtype=torch.float16, # T4/P100 have no bf16\n",
" bnb_4bit_use_double_quant=True,\n",
")\n",
"model = AutoModelForCausalLM.from_pretrained(MODEL_NAME, quantization_config=bnb, device_map=\"auto\")\n",
"print(\"loaded\", MODEL_NAME)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## 3. Build synthetic pairs: human text -> \"AI-style\" version\n",
"The model rewrites each human text in stiff, generic AI style. We then flip it: input = AI-style, target = the real human text."
]
},
{
"cell_type": "code",
"metadata": {},
"execution_count": null,
"outputs": [],
"source": [
"AI_STYLE_SYS = (\n",
" \"Rewrite the user's text the way a generic AI assistant would: formal, polished, balanced and \"\n",
" \"slightly verbose, using phrases like 'it is important to note' and transitions like 'Moreover' or \"\n",
" \"'Furthermore'. No slang, no typos, no contractions, no personal quirks. Keep exactly the same meaning \"\n",
" \"and a similar length. Output ONLY the rewritten text.\"\n",
")\n",
"\n",
"@torch.no_grad()\n",
"def to_ai_style(texts, bs=8, max_new_tokens=320):\n",
" tokenizer.padding_side = \"left\"\n",
" order = np.argsort([len(t) for t in texts]) # batch similar lengths together\n",
" results = [None] * len(texts)\n",
" for i in tqdm(range(0, len(texts), bs)):\n",
" idx = order[i:i+bs]\n",
" prompts = [tokenizer.apply_chat_template(\n",
" [{\"role\": \"system\", \"content\": AI_STYLE_SYS}, {\"role\": \"user\", \"content\": texts[j]}],\n",
" tokenize=False, add_generation_prompt=True) for j in idx]\n",
" enc = tokenizer(prompts, return_tensors=\"pt\", padding=True, truncation=True, max_length=768).to(model.device)\n",
" out = model.generate(**enc, max_new_tokens=max_new_tokens, do_sample=True, temperature=0.7,\n",
" top_p=0.9, pad_token_id=tokenizer.pad_token_id)\n",
" dec = tokenizer.batch_decode(out[:, enc[\"input_ids\"].shape[1]:], skip_special_tokens=True)\n",
" for j, d in zip(idx, dec):\n",
" results[j] = d.strip()\n",
" return results\n",
"\n",
"PAIRS_CSV = \"/kaggle/working/pairs.csv\"\n",
"if os.path.exists(PAIRS_CSV):\n",
" pairs_df = pd.read_csv(PAIRS_CSV)\n",
" print(\"loaded cached pairs:\", len(pairs_df))\n",
"else:\n",
" ai_versions = to_ai_style(human_texts)\n",
" pairs_df = pd.DataFrame({\"ai_text\": ai_versions, \"human_text\": human_texts})\n",
" pairs_df.to_csv(PAIRS_CSV, index=False)\n",
" print(\"saved\", len(pairs_df))\n",
"torch.cuda.empty_cache(); gc.collect()"
]
},
{
"cell_type": "code",
"metadata": {},
"execution_count": null,
"outputs": [],
"source": [
"# Clean: drop empty / too short / unchanged generations\n",
"pairs_df = pairs_df.dropna()\n",
"pairs_df = pairs_df[pairs_df.ai_text.str.split().str.len() >= 20]\n",
"pairs_df = pairs_df[pairs_df.ai_text.str.strip() != pairs_df.human_text.str.strip()]\n",
"print(\"clean pairs:\", len(pairs_df))\n",
"\n",
"for _, r in pairs_df.head(2).iterrows():\n",
" print(\"AI-STYLE:\", r.ai_text, \"\\nHUMAN :\", r.human_text, \"\\n\" + \"-\"*80)\n",
"\n",
"split = Dataset.from_pandas(pairs_df.reset_index(drop=True)).train_test_split(test_size=0.05, seed=SEED)\n",
"train_ds, eval_ds = split[\"train\"], split[\"test\"]\n",
"print(len(train_ds), len(eval_ds))"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## 4. Add LoRA and tokenize (loss only on the human completion)"
]
},
{
"cell_type": "code",
"metadata": {},
"execution_count": null,
"outputs": [],
"source": [
"from peft import LoraConfig, get_peft_model, prepare_model_for_kbit_training\n",
"\n",
"model = prepare_model_for_kbit_training(model, use_gradient_checkpointing=True)\n",
"lora_config = LoraConfig(\n",
" r=32, lora_alpha=64, lora_dropout=0.05, bias=\"none\", task_type=\"CAUSAL_LM\",\n",
" target_modules=[\"q_proj\", \"k_proj\", \"v_proj\", \"o_proj\", \"gate_proj\", \"up_proj\", \"down_proj\"], # Qwen/Llama names\n",
")\n",
"model = get_peft_model(model, lora_config)\n",
"model.print_trainable_parameters()"
]
},
{
"cell_type": "code",
"metadata": {},
"execution_count": null,
"outputs": [],
"source": [
"INSTR = \"Rewrite the following text so it sounds natural and human-written. Keep the same meaning.\\n\\n{text}\"\n",
"\n",
"def build_prompt(text):\n",
" return tokenizer.apply_chat_template(\n",
" [{\"role\": \"user\", \"content\": INSTR.format(text=text)}], tokenize=False, add_generation_prompt=True)\n",
"\n",
"MAX_PROMPT, MAX_COMP = 512, 320\n",
"tokenizer.padding_side = \"right\"\n",
"\n",
"def tokenize_fn(batch):\n",
" out = {\"input_ids\": [], \"attention_mask\": [], \"labels\": []}\n",
" for ai, hu in zip(batch[\"ai_text\"], batch[\"human_text\"]):\n",
" p = tokenizer(build_prompt(ai), add_special_tokens=False)[\"input_ids\"][:MAX_PROMPT]\n",
" c = tokenizer(hu, add_special_tokens=False)[\"input_ids\"][:MAX_COMP] + [tokenizer.eos_token_id]\n",
" ids = p + c\n",
" out[\"input_ids\"].append(ids)\n",
" out[\"attention_mask\"].append([1] * len(ids))\n",
" out[\"labels\"].append([-100] * len(p) + c) # prompt masked, completion learned\n",
" return out\n",
"\n",
"tok_train = train_ds.map(tokenize_fn, batched=True, remove_columns=train_ds.column_names)\n",
"tok_eval = eval_ds.map(tokenize_fn, batched=True, remove_columns=eval_ds.column_names)\n",
"\n",
"# Sanity check: labels must NOT be all -100\n",
"ex = tok_train[0]\n",
"n_learn = sum(l != -100 for l in ex[\"labels\"])\n",
"print(\"tokens:\", len(ex[\"input_ids\"]), \"| learned tokens:\", n_learn)\n",
"assert n_learn > 0\n",
"print(tokenizer.decode([l for l in ex[\"labels\"] if l != -100])[:300])"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## 5. Train\n",
"Run once with `SMOKE_TEST = True` (loss should be a real number and drop). Then set it to `False` and re-run this cell for the full run."
]
},
{
"cell_type": "code",
"metadata": {},
"execution_count": null,
"outputs": [],
"source": [
"from transformers import TrainingArguments, Trainer, DataCollatorForSeq2Seq\n",
"\n",
"args = TrainingArguments(\n",
" output_dir=\"/kaggle/working/humanize-e-ckpt\",\n",
" num_train_epochs=2,\n",
" max_steps=20 if SMOKE_TEST else -1,\n",
" per_device_train_batch_size=2, per_device_eval_batch_size=2,\n",
" gradient_accumulation_steps=8,\n",
" learning_rate=2e-4, lr_scheduler_type=\"cosine\", warmup_ratio=0.05,\n",
" weight_decay=0.01,\n",
" fp16=True, # use bf16=True instead on L4/A100\n",
" optim=\"paged_adamw_8bit\",\n",
" gradient_checkpointing=True, gradient_checkpointing_kwargs={\"use_reentrant\": False},\n",
" eval_strategy=\"steps\", eval_steps=50,\n",
" save_strategy=\"steps\", save_steps=100, save_total_limit=2,\n",
" logging_steps=5, group_by_length=False,\n",
" remove_unused_columns=False, report_to=\"none\",\n",
")\n",
"\n",
"trainer = Trainer(\n",
" model=model, args=args,\n",
" train_dataset=tok_train, eval_dataset=tok_eval,\n",
" data_collator=DataCollatorForSeq2Seq(tokenizer, padding=True, label_pad_token_id=-100),\n",
")\n",
"trainer.train()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## 6. Save the LoRA adapter"
]
},
{
"cell_type": "code",
"metadata": {},
"execution_count": null,
"outputs": [],
"source": [
"SAVE_DIR = \"/kaggle/working/humanize-e-adapter\"\n",
"trainer.model.save_pretrained(SAVE_DIR)\n",
"tokenizer.save_pretrained(SAVE_DIR)\n",
"print(\"saved to\", SAVE_DIR)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## 7. Try it (same prompt template as training)"
]
},
{
"cell_type": "code",
"metadata": {},
"execution_count": null,
"outputs": [],
"source": [
"model.eval()\n",
"\n",
"@torch.no_grad()\n",
"def humanize(text, max_new_tokens=320, temperature=0.8):\n",
" enc = tokenizer(build_prompt(text), return_tensors=\"pt\", add_special_tokens=False).to(model.device)\n",
" out = model.generate(**enc, max_new_tokens=max_new_tokens, do_sample=True, temperature=temperature,\n",
" top_p=0.9, repetition_penalty=1.05, use_cache=True, pad_token_id=tokenizer.pad_token_id)\n",
" return tokenizer.decode(out[0][enc[\"input_ids\"].shape[1]:], skip_special_tokens=True).strip()\n",
"\n",
"tests = [\n",
" \"In order to provide you with the most comprehensive solution, it is important to note that we have conducted extensive research on this topic.\",\n",
" \"Moreover, the analysis reveals several important implications that should be carefully considered by all stakeholders.\",\n",
"]\n",
"for t in tests:\n",
" print(\"AI :\", t); print(\"HUMAN:\", humanize(t), \"\\n\")"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## 8. Rough quality check with the Kaggle dataset as a detector\n",
"We train a quick TF-IDF classifier on the dataset (AI vs human), **excluding the held-out eval texts**, and compare its \"AI probability\" before vs after humanizing. This is only a proxy. It does not predict how real detectors (GPTZero, Turnitin, etc.) will score the text."
]
},
{
"cell_type": "code",
"metadata": {},
"execution_count": null,
"outputs": [],
"source": [
"from sklearn.feature_extraction.text import TfidfVectorizer\n",
"from sklearn.linear_model import LogisticRegression\n",
"\n",
"d = df[[TEXT_COL, LABEL_COL]].dropna()\n",
"d = d[~d[TEXT_COL].astype(str).isin(set(eval_ds[\"human_text\"]))]\n",
"y = (d[LABEL_COL].astype(str).str.lower() == str(AI_VALUE).lower()).astype(int)\n",
"vec = TfidfVectorizer(ngram_range=(1, 2), min_df=3, max_features=200000, sublinear_tf=True)\n",
"clf = LogisticRegression(max_iter=1000).fit(vec.fit_transform(d[TEXT_COL].astype(str)), y)\n",
"\n",
"n = min(40, len(eval_ds))\n",
"src = eval_ds[\"ai_text\"][:n]\n",
"outs = [humanize(t) for t in tqdm(src)]\n",
"p_before = clf.predict_proba(vec.transform(src))[:, 1].mean()\n",
"p_after = clf.predict_proba(vec.transform(outs))[:, 1].mean()\n",
"p_real = clf.predict_proba(vec.transform(eval_ds[\"human_text\"][:n]))[:, 1].mean()\n",
"print(f\"P(AI) AI-style input : {p_before:.2f}\")\n",
"print(f\"P(AI) model output : {p_after:.2f}\")\n",
"print(f\"P(AI) real human text: {p_real:.2f}\")"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## 9. Load the adapter later (new notebook / your app)\n",
"```python\n",
"import torch\n",
"from transformers import AutoTokenizer, AutoModelForCausalLM, BitsAndBytesConfig\n",
"from peft import PeftModel\n",
"\n",
"BASE = \"Qwen/Qwen2.5-3B-Instruct\"\n",
"tok = AutoTokenizer.from_pretrained(\"humanize-e-adapter\")\n",
"base = AutoModelForCausalLM.from_pretrained(BASE, device_map=\"auto\",\n",
" quantization_config=BitsAndBytesConfig(load_in_4bit=True, bnb_4bit_quant_type=\"nf4\", bnb_4bit_compute_dtype=torch.float16))\n",
"model = PeftModel.from_pretrained(base, \"humanize-e-adapter\").eval()\n",
"# then reuse build_prompt() + humanize() from section 7\n",
"```\n",
"Upload the adapter folder to Hugging Face with `huggingface_hub.upload_folder(...)`, or download it from `/kaggle/working`.\n",
"\n",
"**Tips:** if outputs look too similar to the input, raise `N_PAIRS` and train 3 epochs. If they ramble, lower the temperature to 0.6. Mix in other human styles (news, social, academic) by sampling evenly across the dataset's domain column if it has one."
]
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3",
"language": "python",
"name": "python3"
},
"language_info": {
"name": "python"
}
},
"nbformat": 4,
"nbformat_minor": 4
} |