---
title: Catalyst Humanizer
emoji: ✨
colorFrom: indigo
colorTo: pink
sdk: gradio
sdk_version: 5.49.1
python_version: "3.10"
app_file: app.py
pinned: true
license: apache-2.0
short_description: Detect AI writing and humanize it, Word layout kept
---

# Catalyst Humanizer — by CS Software Solutions

One Space, three tools in a navigation bar:

| Tab | What it does |
|---|---|
| 🚀 **Smart Humanize** | Detector + humanizer together: scores every sentence with the AI detector, rewrites **only the flagged sentences**, re-checks, and repeats until the target AI % or the pass limit is reached. |
| 🔍 **AI Detector** | Highlighted AI-likelihood view (cyan = high, purple = moderate) + PDF report. |
| ✨ **Humanizer** | Full paraphrase of every eligible paragraph (Parrot T5), with the detector score before/after. |
| 📊 **Dashboard** | Signed-in users: totals, words processed, average AI reduction, today's usage vs limits, 14-day chart, full run history, re-download of recent results. |
| 👤 **Account** | Sign in / sign up / profile · **🔒 Change password** · **🔑 Forgot password** (6-digit email code, or recovery key) · create a new recovery key. |
| 🛡️ **Admin** | Only for `ADMIN_EMAILS`. **Overview** (users, runs, guests, errors, job time, AI reduction, CPU fallbacks, 30-day charts, most active users) · **Activity** (every run incl. guests, filter by user/tool/errors, CSV export) · **Users** (search, click a row for full history, per-user daily limits, enable/disable, reset password, delete, CSV export). |

Shortcut buttons link the tools: *Humanize this text →* (Detector → Smart Humanize) and
*Send result to AI Detector →* (Humanizer → Detector).

## Plans (plans.py)

| Plan | Price | Documents (reports) | Paste | Smart passes |
|---|---|---|---|---|
| Guest | free | — | 300 words, 15 runs/day | 2 |
| Free account | free | 2 (lifetime trial) | 300 words, 25 runs/day | 2 |
| Student | ₹4,999 / $59 per month | 5 per day | 2,000 words, 100/day | 4 |
| Agency | ₹7,999 / $95 per month | 10 per day | 3,000 words, 300/day | 4 |
| Pro | ₹10,000 / $119 per month | unlimited | 5,000 words, unlimited | 4 |
| **Admin** (`ADMIN_EMAILS`) | **free, unlimited, never charged** | ∞ | 20,000 words | 4 |

Prices can be changed with the Space variables `PRICE_STUDENT`, `PRICE_AGENCY`, `PRICE_PRO` (₹) and
`PRICE_STUDENT_USD`, `PRICE_AGENCY_USD`, `PRICE_PRO_USD` ($); `PRICE_CURRENCY` (INR / USD) sets which one the Pricing page shows first. Daily limits reset at
midnight IST (`DAY_TZ_OFFSET_MIN`, default 330).

**Monthly subscriptions.** The users table keeps `plan_started` (subscribed since), `period_start` (current month
from), `plan_until` (paid until: the same date next calendar month) and `plan_status` (active / expired / free).
A plan that is not renewed by `plan_until` is switched off automatically (checked every hour and whenever the user
signs in or runs a document): the user goes back to the free plan, gets an email, and the event is written to
`plan_history`. Renewing before the end date adds a month to the end date; renewing after it starts a new
subscription from that day. Admin → Users shows every subscription (start date, paid until, days left, status) and
the full plan history.

**Activating / renewing a plan today:** Admin → Users → 💎 Plans & credits → choose the plan and number of months →
Activate plan. Document credits can still be added there for special cases.

**Razorpay (next step):** in the payment webhook, after verifying the signature, call
`plans.activate_plan(email, "student", 1, source="razorpay", amount=...)` (1 = months) or
`plans.add_credits(email, 1, source="razorpay", amount=...)` — the same functions the admin buttons use.

Optional variables: `PAY_WHATSAPP` (digits with country code), `PAY_EMAIL`, `PROMO_TEXT`, `FREE_TRIAL_DOCS` (default 2).

## Access rules

- **Paste text** — open to everyone, no sign-in (limit `MAX_PASTE_WORDS`, default 300 words; longer texts → upload a document).
- **Document upload** (.docx / .pdf / .txt) — requires a free account. Upload tabs show a lock card
  until the user signs in, and the server also rejects uploads from signed-out sessions.

## Setup (Settings → Variables and secrets)

| Name | Type | Purpose |
|---|---|---|
| `HF_TOKEN` | secret | Read token of an account that accepted the terms of the gated `coai/roberta-ai-detector-v2` model |
| `ADMIN_EMAILS` | secret | Comma-separated admin emails |
| `APP_NAME` | variable (optional) | Header title (default "Catalyst Humanizer") |
| `REPORT_BRAND` | variable (optional) | Header text on PDF report pages |
| `MAX_PASTE_WORDS` | variable (optional) | Paste limit |
| `DETECTOR_MODEL`, `PARA_MODEL` | variable (optional) | Swap models (`PARA_MODEL` = paraphraser; the older name `PARROT_MODEL` still works) |
| `USERS_REPO` / `USERS_REPO_TOKEN` | secret | Private dataset (e.g. `you/suite-users`) + write token: backs up accounts so they survive restarts |
| `DAILY_DOC_LIMIT` | variable (optional) | Document runs per signed-in user per day (default 20, 0 = unlimited) |
| `DAILY_TEXT_LIMIT` | variable (optional) | Pasted-text runs per signed-in user per day (default 100) |
| `DAILY_GUEST_TEXT_LIMIT` | variable (optional) | Pasted-text runs per guest (per IP) per day (default 15) |
| `CONCURRENCY` | variable (optional) | Jobs of the same kind running at once (default 3) |
| `QUEUE_SIZE` | variable (optional) | Max waiting jobs per button (default 60) |
| `HISTORY_DAYS` | variable (optional) | How long run history is kept (default 180 days) |
| `BREVO_API_KEY` | secret | Brevo **API key** (`xkeysib-…`, Brevo → SMTP & API → API Keys). Turns on result emails, email reset codes, welcome and plan mails |
| `MAIL_FROM` | variable | Sender address verified in Brevo (e.g. `noreply@cssoftwaresolutions.in`) |
| `MAIL_FROM_NAME`, `MAIL_MAX_MB` | variable (optional) | Sender name; max attachment size (default 8 MB) |

Admins (`ADMIN_EMAILS`) are never limited.

## Email (mailer.py)

Hugging Face Spaces only allow outgoing ports 80/443/8080, so Brevo's SMTP relay (`smtp-relay.brevo.com:587`)
cannot connect from a Space. The app uses Brevo's HTTPS API with the same account instead.
- Document results (output file + PDF report) are emailed to signed-in users (switch off in Account).
- Forgot password → *Email me a 6-digit code* (15 min, 3 codes/hour, 5 tries).
- Welcome mail on sign-up; plan-activated mail when a plan is activated.
- Admin → Users → 📧 Email: status, *Send test email*, delivery log with the exact Brevo error and fix.
- In Brevo: verify the sender, and **Security → Authorised IPs → deactivate IP blocking** (Space IPs change).

## "Result ready" notifications (push.py, static/)

- Customers press **🔔 Notify me** (or any run button) and allow notifications once.
- When a job finishes, the server sends a **Web Push** (VAPID, `push.py`) to their browser's service worker
  (`static/cs-notify-sw.js`), so the notification arrives even when the tab is in the background, the phone is
  locked or they are in another app. The page also shows its own notification, a chime, a flashing tab title and an
  animated toast — all with the same tag, so only one notification appears.
- Needs `cryptography` (in requirements.txt). VAPID keys are created once and stored in the user database
  (backed up with `USERS_REPO`); optional `VAPID_PRIVATE_KEY` / `VAPID_CONTACT` variables.
- Works on the custom domain (www.catalysthumanizer.com), not inside the huggingface.co page (browsers block
  notifications in embedded frames). iPhone: only when the site is added to the home screen.

## Dropped connections

Jobs run in the background on the server, not inside the browser connection. If a customer's connection
drops (mobile data, switching apps, Wi-Fi) the job keeps running; when the page reconnects it asks again
automatically and picks up the same job — progress continues, or the finished result appears at once — with
no restart and no second charge. Usage is counted and the "ready" notification is sent when the job ends, even
if the page was closed. Unclaimed results are kept for 3 hours (`JOB_KEEP_SECONDS`).

## Passwords

- **Recovery key**: shown once when an account is created (format `XXXX-XXXX-XXXX-XXXX`, stored only as a hash).
  Forgot password = email + recovery key + new password; the used key is replaced by a new one, shown on screen.
- **Change password**: Account → 🔒 Change password (current + new password).
- **New recovery key**: Account → 🔑 Recovery key (cancels the old key). Accounts created before this update
  have no key until the user creates one here.
- **No key?** Forgot password → *Ask the admin*: the user leaves a WhatsApp/phone number; the request appears in
  Admin → Users → Reset password. Set a temporary password there and send it to them; the request is marked done.
- 5 wrong recovery keys per email per hour, then the form waits an hour.

## Multiple users / commercial hosting

- The app runs up to `CONCURRENCY` (default 4) jobs of each tool at the same time, verified with 3 simultaneous users.
- **ZeroGPU is not suitable for paying customers**: it caps GPU slots per Space (users wait "for a GPU slot on
  this Space") and visitors who are not logged in to Hugging Face get only ~2 minutes of GPU a day.
- **For launch use dedicated hardware** (Settings → Space hardware), e.g. *Nvidia T4 small* ($0.40/hour), and set
  **Sleep time → Don't sleep** so customers never hit a cold start. No code change is needed: the app detects
  ZeroGPU only through `SPACES_ZERO_GPU` and otherwise uses the GPU directly (or the CPU on CPU hardware).
- **CPU Upgrade (8 vCPU / 32 GB)** works too: jobs go straight to the CPU and share the cores through
  `runtime.py` (`CPU_JOBS` jobs at once, default = cores ÷ 2 = 4). Pastes stay fast; long documents take minutes.
- One Space = one machine. Beyond ~20–30 simultaneous heavy jobs, move the two models to an autoscaling endpoint.

## Automatic proofread, pay per use, yearly plans

**Automatic proofread.** Every humanized text or document (Smart Humanize and Humanizer) now goes through the
final check & proofread by itself; the customer downloads the proofread file. Because that proofread is the final
check, the humanizer's own language-model polish is not run a second time. If the proofread cannot run, the
humanized result is delivered and the message says so. `AUTO_PROOFREAD=0` = only on request (the button stays).
Note: the proofread reads the whole document, so on a document with few flagged paragraphs it costs more than the
old polish of the changed paragraphs only.

**Pay per use.** A signed-in customer without credits or a plan can still run a document (`PAY_PER_USE=0` = the
old "credits needed" block). The job runs, the scores are shown, and the result is locked behind a charge worked
out from what the job used:

    cost  = GPU seconds x COST_GPU_HOUR_USD / 3600          (Hugging Face Nvidia T4 small: 0.40)
          + idle share of the Space                          (x (COST_IDLE_FACTOR - 1); default factor 5)
          + what the language model charged for this job     (the job's [cost] account)
    price = cost x (1 + PRICE_MARGIN/100) x (1 + PRICE_GST/100) x PRICE_USD_INR, rounded up to a rupee

Defaults: margin 200%, GST 18%, 85 rupees to the dollar, smallest charge Rs 10 (`PRICE_MIN_INR`). Example: 5 minutes
of GPU and 20 cents of language model -> cost $0.37 -> Rs 111. A pop-up shows the price; its Pay button opens
`PAY_URL` (a payment page; `{amount}`, `{ref}`, `{email}`, `{item}` are filled in) or, while that is not set, the
WhatsApp / email contact. The customer then presses "I have paid" to download.

*Until a payment gateway is connected, a charge is confirmed by hand:* Admin -> Plans & credits -> Pay-per-use
charges -> enter the reference (CS-XXXXXXXX) -> Mark paid & unlock. A gateway webhook only has to call
`plans.settle_charge(ref, source="razorpay", amount=...)`. Every charge is logged with its full breakdown:
`[charge] CS-... : 5120 words, 300s -> GPU $ + Space share $ + language model $ = cost $; +200% +18% GST = Rs`.
Locked files live in the Space's temporary folder: a Space restart before payment loses them (the customer is told
to contact you with the reference).

**Yearly plans.** Scholar and Pro can be bought for a year at `YEARLY_DISCOUNT` percent (default 20) off twelve
monthly payments: Rs 9,590 and Rs 28,790. The Pricing page shows them and has two more buy buttons. To activate:
Admin -> Plans & credits -> Activate plan with Months = 12.

## Development and testing

```
python -m venv .venv && .venv/Scripts/pip install -r requirements.txt gradio==5.49.1 pytest   # (bin/ on Linux)
python -m pytest tests                  # 79 fast unit tests, no model download, no network
python devtools/run_local.py            # the app on http://localhost:7860
python devtools/e2e.py smart --sample social --passes 3     # real models + Groq on a sample text
python devtools/e2e.py docx             # Word path: checks headings, bold/italic, superscripts, table, references
```

- Secrets for local runs go in `.env` (git-ignored; `KEY=value` lines, same names as the Space secrets).
  `devtools/env.py` loads it; the Space itself keeps using Settings → Variables and secrets.
- `tests/` covers what must never regress: the rewrite guard (numbers, relation signs such as `p < 0.01`,
  citations, brackets, terms, typography), the publication rules, sentence splitting, in-place Word editing with
  formatting, the detector score cache and the final-polish protection.
- `HUM_DEBUG=1` prints every candidate of every sentence with its AI score and the reason each rejected version
  was rejected (server log) — the quickest way to see why a sentence stayed unchanged.
- `SKIP_MODEL_LOAD=1` delays loading the paraphraser and detector to first use (tests, tools).

**What a rewrite may never do** (checked for every version, whatever wrote it): change or drop a number, unit,
relation sign (`<`, `=`, `≥` ...), citation, acronym or protected term; add or duplicate a citation; leave a bracket
open; change negation or the direction of a comparison; strengthen a hedged claim; add a cause ("because",
"therefore") or a "we" the author did not write; split or merge sentences. Characters that language models
substitute (no-break hyphens, curly quotes, the minus sign) are turned back into the author's own typography.

**Meaning check.** After the rules, every rewritten sentence is read once more against the author's sentence by
the language model (`groqai.verify`, `HUM_VERIFY=0` = off). It catches what rules and a similarity score cannot:
a p-value moved to another claim, "increased by 38% compared with the control" turned into "the control was 38%
lower", a dropped method, "but" turned into "so", an association turned into a cause. A sentence that fails gets
the next version that passes (`HUM_VERIFY_ALTS`, default 3), else the author's wording. On 16 subject samples it
replaced about one rewrite in three; expect a somewhat higher AI score in exchange for text that says what the
author said.

**Groq limits.** Groq counts rate limits per model. When a model reaches its per-minute or daily limit the request
goes to the next model in `GROQ_MODELS` at once (then to the next key), and back to the preferred model when its
limit has passed, so one key now gives the quota of every listed model instead of only the first.

**Subjects.** `python devtools/batch.py` runs Smart Humanize on one sample per subject (engineering, computing,
medicine, public health, economics, finance, psychology, education, law, chemistry, biology, environment,
management, history) and writes every sentence before / after with its checks to `devtools/logs/batch.md`.

**Speed.** Detector scores are cached per text and new texts are scored shortest-first (`DETECTOR_CACHE`, default
20,000 texts). On CPU hardware only the int8 copy of the paraphraser is kept in memory, and the wording /
faithfulness models are loaded only when the chosen `PRIORITY` uses them.

## Add money (Pricing tab) and PhonePe

Pricing tab -> **Add money to your wallet**: pick ₹100 / ₹250 / ₹500 / ₹1,000 or type a whole number of rupees
(₹50 to ₹20,000; `₹1,250` and `1250` both work). The customer is sent to PhonePe, pays by UPI / card / net banking and
comes back to the Pricing tab with the balance updated. If they close the page, PhonePe's webhook (and a background
poll) still credits the wallet, exactly once. Without PhonePe credentials the button shows the WhatsApp/UPI fallback.

**Try it locally without credentials:** set the environment variables `PHONEPE_ENV=mock` and
`PUBLIC_URL=http://127.0.0.1:7860`. A fake PhonePe page with **Pay** / **Cancel** buttons is served at
`/phonepe/mock/pay`; no money moves. The simulator switches itself off on a Hugging Face Space (`SPACE_ID` is set
there), so it can never reach customers. Real PhonePe: `PHONEPE_ENV` defaults to `production`; use `sandbox` while testing.

## Files

- `app.py` — UI, navigation bar, sign-in gating, handlers
- `smart.py` — detect → rewrite flagged sentences → re-check loop (in-place Word editing)
- `detector.py` — AI detector + PDF/HTML report
- `docx_report.py` — Word highlighting + LibreOffice conversion
- `humanizer.py` — Parrot paraphraser
- `auth.py` — accounts, sign-in sessions, limits, admin user actions
- `dashboard.py` — run history, user dashboard, admin analytics
- `plans.py` — pricing plans, quotas, credits, upgrade prompts, manual/Razorpay activation
- `textparse.py` — pasted-text parsing, long-sentence handling, rewrite guard
- `mailer.py` — result emails, reset codes, welcome/plan mails (Brevo API)
- `groqai.py` — Groq quality check of Parrot's output (repairs terms/grammar, rejects meaning changes)
- `push.py` — Web Push notifications when a result is ready
- `static/` — notification service worker + icon
- `runtime.py` — CPU core sharing between simultaneous users (CPU hardware)
- `packages.txt` — LibreOffice + fonts (Word → PDF reports)
- `logo.png`, `favicon.png`

## Groq quality check (groqai.py) — optional, recommended

**Parrot does all the rewriting.** With the `GROQ_API_KEY` secret set, Groq then checks every sentence Parrot changed:
it gets the original (A) and Parrot's paraphrase (B) and makes only the minimum edits to B — puts back any
technical term, acronym, number, unit or citation exactly as in A, fixes broken grammar and casual wording, and
keeps Parrot's phrasing everywhere else. If B changed the meaning and cannot be repaired, it is rejected and the
original sentence is kept. The local guard checks Groq's result once more. Without Groq (no key, quota used up,
network error) the local checks alone are used, exactly as before.
Admin → Users → ✅ Groq quality check shows the model, checked / repaired / rejected counts, tokens and last error.

| Variable | Default | Purpose |
|---|---|---|
| `GROQ_API_KEY` | — | secret; turns the check on |
| `GROQ_MODELS` | `openai/gpt-oss-120b,openai/gpt-oss-20b` | first model available on the key is used; the next one when it is at a rate limit. Keep expensive models out of this list: `qwen/qwen3.8-27b` costs about six times more per token |
| `GROQ_LIGHT_MODEL` / `GROQ_LIGHT_KINDS` | `openai/gpt-oss-20b` / `verify,grammar` | half-price model for the tasks that only judge or correct a sentence; `GROQ_LIGHT_MODEL=` (empty) = main model for everything |
| `GROQ_JOB_BUDGET` | `1.00` | ceiling in US$ for one job; past it the job continues with the local checks (0 = no ceiling) |
| `GROQ_PRICES` | built-in table | `model=in/out` per million tokens, comma separated, when Groq changes its prices |
| `POLISH_CHANGED_ONLY` | `1` | the final polish is sent only for paragraphs the tool changed; `0` = every paragraph |
| `GROQ_BATCH` / `GROQ_MAX_WAIT` | `10` / `60` | sentence pairs per request / max seconds to wait on rate limits |

**What it costs (measured, October 2026 prices: gpt-oss-120b $0.15 in / $0.60 out per million tokens, gpt-oss-20b
half of that; input read from Groq's prompt cache is billed at half price).** One full pass of every task over
5,000 words is about 6-7 US cents: the natural-style rewrites are a little over half of it (they return several
versions of each sentence), the quality check and final polish about 15% each, the meaning check and grammar check
together under 10%. A fully AI-written 5,000-word document that needs 2-4 passes therefore costs roughly 10-25
cents; a document with only some flagged paragraphs costs proportionally less, because only flagged sentences are
rewritten and only changed paragraphs are polished. Every job prints its own bill in the Space log:
`[cost] Smart Humanize, 5120 words: 14 requests, 61.2k tokens in (38% from cache) + 19.8k out = $0.0191 | variants ...`.
The free tier (200K tokens per model per day) covers only a handful of documents.
Customer text is sent to Groq — mention this
in your privacy policy.

## Smart Humanize details

- **Best of 3**: Parrot samples 3 paraphrases of every sentence in one pass (`HUM_CANDIDATES`, default 3); lost
  technical terms are copied back in place, broken versions are dropped, and the AI detector picks the most
  human-sounding survivor (`HUM_SELECT=0` turns the detector choice off). Groq then checks the chosen sentence.
- **100% AI-written box**: Parrot writes 5 versions of every sentence instead of 3 (`HUM_CANDIDATES_AI`), the
  plain-wording rules add more options, the detector ranks them, and Groq checks the 2 most human versions
  (`HUM_GROQ_SHORTLIST`). The most human version Groq accepts is kept; if both are rejected, the next
  version or the original sentence is used. Parrot still does all the rewriting; Groq only checks.
  It also aims for 70% new wording per sentence (`HUM_AI_CHANGE_TARGET`, same measure as the "Vocabulary
  changed" gauge): sentences below it get up to 2 extra Parrot rounds with more variation (`HUM_AI_ROUNDS`),
  versions that reach it are preferred, and Groq's check may not pull the wording back toward the original.
  In Smart Humanize each pass already rewrites again, so the inner extra rounds drop to 1 (`SMART_AI_ROUNDS`)
  and the separate retry is skipped in this mode. Speed settings: `HUM_GPU_BATCH` (32), `GROQ_PARALLEL` (5),
  `GROQ_BATCH` (12).
- Speed: sentences are batched shortest-first and generation stops just past the input length; Groq batches run
  3 at a time (`GROQ_PARALLEL`).

- Sentences with P(AI) ≥ the moderate threshold (default 0.50) are rewritten; each later pass uses
  slightly bolder variation.
- A rewrite is rejected (original kept) if it drops numbers or changes length drastically; dropped
  citations like `[12]` or `(Kumar et al., 2021)` are re-attached.
- **Technical terms are protected**: a rewrite is rejected if it changes an acronym/symbol/model name
  (CNN, LiFePO4, ResNet-50, 5G, α), a term defined in the document ("phase change material (PCM)"), a phrase
  or word the document uses repeatedly ("state of charge", "battery"), or a word whose synonyms change the
  meaning ("accuracy", "model", "layer", "voltage"…). Rejected sentences get one calmer retry; if that also
  damages a term, the original sentence is kept.
- In Word files only the flagged sentence's text is replaced; the rest of the paragraph keeps its
  formatting. Citation fields, hyperlinks and superscript/subscript runs are never edited — the prose
  around them is rewritten instead.

Scores are indicators from an open-source detector and will differ from Turnitin, GPTZero, etc.

## Licences

Paraphraser: humarin/chatgpt_paraphraser_on_T5_base, OpenRAIL (default) · Parrot paraphraser (fallback): Apache-2.0 · RoBERTa AI detector: Apache-2.0.

## Final grammar check (Groq)
After all rewriting is finished, Groq corrects grammar, spelling and punctuation only (no rewording). A correction is
kept only if it changes at most 25% of a sentence's words (`HUM_GRAMMAR_MAX_EDIT`) and keeps every technical term,
number and citation. Smart Humanize runs it once on the final version. `HUM_GRAMMAR=0` turns it off. Without a Groq
key the step is skipped.

## Report-guided rewriting (Smart Humanize) — image processing
Upload an AI or similarity report PDF from any checker (Turnitin, iThenticate, GPTZero, ZeroGPT, Copyleaks, Academi,
scanned reports too). `report_flags.py` renders every page to an image and, for every word, looks at the pixels in
the word's box: a tinted background (cyan, purple, yellow ... even very pale tints) = highlighted; coloured letters =
coloured text. Word positions come from the PDF text layer, or from Tesseract OCR for scanned pages (parallel).
Flagged words are grouped into the report's sentences (cyan = High, purple = Moderate), matched to the document
(word-trigram overlap >= 50%) and only those sentences are rewritten in one pass. The result shows the report's own
score -> an estimated score, and a table of every flagged sentence with its rephrased version.
Flagged sentences get the strongest rewrite: `restructure.py` reshapes them first (clause order, splits,
stock openers; rule-based, no AI model), then Parrot with the 100% AI settings and a 60% new-wording target.
Rows where less than half the wording changed are marked "Likely still flagged" for the writer to revise.
Needs `tesseract-ocr` (packages.txt) and `pytesseract` (requirements.txt) for scanned reports.


## Final polish (Groq)

After all rewriting, the finished text goes through a final polish, paragraph by paragraph. Groq sees each
paragraph together with the author's ORIGINAL paragraph. **Unclear or garbled sentences are repaired** to the
original meaning (clarity wins over the AI score for these); a repair that adds facts or drifts in meaning is
rejected and the clear original sentence is put back. Clear sentences get the light polish: grammar, technical
academic wording (wrong-sense or informal words, redundancy) and flow between sentences (logical links, varied
openings, smoother word order). Sentences are never merged, split, dropped or reordered, so the Word layout stays
intact. A polished sentence is kept only if it keeps every term, number and citation, keeps its meaning (meaning
check), changes at most `POLISH_MAX_EDIT` of its words and, with `POLISH_KEEP_HUMAN=1`, does not turn a sentence
the detector reads as human into one it reads as AI-written; otherwise the grammar-only correction is used.
LanguageTool then fixes any remaining mechanical errors.

**The polish never undoes the humanization.** The editor model tends to call a correctly humanized sentence
"unclear" and rewrite it back towards the original. A repair is therefore accepted as a repair only when the
sentence really fails the clarity checks against the author's sentence (defects, structure, grammar, meaning). A
sentence that passes them is clear: its "repair" is treated as an optional edit, kept only if it does not raise the
AI score of the sentence read with its neighbours, and all optional edits of a paragraph are undone together if the
paragraph as a whole reads as AI-written again (`POLISH_KEEP_HUMAN=0` switches this protection off).

| Variable | Default | Effect |
|---|---|---|
| `POLISH` | `1` | `0` = grammar-only final check |
| `POLISH_MAX_EDIT` | `0.6` | Largest share of a sentence's words the polish may change |
| `POLISH_KEEP_HUMAN` | `1` | `0` = always keep the polished wording, even if the AI score rises |

## Token use and key distribution (groqai.py)

- **Changes-only replies**: the grammar check, quality check and polish return only the sentences they changed;
  every other id is listed under `"unchanged"`. Ids missing from both lists are retried, so nothing is skipped.
  This cuts output tokens by roughly 80% on a typical document.
- **Size-based batches**: a request holds up to `GROQ_BATCH` sentences (default 20) and about `GROQ_BATCH_CHARS`
  characters of text (default 6000), so the fixed system prompt is shared by more sentences (about 40% fewer requests).
- **Token-aware key choice**: each Groq reply reports the key's remaining tokens per minute; the next request goes
  to the key with the most room (requests in flight are booked against it), and waits only when no key has room.
- **Right-sized `max_tokens`**: sized to the reply, plus a separate allowance for the hidden reasoning of
  reasoning models (`GROQ_REASONING`: low / medium / high). A reply cut off at the limit is retried once with a
  larger budget, and complete items are salvaged from a cut-off reply.
- **Cache**: a sentence already checked (grammar / quality check) is not sent again (`GROQ_CACHE`, default 20000).
- Admin → Groq status now shows tokens per key and cache hits.

## Rigorous proofread: quality settings

- Repairs of unclear or garbled sentences are judged against the author's original sentence (they used to be
  rejected for not resembling the garbled text). They count as required fixes and are never undone or reworded
  by the AI-score steps.
- A sentence with a grammar error gets the full correction when its AI rise stays within `PROOF_REQ_RISE`;
  otherwise it gets the minimal grammar fix.
- The full proofread runs with `PROOF_REASONING` (default `medium`); the grammar audit stays on `low`.

| Variable | Default | Effect |
|---|---|---|
| `PROOF_KEEP_RISE` | `0.05` | Largest AI rise allowed for a style-only edit (was 0.03) |
| `PROOF_ALT_RISE` | `0.10` | A corrected sentence is reworded for the AI score only above this rise |
| `PROOF_MAX_DOC_RISE` | `2.0` | Document AI points allowed before style edits are undone (was 1.0) |
| `PROOF_VARIANTS` | `3` | Alternative versions per sentence when rewording (was 4) |
| `PROOF_REPAIR_SIM` | `0.60` | Minimum similarity of a repair to the current sentence (original known) |
| `PROOF_REPAIR_SIM_NOREF` | `0.70` | Same, when no original document is given |
| `PROOF_REASONING` | `medium` | Reasoning effort of the full proofread |

## Paraphrase model

Parrot (`prithivida/parrot_paraphraser_on_T5`) is the paraphraser. `PARA_MODEL` (or the older `PARROT_MODEL`) can
name another T5 paraphraser with the same `paraphrase:` input, e.g. `humarin/chatgpt_paraphraser_on_T5_base`
(`PARA_REP_PENALTY`, default 1.0 for Parrot and 1.15 for other models, discourages copying the input back).

## Adversarial Paraphrasing (detector-guided rewriting, adversarial.py)

Technique from Cheng et al. (2025), *Adversarial Paraphrasing: A Universal Attack for Humanizing AI-Generated Text*
(arXiv:2506.07001). While Parrot writes a sentence, the AI detector chooses the words: at every step Parrot proposes
its most likely next tokens, each one is appended to the text so far and scored by the detector, and the most
human-reading one is kept. Parrot keeps the sentence fluent (only its own likely tokens are allowed); the detector
steers the wording. The result is one extra version per sentence next to Parrot's sampled versions, and it goes
through exactly the same checks (terms, numbers, citations, meaning, fluency, clarity gate, Groq), so a damaged
version is never chosen.

| Variable | Default | Effect |
|---|---|---|
| `HUM_ADV` | `ai` | `ai` = "100% AI-written" box and report-guided rewriting · `always` · `off` |
| `ADV_K` | `8` | Candidate words scored per step (the paper uses 50; more = stronger, slower) |
| `ADV_TOP_P` / `ADV_MIN_P` | `0.95` / `0.05` | Only Parrot's likely words are candidates (keeps grammar intact) |
| `ADV_EVERY` | `1` | Guide every n-th word (`2` halves the detector cost) |
| `ADV_BATCH` / `ADV_MAX` | `16` / `400` | Sentences decoded together / most sentence pieces per job |
| `ADV_ON_ZERO` | `0` | ZeroGPU hardware: off (it would need nested GPU calls); `1` = run on CPU |

Cost: about 2-3x the paraphrasing time for the sentences it is used on (dedicated GPU). Admin → Users shows its
counters next to the Groq status.

## Protection against construct renaming and other rewrite errors (textparse.py)

Checked on every rewrite, in every mode (including report-guided rewriting, where ordinary words are free):
- **Study constructs are locked.** Terms defined with an abbreviation ("travel satisfaction (TS)") and the Keywords
  line are read from the whole document; a rewrite that drops one, or brings in an everyday synonym for it
  ("pleasure"/"contentment" for satisfaction, "closeness" for intimacy, "allegiance" for loyalty), is rejected.
- **Abbreviations** may not change ("EWB" -> "EWD") or appear from nowhere.
- **Statistics symbols** (< > ≤ ≥ = ± ×) must survive ("p < .001" never becomes "p .001").
- **Brackets** must stay balanced ("(TS amongst ...").
- **Foreign words** ("well-être") and **off-topic vocabulary** (2+ uncommon words that occur nowhere in the
  document, e.g. "organizational recruitment" in a tourism paper; `HUM_OFFTOPIC`, 0 = off) are rejected.
A rejected rewrite falls back to another checked version, or to the author's sentence.