Spaces:
Paused
Paused
unknown
Claude Opus 5.5
Product name "Catalyst Humanizer" everywhere (was "CS AI Writing Suite"): the top bar, page title, footer and trademark note, About and Contact pages, emails (sender name), PhonePe payment notes, detector PDF reports and the Space card; CS Software Solutions stays as the company
5a7132a | title: Catalyst Humanizer | |
| emoji: ✨ | |
| colorFrom: indigo | |
| colorTo: pink | |
| sdk: gradio | |
| sdk_version: 5.49.1 | |
| python_version: "3.10" | |
| app_file: app.py | |
| pinned: true | |
| license: apache-2.0 | |
| short_description: Detect AI writing and humanize it, Word layout kept | |
| # Catalyst Humanizer — by CS Software Solutions | |
| One Space, three tools in a navigation bar: | |
| | Tab | What it does | | |
| |---|---| | |
| | 🚀 **Smart Humanize** | Detector + humanizer together: scores every sentence with the AI detector, rewrites **only the flagged sentences**, re-checks, and repeats until the target AI % or the pass limit is reached. | | |
| | 🔍 **AI Detector** | Highlighted AI-likelihood view (cyan = high, purple = moderate) + PDF report. | | |
| | ✨ **Humanizer** | Full paraphrase of every eligible paragraph (Parrot T5), with the detector score before/after. | | |
| | 📊 **Dashboard** | Signed-in users: totals, words processed, average AI reduction, today's usage vs limits, 14-day chart, full run history, re-download of recent results. | | |
| | 👤 **Account** | Sign in / sign up / profile · **🔒 Change password** · **🔑 Forgot password** (6-digit email code, or recovery key) · create a new recovery key. | | |
| | 🛡️ **Admin** | Only for `ADMIN_EMAILS`. **Overview** (users, runs, guests, errors, job time, AI reduction, CPU fallbacks, 30-day charts, most active users) · **Activity** (every run incl. guests, filter by user/tool/errors, CSV export) · **Users** (search, click a row for full history, per-user daily limits, enable/disable, reset password, delete, CSV export). | | |
| Shortcut buttons link the tools: *Humanize this text →* (Detector → Smart Humanize) and | |
| *Send result to AI Detector →* (Humanizer → Detector). | |
| ## Plans (plans.py) | |
| | Plan | Price | Documents (reports) | Paste | Smart passes | | |
| |---|---|---|---|---| | |
| | Guest | free | — | 300 words, 15 runs/day | 2 | | |
| | Free account | free | 2 (lifetime trial) | 300 words, 25 runs/day | 2 | | |
| | Student | ₹4,999 / $59 per month | 5 per day | 2,000 words, 100/day | 4 | | |
| | Agency | ₹7,999 / $95 per month | 10 per day | 3,000 words, 300/day | 4 | | |
| | Pro | ₹10,000 / $119 per month | unlimited | 5,000 words, unlimited | 4 | | |
| | **Admin** (`ADMIN_EMAILS`) | **free, unlimited, never charged** | ∞ | 20,000 words | 4 | | |
| Prices can be changed with the Space variables `PRICE_STUDENT`, `PRICE_AGENCY`, `PRICE_PRO` (₹) and | |
| `PRICE_STUDENT_USD`, `PRICE_AGENCY_USD`, `PRICE_PRO_USD` ($); `PRICE_CURRENCY` (INR / USD) sets which one the Pricing page shows first. Daily limits reset at | |
| midnight IST (`DAY_TZ_OFFSET_MIN`, default 330). | |
| **Monthly subscriptions.** The users table keeps `plan_started` (subscribed since), `period_start` (current month | |
| from), `plan_until` (paid until: the same date next calendar month) and `plan_status` (active / expired / free). | |
| A plan that is not renewed by `plan_until` is switched off automatically (checked every hour and whenever the user | |
| signs in or runs a document): the user goes back to the free plan, gets an email, and the event is written to | |
| `plan_history`. Renewing before the end date adds a month to the end date; renewing after it starts a new | |
| subscription from that day. Admin → Users shows every subscription (start date, paid until, days left, status) and | |
| the full plan history. | |
| **Activating / renewing a plan today:** Admin → Users → 💎 Plans & credits → choose the plan and number of months → | |
| Activate plan. Document credits can still be added there for special cases. | |
| **Razorpay (next step):** in the payment webhook, after verifying the signature, call | |
| `plans.activate_plan(email, "student", 1, source="razorpay", amount=...)` (1 = months) or | |
| `plans.add_credits(email, 1, source="razorpay", amount=...)` — the same functions the admin buttons use. | |
| Optional variables: `PAY_WHATSAPP` (digits with country code), `PAY_EMAIL`, `PROMO_TEXT`, `FREE_TRIAL_DOCS` (default 2). | |
| ## Access rules | |
| - **Paste text** — open to everyone, no sign-in (limit `MAX_PASTE_WORDS`, default 300 words; longer texts → upload a document). | |
| - **Document upload** (.docx / .pdf / .txt) — requires a free account. Upload tabs show a lock card | |
| until the user signs in, and the server also rejects uploads from signed-out sessions. | |
| ## Setup (Settings → Variables and secrets) | |
| | Name | Type | Purpose | | |
| |---|---|---| | |
| | `HF_TOKEN` | secret | Read token of an account that accepted the terms of the gated `coai/roberta-ai-detector-v2` model | | |
| | `ADMIN_EMAILS` | secret | Comma-separated admin emails | | |
| | `APP_NAME` | variable (optional) | Header title (default "Catalyst Humanizer") | | |
| | `REPORT_BRAND` | variable (optional) | Header text on PDF report pages | | |
| | `MAX_PASTE_WORDS` | variable (optional) | Paste limit | | |
| | `DETECTOR_MODEL`, `PARA_MODEL` | variable (optional) | Swap models (`PARA_MODEL` = paraphraser; the older name `PARROT_MODEL` still works) | | |
| | `USERS_REPO` / `USERS_REPO_TOKEN` | secret | Private dataset (e.g. `you/suite-users`) + write token: backs up accounts so they survive restarts | | |
| | `DAILY_DOC_LIMIT` | variable (optional) | Document runs per signed-in user per day (default 20, 0 = unlimited) | | |
| | `DAILY_TEXT_LIMIT` | variable (optional) | Pasted-text runs per signed-in user per day (default 100) | | |
| | `DAILY_GUEST_TEXT_LIMIT` | variable (optional) | Pasted-text runs per guest (per IP) per day (default 15) | | |
| | `CONCURRENCY` | variable (optional) | Jobs of the same kind running at once (default 3) | | |
| | `QUEUE_SIZE` | variable (optional) | Max waiting jobs per button (default 60) | | |
| | `HISTORY_DAYS` | variable (optional) | How long run history is kept (default 180 days) | | |
| | `BREVO_API_KEY` | secret | Brevo **API key** (`xkeysib-…`, Brevo → SMTP & API → API Keys). Turns on result emails, email reset codes, welcome and plan mails | | |
| | `MAIL_FROM` | variable | Sender address verified in Brevo (e.g. `noreply@cssoftwaresolutions.in`) | | |
| | `MAIL_FROM_NAME`, `MAIL_MAX_MB` | variable (optional) | Sender name; max attachment size (default 8 MB) | | |
| Admins (`ADMIN_EMAILS`) are never limited. | |
| ## Email (mailer.py) | |
| Hugging Face Spaces only allow outgoing ports 80/443/8080, so Brevo's SMTP relay (`smtp-relay.brevo.com:587`) | |
| cannot connect from a Space. The app uses Brevo's HTTPS API with the same account instead. | |
| - Document results (output file + PDF report) are emailed to signed-in users (switch off in Account). | |
| - Forgot password → *Email me a 6-digit code* (15 min, 3 codes/hour, 5 tries). | |
| - Welcome mail on sign-up; plan-activated mail when a plan is activated. | |
| - Admin → Users → 📧 Email: status, *Send test email*, delivery log with the exact Brevo error and fix. | |
| - In Brevo: verify the sender, and **Security → Authorised IPs → deactivate IP blocking** (Space IPs change). | |
| ## "Result ready" notifications (push.py, static/) | |
| - Customers press **🔔 Notify me** (or any run button) and allow notifications once. | |
| - When a job finishes, the server sends a **Web Push** (VAPID, `push.py`) to their browser's service worker | |
| (`static/cs-notify-sw.js`), so the notification arrives even when the tab is in the background, the phone is | |
| locked or they are in another app. The page also shows its own notification, a chime, a flashing tab title and an | |
| animated toast — all with the same tag, so only one notification appears. | |
| - Needs `cryptography` (in requirements.txt). VAPID keys are created once and stored in the user database | |
| (backed up with `USERS_REPO`); optional `VAPID_PRIVATE_KEY` / `VAPID_CONTACT` variables. | |
| - Works on the custom domain (www.catalysthumanizer.com), not inside the huggingface.co page (browsers block | |
| notifications in embedded frames). iPhone: only when the site is added to the home screen. | |
| ## Dropped connections | |
| Jobs run in the background on the server, not inside the browser connection. If a customer's connection | |
| drops (mobile data, switching apps, Wi-Fi) the job keeps running; when the page reconnects it asks again | |
| automatically and picks up the same job — progress continues, or the finished result appears at once — with | |
| no restart and no second charge. Usage is counted and the "ready" notification is sent when the job ends, even | |
| if the page was closed. Unclaimed results are kept for 3 hours (`JOB_KEEP_SECONDS`). | |
| ## Passwords | |
| - **Recovery key**: shown once when an account is created (format `XXXX-XXXX-XXXX-XXXX`, stored only as a hash). | |
| Forgot password = email + recovery key + new password; the used key is replaced by a new one, shown on screen. | |
| - **Change password**: Account → 🔒 Change password (current + new password). | |
| - **New recovery key**: Account → 🔑 Recovery key (cancels the old key). Accounts created before this update | |
| have no key until the user creates one here. | |
| - **No key?** Forgot password → *Ask the admin*: the user leaves a WhatsApp/phone number; the request appears in | |
| Admin → Users → Reset password. Set a temporary password there and send it to them; the request is marked done. | |
| - 5 wrong recovery keys per email per hour, then the form waits an hour. | |
| ## Multiple users / commercial hosting | |
| - The app runs up to `CONCURRENCY` (default 4) jobs of each tool at the same time, verified with 3 simultaneous users. | |
| - **ZeroGPU is not suitable for paying customers**: it caps GPU slots per Space (users wait "for a GPU slot on | |
| this Space") and visitors who are not logged in to Hugging Face get only ~2 minutes of GPU a day. | |
| - **For launch use dedicated hardware** (Settings → Space hardware), e.g. *Nvidia T4 small* ($0.40/hour), and set | |
| **Sleep time → Don't sleep** so customers never hit a cold start. No code change is needed: the app detects | |
| ZeroGPU only through `SPACES_ZERO_GPU` and otherwise uses the GPU directly (or the CPU on CPU hardware). | |
| - **CPU Upgrade (8 vCPU / 32 GB)** works too: jobs go straight to the CPU and share the cores through | |
| `runtime.py` (`CPU_JOBS` jobs at once, default = cores ÷ 2 = 4). Pastes stay fast; long documents take minutes. | |
| - One Space = one machine. Beyond ~20–30 simultaneous heavy jobs, move the two models to an autoscaling endpoint. | |
| ## Automatic proofread, pay per use, yearly plans | |
| **Automatic proofread.** Every humanized text or document (Smart Humanize and Humanizer) now goes through the | |
| final check & proofread by itself; the customer downloads the proofread file. Because that proofread is the final | |
| check, the humanizer's own language-model polish is not run a second time. If the proofread cannot run, the | |
| humanized result is delivered and the message says so. `AUTO_PROOFREAD=0` = only on request (the button stays). | |
| Note: the proofread reads the whole document, so on a document with few flagged paragraphs it costs more than the | |
| old polish of the changed paragraphs only. | |
| **Pay per use.** A signed-in customer without credits or a plan can still run a document (`PAY_PER_USE=0` = the | |
| old "credits needed" block). The job runs, the scores are shown, and the result is locked behind a charge worked | |
| out from what the job used: | |
| cost = GPU seconds x COST_GPU_HOUR_USD / 3600 (Hugging Face Nvidia T4 small: 0.40) | |
| + idle share of the Space (x (COST_IDLE_FACTOR - 1); default factor 5) | |
| + what the language model charged for this job (the job's [cost] account) | |
| price = cost x (1 + PRICE_MARGIN/100) x (1 + PRICE_GST/100) x PRICE_USD_INR, rounded up to a rupee | |
| Defaults: margin 200%, GST 18%, 85 rupees to the dollar, smallest charge Rs 10 (`PRICE_MIN_INR`). Example: 5 minutes | |
| of GPU and 20 cents of language model -> cost $0.37 -> Rs 111. A pop-up shows the price; its Pay button opens | |
| `PAY_URL` (a payment page; `{amount}`, `{ref}`, `{email}`, `{item}` are filled in) or, while that is not set, the | |
| WhatsApp / email contact. The customer then presses "I have paid" to download. | |
| *Until a payment gateway is connected, a charge is confirmed by hand:* Admin -> Plans & credits -> Pay-per-use | |
| charges -> enter the reference (CS-XXXXXXXX) -> Mark paid & unlock. A gateway webhook only has to call | |
| `plans.settle_charge(ref, source="razorpay", amount=...)`. Every charge is logged with its full breakdown: | |
| `[charge] CS-... : 5120 words, 300s -> GPU $ + Space share $ + language model $ = cost $; +200% +18% GST = Rs`. | |
| Locked files live in the Space's temporary folder: a Space restart before payment loses them (the customer is told | |
| to contact you with the reference). | |
| **Yearly plans.** Scholar and Pro can be bought for a year at `YEARLY_DISCOUNT` percent (default 20) off twelve | |
| monthly payments: Rs 9,590 and Rs 28,790. The Pricing page shows them and has two more buy buttons. To activate: | |
| Admin -> Plans & credits -> Activate plan with Months = 12. | |
| ## Development and testing | |
| ``` | |
| python -m venv .venv && .venv/Scripts/pip install -r requirements.txt gradio==5.49.1 pytest # (bin/ on Linux) | |
| python -m pytest tests # 79 fast unit tests, no model download, no network | |
| python devtools/run_local.py # the app on http://localhost:7860 | |
| python devtools/e2e.py smart --sample social --passes 3 # real models + Groq on a sample text | |
| python devtools/e2e.py docx # Word path: checks headings, bold/italic, superscripts, table, references | |
| ``` | |
| - Secrets for local runs go in `.env` (git-ignored; `KEY=value` lines, same names as the Space secrets). | |
| `devtools/env.py` loads it; the Space itself keeps using Settings → Variables and secrets. | |
| - `tests/` covers what must never regress: the rewrite guard (numbers, relation signs such as `p < 0.01`, | |
| citations, brackets, terms, typography), the publication rules, sentence splitting, in-place Word editing with | |
| formatting, the detector score cache and the final-polish protection. | |
| - `HUM_DEBUG=1` prints every candidate of every sentence with its AI score and the reason each rejected version | |
| was rejected (server log) — the quickest way to see why a sentence stayed unchanged. | |
| - `SKIP_MODEL_LOAD=1` delays loading the paraphraser and detector to first use (tests, tools). | |
| **What a rewrite may never do** (checked for every version, whatever wrote it): change or drop a number, unit, | |
| relation sign (`<`, `=`, `≥` ...), citation, acronym or protected term; add or duplicate a citation; leave a bracket | |
| open; change negation or the direction of a comparison; strengthen a hedged claim; add a cause ("because", | |
| "therefore") or a "we" the author did not write; split or merge sentences. Characters that language models | |
| substitute (no-break hyphens, curly quotes, the minus sign) are turned back into the author's own typography. | |
| **Meaning check.** After the rules, every rewritten sentence is read once more against the author's sentence by | |
| the language model (`groqai.verify`, `HUM_VERIFY=0` = off). It catches what rules and a similarity score cannot: | |
| a p-value moved to another claim, "increased by 38% compared with the control" turned into "the control was 38% | |
| lower", a dropped method, "but" turned into "so", an association turned into a cause. A sentence that fails gets | |
| the next version that passes (`HUM_VERIFY_ALTS`, default 3), else the author's wording. On 16 subject samples it | |
| replaced about one rewrite in three; expect a somewhat higher AI score in exchange for text that says what the | |
| author said. | |
| **Groq limits.** Groq counts rate limits per model. When a model reaches its per-minute or daily limit the request | |
| goes to the next model in `GROQ_MODELS` at once (then to the next key), and back to the preferred model when its | |
| limit has passed, so one key now gives the quota of every listed model instead of only the first. | |
| **Subjects.** `python devtools/batch.py` runs Smart Humanize on one sample per subject (engineering, computing, | |
| medicine, public health, economics, finance, psychology, education, law, chemistry, biology, environment, | |
| management, history) and writes every sentence before / after with its checks to `devtools/logs/batch.md`. | |
| **Speed.** Detector scores are cached per text and new texts are scored shortest-first (`DETECTOR_CACHE`, default | |
| 20,000 texts). On CPU hardware only the int8 copy of the paraphraser is kept in memory, and the wording / | |
| faithfulness models are loaded only when the chosen `PRIORITY` uses them. | |
| ## Add money (Pricing tab) and PhonePe | |
| Pricing tab -> **Add money to your wallet**: pick ₹100 / ₹250 / ₹500 / ₹1,000 or type a whole number of rupees | |
| (₹50 to ₹20,000; `₹1,250` and `1250` both work). The customer is sent to PhonePe, pays by UPI / card / net banking and | |
| comes back to the Pricing tab with the balance updated. If they close the page, PhonePe's webhook (and a background | |
| poll) still credits the wallet, exactly once. Without PhonePe credentials the button shows the WhatsApp/UPI fallback. | |
| **Try it locally without credentials:** set the environment variables `PHONEPE_ENV=mock` and | |
| `PUBLIC_URL=http://127.0.0.1:7860`. A fake PhonePe page with **Pay** / **Cancel** buttons is served at | |
| `/phonepe/mock/pay`; no money moves. The simulator switches itself off on a Hugging Face Space (`SPACE_ID` is set | |
| there), so it can never reach customers. Real PhonePe: `PHONEPE_ENV` defaults to `production`; use `sandbox` while testing. | |
| ## Files | |
| - `app.py` — UI, navigation bar, sign-in gating, handlers | |
| - `smart.py` — detect → rewrite flagged sentences → re-check loop (in-place Word editing) | |
| - `detector.py` — AI detector + PDF/HTML report | |
| - `docx_report.py` — Word highlighting + LibreOffice conversion | |
| - `humanizer.py` — Parrot paraphraser | |
| - `auth.py` — accounts, sign-in sessions, limits, admin user actions | |
| - `dashboard.py` — run history, user dashboard, admin analytics | |
| - `plans.py` — pricing plans, quotas, credits, upgrade prompts, manual/Razorpay activation | |
| - `textparse.py` — pasted-text parsing, long-sentence handling, rewrite guard | |
| - `mailer.py` — result emails, reset codes, welcome/plan mails (Brevo API) | |
| - `groqai.py` — Groq quality check of Parrot's output (repairs terms/grammar, rejects meaning changes) | |
| - `push.py` — Web Push notifications when a result is ready | |
| - `static/` — notification service worker + icon | |
| - `runtime.py` — CPU core sharing between simultaneous users (CPU hardware) | |
| - `packages.txt` — LibreOffice + fonts (Word → PDF reports) | |
| - `logo.png`, `favicon.png` | |
| ## Groq quality check (groqai.py) — optional, recommended | |
| **Parrot does all the rewriting.** With the `GROQ_API_KEY` secret set, Groq then checks every sentence Parrot changed: | |
| it gets the original (A) and Parrot's paraphrase (B) and makes only the minimum edits to B — puts back any | |
| technical term, acronym, number, unit or citation exactly as in A, fixes broken grammar and casual wording, and | |
| keeps Parrot's phrasing everywhere else. If B changed the meaning and cannot be repaired, it is rejected and the | |
| original sentence is kept. The local guard checks Groq's result once more. Without Groq (no key, quota used up, | |
| network error) the local checks alone are used, exactly as before. | |
| Admin → Users → ✅ Groq quality check shows the model, checked / repaired / rejected counts, tokens and last error. | |
| | Variable | Default | Purpose | | |
| |---|---|---| | |
| | `GROQ_API_KEY` | — | secret; turns the check on | | |
| | `GROQ_MODELS` | `openai/gpt-oss-120b,openai/gpt-oss-20b` | first model available on the key is used; the next one when it is at a rate limit. Keep expensive models out of this list: `qwen/qwen3.8-27b` costs about six times more per token | | |
| | `GROQ_LIGHT_MODEL` / `GROQ_LIGHT_KINDS` | `openai/gpt-oss-20b` / `verify,grammar` | half-price model for the tasks that only judge or correct a sentence; `GROQ_LIGHT_MODEL=` (empty) = main model for everything | | |
| | `GROQ_JOB_BUDGET` | `1.00` | ceiling in US$ for one job; past it the job continues with the local checks (0 = no ceiling) | | |
| | `GROQ_PRICES` | built-in table | `model=in/out` per million tokens, comma separated, when Groq changes its prices | | |
| | `POLISH_CHANGED_ONLY` | `1` | the final polish is sent only for paragraphs the tool changed; `0` = every paragraph | | |
| | `GROQ_BATCH` / `GROQ_MAX_WAIT` | `10` / `60` | sentence pairs per request / max seconds to wait on rate limits | | |
| **What it costs (measured, October 2026 prices: gpt-oss-120b $0.15 in / $0.60 out per million tokens, gpt-oss-20b | |
| half of that; input read from Groq's prompt cache is billed at half price).** One full pass of every task over | |
| 5,000 words is about 6-7 US cents: the natural-style rewrites are a little over half of it (they return several | |
| versions of each sentence), the quality check and final polish about 15% each, the meaning check and grammar check | |
| together under 10%. A fully AI-written 5,000-word document that needs 2-4 passes therefore costs roughly 10-25 | |
| cents; a document with only some flagged paragraphs costs proportionally less, because only flagged sentences are | |
| rewritten and only changed paragraphs are polished. Every job prints its own bill in the Space log: | |
| `[cost] Smart Humanize, 5120 words: 14 requests, 61.2k tokens in (38% from cache) + 19.8k out = $0.0191 | variants ...`. | |
| The free tier (200K tokens per model per day) covers only a handful of documents. | |
| Customer text is sent to Groq — mention this | |
| in your privacy policy. | |
| ## Smart Humanize details | |
| - **Best of 3**: Parrot samples 3 paraphrases of every sentence in one pass (`HUM_CANDIDATES`, default 3); lost | |
| technical terms are copied back in place, broken versions are dropped, and the AI detector picks the most | |
| human-sounding survivor (`HUM_SELECT=0` turns the detector choice off). Groq then checks the chosen sentence. | |
| - **100% AI-written box**: Parrot writes 5 versions of every sentence instead of 3 (`HUM_CANDIDATES_AI`), the | |
| plain-wording rules add more options, the detector ranks them, and Groq checks the 2 most human versions | |
| (`HUM_GROQ_SHORTLIST`). The most human version Groq accepts is kept; if both are rejected, the next | |
| version or the original sentence is used. Parrot still does all the rewriting; Groq only checks. | |
| It also aims for 70% new wording per sentence (`HUM_AI_CHANGE_TARGET`, same measure as the "Vocabulary | |
| changed" gauge): sentences below it get up to 2 extra Parrot rounds with more variation (`HUM_AI_ROUNDS`), | |
| versions that reach it are preferred, and Groq's check may not pull the wording back toward the original. | |
| In Smart Humanize each pass already rewrites again, so the inner extra rounds drop to 1 (`SMART_AI_ROUNDS`) | |
| and the separate retry is skipped in this mode. Speed settings: `HUM_GPU_BATCH` (32), `GROQ_PARALLEL` (5), | |
| `GROQ_BATCH` (12). | |
| - Speed: sentences are batched shortest-first and generation stops just past the input length; Groq batches run | |
| 3 at a time (`GROQ_PARALLEL`). | |
| - Sentences with P(AI) ≥ the moderate threshold (default 0.50) are rewritten; each later pass uses | |
| slightly bolder variation. | |
| - A rewrite is rejected (original kept) if it drops numbers or changes length drastically; dropped | |
| citations like `[12]` or `(Kumar et al., 2021)` are re-attached. | |
| - **Technical terms are protected**: a rewrite is rejected if it changes an acronym/symbol/model name | |
| (CNN, LiFePO4, ResNet-50, 5G, α), a term defined in the document ("phase change material (PCM)"), a phrase | |
| or word the document uses repeatedly ("state of charge", "battery"), or a word whose synonyms change the | |
| meaning ("accuracy", "model", "layer", "voltage"…). Rejected sentences get one calmer retry; if that also | |
| damages a term, the original sentence is kept. | |
| - In Word files only the flagged sentence's text is replaced; the rest of the paragraph keeps its | |
| formatting. Citation fields, hyperlinks and superscript/subscript runs are never edited — the prose | |
| around them is rewritten instead. | |
| Scores are indicators from an open-source detector and will differ from Turnitin, GPTZero, etc. | |
| ## Licences | |
| Paraphraser: humarin/chatgpt_paraphraser_on_T5_base, OpenRAIL (default) · Parrot paraphraser (fallback): Apache-2.0 · RoBERTa AI detector: Apache-2.0. | |
| ## Final grammar check (Groq) | |
| After all rewriting is finished, Groq corrects grammar, spelling and punctuation only (no rewording). A correction is | |
| kept only if it changes at most 25% of a sentence's words (`HUM_GRAMMAR_MAX_EDIT`) and keeps every technical term, | |
| number and citation. Smart Humanize runs it once on the final version. `HUM_GRAMMAR=0` turns it off. Without a Groq | |
| key the step is skipped. | |
| ## Report-guided rewriting (Smart Humanize) — image processing | |
| Upload an AI or similarity report PDF from any checker (Turnitin, iThenticate, GPTZero, ZeroGPT, Copyleaks, Academi, | |
| scanned reports too). `report_flags.py` renders every page to an image and, for every word, looks at the pixels in | |
| the word's box: a tinted background (cyan, purple, yellow ... even very pale tints) = highlighted; coloured letters = | |
| coloured text. Word positions come from the PDF text layer, or from Tesseract OCR for scanned pages (parallel). | |
| Flagged words are grouped into the report's sentences (cyan = High, purple = Moderate), matched to the document | |
| (word-trigram overlap >= 50%) and only those sentences are rewritten in one pass. The result shows the report's own | |
| score -> an estimated score, and a table of every flagged sentence with its rephrased version. | |
| Flagged sentences get the strongest rewrite: `restructure.py` reshapes them first (clause order, splits, | |
| stock openers; rule-based, no AI model), then Parrot with the 100% AI settings and a 60% new-wording target. | |
| Rows where less than half the wording changed are marked "Likely still flagged" for the writer to revise. | |
| Needs `tesseract-ocr` (packages.txt) and `pytesseract` (requirements.txt) for scanned reports. | |
| ## Final polish (Groq) | |
| After all rewriting, the finished text goes through a final polish, paragraph by paragraph. Groq sees each | |
| paragraph together with the author's ORIGINAL paragraph. **Unclear or garbled sentences are repaired** to the | |
| original meaning (clarity wins over the AI score for these); a repair that adds facts or drifts in meaning is | |
| rejected and the clear original sentence is put back. Clear sentences get the light polish: grammar, technical | |
| academic wording (wrong-sense or informal words, redundancy) and flow between sentences (logical links, varied | |
| openings, smoother word order). Sentences are never merged, split, dropped or reordered, so the Word layout stays | |
| intact. A polished sentence is kept only if it keeps every term, number and citation, keeps its meaning (meaning | |
| check), changes at most `POLISH_MAX_EDIT` of its words and, with `POLISH_KEEP_HUMAN=1`, does not turn a sentence | |
| the detector reads as human into one it reads as AI-written; otherwise the grammar-only correction is used. | |
| LanguageTool then fixes any remaining mechanical errors. | |
| **The polish never undoes the humanization.** The editor model tends to call a correctly humanized sentence | |
| "unclear" and rewrite it back towards the original. A repair is therefore accepted as a repair only when the | |
| sentence really fails the clarity checks against the author's sentence (defects, structure, grammar, meaning). A | |
| sentence that passes them is clear: its "repair" is treated as an optional edit, kept only if it does not raise the | |
| AI score of the sentence read with its neighbours, and all optional edits of a paragraph are undone together if the | |
| paragraph as a whole reads as AI-written again (`POLISH_KEEP_HUMAN=0` switches this protection off). | |
| | Variable | Default | Effect | | |
| |---|---|---| | |
| | `POLISH` | `1` | `0` = grammar-only final check | | |
| | `POLISH_MAX_EDIT` | `0.6` | Largest share of a sentence's words the polish may change | | |
| | `POLISH_KEEP_HUMAN` | `1` | `0` = always keep the polished wording, even if the AI score rises | | |
| ## Token use and key distribution (groqai.py) | |
| - **Changes-only replies**: the grammar check, quality check and polish return only the sentences they changed; | |
| every other id is listed under `"unchanged"`. Ids missing from both lists are retried, so nothing is skipped. | |
| This cuts output tokens by roughly 80% on a typical document. | |
| - **Size-based batches**: a request holds up to `GROQ_BATCH` sentences (default 20) and about `GROQ_BATCH_CHARS` | |
| characters of text (default 6000), so the fixed system prompt is shared by more sentences (about 40% fewer requests). | |
| - **Token-aware key choice**: each Groq reply reports the key's remaining tokens per minute; the next request goes | |
| to the key with the most room (requests in flight are booked against it), and waits only when no key has room. | |
| - **Right-sized `max_tokens`**: sized to the reply, plus a separate allowance for the hidden reasoning of | |
| reasoning models (`GROQ_REASONING`: low / medium / high). A reply cut off at the limit is retried once with a | |
| larger budget, and complete items are salvaged from a cut-off reply. | |
| - **Cache**: a sentence already checked (grammar / quality check) is not sent again (`GROQ_CACHE`, default 20000). | |
| - Admin → Groq status now shows tokens per key and cache hits. | |
| ## Rigorous proofread: quality settings | |
| - Repairs of unclear or garbled sentences are judged against the author's original sentence (they used to be | |
| rejected for not resembling the garbled text). They count as required fixes and are never undone or reworded | |
| by the AI-score steps. | |
| - A sentence with a grammar error gets the full correction when its AI rise stays within `PROOF_REQ_RISE`; | |
| otherwise it gets the minimal grammar fix. | |
| - The full proofread runs with `PROOF_REASONING` (default `medium`); the grammar audit stays on `low`. | |
| | Variable | Default | Effect | | |
| |---|---|---| | |
| | `PROOF_KEEP_RISE` | `0.05` | Largest AI rise allowed for a style-only edit (was 0.03) | | |
| | `PROOF_ALT_RISE` | `0.10` | A corrected sentence is reworded for the AI score only above this rise | | |
| | `PROOF_MAX_DOC_RISE` | `2.0` | Document AI points allowed before style edits are undone (was 1.0) | | |
| | `PROOF_VARIANTS` | `3` | Alternative versions per sentence when rewording (was 4) | | |
| | `PROOF_REPAIR_SIM` | `0.60` | Minimum similarity of a repair to the current sentence (original known) | | |
| | `PROOF_REPAIR_SIM_NOREF` | `0.70` | Same, when no original document is given | | |
| | `PROOF_REASONING` | `medium` | Reasoning effort of the full proofread | | |
| ## Paraphrase model | |
| Parrot (`prithivida/parrot_paraphraser_on_T5`) is the paraphraser. `PARA_MODEL` (or the older `PARROT_MODEL`) can | |
| name another T5 paraphraser with the same `paraphrase:` input, e.g. `humarin/chatgpt_paraphraser_on_T5_base` | |
| (`PARA_REP_PENALTY`, default 1.0 for Parrot and 1.15 for other models, discourages copying the input back). | |
| ## Adversarial Paraphrasing (detector-guided rewriting, adversarial.py) | |
| Technique from Cheng et al. (2025), *Adversarial Paraphrasing: A Universal Attack for Humanizing AI-Generated Text* | |
| (arXiv:2506.07001). While Parrot writes a sentence, the AI detector chooses the words: at every step Parrot proposes | |
| its most likely next tokens, each one is appended to the text so far and scored by the detector, and the most | |
| human-reading one is kept. Parrot keeps the sentence fluent (only its own likely tokens are allowed); the detector | |
| steers the wording. The result is one extra version per sentence next to Parrot's sampled versions, and it goes | |
| through exactly the same checks (terms, numbers, citations, meaning, fluency, clarity gate, Groq), so a damaged | |
| version is never chosen. | |
| | Variable | Default | Effect | | |
| |---|---|---| | |
| | `HUM_ADV` | `ai` | `ai` = "100% AI-written" box and report-guided rewriting · `always` · `off` | | |
| | `ADV_K` | `8` | Candidate words scored per step (the paper uses 50; more = stronger, slower) | | |
| | `ADV_TOP_P` / `ADV_MIN_P` | `0.95` / `0.05` | Only Parrot's likely words are candidates (keeps grammar intact) | | |
| | `ADV_EVERY` | `1` | Guide every n-th word (`2` halves the detector cost) | | |
| | `ADV_BATCH` / `ADV_MAX` | `16` / `400` | Sentences decoded together / most sentence pieces per job | | |
| | `ADV_ON_ZERO` | `0` | ZeroGPU hardware: off (it would need nested GPU calls); `1` = run on CPU | | |
| Cost: about 2-3x the paraphrasing time for the sentences it is used on (dedicated GPU). Admin → Users shows its | |
| counters next to the Groq status. | |
| ## Protection against construct renaming and other rewrite errors (textparse.py) | |
| Checked on every rewrite, in every mode (including report-guided rewriting, where ordinary words are free): | |
| - **Study constructs are locked.** Terms defined with an abbreviation ("travel satisfaction (TS)") and the Keywords | |
| line are read from the whole document; a rewrite that drops one, or brings in an everyday synonym for it | |
| ("pleasure"/"contentment" for satisfaction, "closeness" for intimacy, "allegiance" for loyalty), is rejected. | |
| - **Abbreviations** may not change ("EWB" -> "EWD") or appear from nowhere. | |
| - **Statistics symbols** (< > ≤ ≥ = ± ×) must survive ("p < .001" never becomes "p .001"). | |
| - **Brackets** must stay balanced ("(TS amongst ..."). | |
| - **Foreign words** ("well-être") and **off-topic vocabulary** (2+ uncommon words that occur nowhere in the | |
| document, e.g. "organizational recruitment" in a tourism paper; `HUM_OFFTOPIC`, 0 = off) are rejected. | |
| A rejected rewrite falls back to another checked version, or to the author's sentence. |