ATTACK_CATALOG.md

Perturbation catalogue & invariance assumptions

ID	Transformation	Intended use	Invariance assumption
original	No change	Baseline, every source ID	Known original gold labels
format	Uppercase, surrounding quotes, punctuation suffix	Surface robustness; case insensitive engine	Low-risk, still not human validated
typo	Deterministically transpose two internal characters of an eligible token	Spelling noise stability	Typically low semantic impact, not guaranteed
synonym	One predefined language-specific word replacement	Small lexical sensitivity check	Replacement may shift nuance; only changed rows are scored
context	Append one neutral-sounding sentence about research context	Test sensitivity to additional context	May change rhetorical emphasis and true class
truncate	Keep ~65% of source words	Destructive information loss	NOT stable: no transformed accuracy or accuracy-delta claim

Coverage details

• If a dictionary replacement cannot match, the text and prediction are unchanged and remain in attempted-row denominator for flipRate.
• Modified-subset accuracy is computed on actually modified rows only; compare to original accuracy for the same IDs, not all source examples.
• This tool does not claim that a statistical flip is automatically an adverse effect. Meaning-preservation assumptions require independent annotation.
• Test randomness is not used in the perturbations. Every transformation is deterministic and reproducible from the same inputs.
• No external paraphrasing model or paid API is invoked.