Spaces:
Running
Running
| # Perturbation catalogue & invariance assumptions | |
| | ID | Transformation | Intended use | Invariance assumption | | |
| |---|---|---|---| | |
| | `original` | No change | Baseline, every source ID | Known original gold labels | | |
| | `format` | Uppercase, surrounding quotes, punctuation suffix | Surface robustness; case insensitive engine | Low-risk, still not human validated | | |
| | `typo` | Deterministically transpose two internal characters of an eligible token | Spelling noise stability | Typically low semantic impact, not guaranteed | | |
| | `synonym` | One predefined language-specific word replacement | Small lexical sensitivity check | Replacement may shift nuance; only changed rows are scored | | |
| | `context` | Append one neutral-sounding sentence about research context | Test sensitivity to additional context | May change rhetorical emphasis and true class | | |
| | `truncate` | Keep ~65% of source words | Destructive information loss | **NOT stable**: no transformed accuracy or accuracy-delta claim | | |
| ## Coverage details | |
| - If a dictionary replacement cannot match, the text and prediction are unchanged and remain in attempted-row denominator for `flipRate`. | |
| - Modified-subset accuracy is computed on actually modified rows only; compare to original accuracy for the same IDs, not all source examples. | |
| - This tool does not claim that a statistical flip is automatically an adverse effect. Meaning-preservation assumptions require independent annotation. | |
| - Test randomness is not used in the perturbations. Every transformation is deterministic and reproducible from the same inputs. | |
| - No external paraphrasing model or paid API is invoked. | |