| # TabFix: frozen end-to-end test diagnostic |
|
|
| Checkpoint: `Antix5/tabfix-preview@044567585e7943aab9252339cfe7bb1c2c44509b`. Threshold: 0.9. Split: test. GPU BF16 (L40S). |
|
|
| Completed 55 tables. Exactly repaired **23/45** faulty cells; harmed **55/6193** initially valid cells. |
| Detection cell/category precision: 35/161; recall: 35/46. Hard-negative tables harmed: 11/18. |
|
|
| The detector receives no gold locations. Repairs use predicted spans. Gold is used only to score the final cells. Category counts can overlap on compound cells; overall cell counts are deduplicated. |
|
|
| | Category | Detected / gold | Exact repair / gold | False category proposals | Valid cells harmed | |
| |---|---:|---:|---:|---:| |
| | text.encoding | 2/2 | 1/2 | 0 | 0 | |
| | text.invisible | 2/2 | 0/2 | 0 | 0 | |
| | text.spelling | 2/2 | 1/2 | 22 | 10 | |
| | format.number | 2/2 | 2/2 | 10 | 0 | |
| | format.date | 2/2 | 0/2 | 28 | 28 | |
| | format.boolean | 2/2 | 2/2 | 1 | 0 | |
| | format.case | 2/2 | 2/2 | 0 | 0 | |
| | format.whitespace | 3/3 | 3/3 | 0 | 0 | |
| | format.unit | 3/3 | 3/3 | 0 | 0 | |
| | schema.type | 2/2 | 2/2 | 0 | 0 | |
| | schema.identifier | 2/2 | 1/2 | 0 | 0 | |
| | schema.category | 7/7 | 7/7 | 0 | 0 | |
| | consistency.unit | 0/2 | 0/2 | 15 | 1 | |
| | consistency.dependency | 0/3 | 0/3 | 0 | 0 | |
| | consistency.temporal | 0/2 | 0/2 | 4 | 4 | |
| | missing.required | 2/2 | 0/2 | 29 | 12 | |
| | missing.conditional | 2/2 | 0/2 | 17 | 0 | |
| | structure.shift | 0/4 | 0/4 | 0 | 0 | |
|
|
| ## Limits |
|
|
| This is a small, deliberately stratified synthetic diagnostic, not an estimate of production accuracy. Positive category slots cover two languages each; language comparisons are confounded by different tables and errors. The model and threshold were frozen before inference. No threshold or weights were tuned on these test results. Exact reference matching may reject another valid repair. Overlapping proposals are refused; structural shifts are unsupported. The current pipeline attempts generation for missing-value proposals even when the intended value is not inferable; report these failures rather than treating them as supported reconstruction. |
|
|
| Inspect every faulty or changed cell in `review.html`. Raw cases, the frozen selection, category counts and slices are retained alongside this report. |
|
|
| ## Interpretation and next priorities |
|
|
| The pipeline is not ready for automatic correction: it changed more originally valid cells (55) than it repaired faulty cells (23) on this sample. The leading damage categories are date formatting (28), required missing values (12), spelling (10), temporal consistency (4) and unit consistency (1). These are observed errors, not a causal attribution to training data alone. |
|
|
| Examples inspected directly: `04/01/2026` became `2026-01-02-02666`; `Gummifuß` became `Gummifußs`; a valid empty cell received `No additional handling required</cell`. These are not merely alternative valid formatting choices. |
|
|
| Detection found none of the annotated unit, arithmetic-dependency or temporal-consistency errors (0/2, 0/3, 0/2). Invisible-character errors were detected but neither was exactly repaired (0/2), exposing a localization/generation/application gap. |
|
|
| Next work should (1) require abstention for missing values without a uniquely inferable replacement, (2) audit predicted span boundaries and in-place replacement composition using these saved failures, and (3) improve valid-date, multilingual-spelling and relationship examples with schema-disjoint validation. Thresholds and future checkpoint selection should be assessed on validation data; this test diagnostic is now observed and must not become a tuning set. A lower validation cross-entropy does not guarantee fewer harmful edits. |
|
|
