---
license: llama3.3
base_model: meta-llama/Llama-3.3-70B-Instruct
library_name: peft
pipeline_tag: text-generation
tags:
  - lora
  - peft
  - chart-question-answering
  - data-visualization
  - misleading-charts
datasets:
  - rodriguescarson/adaption-chart-qa-with-axis-tricks
language:
  - en
---

> **Archived intermediate checkpoint.** This is the v5 run from the AutoScientist
> Challenge 2026: held-out 78, on-data 85. The adapter that was actually submitted,
> held-out 85.9, is at [polychart-shown-is-not-supported-lora](https://huggingface.co/rodriguescarson/polychart-shown-is-not-supported-lora).
> This copy is kept only so the run history survives the challenge.

# PolyChart — Shown Is Not Supported (LoRA)

A LoRA adapter for `Llama-3.3-70B-Instruct` that reads charts under a **measure-then-claim**
contract: a claim is made only when the underlying values support it, and the distance
between what a chart *shows* and what its data *supports* is reported as a graded quantity
rather than a yes/no label.

Trained on Adaption's AutoScientist for the AutoScientist Challenge 2026.

## Scores

| | Base | This adapter |
|---|---|---|
| On-dataset win rate | 15 | **85** |
| **Held-out category win rate** | 22 | **78** |

**Read 78, not 85.** The two measure different things and this is the most important
caveat on this model. The on-dataset figure is computed on the training distribution; the
held-out figure is computed on data the model never saw. In the study behind this adapter
they sometimes moved in *opposite* directions — one matched pair went 87.2 → 81.0
on-dataset while going 66.3 → 77.3 held-out. Optimising the displayed number can select
the weaker model.

## What is novel here: severity as a continuous target

Every released misleading-chart dataset we know of, including Misviz (ACL 2026), asks a
yes/no question: is this chart misleading? This model is trained on **Tufte's Lie Factor as
a continuous, signed target** — −1.0 where an inverted axis reverses a real trend, 0.017
where an expanded axis flattens one, 4.69 where a truncated baseline invents one. A binary
label collapses all three into the same bit.

Training composition is failure-weighted. Scoring the base model cell-by-cell over
question-type × distortion located the hard cells — continuous severity estimation, precise
value extraction under a distorted axis, and mechanism naming — and the 2,129-row dataset
was weighted toward them (severity 27%), with already-solved cells removed.

That targeting is measurable: the *uniform* version of this dataset on a 109B base
(Llama-4-Scout) reached 76 held-out; this failure-weighted version on a smaller 70B base
reached 78. Capacity sets the ceiling; composition decides where in that ceiling you land.

## Training

| | |
|---|---|
| Base | `Llama-3.3-70B-Instruct` |
| Method | LoRA, r=64, alpha=128 |
| Target modules | all linear |
| Epochs | 3 |
| Rows | 2,129, failure-weighted |
| Eval loss | 0.944 → 0.861 (no overfit) |

Every row passed an independent verifier sharing no code with the generator. It caught nine
silent failures in our own pipeline before shipping, four of them inside its own scaffolding.

## A retraction

**An earlier version of this card claimed "curation beats scale," on the basis that
expanding a seed from 1,882 to 21,882 rows dropped the held-out category from 76 to 72.
That claim is withdrawn.**

It rested on a single unpaired comparison. Repeating the test properly — paired within seed,
across 13 seeds and 176 fine-tuning jobs — reversed the conclusion: augmentation improved
the held-out score on **13 of 13 seeds, mean +14 points, two-sided sign test p = 0.0002**,
and the platform's advertised "+16 points from crossing 20,000 datapoints" reproduced at
**+17.4**.

Held-out difficulty spans 44.5 to 85.9 across those seeds, roughly three times the effect
being measured, so any cross-seed comparison mostly reports which seeds landed in which arm.
That is precisely the error the original claim made.

What survives is narrower and is stated above: curation decides *where within a base's
ceiling* you land, and expansion helps most where the seed under-covers its category. The
two are not competing prescriptions. A model whose thesis is *shown is not supported* owes
that test to its own claims first, which is why the retraction is recorded here rather than
quietly deleted.

## Intended use and limits

Chart-interpretation QA and misleading-encoding detection. Not a general-purpose assistant.

- One task family, one platform, one base model.
- Held-out evaluations fire on roughly a third of jobs and cannot be triggered on demand,
  so several seeds contribute one draw per arm against a metric with sd 3–5 noise.
  Direction is reported rather than a pooled effect size.
- The evaluation domain is assigned by the platform, not chosen. Across six purpose-built
  datasets, five were scored against `general` and none against its own topic — so a
  held-out number is interpretable only alongside the domain it was computed against.
- Inherits the biases and failure modes of `Llama-3.3-70B-Instruct`.

## Use

```python
from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer

base = "meta-llama/Llama-3.3-70B-Instruct"
model = AutoModelForCausalLM.from_pretrained(base, device_map="auto")
model = PeftModel.from_pretrained(model, "rodriguescarson/polychart-shown-is-not-supported-lora")
tok = AutoTokenizer.from_pretrained(base)
```

## Links

- Client + platform findings: https://github.com/rodriguescarson/adaption-autoscientist-api
- Live Lie Factor demo: https://lie-factor.carsonrodrigues.com
- Datasets: https://huggingface.co/rodriguescarson

## License

Llama 3.3 Community License, inherited from the base model.