Update KDL-Frontier-Parser-nano ParseBench results
#4
by Bapt120 - opened
Hi! We've been running KDL-Frontier-Parser-nano through ParseBench and wanted to share what we're seeing.
We reran the model several times and keep getting the numbers below. They differ from the values currently in the YAML, which might be from an earlier version of the bench or a reporting mix-up. It's very possible we're missing something on our side too, so we'd love your take.
| Metric | Current | Our rerun |
|---|---|---|
| Overall | 76.36 | 72.38 |
| Tables | 85.56 | 85.73 |
| Charts | 63.41 | 58.61 |
| Content faithfulness | 87.19 | 87.75 |
| Semantic formatting | 66.81 | 55.06 |
| Visual grounding | 78.84 | 74.75 |
To reproduce:
vllm serve KDLAI/KDL-Frontier-Parser-nano --revision a6cb7d2a9ac5fd13f527764f5411c9ec3ad2c4ec \
--served-model-name kdl-frontier-parser-nano --max-model-len 8192 --gpu-memory-utilization 0.85 \
--max-num-seqs 24 --trust-remote-code --limit-mm-per-prompt '{"image": 1}'
KDL_NANO_ENDPOINT_URL=http://localhost:8807/v1 uv run parse-bench run kdl_frontier_nano