pizza-order-parser / README.md
multimodalart's picture
multimodalart HF Staff
Upload folder using huggingface_hub
5140215 verified
|
Raw History Blame Contribute Delete
2.81 kB
metadata
title: Parsed, Plausible, Wrong — Pizza Order Parser
emoji: 🍕
colorFrom: indigo
colorTo: gray
sdk: gradio
sdk_version: 6.32.0
app_file: app.py
python_version: '3.12'
short_description: Pizza-order semantic parsers from arXiv:2611.17169
startup_duration_timeout: 30m

Parsed, Plausible, Wrong — pizza order parser

Live demo of the semantic parsers released with Parsed, Plausible, Wrong: Output Format Controls Silent Errors in Small-Model Semantic Parsing (arXiv:2611.17169, IDM Lab, ITMO University).

Type a spoken-style pizza order. The model answers a structured WORDS ticket (pizza 2 medium ham no olives), decoded live to the PIZZA benchmark's native s-expression, plus a readable ticket. Orders that aren't from the menu — off-menu items, chatter, gibberish — are answered ? whole, never half-parsed: a loud refusal instead of a silent wrong pizza. Spoken self-corrections ("a pizza with ham, actually make that bacon") are applied.

Two models ship with the paper, both selectable:

model base exact match silent errors
umarsheikh303/pizza-words-gguf Qwen3.5-0.8B 88.2% 3.8%
umarsheikh303/pizza-lfm2-words-gguf LFM2.5-350M 79.3% 10.2%

Both run in llama.cpp (GGUF Q8_0) under the exact prompt, greedy decoding, stop string and token cap of their export manifests — the same contract the GGUFs were server-verified under (99.0% / 93% agreement with the training runs over 200 orders). The WORDS decoder and the benchmark's unordered exact-match matcher are ports of the paper's own reference implementation (UmarGit/coffee-llm, src/bench/tasks/pizza/{formats,ladder}.py).

The finding: writing the same target tree as natural words instead of the dataset's native s-expression keeps accuracy (87.5% vs 85.8% exact match, mean of 5 seeds) while cutting silent errors — parses that look fine and are wrong — from 10.3% to 4.5%, with 3.8× fewer tokens and 3.8× lower latency.

Trained on the PIZZA benchmark (Arkoudas et al. 2022, CC-BY-NC-4.0 — non-commercial research use only).

Files

  • app.py — Gradio app: prompt contract from the export manifests, greedy llama.cpp inference, WORDS→EXR decode, readable ticket
  • pizza_words.py — port of the paper's WORDS decoder + benchmark matcher
  • data/pizza-words-q8_0.json, data/pizza-lfm2-words-q8_0.json — the authors' export manifests (prompt, stop, cap), shipped verbatim
  • data/catalog.json — the PIZZA catalog (entity words), from the authors' site data, CC-BY-NC-4.0