fix(zh): slice entity text from original input to avoid BERT tokenizer spacesRunning
Robin+1: Claude Sonnet 4.6
f90826c
The transformers token-classification pipeline with aggregation_strategy="simple"
emits Chinese entity text with whitespace between characters (e.g. "马 云"
instead of "马云"). This breaks downstream string matching and recall metrics.
Fix: use the start/end character offsets returned by the pipeline to slice
the entity text directly from the original input string. The offsets are
correct; only the joined token text is wrong.
Also adds scripts/test_remote_api.py — end-to-end API test harness that hits
the deployed HF Space, exercises every routing branch (EN/ZH/AR/Mixed/auto/
min_entities/threshold/edge), and writes a Markdown report with timings
and per-case recall.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
