docs/DATA.md

ViDoRe V3 HR: verified data contract

Inspected locally on 2026-10-19 with uv run vdr download --subset hr and uv run vdr inspect. Statistics below are computed from downloaded Parquet tables, not inferred from the dataset card.

Source and reproducibility

• Dataset: vidore/vidore_v3_hr.
• Snapshot: 0cdf0979f2c5a0fd3e335e6373b9da48a9fe3bc3.
• Four configurations, each with a single test split; no new train/dev split.
• Downloaded Parquet size: 442,735,193 bytes (442.74 MB / 422.23 MiB).
• Original PDFs are available upstream but are not needed or downloaded: the corpus already contains page images and extracted markdown.
• Raw files: data/hr/snapshots/<revision>/<config>/*.parquet.
• Provenance: data/hr/manifest.json; full observed schema/statistics: data/hr/inspection.json. Both are local, ignored artifacts.

To reproduce this snapshot explicitly:

uv run vdr download --subset hr --revision 0cdf0979f2c5a0fd3e335e6373b9da48a9fe3bc3
uv run vdr inspect

Counts and suitability

Item	Observed
Documents	14
Page images / retrieval candidates	1,110
Queries, all languages	1,908
English queries (language == "english")	318
Qrels, all languages	10,386
Qrels for English queries	1,731
Distinct relevant pages for English queries	613
Pages with nonempty markdown	1,098
Pages with empty/missing markdown	12
Total markdown characters	3,443,946

Each of english, french, spanish, italian, german, portuguese has 318 queries. Filter by the actual language field; do not infer language from ID ranges. The loader accepts normalized english or en.

All 318 English queries have positive relevance judgments. All page/query IDs are unique, qrel pairs are unique, all qrel references exist, and all corpus documents have metadata. All nine implemented integrity checks passed.

Document metadata lists charts in 13 documents, tables in 10, and images in 8. English qrel content labels include Text (1,654), Chart (234), Table (206), Infographic (65), Image (13), Other (3), and N/A (If relevance score=0) (1). Labels can overlap, so these counts must not be added as disjoint categories. All observed scores are nevertheless 1 or 2: that single N/A label is an upstream annotation inconsistency. Preserve it and the original scores unchanged.

Six evidence-selected pages were decoded and manually reviewed in data/hr/samples/contact-sheet.jpg: they contain dense tables, line charts, scatter plots and pie charts. All six sampled images are 1654 × 2339 pixels; dimensions of the remaining pages were not measured during initial inspection. These inspection JPEGs are previews, not the full-resolution inputs for visual indexing.

The subsequent full PNG export measured every page: 967 are 1654 × 2339, 1 is 1653 × 2339, 54 are landscape 2339 × 1654, 5 are 1386 × 1969, 33 are 1386 × 1970, and 50 are 1387 × 1970. Original-size RGB PNGs total 503,571,673 bytes and live under data/pages/hr/<revision>/; the page manifest records each image's dimensions and source attribution.

Decision: keep HR. It has useful visual evidence, supplied BM25 text, and a manageable initial download. Actual visual-index size and hosting fit still need measurement in the next milestone. HR here means EU employment/labour reports, not an internal employee handbook; future demo examples should match this domain.

Observed schemas

corpus — 1,110 rows

Field	Type	Meaning
corpus_id	int64	Stable page ID; join to qrels
image	Hugging Face Image	Embedded page image; decoded as PIL on access
doc_id	string	Source document; join to document metadata
markdown	string	Provided OCR/parsed page text for BM25
page_number_in_doc	int64	Source page position; observed range 0–201

Use corpus_id as the retrieval/citation identifier, never the row offset or page_number_in_doc. Do not assume printed page numbers equal stored positions. Keep all 1,110 pages as retrieval candidates, including pages without qrels and the 12 without text. An empty text page contributes an empty BM25 document.

queries — 1,908 rows

Field	Type
query_id	int64
query, language	string
query_types	list[string]
query_format	string
content_type, raw_answers	list[string]
query_generator, query_generation_pipeline	string
source_type, query_type_for_generation	string
answer	string

raw_answers contains reference answers and answer the combined reference answer. Neither is used for retrieval indexing or v1 answer evaluation. Benchmark query/answer text is not copied into committed docs or demo examples.

qrels — 10,386 rows

Field	Type
query_id, corpus_id, score	int64
content_type	list[string]
bounding_boxes	list[struct{annotator, x1, x2, y1, y2: int64}]

Join queries.query_id → qrels.query_id → corpus.corpus_id. Use an ID mapping, not positional indexing. Scope qrels to the selected English query IDs.

Observed score counts across all languages: 1 → 8,700; 2 → 1,686. The dataset card defines 1 as critically relevant (partial evidence) and 2 as fully relevant. Keep graded scores for NDCG and treat scores greater than zero as relevant for recall. Never treat unjudged pages as additional positive labels. Bounding boxes are preserved but unused in v1; their coordinate conventions have not been validated for drawing overlays.

documents_metadata — 14 rows

Field	Type
file_name, doc_id, url, doc_type, doc_language	string
doc_year	int64
visual_types	list[string]
page_number	int64
license	string

Licensing and attribution

The pinned dataset card licenses annotations, qrels and generated metadata under CC BY 4.0. Source documents and parsed markdown inherit the individual publishers' licenses, as recorded in documents_metadata.license; all 14 observed entries are cc-by-4.0.

For demo redistribution, retain ViDoRe attribution, source document URLs and publisher attribution, link CC BY 4.0, and state image resizing/compression. Keep per-document provenance with exported demo artifacts. No raw dataset, benchmark queries/answers, or images belong in Git.

Benchmark reference: Loison et al., ViDoRe V3 (2026).