IBEM-im2typst / DATA_PROVENANCE.md
dbcccc's picture
Upload 22 files
8c62819 verified
|
Raw History Blame Contribute Delete
1.84 kB
# Data provenance
## Source dataset
The model was trained only on formula crops derived from **The IBEM Dataset: a
large printed scientific image dataset for indexing and searching mathematical
expressions**, version 1.0:
- Official record: <https://zenodo.org/records/7963703>
- DOI: <https://doi.org/10.5281/zenodo.7963703>
- License: Creative Commons Attribution 4.0
- Authors: Dan Anitei, Joan Andreu Sánchez, and José Miguel Benedí
IBEM contains 600 scientific documents and 8,272 page images, with annotations
for isolated and embedded mathematical expressions. The original images are
machine-typeset page images, not handwriting. Their fonts and styles are not
uniform because they come from different papers.
## Preparation used for this model
- Formula images are grayscale bounding-box crops from the original IBEM page
JPEGs with a deterministic two-pixel crop margin.
- Images were not rerendered into one Typst font.
- Labels were derived deterministically from IBEM `latex_norm` annotations and
converted to package-free Typst with semantic spacing canonicalization.
- Splits are document-disjoint.
- Training: 129,303 crops from 480 documents.
- Validation: 16,899 crops from 60 different documents.
- Dataset lock SHA-256:
`10dae9db6eab4de9d36ee28ca99760bc0c6231fab77311004452dee719f64271`.
- The calibration and test documents were not opened for this checkpoint.
No Fusion dataset, synthetic formula corpus, teacher-generated labels,
external pretrained model, or fused samples were used in this model lineage.
## What is included
No image or annotation from the raw IBEM archive is redistributed here. The
end-to-end smoke-test image at `example/formula.png` is an original
project-authored formula rendered with Typst; it is covered by the project MIT
License and is not training or evaluation data.