chemistry-draw-ocsr — selected DECIMER ONNX artifacts

Selected, optimized ONNX artifacts from a reproducible decimer-onnx-toolchain (dtx) evaluation for self-contained inference (ONNX Runtime, no TensorFlow required at runtime). This repository packages the selected variants of three distinct source models (different origins/weights/tokenizers, evaluated independently): ocsr-printed, ocsr-handdrawn, and image-classifier. These artifacts are optimized for inclusion in chemistry_draw. It contains only the selected artifacts (models/vocabs/metadata) and documentation.

Selected artifacts

Folder	Task	Selected variant	Files	Source model	Primary result (frozen eval)
ocsr-printed-fp32/	OCSR (printed chemical structures)	FP32	encoder.onnx, decoder_step.onnx, vocab.json, export.fp32.meta.json	DECIMER v2 (printed), Zenodo 8300489 (CC-BY-4.0)	Canonical RDKit 0.7041 (614/872, n=1000), exact 0.012, valid 0.969
ocsr-handdrawn-fp32/	OCSR (structures, model trained for hand-drawn)	FP32	encoder.onnx, decoder_step.onnx, vocab.json, export.fp32.meta.json	DECIMER v2 Hand-drawn (upstream specialist), Zenodo 10781330 (CC-BY-4.0)	Canonical RDKit 0.7300 (711/974, n=1000), tf_match 0.912
image-classifier-fp16/	Image classification (chemical structure vs not)	E4 FP16	classifier.fp16.onnx, e4.fp16.meta.json	Iagea/DECIMER-Image-Classifier (MIT, trained weights from upstream pinned commit)	Binary acc 0.7679 (n=754 non-reaction of 1000), TF parity 0.998, 7.78 MB (-49%)

Selection rationale (per artifact)

Selection criterion: accuracy ≥70% on the primary metric (frozen eval, same IDs), with robustness (not a borderline eval-only win). Compression/latency improvements are adopted when they do not violate the floor; all tested variants (including non-selected) were recorded in the source toolchain report.

ocsr-printed-fp32/ — why FP32

• Passes the floor: FP32 achieves 0.7041 canonical RDKit (614/872 comparable, n=1000, eval), valid 0.969.
• No compressed/mixed variant was robustly ≥70%: INT8 dynamic E1 reached 0.6776 on eval (<70%); static/hybrid E3 collapsed to 0.485. Mixed encoder/decoder and selective block-wise quantization variants were borderline on eval/pilot but dropped below 70% on held-out (remainder), e.g. FP32enc+e1sdec 0.7000 (eval) → 0.6870 (remainder), selC 0.703→0.6944, selF 0.705→0.6925, selMid 0.701→0.6880 (and others similar). OCSR FP16 was not viable on CPU ORT for this export.
• Decision: FP32 (~301 MB enc+dec) is the only variant demonstrated to meet the ≥70% floor for printed structures in this evaluation, so it is selected for reliability over compression.

ocsr-handdrawn-fp32/ — why FP32 (and not E1 dynamic INT8)

• Both FP32 and E1 pass the floor on eval, but with clear tradeoffs:
  • FP32: canonical 0.7300 (711/974), tf_match 0.912, p50/p95 1115 / 2156 ms (n_measured=700), ~301 MB.
  • E1 dynamic INT8 all: canonical 0.7149 (692/968, −1.51 pp), tf_match 0.858 (−5.4 pp), p50/p95 2860 / 3951 ms (~2.6× slower; dynamic INT8 adds runtime overhead on CPU and is not guaranteed faster), ~81 MB.
  • E3 hybrid: 0.000 canonical (severe degradation) — rejected.
• Cross-domain note: this model (upstream hand-drawn–trained) generalized well to printed but is not strictly better than the printed model in-domain: on printed eval-1000 it scored 0.694 canonical (628/905) vs 0.704 for the printed model (614/872), while the printed model degraded on hand-drawn (0.677 pilot). The models are distinct with non-exclusive specialization, but neither is uniformly superior across both domains.
• Decision: although E1 is smaller and meets ≥70%, it degrades accuracy/fidelity and is substantially slower in this setup. FP32 is selected as the default for chemistry_draw to prioritize quality, TF-fidelity, and latency while meeting the floor. (E1 remains a recorded, non-selected alternative if size is prioritized over speed/fidelity.)

image-classifier-fp16/ — why E4 FP16

• Meets the goal (accuracy + compression): E4 FP16 achieves 0.7679 binary accuracy (non-reaction, n=754 of 1000 rows) with TF parity 0.998 and 7.78 MB (~49% smaller than FP32 15.35 MB), with comparable CPU latency — satisfying ≥70% while improving size.
• Alternatives rejected/not selected: INT8 on Conv degraded accuracy below the floor — E1 dynamic INT8 all 0.4085, E2 static QDQ (holdout calib) 0.5345. E1s selective (no Conv) preserved accuracy (0.7666) but provided no meaningful compression (~16 MB), so it does not meet the compression objective.
• Decision: E4 FP16 is the only evaluated variant that meets ≥70% accuracy and delivers substantial compression with high fidelity, hence selected.

Model details

OCSR (printed / hand-drawn)

• Architecture: two ONNX graphs per model — encoder.onnx + decoder_step.onnx (1-step decoder) with an external greedy decoding loop.
• Vocabularies: each OCSR model ships its own vocab.json (printed and hand-drawn tokenizers differ); use the vocab from the same folder as the model.
• Primary metric: RDKit canonical SMILES equality on comparable pairs (both expected and predicted parse). Secondary: exact string match and TF fidelity.
• Preprocessing/decoding: follows DECIMER conventions (image preprocessing + autoregressive greedy decode over the exported graphs). Implementation must match the export/runtime contract for parity.

Image classifier (structure vs non-structure)

• Variant: FP16 ONNX (classifier.fp16.onnx).
• Binary by design (0/1): the model emits a single logit → sigmoid → structure / no-structure (threshold score <= 0.000089 => structure). It does not distinguish one vs several molecules, and it does not detect reactions — those are dataset labels used to slice the binary metric, not model outputs. Any routing beyond binary (e.g. "crop to one molecule", "reaction not supported") needs a separate detection signal.
• Protocol: binary mapping one/several_molecule → structure, other → no-structure; reactions excluded from binary evaluation and reported separately.

Inference contract (inputs / outputs)

Exact tensor names and shapes (verified against the published artifacts with ONNX Runtime) so integrators can wire the graphs without guesswork:

OCSR graphs (ocsr-printed-fp32/, ocsr-handdrawn-fp32/):

• encoder.onnx: input image float32 [1, 512, 512, 3] (NHWC, DECIMER-style preprocessed image) → output Identity:0 float32 [1, 256, 512] (encoder memory).
• decoder_step.onnx: inputs tokens int32 [1, seq] and enc_out float32 [1, 256, 512] → output Identity:0 float32 [1, seq, 156] — already softmaxed probabilities over the 156-token vocabulary (verified: rows sum to 1; do not apply softmax again).
• Decoding (greedy): seed with start_id = 13; at each step take argmax of the last position, append the token, stop at end_id = 14 or max_len = 302.
• vocab.json (per folder — printed and hand-drawn tokenizers differ, always pair model + vocab from the same folder): token_to_id / id_to_token, start_id/end_id, pad/unk tokens, digit_remap (DECIMER digit convention), format/source/notes.
• Output per prediction (reference runtime DecimerONNX.predict, src/runtime/api.py):
  • smiles — decoded SMILES string (use as the model output; validate with RDKit for critical use);
  • token_ids — generated ids including the leading start_id;
  • confidences — probability of each chosen token (values in [0,1], from the softmaxed decoder output) — the input for confidence gating (in-domain mean-token thresholds ≈ 0.928 printed / 0.901 hand-drawn, re-calibrate on real inputs);
  • incomplete — true when decoding hit max_len without emitting end_id (truncated/unreliable output: treat as failure or retry, do not return as-is).

Classifier graph (image-classifier-fp16/):

• Inputs: input_2 float32 [N, 224, 224, 3] (preprocessed image) plus two auxiliary float inputs [1, 1, 1, 3] (45214:0, 45216:0) that the validated harness feeds as zeros.
• Output: Identity:0 float32 [N, 1] — a single logit.
• Post-processing: score = sigmoid(logit); label = structure if score <= 0.000089, else no-structure (upstream threshold convention — note the direction: low score = structure). Binary only.

Selection scores (weighted rubric)

Selection used an accuracy-determinant weighted rubric (0–100, per dossier, gate ≥70% + robustness on primary metric). Scores below are for the selected variant in each dossier (see docs/selection-scoring.md in the source toolchain for full per-variant breakdown and criteria/weights).

Dossier	Selected variant	Score	Summary
ocsr-printed	FP32	79.3	Only variant robustly ≥70% canonical (accuracy determinant); compressed/mixed alternatives did not hold the floor on held-out.
ocsr-handdrawn	FP32	88.3	Best quality/fidelity/latency among variants meeting ≥70%; E1 met accuracy but with trade-offs.
image-classifier	E4 FP16	93.7	Meets ≥70% (determinant) and maximizes compression with high TF fidelity (best objective balance).

Benchmark summary (frozen evaluation)

Evaluation used frozen, curated datasets (seed 20260308, same-ID/paired protocol within dossier, CPU/ONNX Runtime). Floor: ≥70% on frozen eval. Decisions were based on primary-metric results and robustness (e.g. held-out checks), not on eval-only borderline wins.

Dossier	Variant	Split	n	Primary result	Notes
ocsr-printed	FP32	eval-1000	1000	0.7041 canonical (614/872)	exact 0.012, valid 0.969; p50/p95 ~2317/5457 ms (CPU, solo run)
ocsr-handdrawn	FP32	eval-1000	1000	0.7300 canonical (711/974)	tf_match 0.912; meets ≥70% floor
image-classifier	E4 FP16	eval-1000	1000	0.7679 binary acc (n=754 non-reaction)	parity TF 0.998; 7.78 MB

All tested variants (including non-selected quantizations/mixes) were recorded in the source toolchain’s benchmark report; only the selected artifacts are published here. Execution reality: a few runs had CPU contention that skews reported p50/p95 (accuracy unaffected), and some runs required retries due to transient ONNX load/runtime issues; these are documented in the source evaluation records. Formal paired statistics (McNemar/bootstrap CI) were not automated.

Limitations

• Predictions are probabilistic; validate SMILES (e.g. RDKit parse/canonicalization) for critical use.
• No compressed OCSR variant met the ≥70% canonical-accuracy floor robustly for printed structures in this evaluation (CPU/ORT), hence FP32 selection.
• Latency depends on hardware, runtime version, and providers; reported values are reference CPU measurements.
• Classifier binary results exclude reaction images per protocol.

Model sources (per artifact)

Artifact	Original source	License / attribution
ocsr-printed-fp32/	DECIMER v2 (printed OCSR): Zenodo 8300489 — Rajan et al.	CC-BY-4.0. Cite the original DECIMER work/authors.
ocsr-handdrawn-fp32/	DECIMER v2 Hand-drawn specialist OCSR: Zenodo 10781330 — Rajan et al.	CC-BY-4.0. Cite the original DECIMER work/authors.
image-classifier-fp16/	DECIMER Image Classifier: Iagea/DECIMER-Image-Classifier (trained weights from upstream at a pinned commit)	MIT (upstream). Retain upstream attribution.

Licenses and attribution

This package redistributes selected derived ONNX artifacts. Retain required attribution and comply with the original model licenses above. Toolchain/runtime code in the source project: Apache-2.0. Verify license compatibility for your use case. This repository provides converted artifacts for convenience.

Provenance

Generated by decimer-onnx-toolchain (dtx): fetch (pinned sources/checksums) → FP32 ONNX export → quantization/selection → frozen-benchmark evaluation (seed 20260308) → reporting.