CERTA-BioMistral-7B-LLMINT8

This repository does not contain the original model weights. It provides the CERTA certification configuration and serving wrapper for the referenced base model BioMistral/BioMistral-7B. The certificate is calibrated for the clinical accept/reject decision task and evaluation distribution described in the accompanying paper; deployment on a different task or population requires recalibration.

CERTA is a decision-level certify-or-escalate layer around a quantized model: a decision is served by the quantized model only when its margin clears the calibrated threshold for its predicted class at the requested risk level, and is otherwise escalated to the full-precision (BF16) reference. This repository does not fine-tune or modify the base model.

Compression method: LLMINT8. Deployable (memory-reducing quantizer). This is a deployable LLM.int8() artifact (published where the 8-bit route loads).

What CERTA certifies (and what it does not)

• Certified: compression-induced disagreement with the BF16 reference on served decisions, bounded at risk level alpha with confidence 1 - delta (delta = 0.10), for the clinical accept/reject verification decision calibrated on PMC-Patients.
• Not certified: clinical correctness of the reference decision; and any guarantee on a different task or population (re-calibrate with export_thresholds.py).
• The thresholds in certa_thresholds.json are the exact calibrated values from the study, not re-estimated here.

Class-conditional operating point (alpha = 5%)

Escalation 5.4% ; amortized size 52.7% of the BF16 reference. Full thresholds (alpha = 1/5/10%, marginal and per-class) are in certa_thresholds.json.

Hardware

The wrapper is lightweight, but it does not remove the compute needed to run the model. At load time it downloads the base model BioMistral/BioMistral-7B and quantizes it with bitsandbytes, so a CUDA GPU with enough memory to hold the LLMINT8 model is required for inference. The wrapper only removes the need to re-host a separate copy of the (quantized) weights.

Usage

from certa_handler import EndpointHandler
h = EndpointHandler(".")                       # downloads + quantizes BioMistral/BioMistral-7B on a CUDA GPU
out = h({"inputs": "<YES/NO clinical-verifier prompt>", "alpha": 0.05})
# escalate=True -> refer this decision to the full-precision (BF16) reference model

License and attribution

Model name: CERTA BioMistral 7B LLMINT8

Quantized derivative of BioMistral/BioMistral-7B, licensed under the Apache License, Version 2.0. Only quantization (NF4 / LLM.int8()) was applied; no other modification.

Quantized derivative of BioMistral/BioMistral-7B, licensed under the Apache License, Version 2.0 (https://www.apache.org/licenses/LICENSE-2.0); a license copy is included.

Upstream license files included: LICENSE.

CERTA contribution license. The CERTA-authored files (certa_handler.py, certa_config.json, certa_thresholds.json, and the CERTA text in this card) are Copyright (c) 2026 Ali H. Lazem and William J. Teahan, licensed under Apache-2.0 (see LICENSE-CERTA). This does not relicense the model weights, which remain under the upstream license(s) above.

Citation

Lazem, A. H. and Teahan, W. J. Do We Need the Largest LLMs for Every Clinical Decision? (2026). arXiv:2610.26937.

• Paper: https://arxiv.org/abs/2610.26937
• Code: https://github.com/Ali-Lazem/CERTA

@misc{lazem2026certa,
  title         = {Do We Need the Largest LLMs for Every Clinical Decision?},
  author        = {Lazem, Ali H. and Teahan, William J.},
  year          = {2026},
  eprint        = {2610.26937},
  archivePrefix = {arXiv},
  primaryClass  = {cs.CL}
}