rnaseek-full / validation /pi-update /evo_native_reference.json
schen647's picture
Add validated RiNALMo-Giga and Evo 2 benchmarks, revised figures and portable environments
e40c47f verified
Raw History Blame Contribute Delete
792 Bytes
{
"reference": "NVIDIA BioNeMo 2.6.1 unmodified bionemo.evo2.run.predict, model-size 1b, BF16",
"source_archive_sha256": "ea4a3f5c9c26d5edc10bdc85165c090ad0ff23ac2670d4f61244f5f0d9d5e817",
"sequence": "GCTGTCACCGGATCCGGTCTGATGAGTCCGGACGAAACAGCGCTGTCACCGGATCCGGTCTGATGAGTCCGGACGAAACAGC",
"sequence_length": 82,
"adapter_next_token_loss": 0.8353613615036011,
"native_next_token_loss": 0.8352132439613342,
"logits_relative_l2": 0.0027049260679632425,
"logits_mean_absolute_error": 0.05994144082069397,
"argmax_agreement": 0.9878048300743103,
"passed": true,
"scope": "Single-sequence numerical compatibility check; not a regression benchmark or a claim of bitwise equivalence. Acceptance: relative logit L2 below 0.03 and absolute next-token loss difference below 0.01."
}