wasarou's picture
Add RGI manuscript evaluator rescoring
aefc53e verified
|
Raw History Blame Contribute Delete
7.98 kB

Re-evaluation with the RGI manuscript method

Archived predictions were rescored; no model inference or weight editing was performed. R/S uses the manuscript's legacy RDKit center assignment. E/Z includes only acyclic, nonaromatic, heavy-atom-decidable defined bonds. Geometry uses the manuscript's independent canonical-SMILES ETKDGv3/UFF/stereo-validated reference preparation.

The joint count is the conjunction of the manuscript's two per-system classifications. It is an additional summary, not the original common-mapping full-stereochemistry gate. The existing 55/50/104 split and previous validation/test exposure are unchanged. The original full-stereochemistry repair requirement remains unmet.

All geometry entries below are medians of per-system RMS errors.

development

Checkpoint N R/S E/Z Joint Bond (A) Angle (degrees)
boltz2_base.ckpt 55 21 41 20 0.038695 2.8723
boltz2_wsp_absolute_vectors_distances_v17.ckpt 55 24 41 23 0.038138 2.8443
boltz2_wsp_absolute_vectors_double_bond_v15.ckpt 55 21 42 20 0.038452 2.8680
boltz2_wsp_absolute_vectors_fourfold_v16.ckpt 55 22 42 21 0.039179 2.7212
boltz2_wsp_absolute_vectors_only_v11.ckpt 55 24 41 23 0.038121 2.8315
boltz2_wsp_balanced_geometry_v9.ckpt 55 21 42 20 0.038385 2.7390
boltz2_wsp_both_digit_suppression_v19.ckpt 55 19 42 17 0.044122 2.7141
boltz2_wsp_denoiser_distances_only_v12.ckpt 55 21 41 20 0.038440 2.8695
boltz2_wsp_denoiser_name_projection_v7.ckpt 55 21 40 19 0.074566 4.8459
boltz2_wsp_direct_input_balance_v6.ckpt 55 9 25 5 2.505421 46.2875
boltz2_wsp_direct_input_conditioning_v5.ckpt 55 22 43 21 0.044555 3.1729
boltz2_wsp_double_bond_contrast_v14.ckpt 55 21 41 19 0.038672 2.9020
boltz2_wsp_input_center_preserving_v8.ckpt 55 24 42 22 0.046064 3.1451
boltz2_wsp_input_digit_suppression_v18.ckpt 55 21 42 18 0.036351 2.9269
boltz2_wsp_input_geometry_v1.ckpt 55 23 40 22 0.038254 2.9360
boltz2_wsp_input_geometry_v3.ckpt 55 21 42 20 0.038556 2.7189
boltz2_wsp_input_reference_mask_v20.ckpt 55 20 41 19 0.038081 2.8679
boltz2_wsp_ligand_name_projection_v4.ckpt 55 20 41 19 0.038429 2.8946
boltz2_wsp_neighbor_program_control_v26.ckpt 55 21 41 20 0.037731 2.8868
boltz2_wsp_orthogonal_atom_geometry_v22.ckpt 55 21 41 20 0.038386 2.8960
boltz2_wsp_orthogonal_input_geometry_v24.ckpt 55 20 41 19 0.038285 2.9150
boltz2_wsp_orthogonal_pair_geometry_v23.ckpt 55 21 41 19 0.039263 2.8838
boltz2_wsp_programmed_neighbor_prior_v25.ckpt 55 20 41 19 0.038947 2.9092
boltz2_wsp_protected_distance_direction_v13.ckpt 55 22 41 20 0.038652 2.8792
boltz2_wsp_reference_distances_v2.ckpt 55 23 40 22 0.038295 2.9338
boltz2_wsp_reference_mask_combined_v21.ckpt 55 20 42 19 0.038563 2.8850
boltz2_wsp_relative_vectors_only_v10.ckpt 55 20 41 19 0.038319 2.8840
boltz2_wsp_reserved_atom_channels_v28.ckpt 55 22 39 20 0.098797 6.0517
boltz2_wsp_reserved_denoiser_channels_v27.ckpt 55 8 34 7 0.586642 34.4782
boltz2_wsp_reserved_token_channels_v29.ckpt 55 8 28 7 0.520425 23.8152
boltz2_wsp_reserved_token_condition_bias_v30.ckpt 55 8 25 1 0.640224 29.1628

v2_validation_seed_1

Checkpoint N R/S E/Z Joint Bond (A) Angle (degrees)
boltz2_base.ckpt 50 27 43 22 0.030402 2.4558
boltz2_wsp_reference_distances_v2.ckpt 50 25 43 23 0.030114 2.5120

v2_test_seed_1

Checkpoint N R/S E/Z Joint Bond (A) Angle (degrees)
boltz2_base.ckpt 104 41 79 35 0.037237 2.6254
boltz2_wsp_reference_distances_v2.ckpt 104 44 78 36 0.036704 2.6128

v2_test_seed_2

Checkpoint N R/S E/Z Joint Bond (A) Angle (degrees)
boltz2_base.ckpt 104 43 79 34 0.036447 2.6106
boltz2_wsp_reference_distances_v2.ckpt 104 42 79 32 0.035559 2.6137

v2_test_seed_3

Checkpoint N R/S E/Z Joint Bond (A) Angle (degrees)
boltz2_base.ckpt 104 41 78 34 0.036223 2.6611
boltz2_wsp_reference_distances_v2.ckpt 104 43 80 35 0.036433 2.5693

v2_nondevelopment_seed_1

Checkpoint N R/S E/Z Joint Bond (A) Angle (degrees)
boltz2_base.ckpt 154 68 122 57 0.034383 2.5826
boltz2_wsp_reference_distances_v2.ckpt 154 69 121 59 0.034468 2.5866

v11_validation_seed_1

Checkpoint N R/S E/Z Joint Bond (A) Angle (degrees)
boltz2_base.ckpt 50 27 43 22 0.030402 2.4558
boltz2_wsp_absolute_vectors_only_v11.ckpt 50 25 43 23 0.030142 2.4849

v11_test_seed_1

Checkpoint N R/S E/Z Joint Bond (A) Angle (degrees)
boltz2_base.ckpt 104 41 79 35 0.037237 2.6254
boltz2_wsp_absolute_vectors_only_v11.ckpt 104 44 78 36 0.036368 2.6236

v11_test_seed_2

Checkpoint N R/S E/Z Joint Bond (A) Angle (degrees)
boltz2_base.ckpt 104 43 79 34 0.036447 2.6106
boltz2_wsp_absolute_vectors_only_v11.ckpt 104 42 77 32 0.035828 2.6562

v11_test_seed_3

Checkpoint N R/S E/Z Joint Bond (A) Angle (degrees)
boltz2_base.ckpt 104 41 78 34 0.036223 2.6611
boltz2_wsp_absolute_vectors_only_v11.ckpt 104 43 80 35 0.036183 2.5556

v11_nondevelopment_seed_1

Checkpoint N R/S E/Z Joint Bond (A) Angle (degrees)
boltz2_base.ckpt 154 68 122 57 0.034383 2.5826
boltz2_wsp_absolute_vectors_only_v11.ckpt 154 69 121 59 0.033988 2.5827

Paired effects

Average repeated seeds within each system, then bootstrap sequence/scaffold groups 10,000 times with resampling seed 20261103. These intervals are conditional on the evaluated seeds and do not correct for candidate selection.

v2: nondevelopment_seed_1

Metric Mean change Cluster 95% CI
match 0.00649351 [-0.03896358, 0.05625000]
ez_match -0.00649351 [-0.02255745, 0.00000000]
joint 0.01298701 [-0.02857143, 0.05194805]
emb_brms 0.00046358 [-0.00019102, 0.00120255]
emb_arms 0.00892608 [-0.06281499, 0.08946591]

v2: test_seeds_1_2_3

Metric Mean change Cluster 95% CI
match 0.01282051 [-0.02272889, 0.05246914]
ez_match 0.00320513 [-0.01980198, 0.02564103]
joint 0.00000000 [-0.03300330, 0.03205128]
emb_brms 0.00011564 [-0.00041220, 0.00068050]
emb_arms -0.04241056 [-0.12946040, 0.03628711]

v11: nondevelopment_seed_1

Metric Mean change Cluster 95% CI
match 0.00649351 [-0.03896358, 0.05625000]
ez_match -0.00649351 [-0.02255745, 0.00000000]
joint 0.01298701 [-0.02857143, 0.05194805]
emb_brms 0.00064088 [-0.00012454, 0.00129773]
emb_arms 0.04754782 [-0.03673725, 0.11713533]

v11: test_seeds_1_2_3

Metric Mean change Cluster 95% CI
match 0.01282051 [-0.02272889, 0.05246914]
ez_match -0.00320513 [-0.02380952, 0.01388889]
joint 0.00000000 [-0.03300330, 0.03205128]
emb_brms 0.00000198 [-0.00045463, 0.00047334]
emb_arms -0.05296255 [-0.14310073, 0.01931832]