|
Download evaluation/paper_evaluation.md from wasarou/weight-space-programming-boltz2: direct link, hf CLI and curl.
- Browser
- Download file 7.98 kB
-
https://huggingface.co/wasarou/weight-space-programming-boltz2/resolve/main/evaluation/paper_evaluation.md
- Command line
-
hf download hf://wasarou/weight-space-programming-boltz2/evaluation/paper_evaluation.md
-
curl -L -o paper_evaluation.md https://huggingface.co/wasarou/weight-space-programming-boltz2/resolve/main/evaluation/paper_evaluation.md
7.98 kB
| # Re-evaluation with the RGI manuscript method | |
| Archived predictions were rescored; no model inference or weight editing was performed. | |
| R/S uses the manuscript's legacy RDKit center assignment. E/Z includes only acyclic, | |
| nonaromatic, heavy-atom-decidable defined bonds. Geometry uses the manuscript's | |
| independent canonical-SMILES ETKDGv3/UFF/stereo-validated reference preparation. | |
| The joint count is the conjunction of the manuscript's two per-system classifications. | |
| It is an additional summary, not the original common-mapping full-stereochemistry gate. | |
| The existing 55/50/104 split and previous validation/test exposure are unchanged. | |
| The original full-stereochemistry repair requirement remains unmet. | |
| All geometry entries below are medians of per-system RMS errors. | |
| ## development | |
| | Checkpoint | N | R/S | E/Z | Joint | Bond (A) | Angle (degrees) | | |
| |---|---:|---:|---:|---:|---:|---:| | |
| | boltz2_base.ckpt | 55 | 21 | 41 | 20 | 0.038695 | 2.8723 | | |
| | boltz2_wsp_absolute_vectors_distances_v17.ckpt | 55 | 24 | 41 | 23 | 0.038138 | 2.8443 | | |
| | boltz2_wsp_absolute_vectors_double_bond_v15.ckpt | 55 | 21 | 42 | 20 | 0.038452 | 2.8680 | | |
| | boltz2_wsp_absolute_vectors_fourfold_v16.ckpt | 55 | 22 | 42 | 21 | 0.039179 | 2.7212 | | |
| | boltz2_wsp_absolute_vectors_only_v11.ckpt | 55 | 24 | 41 | 23 | 0.038121 | 2.8315 | | |
| | boltz2_wsp_balanced_geometry_v9.ckpt | 55 | 21 | 42 | 20 | 0.038385 | 2.7390 | | |
| | boltz2_wsp_both_digit_suppression_v19.ckpt | 55 | 19 | 42 | 17 | 0.044122 | 2.7141 | | |
| | boltz2_wsp_denoiser_distances_only_v12.ckpt | 55 | 21 | 41 | 20 | 0.038440 | 2.8695 | | |
| | boltz2_wsp_denoiser_name_projection_v7.ckpt | 55 | 21 | 40 | 19 | 0.074566 | 4.8459 | | |
| | boltz2_wsp_direct_input_balance_v6.ckpt | 55 | 9 | 25 | 5 | 2.505421 | 46.2875 | | |
| | boltz2_wsp_direct_input_conditioning_v5.ckpt | 55 | 22 | 43 | 21 | 0.044555 | 3.1729 | | |
| | boltz2_wsp_double_bond_contrast_v14.ckpt | 55 | 21 | 41 | 19 | 0.038672 | 2.9020 | | |
| | boltz2_wsp_input_center_preserving_v8.ckpt | 55 | 24 | 42 | 22 | 0.046064 | 3.1451 | | |
| | boltz2_wsp_input_digit_suppression_v18.ckpt | 55 | 21 | 42 | 18 | 0.036351 | 2.9269 | | |
| | boltz2_wsp_input_geometry_v1.ckpt | 55 | 23 | 40 | 22 | 0.038254 | 2.9360 | | |
| | boltz2_wsp_input_geometry_v3.ckpt | 55 | 21 | 42 | 20 | 0.038556 | 2.7189 | | |
| | boltz2_wsp_input_reference_mask_v20.ckpt | 55 | 20 | 41 | 19 | 0.038081 | 2.8679 | | |
| | boltz2_wsp_ligand_name_projection_v4.ckpt | 55 | 20 | 41 | 19 | 0.038429 | 2.8946 | | |
| | boltz2_wsp_neighbor_program_control_v26.ckpt | 55 | 21 | 41 | 20 | 0.037731 | 2.8868 | | |
| | boltz2_wsp_orthogonal_atom_geometry_v22.ckpt | 55 | 21 | 41 | 20 | 0.038386 | 2.8960 | | |
| | boltz2_wsp_orthogonal_input_geometry_v24.ckpt | 55 | 20 | 41 | 19 | 0.038285 | 2.9150 | | |
| | boltz2_wsp_orthogonal_pair_geometry_v23.ckpt | 55 | 21 | 41 | 19 | 0.039263 | 2.8838 | | |
| | boltz2_wsp_programmed_neighbor_prior_v25.ckpt | 55 | 20 | 41 | 19 | 0.038947 | 2.9092 | | |
| | boltz2_wsp_protected_distance_direction_v13.ckpt | 55 | 22 | 41 | 20 | 0.038652 | 2.8792 | | |
| | boltz2_wsp_reference_distances_v2.ckpt | 55 | 23 | 40 | 22 | 0.038295 | 2.9338 | | |
| | boltz2_wsp_reference_mask_combined_v21.ckpt | 55 | 20 | 42 | 19 | 0.038563 | 2.8850 | | |
| | boltz2_wsp_relative_vectors_only_v10.ckpt | 55 | 20 | 41 | 19 | 0.038319 | 2.8840 | | |
| | boltz2_wsp_reserved_atom_channels_v28.ckpt | 55 | 22 | 39 | 20 | 0.098797 | 6.0517 | | |
| | boltz2_wsp_reserved_denoiser_channels_v27.ckpt | 55 | 8 | 34 | 7 | 0.586642 | 34.4782 | | |
| | boltz2_wsp_reserved_token_channels_v29.ckpt | 55 | 8 | 28 | 7 | 0.520425 | 23.8152 | | |
| | boltz2_wsp_reserved_token_condition_bias_v30.ckpt | 55 | 8 | 25 | 1 | 0.640224 | 29.1628 | | |
| ## v2_validation_seed_1 | |
| | Checkpoint | N | R/S | E/Z | Joint | Bond (A) | Angle (degrees) | | |
| |---|---:|---:|---:|---:|---:|---:| | |
| | boltz2_base.ckpt | 50 | 27 | 43 | 22 | 0.030402 | 2.4558 | | |
| | boltz2_wsp_reference_distances_v2.ckpt | 50 | 25 | 43 | 23 | 0.030114 | 2.5120 | | |
| ## v2_test_seed_1 | |
| | Checkpoint | N | R/S | E/Z | Joint | Bond (A) | Angle (degrees) | | |
| |---|---:|---:|---:|---:|---:|---:| | |
| | boltz2_base.ckpt | 104 | 41 | 79 | 35 | 0.037237 | 2.6254 | | |
| | boltz2_wsp_reference_distances_v2.ckpt | 104 | 44 | 78 | 36 | 0.036704 | 2.6128 | | |
| ## v2_test_seed_2 | |
| | Checkpoint | N | R/S | E/Z | Joint | Bond (A) | Angle (degrees) | | |
| |---|---:|---:|---:|---:|---:|---:| | |
| | boltz2_base.ckpt | 104 | 43 | 79 | 34 | 0.036447 | 2.6106 | | |
| | boltz2_wsp_reference_distances_v2.ckpt | 104 | 42 | 79 | 32 | 0.035559 | 2.6137 | | |
| ## v2_test_seed_3 | |
| | Checkpoint | N | R/S | E/Z | Joint | Bond (A) | Angle (degrees) | | |
| |---|---:|---:|---:|---:|---:|---:| | |
| | boltz2_base.ckpt | 104 | 41 | 78 | 34 | 0.036223 | 2.6611 | | |
| | boltz2_wsp_reference_distances_v2.ckpt | 104 | 43 | 80 | 35 | 0.036433 | 2.5693 | | |
| ## v2_nondevelopment_seed_1 | |
| | Checkpoint | N | R/S | E/Z | Joint | Bond (A) | Angle (degrees) | | |
| |---|---:|---:|---:|---:|---:|---:| | |
| | boltz2_base.ckpt | 154 | 68 | 122 | 57 | 0.034383 | 2.5826 | | |
| | boltz2_wsp_reference_distances_v2.ckpt | 154 | 69 | 121 | 59 | 0.034468 | 2.5866 | | |
| ## v11_validation_seed_1 | |
| | Checkpoint | N | R/S | E/Z | Joint | Bond (A) | Angle (degrees) | | |
| |---|---:|---:|---:|---:|---:|---:| | |
| | boltz2_base.ckpt | 50 | 27 | 43 | 22 | 0.030402 | 2.4558 | | |
| | boltz2_wsp_absolute_vectors_only_v11.ckpt | 50 | 25 | 43 | 23 | 0.030142 | 2.4849 | | |
| ## v11_test_seed_1 | |
| | Checkpoint | N | R/S | E/Z | Joint | Bond (A) | Angle (degrees) | | |
| |---|---:|---:|---:|---:|---:|---:| | |
| | boltz2_base.ckpt | 104 | 41 | 79 | 35 | 0.037237 | 2.6254 | | |
| | boltz2_wsp_absolute_vectors_only_v11.ckpt | 104 | 44 | 78 | 36 | 0.036368 | 2.6236 | | |
| ## v11_test_seed_2 | |
| | Checkpoint | N | R/S | E/Z | Joint | Bond (A) | Angle (degrees) | | |
| |---|---:|---:|---:|---:|---:|---:| | |
| | boltz2_base.ckpt | 104 | 43 | 79 | 34 | 0.036447 | 2.6106 | | |
| | boltz2_wsp_absolute_vectors_only_v11.ckpt | 104 | 42 | 77 | 32 | 0.035828 | 2.6562 | | |
| ## v11_test_seed_3 | |
| | Checkpoint | N | R/S | E/Z | Joint | Bond (A) | Angle (degrees) | | |
| |---|---:|---:|---:|---:|---:|---:| | |
| | boltz2_base.ckpt | 104 | 41 | 78 | 34 | 0.036223 | 2.6611 | | |
| | boltz2_wsp_absolute_vectors_only_v11.ckpt | 104 | 43 | 80 | 35 | 0.036183 | 2.5556 | | |
| ## v11_nondevelopment_seed_1 | |
| | Checkpoint | N | R/S | E/Z | Joint | Bond (A) | Angle (degrees) | | |
| |---|---:|---:|---:|---:|---:|---:| | |
| | boltz2_base.ckpt | 154 | 68 | 122 | 57 | 0.034383 | 2.5826 | | |
| | boltz2_wsp_absolute_vectors_only_v11.ckpt | 154 | 69 | 121 | 59 | 0.033988 | 2.5827 | | |
| ## Paired effects | |
| Average repeated seeds within each system, then bootstrap sequence/scaffold groups | |
| 10,000 times with resampling seed 20261103. These intervals are conditional on the | |
| evaluated seeds and do not correct for candidate selection. | |
| ### v2: nondevelopment_seed_1 | |
| | Metric | Mean change | Cluster 95% CI | | |
| |---|---:|---| | |
| | match | 0.00649351 | [-0.03896358, 0.05625000] | | |
| | ez_match | -0.00649351 | [-0.02255745, 0.00000000] | | |
| | joint | 0.01298701 | [-0.02857143, 0.05194805] | | |
| | emb_brms | 0.00046358 | [-0.00019102, 0.00120255] | | |
| | emb_arms | 0.00892608 | [-0.06281499, 0.08946591] | | |
| ### v2: test_seeds_1_2_3 | |
| | Metric | Mean change | Cluster 95% CI | | |
| |---|---:|---| | |
| | match | 0.01282051 | [-0.02272889, 0.05246914] | | |
| | ez_match | 0.00320513 | [-0.01980198, 0.02564103] | | |
| | joint | 0.00000000 | [-0.03300330, 0.03205128] | | |
| | emb_brms | 0.00011564 | [-0.00041220, 0.00068050] | | |
| | emb_arms | -0.04241056 | [-0.12946040, 0.03628711] | | |
| ### v11: nondevelopment_seed_1 | |
| | Metric | Mean change | Cluster 95% CI | | |
| |---|---:|---| | |
| | match | 0.00649351 | [-0.03896358, 0.05625000] | | |
| | ez_match | -0.00649351 | [-0.02255745, 0.00000000] | | |
| | joint | 0.01298701 | [-0.02857143, 0.05194805] | | |
| | emb_brms | 0.00064088 | [-0.00012454, 0.00129773] | | |
| | emb_arms | 0.04754782 | [-0.03673725, 0.11713533] | | |
| ### v11: test_seeds_1_2_3 | |
| | Metric | Mean change | Cluster 95% CI | | |
| |---|---:|---| | |
| | match | 0.01282051 | [-0.02272889, 0.05246914] | | |
| | ez_match | -0.00320513 | [-0.02380952, 0.01388889] | | |
| | joint | 0.00000000 | [-0.03300330, 0.03205128] | | |
| | emb_brms | 0.00000198 | [-0.00045463, 0.00047334] | | |
| | emb_arms | -0.05296255 | [-0.14310073, 0.01931832] | | |